2024-04-13T02:56:06.317 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-13T02:56:06.317 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --name vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi --verbose --archive /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237 --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/orig.config.yaml 2024-04-13T02:56:06.369 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T02:56:06.467 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237 branch: wip-vshankar-testing-20240412.042930 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: vshankar@redhat.com first_in_suite: false job_id: '7654237' kernel: client: sha1: distro flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240412.042930 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240412.042930 sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7517 sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 sleep_before_teardown: 0 subset: 55/1024 suite: fs suite_branch: wip-vshankar-testing-20240412.042930 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyYDU5m3TDDQA01tQdtRqc5190+eS7O5PDuIy4C3rpyY48TgQcZeo44AnjuDICd/T+RwJqL3qnqghlndxPkuK8= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrnsrNDJBSUskISoAmoVnGd/SkC+aeSpHNjd8AYVLKsGuh8E4E6f1RCpN0JYFOVOBR1GbmNXdlypbubhUzF36Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-12_17:34:28 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-13T02:56:06.468 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa; will attempt to use it 2024-04-13T02:56:06.468 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa/tasks 2024-04-13T02:56:06.468 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-13T02:56:06.470 INFO:teuthology.task.internal:Checking packages... 2024-04-13T02:56:06.496 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '218a691417ef57c5d22a8d5cf3c5f41f7b542838' 2024-04-13T02:56:06.496 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-13T02:56:06.497 INFO:teuthology.packaging:ref: None 2024-04-13T02:56:06.497 INFO:teuthology.packaging:tag: None 2024-04-13T02:56:06.497 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240412.042930 2024-04-13T02:56:06.497 INFO:teuthology.packaging:sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T02:56:06.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240412.042930 2024-04-13T02:56:06.647 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2884.g218a6914 2024-04-13T02:56:06.648 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-13T02:56:06.657 INFO:teuthology.task.internal:no buildpackages task found 2024-04-13T02:56:06.658 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-13T02:56:06.680 INFO:teuthology.task.internal:Saving configuration 2024-04-13T02:56:06.697 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-13T02:56:06.704 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T02:56:06.726 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 02:51:04.234176', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyYDU5m3TDDQA01tQdtRqc5190+eS7O5PDuIy4C3rpyY48TgQcZeo44AnjuDICd/T+RwJqL3qnqghlndxPkuK8='} 2024-04-13T02:56:06.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 02:51:04.232760', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrnsrNDJBSUskISoAmoVnGd/SkC+aeSpHNjd8AYVLKsGuh8E4E6f1RCpN0JYFOVOBR1GbmNXdlypbubhUzF36Y='} 2024-04-13T02:56:06.748 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-13T02:56:06.754 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-13T02:56:06.754 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-13T02:56:06.754 INFO:teuthology.run_tasks:Running task console_log... 2024-04-13T02:56:06.840 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8ccfa45160>, signals=[15]) 2024-04-13T02:56:06.840 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-13T02:56:06.847 INFO:teuthology.task.internal:Opening connections... 2024-04-13T02:56:06.847 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T02:56:06.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:56:06.936 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T02:56:06.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:56:07.023 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-13T02:56:07.029 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-13T02:56:07.047 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-13T02:56:07.047 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-13T02:56:07.104 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-13T02:56:07.105 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-13T02:56:07.105 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T02:56:07.105 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-13T02:56:07.105 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-13T02:56:07.105 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-13T02:56:07.126 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-04-13T02:56:07.144 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-04-13T02:56:07.144 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-04-13T02:56:07.200 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-04-13T02:56:07.200 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-04-13T02:56:07.200 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-13T02:56:07.201 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-13T02:56:07.202 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-04-13T02:56:07.234 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-13T02:56:07.243 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-13T02:56:07.272 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-13T02:56:07.272 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-13T02:56:07.275 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-04-13T02:56:07.291 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-13T02:56:07.296 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-13T02:56:07.297 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-13T02:56:07.331 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-04-13T02:56:07.422 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-13T02:56:07.476 INFO:teuthology.run_tasks:Running task kernel... 2024-04-13T02:56:07.494 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-13T02:56:07.494 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-13T02:56:07.494 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-13T02:56:07.495 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-13T02:56:07.495 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-13T02:56:07.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T02:56:07.511 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-13T02:56:07.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T02:56:07.513 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-04-13T02:56:07.568 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-361.el9.x86_64 2024-04-13T02:56:07.569 INFO:teuthology.task.kernel:Running kernel on smithi106: 5.14.0-361.el9.x86_64 2024-04-13T02:56:07.569 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum install -y kernel 2024-04-13T02:56:07.570 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-361.el9.x86_64 2024-04-13T02:56:07.570 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-361.el9.x86_64 2024-04-13T02:56:07.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-04-13T02:56:09.700 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 15 MB/s | 8.1 MB 00:00 2024-04-13T02:56:09.752 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 16 MB/s | 8.1 MB 00:00 2024-04-13T02:56:11.350 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.5 MB 00:00 2024-04-13T02:56:11.368 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 8.7 MB/s | 2.5 MB 00:00 2024-04-13T02:56:12.216 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.0 MB 00:00 2024-04-13T02:56:12.742 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-13T02:56:23.708 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-04-13T02:56:23.834 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-04-13T02:56:27.743 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-04-13T02:56:28.211 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-13T02:56:29.185 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-13T02:56:29.842 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-13T02:56:30.095 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-04-13T02:56:30.944 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-13T02:56:35.221 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-13T02:56:36.069 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-13T02:56:37.474 INFO:teuthology.orchestra.run.smithi106.stderr:Modular dependency problems: 2024-04-13T02:56:37.474 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 1: conflicting requests 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 2: conflicting requests 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 3: conflicting requests 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 4: conflicting requests 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 5: conflicting requests 2024-04-13T02:56:37.475 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 6: conflicting requests 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 7: conflicting requests 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 8: conflicting requests 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 9: conflicting requests 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 10: conflicting requests 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.476 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 11: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 12: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 13: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 14: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 15: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 16: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 17: conflicting requests 2024-04-13T02:56:37.477 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 18: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 19: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 20: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 21: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 22: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 23: conflicting requests 2024-04-13T02:56:37.478 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 24: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 25: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 26: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 27: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 28: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 29: conflicting requests 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.479 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 30: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 31: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 32: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 33: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 34: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 35: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 36: conflicting requests 2024-04-13T02:56:37.480 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 37: conflicting requests 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 39: conflicting requests 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.481 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.482 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.483 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.484 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T02:56:37.819 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-13T02:56:37.819 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-13T02:56:37.932 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout: Package Architecture Version Repository Size 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-13T02:56:37.944 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:Install 4 Packages 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 77 M 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 123 M 2024-04-13T02:56:37.945 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-13T02:56:38.230 INFO:teuthology.orchestra.run.smithi106.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 1.2 MB/s | 214 kB 00:00 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:Modular dependency problems: 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 1: conflicting requests 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 2: conflicting requests 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 3: conflicting requests 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.301 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 4: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 5: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 6: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 7: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 8: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 9: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 10: conflicting requests 2024-04-13T02:56:38.302 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 11: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 12: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 13: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 14: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 15: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 16: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 17: conflicting requests 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.303 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 18: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 19: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 20: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 21: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 22: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 23: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 24: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 25: conflicting requests 2024-04-13T02:56:38.304 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 26: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 27: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 28: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 29: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 30: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 31: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 32: conflicting requests 2024-04-13T02:56:38.305 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 33: conflicting requests 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 34: conflicting requests 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 35: conflicting requests 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 36: conflicting requests 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.306 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 37: conflicting requests 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 39: conflicting requests 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.307 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.308 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.309 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.310 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T02:56:38.684 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-13T02:56:38.685 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-13T02:56:38.755 INFO:teuthology.orchestra.run.smithi106.stdout:(2/4): kernel-core-5.14.0-435.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-04-13T02:56:38.788 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-13T02:56:38.799 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-13T02:56:38.800 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-04-13T02:56:38.800 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:56:38.800 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 77 M 2024-04-13T02:56:38.800 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 123 M 2024-04-13T02:56:38.800 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-13T02:56:39.462 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 647 kB/s | 214 kB 00:00 2024-04-13T02:56:39.572 INFO:teuthology.orchestra.run.smithi106.stdout:(3/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 22 MB/s | 34 MB 00:01 2024-04-13T02:56:39.948 INFO:teuthology.orchestra.run.smithi106.stdout:(4/4): kernel-modules-core-5.14.0-435.el9.x86_6 16 MB/s | 28 MB 00:01 2024-04-13T02:56:39.948 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-13T02:56:39.948 INFO:teuthology.orchestra.run.smithi106.stdout:Total 38 MB/s | 77 MB 00:02 2024-04-13T02:56:40.614 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-13T02:56:40.951 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-13T02:56:40.952 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-13T02:56:41.079 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-435.el9.x86_64.rpm 7.8 MB/s | 15 MB 00:01 2024-04-13T02:56:41.632 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-core-5.14.0-435.el9.x86_6 13 MB/s | 28 MB 00:02 2024-04-13T02:56:41.946 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-04-13T02:56:41.946 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-13T02:56:41.946 INFO:teuthology.orchestra.run.smithi114.stdout:Total 24 MB/s | 77 MB 00:03 2024-04-13T02:56:41.961 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-13T02:56:41.961 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-13T02:56:42.618 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-13T02:56:42.976 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-13T02:56:42.976 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-13T02:56:44.044 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-13T02:56:44.045 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-13T02:56:44.423 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-13T02:56:44.999 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-13T02:56:45.190 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:56:46.520 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-13T02:56:47.106 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-13T02:56:47.188 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:56:47.905 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:56:48.052 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:56:49.953 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:56:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:56:52.766 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:56:52.819 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:56:54.780 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:56:54.840 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:56:57.555 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:56:59.570 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:26.915 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:26.916 INFO:teuthology.orchestra.run.smithi106.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-13T02:57:26.916 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:57:26.962 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:27.273 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:27.273 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-13T02:57:27.273 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:57:27.273 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:57:27.710 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:57:27.711 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-13T02:57:28.067 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q kernel | sort -rV | head -n 1 2024-04-13T02:57:28.167 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:28.167 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-13T02:57:28.167 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-13T02:57:28.167 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-13T02:57:28.167 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-13T02:57:28.167 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-13T02:57:28.168 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=None, version=distro) 2024-04-13T02:57:28.168 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q kernel | sort -rV | head -n 1 2024-04-13T02:57:28.247 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:28.248 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-13T02:57:28.248 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2-tools 2024-04-13T02:57:28.292 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2-tools 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.06 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 68.el9 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Group : Unspecified 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 8264254 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2024-04-13T02:57:28.293 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : builder@centos.org 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Support tools for GRUB. 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2024-04-13T02:57:28.294 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T02:57:28.295 INFO:teuthology.orchestra.run.smithi106.stdout:This subpackage provides tools for support of all platforms. 2024-04-13T02:57:28.295 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-13T02:57:28.296 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 5.14.0-435.el9.x86_64 2024-04-13T02:57:28.296 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-13T02:57:28.886 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2024-04-13T02:57:28.910 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:28.911 INFO:teuthology.orchestra.run.smithi114.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-13T02:57:28.911 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:57:28.957 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:29.336 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:29.336 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-13T02:57:29.336 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-13T02:57:29.336 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-13T02:57:29.783 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-13T02:57:29.783 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:57:29.783 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-13T02:57:29.783 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:29.784 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-13T02:57:29.784 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-13T02:57:29.784 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-13T02:57:29.784 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:57:29.784 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-13T02:57:30.146 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-04-13T02:57:30.156 INFO:teuthology.orchestra.run.smithi106.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-13T02:57:30.195 INFO:teuthology.orchestra.run.smithi106.stderr:done 2024-04-13T02:57:30.196 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-13T02:57:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.wumqDrXcDd 2024-04-13T02:57:30.212 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wumqDrXcDd 2024-04-13T02:57:30.246 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:30.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-13T02:57:30.246 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-13T02:57:30.246 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-13T02:57:30.246 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-13T02:57:30.246 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-04-13T02:57:30.247 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-04-13T02:57:30.247 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-04-13T02:57:30.281 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.wumqDrXcDd 2024-04-13T02:57:30.327 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-13T02:57:30.327 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-13T02:57:30.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 68.el9 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-13T02:57:30.360 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8264254 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-13T02:57:30.361 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-13T02:57:30.362 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-13T02:57:30.362 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-04-13T02:57:30.362 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T02:57:30.362 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-04-13T02:57:30.363 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-13T02:57:30.363 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-435.el9.x86_64 2024-04-13T02:57:30.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-13T02:57:30.419 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.wumqDrXcDd is 6KB 2024-04-13T02:57:30.468 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.wumqDrXcDd 2024-04-13T02:57:30.482 DEBUG:teuthology.orchestra.run.smithi106:> sudo /bin/ls /boot/loader/entries || true 2024-04-13T02:57:30.546 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-13T02:57:30.547 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-13T02:57:30.547 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-13T02:57:30.547 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-13T02:57:30.548 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-13T02:57:30.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2024-04-13T02:57:30.980 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-04-13T02:57:32.392 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-13T02:57:32.421 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-04-13T02:57:32.423 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-04-13T02:57:32.438 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.2SurfXOEh9 2024-04-13T02:57:32.438 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2SurfXOEh9 2024-04-13T02:57:32.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.2SurfXOEh9 2024-04-13T02:57:32.645 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.2SurfXOEh9 is 6KB 2024-04-13T02:57:32.694 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.2SurfXOEh9 2024-04-13T02:57:32.708 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-04-13T02:57:32.782 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-13T02:57:32.782 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-13T02:57:32.782 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-13T02:57:32.782 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-13T02:57:32.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-13T02:57:32.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-04-13T02:58:00.723 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-13T02:58:00.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-13T02:58:00.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:58:02.918 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-13T02:58:02.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T02:58:02.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:58:37.471 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T02:58:46.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T02:58:46.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:58:49.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T02:59:00.734 DEBUG:teuthology.orchestra.remote:timed out 2024-04-13T02:59:01.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T02:59:01.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:59:04.610 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T02:59:09.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-13T02:59:09.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:59:10.219 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-13T02:59:10.691 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-13T02:59:10.691 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-435.el9.x86_64"... 2024-04-13T02:59:10.691 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-13T02:59:10.705 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-435.el9.x86_64 2024-04-13T02:59:10.705 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-13T02:59:10.705 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-13T02:59:10.705 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-13T02:59:11.706 DEBUG:teuthology.parallel:result is None 2024-04-13T02:59:19.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T02:59:19.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:59:22.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T02:59:40.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T02:59:40.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T02:59:43.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T03:00:04.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:00:04.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T03:00:07.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T03:00:31.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:00:31.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T03:00:34.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-04-13T03:01:01.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:01:01.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T03:01:02.268 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-04-13T03:01:02.717 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:01:02.717 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-435.el9.x86_64"... 2024-04-13T03:01:02.717 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-04-13T03:01:02.731 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-435.el9.x86_64 2024-04-13T03:01:02.731 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-13T03:01:02.731 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-13T03:01:02.731 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-13T03:01:03.732 DEBUG:teuthology.parallel:result is None 2024-04-13T03:01:03.732 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-13T03:01:03.740 INFO:teuthology.task.internal:Creating test directory... 2024-04-13T03:01:03.740 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-13T03:01:03.744 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-13T03:01:03.760 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-13T03:01:03.784 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-13T03:01:03.817 INFO:teuthology.task.internal:Creating archive directory... 2024-04-13T03:01:03.818 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-13T03:01:03.820 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-13T03:01:03.913 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-13T03:01:03.920 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-13T03:01:03.920 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-13T03:01:03.923 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-13T03:01:04.024 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T03:01:04.030 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T03:01:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T03:01:04.117 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T03:01:04.118 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-13T03:01:04.125 INFO:teuthology.task.internal:Configuring sudo... 2024-04-13T03:01:04.126 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-13T03:01:04.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-13T03:01:04.157 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-13T03:01:04.166 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-13T03:01:04.166 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-13T03:01:04.200 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-13T03:01:04.215 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T03:01:04.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T03:01:04.348 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T03:01:04.433 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T03:01:04.477 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:01:04.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-13T03:01:04.547 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T03:01:04.588 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T03:01:04.645 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T03:01:04.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T03:01:04.775 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:01:04.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-13T03:01:04.847 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-13T03:01:04.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-04-13T03:01:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T03:01:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T03:01:05.358 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-13T03:01:05.365 INFO:teuthology.task.internal:Starting timer... 2024-04-13T03:01:05.365 INFO:teuthology.run_tasks:Running task pcp... 2024-04-13T03:01:05.396 INFO:teuthology.run_tasks:Running task selinux... 2024-04-13T03:01:05.428 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-13T03:01:05.429 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-13T03:01:05.488 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-13T03:01:05.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-04-13T03:01:05.576 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-04-13T03:01:05.579 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-13T03:01:05.579 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-13T03:01:05.604 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-13T03:01:05.604 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-04-13T03:01:05.696 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-04-13T03:01:05.697 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive'} 2024-04-13T03:01:05.697 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T03:01:05.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:01:05.728 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T03:01:05.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:01:05.785 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-13T03:01:05.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-13T03:01:05.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-04-13T03:01:06.166 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-13T03:01:06.176 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-13T03:01:06.193 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-13T03:01:06.194 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-13T03:05:52.714 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-04-13T03:05:52.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-13T03:05:52.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T03:05:52.807 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-13T03:05:52.885 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-13T03:05:52.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:05:52.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T03:05:52.974 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-04-13T03:05:53.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-13T03:05:53.049 INFO:teuthology.run_tasks:Running task clock... 2024-04-13T03:05:53.060 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-13T03:05:53.060 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-13T03:05:53.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T03:05:53.063 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-13T03:05:53.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T03:05:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-13T03:05:53.109 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-13T03:05:53.121 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-13T03:05:53.138 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-13T03:05:53.138 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-13T03:05:53.151 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-13T03:05:53.167 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-04-13T03:05:53.168 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-13T03:05:53.181 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-04-13T03:05:53.184 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-13T03:05:53.197 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-13T03:05:53.214 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-13T03:05:53.235 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-13T03:05:53.238 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T03:05:53.238 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-13T03:05:53.238 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.238 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.238 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.239 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:05:53.271 INFO:teuthology.run_tasks:Running task install... 2024-04-13T03:05:53.293 DEBUG:teuthology.task.install:project ceph 2024-04-13T03:05:53.293 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}} 2024-04-13T03:05:53.293 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'} 2024-04-13T03:05:53.293 INFO:teuthology.task.install:Using flavor: default 2024-04-13T03:05:53.304 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-13T03:05:53.304 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-13T03:05:53.305 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'tag': None, 'wait_for_package': False} 2024-04-13T03:05:53.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T03:05:53.309 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'tag': None, 'wait_for_package': False} 2024-04-13T03:05:53.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T03:05:53.463 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/ 2024-04-13T03:05:53.463 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2884.g218a6914 2024-04-13T03:05:53.471 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/ 2024-04-13T03:05:53.471 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2884.g218a6914 2024-04-13T03:05:53.577 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-13T03:05:53.577 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:05:53.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-13T03:05:53.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-13T03:05:53.607 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/218a691417ef57c5d22a8d5cf3c5f41f7b542838/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-13T03:05:53.639 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-13T03:05:53.639 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:05:53.639 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-13T03:05:53.669 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-13T03:05:53.669 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/218a691417ef57c5d22a8d5cf3c5f41f7b542838/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-13T03:05:53.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-13T03:05:53.738 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-13T03:05:53.757 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-13T03:05:53.817 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-13T03:05:53.823 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-04-13T03:05:53.825 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-04-13T03:05:53.883 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-13T03:05:53.884 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-13T03:05:54.066 INFO:teuthology.orchestra.run.smithi114.stdout:74 files removed 2024-04-13T03:05:54.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-13T03:05:54.130 INFO:teuthology.orchestra.run.smithi106.stdout:74 files removed 2024-04-13T03:05:54.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-13T03:05:54.852 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 180 kB/s | 83 kB 00:00 2024-04-13T03:05:54.908 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 184 kB/s | 83 kB 00:00 2024-04-13T03:05:55.222 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-13T03:05:55.258 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-13T03:05:55.603 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-13T03:05:55.667 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-13T03:05:56.072 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 19 MB/s | 8.1 MB 00:00 2024-04-13T03:05:56.119 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 20 MB/s | 8.1 MB 00:00 2024-04-13T03:05:57.764 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.5 MB 00:00 2024-04-13T03:05:57.787 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.5 MB 00:00 2024-04-13T03:05:58.652 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-13T03:05:58.804 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-13T03:06:00.412 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-13T03:06:00.755 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-13T03:06:04.604 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-13T03:06:04.806 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-13T03:06:06.014 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-04-13T03:06:06.307 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-13T03:06:07.098 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-13T03:06:07.425 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-13T03:06:12.090 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-13T03:06:12.450 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-13T03:06:14.649 INFO:teuthology.orchestra.run.smithi106.stderr:Modular dependency problems: 2024-04-13T03:06:14.649 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:06:14.649 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 1: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 2: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 3: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 4: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 5: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 6: conflicting requests 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.650 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 7: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 8: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 9: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 10: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 11: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 12: conflicting requests 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.651 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 13: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 14: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 15: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 16: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 17: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 18: conflicting requests 2024-04-13T03:06:14.652 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 19: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 20: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 21: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 22: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 23: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 24: conflicting requests 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.653 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 25: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 26: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 27: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 28: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 29: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 30: conflicting requests 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.654 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 31: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 32: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 33: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 34: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 35: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 36: conflicting requests 2024-04-13T03:06:14.655 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 37: conflicting requests 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 39: conflicting requests 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.656 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.657 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T03:06:14.658 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T03:06:14.659 INFO:teuthology.orchestra.run.smithi106.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.008 INFO:teuthology.orchestra.run.smithi114.stderr:Modular dependency problems: 2024-04-13T03:06:15.008 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 1: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 2: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 3: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 4: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 5: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 6: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 7: conflicting requests 2024-04-13T03:06:15.009 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 8: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 9: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 10: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 11: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 12: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 13: conflicting requests 2024-04-13T03:06:15.010 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 14: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 15: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 16: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 17: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 18: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 19: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 20: conflicting requests 2024-04-13T03:06:15.011 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 21: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 22: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 23: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 24: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 25: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 26: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 27: conflicting requests 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.012 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 28: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 29: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 30: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 31: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 32: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 33: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 34: conflicting requests 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.013 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 35: conflicting requests 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 36: conflicting requests 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 37: conflicting requests 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.014 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 39: conflicting requests 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.015 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T03:06:15.016 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.017 INFO:teuthology.orchestra.run.smithi114.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T03:06:15.111 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.112 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.125 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-13T03:06:15.126 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.127 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-13T03:06:15.128 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.129 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.132 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.133 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-13T03:06:15.227 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout:========================================================================================= 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout:========================================================================================= 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-2884.g218a6914.el9 ceph 6.6 k 2024-04-13T03:06:15.239 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.4 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 891 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2884.g218a6914.el9 ceph 145 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-2884.g218a6914.el9 ceph 1.5 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 142 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 3.5 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 7.4 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 49 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 10 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-2884.g218a6914.el9 ceph 49 M 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 264 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 763 k 2024-04-13T03:06:15.240 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 225 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-2884.g218a6914.el9 ceph 27 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 18 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 32 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 746 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 126 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-13T03:06:15.241 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-2884.g218a6914.el9 ceph 162 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-2884.g218a6914.el9 ceph 321 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 298 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 100 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 85 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 170 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-13T03:06:15.242 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.4 M 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-13T03:06:15.243 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 21 M 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 34 k 2024-04-13T03:06:15.244 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-2884.g218a6914.el9 ceph 2.4 M 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 249 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-2884.g218a6914.el9 ceph 4.7 M 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 17 M 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 16 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-2884.g218a6914.el9 ceph 25 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-13T03:06:15.245 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-2884.g218a6914.el9 ceph 163 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 500 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.2 M 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-13T03:06:15.246 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-13T03:06:15.247 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 45 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 135 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-13T03:06:15.248 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-13T03:06:15.249 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-13T03:06:15.250 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-13T03:06:15.251 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:06:15.252 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout:========================================================================================= 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout:Install 150 Packages 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 24 Packages 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 229 M 2024-04-13T03:06:15.253 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-13T03:06:15.474 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.474 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.488 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-13T03:06:15.489 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-13T03:06:15.490 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-13T03:06:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.496 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-13T03:06:15.497 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-13T03:06:15.590 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout:========================================================================================= 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout:========================================================================================= 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.0.0-2884.g218a6914.el9 ceph 6.6 k 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.4 M 2024-04-13T03:06:15.606 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 891 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2884.g218a6914.el9 ceph 145 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.0.0-2884.g218a6914.el9 ceph 1.5 M 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 142 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 3.5 M 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 7.4 M 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 49 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 10 M 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.0.0-2884.g218a6914.el9 ceph 49 M 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 264 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 763 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 225 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.0.0-2884.g218a6914.el9 ceph 27 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 18 k 2024-04-13T03:06:15.607 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 32 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 746 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 126 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.0.0-2884.g218a6914.el9 ceph 162 k 2024-04-13T03:06:15.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.0.0-2884.g218a6914.el9 ceph 321 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 298 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 100 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 85 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 170 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-13T03:06:15.609 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.4 M 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-13T03:06:15.610 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 21 M 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 34 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.0.0-2884.g218a6914.el9 ceph 2.4 M 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 249 k 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.0.0-2884.g218a6914.el9 ceph 4.7 M 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 17 M 2024-04-13T03:06:15.611 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 16 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.0.0-2884.g218a6914.el9 ceph 25 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.0.0-2884.g218a6914.el9 ceph 163 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-13T03:06:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 500 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.2 M 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-13T03:06:15.613 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-13T03:06:15.614 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 45 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 135 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-13T03:06:15.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-13T03:06:15.616 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-13T03:06:15.617 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-13T03:06:15.618 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-13T03:06:15.619 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:========================================================================================= 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:Install 150 Packages 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 24 Packages 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 229 M 2024-04-13T03:06:15.620 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-13T03:06:16.250 INFO:teuthology.orchestra.run.smithi106.stdout:(1/174): ceph-19.0.0-2884.g218a6914.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-13T03:06:16.460 INFO:teuthology.orchestra.run.smithi114.stdout:(1/174): ceph-19.0.0-2884.g218a6914.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-13T03:06:17.183 INFO:teuthology.orchestra.run.smithi106.stdout:(2/174): ceph-fuse-19.0.0-2884.g218a6914.el9.x8 954 kB/s | 891 kB 00:00 2024-04-13T03:06:17.358 INFO:teuthology.orchestra.run.smithi106.stdout:(3/174): ceph-immutable-object-cache-19.0.0-288 828 kB/s | 145 kB 00:00 2024-04-13T03:06:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:(2/174): ceph-fuse-19.0.0-2884.g218a6914.el9.x8 463 kB/s | 891 kB 00:01 2024-04-13T03:06:18.660 INFO:teuthology.orchestra.run.smithi114.stdout:(3/174): ceph-immutable-object-cache-19.0.0-288 526 kB/s | 145 kB 00:00 2024-04-13T03:06:18.842 INFO:teuthology.orchestra.run.smithi106.stdout:(4/174): ceph-mds-19.0.0-2884.g218a6914.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-13T03:06:19.484 INFO:teuthology.orchestra.run.smithi106.stdout:(5/174): ceph-mgr-19.0.0-2884.g218a6914.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-04-13T03:06:19.559 INFO:teuthology.orchestra.run.smithi106.stdout:(6/174): ceph-base-19.0.0-2884.g218a6914.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-13T03:06:19.760 INFO:teuthology.orchestra.run.smithi106.stdout:(7/174): ceph-common-19.0.0-2884.g218a6914.el9. 5.4 MB/s | 21 MB 00:03 2024-04-13T03:06:20.444 INFO:teuthology.orchestra.run.smithi114.stdout:(4/174): ceph-base-19.0.0-2884.g218a6914.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-13T03:06:20.969 INFO:teuthology.orchestra.run.smithi114.stdout:(5/174): ceph-mgr-19.0.0-2884.g218a6914.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-13T03:06:21.026 INFO:teuthology.orchestra.run.smithi106.stdout:(8/174): ceph-radosgw-19.0.0-2884.g218a6914.el9 7.9 MB/s | 10 MB 00:01 2024-04-13T03:06:21.093 INFO:teuthology.orchestra.run.smithi106.stdout:(9/174): ceph-mon-19.0.0-2884.g218a6914.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-04-13T03:06:21.110 INFO:teuthology.orchestra.run.smithi106.stdout:(10/174): ceph-selinux-19.0.0-2884.g218a6914.el 301 kB/s | 25 kB 00:00 2024-04-13T03:06:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:(11/174): cephfs-mirror-19.0.0-2884.g218a6914.e 2.6 MB/s | 225 kB 00:00 2024-04-13T03:06:21.260 INFO:teuthology.orchestra.run.smithi106.stdout:(12/174): cephfs-shell-19.0.0-2884.g218a6914.el 405 kB/s | 27 kB 00:00 2024-04-13T03:06:21.319 INFO:teuthology.orchestra.run.smithi106.stdout:(13/174): libcephfs-devel-19.0.0-2884.g218a6914 544 kB/s | 32 kB 00:00 2024-04-13T03:06:21.369 INFO:teuthology.orchestra.run.smithi114.stdout:(6/174): ceph-mds-19.0.0-2884.g218a6914.el9.x86 894 kB/s | 2.4 MB 00:02 2024-04-13T03:06:21.419 INFO:teuthology.orchestra.run.smithi106.stdout:(14/174): libcephfs2-19.0.0-2884.g218a6914.el9. 7.3 MB/s | 746 kB 00:00 2024-04-13T03:06:21.486 INFO:teuthology.orchestra.run.smithi106.stdout:(15/174): libcephsqlite-19.0.0-2884.g218a6914.e 2.4 MB/s | 163 kB 00:00 2024-04-13T03:06:21.545 INFO:teuthology.orchestra.run.smithi106.stdout:(16/174): librados-devel-19.0.0-2884.g218a6914. 2.1 MB/s | 126 kB 00:00 2024-04-13T03:06:21.586 INFO:teuthology.orchestra.run.smithi114.stdout:(7/174): ceph-common-19.0.0-2884.g218a6914.el9. 3.7 MB/s | 21 MB 00:05 2024-04-13T03:06:21.620 INFO:teuthology.orchestra.run.smithi106.stdout:(17/174): libradosstriper1-19.0.0-2884.g218a691 6.5 MB/s | 500 kB 00:00 2024-04-13T03:06:22.129 INFO:teuthology.orchestra.run.smithi106.stdout:(18/174): librgw2-19.0.0-2884.g218a6914.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-13T03:06:22.179 INFO:teuthology.orchestra.run.smithi106.stdout:(19/174): python3-ceph-argparse-19.0.0-2884.g21 907 kB/s | 45 kB 00:00 2024-04-13T03:06:22.246 INFO:teuthology.orchestra.run.smithi106.stdout:(20/174): python3-ceph-common-19.0.0-2884.g218a 2.0 MB/s | 135 kB 00:00 2024-04-13T03:06:22.313 INFO:teuthology.orchestra.run.smithi106.stdout:(21/174): python3-cephfs-19.0.0-2884.g218a6914. 2.4 MB/s | 162 kB 00:00 2024-04-13T03:06:22.370 INFO:teuthology.orchestra.run.smithi114.stdout:(8/174): ceph-mon-19.0.0-2884.g218a6914.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-04-13T03:06:22.379 INFO:teuthology.orchestra.run.smithi106.stdout:(22/174): python3-rados-19.0.0-2884.g218a6914.e 4.7 MB/s | 321 kB 00:00 2024-04-13T03:06:22.420 INFO:teuthology.orchestra.run.smithi114.stdout:(9/174): ceph-selinux-19.0.0-2884.g218a6914.el9 506 kB/s | 25 kB 00:00 2024-04-13T03:06:22.455 INFO:teuthology.orchestra.run.smithi106.stdout:(23/174): python3-rbd-19.0.0-2884.g218a6914.el9 3.9 MB/s | 298 kB 00:00 2024-04-13T03:06:22.522 INFO:teuthology.orchestra.run.smithi106.stdout:(24/174): python3-rgw-19.0.0-2884.g218a6914.el9 1.5 MB/s | 100 kB 00:00 2024-04-13T03:06:22.589 INFO:teuthology.orchestra.run.smithi106.stdout:(25/174): rbd-fuse-19.0.0-2884.g218a6914.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-13T03:06:22.931 INFO:teuthology.orchestra.run.smithi106.stdout:(26/174): rbd-mirror-19.0.0-2884.g218a6914.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-13T03:06:22.998 INFO:teuthology.orchestra.run.smithi106.stdout:(27/174): rbd-nbd-19.0.0-2884.g218a6914.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-13T03:06:23.086 INFO:teuthology.orchestra.run.smithi106.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2884.g 386 kB/s | 34 kB 00:00 2024-04-13T03:06:23.137 INFO:teuthology.orchestra.run.smithi114.stdout:(10/174): ceph-radosgw-19.0.0-2884.g218a6914.el 6.4 MB/s | 10 MB 00:01 2024-04-13T03:06:23.148 INFO:teuthology.orchestra.run.smithi106.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2884.g218a691 2.2 MB/s | 142 kB 00:00 2024-04-13T03:06:23.204 INFO:teuthology.orchestra.run.smithi114.stdout:(11/174): cephfs-mirror-19.0.0-2884.g218a6914.e 3.3 MB/s | 225 kB 00:00 2024-04-13T03:06:23.280 INFO:teuthology.orchestra.run.smithi114.stdout:(12/174): cephfs-shell-19.0.0-2884.g218a6914.el 360 kB/s | 27 kB 00:00 2024-04-13T03:06:23.354 INFO:teuthology.orchestra.run.smithi114.stdout:(13/174): libcephfs-devel-19.0.0-2884.g218a6914 423 kB/s | 32 kB 00:00 2024-04-13T03:06:23.499 INFO:teuthology.orchestra.run.smithi106.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2884.g218a6 10 MB/s | 3.5 MB 00:00 2024-04-13T03:06:23.530 INFO:teuthology.orchestra.run.smithi114.stdout:(14/174): libcephfs2-19.0.0-2884.g218a6914.el9. 4.2 MB/s | 746 kB 00:00 2024-04-13T03:06:23.604 INFO:teuthology.orchestra.run.smithi114.stdout:(15/174): libcephsqlite-19.0.0-2884.g218a6914.e 2.1 MB/s | 163 kB 00:00 2024-04-13T03:06:23.689 INFO:teuthology.orchestra.run.smithi114.stdout:(16/174): librados-devel-19.0.0-2884.g218a6914. 1.5 MB/s | 126 kB 00:00 2024-04-13T03:06:23.838 INFO:teuthology.orchestra.run.smithi114.stdout:(17/174): libradosstriper1-19.0.0-2884.g218a691 3.3 MB/s | 500 kB 00:00 2024-04-13T03:06:24.207 INFO:teuthology.orchestra.run.smithi106.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-13T03:06:24.282 INFO:teuthology.orchestra.run.smithi106.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2884.g21 3.3 MB/s | 249 kB 00:00 2024-04-13T03:06:24.349 INFO:teuthology.orchestra.run.smithi106.stdout:(33/174): ceph-mgr-rook-19.0.0-2884.g218a6914.e 738 kB/s | 49 kB 00:00 2024-04-13T03:06:24.417 INFO:teuthology.orchestra.run.smithi106.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2884.g2 245 kB/s | 16 kB 00:00 2024-04-13T03:06:24.492 INFO:teuthology.orchestra.run.smithi106.stdout:(35/174): ceph-volume-19.0.0-2884.g218a6914.el9 3.4 MB/s | 264 kB 00:00 2024-04-13T03:06:24.634 INFO:teuthology.orchestra.run.smithi106.stdout:(36/174): cephadm-19.0.0-2884.g218a6914.el9.noa 5.3 MB/s | 763 kB 00:00 2024-04-13T03:06:24.682 INFO:teuthology.orchestra.run.smithi114.stdout:(18/174): librgw2-19.0.0-2884.g218a6914.el9.x86 6.2 MB/s | 5.2 MB 00:00 2024-04-13T03:06:24.701 INFO:teuthology.orchestra.run.smithi106.stdout:(37/174): cephfs-top-19.0.0-2884.g218a6914.el9. 273 kB/s | 18 kB 00:00 2024-04-13T03:06:24.739 INFO:teuthology.orchestra.run.smithi114.stdout:(19/174): python3-ceph-argparse-19.0.0-2884.g21 775 kB/s | 45 kB 00:00 2024-04-13T03:06:24.784 INFO:teuthology.orchestra.run.smithi106.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 211 kB/s | 18 kB 00:00 2024-04-13T03:06:24.798 INFO:teuthology.orchestra.run.smithi114.stdout:(20/174): python3-ceph-common-19.0.0-2884.g218a 2.3 MB/s | 135 kB 00:00 2024-04-13T03:06:24.856 INFO:teuthology.orchestra.run.smithi114.stdout:(21/174): python3-cephfs-19.0.0-2884.g218a6914. 2.7 MB/s | 162 kB 00:00 2024-04-13T03:06:24.931 INFO:teuthology.orchestra.run.smithi114.stdout:(22/174): python3-rados-19.0.0-2884.g218a6914.e 4.2 MB/s | 321 kB 00:00 2024-04-13T03:06:24.943 INFO:teuthology.orchestra.run.smithi106.stdout:(39/174): ceph-osd-19.0.0-2884.g218a6914.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-13T03:06:24.985 INFO:teuthology.orchestra.run.smithi106.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-13T03:06:24.998 INFO:teuthology.orchestra.run.smithi114.stdout:(23/174): python3-rbd-19.0.0-2884.g218a6914.el9 4.4 MB/s | 298 kB 00:00 2024-04-13T03:06:25.027 INFO:teuthology.orchestra.run.smithi106.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-04-13T03:06:25.057 INFO:teuthology.orchestra.run.smithi114.stdout:(24/174): python3-rgw-19.0.0-2884.g218a6914.el9 1.7 MB/s | 100 kB 00:00 2024-04-13T03:06:25.094 INFO:teuthology.orchestra.run.smithi106.stdout:(42/174): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-13T03:06:25.119 INFO:teuthology.orchestra.run.smithi106.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-13T03:06:25.124 INFO:teuthology.orchestra.run.smithi114.stdout:(25/174): rbd-fuse-19.0.0-2884.g218a6914.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-13T03:06:25.186 INFO:teuthology.orchestra.run.smithi106.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 218 kB 00:00 2024-04-13T03:06:25.236 INFO:teuthology.orchestra.run.smithi106.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-13T03:06:25.312 INFO:teuthology.orchestra.run.smithi106.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-13T03:06:25.353 INFO:teuthology.orchestra.run.smithi106.stdout:(47/174): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-04-13T03:06:25.395 INFO:teuthology.orchestra.run.smithi106.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-13T03:06:25.437 INFO:teuthology.orchestra.run.smithi106.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-13T03:06:25.488 INFO:teuthology.orchestra.run.smithi106.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-13T03:06:25.541 INFO:teuthology.orchestra.run.smithi114.stdout:(26/174): rbd-mirror-19.0.0-2884.g218a6914.el9. 7.4 MB/s | 3.1 MB 00:00 2024-04-13T03:06:25.608 INFO:teuthology.orchestra.run.smithi114.stdout:(27/174): rbd-nbd-19.0.0-2884.g218a6914.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-13T03:06:25.666 INFO:teuthology.orchestra.run.smithi114.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2884.g 586 kB/s | 34 kB 00:00 2024-04-13T03:06:25.734 INFO:teuthology.orchestra.run.smithi114.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2884.g218a691 2.1 MB/s | 142 kB 00:00 2024-04-13T03:06:26.121 INFO:teuthology.orchestra.run.smithi106.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-04-13T03:06:26.225 INFO:teuthology.orchestra.run.smithi114.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2884.g218a6 7.2 MB/s | 3.5 MB 00:00 2024-04-13T03:06:26.229 INFO:teuthology.orchestra.run.smithi106.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 107 kB 00:00 2024-04-13T03:06:26.330 INFO:teuthology.orchestra.run.smithi106.stdout:(53/174): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-04-13T03:06:26.355 INFO:teuthology.orchestra.run.smithi106.stdout:(54/174): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-13T03:06:26.439 INFO:teuthology.orchestra.run.smithi106.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-13T03:06:26.464 INFO:teuthology.orchestra.run.smithi106.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-04-13T03:06:26.573 INFO:teuthology.orchestra.run.smithi106.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-13T03:06:26.682 INFO:teuthology.orchestra.run.smithi106.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-13T03:06:26.790 INFO:teuthology.orchestra.run.smithi106.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-13T03:06:26.907 INFO:teuthology.orchestra.run.smithi106.stdout:(60/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-13T03:06:26.974 INFO:teuthology.orchestra.run.smithi106.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-13T03:06:27.016 INFO:teuthology.orchestra.run.smithi106.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-13T03:06:27.075 INFO:teuthology.orchestra.run.smithi106.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-13T03:06:27.125 INFO:teuthology.orchestra.run.smithi106.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-13T03:06:27.184 INFO:teuthology.orchestra.run.smithi106.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-13T03:06:27.234 INFO:teuthology.orchestra.run.smithi114.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-04-13T03:06:27.234 INFO:teuthology.orchestra.run.smithi106.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-13T03:06:27.284 INFO:teuthology.orchestra.run.smithi106.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-04-13T03:06:27.317 INFO:teuthology.orchestra.run.smithi114.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2884.g21 2.9 MB/s | 249 kB 00:00 2024-04-13T03:06:27.376 INFO:teuthology.orchestra.run.smithi114.stdout:(33/174): ceph-mgr-rook-19.0.0-2884.g218a6914.e 845 kB/s | 49 kB 00:00 2024-04-13T03:06:27.426 INFO:teuthology.orchestra.run.smithi114.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2884.g2 327 kB/s | 16 kB 00:00 2024-04-13T03:06:27.502 INFO:teuthology.orchestra.run.smithi114.stdout:(35/174): ceph-volume-19.0.0-2884.g218a6914.el9 3.4 MB/s | 264 kB 00:00 2024-04-13T03:06:27.660 INFO:teuthology.orchestra.run.smithi114.stdout:(36/174): cephadm-19.0.0-2884.g218a6914.el9.noa 4.7 MB/s | 763 kB 00:00 2024-04-13T03:06:27.729 INFO:teuthology.orchestra.run.smithi114.stdout:(37/174): cephfs-top-19.0.0-2884.g218a6914.el9. 264 kB/s | 18 kB 00:00 2024-04-13T03:06:27.905 INFO:teuthology.orchestra.run.smithi114.stdout:(38/174): ceph-osd-19.0.0-2884.g218a6914.el9.x8 2.6 MB/s | 17 MB 00:06 2024-04-13T03:06:27.980 INFO:teuthology.orchestra.run.smithi114.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 70 kB/s | 18 kB 00:00 2024-04-13T03:06:28.460 INFO:teuthology.orchestra.run.smithi106.stdout:(68/174): ceph-test-19.0.0-2884.g218a6914.el9.x 6.6 MB/s | 49 MB 00:07 2024-04-13T03:06:28.630 INFO:teuthology.orchestra.run.smithi114.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-04-13T03:06:28.806 INFO:teuthology.orchestra.run.smithi114.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 227 kB/s | 187 kB 00:00 2024-04-13T03:06:29.301 INFO:teuthology.orchestra.run.smithi106.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-04-13T03:06:29.356 INFO:teuthology.orchestra.run.smithi114.stdout:(42/174): ceph-test-19.0.0-2884.g218a6914.el9.x 7.0 MB/s | 49 MB 00:06 2024-04-13T03:06:29.502 INFO:teuthology.orchestra.run.smithi106.stdout:(70/174): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-13T03:06:29.610 INFO:teuthology.orchestra.run.smithi106.stdout:(71/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-13T03:06:29.694 INFO:teuthology.orchestra.run.smithi106.stdout:(72/174): openblas-openmp-0.3.21-2.el9.x86_64.r 1.9 MB/s | 4.7 MB 00:02 2024-04-13T03:06:30.048 INFO:teuthology.orchestra.run.smithi114.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 157 kB/s | 195 kB 00:01 2024-04-13T03:06:30.265 INFO:teuthology.orchestra.run.smithi114.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 240 kB/s | 218 kB 00:00 2024-04-13T03:06:30.511 INFO:teuthology.orchestra.run.smithi106.stdout:(73/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 4.1 MB 00:03 2024-04-13T03:06:30.590 INFO:teuthology.orchestra.run.smithi114.stdout:(45/174): libgfortran-11.4.1-3.el9.x86_64.rpm 410 kB/s | 803 kB 00:01 2024-04-13T03:06:30.932 INFO:teuthology.orchestra.run.smithi114.stdout:(46/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 287 kB/s | 253 kB 00:00 2024-04-13T03:06:31.111 INFO:teuthology.orchestra.run.smithi106.stdout:(74/174): protobuf-3.14.0-13.el9.x86_64.rpm 729 kB/s | 1.0 MB 00:01 2024-04-13T03:06:31.349 INFO:teuthology.orchestra.run.smithi114.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 141 kB/s | 106 kB 00:00 2024-04-13T03:06:31.716 INFO:teuthology.orchestra.run.smithi114.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 173 kB/s | 135 kB 00:00 2024-04-13T03:06:32.083 INFO:teuthology.orchestra.run.smithi114.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 170 kB/s | 125 kB 00:00 2024-04-13T03:06:32.400 INFO:teuthology.orchestra.run.smithi114.stdout:(50/174): python3-cryptography-36.0.1-4.el9.x86 588 kB/s | 1.2 MB 00:02 2024-04-13T03:06:32.558 INFO:teuthology.orchestra.run.smithi114.stdout:(51/174): python3-urllib3-1.26.5-5.el9.noarch.r 255 kB/s | 215 kB 00:00 2024-04-13T03:06:33.319 INFO:teuthology.orchestra.run.smithi106.stdout:(75/174): python3-attrs-20.3.0-7.el9.noarch.rpm 31 kB/s | 87 kB 00:02 2024-04-13T03:06:33.333 INFO:teuthology.orchestra.run.smithi114.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 138 kB/s | 107 kB 00:00 2024-04-13T03:06:33.533 INFO:teuthology.orchestra.run.smithi114.stdout:(53/174): automake-1.16.2-8.el9.noarch.rpm 475 kB/s | 689 kB 00:01 2024-04-13T03:06:33.586 INFO:teuthology.orchestra.run.smithi106.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 920 kB/s | 245 kB 00:00 2024-04-13T03:06:33.745 INFO:teuthology.orchestra.run.smithi106.stdout:(77/174): postgresql-server-13.14-1.el9.x86_64. 1.4 MB/s | 5.8 MB 00:04 2024-04-13T03:06:33.812 INFO:teuthology.orchestra.run.smithi106.stdout:(78/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-04-13T03:06:33.837 INFO:teuthology.orchestra.run.smithi106.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-04-13T03:06:33.900 INFO:teuthology.orchestra.run.smithi114.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 38 kB/s | 21 kB 00:00 2024-04-13T03:06:33.904 INFO:teuthology.orchestra.run.smithi106.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-04-13T03:06:33.996 INFO:teuthology.orchestra.run.smithi106.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 379 kB/s | 35 kB 00:00 2024-04-13T03:06:34.096 INFO:teuthology.orchestra.run.smithi106.stdout:(82/174): python3-mako-1.1.4-6.el9.noarch.rpm 666 kB/s | 172 kB 00:00 2024-04-13T03:06:34.150 INFO:teuthology.orchestra.run.smithi114.stdout:(55/174): bison-3.7.4-5.el9.x86_64.rpm 539 kB/s | 943 kB 00:01 2024-04-13T03:06:34.438 INFO:teuthology.orchestra.run.smithi106.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-04-13T03:06:34.543 INFO:teuthology.orchestra.run.smithi114.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 51 kB/s | 33 kB 00:00 2024-04-13T03:06:34.618 INFO:teuthology.orchestra.run.smithi114.stdout:(57/174): flex-2.6.4-9.el9.x86_64.rpm 289 kB/s | 313 kB 00:01 2024-04-13T03:06:34.780 INFO:teuthology.orchestra.run.smithi106.stdout:(84/174): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-04-13T03:06:35.005 INFO:teuthology.orchestra.run.smithi106.stdout:(85/174): python3-packaging-20.9-5.el9.noarch.r 343 kB/s | 77 kB 00:00 2024-04-13T03:06:35.068 INFO:teuthology.orchestra.run.smithi114.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-04-13T03:06:35.122 INFO:teuthology.orchestra.run.smithi106.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-04-13T03:06:35.485 INFO:teuthology.orchestra.run.smithi114.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 185 kB/s | 160 kB 00:00 2024-04-13T03:06:35.539 INFO:teuthology.orchestra.run.smithi106.stdout:(87/174): python3-protobuf-3.14.0-13.el9.noarch 646 kB/s | 269 kB 00:00 2024-04-13T03:06:35.710 INFO:teuthology.orchestra.run.smithi114.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-04-13T03:06:35.807 INFO:teuthology.orchestra.run.smithi106.stdout:(88/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-04-13T03:06:36.231 INFO:teuthology.orchestra.run.smithi106.stdout:(89/174): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-04-13T03:06:36.332 INFO:teuthology.orchestra.run.smithi106.stdout:(90/174): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-04-13T03:06:36.432 INFO:teuthology.orchestra.run.smithi106.stdout:(91/174): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-04-13T03:06:36.699 INFO:teuthology.orchestra.run.smithi106.stdout:(92/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-04-13T03:06:36.799 INFO:teuthology.orchestra.run.smithi106.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-04-13T03:06:36.894 INFO:teuthology.orchestra.run.smithi114.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 470 kB/s | 662 kB 00:01 2024-04-13T03:06:36.975 INFO:teuthology.orchestra.run.smithi106.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 252 kB/s | 44 kB 00:00 2024-04-13T03:06:37.102 INFO:teuthology.orchestra.run.smithi114.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 427 kB/s | 582 kB 00:01 2024-04-13T03:06:37.325 INFO:teuthology.orchestra.run.smithi106.stdout:(95/174): qatlib-24.02.0-1.el9.x86_64.rpm 631 kB/s | 221 kB 00:00 2024-04-13T03:06:37.433 INFO:teuthology.orchestra.run.smithi106.stdout:(96/174): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-13T03:06:37.486 INFO:teuthology.orchestra.run.smithi114.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-04-13T03:06:37.542 INFO:teuthology.orchestra.run.smithi106.stdout:(97/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-04-13T03:06:37.753 INFO:teuthology.orchestra.run.smithi114.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-04-13T03:06:37.967 INFO:teuthology.orchestra.run.smithi106.stdout:(98/174): socat-1.7.4.1-5.el9.x86_64.rpm 717 kB/s | 305 kB 00:00 2024-04-13T03:06:38.076 INFO:teuthology.orchestra.run.smithi106.stdout:(99/174): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-04-13T03:06:38.268 INFO:teuthology.orchestra.run.smithi106.stdout:(100/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 489 kB/s | 94 kB 00:00 2024-04-13T03:06:38.451 INFO:teuthology.orchestra.run.smithi106.stdout:(101/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-04-13T03:06:38.453 INFO:teuthology.orchestra.run.smithi114.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 302 kB/s | 292 kB 00:00 2024-04-13T03:06:38.512 INFO:teuthology.orchestra.run.smithi114.stdout:(66/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 706 kB/s | 3.0 MB 00:04 2024-04-13T03:06:38.560 INFO:teuthology.orchestra.run.smithi106.stdout:(102/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-13T03:06:38.644 INFO:teuthology.orchestra.run.smithi106.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-13T03:06:38.645 INFO:teuthology.orchestra.run.smithi114.stdout:(67/174): lua-5.4.4-4.el9.x86_64.rpm 211 kB/s | 188 kB 00:00 2024-04-13T03:06:38.902 INFO:teuthology.orchestra.run.smithi106.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-04-13T03:06:39.054 INFO:teuthology.orchestra.run.smithi114.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-04-13T03:06:39.611 INFO:teuthology.orchestra.run.smithi106.stdout:(105/174): python3-babel-2.9.1-2.el9.noarch.rpm 718 kB/s | 6.0 MB 00:08 2024-04-13T03:06:39.644 INFO:teuthology.orchestra.run.smithi106.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 739 kB/s | 548 kB 00:00 2024-04-13T03:06:39.744 INFO:teuthology.orchestra.run.smithi106.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-13T03:06:40.153 INFO:teuthology.orchestra.run.smithi106.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-13T03:06:40.253 INFO:teuthology.orchestra.run.smithi106.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-13T03:06:40.279 INFO:teuthology.orchestra.run.smithi106.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 463 kB/s | 309 kB 00:00 2024-04-13T03:06:40.363 INFO:teuthology.orchestra.run.smithi106.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-04-13T03:06:40.396 INFO:teuthology.orchestra.run.smithi106.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-13T03:06:40.471 INFO:teuthology.orchestra.run.smithi106.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-13T03:06:40.589 INFO:teuthology.orchestra.run.smithi106.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-13T03:06:40.664 INFO:teuthology.orchestra.run.smithi106.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-13T03:06:40.697 INFO:teuthology.orchestra.run.smithi106.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-13T03:06:40.773 INFO:teuthology.orchestra.run.smithi106.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-13T03:06:40.798 INFO:teuthology.orchestra.run.smithi106.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-13T03:06:40.898 INFO:teuthology.orchestra.run.smithi106.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-13T03:06:40.923 INFO:teuthology.orchestra.run.smithi106.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-13T03:06:41.024 INFO:teuthology.orchestra.run.smithi106.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-13T03:06:41.040 INFO:teuthology.orchestra.run.smithi106.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-13T03:06:41.141 INFO:teuthology.orchestra.run.smithi106.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-13T03:06:41.200 INFO:teuthology.orchestra.run.smithi106.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-13T03:06:41.258 INFO:teuthology.orchestra.run.smithi106.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-13T03:06:41.292 INFO:teuthology.orchestra.run.smithi106.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-13T03:06:41.359 INFO:teuthology.orchestra.run.smithi106.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-13T03:06:41.392 INFO:teuthology.orchestra.run.smithi106.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-13T03:06:41.459 INFO:teuthology.orchestra.run.smithi106.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-13T03:06:41.484 INFO:teuthology.orchestra.run.smithi106.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-13T03:06:41.585 INFO:teuthology.orchestra.run.smithi106.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-13T03:06:41.643 INFO:teuthology.orchestra.run.smithi106.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-04-13T03:06:41.694 INFO:teuthology.orchestra.run.smithi106.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-13T03:06:41.752 INFO:teuthology.orchestra.run.smithi106.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-13T03:06:41.754 INFO:teuthology.orchestra.run.smithi114.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 602 kB/s | 1.6 MB 00:02 2024-04-13T03:06:41.803 INFO:teuthology.orchestra.run.smithi106.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-13T03:06:41.861 INFO:teuthology.orchestra.run.smithi106.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-13T03:06:41.912 INFO:teuthology.orchestra.run.smithi106.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-13T03:06:41.970 INFO:teuthology.orchestra.run.smithi106.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-13T03:06:42.020 INFO:teuthology.orchestra.run.smithi106.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-13T03:06:42.087 INFO:teuthology.orchestra.run.smithi106.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-13T03:06:42.129 INFO:teuthology.orchestra.run.smithi106.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-13T03:06:42.196 INFO:teuthology.orchestra.run.smithi106.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-13T03:06:42.238 INFO:teuthology.orchestra.run.smithi106.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-13T03:06:42.322 INFO:teuthology.orchestra.run.smithi106.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-13T03:06:42.347 INFO:teuthology.orchestra.run.smithi106.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-13T03:06:42.447 INFO:teuthology.orchestra.run.smithi106.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-13T03:06:42.473 INFO:teuthology.orchestra.run.smithi106.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-04-13T03:06:42.565 INFO:teuthology.orchestra.run.smithi106.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-13T03:06:42.607 INFO:teuthology.orchestra.run.smithi106.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-13T03:06:42.924 INFO:teuthology.orchestra.run.smithi106.stdout:(150/174): librados2-19.0.0-2884.g218a6914.el9. 9.5 MB/s | 3.4 MB 00:00 2024-04-13T03:06:43.057 INFO:teuthology.orchestra.run.smithi106.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-04-13T03:06:43.183 INFO:teuthology.orchestra.run.smithi106.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-04-13T03:06:43.233 INFO:teuthology.orchestra.run.smithi106.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 839 kB/s | 42 kB 00:00 2024-04-13T03:06:43.308 INFO:teuthology.orchestra.run.smithi106.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 3.3 MB/s | 249 kB 00:00 2024-04-13T03:06:43.367 INFO:teuthology.orchestra.run.smithi106.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-04-13T03:06:43.408 INFO:teuthology.orchestra.run.smithi106.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 878 kB/s | 36 kB 00:00 2024-04-13T03:06:43.467 INFO:teuthology.orchestra.run.smithi106.stdout:(157/174): librbd1-19.0.0-2884.g218a6914.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-13T03:06:43.504 INFO:teuthology.orchestra.run.smithi114.stdout:(70/174): postgresql-contrib-13.14-1.el9.x86_64 521 kB/s | 911 kB 00:01 2024-04-13T03:06:43.526 INFO:teuthology.orchestra.run.smithi106.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-13T03:06:43.576 INFO:teuthology.orchestra.run.smithi106.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-13T03:06:43.618 INFO:teuthology.orchestra.run.smithi106.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 250 kB/s | 23 kB 00:00 2024-04-13T03:06:43.660 INFO:teuthology.orchestra.run.smithi106.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-13T03:06:43.710 INFO:teuthology.orchestra.run.smithi106.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-04-13T03:06:43.752 INFO:teuthology.orchestra.run.smithi106.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-04-13T03:06:43.778 INFO:teuthology.orchestra.run.smithi106.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 955 kB/s | 64 kB 00:00 2024-04-13T03:06:43.819 INFO:teuthology.orchestra.run.smithi106.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 430 kB/s | 29 kB 00:00 2024-04-13T03:06:43.903 INFO:teuthology.orchestra.run.smithi106.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-04-13T03:06:43.978 INFO:teuthology.orchestra.run.smithi106.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-04-13T03:06:44.070 INFO:teuthology.orchestra.run.smithi106.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 5.0 MB/s | 465 kB 00:00 2024-04-13T03:06:44.129 INFO:teuthology.orchestra.run.smithi106.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 10 MB/s | 2.3 MB 00:00 2024-04-13T03:06:44.171 INFO:teuthology.orchestra.run.smithi106.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-04-13T03:06:44.238 INFO:teuthology.orchestra.run.smithi106.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2024-04-13T03:06:44.279 INFO:teuthology.orchestra.run.smithi114.stdout:(71/174): postgresql-private-libs-13.14-1.el9.x 172 kB/s | 133 kB 00:00 2024-04-13T03:06:44.313 INFO:teuthology.orchestra.run.smithi106.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-13T03:06:44.429 INFO:teuthology.orchestra.run.smithi114.stdout:(72/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 732 kB/s | 4.1 MB 00:05 2024-04-13T03:06:44.538 INFO:teuthology.orchestra.run.smithi106.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 247 kB/s | 74 kB 00:00 2024-04-13T03:06:45.080 INFO:teuthology.orchestra.run.smithi114.stdout:(73/174): openblas-openmp-0.3.21-2.el9.x86_64.r 739 kB/s | 4.7 MB 00:06 2024-04-13T03:06:45.788 INFO:teuthology.orchestra.run.smithi114.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 124 kB/s | 87 kB 00:00 2024-04-13T03:06:46.288 INFO:teuthology.orchestra.run.smithi114.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 556 kB/s | 1.0 MB 00:01 2024-04-13T03:06:47.164 INFO:teuthology.orchestra.run.smithi114.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 280 kB/s | 245 kB 00:00 2024-04-13T03:06:48.064 INFO:teuthology.orchestra.run.smithi114.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 275 kB/s | 248 kB 00:00 2024-04-13T03:06:48.656 INFO:teuthology.orchestra.run.smithi114.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-13T03:06:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-04-13T03:06:50.048 INFO:teuthology.orchestra.run.smithi114.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-04-13T03:06:50.681 INFO:teuthology.orchestra.run.smithi114.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-04-13T03:06:52.432 INFO:teuthology.orchestra.run.smithi114.stdout:(82/174): postgresql-server-13.14-1.el9.x86_64. 726 kB/s | 5.8 MB 00:08 2024-04-13T03:06:53.373 INFO:teuthology.orchestra.run.smithi114.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 266 kB/s | 250 kB 00:00 2024-04-13T03:06:53.907 INFO:teuthology.orchestra.run.smithi114.stdout:(84/174): python3-babel-2.9.1-2.el9.noarch.rpm 752 kB/s | 6.0 MB 00:08 2024-04-13T03:06:54.291 INFO:teuthology.orchestra.run.smithi114.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 242 kB/s | 222 kB 00:00 2024-04-13T03:06:54.608 INFO:teuthology.orchestra.run.smithi114.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-04-13T03:06:54.916 INFO:teuthology.orchestra.run.smithi114.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-04-13T03:06:55.550 INFO:teuthology.orchestra.run.smithi114.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 286 kB/s | 269 kB 00:00 2024-04-13T03:06:55.691 INFO:teuthology.orchestra.run.smithi114.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 205 kB/s | 159 kB 00:00 2024-04-13T03:06:56.342 INFO:teuthology.orchestra.run.smithi114.stdout:(90/174): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-04-13T03:06:56.492 INFO:teuthology.orchestra.run.smithi114.stdout:(91/174): python3-pyasn1-modules-0.4.8-6.el9.no 296 kB/s | 279 kB 00:00 2024-04-13T03:06:57.018 INFO:teuthology.orchestra.run.smithi114.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-04-13T03:06:57.218 INFO:teuthology.orchestra.run.smithi114.stdout:(93/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 799 kB/s | 5.1 MB 00:06 2024-04-13T03:06:57.710 INFO:teuthology.orchestra.run.smithi114.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-04-13T03:06:57.910 INFO:teuthology.orchestra.run.smithi114.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 64 kB/s | 44 kB 00:00 2024-04-13T03:06:58.005 INFO:teuthology.orchestra.run.smithi106.stdout:(174/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 771 kB/s | 16 MB 00:21 2024-04-13T03:06:58.010 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-13T03:06:58.010 INFO:teuthology.orchestra.run.smithi106.stdout:Total 5.4 MB/s | 229 MB 00:42 2024-04-13T03:06:58.527 INFO:teuthology.orchestra.run.smithi114.stdout:(96/174): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-04-13T03:06:58.577 INFO:teuthology.orchestra.run.smithi114.stdout:(97/174): qatlib-24.02.0-1.el9.x86_64.rpm 255 kB/s | 221 kB 00:00 2024-04-13T03:06:59.161 INFO:teuthology.orchestra.run.smithi114.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-04-13T03:06:59.334 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-13T03:06:59.586 INFO:teuthology.orchestra.run.smithi114.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 302 kB/s | 305 kB 00:01 2024-04-13T03:06:59.618 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-13T03:06:59.618 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-13T03:06:59.878 INFO:teuthology.orchestra.run.smithi114.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 81 kB/s | 58 kB 00:00 2024-04-13T03:07:00.320 INFO:teuthology.orchestra.run.smithi114.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 128 kB/s | 94 kB 00:00 2024-04-13T03:07:00.595 INFO:teuthology.orchestra.run.smithi114.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-04-13T03:07:00.770 INFO:teuthology.orchestra.run.smithi114.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-04-13T03:07:00.920 INFO:teuthology.orchestra.run.smithi114.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-13T03:07:00.945 INFO:teuthology.orchestra.run.smithi114.stdout:(105/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-04-13T03:07:01.379 INFO:teuthology.orchestra.run.smithi114.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 713 kB/s | 309 kB 00:00 2024-04-13T03:07:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-13T03:07:01.463 INFO:teuthology.orchestra.run.smithi114.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-04-13T03:07:01.555 INFO:teuthology.orchestra.run.smithi114.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-13T03:07:01.638 INFO:teuthology.orchestra.run.smithi114.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-04-13T03:07:01.731 INFO:teuthology.orchestra.run.smithi114.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-04-13T03:07:01.775 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-13T03:07:01.775 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-13T03:07:01.789 INFO:teuthology.orchestra.run.smithi114.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-13T03:07:01.823 INFO:teuthology.orchestra.run.smithi114.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-13T03:07:01.898 INFO:teuthology.orchestra.run.smithi114.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-04-13T03:07:01.999 INFO:teuthology.orchestra.run.smithi114.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-04-13T03:07:02.032 INFO:teuthology.orchestra.run.smithi114.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-13T03:07:02.099 INFO:teuthology.orchestra.run.smithi114.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-13T03:07:02.133 INFO:teuthology.orchestra.run.smithi114.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-13T03:07:02.208 INFO:teuthology.orchestra.run.smithi114.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-13T03:07:02.250 INFO:teuthology.orchestra.run.smithi114.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-13T03:07:02.325 INFO:teuthology.orchestra.run.smithi114.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-13T03:07:02.359 INFO:teuthology.orchestra.run.smithi114.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-13T03:07:02.434 INFO:teuthology.orchestra.run.smithi114.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-13T03:07:02.543 INFO:teuthology.orchestra.run.smithi114.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-13T03:07:02.652 INFO:teuthology.orchestra.run.smithi114.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-13T03:07:02.744 INFO:teuthology.orchestra.run.smithi114.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-04-13T03:07:02.769 INFO:teuthology.orchestra.run.smithi114.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-13T03:07:02.836 INFO:teuthology.orchestra.run.smithi114.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-13T03:07:02.861 INFO:teuthology.orchestra.run.smithi114.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-13T03:07:02.928 INFO:teuthology.orchestra.run.smithi114.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-13T03:07:02.970 INFO:teuthology.orchestra.run.smithi114.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-13T03:07:03.028 INFO:teuthology.orchestra.run.smithi114.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-13T03:07:03.062 INFO:teuthology.orchestra.run.smithi114.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-04-13T03:07:03.121 INFO:teuthology.orchestra.run.smithi114.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-13T03:07:03.154 INFO:teuthology.orchestra.run.smithi114.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-13T03:07:03.213 INFO:teuthology.orchestra.run.smithi114.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-13T03:07:03.272 INFO:teuthology.orchestra.run.smithi114.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-04-13T03:07:03.305 INFO:teuthology.orchestra.run.smithi114.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-13T03:07:03.364 INFO:teuthology.orchestra.run.smithi114.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-13T03:07:03.397 INFO:teuthology.orchestra.run.smithi114.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-13T03:07:03.456 INFO:teuthology.orchestra.run.smithi114.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-13T03:07:03.490 INFO:teuthology.orchestra.run.smithi114.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-13T03:07:03.548 INFO:teuthology.orchestra.run.smithi114.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-13T03:07:03.582 INFO:teuthology.orchestra.run.smithi114.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-13T03:07:03.640 INFO:teuthology.orchestra.run.smithi114.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 976 kB/s | 90 kB 00:00 2024-04-13T03:07:03.682 INFO:teuthology.orchestra.run.smithi114.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-13T03:07:03.732 INFO:teuthology.orchestra.run.smithi114.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-13T03:07:03.774 INFO:teuthology.orchestra.run.smithi114.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-13T03:07:04.550 INFO:teuthology.orchestra.run.smithi114.stdout:(149/174): librados2-19.0.0-2884.g218a6914.el9. 4.4 MB/s | 3.4 MB 00:00 2024-04-13T03:07:04.684 INFO:teuthology.orchestra.run.smithi114.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-13T03:07:05.092 INFO:teuthology.orchestra.run.smithi114.stdout:(151/174): librbd1-19.0.0-2884.g218a6914.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-04-13T03:07:08.084 INFO:teuthology.orchestra.run.smithi114.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 626 kB/s | 1.8 MB 00:02 2024-04-13T03:07:08.142 INFO:teuthology.orchestra.run.smithi114.stdout:(153/174): NetworkManager-1.47.4-1.el9.x86_64.r 684 kB/s | 2.3 MB 00:03 2024-04-13T03:07:08.709 INFO:teuthology.orchestra.run.smithi114.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 67 kB/s | 42 kB 00:00 2024-04-13T03:07:08.772 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-13T03:07:08.875 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-13T03:07:08.910 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-13T03:07:08.948 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T03:07:08.958 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T03:07:09.006 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-13T03:07:09.060 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-13T03:07:09.068 INFO:teuthology.orchestra.run.smithi114.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 269 kB/s | 249 kB 00:00 2024-04-13T03:07:09.093 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-13T03:07:09.123 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-13T03:07:09.172 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-13T03:07:09.410 INFO:teuthology.orchestra.run.smithi114.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 102 kB/s | 71 kB 00:00 2024-04-13T03:07:09.494 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-13T03:07:09.566 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:07:09.672 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:07:09.701 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:07:09.710 INFO:teuthology.orchestra.run.smithi114.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 57 kB/s | 36 kB 00:00 2024-04-13T03:07:09.776 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:07:09.805 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T03:07:09.841 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-13T03:07:10.025 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-13T03:07:10.086 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-13T03:07:10.106 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:07:10.209 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:07:10.232 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T03:07:10.285 INFO:teuthology.orchestra.run.smithi114.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 220 kB/s | 192 kB 00:00 2024-04-13T03:07:10.313 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T03:07:10.353 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-13T03:07:10.375 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T03:07:10.391 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T03:07:10.584 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T03:07:10.624 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T03:07:10.644 INFO:teuthology.orchestra.run.smithi114.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 274 kB/s | 256 kB 00:00 2024-04-13T03:07:10.655 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-13T03:07:10.855 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-13T03:07:10.861 INFO:teuthology.orchestra.run.smithi114.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 40 kB/s | 23 kB 00:00 2024-04-13T03:07:10.985 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T03:07:11.045 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T03:07:11.145 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:11.188 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:11.378 INFO:teuthology.orchestra.run.smithi114.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 148 kB/s | 108 kB 00:00 2024-04-13T03:07:11.396 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:11.499 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-13T03:07:11.587 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-13T03:07:11.618 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-13T03:07:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-13T03:07:11.678 INFO:teuthology.orchestra.run.smithi114.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 190 kB/s | 155 kB 00:00 2024-04-13T03:07:11.717 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-13T03:07:11.751 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-13T03:07:11.794 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-13T03:07:11.839 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-13T03:07:11.872 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-13T03:07:11.940 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-13T03:07:11.993 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-13T03:07:12.033 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-13T03:07:12.077 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-13T03:07:12.145 INFO:teuthology.orchestra.run.smithi114.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 177 kB/s | 136 kB 00:00 2024-04-13T03:07:12.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-13T03:07:12.187 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 39/198 2024-04-13T03:07:12.250 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 40/198 2024-04-13T03:07:12.284 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 41/198 2024-04-13T03:07:12.337 INFO:teuthology.orchestra.run.smithi114.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 97 kB/s | 64 kB 00:00 2024-04-13T03:07:12.353 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-13T03:07:12.398 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-13T03:07:12.472 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-13T03:07:12.615 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-13T03:07:12.667 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-13T03:07:12.729 INFO:teuthology.orchestra.run.smithi114.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 49 kB/s | 29 kB 00:00 2024-04-13T03:07:13.191 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-13T03:07:13.271 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-13T03:07:13.362 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-13T03:07:13.407 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T03:07:13.417 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T03:07:13.448 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-13T03:07:13.488 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-13T03:07:13.526 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 53/198 2024-04-13T03:07:13.565 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-13T03:07:13.830 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-13T03:07:13.986 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-13T03:07:14.067 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-13T03:07:14.110 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-13T03:07:14.245 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-13T03:07:14.289 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-13T03:07:14.352 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-13T03:07:14.424 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-13T03:07:14.437 INFO:teuthology.orchestra.run.smithi114.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 590 kB/s | 1.2 MB 00:02 2024-04-13T03:07:14.463 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-13T03:07:14.500 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-13T03:07:14.547 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-13T03:07:14.583 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-13T03:07:14.626 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-13T03:07:14.677 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-13T03:07:14.811 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-13T03:07:14.857 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-13T03:07:14.887 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-13T03:07:14.928 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-13T03:07:15.023 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-13T03:07:15.062 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-13T03:07:15.102 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-13T03:07:15.128 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-13T03:07:15.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-13T03:07:15.234 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-13T03:07:15.352 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-13T03:07:15.385 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-13T03:07:15.412 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-13T03:07:15.447 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-13T03:07:15.486 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-13T03:07:15.517 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-13T03:07:15.547 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-13T03:07:15.582 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-13T03:07:15.615 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-13T03:07:15.656 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-13T03:07:15.691 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-13T03:07:15.778 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-13T03:07:15.821 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-13T03:07:15.853 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-13T03:07:15.899 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-13T03:07:15.970 INFO:teuthology.orchestra.run.smithi114.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 681 kB/s | 2.2 MB 00:03 2024-04-13T03:07:15.997 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-13T03:07:16.045 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-13T03:07:16.776 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-13T03:07:16.826 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-13T03:07:16.859 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-13T03:07:16.958 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-13T03:07:17.009 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-13T03:07:17.096 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-13T03:07:17.128 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-13T03:07:17.162 INFO:teuthology.orchestra.run.smithi114.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 390 kB/s | 465 kB 00:01 2024-04-13T03:07:17.304 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-13T03:07:17.334 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-13T03:07:17.412 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-13T03:07:17.754 INFO:teuthology.orchestra.run.smithi114.stdout:(169/174): util-linux-user-2.37.4-18.el9.x86_64 53 kB/s | 31 kB 00:00 2024-04-13T03:07:18.005 INFO:teuthology.orchestra.run.smithi114.stdout:(170/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 774 kB/s | 16 MB 00:21 2024-04-13T03:07:18.055 INFO:teuthology.orchestra.run.smithi114.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 646 kB/s | 2.3 MB 00:03 2024-04-13T03:07:18.061 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-13T03:07:18.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-13T03:07:18.780 INFO:teuthology.orchestra.run.smithi114.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 102 kB/s | 74 kB 00:00 2024-04-13T03:07:18.872 INFO:teuthology.orchestra.run.smithi114.stdout:(173/174): xfsdump-3.1.12-4.el9.x86_64.rpm 289 kB/s | 323 kB 00:01 2024-04-13T03:07:19.415 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-13T03:07:19.494 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-13T03:07:19.521 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-13T03:07:19.834 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-13T03:07:19.859 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T03:07:19.950 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T03:07:19.980 INFO:teuthology.orchestra.run.smithi114.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 557 kB/s | 1.1 MB 00:01 2024-04-13T03:07:19.980 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 113/198 2024-04-13T03:07:19.985 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-13T03:07:19.985 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.6 MB/s | 229 MB 01:04 2024-04-13T03:07:20.017 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-13T03:07:20.484 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T03:07:20.513 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T03:07:20.584 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T03:07:20.606 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 117/198 2024-04-13T03:07:21.268 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-13T03:07:21.551 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-13T03:07:21.551 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-13T03:07:22.835 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:22.877 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:23.417 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:23.455 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T03:07:23.545 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T03:07:23.545 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-13T03:07:23.545 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-13T03:07:23.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:23.581 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T03:07:23.674 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-13T03:07:23.674 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-13T03:07:30.719 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-13T03:07:30.809 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-13T03:07:30.836 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-13T03:07:30.857 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T03:07:30.867 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T03:07:30.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-13T03:07:31.003 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-13T03:07:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-13T03:07:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-13T03:07:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-13T03:07:31.459 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-13T03:07:31.511 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:07:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:07:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:07:31.745 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:07:31.775 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T03:07:31.812 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-13T03:07:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-13T03:07:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-13T03:07:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:07:32.196 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:07:32.218 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T03:07:32.266 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T03:07:32.314 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-13T03:07:32.336 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T03:07:32.351 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T03:07:32.510 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T03:07:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T03:07:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-13T03:07:32.760 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-13T03:07:32.886 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T03:07:32.930 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T03:07:33.031 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:33.073 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:33.292 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T03:07:33.401 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-13T03:07:33.496 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-13T03:07:33.528 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-13T03:07:33.562 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-13T03:07:33.626 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-13T03:07:33.662 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-13T03:07:33.705 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-13T03:07:33.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-13T03:07:33.765 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-13T03:07:33.833 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-13T03:07:33.887 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-13T03:07:33.918 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-13T03:07:33.953 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-13T03:07:34.029 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-13T03:07:34.064 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 39/198 2024-04-13T03:07:34.107 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 40/198 2024-04-13T03:07:34.136 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 41/198 2024-04-13T03:07:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-13T03:07:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-13T03:07:34.324 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-13T03:07:34.459 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-13T03:07:34.511 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-13T03:07:35.036 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-13T03:07:35.115 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-13T03:07:35.193 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-13T03:07:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T03:07:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T03:07:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-13T03:07:35.374 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-13T03:07:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 53/198 2024-04-13T03:07:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-13T03:07:35.641 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T03:07:35.641 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-13T03:07:35.642 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:35.764 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-13T03:07:35.801 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T03:07:35.816 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T03:07:35.816 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:35.816 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-13T03:07:35.816 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:35.904 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-13T03:07:35.977 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-13T03:07:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-13T03:07:36.158 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-13T03:07:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-13T03:07:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-13T03:07:36.342 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-13T03:07:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-13T03:07:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-13T03:07:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-13T03:07:36.494 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-13T03:07:36.537 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-13T03:07:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-13T03:07:36.711 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-13T03:07:36.760 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-13T03:07:36.791 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-13T03:07:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-13T03:07:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-13T03:07:36.965 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-13T03:07:37.005 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-13T03:07:37.031 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-13T03:07:37.056 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-13T03:07:37.144 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-13T03:07:37.268 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T03:07:37.272 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T03:07:37.294 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:37.313 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-13T03:07:37.340 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-13T03:07:37.375 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-13T03:07:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-13T03:07:37.448 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-13T03:07:37.478 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-13T03:07:37.512 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-13T03:07:37.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T03:07:37.541 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T03:07:37.541 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:37.541 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-13T03:07:37.542 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T03:07:37.542 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T03:07:37.542 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:37.545 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-13T03:07:37.587 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-13T03:07:37.621 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-13T03:07:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-13T03:07:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-13T03:07:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-13T03:07:37.829 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-13T03:07:37.927 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-13T03:07:37.975 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-13T03:07:37.986 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T03:07:38.008 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T03:07:38.008 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:38.009 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-13T03:07:38.009 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T03:07:38.009 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T03:07:38.009 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:38.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-13T03:07:38.165 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-13T03:07:38.237 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-13T03:07:38.422 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-13T03:07:38.470 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-13T03:07:38.540 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-13T03:07:38.637 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-13T03:07:38.705 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-13T03:07:38.756 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-13T03:07:38.777 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-13T03:07:38.790 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-13T03:07:38.819 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-13T03:07:38.853 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-13T03:07:38.884 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-13T03:07:38.887 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-13T03:07:38.921 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-13T03:07:38.938 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-13T03:07:38.956 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-13T03:07:39.017 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-13T03:07:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-13T03:07:39.252 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-13T03:07:39.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-13T03:07:39.349 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T03:07:39.366 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-13T03:07:39.404 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T03:07:39.436 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 139/198 2024-04-13T03:07:39.497 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 140/198 2024-04-13T03:07:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T03:07:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-13T03:07:40.127 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-13T03:07:40.595 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T03:07:40.629 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T03:07:41.354 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-13T03:07:41.435 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-13T03:07:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-13T03:07:41.598 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T03:07:41.630 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T03:07:41.730 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T03:07:41.777 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-13T03:07:41.800 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T03:07:41.890 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T03:07:41.893 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 144/198 2024-04-13T03:07:41.915 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T03:07:41.921 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 113/198 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T03:07:41.937 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:41.958 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-13T03:07:41.982 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T03:07:41.993 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T03:07:42.083 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 147/198 2024-04-13T03:07:42.425 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T03:07:42.461 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T03:07:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T03:07:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 117/198 2024-04-13T03:07:44.824 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:44.867 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:45.409 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T03:07:45.445 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T03:07:45.542 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T03:07:45.542 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-13T03:07:45.542 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-13T03:07:45.542 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T03:07:46.643 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 148/198 2024-04-13T03:07:46.713 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 149/198 2024-04-13T03:07:47.588 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-13T03:07:47.622 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:07:47.644 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:07:47.645 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:47.645 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-13T03:07:47.645 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T03:07:47.645 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T03:07:47.645 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:47.690 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T03:07:47.711 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T03:07:47.711 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:47.711 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-13T03:07:47.712 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:47.757 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T03:07:47.779 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:48.082 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T03:07:48.105 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:07:48.157 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 155/198 2024-04-13T03:07:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 156/198 2024-04-13T03:07:48.214 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-13T03:07:48.401 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 158/198 2024-04-13T03:07:48.454 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-13T03:07:48.507 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-13T03:07:48.562 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-13T03:07:48.597 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-13T03:07:48.629 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 163/198 2024-04-13T03:07:48.661 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-13T03:07:48.691 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-13T03:07:48.752 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-13T03:07:48.819 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-13T03:07:48.857 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-13T03:07:49.411 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:07:49.456 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:07:49.476 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:07:50.300 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 170/198 2024-04-13T03:07:50.401 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-13T03:07:50.489 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-13T03:07:50.520 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-13T03:07:50.545 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-13T03:07:50.571 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-13T03:07:50.596 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-13T03:07:50.631 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-13T03:07:50.631 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T03:07:50.719 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T03:07:50.748 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-13T03:07:50.749 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T03:07:50.816 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T03:07:50.842 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-13T03:07:50.884 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-13T03:07:50.910 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-13T03:07:50.934 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-13T03:07:50.968 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-13T03:07:51.016 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-13T03:07:51.055 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-13T03:07:51.081 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-13T03:07:51.106 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-13T03:07:51.131 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-13T03:07:51.165 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-13T03:07:51.171 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:07:51.171 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:07:51.351 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:07:51.351 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T03:07:51.418 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T03:07:51.442 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-13T03:07:51.475 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-13T03:07:51.502 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-13T03:07:51.527 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-13T03:07:51.593 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T03:07:57.226 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T03:07:57.226 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 1/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 3/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 4/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 5/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 6/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 7/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 8/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 9/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 13/198 2024-04-13T03:07:57.227 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 14/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 15/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 17/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 18/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 19/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 20/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 21/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 22/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 23/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 24/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 25/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 26/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 27/198 2024-04-13T03:07:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 28/198 2024-04-13T03:07:57.243 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 29/198 2024-04-13T03:07:57.243 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 30/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 31/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 32/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 33/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 34/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 35/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 36/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 37/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 38/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 39/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-13T03:07:57.244 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-13T03:07:57.245 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-13T03:07:57.246 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-13T03:07:57.247 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-13T03:07:57.248 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-13T03:07:57.249 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-13T03:07:57.250 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-13T03:07:57.251 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-13T03:07:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-13T03:07:57.253 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-13T03:07:57.254 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-13T03:07:57.255 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-13T03:07:57.255 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-13T03:07:57.255 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-13T03:07:57.255 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-13T03:07:57.255 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-13T03:07:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-04-13T03:07:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:57.482 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T03:07:57.497 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T03:07:57.497 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:57.497 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-13T03:07:57.498 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:59.003 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T03:07:59.029 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T03:07:59.029 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:59.030 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-13T03:07:59.030 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T03:07:59.030 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T03:07:59.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:59.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T03:07:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T03:07:59.285 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:59.285 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-13T03:07:59.285 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T03:07:59.285 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T03:07:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T03:07:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T03:07:59.743 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:07:59.743 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-13T03:07:59.744 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T03:07:59.744 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T03:07:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:07:59.868 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-13T03:07:59.916 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-13T03:07:59.979 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-13T03:08:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-13T03:08:00.222 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-13T03:08:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:00.316 INFO:teuthology.orchestra.run.smithi106.stdout: acl-2.3.1-4.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:00.317 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.318 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.319 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-13T03:08:00.320 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: jq-1.6-16.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-13T03:08:00.321 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-13T03:08:00.322 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-13T03:08:00.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-13T03:08:00.324 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-13T03:08:00.325 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-13T03:08:00.326 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-13T03:08:00.327 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-13T03:08:00.328 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-13T03:08:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-13T03:08:00.509 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-13T03:08:00.546 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-13T03:08:00.579 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-13T03:08:00.611 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-13T03:08:00.647 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-13T03:08:00.674 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-13T03:08:00.837 DEBUG:teuthology.parallel:result is None 2024-04-13T03:08:01.017 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T03:08:01.048 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T03:08:01.090 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 139/198 2024-04-13T03:08:01.149 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 140/198 2024-04-13T03:08:01.188 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T03:08:02.228 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T03:08:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T03:08:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T03:08:03.265 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T03:08:03.365 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T03:08:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 144/198 2024-04-13T03:08:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T03:08:03.573 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:03.618 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T03:08:03.628 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T03:08:03.739 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 147/198 2024-04-13T03:08:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 148/198 2024-04-13T03:08:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 149/198 2024-04-13T03:08:09.239 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-13T03:08:09.267 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:08:09.289 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:08:09.289 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:08:09.289 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-13T03:08:09.289 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T03:08:09.290 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T03:08:09.290 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:09.334 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T03:08:09.356 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T03:08:09.356 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:08:09.356 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-13T03:08:09.356 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:09.401 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:08:09.423 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:08:09.423 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:08:09.423 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-13T03:08:09.424 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T03:08:09.424 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T03:08:09.424 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:09.727 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T03:08:09.749 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:09.801 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 155/198 2024-04-13T03:08:09.832 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 156/198 2024-04-13T03:08:09.859 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-13T03:08:10.050 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 158/198 2024-04-13T03:08:10.106 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-13T03:08:10.152 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-13T03:08:10.198 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-13T03:08:10.241 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-13T03:08:10.274 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 163/198 2024-04-13T03:08:10.314 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-13T03:08:10.344 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-13T03:08:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-13T03:08:10.478 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-13T03:08:10.510 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-13T03:08:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:08:11.107 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:08:11.126 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T03:08:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 170/198 2024-04-13T03:08:12.044 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-13T03:08:12.124 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-13T03:08:12.155 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-13T03:08:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-13T03:08:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-13T03:08:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-13T03:08:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-13T03:08:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T03:08:12.329 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T03:08:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-13T03:08:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T03:08:12.452 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T03:08:12.478 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-13T03:08:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-13T03:08:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-13T03:08:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-13T03:08:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-13T03:08:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-13T03:08:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-13T03:08:12.692 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-13T03:08:12.717 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-13T03:08:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-13T03:08:12.770 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-13T03:08:12.774 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:08:12.774 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:08:12.954 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T03:08:12.954 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T03:08:13.037 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T03:08:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-13T03:08:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-13T03:08:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-13T03:08:13.154 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-13T03:08:13.219 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T03:08:18.811 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 1/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 3/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 4/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 5/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 6/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 7/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 8/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 9/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T03:08:18.812 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 13/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 14/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 15/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 17/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 18/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 19/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 20/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 21/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 22/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 23/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 24/198 2024-04-13T03:08:18.813 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 25/198 2024-04-13T03:08:18.830 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 26/198 2024-04-13T03:08:18.830 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 27/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 28/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 29/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 30/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 31/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 32/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 33/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 34/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 35/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 36/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 37/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 38/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 39/198 2024-04-13T03:08:18.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-13T03:08:18.832 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-13T03:08:18.833 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-13T03:08:18.834 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-13T03:08:18.835 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-13T03:08:18.836 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-13T03:08:18.837 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-13T03:08:18.838 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-13T03:08:18.839 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-13T03:08:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-13T03:08:18.841 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-13T03:08:18.842 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-13T03:08:18.843 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: acl-2.3.1-4.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-13T03:08:21.647 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.649 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.650 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-13T03:08:21.651 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: jq-1.6-16.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-13T03:08:21.652 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-04-13T03:08:21.653 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.14-1.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-13T03:08:21.654 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-13T03:08:21.655 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-13T03:08:21.656 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-13T03:08:21.657 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-13T03:08:21.658 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.659 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-13T03:08:21.660 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:21.661 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-13T03:08:22.146 DEBUG:teuthology.parallel:result is None 2024-04-13T03:08:22.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T03:08:22.302 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-13T03:08:22.328 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-2884.g218a6914.el9 2024-04-13T03:08:22.329 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2884.g218a6914.el9 2024-04-13T03:08:22.329 INFO:teuthology.task.install:The correct ceph version 19.0.0-2884.g218a6914 is installed. 2024-04-13T03:08:22.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T03:08:22.483 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-13T03:08:22.507 INFO:teuthology.orchestra.run.smithi114.stdout:19.0.0-2884.g218a6914.el9 2024-04-13T03:08:22.508 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2884.g218a6914.el9 2024-04-13T03:08:22.508 INFO:teuthology.task.install:The correct ceph version 19.0.0-2884.g218a6914 is installed. 2024-04-13T03:08:22.509 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-13T03:08:22.510 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:22.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-13T03:08:22.554 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:22.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-13T03:08:22.589 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-13T03:08:22.589 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:22.589 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-13T03:08:22.630 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-13T03:08:22.706 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:22.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-04-13T03:08:22.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-13T03:08:22.805 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-13T03:08:22.805 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:22.805 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-13T03:08:22.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-13T03:08:22.909 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:22.910 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-13T03:08:22.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-13T03:08:23.005 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-13T03:08:23.006 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:23.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-13T03:08:23.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-13T03:08:23.107 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:23.107 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-04-13T03:08:23.134 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-13T03:08:23.201 INFO:teuthology.run_tasks:Running task ceph... 2024-04-13T03:08:23.294 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-13T03:08:23.294 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2024-04-13T03:08:23.296 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-04-13T03:08:23.324 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-13T03:08:23.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-13T03:08:23.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-13T03:08:23.394 INFO:tasks.ceph:Creating extra log directories... 2024-04-13T03:08:23.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-13T03:08:23.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-13T03:08:23.470 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-13T03:08:23.470 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-13T03:08:23.470 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-13T03:08:23.470 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:23.471 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-13T03:08:23.518 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:23.637 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:23.664 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:23.664 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-13T03:08:23.679 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:23.793 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:23.825 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-13T03:08:23.825 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-13T03:08:23.825 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237', 'branch': 'wip-vshankar-testing-20240412.042930', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7654237', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240412.042930'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240412.042930', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7517, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'sleep_before_teardown': 0, 'subset': '55/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240412.042930', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi106.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyYDU5m3TDDQA01tQdtRqc5190+eS7O5PDuIy4C3rpyY48TgQcZeo44AnjuDICd/T+RwJqL3qnqghlndxPkuK8=', 'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrnsrNDJBSUskISoAmoVnGd/SkC+aeSpHNjd8AYVLKsGuh8E4E6f1RCpN0JYFOVOBR1GbmNXdlypbubhUzF36Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-12_17:34:28', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-13T03:08:23.826 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-13T03:08:23.829 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-13T03:08:23.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2024-04-13T03:08:23.883 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-04-13T03:08:23.942 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:23.942 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-13T03:08:23.963 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-13T03:08:23.963 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-13 03:07:51.912067876 +0000 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-13 03:05:05.847059093 +0000 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-13 03:05:05.847059093 +0000 2024-04-13T03:08:24.018 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-13 03:05:05.847059093 +0000 2024-04-13T03:08:24.019 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-13T03:08:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-13T03:08:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-13T03:08:24.083 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000259811 s, 2.0 MB/s 2024-04-13T03:08:24.084 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-13T03:08:24.140 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-13T03:08:24.195 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-13T03:08:24.195 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.195 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-13 03:07:51.912067876 +0000 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-13 03:05:06.211050256 +0000 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-13 03:05:06.211050256 +0000 2024-04-13T03:08:24.196 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-13 03:05:06.211050256 +0000 2024-04-13T03:08:24.196 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-13T03:08:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-13T03:08:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-13T03:08:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000278285 s, 1.8 MB/s 2024-04-13T03:08:24.262 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-13T03:08:24.318 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-13 03:07:51.913067852 +0000 2024-04-13T03:08:24.372 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-13 03:05:06.551042002 +0000 2024-04-13T03:08:24.373 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-13 03:05:06.551042002 +0000 2024-04-13T03:08:24.373 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-13 03:05:06.551042002 +0000 2024-04-13T03:08:24.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-13T03:08:24.438 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-13T03:08:24.438 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-13T03:08:24.438 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000227885 s, 2.2 MB/s 2024-04-13T03:08:24.439 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-13T03:08:24.495 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-13 03:07:51.913067852 +0000 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-13 03:05:06.886033868 +0000 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-13 03:05:06.886033868 +0000 2024-04-13T03:08:24.550 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-13 03:05:06.886033868 +0000 2024-04-13T03:08:24.553 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-13T03:08:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-13T03:08:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-13T03:08:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000224548 s, 2.3 MB/s 2024-04-13T03:08:24.619 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-13T03:08:24.675 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:24.675 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:24.675 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-04-13T03:08:24.693 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-13T03:08:24.693 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-13 03:08:13.593011191 +0000 2024-04-13T03:08:24.750 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-13 03:05:29.987253694 +0000 2024-04-13T03:08:24.751 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-13 03:05:29.987253694 +0000 2024-04-13T03:08:24.751 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-13 03:05:29.987253694 +0000 2024-04-13T03:08:24.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-13T03:08:24.818 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-13T03:08:24.818 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-13T03:08:24.818 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000263797 s, 1.9 MB/s 2024-04-13T03:08:24.819 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-13T03:08:24.877 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-04-13T03:08:24.932 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-13T03:08:24.932 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-13 03:08:13.593011191 +0000 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-13 03:05:30.326246535 +0000 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-13 03:05:30.326246535 +0000 2024-04-13T03:08:24.933 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-13 03:05:30.326246535 +0000 2024-04-13T03:08:24.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-13T03:08:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-13T03:08:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-13T03:08:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000184643 s, 2.8 MB/s 2024-04-13T03:08:24.999 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-13T03:08:25.055 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-04-13T03:08:25.110 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-13 03:08:13.594011170 +0000 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-13 03:05:30.660239481 +0000 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-13 03:05:30.660239481 +0000 2024-04-13T03:08:25.111 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-13 03:05:30.660239481 +0000 2024-04-13T03:08:25.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-13T03:08:25.177 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-13T03:08:25.177 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-13T03:08:25.177 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000210174 s, 2.4 MB/s 2024-04-13T03:08:25.178 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-13T03:08:25.234 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T03:08:25.289 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-13 03:08:13.594011170 +0000 2024-04-13T03:08:25.290 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-13 03:05:30.993232448 +0000 2024-04-13T03:08:25.290 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-13 03:05:30.993232448 +0000 2024-04-13T03:08:25.290 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-13 03:05:30.993232448 +0000 2024-04-13T03:08:25.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-13T03:08:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-13T03:08:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-13T03:08:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000188282 s, 2.7 MB/s 2024-04-13T03:08:25.356 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-13T03:08:25.411 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:25.411 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-13T03:08:25.411 INFO:tasks.ceph:Generating config... 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] debug client = 20 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] debug ms = 1 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] pid file = 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-13T03:08:25.413 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-13T03:08:25.413 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-13T03:08:25.413 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-13T03:08:25.414 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-13T03:08:25.414 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-13T03:08:25.414 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-13T03:08:25.414 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-13T03:08:25.414 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-13T03:08:25.415 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-13T03:08:25.415 INFO:tasks.ceph:Setting up mon.a... 2024-04-13T03:08:25.415 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-13T03:08:25.465 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.keyring 2024-04-13T03:08:25.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-13T03:08:25.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-13T03:08:25.578 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.106'), ('mon.b', '172.21.15.114'), ('mon.c', '[v2:172.21.15.114:3301,v1:172.21.15.114:6790]')] 2024-04-13T03:08:25.579 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-13T03:08:25.579 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:25.580 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-13T03:08:25.633 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --add b 172.21.15.114 --addv c '[v2:172.21.15.114:3301,v1:172.21.15.114:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:25.718 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:25.718 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:08:25.718 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = quincy 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:fsid d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2024-04-13T03:08:25.718480+0000 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-13T03:08:25.718480+0000 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 17 (quincy) 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:2: [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] mon.c 2024-04-13T03:08:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-13T03:08:25.720 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-13T03:08:25.735 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d7ff0368-2781-4238-b477-50485b08a223... 2024-04-13T03:08:25.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:08:25.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:08:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-13T03:08:25.825 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:08:25.826 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.827 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.828 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-13T03:08:25.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-13T03:08:25.830 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:08:25.831 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-13T03:08:25.832 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:08:25.833 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-13T03:08:25.834 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-13T03:08:25.834 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:08:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-13T03:08:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:08:25.836 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.837 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.838 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.839 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:08:25.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.841 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-04-13T03:08:25.842 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:08:25.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:08:25.844 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-13T03:08:25.845 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:08:25.846 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = True 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = True 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:08:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:08:25.849 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-13T03:08:25.849 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-13T03:08:25.945 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-13T03:08:25.945 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:25.945 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-13T03:08:25.959 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:25.960 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-13T03:08:26.014 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:08:26.014 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:26.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-13T03:08:26.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-13T03:08:26.093 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:26.093 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:26.148 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:08:26.148 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:26.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-13T03:08:26.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-13T03:08:26.189 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:26.189 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:26.244 INFO:tasks.ceph:Setting up mon nodes... 2024-04-13T03:08:26.244 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-13T03:08:26.245 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-13T03:08:26.299 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-13T03:08:26.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-13T03:08:26.386 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-13T03:08:26.388 INFO:tasks.ceph:Setting up mds nodes... 2024-04-13T03:08:26.388 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-13T03:08:26.443 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-13T03:08:26.446 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-13T03:08:26.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-13T03:08:26.570 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-13T03:08:26.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-13T03:08:26.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-13T03:08:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-13T03:08:26.657 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-13T03:08:26.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-13T03:08:26.778 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-13T03:08:26.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-13T03:08:26.808 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-13T03:08:26.808 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-13T03:08:26.850 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-13T03:08:26.865 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-13T03:08:26.948 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-13T03:08:26.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-13T03:08:27.010 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-13T03:08:27.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-13T03:08:27.108 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-13T03:08:27.124 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-13T03:08:27.124 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-13T03:08:27.124 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-13T03:08:27.151 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:27.151 INFO:tasks.ceph:role: osd.0 2024-04-13T03:08:27.151 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:08:27.151 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-13T03:08:27.224 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:27.224 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:27.224 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:27.224 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:27.224 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:27.225 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:27.225 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:27.225 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:27.225 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:27.225 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:27.737 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-13T03:08:27.739 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-13T03:08:27.739 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-13T03:08:27.870 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-13T03:08:27.902 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-13T03:08:27.969 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:27.969 INFO:tasks.ceph:role: osd.1 2024-04-13T03:08:27.970 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:08:27.970 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:28.046 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:28.501 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-13T03:08:28.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-13T03:08:28.504 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-13T03:08:28.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-13T03:08:28.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-13T03:08:28.701 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:28.702 INFO:tasks.ceph:role: osd.2 2024-04-13T03:08:28.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:08:28.702 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:28.776 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:29.259 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-13T03:08:29.261 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-13T03:08:29.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-13T03:08:29.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-13T03:08:29.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-13T03:08:29.488 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:29.489 INFO:tasks.ceph:role: osd.3 2024-04-13T03:08:29.489 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:08:29.489 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-13T03:08:29.563 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:29.563 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:29.563 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:29.563 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:29.563 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:29.564 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:29.564 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:29.564 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:29.564 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:29.564 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:30.079 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-13T03:08:30.081 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-13T03:08:30.081 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-13T03:08:30.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-13T03:08:30.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.375+0000 7f7b96e15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-13T03:08:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.376+0000 7f7b96e15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-13T03:08:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.376+0000 7f7b96e15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-13T03:08:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.376+0000 7f7b96e15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-13T03:08:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.376+0000 7f7b96e15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-13T03:08:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:30.376+0000 7f7b96e15640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-13T03:08:33.223 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-13T03:08:33.253 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:33.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.342+0000 7fb0150f7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-13T03:08:33.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.342+0000 7fb0150f7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-13T03:08:33.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.342+0000 7fb0150f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-13T03:08:33.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.342+0000 7fb0150f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-13T03:08:33.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.342+0000 7fb0150f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-13T03:08:33.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:33.343+0000 7fb0150f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-13T03:08:36.186 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-13T03:08:36.215 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:36.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.305+0000 7fe373f0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-13T03:08:36.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.305+0000 7fe373f0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-13T03:08:36.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.305+0000 7fe373f0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-13T03:08:36.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.305+0000 7fe373f0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-13T03:08:36.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.305+0000 7fe373f0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-13T03:08:36.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:36.306+0000 7fe373f0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-13T03:08:39.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-13T03:08:39.187 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.274+0000 7f118f781640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-13T03:08:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.274+0000 7f118f781640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-13T03:08:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.274+0000 7f118f781640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-13T03:08:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.274+0000 7f118f781640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-13T03:08:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.274+0000 7f118f781640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-13T03:08:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:08:39.275+0000 7f118f781640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-13T03:08:42.131 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-13T03:08:42.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-13T03:08:42.194 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:42.194 INFO:tasks.ceph:role: osd.4 2024-04-13T03:08:42.194 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:08:42.195 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-13T03:08:42.272 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:42.273 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:42.779 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-04-13T03:08:42.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-04-13T03:08:42.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-13T03:08:42.891 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-13T03:08:42.928 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-13T03:08:42.997 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:42.997 INFO:tasks.ceph:role: osd.5 2024-04-13T03:08:42.997 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:08:42.998 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:43.071 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:43.072 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:43.072 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:43.072 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:43.538 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-04-13T03:08:43.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-04-13T03:08:43.540 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-13T03:08:43.634 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-13T03:08:43.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-13T03:08:43.732 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:43.732 INFO:tasks.ceph:role: osd.6 2024-04-13T03:08:43.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:08:43.732 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-13T03:08:43.806 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:43.807 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:44.289 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-04-13T03:08:44.291 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-04-13T03:08:44.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-13T03:08:44.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-13T03:08:44.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-13T03:08:44.522 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-13T03:08:44.522 INFO:tasks.ceph:role: osd.7 2024-04-13T03:08:44.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:08:44.522 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-13T03:08:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-04-13T03:08:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-13T03:08:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-13T03:08:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-13T03:08:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-13T03:08:45.108 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-04-13T03:08:45.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-04-13T03:08:45.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-13T03:08:45.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-13T03:08:45.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:45.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-13T03:08:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-13T03:08:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-13T03:08:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-13T03:08:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-13T03:08:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:45.376+0000 7f4eb501f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-13T03:08:48.230 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-13T03:08:48.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.353+0000 7f85f480e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-13T03:08:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.353+0000 7f85f480e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-13T03:08:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.353+0000 7f85f480e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-13T03:08:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.353+0000 7f85f480e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-13T03:08:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.353+0000 7f85f480e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-13T03:08:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:48.354+0000 7f85f480e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-13T03:08:51.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-13T03:08:51.232 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.324+0000 7fe09e79a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.324+0000 7fe09e79a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.324+0000 7fe09e79a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.324+0000 7fe09e79a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.324+0000 7fe09e79a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-13T03:08:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:51.325+0000 7fe09e79a640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-13T03:08:53.827 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:53.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:08:54.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-13T03:08:54.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-13T03:08:54.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:08:54.299+0000 7f1830010640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-13T03:08:57.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-13T03:08:57.184 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-13T03:08:57.184 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.184 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-13T03:08:57.217 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-13T03:08:57.286 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.286 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-13T03:08:57.355 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.355 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-13T03:08:57.424 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.424 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-13T03:08:57.493 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.493 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-13T03:08:57.561 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:57.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-13T03:08:57.628 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.628 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-13T03:08:57.657 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.657 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-13T03:08:57.725 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-13T03:08:57.793 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.794 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-13T03:08:57.860 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-13T03:08:57.927 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-13T03:08:57.993 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:57.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-13T03:08:58.059 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:58.059 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-13T03:08:58.075 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:08:58.075 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-13T03:08:58.132 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:58.132 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-13T03:08:58.146 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:08:58.146 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-13T03:08:58.201 INFO:tasks.ceph:Adding keys to all mons... 2024-04-13T03:08:58.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-13T03:08:58.203 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-13T03:08:58.244 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2024-04-13T03:08:58.244 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA== 2024-04-13T03:08:58.244 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ== 2024-04-13T03:08:58.245 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw== 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2024-04-13T03:08:58.246 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw== 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout:[client.2] 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ== 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA== 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout:[client.3] 2024-04-13T03:08:58.247 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ== 2024-04-13T03:08:58.266 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.y] 2024-04-13T03:08:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ== 2024-04-13T03:08:58.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw== 2024-04-13T03:08:58.268 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout:[client.2] 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ== 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout:[client.1] 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA== 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout:[client.3] 2024-04-13T03:08:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ== 2024-04-13T03:08:58.270 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-13T03:08:58.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-13T03:08:58.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.479 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.533 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.639 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-13T03:08:58.696 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-13T03:08:58.748 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.805 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-13T03:08:58.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.871 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.928 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:58.985 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:59.037 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:59.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-13T03:08:59.091 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.147 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.204 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.206 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.263 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-13T03:08:59.318 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-13T03:08:59.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-13T03:08:59.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-13T03:08:59.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-13T03:08:59.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-13T03:08:59.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-13T03:08:59.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-13T03:08:59.621 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-13T03:08:59.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-13T03:08:59.785 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-13T03:08:59.811 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:59.813 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-13T03:08:59.867 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-13T03:08:59.867 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-13T03:08:59.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-13T03:08:59.869 INFO:tasks.ceph.mon.a:Started 2024-04-13T03:08:59.869 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-13T03:08:59.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-13T03:08:59.909 INFO:tasks.ceph.mon.b:Started 2024-04-13T03:08:59.909 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-13T03:08:59.909 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-13T03:08:59.911 INFO:tasks.ceph.mon.c:Started 2024-04-13T03:08:59.911 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-13T03:08:59.912 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-13T03:08:59.912 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-13T03:08:59.914 INFO:tasks.ceph.mgr.y:Started 2024-04-13T03:08:59.914 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-13T03:08:59.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-13T03:08:59.916 INFO:tasks.ceph.mgr.x:Started 2024-04-13T03:08:59.916 INFO:tasks.ceph:Setting crush tunables to default 2024-04-13T03:08:59.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph osd crush tunables default 2024-04-13T03:09:00.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.017+0000 7fc8bffbb640 1 Processor -- start 2024-04-13T03:09:00.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.017+0000 7fc8bffbb640 1 -- start start 2024-04-13T03:09:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b80d0c90 con 0x7fc8b80d0820 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b806e280 con 0x7fc8b815e930 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bffbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b806e560 con 0x7fc8b806ed70 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36290/0 (socket says 172.21.15.106:36290) 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 learned_addr learned my addr 172.21.15.106:0/2296251546 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 msgr2=0x7fc8b80cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 msgr2=0x7fc8b80cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bd556640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bdd57640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bd556640 1 -- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 msgr2=0x7fc8b80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bd556640 1 -- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 msgr2=0x7fc8b80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bd556640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bd556640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bdd57640 1 -- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 msgr2=0x7fc8b80b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bdd57640 1 -- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 msgr2=0x7fc8b80b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bdd57640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-13T03:09:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.018+0000 7fc8bdd57640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T03:09:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.106:3300/0 conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8bdd57640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8bd556640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 msgr2=0x7fc8b80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 msgr2=0x7fc8b80b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.219+0000 7fc8be558640 1 -- 172.21.15.106:0/2296251546 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b80cccf0 con 0x7fc8b80d0820 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8be558640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cd7d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80b50f0 tx=0x7fc8b002bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eda83d71cac78f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8bcd55640 1 -- 172.21.15.106:0/2296251546 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0039070 con 0x7fc8b80d0820 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8bcd55640 1 -- 172.21.15.106:0/2296251546 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b002ca20 con 0x7fc8b80d0820 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8bcd55640 1 -- 172.21.15.106:0/2296251546 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000f040 con 0x7fc8b80d0820 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8bffbb640 1 -- 172.21.15.106:0/2296251546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 msgr2=0x7fc8b80cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.220+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cd7d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80b50f0 tx=0x7fc8b002bc70 comp rx=0 tx=0).stop 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 -- 172.21.15.106:0/2296251546 shutdown_connections 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cd7d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/2296251546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b806ed70 0x7fc8b80d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/2296251546 >> v2:172.21.15.114:3300/0 conn(0x7fc8b815e930 0x7fc8b80b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 -- 172.21.15.106:0/2296251546 >> 172.21.15.106:0/2296251546 conn(0x7fc8b80cef80 msgr2=0x7fc8b80cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 -- 172.21.15.106:0/2296251546 shutdown_connections 2024-04-13T03:09:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.221+0000 7fc8bffbb640 1 -- 172.21.15.106:0/2296251546 wait complete. 2024-04-13T03:09:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.222+0000 7fc8bffbb640 1 Processor -- start 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.222+0000 7fc8bffbb640 1 -- start start 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8b806ed70 0x7fc8b80cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b815e930 0x7fc8b81f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b806f1d0 con 0x7fc8b80d0820 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80cd2b0 con 0x7fc8b806ed70 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b80ce830 con 0x7fc8b815e930 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36312/0 (socket says 172.21.15.106:36312) 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 -- 172.21.15.106:0/3699129123 learned_addr learned my addr 172.21.15.106:0/3699129123 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bdd57640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8b806ed70 0x7fc8b80cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8be558640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b815e930 0x7fc8b81f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 -- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b815e930 msgr2=0x7fc8b81f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b815e930 0x7fc8b81f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 -- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8b806ed70 msgr2=0x7fc8b80cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8b806ed70 0x7fc8b80cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 -- 172.21.15.106:0/3699129123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b815c2a0 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bd556640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800afd0 tx=0x7fc8a8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8014610 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a8019070 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b8160ec0 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80149d0 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.223+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b815be10 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.224+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8a801d410 con 0x7fc8b80d0820 2024-04-13T03:09:00.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.224+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8a80086f0 con 0x7fc8b80d0820 2024-04-13T03:09:00.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.224+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b8068a10 con 0x7fc8b80d0820 2024-04-13T03:09:00.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.227+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc8a8008940 con 0x7fc8b80d0820 2024-04-13T03:09:00.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.293+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc8b80c6280 con 0x7fc8b80d0820 2024-04-13T03:09:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.294+0000 7fc8b67fc640 1 -- 172.21.15.106:0/3699129123 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc8a8008b20 con 0x7fc8b80d0820 2024-04-13T03:09:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2024-04-13T03:09:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 msgr2=0x7fc8b80cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800afd0 tx=0x7fc8a8007700 comp rx=0 tx=0).stop 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 shutdown_connections 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8b815e930 0x7fc8b81f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8b80d0820 0x7fc8b80cbce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 --2- 172.21.15.106:0/3699129123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8b806ed70 0x7fc8b80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 >> 172.21.15.106:0/3699129123 conn(0x7fc8b80cef80 msgr2=0x7fc8b815d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 shutdown_connections 2024-04-13T03:09:00.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:00.295+0000 7fc8bffbb640 1 -- 172.21.15.106:0/3699129123 wait complete. 2024-04-13T03:09:00.307 INFO:tasks.ceph:check_enable_crimson: False 2024-04-13T03:09:00.307 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-13T03:09:00.308 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:00.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-13T03:09:00.334 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:00.334 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-13T03:09:00.401 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:00.402 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-13T03:09:00.468 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:00.468 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-13T03:09:00.535 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:09:00.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-13T03:09:00.562 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:09:00.562 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-13T03:09:00.630 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:09:00.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-13T03:09:00.653 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:00.651+0000 7f8d5dc2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T03:09:00.679 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:09:00.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-13T03:09:00.685 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:00.684+0000 7f0d05a49040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T03:09:00.708 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:00.707+0000 7f8d5dc2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T03:09:00.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 3f0d52df-7c5b-4676-8ea0-28475c30a296 0 2024-04-13T03:09:00.739 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:00.739+0000 7f0d05a49040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T03:09:00.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.874+0000 7f92ac0de640 1 Processor -- start 2024-04-13T03:09:00.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.874+0000 7f92ac0de640 1 -- start start 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92ac0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92ac0de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f92a406f560 0x7f92a415f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92ac0de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a4159530 con 0x7f92a415fc90 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92ac0de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f92a40d0700 con 0x7f92a406f560 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92ac0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f92a406fcb0 con 0x7f92a4159150 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52650/0 (socket says 172.21.15.114:52650) 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3979819818 learned_addr learned my addr 172.21.15.114:0/3979819818 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9679640 1 --2- 172.21.15.114:0/3979819818 >> v2:172.21.15.106:3300/0 conn(0x7f92a406f560 0x7f92a415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3979819818 >> v2:172.21.15.106:3300/0 conn(0x7f92a406f560 msgr2=0x7f92a415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 --2- 172.21.15.114:0/3979819818 >> v2:172.21.15.106:3300/0 conn(0x7f92a406f560 0x7f92a415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3979819818 >> v1:172.21.15.106:6789/0 conn(0x7f92a415fc90 legacy=0x7f92a40ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.875+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3979819818 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a406ff90 con 0x7f92a4159150 2024-04-13T03:09:00.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.876+0000 7f92a9e7a640 1 --2- 172.21.15.114:0/3979819818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9294009870 tx=0x7f929402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91fce68a9d3de60 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:00.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.876+0000 7f92a8e78640 1 -- 172.21.15.114:0/3979819818 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294034070 con 0x7f92a4159150 2024-04-13T03:09:00.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.876+0000 7f92a8e78640 1 -- 172.21.15.114:0/3979819818 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9294030840 con 0x7f92a4159150 2024-04-13T03:09:00.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.876+0000 7f92a8e78640 1 -- 172.21.15.114:0/3979819818 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294030c10 con 0x7f92a4159150 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 -- 172.21.15.114:0/3979819818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 msgr2=0x7f92a406d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3979819818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9294009870 tx=0x7f929402b980 comp rx=0 tx=0).stop 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 -- 172.21.15.114:0/3979819818 shutdown_connections 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3979819818 >> v2:172.21.15.106:3300/0 conn(0x7f92a406f560 0x7f92a415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3979819818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a4159150 0x7f92a406d9b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 -- 172.21.15.114:0/3979819818 >> 172.21.15.114:0/3979819818 conn(0x7f92a40d0c30 msgr2=0x7f92a40d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 -- 172.21.15.114:0/3979819818 shutdown_connections 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.877+0000 7f92ac0de640 1 -- 172.21.15.114:0/3979819818 wait complete. 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.878+0000 7f92ac0de640 1 Processor -- start 2024-04-13T03:09:00.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.878+0000 7f92ac0de640 1 -- start start 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a4159150 0x7f92a41e9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 0x7f92a41e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92a406f9c0 con 0x7f92a4159150 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92a415f230 con 0x7f92a406f560 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92ac0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f92a40cf240 con 0x7f92a415fc90 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42522/0 (socket says 172.21.15.114:42522) 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3272740741 learned_addr learned my addr 172.21.15.114:0/3272740741 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92aa67b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 0x7f92a41e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9679640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a4159150 0x7f92a41e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 msgr2=0x7f92a41e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 0x7f92a41e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3272740741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a4159150 msgr2=0x7f92a41e9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a4159150 0x7f92a41e9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 -- 172.21.15.114:0/3272740741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a4141330 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92aa67b640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 0x7f92a41e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.879+0000 7f92a9e7a640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294048070 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9294030640 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a4142b70 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294034040 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a4143130 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f929404b070 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.880+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f929404dce0 con 0x7f92a406f560 2024-04-13T03:09:00.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.881+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a406ca80 con 0x7f92a406f560 2024-04-13T03:09:00.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.883+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f92a406ca80 con 0x7f92a406f560 2024-04-13T03:09:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.955+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3f0d52df-7c5b-4676-8ea0-28475c30a296", "id": 0} v 0) -- 0x7f92a4069120 con 0x7f92a406f560 2024-04-13T03:09:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.958+0000 7f929a7fc640 1 -- 172.21.15.114:0/3272740741 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f0d52df-7c5b-4676-8ea0-28475c30a296", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f92940307f0 con 0x7f92a406f560 2024-04-13T03:09:00.958 INFO:teuthology.orchestra.run.smithi114.stdout:0 2024-04-13T03:09:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 msgr2=0x7f92a4152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294004730 comp rx=0 tx=0).stop 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 shutdown_connections 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92a415fc90 0x7f92a41e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a4159150 0x7f92a41e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 --2- 172.21.15.114:0/3272740741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a406f560 0x7f92a4152590 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 >> 172.21.15.114:0/3272740741 conn(0x7f92a40d0c30 msgr2=0x7f92a40ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 shutdown_connections 2024-04-13T03:09:00.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:00.959+0000 7f92ac0de640 1 -- 172.21.15.114:0/3272740741 wait complete. 2024-04-13T03:09:00.969 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 611ca47e-a863-4c90-baef-28b486fd0bc9 1 2024-04-13T03:09:01.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.069+0000 7fc38f6de640 1 Processor -- start 2024-04-13T03:09:01.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 -- start start 2024-04-13T03:09:01.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc3880d0090 0x7fc3880d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc3880725c0 0x7fc388072a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc388162530 con 0x7fc38806d7e0 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc388072f70 con 0x7fc3880725c0 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.070+0000 7fc38f6de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc388163420 con 0x7fc3880d0090 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc3880d0090 0x7fc3880d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc3880d0090 0x7fc3880d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42526/0 (socket says 172.21.15.114:42526) 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 -- 172.21.15.114:0/321455460 learned_addr learned my addr 172.21.15.114:0/321455460 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38dc7b640 1 --2- 172.21.15.114:0/321455460 >> v2:172.21.15.106:3300/0 conn(0x7fc3880725c0 0x7fc388072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 -- 172.21.15.114:0/321455460 >> v2:172.21.15.106:3300/0 conn(0x7fc3880725c0 msgr2=0x7fc388072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 --2- 172.21.15.114:0/321455460 >> v2:172.21.15.106:3300/0 conn(0x7fc3880725c0 0x7fc388072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 -- 172.21.15.114:0/321455460 >> v1:172.21.15.114:6789/0 conn(0x7fc38806d7e0 legacy=0x7fc38806f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.071+0000 7fc38cc79640 1 -- 172.21.15.114:0/321455460 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc388162810 con 0x7fc3880d0090 2024-04-13T03:09:01.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38cc79640 1 --2- 172.21.15.114:0/321455460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880d0090 0x7fc3880d04f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3880d1c30 tx=0x7fc37802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770cbcd16e4f8b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc37ffff640 1 -- 172.21.15.114:0/321455460 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378034070 con 0x7fc3880d0090 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc37ffff640 1 -- 172.21.15.114:0/321455460 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378030860 con 0x7fc3880d0090 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc37ffff640 1 -- 172.21.15.114:0/321455460 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378030c30 con 0x7fc3880d0090 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 -- 172.21.15.114:0/321455460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880d0090 msgr2=0x7fc3880d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 --2- 172.21.15.114:0/321455460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880d0090 0x7fc3880d04f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3880d1c30 tx=0x7fc37802b9a0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 -- 172.21.15.114:0/321455460 shutdown_connections 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 --2- 172.21.15.114:0/321455460 >> v2:172.21.15.106:3300/0 conn(0x7fc3880725c0 0x7fc388072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 --2- 172.21.15.114:0/321455460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880d0090 0x7fc3880d04f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 -- 172.21.15.114:0/321455460 >> 172.21.15.114:0/321455460 conn(0x7fc3880d0c30 msgr2=0x7fc3880d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 -- 172.21.15.114:0/321455460 shutdown_connections 2024-04-13T03:09:01.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.072+0000 7fc38f6de640 1 -- 172.21.15.114:0/321455460 wait complete. 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 Processor -- start 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 -- start start 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 0x7fc38814ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3880d0090 0x7fc38815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881630b0 con 0x7fc3880d0090 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc388162dd0 con 0x7fc3880725c0 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38f6de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc38806dbc0 con 0x7fc38806d7e0 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38cc79640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38d47a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 0x7fc38814ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38cc79640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42532/0 (socket says 172.21.15.114:42532) 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38cc79640 1 -- 172.21.15.114:0/3487953069 learned_addr learned my addr 172.21.15.114:0/3487953069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.074+0000 7fc38dc7b640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3880d0090 0x7fc38815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 -- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 msgr2=0x7fc38814ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 0x7fc38814ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 -- 172.21.15.114:0/3487953069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3880d0090 msgr2=0x7fc38815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3880d0090 0x7fc38815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 -- 172.21.15.114:0/3487953069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3881601a0 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38d47a640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 0x7fc38814ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38cc79640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc37802b9e0 tx=0x7fc3780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378034040 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37803ad20 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378044610 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc388160480 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38815f450 con 0x7fc3880725c0 2024-04-13T03:09:01.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc378037070 con 0x7fc3880725c0 2024-04-13T03:09:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.075+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc37804d7a0 con 0x7fc3880725c0 2024-04-13T03:09:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.076+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388069120 con 0x7fc3880725c0 2024-04-13T03:09:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.078+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc37803f070 con 0x7fc3880725c0 2024-04-13T03:09:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.149+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "611ca47e-a863-4c90-baef-28b486fd0bc9", "id": 1} v 0) -- 0x7fc38806c480 con 0x7fc3880725c0 2024-04-13T03:09:01.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.153+0000 7fc37dffb640 1 -- 172.21.15.114:0/3487953069 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "611ca47e-a863-4c90-baef-28b486fd0bc9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc37805a020 con 0x7fc3880725c0 2024-04-13T03:09:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 msgr2=0x7fc38815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc37802b9e0 tx=0x7fc3780046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 shutdown_connections 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3880d0090 0x7fc38815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3880725c0 0x7fc38815bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 --2- 172.21.15.114:0/3487953069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc38806d7e0 0x7fc38814ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 >> 172.21.15.114:0/3487953069 conn(0x7fc3880d0c30 msgr2=0x7fc38806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 shutdown_connections 2024-04-13T03:09:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.154+0000 7fc38f6de640 1 -- 172.21.15.114:0/3487953069 wait complete. 2024-04-13T03:09:01.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new b2cfad75-5479-41fc-b4a5-8f481ce3d6ec 2 2024-04-13T03:09:01.263 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.262+0000 7f0d05a49040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T03:09:01.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.269+0000 7fc76c5e9640 1 Processor -- start 2024-04-13T03:09:01.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.269+0000 7fc76c5e9640 1 -- start start 2024-04-13T03:09:01.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.270+0000 7fc76c5e9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc764159220 0x7fc7640b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.270+0000 7fc76c5e9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc764159600 con 0x7fc76415c2d0 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.270+0000 7fc76c5e9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76415c6e0 con 0x7fc764072e10 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.270+0000 7fc76c5e9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7640b4690 con 0x7fc764159220 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.270+0000 7fc76a385640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc764159220 0x7fc7640b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc764159220 0x7fc7640b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42536/0 (socket says 172.21.15.114:42536) 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc769b84640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc764072e10 0x7fc76415bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54698/0 (socket says 172.21.15.114:54698) 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 -- 172.21.15.114:0/1294731915 learned_addr learned my addr 172.21.15.114:0/1294731915 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 -- 172.21.15.114:0/1294731915 >> v1:172.21.15.114:6789/0 conn(0x7fc764072e10 legacy=0x7fc76415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 -- 172.21.15.114:0/1294731915 >> v1:172.21.15.106:6789/0 conn(0x7fc76415c2d0 legacy=0x7fc7640cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 -- 172.21.15.114:0/1294731915 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76415c960 con 0x7fc764159220 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.271+0000 7fc76a385640 1 --2- 172.21.15.114:0/1294731915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc764159220 0x7fc7640b5150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc760009870 tx=0x7fc76002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933a338257310dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.272+0000 7fc769383640 1 -- 172.21.15.114:0/1294731915 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76002f840 con 0x7fc764159220 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.272+0000 7fc769383640 1 -- 172.21.15.114:0/1294731915 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76002f9e0 con 0x7fc764159220 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.272+0000 7fc769383640 1 -- 172.21.15.114:0/1294731915 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76002fdb0 con 0x7fc764159220 2024-04-13T03:09:01.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.272+0000 7fc76c5e9640 1 -- 172.21.15.114:0/1294731915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc764159220 msgr2=0x7fc7640b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.272+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/1294731915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc764159220 0x7fc7640b5150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc760009870 tx=0x7fc76002ba20 comp rx=0 tx=0).stop 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.273+0000 7fc76c5e9640 1 -- 172.21.15.114:0/1294731915 shutdown_connections 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.273+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/1294731915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc764159220 0x7fc7640b5150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.273+0000 7fc76c5e9640 1 -- 172.21.15.114:0/1294731915 >> 172.21.15.114:0/1294731915 conn(0x7fc76406f180 msgr2=0x7fc76406f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.273+0000 7fc76c5e9640 1 -- 172.21.15.114:0/1294731915 shutdown_connections 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.273+0000 7fc76c5e9640 1 -- 172.21.15.114:0/1294731915 wait complete. 2024-04-13T03:09:01.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.274+0000 7fc76c5e9640 1 Processor -- start 2024-04-13T03:09:01.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.275+0000 7fc76c5e9640 1 -- start start 2024-04-13T03:09:01.278 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.275+0000 7f8d5dc2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc764159220 0x7fc7640cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 0x7fc7640cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7640732a0 con 0x7fc764159220 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7640b4c30 con 0x7fc76415c2d0 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76c5e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc764073420 con 0x7fc764072e10 2024-04-13T03:09:01.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76ab86640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 0x7fc7640cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52668/0 (socket says 172.21.15.114:52668) 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 -- 172.21.15.114:0/2076398055 learned_addr learned my addr 172.21.15.114:0/2076398055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc769b84640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc764159220 0x7fc7640cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 -- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 msgr2=0x7fc7640cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 0x7fc7640cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 -- 172.21.15.114:0/2076398055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc764159220 msgr2=0x7fc7640cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc764159220 0x7fc7640cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 -- 172.21.15.114:0/2076398055 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7640aa340 con 0x7fc764072e10 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76ab86640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 0x7fc7640cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc76a385640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc760004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760041400 con 0x7fc764072e10 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.276+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760046070 con 0x7fc764072e10 2024-04-13T03:09:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7640aa620 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760041800 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7640aabe0 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc760040070 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc76004b9c0 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.277+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc764069120 con 0x7fc764072e10 2024-04-13T03:09:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.280+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc760033030 con 0x7fc764072e10 2024-04-13T03:09:01.317 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.316+0000 7f0d05a49040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T03:09:01.333 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.332+0000 7f8d5dc2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T03:09:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.353+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b2cfad75-5479-41fc-b4a5-8f481ce3d6ec", "id": 2} v 0) -- 0x7fc76406e110 con 0x7fc764072e10 2024-04-13T03:09:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.358+0000 7fc756ffd640 1 -- 172.21.15.114:0/2076398055 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b2cfad75-5479-41fc-b4a5-8f481ce3d6ec", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc76002f8e0 con 0x7fc764072e10 2024-04-13T03:09:01.360 INFO:teuthology.orchestra.run.smithi114.stdout:2 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 msgr2=0x7fc76414bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc760004730 comp rx=0 tx=0).stop 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 shutdown_connections 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc76415c2d0 0x7fc7640cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc764159220 0x7fc7640cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 --2- 172.21.15.114:0/2076398055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc764072e10 0x7fc76414bfc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 >> 172.21.15.114:0/2076398055 conn(0x7fc76406f180 msgr2=0x7fc7640b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 shutdown_connections 2024-04-13T03:09:01.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.359+0000 7fc76c5e9640 1 -- 172.21.15.114:0/2076398055 wait complete. 2024-04-13T03:09:01.362 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.361+0000 7f0d05a49040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T03:09:01.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 687c55b8-bccf-42fe-b20c-85d0fc53095b 3 2024-04-13T03:09:01.381 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.379+0000 7f8d5dc2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T03:09:01.416 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.415+0000 7f0d05a49040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T03:09:01.437 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.435+0000 7f8d5dc2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T03:09:01.458 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.458+0000 7f0d05a49040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T03:09:01.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.469+0000 7f265438c640 1 Processor -- start 2024-04-13T03:09:01.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.469+0000 7f265438c640 1 -- start start 2024-04-13T03:09:01.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f265438c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f264c06e8a0 0x7f264c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f265438c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f264c06f800 0x7f264c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f265438c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c070160 con 0x7f264c159120 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f265438c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f264c0cf700 con 0x7f264c06e8a0 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f265438c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f264c15c400 con 0x7f264c06f800 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f264c06f800 0x7f264c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f264c06f800 0x7f264c06fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42554/0 (socket says 172.21.15.114:42554) 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 -- 172.21.15.114:0/4228652385 learned_addr learned my addr 172.21.15.114:0/4228652385 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2651927640 1 --2- 172.21.15.114:0/4228652385 >> v2:172.21.15.106:3300/0 conn(0x7f264c06e8a0 0x7f264c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 -- 172.21.15.114:0/4228652385 >> v2:172.21.15.106:3300/0 conn(0x7f264c06e8a0 msgr2=0x7f264c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 --2- 172.21.15.114:0/4228652385 >> v2:172.21.15.106:3300/0 conn(0x7f264c06e8a0 0x7f264c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 -- 172.21.15.114:0/4228652385 >> v1:172.21.15.106:6789/0 conn(0x7f264c159120 legacy=0x7f264c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.470+0000 7f2652929640 1 -- 172.21.15.114:0/4228652385 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c15c6e0 con 0x7f264c06f800 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.471+0000 7f2652929640 1 --2- 172.21.15.114:0/4228652385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c06fc20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f264c0b5ce0 tx=0x7f264402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba2afa4bceba67a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.471+0000 7f2651126640 1 -- 172.21.15.114:0/4228652385 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264402cb20 con 0x7f264c06f800 2024-04-13T03:09:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.471+0000 7f2651126640 1 -- 172.21.15.114:0/4228652385 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264402ccc0 con 0x7f264c06f800 2024-04-13T03:09:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.471+0000 7f2651126640 1 -- 172.21.15.114:0/4228652385 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644034610 con 0x7f264c06f800 2024-04-13T03:09:01.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 -- 172.21.15.114:0/4228652385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 msgr2=0x7f264c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 --2- 172.21.15.114:0/4228652385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c06fc20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f264c0b5ce0 tx=0x7f264402bf80 comp rx=0 tx=0).stop 2024-04-13T03:09:01.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 -- 172.21.15.114:0/4228652385 shutdown_connections 2024-04-13T03:09:01.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 --2- 172.21.15.114:0/4228652385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c06fc20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 --2- 172.21.15.114:0/4228652385 >> v2:172.21.15.106:3300/0 conn(0x7f264c06e8a0 0x7f264c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 -- 172.21.15.114:0/4228652385 >> 172.21.15.114:0/4228652385 conn(0x7f264c0d1b60 msgr2=0x7f264c06f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 -- 172.21.15.114:0/4228652385 shutdown_connections 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.472+0000 7f265438c640 1 -- 172.21.15.114:0/4228652385 wait complete. 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.473+0000 7f265438c640 1 Processor -- start 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.473+0000 7f265438c640 1 -- start start 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f264c06e8a0 0x7f264c1f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 0x7f264c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f264c15c0c0 con 0x7f264c06e8a0 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f264c159500 con 0x7f264c06f800 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0cf320 con 0x7f264c159120 2024-04-13T03:09:01.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2652929640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 0x7f264c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42568/0 (socket says 172.21.15.114:42568) 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2652128640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f264c06e8a0 0x7f264c1f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2652929640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 0x7f264c06c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52680/0 (socket says 172.21.15.114:52680) 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 -- 172.21.15.114:0/1588329917 learned_addr learned my addr 172.21.15.114:0/1588329917 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 -- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 msgr2=0x7f264c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 0x7f264c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 -- 172.21.15.114:0/1588329917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f264c06e8a0 msgr2=0x7f264c1f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f264c06e8a0 0x7f264c1f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 -- 172.21.15.114:0/1588329917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0cc310 con 0x7f264c06f800 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2651927640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9e0 tx=0x7f263c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c017070 con 0x7f264c06f800 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c00c9d0 con 0x7f264c06f800 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00cdc0 con 0x7f264c06f800 2024-04-13T03:09:01.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c0cb270 con 0x7f264c06f800 2024-04-13T03:09:01.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.474+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c0cb830 con 0x7f264c06f800 2024-04-13T03:09:01.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.475+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f263c01a070 con 0x7f264c06f800 2024-04-13T03:09:01.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.475+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f263c006660 con 0x7f264c06f800 2024-04-13T03:09:01.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.475+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c069120 con 0x7f264c06f800 2024-04-13T03:09:01.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.478+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f263c006020 con 0x7f264c06f800 2024-04-13T03:09:01.481 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.480+0000 7f8d5dc2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T03:09:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.550+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "687c55b8-bccf-42fe-b20c-85d0fc53095b", "id": 3} v 0) -- 0x7f264c0ce340 con 0x7f264c06f800 2024-04-13T03:09:01.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.556+0000 7f2642ffd640 1 -- 172.21.15.114:0/1588329917 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "687c55b8-bccf-42fe-b20c-85d0fc53095b", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f263c005990 con 0x7f264c06f800 2024-04-13T03:09:01.557 INFO:teuthology.orchestra.run.smithi114.stdout:3 2024-04-13T03:09:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 msgr2=0x7f264c1f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9e0 tx=0x7f263c00eea0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 shutdown_connections 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f264c159120 0x7f264c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f264c06f800 0x7f264c1f4280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 --2- 172.21.15.114:0/1588329917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f264c06e8a0 0x7f264c1f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 >> 172.21.15.114:0/1588329917 conn(0x7f264c0d1b60 msgr2=0x7f264c0cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 shutdown_connections 2024-04-13T03:09:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.557+0000 7f265438c640 1 -- 172.21.15.114:0/1588329917 wait complete. 2024-04-13T03:09:01.568 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 8daeed25-5121-4623-86b7-f96990e6f838 4 2024-04-13T03:09:01.577 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.576+0000 7f0d05a49040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T03:09:01.602 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.601+0000 7f8d5dc2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T03:09:01.645 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.644+0000 7f0d05a49040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T03:09:01.676 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.674+0000 7f8d5dc2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T03:09:01.691 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.689+0000 7f0d05a49040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T03:09:01.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.701+0000 7fed0ea36640 1 Processor -- start 2024-04-13T03:09:01.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.701+0000 7fed0ea36640 1 -- start start 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed0ea36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed08073520 0x7fed080b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed0ea36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed08159600 con 0x7fed08159220 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed0ea36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed080b4690 con 0x7fed0815bbc0 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed0ea36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fed0815c710 con 0x7fed08073520 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed077fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed08073520 0x7fed080b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed077fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed08073520 0x7fed080b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42582/0 (socket says 172.21.15.114:42582) 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.702+0000 7fed077fe640 1 -- 172.21.15.114:0/3033799616 learned_addr learned my addr 172.21.15.114:0/3033799616 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed077fe640 1 -- 172.21.15.114:0/3033799616 >> v1:172.21.15.114:6789/0 conn(0x7fed0815bbc0 legacy=0x7fed0815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed077fe640 1 -- 172.21.15.114:0/3033799616 >> v1:172.21.15.106:6789/0 conn(0x7fed08159220 legacy=0x7fed08072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed077fe640 1 -- 172.21.15.114:0/3033799616 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0815c960 con 0x7fed08073520 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed077fe640 1 --2- 172.21.15.114:0/3033799616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed08073520 0x7fed080b5150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009870 tx=0x7fecf802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3338a27249804470 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed06ffd640 1 -- 172.21.15.114:0/3033799616 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf802f840 con 0x7fed08073520 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed06ffd640 1 -- 172.21.15.114:0/3033799616 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf802f9e0 con 0x7fed08073520 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.703+0000 7fed06ffd640 1 -- 172.21.15.114:0/3033799616 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf802fdb0 con 0x7fed08073520 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 -- 172.21.15.114:0/3033799616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed08073520 msgr2=0x7fed080b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 --2- 172.21.15.114:0/3033799616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed08073520 0x7fed080b5150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009870 tx=0x7fecf802ba20 comp rx=0 tx=0).stop 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 -- 172.21.15.114:0/3033799616 shutdown_connections 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 --2- 172.21.15.114:0/3033799616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed08073520 0x7fed080b5150 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 -- 172.21.15.114:0/3033799616 >> 172.21.15.114:0/3033799616 conn(0x7fed0806f180 msgr2=0x7fed0806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 -- 172.21.15.114:0/3033799616 shutdown_connections 2024-04-13T03:09:01.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.704+0000 7fed0ea36640 1 -- 172.21.15.114:0/3033799616 wait complete. 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.705+0000 7fed0ea36640 1 Processor -- start 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.705+0000 7fed0ea36640 1 -- start start 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed08159220 0x7fed080c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed080cf400 con 0x7fed08159220 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fed080b4c30 con 0x7fed0815bbc0 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0ea36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fed080cf580 con 0x7fed08073520 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed07fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0cfd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed07fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52696/0 (socket says 172.21.15.114:52696) 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed0cfd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42588/0 (socket says 172.21.15.114:42588) 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed07fff640 1 -- 172.21.15.114:0/4270847303 learned_addr learned my addr 172.21.15.114:0/4270847303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed077fe640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed08159220 0x7fed080c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.706+0000 7fed07fff640 1 -- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 msgr2=0x7fed0806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed07fff640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed07fff640 1 -- 172.21.15.114:0/4270847303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed08159220 msgr2=0x7fed080c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed07fff640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed08159220 0x7fed080c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed07fff640 1 -- 172.21.15.114:0/4270847303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed080a9580 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed0cfd3640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed07fff640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c430 tx=0x7fecfc00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0052d0 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc005490 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007260 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed080a9820 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed080a9de0 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fecfc007400 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.707+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fecfc011590 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.708+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed080ce460 con 0x7fed08073520 2024-04-13T03:09:01.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.710+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fed080ce460 con 0x7fed08073520 2024-04-13T03:09:01.725 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.723+0000 7f8d5dc2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T03:09:01.744 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.743+0000 7f0d05a49040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T03:09:01.779 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.777+0000 7f8d5dc2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T03:09:01.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.783+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8daeed25-5121-4623-86b7-f96990e6f838", "id": 4} v 0) -- 0x7fed08069120 con 0x7fed08073520 2024-04-13T03:09:01.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.788+0000 7fed04ff9640 1 -- 172.21.15.114:0/4270847303 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8daeed25-5121-4623-86b7-f96990e6f838", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fed08159600 con 0x7fed08073520 2024-04-13T03:09:01.789 INFO:teuthology.orchestra.run.smithi114.stdout:4 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 msgr2=0x7fed08152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c430 tx=0x7fecfc00c8f0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 shutdown_connections 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0815bbc0 0x7fed0806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed08159220 0x7fed080c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 --2- 172.21.15.114:0/4270847303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed08073520 0x7fed08152190 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 >> 172.21.15.114:0/4270847303 conn(0x7fed0806f180 msgr2=0x7fed080b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 shutdown_connections 2024-04-13T03:09:01.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.789+0000 7fed0ea36640 1 -- 172.21.15.114:0/4270847303 wait complete. 2024-04-13T03:09:01.791 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:01.790+0000 7f0d05a49040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T03:09:01.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 6590374b-3603-439d-ab79-ce2e0094e7f8 5 2024-04-13T03:09:01.829 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:01.828+0000 7f8d5dc2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T03:09:01.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.898+0000 7f8d63e36640 1 Processor -- start 2024-04-13T03:09:01.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 -- start start 2024-04-13T03:09:01.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 0x7f8d5c073270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c159600 con 0x7f8d5c15bbc0 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c0737b0 con 0x7f8d5c159220 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d63e36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c15c700 con 0x7f8d5c072e10 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.899+0000 7f8d61bd2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d613d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 0x7f8d5c073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d61bd2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42604/0 (socket says 172.21.15.114:42604) 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d61bd2640 1 -- 172.21.15.114:0/2509729792 learned_addr learned my addr 172.21.15.114:0/2509729792 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d613d1640 1 -- 172.21.15.114:0/2509729792 >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 msgr2=0x7f8d5c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d613d1640 1 --2- 172.21.15.114:0/2509729792 >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d613d1640 1 -- 172.21.15.114:0/2509729792 >> v1:172.21.15.106:6789/0 conn(0x7f8d5c15bbc0 legacy=0x7f8d5c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d613d1640 1 -- 172.21.15.114:0/2509729792 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0b4690 con 0x7f8d5c072e10 2024-04-13T03:09:01.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.900+0000 7f8d61bd2640 1 --2- 172.21.15.114:0/2509729792 >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d613d1640 1 --2- 172.21.15.114:0/2509729792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 0x7f8d5c073270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097a0 tx=0x7f8d5802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae843038170838a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d60bd0640 1 -- 172.21.15.114:0/2509729792 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58035020 con 0x7f8d5c072e10 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d60bd0640 1 -- 172.21.15.114:0/2509729792 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58031830 con 0x7f8d5c072e10 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d60bd0640 1 -- 172.21.15.114:0/2509729792 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58031c00 con 0x7f8d5c072e10 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d63e36640 1 -- 172.21.15.114:0/2509729792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 msgr2=0x7f8d5c073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.901+0000 7f8d63e36640 1 --2- 172.21.15.114:0/2509729792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 0x7f8d5c073270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097a0 tx=0x7f8d5802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 -- 172.21.15.114:0/2509729792 shutdown_connections 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 --2- 172.21.15.114:0/2509729792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c072e10 0x7f8d5c073270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 --2- 172.21.15.114:0/2509729792 >> v2:172.21.15.114:3300/0 conn(0x7f8d5c159220 0x7f8d5c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 -- 172.21.15.114:0/2509729792 >> 172.21.15.114:0/2509729792 conn(0x7f8d5c06f180 msgr2=0x7f8d5c06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 -- 172.21.15.114:0/2509729792 shutdown_connections 2024-04-13T03:09:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.902+0000 7f8d63e36640 1 -- 172.21.15.114:0/2509729792 wait complete. 2024-04-13T03:09:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.903+0000 7f8d63e36640 1 Processor -- start 2024-04-13T03:09:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.904+0000 7f8d63e36640 1 -- start start 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d5c072e10 0x7f8d5c13dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 0x7f8d5c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b4c30 con 0x7f8d5c072e10 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b4950 con 0x7f8d5c15bbc0 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d63e36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c0cf400 con 0x7f8d5c159220 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52710/0 (socket says 172.21.15.114:52710) 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 -- 172.21.15.114:0/4173311307 learned_addr learned my addr 172.21.15.114:0/4173311307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d623d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 0x7f8d5c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d61bd2640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d5c072e10 0x7f8d5c13dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 -- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 msgr2=0x7f8d5c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 0x7f8d5c06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 -- 172.21.15.114:0/4173311307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d5c072e10 msgr2=0x7f8d5c13dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d5c072e10 0x7f8d5c13dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 -- 172.21.15.114:0/4173311307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c152720 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d623d3640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 0x7f8d5c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d613d1640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d5802bee0 tx=0x7f8d580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.905+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58041070 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5803f070 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58035040 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c152a00 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c152fc0 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d58038020 con 0x7f8d5c159220 2024-04-13T03:09:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f8d58045c60 con 0x7f8d5c159220 2024-04-13T03:09:01.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.906+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c069120 con 0x7f8d5c159220 2024-04-13T03:09:01.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.908+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8d58031800 con 0x7f8d5c159220 2024-04-13T03:09:01.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.980+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6590374b-3603-439d-ab79-ce2e0094e7f8", "id": 5} v 0) -- 0x7f8d5c06fec0 con 0x7f8d5c159220 2024-04-13T03:09:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.986+0000 7f8d527fc640 1 -- 172.21.15.114:0/4173311307 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6590374b-3603-439d-ab79-ce2e0094e7f8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8d58045410 con 0x7f8d5c159220 2024-04-13T03:09:01.987 INFO:teuthology.orchestra.run.smithi114.stdout:5 2024-04-13T03:09:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 msgr2=0x7f8d5c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d5802bee0 tx=0x7f8d580046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 shutdown_connections 2024-04-13T03:09:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8d5c159220 0x7f8d5c06d1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d5c15bbc0 0x7f8d5c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 --2- 172.21.15.114:0/4173311307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d5c072e10 0x7f8d5c13dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 >> 172.21.15.114:0/4173311307 conn(0x7f8d5c06f180 msgr2=0x7f8d5c15bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 shutdown_connections 2024-04-13T03:09:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:01.987+0000 7f8d63e36640 1 -- 172.21.15.114:0/4173311307 wait complete. 2024-04-13T03:09:01.998 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09 6 2024-04-13T03:09:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 Processor -- start 2024-04-13T03:09:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 -- start start 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe81415ea80 0x7fe8140d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140cfe70 con 0x7fe8140d1630 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe81415ee60 con 0x7fe81415ea80 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.140+0000 7fe8193bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe814162af0 con 0x7fe814162550 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe8127fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe812ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe81415ea80 0x7fe8140d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe8127fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52718/0 (socket says 172.21.15.114:52718) 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe8127fc640 1 -- 172.21.15.114:0/3222221072 learned_addr learned my addr 172.21.15.114:0/3222221072 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe812ffd640 1 -- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 msgr2=0x7fe8140b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe812ffd640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe812ffd640 1 -- 172.21.15.114:0/3222221072 >> v1:172.21.15.106:6789/0 conn(0x7fe8140d1630 legacy=0x7fe8140d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe812ffd640 1 -- 172.21.15.114:0/3222221072 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe814162dd0 con 0x7fe81415ea80 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.141+0000 7fe8127fc640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe812ffd640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe81415ea80 0x7fe8140d22b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8040097a0 tx=0x7fe80402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced5558d7475880d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe811ffb640 1 -- 172.21.15.114:0/3222221072 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804035020 con 0x7fe81415ea80 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe811ffb640 1 -- 172.21.15.114:0/3222221072 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe804031830 con 0x7fe81415ea80 2024-04-13T03:09:02.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe811ffb640 1 -- 172.21.15.114:0/3222221072 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804031c00 con 0x7fe81415ea80 2024-04-13T03:09:02.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 -- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe81415ea80 msgr2=0x7fe8140d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe81415ea80 0x7fe8140d22b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8040097a0 tx=0x7fe80402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 -- 172.21.15.114:0/3222221072 shutdown_connections 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe8140b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 --2- 172.21.15.114:0/3222221072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe81415ea80 0x7fe8140d22b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 -- 172.21.15.114:0/3222221072 >> 172.21.15.114:0/3222221072 conn(0x7fe81409da50 msgr2=0x7fe8140b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 -- 172.21.15.114:0/3222221072 shutdown_connections 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.142+0000 7fe8193bb640 1 -- 172.21.15.114:0/3222221072 wait complete. 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 Processor -- start 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 -- start start 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415ea80 0x7fe81406e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe81414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140b3e70 con 0x7fe81415ea80 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe814163390 con 0x7fe8140d1630 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8193bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140d1aa0 con 0x7fe814162550 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe812ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe812ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42620/0 (socket says 172.21.15.114:42620) 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe812ffd640 1 -- 172.21.15.114:0/1628981082 learned_addr learned my addr 172.21.15.114:0/1628981082 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8137fe640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe81414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.144+0000 7fe8127fc640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415ea80 0x7fe81406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 -- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 msgr2=0x7fe81414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe81414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 -- 172.21.15.114:0/1628981082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415ea80 msgr2=0x7fe81406e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415ea80 0x7fe81406e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 -- 172.21.15.114:0/1628981082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8141603b0 con 0x7fe8140d1630 2024-04-13T03:09:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe812ffd640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe80402bee0 tx=0x7fe8040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804035040 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80403b920 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe814160690 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80403bd10 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8140ca450 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe804038020 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.145+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe80403aa20 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.146+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe814068a10 con 0x7fe8140d1630 2024-04-13T03:09:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.148+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe80403acc0 con 0x7fe8140d1630 2024-04-13T03:09:02.172 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.171+0000 7f0d05a49040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T03:09:02.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.221+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09", "id": 6} v 0) -- 0x7fe8140d5350 con 0x7fe8140d1630 2024-04-13T03:09:02.226 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.225+0000 7f8d5dc2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T03:09:02.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.227+0000 7fe7fb7fe640 1 -- 172.21.15.114:0/1628981082 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe80403a430 con 0x7fe8140d1630 2024-04-13T03:09:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:6 2024-04-13T03:09:02.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 msgr2=0x7fe8140c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe80402bee0 tx=0x7fe8040046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 shutdown_connections 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe814162550 0x7fe81414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415ea80 0x7fe81406e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 --2- 172.21.15.114:0/1628981082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8140d1630 0x7fe8140c9f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 >> 172.21.15.114:0/1628981082 conn(0x7fe81409da50 msgr2=0x7fe8140b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 shutdown_connections 2024-04-13T03:09:02.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.228+0000 7fe8193bb640 1 -- 172.21.15.114:0/1628981082 wait complete. 2024-04-13T03:09:02.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new d90993dc-97b5-49ab-803c-c0ba0225626f 7 2024-04-13T03:09:02.308 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.307+0000 7f0d05a49040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T03:09:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.338+0000 7f0684605640 1 Processor -- start 2024-04-13T03:09:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.338+0000 7f0684605640 1 -- start start 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0684605640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 0x7f067c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0684605640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f067c0ceb30 0x7f067c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0684605640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f067c162730 con 0x7f067c0ce1d0 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0684605640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f067c162a10 con 0x7f067c0ceb30 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0684605640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f067c162cf0 con 0x7f067c15eb40 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f06823a1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 0x7f067c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0681ba0640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f067c0ce1d0 0x7f067c0b5640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54720/0 (socket says 172.21.15.114:54720) 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0681ba0640 1 -- 172.21.15.114:0/3628813307 learned_addr learned my addr 172.21.15.114:0/3628813307 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f0682ba2640 1 --2- 172.21.15.114:0/3628813307 >> v2:172.21.15.106:3300/0 conn(0x7f067c0ceb30 0x7f067c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f06823a1640 1 -- 172.21.15.114:0/3628813307 >> v2:172.21.15.106:3300/0 conn(0x7f067c0ceb30 msgr2=0x7f067c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f06823a1640 1 --2- 172.21.15.114:0/3628813307 >> v2:172.21.15.106:3300/0 conn(0x7f067c0ceb30 0x7f067c0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f06823a1640 1 -- 172.21.15.114:0/3628813307 >> v1:172.21.15.114:6789/0 conn(0x7f067c0ce1d0 legacy=0x7f067c0b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f068139f640 1 -- 172.21.15.114:0/3628813307 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1599936590 0 0) 0x7f067c162730 con 0x7f067c0ce1d0 2024-04-13T03:09:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.340+0000 7f06823a1640 1 -- 172.21.15.114:0/3628813307 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c162fd0 con 0x7f067c15eb40 2024-04-13T03:09:02.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.341+0000 7f06823a1640 1 --2- 172.21.15.114:0/3628813307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 0x7f067c0b5100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097a0 tx=0x7f066c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75204dc76325528e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.341+0000 7f068139f640 1 -- 172.21.15.114:0/3628813307 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c035020 con 0x7f067c15eb40 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.341+0000 7f068139f640 1 -- 172.21.15.114:0/3628813307 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066c031830 con 0x7f067c15eb40 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.341+0000 7f068139f640 1 -- 172.21.15.114:0/3628813307 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c031c00 con 0x7f067c15eb40 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 -- 172.21.15.114:0/3628813307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 msgr2=0x7f067c0b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 --2- 172.21.15.114:0/3628813307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 0x7f067c0b5100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097a0 tx=0x7f066c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 -- 172.21.15.114:0/3628813307 shutdown_connections 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 --2- 172.21.15.114:0/3628813307 >> v2:172.21.15.106:3300/0 conn(0x7f067c0ceb30 0x7f067c0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 --2- 172.21.15.114:0/3628813307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c15eb40 0x7f067c0b5100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 -- 172.21.15.114:0/3628813307 >> 172.21.15.114:0/3628813307 conn(0x7f067c0cd6b0 msgr2=0x7f067c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 -- 172.21.15.114:0/3628813307 shutdown_connections 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.342+0000 7f0684605640 1 -- 172.21.15.114:0/3628813307 wait complete. 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.343+0000 7f0684605640 1 Processor -- start 2024-04-13T03:09:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.343+0000 7f0684605640 1 -- start start 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f067c0ce1d0 0x7f067c155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f067c06f8c0 con 0x7f067c0ce1d0 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1634f0 con 0x7f067c15eb40 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f067c0ce790 con 0x7f067c0ceb30 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0682ba2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.114:52746/0 (socket says 172.21.15.114:52746) 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 -- 172.21.15.114:0/2048903759 learned_addr learned my addr 172.21.15.114:0/2048903759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-13T03:09:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0682ba2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42626/0 (socket says 172.21.15.114:42626) 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f06823a1640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f067c0ce1d0 0x7f067c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 -- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 msgr2=0x7f067c152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 -- 172.21.15.114:0/2048903759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f067c0ce1d0 msgr2=0x7f067c155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f067c0ce1d0 0x7f067c155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 -- 172.21.15.114:0/2048903759 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c150130 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0682ba2640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0681ba0640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f067800c500 tx=0x7f067800c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678018070 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0678007580 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678004a20 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c153ff0 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.344+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1545b0 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.345+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f067801b070 con 0x7f067c0ceb30 2024-04-13T03:09:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.345+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0678004bc0 con 0x7f067c0ceb30 2024-04-13T03:09:02.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.345+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c069120 con 0x7f067c0ceb30 2024-04-13T03:09:02.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.347+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f067c150130 con 0x7f067c0ceb30 2024-04-13T03:09:02.369 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.368+0000 7f8d5dc2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T03:09:02.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.420+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d90993dc-97b5-49ab-803c-c0ba0225626f", "id": 7} v 0) -- 0x7f067c0cd080 con 0x7f067c0ceb30 2024-04-13T03:09:02.425 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.425+0000 7f0d05a49040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T03:09:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.426+0000 7f0672ffd640 1 -- 172.21.15.114:0/2048903759 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d90993dc-97b5-49ab-803c-c0ba0225626f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f067801d440 con 0x7f067c0ceb30 2024-04-13T03:09:02.427 INFO:teuthology.orchestra.run.smithi114.stdout:7 2024-04-13T03:09:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 msgr2=0x7f067c14fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f067800c500 tx=0x7f067800c9c0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 shutdown_connections 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f067c0ceb30 0x7f067c14fbf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f067c15eb40 0x7f067c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 --2- 172.21.15.114:0/2048903759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f067c0ce1d0 0x7f067c155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 >> 172.21.15.114:0/2048903759 conn(0x7f067c0cd6b0 msgr2=0x7f067c1592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 shutdown_connections 2024-04-13T03:09:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:09:02.427+0000 7f0684605640 1 -- 172.21.15.114:0/2048903759 wait complete. 2024-04-13T03:09:02.438 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-13T03:09:02.439 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-13T03:09:02.441 INFO:tasks.ceph.osd.0:Started 2024-04-13T03:09:02.441 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-13T03:09:02.441 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-13T03:09:02.445 INFO:tasks.ceph.osd.1:Started 2024-04-13T03:09:02.445 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-13T03:09:02.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-13T03:09:02.447 INFO:tasks.ceph.osd.2:Started 2024-04-13T03:09:02.447 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-13T03:09:02.447 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-13T03:09:02.450 INFO:tasks.ceph.osd.3:Started 2024-04-13T03:09:02.450 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-13T03:09:02.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-13T03:09:02.452 INFO:tasks.ceph.osd.4:Started 2024-04-13T03:09:02.452 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-13T03:09:02.452 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-13T03:09:02.454 INFO:tasks.ceph.osd.5:Started 2024-04-13T03:09:02.455 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-13T03:09:02.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-13T03:09:02.457 INFO:tasks.ceph.osd.6:Started 2024-04-13T03:09:02.457 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-13T03:09:02.457 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-13T03:09:02.459 INFO:tasks.ceph.osd.7:Started 2024-04-13T03:09:02.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:02.481 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.480+0000 7f0d05a49040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T03:09:02.504 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.503+0000 7f8d5dc2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T03:09:02.565 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.564+0000 7f8d5dc2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T03:09:02.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.578+0000 7f55ee7cc640 1 Processor -- start 2024-04-13T03:09:02.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.578+0000 7f55ee7cc640 1 -- start start 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ee7cc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55e815ed90 0x7f55e80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ee7cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f55e80b3d60 0x7f55e80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ee7cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80cfd20 con 0x7f55e815e9b0 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ee7cc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f55e8162af0 con 0x7f55e815ed90 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ee7cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f55e8162dd0 con 0x7f55e80b3d60 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55e77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55e815ed90 0x7f55e80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55e7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f55e815e9b0 0x7f55e80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37848/0 (socket says 172.21.15.106:37848) 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55e77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55e815ed90 0x7f55e80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36338/0 (socket says 172.21.15.106:36338) 2024-04-13T03:09:02.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55e7fff640 1 -- 172.21.15.106:0/147130055 learned_addr learned my addr 172.21.15.106:0/147130055 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.579+0000 7f55ecd69640 1 --2- 172.21.15.106:0/147130055 >> v2:172.21.15.114:3300/0 conn(0x7f55e80b3d60 0x7f55e80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e77fe640 1 -- 172.21.15.106:0/147130055 >> v2:172.21.15.114:3300/0 conn(0x7f55e80b3d60 msgr2=0x7f55e80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e77fe640 1 --2- 172.21.15.106:0/147130055 >> v2:172.21.15.114:3300/0 conn(0x7f55e80b3d60 0x7f55e80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e77fe640 1 -- 172.21.15.106:0/147130055 >> v1:172.21.15.106:6789/0 conn(0x7f55e815e9b0 legacy=0x7f55e80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e77fe640 1 -- 172.21.15.106:0/147130055 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e81630b0 con 0x7f55e815ed90 2024-04-13T03:09:02.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e6ffd640 1 -- 172.21.15.106:0/147130055 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407399841 0 0) 0x7f55e80cfd20 con 0x7f55e815e9b0 2024-04-13T03:09:02.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e77fe640 1 --2- 172.21.15.106:0/147130055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e815ed90 0x7f55e80d2300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55e80cb7b0 tx=0x7f55d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e8b46ef59cb100 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.580+0000 7f55e6ffd640 1 -- 172.21.15.106:0/147130055 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8034070 con 0x7f55e815ed90 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55e6ffd640 1 -- 172.21.15.106:0/147130055 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d8030610 con 0x7f55e815ed90 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55e6ffd640 1 -- 172.21.15.106:0/147130055 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d80309e0 con 0x7f55e815ed90 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 -- 172.21.15.106:0/147130055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e815ed90 msgr2=0x7f55e80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/147130055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e815ed90 0x7f55e80d2300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55e80cb7b0 tx=0x7f55d802b960 comp rx=0 tx=0).stop 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 -- 172.21.15.106:0/147130055 shutdown_connections 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/147130055 >> v2:172.21.15.114:3300/0 conn(0x7f55e80b3d60 0x7f55e80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/147130055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e815ed90 0x7f55e80d2300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 -- 172.21.15.106:0/147130055 >> 172.21.15.106:0/147130055 conn(0x7f55e808bf10 msgr2=0x7f55e80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 -- 172.21.15.106:0/147130055 shutdown_connections 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.581+0000 7f55ee7cc640 1 -- 172.21.15.106:0/147130055 wait complete. 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.582+0000 7f55ee7cc640 1 Processor -- start 2024-04-13T03:09:02.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 -- start start 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55e815e9b0 0x7f55e80cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8162800 con 0x7f55e80cd710 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8162680 con 0x7f55e80b3d60 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ee7cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80d4d00 con 0x7f55e815e9b0 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ecd69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55e7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ecd69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36350/0 (socket says 172.21.15.106:36350) 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55ecd69640 1 -- 172.21.15.106:0/1792765204 learned_addr learned my addr 172.21.15.106:0/1792765204 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:02.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55e7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:46752/0 (socket says 172.21.15.106:46752) 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.583+0000 7f55e77fe640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55e815e9b0 0x7f55e80cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 -- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55e815e9b0 msgr2=0x7f55e80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55e815e9b0 0x7f55e80cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 -- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 msgr2=0x7f55e8152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 -- 172.21.15.106:0/1792765204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80cdd00 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e7fff640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ecd69640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55e000efc0 tx=0x7f55e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0018020 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e0014480 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e80cdfe0 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0014870 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e80b6eb0 con 0x7f55e80cd710 2024-04-13T03:09:02.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f55e0005d10 con 0x7f55e80cd710 2024-04-13T03:09:02.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.584+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f55e0013ab0 con 0x7f55e80cd710 2024-04-13T03:09:02.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.585+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e8068a10 con 0x7f55e80cd710 2024-04-13T03:09:02.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.587+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f55e0022070 con 0x7f55e80cd710 2024-04-13T03:09:02.617 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.616+0000 7f0d05a49040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T03:09:02.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.648+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55e80d53a0 con 0x7f55e80cd710 2024-04-13T03:09:02.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.649+0000 7f55e4ff9640 1 -- 172.21.15.106:0/1792765204 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f55e0010040 con 0x7f55e80cd710 2024-04-13T03:09:02.650 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:02.650 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:02.422340+0000","last_up_change":"0.000000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:02.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 msgr2=0x7f55e806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:02.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55e000efc0 tx=0x7f55e000c490 comp rx=0 tx=0).stop 2024-04-13T03:09:02.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 shutdown_connections 2024-04-13T03:09:02.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55e815e9b0 0x7f55e80cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55e80cd710 0x7f55e806d3e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 --2- 172.21.15.106:0/1792765204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55e80b3d60 0x7f55e8152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:02.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 >> 172.21.15.106:0/1792765204 conn(0x7f55e808bf10 msgr2=0x7f55e80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:02.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 shutdown_connections 2024-04-13T03:09:02.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:02.650+0000 7f55ee7cc640 1 -- 172.21.15.106:0/1792765204 wait complete. 2024-04-13T03:09:02.661 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-13T03:09:02.661 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-13T03:09:02.678 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.676+0000 7f8d5dc2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T03:09:02.716 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.715+0000 7f0d05a49040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T03:09:02.772 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.770+0000 7f8d5dc2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T03:09:02.875 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.874+0000 7f0d05a49040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T03:09:02.917 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:02.916+0000 7f0d05a49040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T03:09:02.926 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.925+0000 7f8d5dc2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T03:09:02.968 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:02.966+0000 7f8d5dc2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T03:09:03.056 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-13T03:09:03.055+0000 7fd445f19640 -1 Falling back to public interface 2024-04-13T03:09:03.060 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-04-13T03:09:03.058+0000 7f2bee489640 -1 Falling back to public interface 2024-04-13T03:09:03.064 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-04-13T03:09:03.063+0000 7f3aa6814640 -1 Falling back to public interface 2024-04-13T03:09:03.067 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-13T03:09:03.067+0000 7f5508c1d640 -1 Falling back to public interface 2024-04-13T03:09:03.068 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-13T03:09:03.067+0000 7f60cfa0f640 -1 Falling back to public interface 2024-04-13T03:09:03.070 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-04-13T03:09:03.069+0000 7f1fb72ac640 -1 Falling back to public interface 2024-04-13T03:09:03.074 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-04-13T03:09:03.073+0000 7f8442a0d640 -1 Falling back to public interface 2024-04-13T03:09:03.076 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:03.076+0000 7f0d05a49040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T03:09:03.079 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-13T03:09:03.078+0000 7ff6d8a11640 -1 Falling back to public interface 2024-04-13T03:09:03.125 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:03.123+0000 7f8d5dc2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T03:09:03.466 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:03.566 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.557+0000 7f102a208640 1 Processor -- start 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.557+0000 7f102a208640 1 -- start start 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415c5d0 0x7f1024073b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f102415fd50 0x7f10240b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 0x7f10240ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10240736a0 con 0x7f102415fd50 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f10240ce910 con 0x7f10240cdf40 2024-04-13T03:09:03.567 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f102a208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10240b4f50 con 0x7f102415c5d0 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f102415fd50 0x7f10240b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f102415fd50 0x7f10240b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41346/0 (socket says 172.21.15.106:41346) 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 -- 172.21.15.106:0/4214774129 learned_addr learned my addr 172.21.15.106:0/4214774129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1023fff640 1 --2- 172.21.15.106:0/4214774129 >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 0x7f10240ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f10237fe640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415c5d0 0x7f1024073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 -- 172.21.15.106:0/4214774129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415c5d0 msgr2=0x7f1024073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415c5d0 0x7f1024073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 -- 172.21.15.106:0/4214774129 >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 msgr2=0x7f10240ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:03.568 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 --2- 172.21.15.106:0/4214774129 >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 0x7f10240ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1022ffd640 1 -- 172.21.15.106:0/4214774129 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1024073050 con 0x7f102415fd50 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.558+0000 7f1023fff640 1 --2- 172.21.15.106:0/4214774129 >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 0x7f10240ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f1022ffd640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415fd50 0x7f10240b45e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10240b0b70 tx=0x7f101402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2085a488c9d40d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f10227fc640 1 -- 172.21.15.106:0/4214774129 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014038070 con 0x7f102415fd50 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f10227fc640 1 -- 172.21.15.106:0/4214774129 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101402f050 con 0x7f102415fd50 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f10227fc640 1 -- 172.21.15.106:0/4214774129 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014031970 con 0x7f102415fd50 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f102a208640 1 -- 172.21.15.106:0/4214774129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415fd50 msgr2=0x7f10240b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.559+0000 7f102a208640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415fd50 0x7f10240b45e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10240b0b70 tx=0x7f101402b7f0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 -- 172.21.15.106:0/4214774129 shutdown_connections 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 --2- 172.21.15.106:0/4214774129 >> v2:172.21.15.114:3300/0 conn(0x7f10240cdf40 0x7f10240ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415fd50 0x7f10240b45e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 --2- 172.21.15.106:0/4214774129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415c5d0 0x7f1024073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 -- 172.21.15.106:0/4214774129 >> 172.21.15.106:0/4214774129 conn(0x7f10240d1530 msgr2=0x7f10240d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:03.569 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 -- 172.21.15.106:0/4214774129 shutdown_connections 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.560+0000 7f102a208640 1 -- 172.21.15.106:0/4214774129 wait complete. 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.561+0000 7f102a208640 1 Processor -- start 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.561+0000 7f102a208640 1 -- start start 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10240cdf40 0x7f1024153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415fd50 0x7f1024159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10240b4b20 con 0x7f102415c5d0 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f102415c9b0 con 0x7f10240cdf40 2024-04-13T03:09:03.570 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10240b4ca0 con 0x7f102415fd50 2024-04-13T03:09:03.571 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.571 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41350/0 (socket says 172.21.15.106:41350) 2024-04-13T03:09:03.571 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 -- 172.21.15.106:0/3602388709 learned_addr learned my addr 172.21.15.106:0/3602388709 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:03.571 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f10237fe640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10240cdf40 0x7f1024153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.571 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1023fff640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415fd50 0x7f1024159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:03.572 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 -- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415fd50 msgr2=0x7f1024159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:03.572 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415fd50 0x7f1024159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.572 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 -- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10240cdf40 msgr2=0x7f1024153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:03.572 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10240cdf40 0x7f1024153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 -- 172.21.15.106:0/3602388709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10241f5e20 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f1022ffd640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1014009960 tx=0x7f10140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014042020 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101403d890 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1024157930 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014038040 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1024157ef0 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1014045020 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.562+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f101403eae0 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.563+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1024075cc0 con 0x7f102415c5d0 2024-04-13T03:09:03.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.565+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f101404c030 con 0x7f102415c5d0 2024-04-13T03:09:03.629 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.627+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f102406eab0 con 0x7f102415c5d0 2024-04-13T03:09:03.630 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.628+0000 7f100bfff640 1 -- 172.21.15.106:0/3602388709 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f101403ed80 con 0x7f102415c5d0 2024-04-13T03:09:03.630 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-13T03:09:03.630 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":10,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:02.422340+0000","last_up_change":"0.000000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 msgr2=0x7f10241f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1014009960 tx=0x7f10140046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 shutdown_connections 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f102415fd50 0x7f1024159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f102415c5d0 0x7f10241f58e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 --2- 172.21.15.106:0/3602388709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10240cdf40 0x7f1024153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 >> 172.21.15.106:0/3602388709 conn(0x7f10240d1530 msgr2=0x7f102415f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 shutdown_connections 2024-04-13T03:09:03.631 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:03.629+0000 7f102a208640 1 -- 172.21.15.106:0/3602388709 wait complete. 2024-04-13T03:09:03.639 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-13T03:09:03.909 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:03.908+0000 7f0d05a49040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T03:09:03.951 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:03.950+0000 7f8d5dc2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:04.184+0000 7f0d05a49040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:04.235+0000 7f0d05a49040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:04.351+0000 7f0d05a49040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:04.226+0000 7f8d5dc2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:04.276+0000 7f8d5dc2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T03:09:04.426 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-04-13T03:09:04.394+0000 7f8d5dc2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T03:09:05.370 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:05.365+0000 7f0ca63d5640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-13T03:09:05.370 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-13T03:09:05.367+0000 7f0ca63d5640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-13T03:09:06.524 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-04-13T03:09:06.523+0000 7f3aa6814640 -1 osd.5 0 log_to_monitors true 2024-04-13T03:09:06.546 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-13T03:09:06.545+0000 7fd445f19640 -1 osd.1 0 log_to_monitors true 2024-04-13T03:09:06.565 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-04-13T03:09:06.564+0000 7f2bee489640 -1 osd.4 0 log_to_monitors true 2024-04-13T03:09:06.592 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-04-13T03:09:06.591+0000 7f1fb72ac640 -1 osd.6 0 log_to_monitors true 2024-04-13T03:09:06.595 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-04-13T03:09:06.594+0000 7f8442a0d640 -1 osd.7 0 log_to_monitors true 2024-04-13T03:09:06.596 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-13T03:09:06.595+0000 7f5508c1d640 -1 osd.0 0 log_to_monitors true 2024-04-13T03:09:06.610 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-13T03:09:06.609+0000 7f60cfa0f640 -1 osd.3 0 log_to_monitors true 2024-04-13T03:09:06.617 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-13T03:09:06.616+0000 7ff6d8a11640 -1 osd.2 0 log_to_monitors true 2024-04-13T03:09:10.398 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-13T03:09:10.397+0000 7f55043e4640 -1 osd.0 0 waiting for initial osdmap 2024-04-13T03:09:10.414 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-13T03:09:10.413+0000 7f54fc215640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.434 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-13T03:09:10.433+0000 7f60cbdf8640 -1 osd.3 0 waiting for initial osdmap 2024-04-13T03:09:10.445 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-13T03:09:10.444+0000 7f60c3480640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.447 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:10.477 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-04-13T03:09:10.473+0000 7f843e7f8640 -1 osd.7 0 waiting for initial osdmap 2024-04-13T03:09:10.480 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-13T03:09:10.479+0000 7ff6d4df8640 -1 osd.2 0 waiting for initial osdmap 2024-04-13T03:09:10.492 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-04-13T03:09:10.491+0000 7f8435fd6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.499 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-13T03:09:10.498+0000 7ff6cc480640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.500 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-04-13T03:09:10.499+0000 7f2bea1f7640 -1 osd.4 0 waiting for initial osdmap 2024-04-13T03:09:10.513 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-04-13T03:09:10.512+0000 7f2be2028640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.518 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-13T03:09:10.517+0000 7fd441bf7640 -1 osd.1 0 waiting for initial osdmap 2024-04-13T03:09:10.536 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-13T03:09:10.535+0000 7fd439a80640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.541 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.540+0000 7ff5f831e640 1 Processor -- start 2024-04-13T03:09:10.541 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.540+0000 7ff5f831e640 1 -- start start 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f831e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 0x7ff5f0169820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f831e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00d37e0 con 0x7ff5f00b8ca0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f831e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00d3ac0 con 0x7ff5f015eba0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f831e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f00d3da0 con 0x7ff5f006d010 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f58b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff5f00b8ca0 0x7ff5f006b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50226/0 (socket says 172.21.15.106:50226) 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f58b9640 1 -- 172.21.15.106:0/1493971798 learned_addr learned my addr 172.21.15.106:0/1493971798 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f68bb640 1 --2- 172.21.15.106:0/1493971798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 0x7ff5f0169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893053131 0 0) 0x7ff5f00d37e0 con 0x7ff5f00b8ca0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5dc003520 con 0x7ff5f00b8ca0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574893957 0 0) 0x7ff5f00d3ac0 con 0x7ff5f015eba0 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f68bb640 1 -- 172.21.15.106:0/1493971798 >> v1:172.21.15.114:6789/0 conn(0x7ff5f015eba0 legacy=0x7ff5f00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.543 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.541+0000 7ff5f68bb640 1 -- 172.21.15.106:0/1493971798 >> v1:172.21.15.106:6789/0 conn(0x7ff5f00b8ca0 legacy=0x7ff5f006b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.544 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.542+0000 7ff5f68bb640 1 -- 172.21.15.106:0/1493971798 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f00d4080 con 0x7ff5f006d010 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.542+0000 7ff5f68bb640 1 --2- 172.21.15.106:0/1493971798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 0x7ff5f0169820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009dc0 tx=0x7ff5e802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575bdf2143be0464 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.542+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8039070 con 0x7ff5f006d010 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.542+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e802ccd0 con 0x7ff5f006d010 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.542+0000 7ff5f50b8640 1 -- 172.21.15.106:0/1493971798 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8034660 con 0x7ff5f006d010 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 -- 172.21.15.106:0/1493971798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 msgr2=0x7ff5f0169820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 --2- 172.21.15.106:0/1493971798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 0x7ff5f0169820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009dc0 tx=0x7ff5e802bf20 comp rx=0 tx=0).stop 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 -- 172.21.15.106:0/1493971798 shutdown_connections 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 --2- 172.21.15.106:0/1493971798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f006d010 0x7ff5f0169820 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 -- 172.21.15.106:0/1493971798 >> 172.21.15.106:0/1493971798 conn(0x7ff5f00d47b0 msgr2=0x7ff5f00d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:10.545 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 -- 172.21.15.106:0/1493971798 shutdown_connections 2024-04-13T03:09:10.546 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.543+0000 7ff5f831e640 1 -- 172.21.15.106:0/1493971798 wait complete. 2024-04-13T03:09:10.546 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.544+0000 7ff5f831e640 1 Processor -- start 2024-04-13T03:09:10.546 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.544+0000 7ff5f831e640 1 -- start start 2024-04-13T03:09:10.547 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f006d010 0x7ff5f0161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:10.547 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f00b8ca0 0x7ff5f01f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f015f100 con 0x7ff5f015eba0 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f015ef80 con 0x7ff5f006d010 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f831e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f00cfee0 con 0x7ff5f00b8ca0 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41466/0 (socket says 172.21.15.106:41466) 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f60ba640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f006d010 0x7ff5f0161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 -- 172.21.15.106:0/3197272113 learned_addr learned my addr 172.21.15.106:0/3197272113 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f58b9640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f00b8ca0 0x7ff5f01f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 -- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f00b8ca0 msgr2=0x7ff5f01f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f00b8ca0 0x7ff5f01f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 -- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f006d010 msgr2=0x7ff5f0161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f006d010 0x7ff5f0161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.548 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 -- 172.21.15.106:0/3197272113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01f46e0 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5f68bb640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009ee0 tx=0x7ff5e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8039050 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.545+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e8045070 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.546+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f0160bf0 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.546+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8009450 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.546+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01611b0 con 0x7ff5f015eba0 2024-04-13T03:09:10.549 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.547+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f0075cc0 con 0x7ff5f015eba0 2024-04-13T03:09:10.550 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.547+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff5e803e850 con 0x7ff5f015eba0 2024-04-13T03:09:10.550 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.547+0000 7ff5e6ffd640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 0x7ff5d406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:10.550 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.547+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7ff5e80ad970 con 0x7ff5f015eba0 2024-04-13T03:09:10.550 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.547+0000 7ff5f60ba640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 0x7ff5d406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:10.550 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.548+0000 7ff5f60ba640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 0x7ff5d406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec000f60 tx=0x7ff5ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:10.553 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.552+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5e807c650 con 0x7ff5f015eba0 2024-04-13T03:09:10.557 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-04-13T03:09:10.556+0000 7f1fb37f8640 -1 osd.6 0 waiting for initial osdmap 2024-04-13T03:09:10.562 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-04-13T03:09:10.561+0000 7f3aa1df7640 -1 osd.5 0 waiting for initial osdmap 2024-04-13T03:09:10.568 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-04-13T03:09:10.567+0000 7f1faae28640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.575 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-04-13T03:09:10.574+0000 7f3a99c80640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T03:09:10.733 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.732+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5f00cf330 con 0x7ff5f015eba0 2024-04-13T03:09:10.734 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.733+0000 7ff5e6ffd640 1 -- 172.21.15.106:0/3197272113 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7ff5e8080950 con 0x7ff5f015eba0 2024-04-13T03:09:10.734 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-13T03:09:10.734 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":12,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:08.373739+0000","last_up_change":"0.000000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 msgr2=0x7ff5d406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 0x7ff5d406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec000f60 tx=0x7ff5ec009040 comp rx=0 tx=0).stop 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 msgr2=0x7ff5f006c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009ee0 tx=0x7ff5e8004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 shutdown_connections 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5d406c860 0x7ff5d406ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.736 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5f00b8ca0 0x7ff5f01f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.737 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5f015eba0 0x7ff5f006c060 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.737 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 --2- 172.21.15.106:0/3197272113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f006d010 0x7ff5f0161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:10.737 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 >> 172.21.15.106:0/3197272113 conn(0x7ff5f00d47b0 msgr2=0x7ff5f00b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:10.737 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 shutdown_connections 2024-04-13T03:09:10.737 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:10.735+0000 7ff5f831e640 1 -- 172.21.15.106:0/3197272113 wait complete. 2024-04-13T03:09:10.745 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-13T03:09:17.552 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:17.651 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.650+0000 7f0569e58640 1 Processor -- start 2024-04-13T03:09:17.651 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.650+0000 7f0569e58640 1 -- start start 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f0569e58640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 0x7f05640cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f0569e58640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05640cfd10 0x7f05640b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f0569e58640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05640d37e0 con 0x7f05640b81d0 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f0569e58640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f05640d3ac0 con 0x7f05640cfd10 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f0569e58640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f05640d3da0 con 0x7f056415eba0 2024-04-13T03:09:17.654 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05640cfd10 0x7f05640b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05640cfd10 0x7f05640b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43748/0 (socket says 172.21.15.106:43748) 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 -- 172.21.15.106:0/2956988272 learned_addr learned my addr 172.21.15.106:0/2956988272 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f05637fe640 1 --2- 172.21.15.106:0/2956988272 >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 0x7f05640cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 -- 172.21.15.106:0/2956988272 >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 msgr2=0x7f05640cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 --2- 172.21.15.106:0/2956988272 >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 0x7f05640cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 -- 172.21.15.106:0/2956988272 >> v1:172.21.15.114:6789/0 conn(0x7f05640b81d0 legacy=0x7f05640b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f055affd640 1 -- 172.21.15.106:0/2956988272 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05640d4080 con 0x7f05640cfd10 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.651+0000 7f05637fe640 1 --2- 172.21.15.106:0/2956988272 >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 0x7f05640cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.652+0000 7f055affd640 1 --2- 172.21.15.106:0/2956988272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f05640b7c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05640cb830 tx=0x7f054c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b2f2a38e1e9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.652+0000 7f0562ffd640 1 -- 172.21.15.106:0/2956988272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c035020 con 0x7f05640cfd10 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.652+0000 7f0562ffd640 1 -- 172.21.15.106:0/2956988272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c031870 con 0x7f05640cfd10 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.652+0000 7f0562ffd640 1 -- 172.21.15.106:0/2956988272 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c031c40 con 0x7f05640cfd10 2024-04-13T03:09:17.655 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 -- 172.21.15.106:0/2956988272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 msgr2=0x7f05640b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 --2- 172.21.15.106:0/2956988272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f05640b7c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05640cb830 tx=0x7f054c02b910 comp rx=0 tx=0).stop 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 -- 172.21.15.106:0/2956988272 shutdown_connections 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 --2- 172.21.15.106:0/2956988272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f05640b7c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 --2- 172.21.15.106:0/2956988272 >> v2:172.21.15.114:3300/0 conn(0x7f056415eba0 0x7f05640cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 -- 172.21.15.106:0/2956988272 >> 172.21.15.106:0/2956988272 conn(0x7f05640d47b0 msgr2=0x7f05640d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 -- 172.21.15.106:0/2956988272 shutdown_connections 2024-04-13T03:09:17.656 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.653+0000 7f0569e58640 1 -- 172.21.15.106:0/2956988272 wait complete. 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.654+0000 7f0569e58640 1 Processor -- start 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.654+0000 7f0569e58640 1 -- start start 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05640b81d0 0x7f056415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 0x7f056415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f056415f100 con 0x7f05640cfd10 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f056415ef80 con 0x7f056415eba0 2024-04-13T03:09:17.659 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0569e58640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f05640b8d10 con 0x7f05640b81d0 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43756/0 (socket says 172.21.15.106:43756) 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 -- 172.21.15.106:0/3607420362 learned_addr learned my addr 172.21.15.106:0/3607420362 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f05637fe640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05640b81d0 0x7f056415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0563fff640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 0x7f056415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 -- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05640b81d0 msgr2=0x7f056415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05640b81d0 0x7f056415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 -- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 msgr2=0x7f056415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 0x7f056415d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f055affd640 1 -- 172.21.15.106:0/3607420362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05641f8190 con 0x7f05640cfd10 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.655+0000 7f0563fff640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 0x7f056415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:17.660 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f055affd640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f054c000c00 tx=0x7f054c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c040070 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c03a420 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05641f8470 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c035040 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.656+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641f8a30 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.657+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0564075cc0 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.657+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f054c04b060 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.657+0000 7f0560ff9640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.658+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f054c0ad6c0 con 0x7f05640cfd10 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.658+0000 7f05637fe640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.661 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.658+0000 7f05637fe640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f055400aac0 tx=0x7f0554008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.663 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.662+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f054c07bed0 con 0x7f05640cfd10 2024-04-13T03:09:17.840 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.839+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0564067ee0 con 0x7f05640cfd10 2024-04-13T03:09:17.841 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.840+0000 7f0560ff9640 1 -- 172.21.15.106:0/3607420362 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f054c07b7d0 con 0x7f05640cfd10 2024-04-13T03:09:17.841 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-13T03:09:17.841 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":16,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:14.407710+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 msgr2=0x7f054406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f055400aac0 tx=0x7f0554008040 comp rx=0 tx=0).stop 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 msgr2=0x7f056415f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f054c000c00 tx=0x7f054c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 shutdown_connections 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056415eba0 0x7f056415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.846 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05640cfd10 0x7f056415f9e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.847 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 --2- 172.21.15.106:0/3607420362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05640b81d0 0x7f056415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.847 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 >> 172.21.15.106:0/3607420362 conn(0x7f05640d47b0 msgr2=0x7f056406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:17.847 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 shutdown_connections 2024-04-13T03:09:17.847 INFO:teuthology.misc.health.smithi106.stderr:2024-04-13T03:09:17.842+0000 7f0569e58640 1 -- 172.21.15.106:0/3607420362 wait complete. 2024-04-13T03:09:17.852 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-13T03:09:17.853 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-13T03:09:17.853 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:09:17.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:09:17.855 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:09:17.855 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:09:17.856 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:09:17.857 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:09:17.857 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:09:17.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:09:17.860 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:09:17.861 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:09:17.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:09:17.863 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:09:17.863 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-13T03:09:17.863 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:17.933 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:09:17.934 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:09:17.947 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:09:17.947 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:09:17.948 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:09:17.947+0000 7f977e7fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-04-13T03:09:17.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.969+0000 7f8e783e0640 1 Processor -- start 2024-04-13T03:09:17.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.969+0000 7f8e783e0640 1 -- start start 2024-04-13T03:09:17.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e783e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e700b6370 0x7f8e700cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e783e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e7015ed20 con 0x7f8e7015e940 2024-04-13T03:09:17.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e783e0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e700b6850 con 0x7f8e700cbdd0 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e783e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8e700cc240 con 0x7f8e700b6370 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7597b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e700b6370 0x7f8e700cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7617c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8e7015e940 0x7f8e700ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49514/0 (socket says 172.21.15.106:49514) 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7597b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e700b6370 0x7f8e700cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43778/0 (socket says 172.21.15.106:43778) 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7617c640 1 -- 172.21.15.106:0/3277491831 learned_addr learned my addr 172.21.15.106:0/3277491831 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7597b640 1 -- 172.21.15.106:0/3277491831 >> v1:172.21.15.114:6789/0 conn(0x7f8e700cbdd0 legacy=0x7f8e700d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7597b640 1 -- 172.21.15.106:0/3277491831 >> v1:172.21.15.106:6789/0 conn(0x7f8e7015e940 legacy=0x7f8e700ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.970+0000 7f8e7597b640 1 -- 172.21.15.106:0/3277491831 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e700b6b30 con 0x7f8e700b6370 2024-04-13T03:09:17.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.971+0000 7f8e7597b640 1 --2- 172.21.15.106:0/3277491831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700b6370 0x7f8e700cb890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e600097a0 tx=0x7f8e6002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9327805c03aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.971+0000 7f8e7517a640 1 -- 172.21.15.106:0/3277491831 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60037070 con 0x7f8e700b6370 2024-04-13T03:09:17.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.971+0000 7f8e7517a640 1 -- 172.21.15.106:0/3277491831 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e60030710 con 0x7f8e700b6370 2024-04-13T03:09:17.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.971+0000 7f8e7517a640 1 -- 172.21.15.106:0/3277491831 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60030ae0 con 0x7f8e700b6370 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 -- 172.21.15.106:0/3277491831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700b6370 msgr2=0x7f8e700cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 --2- 172.21.15.106:0/3277491831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700b6370 0x7f8e700cb890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e600097a0 tx=0x7f8e6002b710 comp rx=0 tx=0).stop 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 -- 172.21.15.106:0/3277491831 shutdown_connections 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 --2- 172.21.15.106:0/3277491831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700b6370 0x7f8e700cb890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 -- 172.21.15.106:0/3277491831 >> 172.21.15.106:0/3277491831 conn(0x7f8e700cf000 msgr2=0x7f8e700cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 -- 172.21.15.106:0/3277491831 shutdown_connections 2024-04-13T03:09:17.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.972+0000 7f8e783e0640 1 -- 172.21.15.106:0/3277491831 wait complete. 2024-04-13T03:09:17.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.973+0000 7f8e783e0640 1 Processor -- start 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.973+0000 7f8e783e0640 1 -- start start 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e700b6370 0x7f8e70155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e7015e940 0x7f8e7015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700b73d0 con 0x7f8e700cbdd0 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700b70f0 con 0x7f8e700b6370 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e783e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8e700d0860 con 0x7f8e7015e940 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7617c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e700b6370 0x7f8e70155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43784/0 (socket says 172.21.15.106:43784) 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 -- 172.21.15.106:0/1904990926 learned_addr learned my addr 172.21.15.106:0/1904990926 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:17.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7697d640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e7015e940 0x7f8e7015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 -- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e7015e940 msgr2=0x7f8e7015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e7015e940 0x7f8e7015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 -- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e700b6370 msgr2=0x7f8e70155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e700b6370 0x7f8e70155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 -- 172.21.15.106:0/1904990926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e701f5340 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.974+0000 7f8e7597b640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e600097a0 tx=0x7f8e600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.975+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60042070 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.975+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e60032a40 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.975+0000 7f8e783e0640 1 -- 172.21.15.106:0/1904990926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701f4d80 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.975+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60037040 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.975+0000 7f8e783e0640 1 -- 172.21.15.106:0/1904990926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7006f410 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.976+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e70069120 con 0x7f8e700cbdd0 2024-04-13T03:09:17.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.976+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e60045070 con 0x7f8e700cbdd0 2024-04-13T03:09:17.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.976+0000 7f8e66ffd640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 0x7f8e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:17.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.976+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8e600ad6b0 con 0x7f8e700cbdd0 2024-04-13T03:09:17.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.976+0000 7f8e7617c640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 0x7f8e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:17.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.977+0000 7f8e7617c640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 0x7f8e5406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0012e0 tx=0x7f8e6c0017a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:17.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:17.981+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e6007b990 con 0x7f8e700cbdd0 2024-04-13T03:09:18.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.155+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e70159630 con 0x7f8e700cbdd0 2024-04-13T03:09:18.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.156+0000 7f8e66ffd640 1 -- 172.21.15.106:0/1904990926 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f8e60080e20 con 0x7f8e700cbdd0 2024-04-13T03:09:18.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:18.157 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:14.407710+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:18.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 msgr2=0x7f8e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 0x7f8e5406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0012e0 tx=0x7f8e6c0017a0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 msgr2=0x7f8e701f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e600097a0 tx=0x7f8e600046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 shutdown_connections 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e5406c8f0 0x7f8e5406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e7015e940 0x7f8e7015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e700cbdd0 0x7f8e701f4360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 --2- 172.21.15.106:0/1904990926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e700b6370 0x7f8e70155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 >> 172.21.15.106:0/1904990926 conn(0x7f8e700cf000 msgr2=0x7f8e700cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 shutdown_connections 2024-04-13T03:09:18.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.158+0000 7f8e64ff9640 1 -- 172.21.15.106:0/1904990926 wait complete. 2024-04-13T03:09:18.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:09:18.170 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.272+0000 7f0909c88640 1 Processor -- start 2024-04-13T03:09:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.272+0000 7f0909c88640 1 -- start start 2024-04-13T03:09:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0909c88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040cd260 0x7f09040c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0909c88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09040d1c30 0x7f0904160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0909c88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09040d21d0 con 0x7f090415c5f0 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0909c88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09040cd8b0 con 0x7f09040d1c30 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0909c88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0904161330 con 0x7f09040cd260 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09040d1c30 0x7f0904160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09040d1c30 0x7f0904160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43792/0 (socket says 172.21.15.106:43792) 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 -- 172.21.15.106:0/2330354983 learned_addr learned my addr 172.21.15.106:0/2330354983 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0902ffd640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040cd260 0x7f09040c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 -- 172.21.15.106:0/2330354983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040cd260 msgr2=0x7f09040c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040cd260 0x7f09040c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 -- 172.21.15.106:0/2330354983 >> v1:172.21.15.114:6789/0 conn(0x7f090415c5f0 legacy=0x7f09040ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.273+0000 7f0903fff640 1 -- 172.21.15.106:0/2330354983 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040b1f40 con 0x7f09040d1c30 2024-04-13T03:09:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0903fff640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09040d1c30 0x7f0904160df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bc6de8a0b92862 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f09027fc640 1 -- 172.21.15.106:0/2330354983 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc02ca70 con 0x7f09040d1c30 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f09027fc640 1 -- 172.21.15.106:0/2330354983 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc02cc10 con 0x7f09040d1c30 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f09027fc640 1 -- 172.21.15.106:0/2330354983 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0343f0 con 0x7f09040d1c30 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 -- 172.21.15.106:0/2330354983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09040d1c30 msgr2=0x7f0904160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09040d1c30 0x7f0904160df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc02bed0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 -- 172.21.15.106:0/2330354983 shutdown_connections 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09040d1c30 0x7f0904160df0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 --2- 172.21.15.106:0/2330354983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040cd260 0x7f09040c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 -- 172.21.15.106:0/2330354983 >> 172.21.15.106:0/2330354983 conn(0x7f09040cef80 msgr2=0x7f09040cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 -- 172.21.15.106:0/2330354983 shutdown_connections 2024-04-13T03:09:18.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.274+0000 7f0909c88640 1 -- 172.21.15.106:0/2330354983 wait complete. 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 Processor -- start 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 -- start start 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09040cd260 0x7f090415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 0x7f09040cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09040b2500 con 0x7f090415c5f0 2024-04-13T03:09:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09040b2870 con 0x7f09040cd260 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.276+0000 7f0909c88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f090415c9d0 con 0x7f09040d1c30 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43794/0 (socket says 172.21.15.106:43794) 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 -- 172.21.15.106:0/2582269548 learned_addr learned my addr 172.21.15.106:0/2582269548 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f09037fe640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09040cd260 0x7f090415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0902ffd640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 0x7f09040cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 -- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 msgr2=0x7f09040cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 0x7f09040cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 -- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09040cd260 msgr2=0x7f090415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09040cd260 0x7f090415f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 -- 172.21.15.106:0/2582269548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040cb730 con 0x7f090415c5f0 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0902ffd640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 0x7f09040cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0903fff640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08fc036040 tx=0x7f08fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc042020 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc03e450 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09040cba10 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc03e840 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.277+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09040cbfd0 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.278+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0904068a10 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.278+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08fc045020 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.279+0000 7f08ebfff640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 0x7f08f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.279+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f08fc0adf30 con 0x7f090415c5f0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.279+0000 7f09037fe640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 0x7f08f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.279+0000 7f09037fe640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 0x7f08f006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006fd0 tx=0x7f08f8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.283+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08fc04c030 con 0x7f090415c5f0 2024-04-13T03:09:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.457+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0904160010 con 0x7f090415c5f0 2024-04-13T03:09:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.457+0000 7f08ebfff640 1 -- 172.21.15.106:0/2582269548 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f08fc07c6b0 con 0x7f090415c5f0 2024-04-13T03:09:18.458 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:18.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 msgr2=0x7f08f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 0x7f08f006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006fd0 tx=0x7f08f8009060 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 msgr2=0x7f09041e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08fc036040 tx=0x7f08fc004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 shutdown_connections 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f08f006c8f0 0x7f08f006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09040d1c30 0x7f09040cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090415c5f0 0x7f09041e6940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 --2- 172.21.15.106:0/2582269548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09040cd260 0x7f090415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 >> 172.21.15.106:0/2582269548 conn(0x7f09040cef80 msgr2=0x7f090406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 shutdown_connections 2024-04-13T03:09:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.459+0000 7f0909c88640 1 -- 172.21.15.106:0/2582269548 wait complete. 2024-04-13T03:09:18.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:18.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.568+0000 7f30f3df0640 1 Processor -- start 2024-04-13T03:09:18.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 -- start start 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30ec15ce00 0x7f30ec0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ec0b5e50 con 0x7f30ec0b1c70 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f30ec0b8310 con 0x7f30ec15ce00 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.569+0000 7f30f3df0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec0b5030 con 0x7f30ec0cd350 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30ec15ce00 0x7f30ec0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f138b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f30ec0b1c70 0x7f30ec0b5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49520/0 (socket says 172.21.15.106:49520) 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30ec15ce00 0x7f30ec0d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43806/0 (socket says 172.21.15.106:43806) 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f138b640 1 -- 172.21.15.106:0/2786616801 learned_addr learned my addr 172.21.15.106:0/2786616801 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f1b8c640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 -- 172.21.15.106:0/2786616801 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 msgr2=0x7f30ec0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 -- 172.21.15.106:0/2786616801 >> v1:172.21.15.106:6789/0 conn(0x7f30ec0b1c70 legacy=0x7f30ec0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f0b8a640 1 -- 172.21.15.106:0/2786616801 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334976396 0 0) 0x7f30dc001110 con 0x7f30ec0b1c70 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.570+0000 7f30f238d640 1 -- 172.21.15.106:0/2786616801 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec0b4d50 con 0x7f30ec15ce00 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f238d640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0d1520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4549af56bd7f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f0b8a640 1 -- 172.21.15.106:0/2786616801 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80315a0 con 0x7f30ec15ce00 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f0b8a640 1 -- 172.21.15.106:0/2786616801 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8037070 con 0x7f30ec15ce00 2024-04-13T03:09:18.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f0b8a640 1 -- 172.21.15.106:0/2786616801 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8031960 con 0x7f30ec15ce00 2024-04-13T03:09:18.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 -- 172.21.15.106:0/2786616801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 msgr2=0x7f30ec0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0d1520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e802b840 comp rx=0 tx=0).stop 2024-04-13T03:09:18.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 -- 172.21.15.106:0/2786616801 shutdown_connections 2024-04-13T03:09:18.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0d1520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 --2- 172.21.15.106:0/2786616801 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 -- 172.21.15.106:0/2786616801 >> 172.21.15.106:0/2786616801 conn(0x7f30ec0b4100 msgr2=0x7f30ec0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 -- 172.21.15.106:0/2786616801 shutdown_connections 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.571+0000 7f30f3df0640 1 -- 172.21.15.106:0/2786616801 wait complete. 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 Processor -- start 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 -- start start 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec0b1c70 0x7f30ec0b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0cdc70 con 0x7f30ec15ce00 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0d1000 con 0x7f30ec0b1c70 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f3df0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec0b7f30 con 0x7f30ec0cd350 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f238d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f238d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43820/0 (socket says 172.21.15.106:43820) 2024-04-13T03:09:18.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f238d640 1 -- 172.21.15.106:0/424438490 learned_addr learned my addr 172.21.15.106:0/424438490 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.573+0000 7f30f1b8c640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec0b1c70 0x7f30ec0b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f138b640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 -- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 msgr2=0x7f30ec0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 -- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec0b1c70 msgr2=0x7f30ec0b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec0b1c70 0x7f30ec0b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 -- 172.21.15.106:0/424438490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec0cb250 con 0x7f30ec15ce00 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f138b640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f238d640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30e80098c0 tx=0x7f30e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8032840 con 0x7f30ec15ce00 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8032a00 con 0x7f30ec15ce00 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec0cb530 con 0x7f30ec15ce00 2024-04-13T03:09:18.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8032df0 con 0x7f30ec15ce00 2024-04-13T03:09:18.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.574+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec0cbaf0 con 0x7f30ec15ce00 2024-04-13T03:09:18.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.575+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec069120 con 0x7f30ec15ce00 2024-04-13T03:09:18.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.575+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30e804c020 con 0x7f30ec15ce00 2024-04-13T03:09:18.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.576+0000 7f30e27fc640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:18.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.576+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f30e802f020 con 0x7f30ec15ce00 2024-04-13T03:09:18.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.576+0000 7f30f1b8c640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:18.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.576+0000 7f30f1b8c640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 0x7f30d006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30e4003b80 tx=0x7f30e40098a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:18.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.580+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30e8040070 con 0x7f30ec15ce00 2024-04-13T03:09:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.792+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30ec068140 con 0x7f30ec15ce00 2024-04-13T03:09:18.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.792+0000 7f30e27fc640 1 -- 172.21.15.106:0/424438490 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f30e807ba80 con 0x7f30ec15ce00 2024-04-13T03:09:18.793 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:18.793 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-13T03:09:18.794 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2024-04-13T03:09:18.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 msgr2=0x7f30d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 0x7f30d006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30e4003b80 tx=0x7f30e40098a0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 msgr2=0x7f30ec0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30e80098c0 tx=0x7f30e80046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 shutdown_connections 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30ec0cd350 0x7f30ec0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30ec15ce00 0x7f30ec0c96e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 --2- 172.21.15.106:0/424438490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec0b1c70 0x7f30ec0b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 >> 172.21.15.106:0/424438490 conn(0x7f30ec0b4100 msgr2=0x7f30ec0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 shutdown_connections 2024-04-13T03:09:18.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:18.794+0000 7f30f3df0640 1 -- 172.21.15.106:0/424438490 wait complete. 2024-04-13T03:09:20.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.916+0000 7fe0438de640 1 Processor -- start 2024-04-13T03:09:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.916+0000 7fe0438de640 1 -- start start 2024-04-13T03:09:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe0438de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe03c0b68d0 0x7fe03c0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe0438de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 0x7fe03c0d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe0438de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03c15eda0 con 0x7fe03c15e9c0 2024-04-13T03:09:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe0438de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe03c0d27f0 con 0x7fe03c0b68d0 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe0438de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe03c162c20 con 0x7fe03c1624b0 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe03c0b68d0 0x7fe03c0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe03c0b68d0 0x7fe03c0ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43826/0 (socket says 172.21.15.106:43826) 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 -- 172.21.15.106:0/3578634986 learned_addr learned my addr 172.21.15.106:0/3578634986 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe041e7b640 1 --2- 172.21.15.106:0/3578634986 >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 0x7fe03c0d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 -- 172.21.15.106:0/3578634986 >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 msgr2=0x7fe03c0d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 --2- 172.21.15.106:0/3578634986 >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 0x7fe03c0d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 -- 172.21.15.106:0/3578634986 >> v1:172.21.15.114:6789/0 conn(0x7fe03c15e9c0 legacy=0x7fe03c0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe040e79640 1 -- 172.21.15.106:0/3578634986 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c162f00 con 0x7fe03c0b68d0 2024-04-13T03:09:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.917+0000 7fe041e7b640 1 --2- 172.21.15.106:0/3578634986 >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 0x7fe03c0d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.918+0000 7fe040e79640 1 --2- 172.21.15.106:0/3578634986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ba290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe03c073be0 tx=0x7fe02c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda3829c9d1d3a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.918+0000 7fe033fff640 1 -- 172.21.15.106:0/3578634986 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c035020 con 0x7fe03c0b68d0 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.918+0000 7fe033fff640 1 -- 172.21.15.106:0/3578634986 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c031870 con 0x7fe03c0b68d0 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.918+0000 7fe033fff640 1 -- 172.21.15.106:0/3578634986 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c031c40 con 0x7fe03c0b68d0 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 -- 172.21.15.106:0/3578634986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c0b68d0 msgr2=0x7fe03c0ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 --2- 172.21.15.106:0/3578634986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ba290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe03c073be0 tx=0x7fe02c02b910 comp rx=0 tx=0).stop 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 -- 172.21.15.106:0/3578634986 shutdown_connections 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 --2- 172.21.15.106:0/3578634986 >> v2:172.21.15.114:3300/0 conn(0x7fe03c1624b0 0x7fe03c0d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 --2- 172.21.15.106:0/3578634986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ba290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 -- 172.21.15.106:0/3578634986 >> 172.21.15.106:0/3578634986 conn(0x7fe03c0d11c0 msgr2=0x7fe03c0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 -- 172.21.15.106:0/3578634986 shutdown_connections 2024-04-13T03:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.919+0000 7fe0438de640 1 -- 172.21.15.106:0/3578634986 wait complete. 2024-04-13T03:09:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.920+0000 7fe0438de640 1 Processor -- start 2024-04-13T03:09:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.920+0000 7fe0438de640 1 -- start start 2024-04-13T03:09:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe03c1624b0 0x7fe03c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c162940 con 0x7fe03c15e9c0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c0d2a40 con 0x7fe03c0b68d0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe03c0d34a0 con 0x7fe03c1624b0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43838/0 (socket says 172.21.15.106:43838) 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 -- 172.21.15.106:0/603160283 learned_addr learned my addr 172.21.15.106:0/603160283 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe04167a640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe041e7b640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe03c1624b0 0x7fe03c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 -- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe03c1624b0 msgr2=0x7fe03c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe03c1624b0 0x7fe03c0cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 -- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe03c0b68d0 msgr2=0x7fe03c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 -- 172.21.15.106:0/603160283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c1f53d0 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe040e79640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe02c02b950 tx=0x7fe02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c040070 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c03a420 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c035040 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.921+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c0c9470 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.922+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c0c9a30 con 0x7fe03c15e9c0 2024-04-13T03:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.923+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03c068a10 con 0x7fe03c15e9c0 2024-04-13T03:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.923+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe02c038020 con 0x7fe03c15e9c0 2024-04-13T03:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.923+0000 7fe031ffb640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 0x7fe02006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.923+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe02c0ad7f0 con 0x7fe03c15e9c0 2024-04-13T03:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.923+0000 7fe04167a640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 0x7fe02006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.924+0000 7fe04167a640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 0x7fe02006ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe038007fd0 tx=0x7fe038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:20.928+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe03c1f53d0 con 0x7fe03c15e9c0 2024-04-13T03:09:21.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.142+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe03c0c9180 con 0x7fe03c15e9c0 2024-04-13T03:09:21.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.142+0000 7fe031ffb640 1 -- 172.21.15.106:0/603160283 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7fe02c07bf70 con 0x7fe03c15e9c0 2024-04-13T03:09:21.144 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:21.144 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-13T03:09:21.145 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.144+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 msgr2=0x7fe02006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.144+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 0x7fe02006ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe038007fd0 tx=0x7fe038009040 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.144+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 msgr2=0x7fe03c1f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.144+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe02c02b950 tx=0x7fe02c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 shutdown_connections 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe02006c920 0x7fe02006ede0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe03c1624b0 0x7fe03c0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe03c15e9c0 0x7fe03c1f4e90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 --2- 172.21.15.106:0/603160283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe03c0b68d0 0x7fe03c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 >> 172.21.15.106:0/603160283 conn(0x7fe03c0d11c0 msgr2=0x7fe03c0b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 shutdown_connections 2024-04-13T03:09:21.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.145+0000 7fe0438de640 1 -- 172.21.15.106:0/603160283 wait complete. 2024-04-13T03:09:21.158 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:21.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.255+0000 7fb79311e640 1 Processor -- start 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.255+0000 7fb79311e640 1 -- start start 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb79311e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb78c0b3980 0x7fb78c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb79311e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c06bfe0 con 0x7fb78c0cefe0 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb79311e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c06c2c0 con 0x7fb78c0b7810 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb79311e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c0cfda0 con 0x7fb78c0b3980 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb790eba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb78c0cefe0 0x7fb78c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49522/0 (socket says 172.21.15.106:49522) 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb790eba640 1 -- 172.21.15.106:0/22672529 learned_addr learned my addr 172.21.15.106:0/22672529 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.256+0000 7fb78bfff640 1 --2- 172.21.15.106:0/22672529 >> v2:172.21.15.114:3300/0 conn(0x7fb78c0b3980 0x7fb78c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554698980 0 0) 0x7fb78c06bfe0 con 0x7fb78c0cefe0 2024-04-13T03:09:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb774003520 con 0x7fb78c0cefe0 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083950628 0 0) 0x7fb774003520 con 0x7fb78c0cefe0 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 >> v2:172.21.15.114:3300/0 conn(0x7fb78c0b3980 msgr2=0x7fb78c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/22672529 >> v2:172.21.15.114:3300/0 conn(0x7fb78c0b3980 0x7fb78c06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 >> v1:172.21.15.114:6789/0 conn(0x7fb78c0b7810 legacy=0x7fb78c0b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c06c5a0 con 0x7fb78c0cefe0 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb77c001670 con 0x7fb78c0cefe0 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 >> v1:172.21.15.106:6789/0 conn(0x7fb78c0cefe0 legacy=0x7fb78c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7740051c0 0x7fb77400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb77400c030 con 0x7fb774004800 2024-04-13T03:09:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb77400c1e0 con 0x7fb7740051c0 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb77400c3e0 con 0x7fb7740039b0 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb790eba640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb7916bb640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7740051c0 0x7fb77400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 -- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb7740039b0 msgr2=0x7fb774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 -- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7740051c0 msgr2=0x7fb77400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7740051c0 0x7fb77400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb78bfff640 1 -- 172.21.15.106:0/22672529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774003520 con 0x7fb774004800 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb7916bb640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7740051c0 0x7fb77400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.257+0000 7fb7916bb640 1 -- 172.21.15.106:0/22672529 reap_dead start 2024-04-13T03:09:21.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.258+0000 7fb78bfff640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb774004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ac0 tx=0x7fb774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15eaabfeebd98cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.258+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780039070 con 0x7fb774004800 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.258+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78002c6b0 con 0x7fb774004800 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.258+0000 7fb78b7fe640 1 -- 172.21.15.106:0/22672529 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780009690 con 0x7fb774004800 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.258+0000 7fb79311e640 1 -- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 msgr2=0x7fb774004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb774004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ac0 tx=0x7fb774005830 comp rx=0 tx=0).stop 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 -- 172.21.15.106:0/22672529 shutdown_connections 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 --2- 172.21.15.106:0/22672529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb774004c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 -- 172.21.15.106:0/22672529 >> 172.21.15.106:0/22672529 conn(0x7fb78c0d4780 msgr2=0x7fb78c0d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 -- 172.21.15.106:0/22672529 shutdown_connections 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.259+0000 7fb79311e640 1 -- 172.21.15.106:0/22672529 wait complete. 2024-04-13T03:09:21.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.260+0000 7fb79311e640 1 Processor -- start 2024-04-13T03:09:21.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.260+0000 7fb79311e640 1 -- start start 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c14ab80 0x7fb78c0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c0d2dc0 0x7fb78c1f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c06d080 con 0x7fb774004800 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c06cb60 con 0x7fb78c14ab80 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c15f0d0 con 0x7fb78c0d2dc0 2024-04-13T03:09:21.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43860/0 (socket says 172.21.15.106:43860) 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 -- 172.21.15.106:0/2150805214 learned_addr learned my addr 172.21.15.106:0/2150805214 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 -- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c0d2dc0 msgr2=0x7fb78c1f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c0d2dc0 0x7fb78c1f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 -- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c14ab80 msgr2=0x7fb78c0d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb78bfff640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c14ab80 0x7fb78c0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c14ab80 0x7fb78c0d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 -- 172.21.15.106:0/2150805214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c154d80 con 0x7fb774004800 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb790eba640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00f8e0 con 0x7fb774004800 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c018070 con 0x7fb774004800 2024-04-13T03:09:21.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c155060 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00fca0 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.261+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c155620 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.262+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78c068a10 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.263+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c01d460 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.263+0000 7fb7897fa640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.263+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb77c08b8c0 con 0x7fb774004800 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.263+0000 7fb78bfff640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.263+0000 7fb78bfff640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 0x7fb77806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb780036000 tx=0x7fb780030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.267+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb77c05a0d0 con 0x7fb774004800 2024-04-13T03:09:21.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.443+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb78c0d31e0 con 0x7fb774004800 2024-04-13T03:09:21.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.444+0000 7fb7897fa640 1 -- 172.21.15.106:0/2150805214 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fb77c0599d0 con 0x7fb774004800 2024-04-13T03:09:21.446 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:21.446 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:14.407710+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:21.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 msgr2=0x7fb77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 0x7fb77806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb780036000 tx=0x7fb780030000 comp rx=0 tx=0).stop 2024-04-13T03:09:21.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 msgr2=0x7fb78c15eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007720 comp rx=0 tx=0).stop 2024-04-13T03:09:21.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 shutdown_connections 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c0d2dc0 0x7fb78c1f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c14ab80 0x7fb78c0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 --2- 172.21.15.106:0/2150805214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb774004800 0x7fb78c15eb90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 >> 172.21.15.106:0/2150805214 conn(0x7fb78c0d4780 msgr2=0x7fb78c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 shutdown_connections 2024-04-13T03:09:21.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.446+0000 7fb79311e640 1 -- 172.21.15.106:0/2150805214 wait complete. 2024-04-13T03:09:21.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:09:21.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:21.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.561+0000 7fa57e9dc640 1 Processor -- start 2024-04-13T03:09:21.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.561+0000 7fa57e9dc640 1 -- start start 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57e9dc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780cd260 0x7fa5780c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57e9dc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa5780d1c30 0x7fa578160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57e9dc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5780d21d0 con 0x7fa57815c5f0 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57e9dc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa5780cd8b0 con 0x7fa5780d1c30 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57e9dc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa578161330 con 0x7fa5780cd260 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57cf79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa5780d1c30 0x7fa578160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57cf79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa5780d1c30 0x7fa578160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43866/0 (socket says 172.21.15.106:43866) 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57cf79640 1 -- 172.21.15.106:0/3774200001 learned_addr learned my addr 172.21.15.106:0/3774200001 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa5777fe640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780cd260 0x7fa5780c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57cf79640 1 -- 172.21.15.106:0/3774200001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780cd260 msgr2=0x7fa5780c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.562+0000 7fa57cf79640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780cd260 0x7fa5780c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa57cf79640 1 -- 172.21.15.106:0/3774200001 >> v1:172.21.15.114:6789/0 conn(0x7fa57815c5f0 legacy=0x7fa5780ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa57cf79640 1 -- 172.21.15.106:0/3774200001 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5780b1f40 con 0x7fa5780d1c30 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa57cf79640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780d1c30 0x7fa578160df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa570009e50 tx=0x7fa57002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54226236ea33e5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa576ffd640 1 -- 172.21.15.106:0/3774200001 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57002cb00 con 0x7fa5780d1c30 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa576ffd640 1 -- 172.21.15.106:0/3774200001 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57002cca0 con 0x7fa5780d1c30 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.563+0000 7fa576ffd640 1 -- 172.21.15.106:0/3774200001 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570034610 con 0x7fa5780d1c30 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3774200001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780d1c30 msgr2=0x7fa578160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780d1c30 0x7fa578160df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa570009e50 tx=0x7fa57002bf60 comp rx=0 tx=0).stop 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3774200001 shutdown_connections 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780d1c30 0x7fa578160df0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3774200001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780cd260 0x7fa5780c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3774200001 >> 172.21.15.106:0/3774200001 conn(0x7fa5780cef80 msgr2=0x7fa5780cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3774200001 shutdown_connections 2024-04-13T03:09:21.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.564+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3774200001 wait complete. 2024-04-13T03:09:21.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.565+0000 7fa57e9dc640 1 Processor -- start 2024-04-13T03:09:21.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.565+0000 7fa57e9dc640 1 -- start start 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 0x7fa5780c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa57815c5f0 0x7fa5780c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5780b2500 con 0x7fa5780cd260 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa5780b2870 con 0x7fa57815c5f0 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57e9dc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa57815c9d0 con 0x7fa5780d1c30 2024-04-13T03:09:21.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43868/0 (socket says 172.21.15.106:43868) 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 -- 172.21.15.106:0/3258342078 learned_addr learned my addr 172.21.15.106:0/3258342078 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa57cf79640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa57815c5f0 0x7fa5780c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa5777fe640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 0x7fa5780c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 -- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 msgr2=0x7fa5780c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 0x7fa5780c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 -- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa57815c5f0 msgr2=0x7fa5780c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa57815c5f0 0x7fa5780c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa577fff640 1 -- 172.21.15.106:0/3258342078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5780b6140 con 0x7fa5780cd260 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.566+0000 7fa5777fe640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 0x7fa5780c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa577fff640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c3c0 tx=0x7fa56800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5680052d0 con 0x7fa5780cd260 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa568005490 con 0x7fa5780cd260 2024-04-13T03:09:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5780b6420 con 0x7fa5780cd260 2024-04-13T03:09:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568007260 con 0x7fa5780cd260 2024-04-13T03:09:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.567+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5780b69e0 con 0x7fa5780cd260 2024-04-13T03:09:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.568+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578068a10 con 0x7fa5780cd260 2024-04-13T03:09:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.568+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa568007400 con 0x7fa5780cd260 2024-04-13T03:09:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.569+0000 7fa574ff9640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 0x7fa56406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.569+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa56808ac10 con 0x7fa5780cd260 2024-04-13T03:09:21.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.569+0000 7fa5777fe640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 0x7fa56406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.569+0000 7fa5777fe640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 0x7fa56406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5780b6f70 tx=0x7fa56c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.573+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa568058d40 con 0x7fa5780cd260 2024-04-13T03:09:21.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.748+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5780c8850 con 0x7fa5780cd260 2024-04-13T03:09:21.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.748+0000 7fa574ff9640 1 -- 172.21.15.106:0/3258342078 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa56805d040 con 0x7fa5780cd260 2024-04-13T03:09:21.751 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.750+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 msgr2=0x7fa56406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.750+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 0x7fa56406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5780b6f70 tx=0x7fa56c008040 comp rx=0 tx=0).stop 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.750+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 msgr2=0x7fa578157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.750+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c3c0 tx=0x7fa56800c880 comp rx=0 tx=0).stop 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 shutdown_connections 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa56406c8f0 0x7fa56406edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5780d1c30 0x7fa5780c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa57815c5f0 0x7fa5780c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 --2- 172.21.15.106:0/3258342078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5780cd260 0x7fa578157830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 >> 172.21.15.106:0/3258342078 conn(0x7fa5780cef80 msgr2=0x7fa57815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 shutdown_connections 2024-04-13T03:09:21.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.751+0000 7fa57e9dc640 1 -- 172.21.15.106:0/3258342078 wait complete. 2024-04-13T03:09:21.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:09:21.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 Processor -- start 2024-04-13T03:09:21.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 -- start start 2024-04-13T03:09:21.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01d015c7d0 0x7f01d0160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d00c60a0 0x7f01d00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00cd550 con 0x7f01d00c60a0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00d2310 con 0x7f01d015c7d0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01d5db4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00b1f30 con 0x7f01d00ccba0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01ceffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d00c60a0 0x7f01d00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01ceffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d00c60a0 0x7f01d00d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43878/0 (socket says 172.21.15.106:43878) 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.861+0000 7f01ceffd640 1 -- 172.21.15.106:0/2466333813 learned_addr learned my addr 172.21.15.106:0/2466333813 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01cffff640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/2466333813 >> v2:172.21.15.114:3300/0 conn(0x7f01d015c7d0 0x7f01d0160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 -- 172.21.15.106:0/2466333813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 msgr2=0x7f01d00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 -- 172.21.15.106:0/2466333813 >> v2:172.21.15.114:3300/0 conn(0x7f01d015c7d0 msgr2=0x7f01d0160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 --2- 172.21.15.106:0/2466333813 >> v2:172.21.15.114:3300/0 conn(0x7f01d015c7d0 0x7f01d0160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 -- 172.21.15.106:0/2466333813 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00b2210 con 0x7f01d00c60a0 2024-04-13T03:09:21.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.862+0000 7f01ceffd640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d00c60a0 0x7f01d00d1c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01d0072aa0 tx=0x7f01c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e37d8d0704deab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01ce7fc640 1 -- 172.21.15.106:0/2466333813 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4035020 con 0x7f01d00c60a0 2024-04-13T03:09:21.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01ce7fc640 1 -- 172.21.15.106:0/2466333813 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4031870 con 0x7f01d00c60a0 2024-04-13T03:09:21.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01ce7fc640 1 -- 172.21.15.106:0/2466333813 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031c40 con 0x7f01d00c60a0 2024-04-13T03:09:21.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 -- 172.21.15.106:0/2466333813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d00c60a0 msgr2=0x7f01d00d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d00c60a0 0x7f01d00d1c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01d0072aa0 tx=0x7f01c402b910 comp rx=0 tx=0).stop 2024-04-13T03:09:21.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 -- 172.21.15.106:0/2466333813 shutdown_connections 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2466333813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d00c60a0 0x7f01d00d1c10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2466333813 >> v2:172.21.15.114:3300/0 conn(0x7f01d015c7d0 0x7f01d0160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 -- 172.21.15.106:0/2466333813 >> 172.21.15.106:0/2466333813 conn(0x7f01d00d2550 msgr2=0x7f01d00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 -- 172.21.15.106:0/2466333813 shutdown_connections 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.863+0000 7f01d5db4640 1 -- 172.21.15.106:0/2466333813 wait complete. 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.864+0000 7f01d5db4640 1 Processor -- start 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 -- start start 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 0x7f01d00b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01d015cbb0 con 0x7f01d015c7d0 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00b27d0 con 0x7f01d00c60a0 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01d5db4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00b80a0 con 0x7f01d00ccba0 2024-04-13T03:09:21.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43888/0 (socket says 172.21.15.106:43888) 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01cffff640 1 -- 172.21.15.106:0/2315610148 learned_addr learned my addr 172.21.15.106:0/2315610148 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.865+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 0x7f01d00b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01ceffd640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 -- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 msgr2=0x7f01d00b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 -- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 msgr2=0x7f01d00b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 0x7f01d00b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 -- 172.21.15.106:0/2315610148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00cb120 con 0x7f01d015c7d0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 0x7f01d00b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01ceffd640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01cffff640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01c800ef00 tx=0x7f01c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8018020 con 0x7f01d015c7d0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8014400 con 0x7f01d015c7d0 2024-04-13T03:09:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d00cb400 con 0x7f01d015c7d0 2024-04-13T03:09:21.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80147f0 con 0x7f01d015c7d0 2024-04-13T03:09:21.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.866+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00cb9c0 con 0x7f01d015c7d0 2024-04-13T03:09:21.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.867+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d0069120 con 0x7f01d015c7d0 2024-04-13T03:09:21.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.867+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01c8005d10 con 0x7f01d015c7d0 2024-04-13T03:09:21.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.868+0000 7f01b7fff640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 0x7f01bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:21.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.868+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 0x7f01bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:21.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.868+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f01c808bf10 con 0x7f01d015c7d0 2024-04-13T03:09:21.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.868+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 0x7f01bc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009af0 tx=0x7f01c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:21.872+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01c8022070 con 0x7f01d015c7d0 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.052+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f01d00d31b0 con 0x7f01d015c7d0 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.052+0000 7f01b7fff640 1 -- 172.21.15.106:0/2315610148 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f01c805a690 con 0x7f01d015c7d0 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:09:22.054 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:09:22.055 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 msgr2=0x7f01bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 0x7f01bc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009af0 tx=0x7f01c0008040 comp rx=0 tx=0).stop 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 msgr2=0x7f01d01f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01c800ef00 tx=0x7f01c800b480 comp rx=0 tx=0).stop 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 shutdown_connections 2024-04-13T03:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01bc06c8f0 0x7f01bc06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00ccba0 0x7f01d00b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d015c7d0 0x7f01d01f3a50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 --2- 172.21.15.106:0/2315610148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00c60a0 0x7f01d00b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 >> 172.21.15.106:0/2315610148 conn(0x7f01d00d2550 msgr2=0x7f01d00704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 shutdown_connections 2024-04-13T03:09:22.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.054+0000 7f01d5db4640 1 -- 172.21.15.106:0/2315610148 wait complete. 2024-04-13T03:09:22.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:09:22.066 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:09:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.169+0000 7fe3a87b4640 1 Processor -- start 2024-04-13T03:09:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.169+0000 7fe3a87b4640 1 -- start start 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a87b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe3a0162490 0x7fe3a0162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a87b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a00d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a87b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a00ba6e0 con 0x7fe3a015e8a0 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a87b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a0163310 con 0x7fe3a0162490 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a87b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a00d2b50 con 0x7fe3a00d2fe0 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe3a0162490 0x7fe3a0162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe3a0162490 0x7fe3a0162920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43890/0 (socket says 172.21.15.106:43890) 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a6550640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe3a015e8a0 0x7fe3a00b9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49538/0 (socket says 172.21.15.106:49538) 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 -- 172.21.15.106:0/3886391365 learned_addr learned my addr 172.21.15.106:0/3886391365 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a6d51640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a00d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 -- 172.21.15.106:0/3886391365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 msgr2=0x7fe3a00d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a00d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 -- 172.21.15.106:0/3886391365 >> v1:172.21.15.106:6789/0 conn(0x7fe3a015e8a0 legacy=0x7fe3a00b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a5d4f640 1 -- 172.21.15.106:0/3886391365 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a006cbd0 con 0x7fe3a0162490 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a554e640 1 -- 172.21.15.106:0/3886391365 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417426006 0 0) 0x7fe39c001110 con 0x7fe3a015e8a0 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.170+0000 7fe3a6d51640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a00d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a5d4f640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a0162920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe390009870 tx=0x7fe39002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a57849ac629bd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a554e640 1 -- 172.21.15.106:0/3886391365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390034070 con 0x7fe3a0162490 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a554e640 1 -- 172.21.15.106:0/3886391365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe390030840 con 0x7fe3a0162490 2024-04-13T03:09:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a554e640 1 -- 172.21.15.106:0/3886391365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390030c10 con 0x7fe3a0162490 2024-04-13T03:09:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 -- 172.21.15.106:0/3886391365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 msgr2=0x7fe3a0162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a0162920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe390009870 tx=0x7fe39002b980 comp rx=0 tx=0).stop 2024-04-13T03:09:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 -- 172.21.15.106:0/3886391365 shutdown_connections 2024-04-13T03:09:22.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a00d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/3886391365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a0162920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 -- 172.21.15.106:0/3886391365 >> 172.21.15.106:0/3886391365 conn(0x7fe3a00b6210 msgr2=0x7fe3a00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.171+0000 7fe3a87b4640 1 -- 172.21.15.106:0/3886391365 shutdown_connections 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.172+0000 7fe3a87b4640 1 -- 172.21.15.106:0/3886391365 wait complete. 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 Processor -- start 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 -- start start 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a014ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3a015e8a0 0x7fe3a01e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a00d27e0 con 0x7fe3a0162490 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a006d190 con 0x7fe3a015e8a0 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a87b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a006c8f0 con 0x7fe3a00d2fe0 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a6d51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a6d51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43904/0 (socket says 172.21.15.106:43904) 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.173+0000 7fe3a6d51640 1 -- 172.21.15.106:0/558354377 learned_addr learned my addr 172.21.15.106:0/558354377 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a5d4f640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3a015e8a0 0x7fe3a01e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6550640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a014ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 -- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 msgr2=0x7fe3a014ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a014ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 -- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3a015e8a0 msgr2=0x7fe3a01e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3a015e8a0 0x7fe3a01e9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 -- 172.21.15.106:0/558354377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a00cd8c0 con 0x7fe3a0162490 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a6d51640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe39800cf20 tx=0x7fe39800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39801a070 con 0x7fe3a0162490 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39800d840 con 0x7fe3a0162490 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a00cdba0 con 0x7fe3a0162490 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800dc30 con 0x7fe3a0162490 2024-04-13T03:09:22.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.174+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a00ce160 con 0x7fe3a0162490 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.175+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a0074760 con 0x7fe3a0162490 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.175+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe39801d070 con 0x7fe3a0162490 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.176+0000 7fe396ffd640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 0x7fe38406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.176+0000 7fe3a6550640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 0x7fe38406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.176+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe39808c080 con 0x7fe3a0162490 2024-04-13T03:09:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.176+0000 7fe3a6550640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 0x7fe38406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000f60 tx=0x7fe39c007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:22.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.180+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe39805a800 con 0x7fe3a0162490 2024-04-13T03:09:22.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.357+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe3a0068a10 con 0x7fe3a0162490 2024-04-13T03:09:22.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.949+0000 7fe396ffd640 1 -- 172.21.15.106:0/558354377 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fe39805a100 con 0x7fe3a0162490 2024-04-13T03:09:22.951 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:09:22.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 msgr2=0x7fe38406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 0x7fe38406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000f60 tx=0x7fe39c007970 comp rx=0 tx=0).stop 2024-04-13T03:09:22.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 msgr2=0x7fe3a00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe39800cf20 tx=0x7fe39800a7d0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 shutdown_connections 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe38406c8f0 0x7fe38406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3a0162490 0x7fe3a00b8990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3a015e8a0 0x7fe3a01e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 --2- 172.21.15.106:0/558354377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3a00d2fe0 0x7fe3a014ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 >> 172.21.15.106:0/558354377 conn(0x7fe3a00b6210 msgr2=0x7fe3a00b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 shutdown_connections 2024-04-13T03:09:22.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:22.952+0000 7fe3a87b4640 1 -- 172.21.15.106:0/558354377 wait complete. 2024-04-13T03:09:22.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-13T03:09:23.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.073+0000 7fb304688640 1 Processor -- start 2024-04-13T03:09:23.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.073+0000 7fb304688640 1 -- start start 2024-04-13T03:09:23.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb304688640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2fc0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb304688640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2fc0cfe70 con 0x7fb2fc162550 2024-04-13T03:09:23.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb304688640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2fc15ee60 con 0x7fb2fc0d1630 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb304688640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc162af0 con 0x7fb2fc15ea80 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb301c23640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb2fc162550 0x7fb2fc0b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54724/0 (socket says 172.21.15.106:54724) 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb301c23640 1 -- 172.21.15.106:0/1498877428 learned_addr learned my addr 172.21.15.106:0/1498877428 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb302424640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2fc0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111066331 0 0) 0x7fb2fc0cfe70 con 0x7fb2fc162550 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.074+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2e8003520 con 0x7fb2fc162550 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494663868 0 0) 0x7fb2fc15ee60 con 0x7fb2fc0d1630 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2fc0cfe70 con 0x7fb2fc0d1630 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1213447629 0 0) 0x7fb2e8003520 con 0x7fb2fc162550 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 msgr2=0x7fb2fc0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:23.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2fc0d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 >> v1:172.21.15.114:6789/0 conn(0x7fb2fc0d1630 legacy=0x7fb2fc0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc162dd0 con 0x7fb2fc162550 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb2ec0015f0 con 0x7fb2fc162550 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 >> v1:172.21.15.106:6789/0 conn(0x7fb2fc162550 legacy=0x7fb2fc0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2e8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8004120 0x7fb2e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004d40 0x7fb2e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc162af0 con 0x7fb2e8004120 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e800bb00 con 0x7fb2e8004d40 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e800bd00 con 0x7fb2fc15ea80 2024-04-13T03:09:23.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8004120 0x7fb2e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb302424640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2e8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb302c25640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004d40 0x7fb2e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 -- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 msgr2=0x7fb2e8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc15ea80 0x7fb2e8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 -- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004d40 msgr2=0x7fb2e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004d40 0x7fb2e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.075+0000 7fb301c23640 1 -- 172.21.15.106:0/1498877428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e8003520 con 0x7fb2e8004120 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.076+0000 7fb301c23640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8004120 0x7fb2e8004800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec007f60 tx=0x7fb2e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a952deecda0a30bc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.076+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec007b80 con 0x7fb2e8004120 2024-04-13T03:09:23.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.076+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ec030070 con 0x7fb2e8004120 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.076+0000 7fb301422640 1 -- 172.21.15.106:0/1498877428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec005e50 con 0x7fb2e8004120 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 -- 172.21.15.106:0/1498877428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8004120 msgr2=0x7fb2e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 --2- 172.21.15.106:0/1498877428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8004120 0x7fb2e8004800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec007f60 tx=0x7fb2e8005830 comp rx=0 tx=0).stop 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb302c25640 1 -- 172.21.15.106:0/1498877428 reap_dead start 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 -- 172.21.15.106:0/1498877428 shutdown_connections 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 -- 172.21.15.106:0/1498877428 >> 172.21.15.106:0/1498877428 conn(0x7fb2fc09da50 msgr2=0x7fb2fc0b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 -- 172.21.15.106:0/1498877428 shutdown_connections 2024-04-13T03:09:23.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.077+0000 7fb304688640 1 -- 172.21.15.106:0/1498877428 wait complete. 2024-04-13T03:09:23.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.078+0000 7fb304688640 1 Processor -- start 2024-04-13T03:09:23.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.078+0000 7fb304688640 1 -- start start 2024-04-13T03:09:23.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.078+0000 7fb304688640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004120 0x7fb2fc161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 0x7fb2fc0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc0d22b0 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc163390 con 0x7fb2e8004120 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc0d2430 con 0x7fb2fc0cd830 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51422/0 (socket says 172.21.15.106:51422) 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 -- 172.21.15.106:0/2966041688 learned_addr learned my addr 172.21.15.106:0/2966041688 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb301c23640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 0x7fb2fc0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302424640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004120 0x7fb2fc161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 -- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 msgr2=0x7fb2fc0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 0x7fb2fc0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 -- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004120 msgr2=0x7fb2fc161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004120 0x7fb2fc161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 -- 172.21.15.106:0/2966041688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc160760 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb301c23640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 0x7fb2fc0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb302c25640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0d5320 tx=0x7fb2f400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4019070 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f4006e20 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc15f9a0 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4003b90 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.079+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc15ff60 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.080+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc068a10 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.081+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2f401c070 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.081+0000 7fb2f2ffd640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 0x7fb2e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.081+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fb2f408ac50 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.081+0000 7fb302424640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 0x7fb2e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.082+0000 7fb302424640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 0x7fb2e006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0c8030 tx=0x7fb2f8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.086+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2f4090020 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.262+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb2fc0d5350 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:09:23.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:09:23.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.954+0000 7fb2f2ffd640 1 -- 172.21.15.106:0/2966041688 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fb2f40592f0 con 0x7fb2fc0cdcb0 2024-04-13T03:09:23.956 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 msgr2=0x7fb2e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 0x7fb2e006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0c8030 tx=0x7fb2f8009190 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 msgr2=0x7fb2fc0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0d5320 tx=0x7fb2f400feb0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 shutdown_connections 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2e006c920 0x7fb2e006ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2fc0cd830 0x7fb2fc0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc0cdcb0 0x7fb2fc0cbcf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 --2- 172.21.15.106:0/2966041688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e8004120 0x7fb2fc161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.959+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 >> 172.21.15.106:0/2966041688 conn(0x7fb2fc09da50 msgr2=0x7fb2fc0d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:23.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.961+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 shutdown_connections 2024-04-13T03:09:23.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:23.961+0000 7fb304688640 1 -- 172.21.15.106:0/2966041688 wait complete. 2024-04-13T03:09:23.984 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:09:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 Processor -- start 2024-04-13T03:09:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 -- start start 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f229c0b7f00 0x7f229c0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 0x7f229c0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0cd4b0 con 0x7f229c15c820 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f229c0706d0 con 0x7f229c0b7f00 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.126+0000 7f22a10de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f229c0709b0 con 0x7f229c0cd0a0 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f229c0b7f00 0x7f229c0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f229b59e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 0x7f229c0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f229c0b7f00 0x7f229c0ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:41538/0 (socket says 172.21.15.106:41538) 2024-04-13T03:09:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 -- 172.21.15.106:0/2199061274 learned_addr learned my addr 172.21.15.106:0/2199061274 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 -- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 msgr2=0x7f229c0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 0x7f229c0d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 -- 172.21.15.106:0/2199061274 >> v1:172.21.15.114:6789/0 conn(0x7f229c15c820 legacy=0x7f229c06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f2293fff640 1 -- 172.21.15.106:0/2199061274 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c070c90 con 0x7f229c0b7f00 2024-04-13T03:09:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.127+0000 7f229b59e640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 0x7f229c0d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.128+0000 7f2293fff640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c0ccb60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2284009870 tx=0x7f228402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fc467d7d0161aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.128+0000 7f229a59c640 1 -- 172.21.15.106:0/2199061274 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284034070 con 0x7f229c0b7f00 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.128+0000 7f229a59c640 1 -- 172.21.15.106:0/2199061274 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2284030840 con 0x7f229c0b7f00 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.128+0000 7f229a59c640 1 -- 172.21.15.106:0/2199061274 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284030c10 con 0x7f229c0b7f00 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 -- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 msgr2=0x7f229c0ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c0ccb60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2284009870 tx=0x7f228402b980 comp rx=0 tx=0).stop 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 -- 172.21.15.106:0/2199061274 shutdown_connections 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c0cd0a0 0x7f229c0d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 --2- 172.21.15.106:0/2199061274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c0ccb60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 -- 172.21.15.106:0/2199061274 >> 172.21.15.106:0/2199061274 conn(0x7f229c0d2510 msgr2=0x7f229c0d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 -- 172.21.15.106:0/2199061274 shutdown_connections 2024-04-13T03:09:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.129+0000 7f22a10de640 1 -- 172.21.15.106:0/2199061274 wait complete. 2024-04-13T03:09:24.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.130+0000 7f22a10de640 1 Processor -- start 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.130+0000 7f22a10de640 1 -- start start 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c15c820 0x7f229c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0d0e20 con 0x7f229c0cd0a0 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0d03c0 con 0x7f229c0b7f00 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f22a10de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f229c0d0fa0 con 0x7f229c15c820 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51438/0 (socket says 172.21.15.106:51438) 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 -- 172.21.15.106:0/2985698037 learned_addr learned my addr 172.21.15.106:0/2985698037 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f229b59e640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c15c820 0x7f229c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f229ad9d640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 -- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c15c820 msgr2=0x7f229c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c15c820 0x7f229c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 -- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 msgr2=0x7f229c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 -- 172.21.15.106:0/2985698037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c0ca4f0 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.131+0000 7f2293fff640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.132+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284048070 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.132+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2284030640 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.132+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c1f3cd0 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.132+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284034040 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.132+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c15d950 con 0x7f229c0cd0a0 2024-04-13T03:09:24.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.133+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c069120 con 0x7f229c0cd0a0 2024-04-13T03:09:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.133+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f228404b070 con 0x7f229c0cd0a0 2024-04-13T03:09:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.134+0000 7f22937fe640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 0x7f227c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.134+0000 7f229ad9d640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 0x7f227c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.134+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f22840ad800 con 0x7f229c0cd0a0 2024-04-13T03:09:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.134+0000 7f229ad9d640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 0x7f227c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f229c0c86c0 tx=0x7f228c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.138+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f229c1f3cd0 con 0x7f229c0cd0a0 2024-04-13T03:09:24.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.348+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f229c15fff0 con 0x7f229c0cd0a0 2024-04-13T03:09:24.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.972+0000 7f22937fe640 1 -- 172.21.15.106:0/2985698037 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f22840ad020 con 0x7f229c0cd0a0 2024-04-13T03:09:24.974 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 msgr2=0x7f227c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 0x7f227c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f229c0c86c0 tx=0x7f228c009040 comp rx=0 tx=0).stop 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 msgr2=0x7f229c1f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284004730 comp rx=0 tx=0).stop 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 shutdown_connections 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f227c06c8f0 0x7f227c06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.979+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c15c820 0x7f229c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.980+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0cd0a0 0x7f229c1f3790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.980+0000 7f22a10de640 1 --2- 172.21.15.106:0/2985698037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b7f00 0x7f229c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:24.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.980+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 >> 172.21.15.106:0/2985698037 conn(0x7f229c0d2510 msgr2=0x7f229c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:24.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.980+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 shutdown_connections 2024-04-13T03:09:24.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:24.981+0000 7f22a10de640 1 -- 172.21.15.106:0/2985698037 wait complete. 2024-04-13T03:09:24.999 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-13T03:09:24.999 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-13T03:09:25.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.104+0000 7f3e04836640 1 Processor -- start 2024-04-13T03:09:25.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.104+0000 7f3e04836640 1 -- start start 2024-04-13T03:09:25.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3e04836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3e04836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3e04836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e000cfc60 con 0x7f3e000cf7d0 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3e04836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3e000d30a0 con 0x7f3e0015e880 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3e04836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000d2a50 con 0x7f3e000d3530 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfed9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfe59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfed9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47754/0 (socket says 172.21.15.106:47754) 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfe59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:41556/0 (socket says 172.21.15.106:41556) 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfed9d640 1 -- 172.21.15.106:0/167694369 learned_addr learned my addr 172.21.15.106:0/167694369 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.105+0000 7f3dfdd9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3e000cf7d0 0x7f3e0006be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:49672/0 (socket says 172.21.15.106:49672) 2024-04-13T03:09:25.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfed9d640 1 -- 172.21.15.106:0/167694369 >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 msgr2=0x7f3e0006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:25.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfed9d640 1 --2- 172.21.15.106:0/167694369 >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfed9d640 1 -- 172.21.15.106:0/167694369 >> v1:172.21.15.114:6789/0 conn(0x7f3e000cf7d0 legacy=0x7f3e0006be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfed9d640 1 -- 172.21.15.106:0/167694369 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e000d2d30 con 0x7f3e000d3530 2024-04-13T03:09:25.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfd59a640 1 -- 172.21.15.106:0/167694369 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177157974 0 0) 0x7f3e000cfc60 con 0x7f3e000cf7d0 2024-04-13T03:09:25.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfe59c640 1 --2- 172.21.15.106:0/167694369 >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:25.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfed9d640 1 --2- 172.21.15.106:0/167694369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009e50 tx=0x7f3df802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7a9e751888b2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:25.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfd59a640 1 -- 172.21.15.106:0/167694369 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df802cb00 con 0x7f3e000d3530 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfd59a640 1 -- 172.21.15.106:0/167694369 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df802cca0 con 0x7f3e000d3530 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.106+0000 7f3dfd59a640 1 -- 172.21.15.106:0/167694369 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8034610 con 0x7f3e000d3530 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 -- 172.21.15.106:0/167694369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 msgr2=0x7f3e000d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 --2- 172.21.15.106:0/167694369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009e50 tx=0x7f3df802bf60 comp rx=0 tx=0).stop 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 -- 172.21.15.106:0/167694369 shutdown_connections 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 --2- 172.21.15.106:0/167694369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e000d3530 0x7f3e000d3970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 --2- 172.21.15.106:0/167694369 >> v2:172.21.15.114:3300/0 conn(0x7f3e0015e880 0x7f3e0006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 -- 172.21.15.106:0/167694369 >> 172.21.15.106:0/167694369 conn(0x7f3e000b6280 msgr2=0x7f3e000b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 -- 172.21.15.106:0/167694369 shutdown_connections 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.107+0000 7f3e04836640 1 -- 172.21.15.106:0/167694369 wait complete. 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.108+0000 7f3e04836640 1 Processor -- start 2024-04-13T03:09:25.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.108+0000 7f3e04836640 1 -- start start 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 0x7f3e0014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 0x7f3e000cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 0x7f3e000c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000b3bd0 con 0x7f3e000cbda0 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000b3a50 con 0x7f3e000d3530 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3e04836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000b78c0 con 0x7f3e0015e880 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfdd9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 0x7f3e000cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 0x7f3e000c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfe59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 0x7f3e0014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfdd9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 0x7f3e000cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47762/0 (socket says 172.21.15.106:47762) 2024-04-13T03:09:25.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfdd9b640 1 -- 172.21.15.106:0/4167216044 learned_addr learned my addr 172.21.15.106:0/4167216044 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfed9d640 1 -- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 msgr2=0x7f3e000cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.109+0000 7f3dfed9d640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 0x7f3e000cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3dfed9d640 1 -- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 msgr2=0x7f3e0014ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3dfed9d640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 0x7f3e0014ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3dfed9d640 1 -- 172.21.15.106:0/4167216044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0015f0c0 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3dfe59c640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 0x7f3e0014ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3dfed9d640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 0x7f3e000c9310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3df8032000 tx=0x7f3df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8043030 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df802ce50 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8009510 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0015f3a0 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.110+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0015f960 con 0x7f3e000cbda0 2024-04-13T03:09:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.111+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e00068a10 con 0x7f3e000cbda0 2024-04-13T03:09:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.111+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3df8046070 con 0x7f3e000cbda0 2024-04-13T03:09:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.112+0000 7f3deeffd640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 0x7f3de006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.112+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f3df80adc10 con 0x7f3e000cbda0 2024-04-13T03:09:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.112+0000 7f3dfe59c640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 0x7f3de006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.112+0000 7f3dfe59c640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 0x7f3de006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009560 tx=0x7f3df0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.116+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3df804d070 con 0x7f3e000cbda0 2024-04-13T03:09:25.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.298+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3e000cff10 con 0x7f3e000cbda0 2024-04-13T03:09:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.958+0000 7f3deeffd640 1 -- 172.21.15.106:0/4167216044 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f3df8082180 con 0x7f3e000cbda0 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 msgr2=0x7f3de006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 0x7f3de006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009560 tx=0x7f3df0009190 comp rx=0 tx=0).stop 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 msgr2=0x7f3e000c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 0x7f3e000c9310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3df8032000 tx=0x7f3df8004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 shutdown_connections 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3de006c8f0 0x7f3de006edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3e0015e880 0x7f3e000cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e000cbda0 0x7f3e000c9310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 --2- 172.21.15.106:0/4167216044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e000d3530 0x7f3e0014ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 >> 172.21.15.106:0/4167216044 conn(0x7f3e000b6280 msgr2=0x7f3e000b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 shutdown_connections 2024-04-13T03:09:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:25.961+0000 7f3e04836640 1 -- 172.21.15.106:0/4167216044 wait complete. 2024-04-13T03:09:25.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:09:26.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.085+0000 7fc832f1e640 1 Processor -- start 2024-04-13T03:09:26.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.085+0000 7fc832f1e640 1 -- start start 2024-04-13T03:09:26.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc832f1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc82c1593b0 0x7fc82c0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc832f1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc82c15fb00 0x7fc82c072d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc832f1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82c160170 con 0x7fc82c070a30 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc832f1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc82c15ff50 con 0x7fc82c1593b0 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc832f1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc82c070e90 con 0x7fc82c15fb00 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc82c1593b0 0x7fc82c0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc82bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc82c070a30 0x7fc82c15f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54738/0 (socket says 172.21.15.106:54738) 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc82c1593b0 0x7fc82c0704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51456/0 (socket says 172.21.15.106:51456) 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc82bfff640 1 -- 172.21.15.106:0/4270871123 learned_addr learned my addr 172.21.15.106:0/4270871123 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc8314bb640 1 --2- 172.21.15.106:0/4270871123 >> v2:172.21.15.114:3300/0 conn(0x7fc82c15fb00 0x7fc82c072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 -- 172.21.15.106:0/4270871123 >> v2:172.21.15.114:3300/0 conn(0x7fc82c15fb00 msgr2=0x7fc82c072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 --2- 172.21.15.106:0/4270871123 >> v2:172.21.15.114:3300/0 conn(0x7fc82c15fb00 0x7fc82c072d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 -- 172.21.15.106:0/4270871123 >> v1:172.21.15.106:6789/0 conn(0x7fc82c070a30 legacy=0x7fc82c15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc830cba640 1 -- 172.21.15.106:0/4270871123 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c0cf390 con 0x7fc82c1593b0 2024-04-13T03:09:26.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.086+0000 7fc82b7fe640 1 -- 172.21.15.106:0/4270871123 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993170984 0 0) 0x7fc82c160170 con 0x7fc82c070a30 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.087+0000 7fc830cba640 1 --2- 172.21.15.106:0/4270871123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c0704f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0cf0e0 tx=0x7fc82002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abc763fcafe6295 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.087+0000 7fc82b7fe640 1 -- 172.21.15.106:0/4270871123 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820035020 con 0x7fc82c1593b0 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.087+0000 7fc82b7fe640 1 -- 172.21.15.106:0/4270871123 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820031830 con 0x7fc82c1593b0 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.087+0000 7fc82b7fe640 1 -- 172.21.15.106:0/4270871123 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820031c00 con 0x7fc82c1593b0 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 -- 172.21.15.106:0/4270871123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 msgr2=0x7fc82c0704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 --2- 172.21.15.106:0/4270871123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c0704f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0cf0e0 tx=0x7fc82002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 -- 172.21.15.106:0/4270871123 shutdown_connections 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 --2- 172.21.15.106:0/4270871123 >> v2:172.21.15.114:3300/0 conn(0x7fc82c15fb00 0x7fc82c072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 --2- 172.21.15.106:0/4270871123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c0704f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 -- 172.21.15.106:0/4270871123 >> 172.21.15.106:0/4270871123 conn(0x7fc82c0d13f0 msgr2=0x7fc82c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 -- 172.21.15.106:0/4270871123 shutdown_connections 2024-04-13T03:09:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.088+0000 7fc832f1e640 1 -- 172.21.15.106:0/4270871123 wait complete. 2024-04-13T03:09:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.089+0000 7fc832f1e640 1 Processor -- start 2024-04-13T03:09:26.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.089+0000 7fc832f1e640 1 -- start start 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 0x7fc82c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 0x7fc82c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0cfc30 con 0x7fc82c1593b0 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0cf950 con 0x7fc82c15fb00 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c0d0150 con 0x7fc82c070a30 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51460/0 (socket says 172.21.15.106:51460) 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 -- 172.21.15.106:0/1468189830 learned_addr learned my addr 172.21.15.106:0/1468189830 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc8314bb640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 0x7fc82c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc830cba640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 0x7fc82c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 -- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 msgr2=0x7fc82c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 0x7fc82c15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 -- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 msgr2=0x7fc82c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 0x7fc82c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 -- 172.21.15.106:0/1468189830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c1f4e00 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc8314bb640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 0x7fc82c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc830cba640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 0x7fc82c15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc82bfff640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e730 tx=0x7fc81800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81800ee40 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc818016020 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818007500 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c1f50e0 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.090+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c1f56a0 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.091+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c15df70 con 0x7fc82c1593b0 2024-04-13T03:09:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.092+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc81800c7f0 con 0x7fc82c1593b0 2024-04-13T03:09:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.092+0000 7fc8297fa640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.092+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fc81808e240 con 0x7fc82c1593b0 2024-04-13T03:09:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.092+0000 7fc830cba640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.093+0000 7fc830cba640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc820036000 tx=0x7fc820008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:26.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.097+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc818020020 con 0x7fc82c1593b0 2024-04-13T03:09:26.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:26.274+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc82c069120 con 0x7fc82c1593b0 2024-04-13T03:09:27.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.987+0000 7fc8297fa640 1 -- 172.21.15.106:0/1468189830 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fc81805c540 con 0x7fc82c1593b0 2024-04-13T03:09:27.988 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data_ec' created 2024-04-13T03:09:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 msgr2=0x7fc81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc820036000 tx=0x7fc820008030 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 msgr2=0x7fc82c142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e730 tx=0x7fc81800ebf0 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 shutdown_connections 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82c15fb00 0x7fc82c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc82c1593b0 0x7fc82c142dc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 --2- 172.21.15.106:0/1468189830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc82c070a30 0x7fc82c15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.990+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 >> 172.21.15.106:0/1468189830 conn(0x7fc82c0d13f0 msgr2=0x7fc82c06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.991+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 shutdown_connections 2024-04-13T03:09:27.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:27.991+0000 7fc832f1e640 1 -- 172.21.15.106:0/1468189830 wait complete. 2024-04-13T03:09:28.010 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-13T03:09:28.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.117+0000 7f4306c36640 1 Processor -- start 2024-04-13T03:09:28.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.117+0000 7f4306c36640 1 -- start start 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f4306c36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f430015e880 0x7f430006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f4306c36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43000cfc60 con 0x7f43000cf7d0 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f4306c36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43000d2a50 con 0x7f43000d3530 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f4306c36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f43000d30a0 con 0x7f430015e880 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f43000cf7d0 0x7f430006be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54750/0 (socket says 172.21.15.106:54750) 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f7fff640 1 -- 172.21.15.106:0/3774872501 learned_addr learned my addr 172.21.15.106:0/3774872501 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f43049d2640 1 --2- 172.21.15.106:0/3774872501 >> v2:172.21.15.114:3300/0 conn(0x7f430015e880 0x7f430006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078506704 0 0) 0x7f43000cfc60 con 0x7f43000cf7d0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e8003520 con 0x7f43000cf7d0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 44274797 0 0) 0x7f42e8003520 con 0x7f43000cf7d0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 >> v2:172.21.15.114:3300/0 conn(0x7f430015e880 msgr2=0x7f430006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 --2- 172.21.15.106:0/3774872501 >> v2:172.21.15.114:3300/0 conn(0x7f430015e880 0x7f430006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 >> v1:172.21.15.114:6789/0 conn(0x7f43000d3530 legacy=0x7f43000d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43000d2d30 con 0x7f43000cf7d0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f42f0001670 con 0x7f43000cf7d0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f43049d2640 1 --2- 172.21.15.106:0/3774872501 >> v2:172.21.15.114:3300/0 conn(0x7f430015e880 0x7f430006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.118+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 >> v1:172.21.15.106:6789/0 conn(0x7f43000cf7d0 legacy=0x7f430006be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80039b0 0x7f42e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43000d30a0 con 0x7f42e80051c0 2024-04-13T03:09:28.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42e800c060 con 0x7f42e8004800 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42e800c260 con 0x7f42e80039b0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f42f7fff640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43049d2640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80039b0 0x7f42e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 -- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80039b0 msgr2=0x7f42e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80039b0 0x7f42e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 -- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e8004800 msgr2=0x7f42e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.119+0000 7f43051d3640 1 -- 172.21.15.106:0/3774872501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e8003520 con 0x7f42e80051c0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f43051d3640 1 -- 172.21.15.106:0/3774872501 reap_dead start 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f43051d3640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f8001e50 tx=0x7f42e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2194f8d6f0a31df server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8001a70 con 0x7f42e80051c0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f8003bb0 con 0x7f42e80051c0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f42f77fe640 1 -- 172.21.15.106:0/3774872501 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80050d0 con 0x7f42e80051c0 2024-04-13T03:09:28.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 -- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 msgr2=0x7f42e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42f8001e50 tx=0x7f42e8005830 comp rx=0 tx=0).stop 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 -- 172.21.15.106:0/3774872501 shutdown_connections 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 --2- 172.21.15.106:0/3774872501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 -- 172.21.15.106:0/3774872501 >> 172.21.15.106:0/3774872501 conn(0x7f43000b6280 msgr2=0x7f43000b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 -- 172.21.15.106:0/3774872501 shutdown_connections 2024-04-13T03:09:28.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.120+0000 7f4306c36640 1 -- 172.21.15.106:0/3774872501 wait complete. 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 Processor -- start 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 -- start start 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 0x7f430015f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43001604e0 0x7f43000cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43000b3bd0 con 0x7f430014da90 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43000b3a50 con 0x7f43001604e0 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.122+0000 7f4306c36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f43000b78c0 con 0x7f42e80051c0 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51474/0 (socket says 172.21.15.106:51474) 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 -- 172.21.15.106:0/735801530 learned_addr learned my addr 172.21.15.106:0/735801530 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f42f7fff640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43001604e0 0x7f43000cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43049d2640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 0x7f430015f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 -- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 msgr2=0x7f430015f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 0x7f430015f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 -- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43001604e0 msgr2=0x7f43000cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43001604e0 0x7f43000cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 -- 172.21.15.106:0/735801530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43000ccb80 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43049d2640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 0x7f430015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f43051d3640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42f8005040 tx=0x7f42f80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8006d60 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f8007030 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43000cce20 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80073e0 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.123+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430015ee70 con 0x7f430014da90 2024-04-13T03:09:28.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.124+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4300068a10 con 0x7f430014da90 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.125+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42f8007580 con 0x7f430014da90 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.125+0000 7f42f57fa640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 0x7f42ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.125+0000 7f43049d2640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 0x7f42ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.125+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f42f80ad980 con 0x7f430014da90 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.126+0000 7f43049d2640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 0x7f42ec06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009cd0 tx=0x7f42fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:28.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.130+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42f8033030 con 0x7f430014da90 2024-04-13T03:09:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.306+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f43001621e0 con 0x7f430014da90 2024-04-13T03:09:28.951 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:09:28.950+0000 7f97787f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:09:28.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:28.988+0000 7f42f57fa640 1 -- 172.21.15.106:0/735801530 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f42f807ba00 con 0x7f430014da90 2024-04-13T03:09:28.989 INFO:teuthology.orchestra.run.smithi106.stderr:set pool 4 allow_ec_overwrites to true 2024-04-13T03:09:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.002+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 msgr2=0x7f42ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.002+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 0x7f42ec06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009cd0 tx=0x7f42fc008040 comp rx=0 tx=0).stop 2024-04-13T03:09:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.002+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 msgr2=0x7f43000c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.002+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42f8005040 tx=0x7f42f80014a0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 shutdown_connections 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42ec06c920 0x7f42ec06ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f430014da90 0x7f43000c9850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43001604e0 0x7f43000cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 --2- 172.21.15.106:0/735801530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e80051c0 0x7f430015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.004+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 >> 172.21.15.106:0/735801530 conn(0x7f43000b6280 msgr2=0x7f430006d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:29.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.006+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 shutdown_connections 2024-04-13T03:09:29.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.007+0000 7f4306c36640 1 -- 172.21.15.106:0/735801530 wait complete. 2024-04-13T03:09:29.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-13T03:09:29.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.124+0000 7f66aa076640 1 Processor -- start 2024-04-13T03:09:29.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 -- start start 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 0x7f66a415d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66a413f8f0 0x7f66a4141e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a4142830 con 0x7f66a41423b0 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f66a4142df0 con 0x7f66a413f8f0 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f66aa076640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66a4142b10 con 0x7f66a413f510 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.125+0000 7f669affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66a413f8f0 0x7f66a4141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66a413f8f0 0x7f66a4141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51488/0 (socket says 172.21.15.106:51488) 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f66a3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f66a41423b0 0x7f66a40704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54764/0 (socket says 172.21.15.106:54764) 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 -- 172.21.15.106:0/4245766142 learned_addr learned my addr 172.21.15.106:0/4245766142 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f66a37fe640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 0x7f66a415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 -- 172.21.15.106:0/4245766142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 msgr2=0x7f66a415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:29.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 0x7f66a415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 -- 172.21.15.106:0/4245766142 >> v1:172.21.15.106:6789/0 conn(0x7f66a41423b0 legacy=0x7f66a40704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 -- 172.21.15.106:0/4245766142 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a415cf30 con 0x7f66a413f8f0 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f66a37fe640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 0x7f66a415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.126+0000 7f669affd640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f8f0 0x7f66a4141e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f668c009870 tx=0x7f668c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaebc6c48ee0bed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66a2ffd640 1 -- 172.21.15.106:0/4245766142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c034070 con 0x7f66a413f8f0 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66a2ffd640 1 -- 172.21.15.106:0/4245766142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c030840 con 0x7f66a413f8f0 2024-04-13T03:09:29.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66a2ffd640 1 -- 172.21.15.106:0/4245766142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c030c10 con 0x7f66a413f8f0 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 -- 172.21.15.106:0/4245766142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f8f0 msgr2=0x7f66a4141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f8f0 0x7f66a4141e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f668c009870 tx=0x7f668c02b980 comp rx=0 tx=0).stop 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 -- 172.21.15.106:0/4245766142 shutdown_connections 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f8f0 0x7f66a4141e70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 --2- 172.21.15.106:0/4245766142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a413f510 0x7f66a415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 -- 172.21.15.106:0/4245766142 >> 172.21.15.106:0/4245766142 conn(0x7f66a40d13f0 msgr2=0x7f66a40d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 -- 172.21.15.106:0/4245766142 shutdown_connections 2024-04-13T03:09:29.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.127+0000 7f66aa076640 1 -- 172.21.15.106:0/4245766142 wait complete. 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.128+0000 7f66aa076640 1 Processor -- start 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 -- start start 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a41423b0 0x7f66a41f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 0x7f66a4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66a406bf90 con 0x7f66a413f510 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66a415dc70 con 0x7f66a41423b0 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66aa076640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66a406c110 con 0x7f66a41526f0 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51500/0 (socket says 172.21.15.106:51500) 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66a37fe640 1 -- 172.21.15.106:0/910226476 learned_addr learned my addr 172.21.15.106:0/910226476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f66a3fff640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 0x7f66a4151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.129+0000 7f669affd640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a41423b0 0x7f66a41f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 -- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 msgr2=0x7f66a4151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 0x7f66a4151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 -- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a41423b0 msgr2=0x7f66a41f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a41423b0 0x7f66a41f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 -- 172.21.15.106:0/910226476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a4152ee0 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a3fff640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 0x7f66a4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a37fe640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f669400c820 tx=0x7f669400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694015070 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694007a30 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694007de0 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a413d810 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.130+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a413ddd0 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.131+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a4069120 con 0x7f66a413f510 2024-04-13T03:09:29.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.131+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6694018070 con 0x7f66a413f510 2024-04-13T03:09:29.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.132+0000 7f66a0ff9640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:29.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.132+0000 7f669affd640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:29.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.132+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f669408bd60 con 0x7f66a413f510 2024-04-13T03:09:29.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.132+0000 7f669affd640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f668c036000 tx=0x7f668c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:29.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.136+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6694059e70 con 0x7f66a413f510 2024-04-13T03:09:29.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:29.350+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f66a40b06f0 con 0x7f66a413f510 2024-04-13T03:09:30.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.005+0000 7f66a0ff9640 1 -- 172.21.15.106:0/910226476 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f6694059770 con 0x7f66a413f510 2024-04-13T03:09:30.007 INFO:teuthology.orchestra.run.smithi106.stderr:added data pool 4 to fsmap 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 msgr2=0x7f668406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f668c036000 tx=0x7f668c03b040 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 msgr2=0x7f66a4153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f669400c820 tx=0x7f669400cce0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 shutdown_connections 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a41526f0 0x7f66a4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a41423b0 0x7f66a41f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 --2- 172.21.15.106:0/910226476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a413f510 0x7f66a4153d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.007+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 >> 172.21.15.106:0/910226476 conn(0x7f66a40d13f0 msgr2=0x7f66a4072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.008+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 shutdown_connections 2024-04-13T03:09:30.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.008+0000 7f66aa076640 1 -- 172.21.15.106:0/910226476 wait complete. 2024-04-13T03:09:30.020 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:30.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.119+0000 7fe161e60640 1 Processor -- start 2024-04-13T03:09:30.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.119+0000 7fe161e60640 1 -- start start 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe161e60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe15c0b68d0 0x7fe15c0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe161e60640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15c15eda0 con 0x7fe15c15e9c0 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe161e60640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15c162c20 con 0x7fe15c1624b0 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe161e60640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe15c0d27f0 con 0x7fe15c0b68d0 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe15c15e9c0 0x7fe15c0b61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54778/0 (socket says 172.21.15.106:54778) 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15b7fe640 1 -- 172.21.15.106:0/2896637837 learned_addr learned my addr 172.21.15.106:0/2896637837 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15affd640 1 --2- 172.21.15.106:0/2896637837 >> v2:172.21.15.106:3300/0 conn(0x7fe15c0b68d0 0x7fe15c0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15a7fc640 1 -- 172.21.15.106:0/2896637837 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542179281 0 0) 0x7fe15c15eda0 con 0x7fe15c15e9c0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15a7fc640 1 -- 172.21.15.106:0/2896637837 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe144003520 con 0x7fe15c15e9c0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15affd640 1 -- 172.21.15.106:0/2896637837 >> v1:172.21.15.114:6789/0 conn(0x7fe15c1624b0 legacy=0x7fe15c0d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15affd640 1 -- 172.21.15.106:0/2896637837 >> v1:172.21.15.106:6789/0 conn(0x7fe15c15e9c0 legacy=0x7fe15c0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.120+0000 7fe15affd640 1 -- 172.21.15.106:0/2896637837 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c162f00 con 0x7fe15c0b68d0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.121+0000 7fe15affd640 1 --2- 172.21.15.106:0/2896637837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0ba290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1500097a0 tx=0x7fe15002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea3570af0c53f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.121+0000 7fe15a7fc640 1 -- 172.21.15.106:0/2896637837 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150034070 con 0x7fe15c0b68d0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.121+0000 7fe15a7fc640 1 -- 172.21.15.106:0/2896637837 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe150030610 con 0x7fe15c0b68d0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.121+0000 7fe15a7fc640 1 -- 172.21.15.106:0/2896637837 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500309e0 con 0x7fe15c0b68d0 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 -- 172.21.15.106:0/2896637837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c0b68d0 msgr2=0x7fe15c0ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 --2- 172.21.15.106:0/2896637837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0ba290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1500097a0 tx=0x7fe15002b960 comp rx=0 tx=0).stop 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 -- 172.21.15.106:0/2896637837 shutdown_connections 2024-04-13T03:09:30.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 --2- 172.21.15.106:0/2896637837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0ba290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 -- 172.21.15.106:0/2896637837 >> 172.21.15.106:0/2896637837 conn(0x7fe15c0d11c0 msgr2=0x7fe15c0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 -- 172.21.15.106:0/2896637837 shutdown_connections 2024-04-13T03:09:30.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.122+0000 7fe161e60640 1 -- 172.21.15.106:0/2896637837 wait complete. 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.123+0000 7fe161e60640 1 Processor -- start 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.123+0000 7fe161e60640 1 -- start start 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe15c1624b0 0x7fe15c160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c162940 con 0x7fe15c15e9c0 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c0d2a40 con 0x7fe15c0b68d0 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe161e60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe15c0d3670 con 0x7fe15c1624b0 2024-04-13T03:09:30.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51520/0 (socket says 172.21.15.106:51520) 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 -- 172.21.15.106:0/1663503783 learned_addr learned my addr 172.21.15.106:0/1663503783 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15b7fe640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15bfff640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe15c1624b0 0x7fe15c160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 -- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe15c1624b0 msgr2=0x7fe15c160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe15c1624b0 0x7fe15c160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 -- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c0b68d0 msgr2=0x7fe15c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.124+0000 7fe15affd640 1 -- 172.21.15.106:0/1663503783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c15f390 con 0x7fe15c15e9c0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe15affd640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe15002bf70 tx=0x7fe1500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150034040 con 0x7fe15c15e9c0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe150030410 con 0x7fe15c15e9c0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c15f670 con 0x7fe15c15e9c0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150039c50 con 0x7fe15c15e9c0 2024-04-13T03:09:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.125+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c15fc30 con 0x7fe15c15e9c0 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.126+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c068a10 con 0x7fe15c15e9c0 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.126+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe150037070 con 0x7fe15c15e9c0 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.126+0000 7fe143fff640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 0x7fe14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.127+0000 7fe15b7fe640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 0x7fe14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.127+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe1500add10 con 0x7fe15c15e9c0 2024-04-13T03:09:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.127+0000 7fe15b7fe640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 0x7fe14806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0011e0 tx=0x7fe14c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.131+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe15007b7b0 con 0x7fe15c15e9c0 2024-04-13T03:09:30.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.348+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe15c15dd30 con 0x7fe15c15e9c0 2024-04-13T03:09:30.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.349+0000 7fe143fff640 1 -- 172.21.15.106:0/1663503783 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7fe15007fab0 con 0x7fe15c15e9c0 2024-04-13T03:09:30.350 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:30.350 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:29.352334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:30.350 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 msgr2=0x7fe14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 0x7fe14806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0011e0 tx=0x7fe14c007d30 comp rx=0 tx=0).stop 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 msgr2=0x7fe15c0c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe15002bf70 tx=0x7fe1500046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 shutdown_connections 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe14806c8f0 0x7fe14806edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe15c1624b0 0x7fe15c160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe15c15e9c0 0x7fe15c0c8c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 --2- 172.21.15.106:0/1663503783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c0b68d0 0x7fe15c0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 >> 172.21.15.106:0/1663503783 conn(0x7fe15c0d11c0 msgr2=0x7fe15c0b8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 shutdown_connections 2024-04-13T03:09:30.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.351+0000 7fe161e60640 1 -- 172.21.15.106:0/1663503783 wait complete. 2024-04-13T03:09:30.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:30.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.461+0000 7f58a557b640 1 Processor -- start 2024-04-13T03:09:30.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.461+0000 7f58a557b640 1 -- start start 2024-04-13T03:09:30.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f58a557b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f58a00ba290 0x7f58a01624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f58a557b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a015eda0 con 0x7f58a00b68d0 2024-04-13T03:09:30.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f58a557b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a0162c20 con 0x7f58a015e9c0 2024-04-13T03:09:30.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f58a557b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f58a01629f0 con 0x7f58a00ba290 2024-04-13T03:09:30.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589e7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f58a00b68d0 0x7f58a00d2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54780/0 (socket says 172.21.15.106:54780) 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589e7fc640 1 -- 172.21.15.106:0/3387925677 learned_addr learned my addr 172.21.15.106:0/3387925677 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3387925677 >> v2:172.21.15.114:3300/0 conn(0x7f58a00ba290 0x7f58a01624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182136354 0 0) 0x7f58a015eda0 con 0x7f58a00b68d0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5888003520 con 0x7f58a00b68d0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.462+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3668686997 0 0) 0x7f5888003520 con 0x7f58a00b68d0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 >> v2:172.21.15.114:3300/0 conn(0x7f58a00ba290 msgr2=0x7f58a01624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 --2- 172.21.15.106:0/3387925677 >> v2:172.21.15.114:3300/0 conn(0x7f58a00ba290 0x7f58a01624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 >> v1:172.21.15.114:6789/0 conn(0x7f58a015e9c0 legacy=0x7f58a00b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a0162f00 con 0x7f58a00b68d0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5894001670 con 0x7f58a00b68d0 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 >> v1:172.21.15.106:6789/0 conn(0x7f58a00b68d0 legacy=0x7f58a00d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58880039b0 0x7f5888003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5888004800 0x7f5888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 0x7f588800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f588800c030 con 0x7f58880051c0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f588800c1e0 con 0x7f5888004800 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f588800c3e0 con 0x7f58880039b0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 0x7f588800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589e7fc640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5888004800 0x7f5888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589effd640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58880039b0 0x7f5888003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 -- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58880039b0 msgr2=0x7f5888003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58880039b0 0x7f5888003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 -- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5888004800 msgr2=0x7f5888004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5888004800 0x7f5888004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589f7fe640 1 -- 172.21.15.106:0/3387925677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5888003520 con 0x7f58880051c0 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.463+0000 7f589effd640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58880039b0 0x7f5888003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:30.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f589f7fe640 1 -- 172.21.15.106:0/3387925677 reap_dead start 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 0x7f588800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ac0 tx=0x7f5888005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b39f8f5895fdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898009690 con 0x7f58880051c0 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5898004020 con 0x7f58880051c0 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f589dffb640 1 -- 172.21.15.106:0/3387925677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898004410 con 0x7f58880051c0 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 -- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 msgr2=0x7f588800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 0x7f588800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ac0 tx=0x7f5888005830 comp rx=0 tx=0).stop 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 -- 172.21.15.106:0/3387925677 shutdown_connections 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 --2- 172.21.15.106:0/3387925677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58880051c0 0x7f588800b9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 -- 172.21.15.106:0/3387925677 >> 172.21.15.106:0/3387925677 conn(0x7f58a00d11c0 msgr2=0x7f58a00d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 -- 172.21.15.106:0/3387925677 shutdown_connections 2024-04-13T03:09:30.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.464+0000 7f58a557b640 1 -- 172.21.15.106:0/3387925677 wait complete. 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.466+0000 7f58a557b640 1 Processor -- start 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.466+0000 7f58a557b640 1 -- start start 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.466+0000 7f58a557b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880051c0 0x7f58a0153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.466+0000 7f58a557b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.466+0000 7f58a557b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58a00b4f10 0x7f58a0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58a557b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00d2a40 con 0x7f58a00c8d20 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58a557b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00d27f0 con 0x7f58880051c0 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58a557b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58a01624b0 con 0x7f58a00b4f10 2024-04-13T03:09:30.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51526/0 (socket says 172.21.15.106:51526) 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 -- 172.21.15.106:0/3645500584 learned_addr learned my addr 172.21.15.106:0/3645500584 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589f7fe640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58a00b4f10 0x7f58a0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589effd640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880051c0 0x7f58a0153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 -- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58a00b4f10 msgr2=0x7f58a0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58a00b4f10 0x7f58a0152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 -- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880051c0 msgr2=0x7f58a0153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880051c0 0x7f58a0153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 -- 172.21.15.106:0/3645500584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a00c99a0 con 0x7f58a00c8d20 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f589e7fc640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5894001e80 tx=0x7f5894007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589400f960 con 0x7f58a00c8d20 2024-04-13T03:09:30.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5894018070 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a00c93e0 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589400fd80 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.467+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a00cd100 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.468+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0068a10 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.468+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f589401d460 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.469+0000 7f58877fe640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 0x7f588c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.469+0000 7f589effd640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 0x7f588c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.469+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f589408be60 con 0x7f58a00c8d20 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.469+0000 7f589effd640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 0x7f588c06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5890001c30 tx=0x7f5890009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.473+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5894059fe0 con 0x7f58a00c8d20 2024-04-13T03:09:30.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.649+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a006cb60 con 0x7f58a00c8d20 2024-04-13T03:09:30.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.650+0000 7f58877fe640 1 -- 172.21.15.106:0/3645500584 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18563 (secure 0 0 0) 0x7f58940598e0 con 0x7f58a00c8d20 2024-04-13T03:09:30.651 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:30.652 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:29.352074+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:30.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 msgr2=0x7f588c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 0x7f588c06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5890001c30 tx=0x7f5890009040 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 msgr2=0x7f58a01f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5894001e80 tx=0x7f5894007720 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 shutdown_connections 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f588c06c920 0x7f588c06ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58a00b4f10 0x7f58a0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a00c8d20 0x7f58a01f84a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 --2- 172.21.15.106:0/3645500584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880051c0 0x7f58a0153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 >> 172.21.15.106:0/3645500584 conn(0x7f58a00d11c0 msgr2=0x7f58a015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 shutdown_connections 2024-04-13T03:09:30.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.652+0000 7f58a557b640 1 -- 172.21.15.106:0/3645500584 wait complete. 2024-04-13T03:09:30.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:30.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.762+0000 7f99ad0de640 1 Processor -- start 2024-04-13T03:09:30.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.762+0000 7f99ad0de640 1 -- start start 2024-04-13T03:09:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99ad0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99ad0de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99ad0de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a8073d60 con 0x7f99a806cc20 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99ad0de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f99a806c6b0 con 0x7f99a80cc910 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99ad0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f99a806c8e0 con 0x7f99a8073920 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a759e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a659c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a659c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47856/0 (socket says 172.21.15.106:47856) 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a6d9d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f99a806cc20 0x7f99a806d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:49710/0 (socket says 172.21.15.106:49710) 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a759e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:41684/0 (socket says 172.21.15.106:41684) 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.763+0000 7f99a659c640 1 -- 172.21.15.106:0/2132294327 learned_addr learned my addr 172.21.15.106:0/2132294327 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a659c640 1 -- 172.21.15.106:0/2132294327 >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 msgr2=0x7f99a80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a659c640 1 --2- 172.21.15.106:0/2132294327 >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a659c640 1 -- 172.21.15.106:0/2132294327 >> v1:172.21.15.114:6789/0 conn(0x7f99a806cc20 legacy=0x7f99a806d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a5d9b640 1 -- 172.21.15.106:0/2132294327 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948068815 0 0) 0x7f99a8073d60 con 0x7f99a806cc20 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a659c640 1 -- 172.21.15.106:0/2132294327 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80cbb50 con 0x7f99a8073920 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a759e640 1 --2- 172.21.15.106:0/2132294327 >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.764+0000 7f99a659c640 1 --2- 172.21.15.106:0/2132294327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f999c009870 tx=0x7f999c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3471099900d1423 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99a5d9b640 1 -- 172.21.15.106:0/2132294327 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c034070 con 0x7f99a8073920 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99a5d9b640 1 -- 172.21.15.106:0/2132294327 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c030840 con 0x7f99a8073920 2024-04-13T03:09:30.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99a5d9b640 1 -- 172.21.15.106:0/2132294327 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c030c10 con 0x7f99a8073920 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 -- 172.21.15.106:0/2132294327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 msgr2=0x7f99a806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 --2- 172.21.15.106:0/2132294327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f999c009870 tx=0x7f999c02b980 comp rx=0 tx=0).stop 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 -- 172.21.15.106:0/2132294327 shutdown_connections 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 --2- 172.21.15.106:0/2132294327 >> v2:172.21.15.114:3300/0 conn(0x7f99a80cc910 0x7f99a80ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 --2- 172.21.15.106:0/2132294327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a806c170 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 -- 172.21.15.106:0/2132294327 >> 172.21.15.106:0/2132294327 conn(0x7f99a80cdee0 msgr2=0x7f99a80ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 -- 172.21.15.106:0/2132294327 shutdown_connections 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.765+0000 7f99ad0de640 1 -- 172.21.15.106:0/2132294327 wait complete. 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 Processor -- start 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 -- start start 2024-04-13T03:09:30.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99a80cc910 0x7f99a815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80cc3f0 con 0x7f99a806cc20 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80cc110 con 0x7f99a80cc910 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.767+0000 7f99ad0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f99a815c9d0 con 0x7f99a8073920 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51536/0 (socket says 172.21.15.106:51536) 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 -- 172.21.15.106:0/3507692729 learned_addr learned my addr 172.21.15.106:0/3507692729 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a659c640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a759e640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99a80cc910 0x7f99a815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 -- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 msgr2=0x7f99a815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a815cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 -- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99a80cc910 msgr2=0x7f99a815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99a80cc910 0x7f99a815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 -- 172.21.15.106:0/3507692729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a815d750 con 0x7f99a806cc20 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99a6d9d640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f999800c8c0 tx=0x7f999800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998018070 con 0x7f99a806cc20 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999800a640 con 0x7f99a806cc20 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a815da30 con 0x7f99a806cc20 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999800aa30 con 0x7f99a806cc20 2024-04-13T03:09:30.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.768+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a815dff0 con 0x7f99a806cc20 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.769+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8069120 con 0x7f99a806cc20 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.770+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f999801b070 con 0x7f99a806cc20 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.770+0000 7f998f7fe640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 0x7f999406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.770+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f999808b490 con 0x7f99a806cc20 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.770+0000 7f99a659c640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 0x7f999406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:30.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.771+0000 7f99a659c640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 0x7f999406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f999c036000 tx=0x7f999c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:30.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.775+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9998058ea0 con 0x7f99a806cc20 2024-04-13T03:09:30.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.952+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99a80b2a30 con 0x7f99a806cc20 2024-04-13T03:09:30.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.953+0000 7f998f7fe640 1 -- 172.21.15.106:0/3507692729 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18563 (secure 0 0 0) 0x7f999805d1a0 con 0x7f99a806cc20 2024-04-13T03:09:30.954 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:30.954 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:29.352074+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:30.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 msgr2=0x7f999406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 0x7f999406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f999c036000 tx=0x7f999c008030 comp rx=0 tx=0).stop 2024-04-13T03:09:30.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 msgr2=0x7f99a8146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:30.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f999800c8c0 tx=0x7f999800ccc0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 shutdown_connections 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f999406c8f0 0x7f999406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99a8073920 0x7f99a815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99a80cc910 0x7f99a815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 --2- 172.21.15.106:0/3507692729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a806cc20 0x7f99a8146300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 >> 172.21.15.106:0/3507692729 conn(0x7f99a80cdee0 msgr2=0x7f99a8159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 shutdown_connections 2024-04-13T03:09:30.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:30.955+0000 7f99ad0de640 1 -- 172.21.15.106:0/3507692729 wait complete. 2024-04-13T03:09:30.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:31.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.081+0000 7fda44308640 1 Processor -- start 2024-04-13T03:09:31.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.081+0000 7fda44308640 1 -- start start 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 0x7fda3c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 0x7fda3c0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda3c0d46f0 0x7fda3c0d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c1625d0 con 0x7fda3c0d46f0 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c163430 con 0x7fda3c0d0a80 2024-04-13T03:09:31.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda44308640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d41b0 con 0x7fda3c0d1900 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.082+0000 7fda428a5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda3c0d46f0 0x7fda3c0d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda3c0d46f0 0x7fda3c0d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51546/0 (socket says 172.21.15.106:51546) 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 -- 172.21.15.106:0/3489192735 learned_addr learned my addr 172.21.15.106:0/3489192735 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 -- 172.21.15.106:0/3489192735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 msgr2=0x7fda3c0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda418a3640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 0x7fda3c0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda420a4640 1 --2- 172.21.15.106:0/3489192735 >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 0x7fda3c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 0x7fda3c0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 -- 172.21.15.106:0/3489192735 >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 msgr2=0x7fda3c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 --2- 172.21.15.106:0/3489192735 >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 0x7fda3c0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 -- 172.21.15.106:0/3489192735 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c162820 con 0x7fda3c0d46f0 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda418a3640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 0x7fda3c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda420a4640 1 --2- 172.21.15.106:0/3489192735 >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 0x7fda3c0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.083+0000 7fda428a5640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d46f0 0x7fda3c0d4b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda3c08e6c0 tx=0x7fda3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50121b0b69140c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda410a2640 1 -- 172.21.15.106:0/3489192735 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3402ca70 con 0x7fda3c0d46f0 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda410a2640 1 -- 172.21.15.106:0/3489192735 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3402cc10 con 0x7fda3c0d46f0 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda410a2640 1 -- 172.21.15.106:0/3489192735 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340343f0 con 0x7fda3c0d46f0 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 -- 172.21.15.106:0/3489192735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d46f0 msgr2=0x7fda3c0d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d46f0 0x7fda3c0d4b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda3c08e6c0 tx=0x7fda3402bed0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 -- 172.21.15.106:0/3489192735 shutdown_connections 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d46f0 0x7fda3c0d4b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 --2- 172.21.15.106:0/3489192735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d1900 0x7fda3c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 --2- 172.21.15.106:0/3489192735 >> v2:172.21.15.114:3300/0 conn(0x7fda3c0d0a80 0x7fda3c0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 -- 172.21.15.106:0/3489192735 >> 172.21.15.106:0/3489192735 conn(0x7fda3c09da70 msgr2=0x7fda3c0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 -- 172.21.15.106:0/3489192735 shutdown_connections 2024-04-13T03:09:31.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.084+0000 7fda44308640 1 -- 172.21.15.106:0/3489192735 wait complete. 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.085+0000 7fda44308640 1 Processor -- start 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.085+0000 7fda44308640 1 -- start start 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda3c0d0a80 0x7fda3c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d46f0 0x7fda3c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1630c0 con 0x7fda3c0d1900 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c162de0 con 0x7fda3c0d0a80 2024-04-13T03:09:31.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda44308640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d0e60 con 0x7fda3c0d46f0 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51554/0 (socket says 172.21.15.106:51554) 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 -- 172.21.15.106:0/947055185 learned_addr learned my addr 172.21.15.106:0/947055185 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda420a4640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda3c0d0a80 0x7fda3c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda428a5640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d46f0 0x7fda3c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 -- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d46f0 msgr2=0x7fda3c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d46f0 0x7fda3c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 -- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda3c0d0a80 msgr2=0x7fda3c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda3c0d0a80 0x7fda3c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.086+0000 7fda418a3640 1 -- 172.21.15.106:0/947055185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1f7a70 con 0x7fda3c0d1900 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda418a3640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00c820 tx=0x7fda2c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c015070 con 0x7fda3c0d1900 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c007a30 con 0x7fda3c0d1900 2024-04-13T03:09:31.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c007e00 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c06b8e0 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.087+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c06bea0 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.088+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c069120 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.088+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda2c018070 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.089+0000 7fda32ffd640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.089+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fda2c08be60 con 0x7fda3c0d1900 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.089+0000 7fda420a4640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.089+0000 7fda420a4640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 0x7fda2006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda38006ab0 tx=0x7fda38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.093+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda2c059f50 con 0x7fda3c0d1900 2024-04-13T03:09:31.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.306+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3c0d53d0 con 0x7fda3c0d1900 2024-04-13T03:09:31.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.307+0000 7fda32ffd640 1 -- 172.21.15.106:0/947055185 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7fda2c059850 con 0x7fda3c0d1900 2024-04-13T03:09:31.308 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:31.308 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:29.352334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:31.309 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-13T03:09:31.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 msgr2=0x7fda2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 0x7fda2006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda38006ab0 tx=0x7fda38008040 comp rx=0 tx=0).stop 2024-04-13T03:09:31.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 msgr2=0x7fda3c06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00c820 tx=0x7fda2c00cc20 comp rx=0 tx=0).stop 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 shutdown_connections 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda2006c8f0 0x7fda2006edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda3c0d46f0 0x7fda3c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda3c0d1900 0x7fda3c06c800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 --2- 172.21.15.106:0/947055185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda3c0d0a80 0x7fda3c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 >> 172.21.15.106:0/947055185 conn(0x7fda3c09da70 msgr2=0x7fda3c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 shutdown_connections 2024-04-13T03:09:31.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.309+0000 7fda44308640 1 -- 172.21.15.106:0/947055185 wait complete. 2024-04-13T03:09:31.321 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:31.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.419+0000 7f74379bb640 1 Processor -- start 2024-04-13T03:09:31.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 -- start start 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74300caf30 0x7f74301630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74300d0bc0 0x7f74300cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300cd700 con 0x7f7430144f10 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f74300cd4d0 con 0x7f74300caf30 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f74379bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f74300cd9e0 con 0x7f74300d0bc0 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f7434f56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74300caf30 0x7f74301630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.420+0000 7f7435757640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7430144f10 0x7f74300ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54788/0 (socket says 172.21.15.106:54788) 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7435757640 1 -- 172.21.15.106:0/642892797 learned_addr learned my addr 172.21.15.106:0/642892797 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:31.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74300caf30 0x7f74301630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51556/0 (socket says 172.21.15.106:51556) 2024-04-13T03:09:31.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7435f58640 1 --2- 172.21.15.106:0/642892797 >> v2:172.21.15.114:3300/0 conn(0x7f74300d0bc0 0x7f74300cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 -- 172.21.15.106:0/642892797 >> v2:172.21.15.114:3300/0 conn(0x7f74300d0bc0 msgr2=0x7f74300cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:31.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 --2- 172.21.15.106:0/642892797 >> v2:172.21.15.114:3300/0 conn(0x7f74300d0bc0 0x7f74300cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 -- 172.21.15.106:0/642892797 >> v1:172.21.15.106:6789/0 conn(0x7f7430144f10 legacy=0x7f74300ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 -- 172.21.15.106:0/642892797 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74300cdcc0 con 0x7f74300caf30 2024-04-13T03:09:31.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.421+0000 7f7434f56640 1 --2- 172.21.15.106:0/642892797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301630c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7430162560 tx=0x7f742002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66396948ed7a7b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f7427fff640 1 -- 172.21.15.106:0/642892797 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420034070 con 0x7f74300caf30 2024-04-13T03:09:31.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f7427fff640 1 -- 172.21.15.106:0/642892797 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420030610 con 0x7f74300caf30 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f7427fff640 1 -- 172.21.15.106:0/642892797 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200309e0 con 0x7f74300caf30 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 -- 172.21.15.106:0/642892797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 msgr2=0x7f74301630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 --2- 172.21.15.106:0/642892797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301630c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7430162560 tx=0x7f742002b960 comp rx=0 tx=0).stop 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 -- 172.21.15.106:0/642892797 shutdown_connections 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 --2- 172.21.15.106:0/642892797 >> v2:172.21.15.114:3300/0 conn(0x7f74300d0bc0 0x7f74300cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 --2- 172.21.15.106:0/642892797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301630c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 -- 172.21.15.106:0/642892797 >> 172.21.15.106:0/642892797 conn(0x7f743006f180 msgr2=0x7f743006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 -- 172.21.15.106:0/642892797 shutdown_connections 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.422+0000 7f74379bb640 1 -- 172.21.15.106:0/642892797 wait complete. 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.423+0000 7f74379bb640 1 Processor -- start 2024-04-13T03:09:31.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.423+0000 7f74379bb640 1 -- start start 2024-04-13T03:09:31.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74300d0bc0 0x7f743015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7430144f10 0x7f74300b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f743006c150 con 0x7f74300caf30 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74300d0fd0 con 0x7f74300d0bc0 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f74379bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f743006c2d0 con 0x7f7430144f10 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51568/0 (socket says 172.21.15.106:51568) 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 -- 172.21.15.106:0/935393593 learned_addr learned my addr 172.21.15.106:0/935393593 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 -- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7430144f10 msgr2=0x7f74300b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7430144f10 0x7f74300b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7434f56640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74300d0bc0 0x7f743015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 -- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74300d0bc0 msgr2=0x7f743015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74300d0bc0 0x7f743015e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 -- 172.21.15.106:0/935393593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7430160880 con 0x7f74300caf30 2024-04-13T03:09:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.424+0000 7f7435757640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c3c0 tx=0x7f742c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.425+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c017070 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.425+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c005ce0 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.425+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7430160b60 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.425+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0072f0 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.425+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7430161120 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.426+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7430068a10 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.426+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f742c01a070 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.426+0000 7f7425ffb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.427+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f742c08b230 con 0x7f74300caf30 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.427+0000 7f7434f56640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.427+0000 7f7434f56640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 0x7f741406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7420036000 tx=0x7f742003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:31.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.431+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f742c059320 con 0x7f74300caf30 2024-04-13T03:09:31.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.607+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74300cb720 con 0x7f74300caf30 2024-04-13T03:09:31.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.608+0000 7f7425ffb640 1 -- 172.21.15.106:0/935393593 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f742c058c20 con 0x7f74300caf30 2024-04-13T03:09:31.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:31.610 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.610+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 msgr2=0x7f741406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.610+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 0x7f741406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7420036000 tx=0x7f742003b040 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.610+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 msgr2=0x7f74301601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.610+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c3c0 tx=0x7f742c00c880 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 shutdown_connections 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7430144f10 0x7f74300b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74300d0bc0 0x7f743015e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 --2- 172.21.15.106:0/935393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74300caf30 0x7f74301601f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:31.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 >> 172.21.15.106:0/935393593 conn(0x7f743006f180 msgr2=0x7f74301425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 shutdown_connections 2024-04-13T03:09:31.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:31.611+0000 7f74379bb640 1 -- 172.21.15.106:0/935393593 wait complete. 2024-04-13T03:09:32.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:09:24.350589+0000', 'modified': '2024-04-13T03:09:29.352334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6835/4025109601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 4025109601}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 4025109601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-04-13T03:09:32.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:09:32.025 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:09:32.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.127+0000 7f01c1d1e640 1 Processor -- start 2024-04-13T03:09:32.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.127+0000 7f01c1d1e640 1 -- start start 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01c1d1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01bc15ea80 0x7f01bc0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01c1d1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01bc162550 0x7f01bc0b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01c1d1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc0cfe70 con 0x7f01bc0d1630 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01c1d1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc15ee60 con 0x7f01bc162550 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01c1d1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc162af0 con 0x7f01bc15ea80 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01bc162550 0x7f01bc0b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01bc162550 0x7f01bc0b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51572/0 (socket says 172.21.15.106:51572) 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 -- 172.21.15.106:0/559152816 learned_addr learned my addr 172.21.15.106:0/559152816 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/559152816 >> v2:172.21.15.114:3300/0 conn(0x7f01bc15ea80 0x7f01bc0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 -- 172.21.15.106:0/559152816 >> v2:172.21.15.114:3300/0 conn(0x7f01bc15ea80 msgr2=0x7f01bc0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 --2- 172.21.15.106:0/559152816 >> v2:172.21.15.114:3300/0 conn(0x7f01bc15ea80 0x7f01bc0d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 -- 172.21.15.106:0/559152816 >> v1:172.21.15.114:6789/0 conn(0x7f01bc0d1630 legacy=0x7f01bc0d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.128+0000 7f01baffd640 1 -- 172.21.15.106:0/559152816 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc162dd0 con 0x7f01bc162550 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01baffd640 1 --2- 172.21.15.106:0/559152816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc162550 0x7f01bc0b3930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0ba750 tx=0x7f01ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f522cbe62a697011 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01ba7fc640 1 -- 172.21.15.106:0/559152816 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac035020 con 0x7f01bc162550 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01ba7fc640 1 -- 172.21.15.106:0/559152816 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac031870 con 0x7f01bc162550 2024-04-13T03:09:32.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01ba7fc640 1 -- 172.21.15.106:0/559152816 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac031c40 con 0x7f01bc162550 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01c1d1e640 1 -- 172.21.15.106:0/559152816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc162550 msgr2=0x7f01bc0b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.129+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/559152816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc162550 0x7f01bc0b3930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0ba750 tx=0x7f01ac02b910 comp rx=0 tx=0).stop 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 -- 172.21.15.106:0/559152816 shutdown_connections 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/559152816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc162550 0x7f01bc0b3930 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/559152816 >> v2:172.21.15.114:3300/0 conn(0x7f01bc15ea80 0x7f01bc0d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 -- 172.21.15.106:0/559152816 >> 172.21.15.106:0/559152816 conn(0x7f01bc09da50 msgr2=0x7f01bc0b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 -- 172.21.15.106:0/559152816 shutdown_connections 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.130+0000 7f01c1d1e640 1 -- 172.21.15.106:0/559152816 wait complete. 2024-04-13T03:09:32.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.131+0000 7f01c1d1e640 1 Processor -- start 2024-04-13T03:09:32.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.131+0000 7f01c1d1e640 1 -- start start 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 0x7f01bc0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01bc162550 0x7f01bc1e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0b3e70 con 0x7f01bc0d1630 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc163390 con 0x7f01bc15ea80 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc0d0320 con 0x7f01bc162550 2024-04-13T03:09:32.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51578/0 (socket says 172.21.15.106:51578) 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1157037855 learned_addr learned my addr 172.21.15.106:0/1157037855 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01baffd640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 0x7f01bc0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bbfff640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01bc162550 0x7f01bc1e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01bc162550 msgr2=0x7f01bc1e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01bc162550 0x7f01bc1e8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 msgr2=0x7f01bc0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 0x7f01bc0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1157037855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc1e9950 con 0x7f01bc0d1630 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01baffd640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 0x7f01bc0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d800 tx=0x7f01b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01a3fff640 1 -- 172.21.15.106:0/1157037855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000b6c0 con 0x7f01bc0d1630 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01a3fff640 1 -- 172.21.15.106:0/1157037855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b0019070 con 0x7f01bc0d1630 2024-04-13T03:09:32.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01a3fff640 1 -- 172.21.15.106:0/1157037855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000ba80 con 0x7f01bc0d1630 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc1f5f90 con 0x7f01bc0d1630 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.132+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc1e9ed0 con 0x7f01bc0d1630 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.134+0000 7f01a3fff640 1 -- 172.21.15.106:0/1157037855 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01b000bc20 con 0x7f01bc0d1630 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.134+0000 7f01a3fff640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 0x7f01a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.134+0000 7f01baffd640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 0x7f01a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.134+0000 7f01a3fff640 1 -- 172.21.15.106:0/1157037855 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01b008c380 con 0x7f01bc0d1630 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.135+0000 7f01baffd640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 0x7f01a806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01ac036000 tx=0x7f01ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.135+0000 7f01c4bfe740 1 Processor -- start 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.135+0000 7f01c4bfe740 1 -- start start 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93b42c0 0x55cde93e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cde931e990 con 0x55cde93e0de0 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cde92c5620 con 0x55cde93e11c0 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cde931a950 con 0x55cde93b42c0 2024-04-13T03:09:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51594/0 (socket says 172.21.15.106:51594) 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1287141955 learned_addr learned my addr 172.21.15.106:0/1287141955 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01baffd640 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bbfff640 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93b42c0 0x55cde93e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93b42c0 msgr2=0x55cde93e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93b42c0 0x55cde93e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 msgr2=0x55cde93e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 -- 172.21.15.106:0/1287141955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde90312d0 con 0x55cde93e0de0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f01b008a570 tx=0x7f01b00599e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fa0ba5dc3ea9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01a17fa640 1 -- 172.21.15.106:0/1287141955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0072020 con 0x55cde93e0de0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01a17fa640 1 -- 172.21.15.106:0/1287141955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.136+0000 7f01a17fa640 1 -- 172.21.15.106:0/1287141955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0062950 con 0x55cde93e0de0 2024-04-13T03:09:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 -- 172.21.15.106:0/1287141955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 msgr2=0x55cde93b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f01b008a570 tx=0x7f01b00599e0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 -- 172.21.15.106:0/1287141955 shutdown_connections 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93b42c0 0x55cde93e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/1287141955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93b3d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 -- 172.21.15.106:0/1287141955 >> 172.21.15.106:0/1287141955 conn(0x55cde93b5ff0 msgr2=0x55cde93feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 -- 172.21.15.106:0/1287141955 shutdown_connections 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.137+0000 7f01c4bfe740 1 -- 172.21.15.106:0/1287141955 wait complete. 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.138+0000 7f01c4bfe740 1 -- start start 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.138+0000 7f01c4bfe740 1 Processor -- start 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.138+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde9412890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cde93b8590 con 0x55cde93e0de0 2024-04-13T03:09:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cde92d89e0 con 0x55cde93e11c0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cde93b7980 con 0x55cde9412410 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 10 client.0 _ll_register_callbacks cb 0x55cde9314220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51598/0 (socket says 172.21.15.106:51598) 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2566191982 learned_addr learned my addr 172.21.15.106:0/2566191982 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01baffd640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bbfff640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde9412890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 msgr2=0x55cde9412890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde9412890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 msgr2=0x55cde93f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2566191982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde90fb6b0 con 0x55cde93e0de0 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01baffd640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:32.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01b00561d0 tx=0x7f01b008a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f133e8c34bb530 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0072040 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01b008c070 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f01b008a270 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 10 client.4431 fetch_fsmap learned FSMap version 7 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55cde93f6350 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f01b0013070 con 0x55cde93e0de0 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 10 client.4431 fetch_fsmap finished waiting for FSMap version 7 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.139+0000 7f01c4bfe740 10 client.4431 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01c4bfe740 20 client.4431 populate_metadata read hostname 'smithi106' 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde93f7590 0x55cde941d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01c4bfe740 4 client.4431 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 --> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cde932c3b0 con 0x55cde93f7590 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01baffd640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde93f7590 0x55cde941d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f019a7fc640 10 client.4431 ms_handle_connect on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.140+0000 7f01baffd640 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde93f7590 0x55cde941d8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741e192c7796cbb7 server_cookie=6e7c1447ee406e63 in_seq=0 out_seq=0 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f019a7fc640 1 -- 172.21.15.106:0/2566191982 <== mds.0 v2:172.21.15.114:6834/4025109601 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cde932c3b0 con 0x55cde93f7590 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f019a7fc640 10 client.4431 handle_command_reply: tid=0 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 client.4431 shutdown 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 msgr2=0x55cde93e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01b00561d0 tx=0x7f01b008a440 comp rx=0 tx=0).stop 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 shutdown_connections 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde93f7590 0x55cde941d8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde9412890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde93f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2566191982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde93e71f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f019a7fc640 0 client.4431 ms_handle_reset on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:32.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 >> 172.21.15.106:0/2566191982 conn(0x55cde93b42c0 msgr2=0x55cde93b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 shutdown_connections 2024-04-13T03:09:32.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2566191982 wait complete. 2024-04-13T03:09:32.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.141+0000 7f01c4bfe740 20 client.4431 trim_cache size 0 max 16384 2024-04-13T03:09:32.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.159+0000 7f01c4bfe740 1 Processor -- start 2024-04-13T03:09:32.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.159+0000 7f01c4bfe740 1 -- start start 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93e11c0 0x55cde93ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde9412410 0x55cde9400e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc2014a0 con 0x55cde93e0de0 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1f9420 con 0x55cde9412410 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc1eaa10 con 0x55cde93e11c0 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51606/0 (socket says 172.21.15.106:51606) 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2280117324 learned_addr learned my addr 172.21.15.106:0/2280117324 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93e11c0 msgr2=0x55cde93ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93e11c0 0x55cde93ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bbfff640 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde9412410 0x55cde9400e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde9412410 msgr2=0x55cde9400e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde9412410 0x55cde9400e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 -- 172.21.15.106:0/2280117324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde93e05f0 con 0x55cde93e0de0 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f01b0058020 tx=0x7f01b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f263b0ba9444502d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01a0ff9640 1 -- 172.21.15.106:0/2280117324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0072040 con 0x55cde93e0de0 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01a0ff9640 1 -- 172.21.15.106:0/2280117324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.160+0000 7f01a0ff9640 1 -- 172.21.15.106:0/2280117324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0056010 con 0x55cde93e0de0 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2280117324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 msgr2=0x55cde9418950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f01b0058020 tx=0x7f01b001f000 comp rx=0 tx=0).stop 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2280117324 shutdown_connections 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde93e11c0 0x55cde93ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde9412410 0x55cde9400e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/2280117324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9418950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2280117324 >> 172.21.15.106:0/2280117324 conn(0x55cde93b42c0 msgr2=0x55cde93b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2280117324 shutdown_connections 2024-04-13T03:09:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.161+0000 7f01c4bfe740 1 -- 172.21.15.106:0/2280117324 wait complete. 2024-04-13T03:09:32.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 -- start start 2024-04-13T03:09:32.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 Processor -- start 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde940d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde940d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cde93b8590 con 0x55cde93e0de0 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cde92d89e0 con 0x55cde93e11c0 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cde93f9510 con 0x55cde9412410 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01c4bfe740 10 client.0 _ll_register_callbacks cb 0x7f01b00596d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:09:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51612/0 (socket says 172.21.15.106:51612) 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.162+0000 7f01bb7fe640 1 -- 172.21.15.106:0/4148591440 learned_addr learned my addr 172.21.15.106:0/4148591440 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bbfff640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde940d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01baffd640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde940d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 -- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 msgr2=0x55cde940d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde940d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 -- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 msgr2=0x55cde940d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde940d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 -- 172.21.15.106:0/4148591440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde94790c0 con 0x55cde93e0de0 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01baffd640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde940d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bbfff640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde940d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01bb7fe640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01b0059d00 tx=0x7f01b000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf85f9c2ebc2add server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0072040 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01b00721e0 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f01b006a050 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 10 client.4437 fetch_fsmap learned FSMap version 7 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55cde94790c0 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b005e120 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f01b00563e0 con 0x55cde93e0de0 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 10 client.4437 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 20 client.4437 populate_metadata read hostname 'smithi106' 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde9479c30 0x55cde947a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 4 client.4437 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.163+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 --> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] -- command(tid 0: {"prefix": "status"}) -- 0x55cde9326220 con 0x55cde9479c30 2024-04-13T03:09:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01baffd640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde9479c30 0x55cde947a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f019a7fc640 10 client.4437 ms_handle_connect on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01baffd640 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde9479c30 0x55cde947a010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec51f894421ef3e server_cookie=98c5af5951e36e9 in_seq=0 out_seq=0 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f019a7fc640 1 -- 172.21.15.106:0/4148591440 <== mds.0 v2:172.21.15.114:6834/4025109601 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55cde9326220 con 0x55cde9479c30 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f019a7fc640 10 client.4437 handle_command_reply: tid=0 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 client.4437 shutdown 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 msgr2=0x55cde9478b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01b0059d00 tx=0x7f01b000bf90 comp rx=0 tx=0).stop 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 shutdown_connections 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x55cde9479c30 0x55cde947a010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cde9412410 0x55cde940d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f019a7fc640 0 client.4437 ms_handle_reset on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cde93e11c0 0x55cde940d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 --2- 172.21.15.106:0/4148591440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cde93e0de0 0x55cde9478b80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 >> 172.21.15.106:0/4148591440 conn(0x55cde93b42c0 msgr2=0x55cde93f86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 shutdown_connections 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 1 -- 172.21.15.106:0/4148591440 wait complete. 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.164+0000 7f01c4bfe740 20 client.4437 trim_cache size 0 max 16384 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4293, 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:09:32.173 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 7.1914966209999998, 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 7, 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 25, 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 24, 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 14.215625142 2024-04-13T03:09:32.174 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 msgr2=0x7f01a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 0x7f01a806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01ac036000 tx=0x7f01ac008030 comp rx=0 tx=0).stop 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 msgr2=0x7f01bc1f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d800 tx=0x7f01b000dcc0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 shutdown_connections 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01a806c8f0 0x7f01a806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01bc162550 0x7f01bc1e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01bc15ea80 0x7f01bc0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 --2- 172.21.15.106:0/1157037855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01bc0d1630 0x7f01bc1f5a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 >> 172.21.15.106:0/1157037855 conn(0x7f01bc09da50 msgr2=0x7f01bc0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 shutdown_connections 2024-04-13T03:09:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.165+0000 7f01c1d1e640 1 -- 172.21.15.106:0/1157037855 wait complete. 2024-04-13T03:09:32.177 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T03:09:32.177 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:32.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:09:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.275+0000 7f96af4b7640 1 Processor -- start 2024-04-13T03:09:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.275+0000 7f96af4b7640 1 -- start start 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96af4b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96a815ca70 0x7f96a806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96af4b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a80b7ef0 0x7f96a80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96af4b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a815ce50 con 0x7f96a80cd390 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96af4b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f96a80cd7c0 con 0x7f96a815ca70 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96af4b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96a8070480 con 0x7f96a80b7ef0 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96a815ca70 0x7f96a806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96a815ca70 0x7f96a806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51622/0 (socket says 172.21.15.106:51622) 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 -- 172.21.15.106:0/2141772507 learned_addr learned my addr 172.21.15.106:0/2141772507 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ada54640 1 --1- 172.21.15.106:0/2141772507 >> v1:172.21.15.106:6789/0 conn(0x7f96a80cd390 0x7f96a8160580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54800/0 (socket says 172.21.15.106:54800) 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96aca52640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a80b7ef0 0x7f96a80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 -- 172.21.15.106:0/2141772507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a80b7ef0 msgr2=0x7f96a80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a80b7ef0 0x7f96a80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 -- 172.21.15.106:0/2141772507 >> v1:172.21.15.106:6789/0 conn(0x7f96a80cd390 legacy=0x7f96a8160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f969ffff640 1 -- 172.21.15.106:0/2141772507 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017689016 0 0) 0x7f96a815ce50 con 0x7f96a80cd390 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.276+0000 7f96ad253640 1 -- 172.21.15.106:0/2141772507 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a8070760 con 0x7f96a815ca70 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.277+0000 7f96ad253640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a815ca70 0x7f96a806fea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009870 tx=0x7f96a402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6dea56c2b25a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.277+0000 7f969ffff640 1 -- 172.21.15.106:0/2141772507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a402f840 con 0x7f96a815ca70 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.277+0000 7f969ffff640 1 -- 172.21.15.106:0/2141772507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a402f9e0 con 0x7f96a815ca70 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.277+0000 7f969ffff640 1 -- 172.21.15.106:0/2141772507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a402fdb0 con 0x7f96a815ca70 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 -- 172.21.15.106:0/2141772507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a815ca70 msgr2=0x7f96a806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a815ca70 0x7f96a806fea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009870 tx=0x7f96a402ba20 comp rx=0 tx=0).stop 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 -- 172.21.15.106:0/2141772507 shutdown_connections 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a80b7ef0 0x7f96a80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 --2- 172.21.15.106:0/2141772507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a815ca70 0x7f96a806fea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 -- 172.21.15.106:0/2141772507 >> 172.21.15.106:0/2141772507 conn(0x7f96a80d2530 msgr2=0x7f96a80d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 -- 172.21.15.106:0/2141772507 shutdown_connections 2024-04-13T03:09:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.278+0000 7f96af4b7640 1 -- 172.21.15.106:0/2141772507 wait complete. 2024-04-13T03:09:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 Processor -- start 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 -- start start 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 0x7f96a81e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a815ca70 0x7f96a80cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96a80b8360 con 0x7f96a80cd390 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8070d20 con 0x7f96a80b7ef0 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.279+0000 7f96af4b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96a80b84e0 con 0x7f96a815ca70 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51636/0 (socket says 172.21.15.106:51636) 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 -- 172.21.15.106:0/1378092974 learned_addr learned my addr 172.21.15.106:0/1378092974 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96ada54640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a815ca70 0x7f96a80cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96ad253640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 0x7f96a81e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 -- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a815ca70 msgr2=0x7f96a80cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a815ca70 0x7f96a80cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 -- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 msgr2=0x7f96a81e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 0x7f96a81e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 -- 172.21.15.106:0/1378092974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a80cc5a0 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96ad253640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 0x7f96a81e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96aca52640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f969800d5f0 tx=0x7f969800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980125f0 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698019070 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a8159480 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980129d0 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.280+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a8159a40 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.281+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a8068a10 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.282+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9698012b70 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.282+0000 7f969dffb640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.282+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f969808c290 con 0x7f96a80cd390 2024-04-13T03:09:32.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.282+0000 7f96ad253640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:32.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.282+0000 7f96ad253640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96a4036000 tx=0x7f96a4039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:32.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.286+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f969805a380 con 0x7f96a80cd390 2024-04-13T03:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.499+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f96a8071320 con 0x7f96a80cd390 2024-04-13T03:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.499+0000 7f969dffb640 1 -- 172.21.15.106:0/1378092974 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f9698059c80 con 0x7f96a80cd390 2024-04-13T03:09:32.500 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 msgr2=0x7f968c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96a4036000 tx=0x7f96a4039040 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 msgr2=0x7f96a8159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f969800d5f0 tx=0x7f969800dab0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 shutdown_connections 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96a815ca70 0x7f96a80cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a80cd390 0x7f96a8159f60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 --2- 172.21.15.106:0/1378092974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96a80b7ef0 0x7f96a81e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 >> 172.21.15.106:0/1378092974 conn(0x7f96a80d2530 msgr2=0x7f96a806fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 shutdown_connections 2024-04-13T03:09:32.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32.501+0000 7f96af4b7640 1 -- 172.21.15.106:0/1378092974 wait complete. 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:09:32.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T03:09:32.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:09:32.514 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-13T03:09:32.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:32.514 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:09:32.529 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:09:32.529 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:09:32.529 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:09:32.530 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:32.531 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:09:32.531 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42577sec preferred_lft 42577sec 2024-04-13T03:09:32.531 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:09:32.531 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:32.531 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-13T03:09:32.532 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:32.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:32.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:32.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:32.718 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:09:32.744 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:09:32.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:32.746 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:09:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:09:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:09:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:09:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:09:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:09:32.817 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:32.818 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:32.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:09:32.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:09:32.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:09:32.818 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:32.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:33.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:33.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:33.113 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:09:33.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:33.128 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-13T03:09:33.182 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:33.182 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:33.182 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:33.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-13T03:09:33.183 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:33.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:33.308 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-13T03:09:33.308 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:33.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:33.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:33.443 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:33.443 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:33.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:09:33.444 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-13T03:09:33.444 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:33.526 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-13T03:09:33.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:09:33.526 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:33.540 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-13T03:09:33.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:09:33.540 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:33.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-13T03:09:33.663 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-13T03:09:33.718 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T03:09:33.719 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:33.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:33.721 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-13T03:09:33.773 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-13T03:09:33.773 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-13T03:09:33.815 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-13T03:09:33.840 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T03:09:33.895 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-13T03:09:33.895 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-13T03:09:33.895 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-13T03:09:33.895 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-13T03:09:33.896 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:33.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:33.897 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-13T03:09:33.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:2024-04-13T03:09:33.900+0000 7f249b171480 -1 init, newargv = 0x563b32a6e0f0 newargc=15 2024-04-13T03:09:33.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[44671]: starting ceph client 2024-04-13T03:09:33.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[44671]: starting fuse 2024-04-13T03:09:33.917 INFO:teuthology.orchestra.run.smithi106.stdout:42 2024-04-13T03:09:33.917 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-13T03:09:33.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-13T03:09:33.917 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-13T03:09:33.917 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-13T03:09:33.917 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-13T03:09:33.918 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.admin")) 2024-04-13T03:09:33.919 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:09:34.038 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.admin.44671.asok 2024-04-13T03:09:34.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:34.048 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44671.asok 2024-04-13T03:09:34.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:34.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44671.asok status 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "218a691417ef57c5d22a8d5cf3c5f41f7b542838", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "admin", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "44671", 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-13T03:09:34.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4455, 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4455 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1529860489 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:09:34.186 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1529860489 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4455 192.168.0.1:0/1529860489", 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/1529860489", 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 7, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 25, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-13T03:09:34.187 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:09:34.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:34.193 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-13T03:09:34.193 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:34.208 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-13T03:09:34.208 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:34.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:34.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:34.276 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-13T03:09:34.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:09:34.384 INFO:teuthology.orchestra.run.smithi106.stdout:# file: . 2024-04-13T03:09:34.384 INFO:teuthology.orchestra.run.smithi106.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-13T03:09:34.385 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:34.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:34.391 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:34.391 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:09:34.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.546+0000 7f94f9755640 1 Processor -- start 2024-04-13T03:09:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.546+0000 7f94f9755640 1 -- start start 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f9755640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f94f415e9b0 0x7f94f40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f9755640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40cfd20 con 0x7f94f40b3d60 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f9755640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f4162af0 con 0x7f94f415ed90 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f9755640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f94f4162dd0 con 0x7f94f415e9b0 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f94f40b3d60 0x7f94f40d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53638/0 (socket says 172.21.15.106:53638) 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f37fe640 1 -- 172.21.15.106:0/2006607095 learned_addr learned my addr 172.21.15.106:0/2006607095 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/2006607095 >> v2:172.21.15.114:3300/0 conn(0x7f94f415e9b0 0x7f94f40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2659012492 0 0) 0x7f94f40cfd20 con 0x7f94f40b3d60 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94dc003520 con 0x7f94f40b3d60 2024-04-13T03:09:34.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2699223732 0 0) 0x7f94dc003520 con 0x7f94f40b3d60 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 >> v2:172.21.15.114:3300/0 conn(0x7f94f415e9b0 msgr2=0x7f94f40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 --2- 172.21.15.106:0/2006607095 >> v2:172.21.15.114:3300/0 conn(0x7f94f415e9b0 0x7f94f40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 >> v1:172.21.15.114:6789/0 conn(0x7f94f415ed90 legacy=0x7f94f40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f41630b0 con 0x7f94f40b3d60 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f94ec0019f0 con 0x7f94f40b3d60 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 >> v1:172.21.15.106:6789/0 conn(0x7f94f40b3d60 legacy=0x7f94f40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94dc004800 0x7f94dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc0051c0 0x7f94dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4162dd0 con 0x7f94dc0039b0 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc00c070 con 0x7f94dc0051c0 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc00c270 con 0x7f94dc004800 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f37fe640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc0051c0 0x7f94dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.547+0000 7f94f27fc640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94dc004800 0x7f94dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 -- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94dc004800 msgr2=0x7f94dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94dc004800 0x7f94dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 -- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc0051c0 msgr2=0x7f94dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc0051c0 0x7f94dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f37fe640 1 -- 172.21.15.106:0/2006607095 reap_dead start 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 -- 172.21.15.106:0/2006607095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc003520 con 0x7f94dc0039b0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94dc003dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ac0 tx=0x7f94dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d588b4a2b0003 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.548+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4039070 con 0x7f94dc0039b0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e402c6b0 con 0x7f94dc0039b0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f1ffb640 1 -- 172.21.15.106:0/2006607095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4009690 con 0x7f94dc0039b0 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 -- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 msgr2=0x7f94dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94dc003dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ac0 tx=0x7f94dc005830 comp rx=0 tx=0).stop 2024-04-13T03:09:34.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 -- 172.21.15.106:0/2006607095 shutdown_connections 2024-04-13T03:09:34.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 --2- 172.21.15.106:0/2006607095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94dc003dd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 -- 172.21.15.106:0/2006607095 >> 172.21.15.106:0/2006607095 conn(0x7f94f408bf10 msgr2=0x7f94f40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:34.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 -- 172.21.15.106:0/2006607095 shutdown_connections 2024-04-13T03:09:34.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.549+0000 7f94f9755640 1 -- 172.21.15.106:0/2006607095 wait complete. 2024-04-13T03:09:34.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.550+0000 7f94f9755640 1 Processor -- start 2024-04-13T03:09:34.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.550+0000 7f94f9755640 1 -- start start 2024-04-13T03:09:34.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94f4153a20 0x7f94f4159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94f40b4160 0x7f94f40b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4162800 con 0x7f94dc0039b0 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4162680 con 0x7f94f4153a20 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f9755640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f94f406eb20 con 0x7f94f40b4160 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51786/0 (socket says 172.21.15.106:51786) 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 -- 172.21.15.106:0/1580261813 learned_addr learned my addr 172.21.15.106:0/1580261813 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f27fc640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94f4153a20 0x7f94f4159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f37fe640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94f40b4160 0x7f94f40b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 -- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94f40b4160 msgr2=0x7f94f40b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94f40b4160 0x7f94f40b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 -- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94f4153a20 msgr2=0x7f94f4159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94f4153a20 0x7f94f4159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.551+0000 7f94f2ffd640 1 -- 172.21.15.106:0/1580261813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40b5170 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94f2ffd640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ac0 tx=0x7f94e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4039050 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e403a040 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f40b4e90 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40044e0 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.552+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f40cc080 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.553+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f40cbb90 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.553+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94e4047020 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.554+0000 7f94db7fe640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.554+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94e40ad700 con 0x7f94dc0039b0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.554+0000 7f94f27fc640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.554+0000 7f94f27fc640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f94f4159a40 tx=0x7f94e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.558+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94e407b9a0 con 0x7f94dc0039b0 2024-04-13T03:09:34.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.733+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94f40b5340 con 0x7f94dc0039b0 2024-04-13T03:09:34.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.734+0000 7f94db7fe640 1 -- 172.21.15.106:0/1580261813 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f94e407b2a0 con 0x7f94dc0039b0 2024-04-13T03:09:34.736 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:34.736 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 msgr2=0x7f94e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f94f4159a40 tx=0x7f94e8009040 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 msgr2=0x7f94f4158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ac0 tx=0x7f94e4004730 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 shutdown_connections 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94f40b4160 0x7f94f40b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94f4153a20 0x7f94f4159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 --2- 172.21.15.106:0/1580261813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94dc0039b0 0x7f94f4158c10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 >> 172.21.15.106:0/1580261813 conn(0x7f94f408bf10 msgr2=0x7f94f40d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 shutdown_connections 2024-04-13T03:09:34.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.736+0000 7f94f9755640 1 -- 172.21.15.106:0/1580261813 wait complete. 2024-04-13T03:09:34.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:09:34.749 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.850+0000 7f6a6546d640 1 Processor -- start 2024-04-13T03:09:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.850+0000 7f6a6546d640 1 -- start start 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a6546d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a6015ed90 0x7f6a600d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a6546d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6a600b3d60 0x7f6a600d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a6546d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a600cfd20 con 0x7f6a6015e9b0 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a6546d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6a60162af0 con 0x7f6a600b3d60 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a6546d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6a60162dd0 con 0x7f6a6015ed90 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6a6015e9b0 0x7f6a600d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53648/0 (socket says 172.21.15.106:53648) 2024-04-13T03:09:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6a600b3d60 0x7f6a600d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5effd640 1 -- 172.21.15.106:0/4185680851 learned_addr learned my addr 172.21.15.106:0/4185680851 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5e7fc640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a6015ed90 0x7f6a600d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/4185680851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a6015ed90 msgr2=0x7f6a600d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5f7fe640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a6015ed90 0x7f6a600d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/4185680851 >> v1:172.21.15.106:6789/0 conn(0x7f6a6015e9b0 legacy=0x7f6a600d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5dffb640 1 -- 172.21.15.106:0/4185680851 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099528403 0 0) 0x7f6a600cfd20 con 0x7f6a6015e9b0 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.851+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/4185680851 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a601630b0 con 0x7f6a600b3d60 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.852+0000 7f6a5f7fe640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600b3d60 0x7f6a600d0200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0f2162f8792761 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.852+0000 7f6a5dffb640 1 -- 172.21.15.106:0/4185680851 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58039070 con 0x7f6a600b3d60 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.852+0000 7f6a5dffb640 1 -- 172.21.15.106:0/4185680851 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5802cce0 con 0x7f6a600b3d60 2024-04-13T03:09:34.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.852+0000 7f6a5dffb640 1 -- 172.21.15.106:0/4185680851 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58034660 con 0x7f6a600b3d60 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 -- 172.21.15.106:0/4185680851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600b3d60 msgr2=0x7f6a600d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600b3d60 0x7f6a600d0200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5802bf30 comp rx=0 tx=0).stop 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 -- 172.21.15.106:0/4185680851 shutdown_connections 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600b3d60 0x7f6a600d0200 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 --2- 172.21.15.106:0/4185680851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a6015ed90 0x7f6a600d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 -- 172.21.15.106:0/4185680851 >> 172.21.15.106:0/4185680851 conn(0x7f6a6008bf10 msgr2=0x7f6a600b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:34.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 -- 172.21.15.106:0/4185680851 shutdown_connections 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.853+0000 7f6a6546d640 1 -- 172.21.15.106:0/4185680851 wait complete. 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.854+0000 7f6a6546d640 1 Processor -- start 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.854+0000 7f6a6546d640 1 -- start start 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a600b3d60 0x7f6a601f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6015e9b0 0x7f6a601601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60162800 con 0x7f6a600cb800 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60162680 con 0x7f6a6015e9b0 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6a600d4d00 con 0x7f6a600b3d60 2024-04-13T03:09:34.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51806/0 (socket says 172.21.15.106:51806) 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/1484675790 learned_addr learned my addr 172.21.15.106:0/1484675790 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a600b3d60 msgr2=0x7f6a601f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a600b3d60 0x7f6a601f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6015e9b0 msgr2=0x7f6a601601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6015e9b0 0x7f6a601601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 -- 172.21.15.106:0/1484675790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a600cd780 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a5f7fe640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58044020 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5803a040 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a601f85c0 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58039050 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.855+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601f8b80 con 0x7f6a600cb800 2024-04-13T03:09:34.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.856+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a60068a10 con 0x7f6a600cb800 2024-04-13T03:09:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.857+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a58003bb0 con 0x7f6a600cb800 2024-04-13T03:09:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.857+0000 7f6a477fe640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 0x7f6a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.857+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a580ae410 con 0x7f6a600cb800 2024-04-13T03:09:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.857+0000 7f6a5effd640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 0x7f6a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:34.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.857+0000 7f6a5effd640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 0x7f6a4c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005880 tx=0x7f6a50007950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:34.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:34.862+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a5804e060 con 0x7f6a600cb800 2024-04-13T03:09:35.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.038+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a600d53a0 con 0x7f6a600cb800 2024-04-13T03:09:35.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.039+0000 7f6a477fe640 1 -- 172.21.15.106:0/1484675790 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f6a600cd780 con 0x7f6a600cb800 2024-04-13T03:09:35.040 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 msgr2=0x7f6a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 0x7f6a4c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005880 tx=0x7f6a50007950 comp rx=0 tx=0).stop 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 msgr2=0x7f6a600cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a58004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 shutdown_connections 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6a4c06c8f0 0x7f6a4c06edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a600cb800 0x7f6a600cbcb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6015e9b0 0x7f6a601601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 --2- 172.21.15.106:0/1484675790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6a600b3d60 0x7f6a601f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 >> 172.21.15.106:0/1484675790 conn(0x7f6a6008bf10 msgr2=0x7f6a600b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 shutdown_connections 2024-04-13T03:09:35.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.041+0000 7f6a6546d640 1 -- 172.21.15.106:0/1484675790 wait complete. 2024-04-13T03:09:35.052 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:09:35.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.155+0000 7fea4b57b640 1 Processor -- start 2024-04-13T03:09:35.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.155+0000 7fea4b57b640 1 -- start start 2024-04-13T03:09:35.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea4b57b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea4406f200 0x7fea4406f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea4b57b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406b8e0 0x7fea4406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea4b57b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea440b5090 con 0x7fea440af800 2024-04-13T03:09:35.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea4b57b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fea440b3ff0 con 0x7fea4406f200 2024-04-13T03:09:35.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea4b57b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fea440b42d0 con 0x7fea4406b8e0 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea4406f200 0x7fea4406f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea4406f200 0x7fea4406f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51820/0 (socket says 172.21.15.106:51820) 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 -- 172.21.15.106:0/86795193 learned_addr learned my addr 172.21.15.106:0/86795193 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea49b18640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406b8e0 0x7fea4406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 -- 172.21.15.106:0/86795193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406b8e0 msgr2=0x7fea4406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406b8e0 0x7fea4406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.156+0000 7fea48b16640 1 -- 172.21.15.106:0/86795193 >> v1:172.21.15.114:6789/0 conn(0x7fea440af800 legacy=0x7fea4415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.157+0000 7fea48b16640 1 -- 172.21.15.106:0/86795193 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea440b45b0 con 0x7fea4406f200 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.157+0000 7fea48b16640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4406f200 0x7fea4406f690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea3400bb10 tx=0x7fea3402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b2a916d42e07ab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.157+0000 7fea3bfff640 1 -- 172.21.15.106:0/86795193 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3403b070 con 0x7fea4406f200 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.157+0000 7fea3bfff640 1 -- 172.21.15.106:0/86795193 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea34036440 con 0x7fea4406f200 2024-04-13T03:09:35.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.157+0000 7fea3bfff640 1 -- 172.21.15.106:0/86795193 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34036810 con 0x7fea4406f200 2024-04-13T03:09:35.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 -- 172.21.15.106:0/86795193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4406f200 msgr2=0x7fea4406f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4406f200 0x7fea4406f690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea3400bb10 tx=0x7fea3402e150 comp rx=0 tx=0).stop 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 -- 172.21.15.106:0/86795193 shutdown_connections 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406b8e0 0x7fea4406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 --2- 172.21.15.106:0/86795193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea4406f200 0x7fea4406f690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 -- 172.21.15.106:0/86795193 >> 172.21.15.106:0/86795193 conn(0x7fea440b2600 msgr2=0x7fea440d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 -- 172.21.15.106:0/86795193 shutdown_connections 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.158+0000 7fea4b57b640 1 -- 172.21.15.106:0/86795193 wait complete. 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.159+0000 7fea4b57b640 1 Processor -- start 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.159+0000 7fea4b57b640 1 -- start start 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 0x7fea440aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 0x7fea4415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea440b3cc0 con 0x7fea440af800 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fea440b4b70 con 0x7fea4406b8e0 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea4b57b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fea4406fef0 con 0x7fea4406f200 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51836/0 (socket says 172.21.15.106:51836) 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 -- 172.21.15.106:0/2610678415 learned_addr learned my addr 172.21.15.106:0/2610678415 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49317640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 0x7fea440aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea48b16640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 0x7fea4415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 -- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 msgr2=0x7fea4415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 0x7fea4415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 -- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 msgr2=0x7fea440aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 0x7fea440aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 -- 172.21.15.106:0/2610678415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea441f5be0 con 0x7fea440af800 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49317640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 0x7fea440aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:35.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea48b16640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 0x7fea4415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.160+0000 7fea49b18640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e7f0 tx=0x7fea3c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.161+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c017070 con 0x7fea440af800 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.161+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c00ee40 con 0x7fea440af800 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.161+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4415c730 con 0x7fea440af800 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.161+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00cb80 con 0x7fea440af800 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.161+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4415ccf0 con 0x7fea440af800 2024-04-13T03:09:35.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.162+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea44068a10 con 0x7fea440af800 2024-04-13T03:09:35.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.162+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea3c01a070 con 0x7fea440af800 2024-04-13T03:09:35.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.163+0000 7fea39ffb640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.163+0000 7fea49317640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.163+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fea3c08c010 con 0x7fea440af800 2024-04-13T03:09:35.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.163+0000 7fea49317640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 0x7fea2806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea40001e00 tx=0x7fea40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.167+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea3c05a190 con 0x7fea440af800 2024-04-13T03:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.341+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fea440cf3f0 con 0x7fea440af800 2024-04-13T03:09:35.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.341+0000 7fea39ffb640 1 -- 172.21.15.106:0/2610678415 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fea3c059a90 con 0x7fea440af800 2024-04-13T03:09:35.343 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 msgr2=0x7fea2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 0x7fea2806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea40001e00 tx=0x7fea40009040 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 msgr2=0x7fea4415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e7f0 tx=0x7fea3c00ebf0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 shutdown_connections 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fea4406f200 0x7fea4415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea440af800 0x7fea4415f150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 --2- 172.21.15.106:0/2610678415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea4406b8e0 0x7fea440aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.343+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 >> 172.21.15.106:0/2610678415 conn(0x7fea440b2600 msgr2=0x7fea4406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.344+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 shutdown_connections 2024-04-13T03:09:35.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.344+0000 7fea4b57b640 1 -- 172.21.15.106:0/2610678415 wait complete. 2024-04-13T03:09:35.355 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:09:35.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.457+0000 7fc0cd7aa640 1 Processor -- start 2024-04-13T03:09:35.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.457+0000 7fc0cd7aa640 1 -- start start 2024-04-13T03:09:35.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0cd7aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc0c815e9b0 0x7fc0c80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0cd7aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0cd7aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80cfd20 con 0x7fc0c815ed90 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0cd7aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c8162af0 con 0x7fc0c815e9b0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0cd7aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c8162dd0 con 0x7fc0c80b3d60 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc0c815e9b0 0x7fc0c80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c67fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc0c815ed90 0x7fc0c80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53664/0 (socket says 172.21.15.106:53664) 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c67fc640 1 -- 172.21.15.106:0/2044767143 learned_addr learned my addr 172.21.15.106:0/2044767143 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c77fe640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c6ffd640 1 -- 172.21.15.106:0/2044767143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 msgr2=0x7fc0c80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c6ffd640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c6ffd640 1 -- 172.21.15.106:0/2044767143 >> v1:172.21.15.106:6789/0 conn(0x7fc0c815ed90 legacy=0x7fc0c80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c5ffb640 1 -- 172.21.15.106:0/2044767143 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091469158 0 0) 0x7fc0c80cfd20 con 0x7fc0c815ed90 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.458+0000 7fc0c6ffd640 1 -- 172.21.15.106:0/2044767143 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c81630b0 con 0x7fc0c815e9b0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.459+0000 7fc0c6ffd640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c815e9b0 0x7fc0c80d2300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009d90 tx=0x7fc0b802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b0235c07060a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.459+0000 7fc0c5ffb640 1 -- 172.21.15.106:0/2044767143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8039070 con 0x7fc0c815e9b0 2024-04-13T03:09:35.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.459+0000 7fc0c5ffb640 1 -- 172.21.15.106:0/2044767143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b802cce0 con 0x7fc0c815e9b0 2024-04-13T03:09:35.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.459+0000 7fc0c5ffb640 1 -- 172.21.15.106:0/2044767143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8034660 con 0x7fc0c815e9b0 2024-04-13T03:09:35.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/2044767143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c815e9b0 msgr2=0x7fc0c80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c815e9b0 0x7fc0c80d2300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009d90 tx=0x7fc0b802bf30 comp rx=0 tx=0).stop 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/2044767143 shutdown_connections 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/2044767143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c815e9b0 0x7fc0c80d2300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/2044767143 >> 172.21.15.106:0/2044767143 conn(0x7fc0c808bf10 msgr2=0x7fc0c80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/2044767143 shutdown_connections 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.460+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/2044767143 wait complete. 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.461+0000 7fc0cd7aa640 1 Processor -- start 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.461+0000 7fc0cd7aa640 1 -- start start 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0c815e9b0 0x7fc0c815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8162800 con 0x7fc0c8155e20 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8162680 con 0x7fc0c815e9b0 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0cd7aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d4d00 con 0x7fc0c80b3d60 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51856/0 (socket says 172.21.15.106:51856) 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 -- 172.21.15.106:0/835633001 learned_addr learned my addr 172.21.15.106:0/835633001 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c6ffd640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c67fc640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0c815e9b0 0x7fc0c815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 -- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 msgr2=0x7fc0c80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 -- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0c815e9b0 msgr2=0x7fc0c815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0c815e9b0 0x7fc0c815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 -- 172.21.15.106:0/835633001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80cbde0 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.462+0000 7fc0c77fe640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c5c0 tx=0x7fc0c000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.463+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0015070 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.463+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c0004220 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.463+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c80cc0c0 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.463+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0003610 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.463+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c80cc680 con 0x7fc0c8155e20 2024-04-13T03:09:35.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.464+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8068a10 con 0x7fc0c8155e20 2024-04-13T03:09:35.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.464+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0c00037b0 con 0x7fc0c8155e20 2024-04-13T03:09:35.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.464+0000 7fc0af7fe640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 0x7fc0b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.464+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc0c008bba0 con 0x7fc0c8155e20 2024-04-13T03:09:35.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.464+0000 7fc0c6ffd640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 0x7fc0b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.465+0000 7fc0c6ffd640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 0x7fc0b406ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8036000 tx=0x7fc0b8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.469+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0c0059640 con 0x7fc0c8155e20 2024-04-13T03:09:35.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.644+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0c815d440 con 0x7fc0c8155e20 2024-04-13T03:09:35.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.644+0000 7fc0af7fe640 1 -- 172.21.15.106:0/835633001 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc0c008b030 con 0x7fc0c8155e20 2024-04-13T03:09:35.645 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:09:35.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 msgr2=0x7fc0b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 0x7fc0b406ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8036000 tx=0x7fc0b8030000 comp rx=0 tx=0).stop 2024-04-13T03:09:35.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 msgr2=0x7fc0c81552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c5c0 tx=0x7fc0c000ca80 comp rx=0 tx=0).stop 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 shutdown_connections 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0b406c920 0x7fc0b406ede0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0c8155e20 0x7fc0c81552d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0c815e9b0 0x7fc0c815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 --2- 172.21.15.106:0/835633001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0c80b3d60 0x7fc0c80cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 >> 172.21.15.106:0/835633001 conn(0x7fc0c808bf10 msgr2=0x7fc0c80b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 shutdown_connections 2024-04-13T03:09:35.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.646+0000 7fc0cd7aa640 1 -- 172.21.15.106:0/835633001 wait complete. 2024-04-13T03:09:35.658 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:09:35.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.758+0000 7f8878b08640 1 Processor -- start 2024-04-13T03:09:35.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f8878b08640 1 -- start start 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f8878b08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f887406db90 0x7f8874072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f8878b08640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f887406c880 con 0x7f8874145f60 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f8878b08640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f887406caf0 con 0x7f887406e020 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f8878b08640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f887406cdd0 con 0x7f887406db90 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f886bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f887406db90 0x7f8874072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f887259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8874145f60 0x7f887406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53678/0 (socket says 172.21.15.106:53678) 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f887259c640 1 -- 172.21.15.106:0/319858413 learned_addr learned my addr 172.21.15.106:0/319858413 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.759+0000 7f886bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f887406db90 0x7f8874072e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51858/0 (socket says 172.21.15.106:51858) 2024-04-13T03:09:35.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.760+0000 7f886bfff640 1 -- 172.21.15.106:0/319858413 >> v1:172.21.15.114:6789/0 conn(0x7f887406e020 legacy=0x7f887415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:35.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.760+0000 7f886bfff640 1 -- 172.21.15.106:0/319858413 >> v1:172.21.15.106:6789/0 conn(0x7f8874145f60 legacy=0x7f887406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.760+0000 7f886bfff640 1 -- 172.21.15.106:0/319858413 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887406d0b0 con 0x7f887406db90 2024-04-13T03:09:35.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.760+0000 7f8871d9b640 1 -- 172.21.15.106:0/319858413 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532206685 0 0) 0x7f887406c880 con 0x7f8874145f60 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.760+0000 7f886bfff640 1 --2- 172.21.15.106:0/319858413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f8874072e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097a0 tx=0x7f885c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea07fe35bd9c6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8871d9b640 1 -- 172.21.15.106:0/319858413 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c035020 con 0x7f887406db90 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8871d9b640 1 -- 172.21.15.106:0/319858413 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c031830 con 0x7f887406db90 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8871d9b640 1 -- 172.21.15.106:0/319858413 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c031c00 con 0x7f887406db90 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 -- 172.21.15.106:0/319858413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 msgr2=0x7f8874072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 --2- 172.21.15.106:0/319858413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f8874072e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097a0 tx=0x7f885c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 -- 172.21.15.106:0/319858413 shutdown_connections 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 --2- 172.21.15.106:0/319858413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f8874072e10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 -- 172.21.15.106:0/319858413 >> 172.21.15.106:0/319858413 conn(0x7f887406f0d0 msgr2=0x7f887406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 -- 172.21.15.106:0/319858413 shutdown_connections 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.761+0000 7f8878b08640 1 -- 172.21.15.106:0/319858413 wait complete. 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.762+0000 7f8878b08640 1 Processor -- start 2024-04-13T03:09:35.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 -- start start 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 0x7f88740b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 0x7f88740cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f88740cdaa0 con 0x7f887406db90 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8874146340 con 0x7f887406e020 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8878b08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f88740cdc20 con 0x7f8874145f60 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f887259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f887259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51870/0 (socket says 172.21.15.106:51870) 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f887259c640 1 -- 172.21.15.106:0/4032066287 learned_addr learned my addr 172.21.15.106:0/4032066287 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.763+0000 7f8872d9d640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 0x7f88740cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f886bfff640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 0x7f88740b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 -- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 msgr2=0x7f88740cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 0x7f88740cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 -- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 msgr2=0x7f88740b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 0x7f88740b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 -- 172.21.15.106:0/4032066287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88740cb340 con 0x7f887406db90 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f8872d9d640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 0x7f88740cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f886bfff640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 0x7f88740b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f887259c640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f886400c500 tx=0x7f886400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864018070 con 0x7f887406db90 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8864007580 con 0x7f887406db90 2024-04-13T03:09:35.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88740cb620 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864004a20 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.764+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741e7420 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.765+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8874068a10 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.765+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f886401b070 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.766+0000 7f886affd640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 0x7f885406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.766+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f886408b160 con 0x7f887406db90 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.766+0000 7f886bfff640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 0x7f885406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:35.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.766+0000 7f886bfff640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 0x7f885406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f885c036000 tx=0x7f885c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:35.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.770+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88640592e0 con 0x7f887406db90 2024-04-13T03:09:35.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.945+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f88740b1640 con 0x7f887406db90 2024-04-13T03:09:35.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.946+0000 7f886affd640 1 -- 172.21.15.106:0/4032066287 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f8864058be0 con 0x7f887406db90 2024-04-13T03:09:35.947 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 msgr2=0x7f885406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 0x7f885406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f885c036000 tx=0x7f885c008030 comp rx=0 tx=0).stop 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 msgr2=0x7f887415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f886400c500 tx=0x7f886400c9c0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 shutdown_connections 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f885406c8f0 0x7f885406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8874145f60 0x7f88740cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f887406e020 0x7f88740b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 --2- 172.21.15.106:0/4032066287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887406db90 0x7f887415c090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:35.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 >> 172.21.15.106:0/4032066287 conn(0x7f887406f0d0 msgr2=0x7f887415f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:35.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 shutdown_connections 2024-04-13T03:09:35.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:35.948+0000 7f8878b08640 1 -- 172.21.15.106:0/4032066287 wait complete. 2024-04-13T03:09:35.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:09:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.060+0000 7f233291e640 1 Processor -- start 2024-04-13T03:09:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.060+0000 7f233291e640 1 -- start start 2024-04-13T03:09:36.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f233291e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 0x7f232c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f233291e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f232c06ed70 0x7f232c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f233291e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232c06e280 con 0x7f232c0d0820 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f233291e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f232c0d0c90 con 0x7f232c06ed70 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f233291e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f232c06e560 con 0x7f232c15e930 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f232c06ed70 0x7f232c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f232c06ed70 0x7f232c0d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51876/0 (socket says 172.21.15.106:51876) 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f2330ebb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f232c0d0820 0x7f232c0cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53694/0 (socket says 172.21.15.106:53694) 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 -- 172.21.15.106:0/1541662006 learned_addr learned my addr 172.21.15.106:0/1541662006 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232bfff640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 0x7f232c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 -- 172.21.15.106:0/1541662006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 msgr2=0x7f232c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 0x7f232c0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 -- 172.21.15.106:0/1541662006 >> v1:172.21.15.106:6789/0 conn(0x7f232c0d0820 legacy=0x7f232c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232b7fe640 1 -- 172.21.15.106:0/1541662006 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c0cccf0 con 0x7f232c06ed70 2024-04-13T03:09:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232affd640 1 -- 172.21.15.106:0/1541662006 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692138600 0 0) 0x7f232c06e280 con 0x7f232c0d0820 2024-04-13T03:09:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.061+0000 7f232bfff640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 0x7f232c0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:36.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.062+0000 7f232b7fe640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c0d02e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2320009870 tx=0x7f232002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfccaa11cc6584b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.062+0000 7f232affd640 1 -- 172.21.15.106:0/1541662006 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320034070 con 0x7f232c06ed70 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.062+0000 7f232affd640 1 -- 172.21.15.106:0/1541662006 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2320030840 con 0x7f232c06ed70 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.062+0000 7f232affd640 1 -- 172.21.15.106:0/1541662006 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320030c10 con 0x7f232c06ed70 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 -- 172.21.15.106:0/1541662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 msgr2=0x7f232c0d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c0d02e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2320009870 tx=0x7f232002b980 comp rx=0 tx=0).stop 2024-04-13T03:09:36.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 -- 172.21.15.106:0/1541662006 shutdown_connections 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c0d02e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 --2- 172.21.15.106:0/1541662006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c15e930 0x7f232c0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 -- 172.21.15.106:0/1541662006 >> 172.21.15.106:0/1541662006 conn(0x7f232c0cef80 msgr2=0x7f232c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 -- 172.21.15.106:0/1541662006 shutdown_connections 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.063+0000 7f233291e640 1 -- 172.21.15.106:0/1541662006 wait complete. 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.064+0000 7f233291e640 1 Processor -- start 2024-04-13T03:09:36.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.064+0000 7f233291e640 1 -- start start 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 0x7f232c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f232c15e930 0x7f232c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f232c06f1d0 con 0x7f232c06ed70 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f232c0cd2b0 con 0x7f232c15e930 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f232c0ce830 con 0x7f232c0d0820 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51882/0 (socket says 172.21.15.106:51882) 2024-04-13T03:09:36.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 -- 172.21.15.106:0/2632469889 learned_addr learned my addr 172.21.15.106:0/2632469889 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232b7fe640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 0x7f232c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f2330ebb640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f232c15e930 0x7f232c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 -- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 msgr2=0x7f232c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 0x7f232c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 -- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f232c15e930 msgr2=0x7f232c0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f232c15e930 0x7f232c0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 -- 172.21.15.106:0/2632469889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1604d0 con 0x7f232c06ed70 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232b7fe640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 0x7f232c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:36.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f232bfff640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f231c00c820 tx=0x7f231c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c015070 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c007a30 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.065+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c0ca910 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.066+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c007de0 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.066+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c0c6490 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.067+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c0c5fd0 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.067+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f231c018070 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.067+0000 7f2328ff9640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.067+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f231c08bd30 con 0x7f232c06ed70 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.067+0000 7f232b7fe640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.068+0000 7f232b7fe640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 0x7f231806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2320036000 tx=0x7f232003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.072+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f232c0c5fd0 con 0x7f232c06ed70 2024-04-13T03:09:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.257+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f232c15e4e0 con 0x7f232c06ed70 2024-04-13T03:09:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.257+0000 7f2328ff9640 1 -- 172.21.15.106:0/2632469889 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f232c15e4e0 con 0x7f232c06ed70 2024-04-13T03:09:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:09:36.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 msgr2=0x7f231806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 0x7f231806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2320036000 tx=0x7f232003b040 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 msgr2=0x7f232c15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f231c00c820 tx=0x7f231c00cce0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 shutdown_connections 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f232c0d0820 0x7f232c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f232c15e930 0x7f232c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 --2- 172.21.15.106:0/2632469889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f232c06ed70 0x7f232c15ff90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 >> 172.21.15.106:0/2632469889 conn(0x7f232c0cef80 msgr2=0x7f232c0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 shutdown_connections 2024-04-13T03:09:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.259+0000 7f233291e640 1 -- 172.21.15.106:0/2632469889 wait complete. 2024-04-13T03:09:36.271 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:09:36.271 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:09:36.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:36.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.426+0000 7f06ab636640 1 Processor -- start 2024-04-13T03:09:36.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.426+0000 7f06ab636640 1 -- start start 2024-04-13T03:09:36.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06ab636640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06a415e9c0 0x7f06a40ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06ab636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a415eda0 con 0x7f06a41624b0 2024-04-13T03:09:36.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06ab636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a4162c20 con 0x7f06a40b68d0 2024-04-13T03:09:36.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06ab636640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f06a40d27f0 con 0x7f06a415e9c0 2024-04-13T03:09:36.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a93d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06a415e9c0 0x7f06a40ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a8bd1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f06a41624b0 0x7f06a40b61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53706/0 (socket says 172.21.15.106:53706) 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a8bd1640 1 -- 172.21.15.106:0/1859116058 learned_addr learned my addr 172.21.15.106:0/1859116058 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a93d2640 1 -- 172.21.15.106:0/1859116058 >> v1:172.21.15.114:6789/0 conn(0x7f06a40b68d0 legacy=0x7f06a40d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a93d2640 1 -- 172.21.15.106:0/1859116058 >> v1:172.21.15.106:6789/0 conn(0x7f06a41624b0 legacy=0x7f06a40b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f069bfff640 1 -- 172.21.15.106:0/1859116058 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698178368 0 0) 0x7f06a415eda0 con 0x7f06a41624b0 2024-04-13T03:09:36.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.427+0000 7f06a93d2640 1 -- 172.21.15.106:0/1859116058 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4162f00 con 0x7f06a415e9c0 2024-04-13T03:09:36.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.428+0000 7f06a93d2640 1 --2- 172.21.15.106:0/1859116058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a415e9c0 0x7f06a40ba290 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06a00097a0 tx=0x7f06a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7943b220b1edb09 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.428+0000 7f069bfff640 1 -- 172.21.15.106:0/1859116058 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035020 con 0x7f06a415e9c0 2024-04-13T03:09:36.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.428+0000 7f069bfff640 1 -- 172.21.15.106:0/1859116058 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a0031830 con 0x7f06a415e9c0 2024-04-13T03:09:36.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.428+0000 7f069bfff640 1 -- 172.21.15.106:0/1859116058 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0031c00 con 0x7f06a415e9c0 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 -- 172.21.15.106:0/1859116058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a415e9c0 msgr2=0x7f06a40ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 --2- 172.21.15.106:0/1859116058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a415e9c0 0x7f06a40ba290 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06a00097a0 tx=0x7f06a002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 -- 172.21.15.106:0/1859116058 shutdown_connections 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 --2- 172.21.15.106:0/1859116058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a415e9c0 0x7f06a40ba290 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 -- 172.21.15.106:0/1859116058 >> 172.21.15.106:0/1859116058 conn(0x7f06a40d11c0 msgr2=0x7f06a40d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 -- 172.21.15.106:0/1859116058 shutdown_connections 2024-04-13T03:09:36.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.429+0000 7f06ab636640 1 -- 172.21.15.106:0/1859116058 wait complete. 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.430+0000 7f06ab636640 1 Processor -- start 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.430+0000 7f06ab636640 1 -- start start 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.430+0000 7f06ab636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 0x7f06a4159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a415e9c0 0x7f06a40c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4162910 con 0x7f06a41624b0 2024-04-13T03:09:36.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40d2a40 con 0x7f06a415e9c0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40d3670 con 0x7f06a40b68d0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51910/0 (socket says 172.21.15.106:51910) 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 -- 172.21.15.106:0/3107402095 learned_addr learned my addr 172.21.15.106:0/3107402095 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a8bd1640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a415e9c0 0x7f06a40c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a93d2640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 0x7f06a4159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 -- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 msgr2=0x7f06a4159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 0x7f06a4159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 -- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a415e9c0 msgr2=0x7f06a40c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a415e9c0 0x7f06a40c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 -- 172.21.15.106:0/3107402095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40c9500 con 0x7f06a41624b0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a93d2640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 0x7f06a4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06a9bd3640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c780 tx=0x7f069c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c017070 con 0x7f06a41624b0 2024-04-13T03:09:36.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c003b70 con 0x7f06a41624b0 2024-04-13T03:09:36.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c005070 con 0x7f06a41624b0 2024-04-13T03:09:36.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.431+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a40c97e0 con 0x7f06a41624b0 2024-04-13T03:09:36.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.432+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a40c91a0 con 0x7f06a41624b0 2024-04-13T03:09:36.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.432+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a4068a10 con 0x7f06a41624b0 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.433+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f069c01a070 con 0x7f06a41624b0 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.433+0000 7f0699ffb640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 0x7f068806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.433+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f069c08b390 con 0x7f06a41624b0 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.433+0000 7f06a93d2640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 0x7f068806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.434+0000 7f06a93d2640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 0x7f068806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06a0036000 tx=0x7f06a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.437+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f069c059510 con 0x7f06a41624b0 2024-04-13T03:09:36.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.611+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06a406bf40 con 0x7f06a41624b0 2024-04-13T03:09:36.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.612+0000 7f0699ffb640 1 -- 172.21.15.106:0/3107402095 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f069c058e10 con 0x7f06a41624b0 2024-04-13T03:09:36.614 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:36.614 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 msgr2=0x7f068806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 0x7f068806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06a0036000 tx=0x7f06a0008030 comp rx=0 tx=0).stop 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 msgr2=0x7f06a415a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c780 tx=0x7f069c00cc40 comp rx=0 tx=0).stop 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 shutdown_connections 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f068806c8f0 0x7f068806edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a41624b0 0x7f06a415a850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a415e9c0 0x7f06a40c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 --2- 172.21.15.106:0/3107402095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40b68d0 0x7f06a4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 >> 172.21.15.106:0/3107402095 conn(0x7f06a40d11c0 msgr2=0x7f06a406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 shutdown_connections 2024-04-13T03:09:36.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.614+0000 7f06ab636640 1 -- 172.21.15.106:0/3107402095 wait complete. 2024-04-13T03:09:36.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:09:36.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:36.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.729+0000 7ff538b8c640 1 Processor -- start 2024-04-13T03:09:36.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.729+0000 7ff538b8c640 1 -- start start 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff538b8c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff538b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff538b8c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff53415fe40 con 0x7ff534070bd0 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff538b8c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff53415f5a0 con 0x7ff5340707f0 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff538b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff53415f880 con 0x7ff5340cf610 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff53259c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.730+0000 7ff532d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff531d9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff534070bd0 0x7ff534162b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:49062/0 (socket says 172.21.15.106:49062) 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff532d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:38822/0 (socket says 172.21.15.106:38822) 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff531d9b640 1 -- 172.21.15.106:0/3079576214 learned_addr learned my addr 172.21.15.106:0/3079576214 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff53259c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:35854/0 (socket says 172.21.15.106:35854) 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff532d9d640 1 -- 172.21.15.106:0/3079576214 >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 msgr2=0x7ff53415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff532d9d640 1 --2- 172.21.15.106:0/3079576214 >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff532d9d640 1 -- 172.21.15.106:0/3079576214 >> v1:172.21.15.114:6789/0 conn(0x7ff534070bd0 legacy=0x7ff534162b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff53159a640 1 -- 172.21.15.106:0/3079576214 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253264536 0 0) 0x7ff53415fe40 con 0x7ff534070bd0 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff532d9d640 1 -- 172.21.15.106:0/3079576214 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53415fb60 con 0x7ff5340cf610 2024-04-13T03:09:36.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.731+0000 7ff53259c640 1 --2- 172.21.15.106:0/3079576214 >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff532d9d640 1 --2- 172.21.15.106:0/3079576214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0097a0 tx=0x7ff52c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6c158790449a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff53159a640 1 -- 172.21.15.106:0/3079576214 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0315a0 con 0x7ff5340cf610 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff53159a640 1 -- 172.21.15.106:0/3079576214 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52c037070 con 0x7ff5340cf610 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff53159a640 1 -- 172.21.15.106:0/3079576214 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c031960 con 0x7ff5340cf610 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 -- 172.21.15.106:0/3079576214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 msgr2=0x7ff5340cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 --2- 172.21.15.106:0/3079576214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0097a0 tx=0x7ff52c02b840 comp rx=0 tx=0).stop 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 -- 172.21.15.106:0/3079576214 shutdown_connections 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 --2- 172.21.15.106:0/3079576214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340cc540 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 --2- 172.21.15.106:0/3079576214 >> v2:172.21.15.114:3300/0 conn(0x7ff5340707f0 0x7ff53415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 -- 172.21.15.106:0/3079576214 >> 172.21.15.106:0/3079576214 conn(0x7ff5340d1650 msgr2=0x7ff5340d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 -- 172.21.15.106:0/3079576214 shutdown_connections 2024-04-13T03:09:36.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.732+0000 7ff538b8c640 1 -- 172.21.15.106:0/3079576214 wait complete. 2024-04-13T03:09:36.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 Processor -- start 2024-04-13T03:09:36.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 -- start start 2024-04-13T03:09:36.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5340707f0 0x7ff5340cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff534163230 con 0x7ff534070bd0 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5340cfaa0 con 0x7ff5340707f0 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.734+0000 7ff538b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5341633b0 con 0x7ff5340cf610 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51912/0 (socket says 172.21.15.106:51912) 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 -- 172.21.15.106:0/1581415166 learned_addr learned my addr 172.21.15.106:0/1581415166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff53259c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5340707f0 0x7ff5340cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff532d9d640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 -- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 msgr2=0x7ff5340c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 -- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5340707f0 msgr2=0x7ff5340cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5340707f0 0x7ff5340cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 -- 172.21.15.106:0/1581415166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5340ada90 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff531d9b640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c490 tx=0x7ff52800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528018070 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5280074e0 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528004a20 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5340c9f60 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.735+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5340ca520 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.736+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff534069120 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.737+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff52801b070 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.737+0000 7ff522ffd640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 0x7ff51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.737+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff52808b0d0 con 0x7ff534070bd0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.737+0000 7ff53259c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 0x7ff51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:36.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.738+0000 7ff53259c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 0x7ff51406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5340b0c40 tx=0x7ff52400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:36.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.741+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff528058b70 con 0x7ff534070bd0 2024-04-13T03:09:36.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.918+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5340c6aa0 con 0x7ff534070bd0 2024-04-13T03:09:36.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.918+0000 7ff522ffd640 1 -- 172.21.15.106:0/1581415166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ff52805ce70 con 0x7ff534070bd0 2024-04-13T03:09:36.919 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 msgr2=0x7ff51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 0x7ff51406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5340b0c40 tx=0x7ff52400b040 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 msgr2=0x7ff5340b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c490 tx=0x7ff52800c950 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 shutdown_connections 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff51406c8f0 0x7ff51406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5340cf610 0x7ff5340c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff534070bd0 0x7ff5340b0490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 --2- 172.21.15.106:0/1581415166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5340707f0 0x7ff5340cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:36.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 >> 172.21.15.106:0/1581415166 conn(0x7ff5340d1650 msgr2=0x7ff5340b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 shutdown_connections 2024-04-13T03:09:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:36.920+0000 7ff538b8c640 1 -- 172.21.15.106:0/1581415166 wait complete. 2024-04-13T03:09:36.932 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:09:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.036+0000 7f7f1135e640 1 Processor -- start 2024-04-13T03:09:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.036+0000 7f7f1135e640 1 -- start start 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f1135e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7f0c15e9b0 0x7f7f0c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f1135e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0c0cfd20 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f1135e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0c162af0 con 0x7f7f0c15ed90 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f1135e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7f0c162dd0 con 0x7f7f0c15e9b0 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7f0c0b3d60 0x7f7f0c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53722/0 (socket says 172.21.15.106:53722) 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0b7fe640 1 -- 172.21.15.106:0/1702726335 learned_addr learned my addr 172.21.15.106:0/1702726335 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0affd640 1 --2- 172.21.15.106:0/1702726335 >> v2:172.21.15.114:3300/0 conn(0x7f7f0c15e9b0 0x7f7f0c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722102633 0 0) 0x7f7f0c0cfd20 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ef8003520 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4010784350 0 0) 0x7f7f0c162af0 con 0x7f7f0c15ed90 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0c0cfd20 con 0x7f7f0c15ed90 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3049171733 0 0) 0x7f7ef8003520 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 >> v2:172.21.15.114:3300/0 conn(0x7f7f0c15e9b0 msgr2=0x7f7f0c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 --2- 172.21.15.106:0/1702726335 >> v2:172.21.15.114:3300/0 conn(0x7f7f0c15e9b0 0x7f7f0c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 >> v1:172.21.15.114:6789/0 conn(0x7f7f0c15ed90 legacy=0x7f7f0c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c1630b0 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f7f040019f0 con 0x7f7f0c0b3d60 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0affd640 1 --2- 172.21.15.106:0/1702726335 >> v2:172.21.15.114:3300/0 conn(0x7f7f0c15e9b0 0x7f7f0c0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.037+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 >> v1:172.21.15.106:6789/0 conn(0x7f7f0c0b3d60 legacy=0x7f7f0c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8003b90 0x7f7ef8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 0x7f7ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef80051c0 0x7f7ef800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c162dd0 con 0x7f7ef8004800 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef800bfe0 con 0x7f7ef80051c0 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef800c1e0 con 0x7f7ef8003b90 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 0x7f7ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0b7fe640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef80051c0 0x7f7ef800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0affd640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8003b90 0x7f7ef8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 -- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8003b90 msgr2=0x7f7ef8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8003b90 0x7f7ef8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 -- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef80051c0 msgr2=0x7f7ef800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef80051c0 0x7f7ef800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f03fff640 1 -- 172.21.15.106:0/1702726335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef8003520 con 0x7f7ef8004800 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.038+0000 7f7f0b7fe640 1 -- 172.21.15.106:0/1702726335 reap_dead start 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f03fff640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 0x7f7ef8004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40018f0 tx=0x7f7ef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec2af4243f67466 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4037070 con 0x7f7ef8004800 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef40052d0 con 0x7f7ef8004800 2024-04-13T03:09:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f0a7fc640 1 -- 172.21.15.106:0/1702726335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4001540 con 0x7f7ef8004800 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 -- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 msgr2=0x7f7ef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 0x7f7ef8004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40018f0 tx=0x7f7ef8005830 comp rx=0 tx=0).stop 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 -- 172.21.15.106:0/1702726335 shutdown_connections 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 --2- 172.21.15.106:0/1702726335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ef8004800 0x7f7ef8004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 -- 172.21.15.106:0/1702726335 >> 172.21.15.106:0/1702726335 conn(0x7f7f0c08bf10 msgr2=0x7f7f0c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 -- 172.21.15.106:0/1702726335 shutdown_connections 2024-04-13T03:09:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.039+0000 7f7f1135e640 1 -- 172.21.15.106:0/1702726335 wait complete. 2024-04-13T03:09:37.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 Processor -- start 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 -- start start 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8004800 0x7f7f0c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 0x7f7f0c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c162800 con 0x7f7f0c155e70 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c162680 con 0x7f7f0c15ff40 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f1135e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c06eb20 con 0x7f7ef8004800 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f03fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f03fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51932/0 (socket says 172.21.15.106:51932) 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f03fff640 1 -- 172.21.15.106:0/3604349069 learned_addr learned my addr 172.21.15.106:0/3604349069 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.041+0000 7f7f0affd640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8004800 0x7f7f0c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f0b7fe640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 0x7f7f0c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 -- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8004800 msgr2=0x7f7f0c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8004800 0x7f7f0c156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 -- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 msgr2=0x7f7f0c1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 0x7f7f0c1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 -- 172.21.15.106:0/3604349069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c156b60 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f0b7fe640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 0x7f7f0c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f03fff640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005040 tx=0x7f7ef4001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4042020 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef4038040 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c06cc90 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4037070 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.042+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c06d250 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.043+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0c068a10 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.044+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ef4045020 con 0x7f7f0c155e70 2024-04-13T03:09:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.044+0000 7f7f037fe640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 0x7f7eec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.044+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ef40ad330 con 0x7f7f0c155e70 2024-04-13T03:09:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.044+0000 7f7f0affd640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 0x7f7eec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.044+0000 7f7f0affd640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 0x7f7eec06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009ce0 tx=0x7f7efc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.048+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ef404c070 con 0x7f7f0c155e70 2024-04-13T03:09:37.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.222+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f0c0b3a60 con 0x7f7f0c155e70 2024-04-13T03:09:37.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.222+0000 7f7f037fe640 1 -- 172.21.15.106:0/3604349069 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7ef407ae60 con 0x7f7f0c155e70 2024-04-13T03:09:37.224 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 msgr2=0x7f7eec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 0x7f7eec06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009ce0 tx=0x7f7efc009190 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 msgr2=0x7f7f0c15fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005040 tx=0x7f7ef4001320 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 shutdown_connections 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eec06c8f0 0x7f7eec06edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0c15ff40 0x7f7f0c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f0c155e70 0x7f7f0c15fa00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 --2- 172.21.15.106:0/3604349069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ef8004800 0x7f7f0c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 >> 172.21.15.106:0/3604349069 conn(0x7f7f0c08bf10 msgr2=0x7f7f0c0d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 shutdown_connections 2024-04-13T03:09:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.224+0000 7f7f1135e640 1 -- 172.21.15.106:0/3604349069 wait complete. 2024-04-13T03:09:37.236 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:09:37.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.337+0000 7f094451e640 1 Processor -- start 2024-04-13T03:09:37.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.337+0000 7f094451e640 1 -- start start 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f094451e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f093c0b5350 0x7f093c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f094451e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093c0b5780 con 0x7f093c15e930 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f094451e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093c06e280 con 0x7f093c0d09f0 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f094451e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f093c06f2b0 con 0x7f093c0b5350 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f09422ba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f093c15e930 0x7f093c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53738/0 (socket says 172.21.15.106:53738) 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.338+0000 7f09422ba640 1 -- 172.21.15.106:0/1231162837 learned_addr learned my addr 172.21.15.106:0/1231162837 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0942abb640 1 --2- 172.21.15.106:0/1231162837 >> v2:172.21.15.114:3300/0 conn(0x7f093c0b5350 0x7f093c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725648750 0 0) 0x7f093c0b5780 con 0x7f093c15e930 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0928003520 con 0x7f093c15e930 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814336459 0 0) 0x7f0928003520 con 0x7f093c15e930 2024-04-13T03:09:37.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 >> v2:172.21.15.114:3300/0 conn(0x7f093c0b5350 msgr2=0x7f093c06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 --2- 172.21.15.106:0/1231162837 >> v2:172.21.15.114:3300/0 conn(0x7f093c0b5350 0x7f093c06ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 >> v1:172.21.15.114:6789/0 conn(0x7f093c0d09f0 legacy=0x7f093c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c06e560 con 0x7f093c15e930 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f09380019f0 con 0x7f093c15e930 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 >> v1:172.21.15.106:6789/0 conn(0x7f093c15e930 legacy=0x7f093c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09280039b0 0x7f0928003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 0x7f0928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09280051c0 0x7f092800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f093c06f2b0 con 0x7f0928004800 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f092800c160 con 0x7f09280039b0 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f092800c360 con 0x7f09280051c0 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 0x7f0928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f09422ba640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09280039b0 0x7f0928003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0942abb640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09280051c0 0x7f092800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 -- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09280051c0 msgr2=0x7f092800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09280051c0 0x7f092800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 -- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09280039b0 msgr2=0x7f0928003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09280039b0 0x7f0928003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0941ab9640 1 -- 172.21.15.106:0/1231162837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0928003520 con 0x7f0928004800 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.339+0000 7f0942abb640 1 -- 172.21.15.106:0/1231162837 reap_dead start 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.340+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 0x7f0928004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f092c001750 tx=0x7f0928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b1ac04b062a05a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.340+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c001370 con 0x7f0928004800 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.340+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c0052d0 con 0x7f0928004800 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.340+0000 7f09412b8640 1 -- 172.21.15.106:0/1231162837 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c003830 con 0x7f0928004800 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 -- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 msgr2=0x7f0928004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 0x7f0928004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f092c001750 tx=0x7f0928005830 comp rx=0 tx=0).stop 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 -- 172.21.15.106:0/1231162837 shutdown_connections 2024-04-13T03:09:37.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 --2- 172.21.15.106:0/1231162837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0928004800 0x7f0928004c80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 -- 172.21.15.106:0/1231162837 >> 172.21.15.106:0/1231162837 conn(0x7f093c0cef80 msgr2=0x7f093c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 -- 172.21.15.106:0/1231162837 shutdown_connections 2024-04-13T03:09:37.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.341+0000 7f094451e640 1 -- 172.21.15.106:0/1231162837 wait complete. 2024-04-13T03:09:37.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.342+0000 7f094451e640 1 Processor -- start 2024-04-13T03:09:37.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.342+0000 7f094451e640 1 -- start start 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 0x7f093c14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 0x7f093c1e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0cd2b0 con 0x7f093c160770 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0ccfd0 con 0x7f093c1e7010 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f094451e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f093c0ce830 con 0x7f0928004800 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51948/0 (socket says 172.21.15.106:51948) 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 -- 172.21.15.106:0/1223814325 learned_addr learned my addr 172.21.15.106:0/1223814325 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0942abb640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 0x7f093c1e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f09422ba640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 0x7f093c14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 -- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 msgr2=0x7f093c14a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 0x7f093c14a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 -- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 msgr2=0x7f093c1e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 0x7f093c1e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 -- 172.21.15.106:0/1223814325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c1612e0 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0942abb640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 0x7f093c1e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f09422ba640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 0x7f093c14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0941ab9640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f092c000c00 tx=0x7f092c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.343+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c041020 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.344+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c005480 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.344+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c1615c0 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.344+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c007a30 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.344+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c15d700 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.344+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c068a10 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.345+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f092c044020 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.345+0000 7f0932ffd640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.345+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f092c0ad910 con 0x7f093c160770 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.345+0000 7f09422ba640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.346+0000 7f09422ba640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 0x7f092006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0938001b50 tx=0x7f0938006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.350+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f092c04b030 con 0x7f093c160770 2024-04-13T03:09:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.525+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f093c0cfce0 con 0x7f093c160770 2024-04-13T03:09:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.525+0000 7f0932ffd640 1 -- 172.21.15.106:0/1223814325 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f092c0303c0 con 0x7f093c160770 2024-04-13T03:09:37.526 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:09:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 msgr2=0x7f092006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 0x7f092006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0938001b50 tx=0x7f0938006040 comp rx=0 tx=0).stop 2024-04-13T03:09:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 msgr2=0x7f093c160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f092c000c00 tx=0x7f092c001120 comp rx=0 tx=0).stop 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 shutdown_connections 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093c1e7010 0x7f093c1e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093c160770 0x7f093c160bd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 --2- 172.21.15.106:0/1223814325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0928004800 0x7f093c14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 >> 172.21.15.106:0/1223814325 conn(0x7f093c0cef80 msgr2=0x7f093c06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 shutdown_connections 2024-04-13T03:09:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.527+0000 7f094451e640 1 -- 172.21.15.106:0/1223814325 wait complete. 2024-04-13T03:09:37.539 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:09:37.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.641+0000 7f22a21b4640 1 Processor -- start 2024-04-13T03:09:37.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.641+0000 7f22a21b4640 1 -- start start 2024-04-13T03:09:37.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f22a21b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f229c0b1c30 0x7f229c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f22a21b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0b5cf0 con 0x7f229c070bb0 2024-04-13T03:09:37.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f22a21b4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0b5fd0 con 0x7f229c0b54a0 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f22a21b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f229c0b62b0 con 0x7f229c0b1c30 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f229bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f229c070bb0 0x7f229c06d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53752/0 (socket says 172.21.15.106:53752) 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f229bfff640 1 -- 172.21.15.106:0/2707167688 learned_addr learned my addr 172.21.15.106:0/2707167688 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f229b7fe640 1 --2- 172.21.15.106:0/2707167688 >> v2:172.21.15.114:3300/0 conn(0x7f229c0b1c30 0x7f229c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f229a7fc640 1 -- 172.21.15.106:0/2707167688 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532170913 0 0) 0x7f229c0b5cf0 con 0x7f229c070bb0 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.642+0000 7f229a7fc640 1 -- 172.21.15.106:0/2707167688 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2284003520 con 0x7f229c070bb0 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.643+0000 7f229b7fe640 1 -- 172.21.15.106:0/2707167688 >> v1:172.21.15.114:6789/0 conn(0x7f229c0b54a0 legacy=0x7f229c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.643+0000 7f229b7fe640 1 -- 172.21.15.106:0/2707167688 >> v1:172.21.15.106:6789/0 conn(0x7f229c070bb0 legacy=0x7f229c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.643+0000 7f229b7fe640 1 -- 172.21.15.106:0/2707167688 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c0b6590 con 0x7f229c0b1c30 2024-04-13T03:09:37.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.643+0000 7f229b7fe640 1 --2- 172.21.15.106:0/2707167688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b1c30 0x7f229c15ee40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097a0 tx=0x7f228c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4889118e9b9c4fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f229a7fc640 1 -- 172.21.15.106:0/2707167688 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c034070 con 0x7f229c0b1c30 2024-04-13T03:09:37.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f229a7fc640 1 -- 172.21.15.106:0/2707167688 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c030610 con 0x7f229c0b1c30 2024-04-13T03:09:37.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f229a7fc640 1 -- 172.21.15.106:0/2707167688 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c0309c0 con 0x7f229c0b1c30 2024-04-13T03:09:37.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 -- 172.21.15.106:0/2707167688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b1c30 msgr2=0x7f229c15ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 --2- 172.21.15.106:0/2707167688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b1c30 0x7f229c15ee40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097a0 tx=0x7f228c02b960 comp rx=0 tx=0).stop 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 -- 172.21.15.106:0/2707167688 shutdown_connections 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 --2- 172.21.15.106:0/2707167688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b1c30 0x7f229c15ee40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 -- 172.21.15.106:0/2707167688 >> 172.21.15.106:0/2707167688 conn(0x7f229c071870 msgr2=0x7f229c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 -- 172.21.15.106:0/2707167688 shutdown_connections 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.644+0000 7f22a21b4640 1 -- 172.21.15.106:0/2707167688 wait complete. 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.645+0000 7f22a21b4640 1 Processor -- start 2024-04-13T03:09:37.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.645+0000 7f22a21b4640 1 -- start start 2024-04-13T03:09:37.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c070bb0 0x7f229c150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b54a0 0x7f229c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f229c06def0 con 0x7f229c0b1c30 2024-04-13T03:09:37.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f229c06dd70 con 0x7f229c0b54a0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f22a21b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f229c06e070 con 0x7f229c070bb0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f229affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f229affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51962/0 (socket says 172.21.15.106:51962) 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f229affd640 1 -- 172.21.15.106:0/3422211160 learned_addr learned my addr 172.21.15.106:0/3422211160 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f229b7fe640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c070bb0 0x7f229c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.646+0000 7f229bfff640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b54a0 0x7f229c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 -- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c070bb0 msgr2=0x7f229c150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c070bb0 0x7f229c150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 -- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b54a0 msgr2=0x7f229c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b54a0 0x7f229c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 -- 172.21.15.106:0/3422211160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c153f10 con 0x7f229c0b1c30 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f229affd640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f229000c3c0 tx=0x7f229000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900052d0 con 0x7f229c0b1c30 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2290005490 con 0x7f229c0b1c30 2024-04-13T03:09:37.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c150c90 con 0x7f229c0b1c30 2024-04-13T03:09:37.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290007260 con 0x7f229c0b1c30 2024-04-13T03:09:37.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.647+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c151c60 con 0x7f229c0b1c30 2024-04-13T03:09:37.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.648+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c068a10 con 0x7f229c0b1c30 2024-04-13T03:09:37.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.648+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2290007400 con 0x7f229c0b1c30 2024-04-13T03:09:37.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.649+0000 7f2283fff640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 0x7f228806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.649+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f229008b200 con 0x7f229c0b1c30 2024-04-13T03:09:37.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.649+0000 7f229b7fe640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 0x7f228806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.649+0000 7f229b7fe640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 0x7f228806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f228c036000 tx=0x7f228c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.653+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2290058d30 con 0x7f229c0b1c30 2024-04-13T03:09:37.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.826+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f229c070720 con 0x7f229c0b1c30 2024-04-13T03:09:37.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.827+0000 7f2283fff640 1 -- 172.21.15.106:0/3422211160 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f229005d030 con 0x7f229c0b1c30 2024-04-13T03:09:37.828 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:37.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 msgr2=0x7f228806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 0x7f228806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f228c036000 tx=0x7f228c008030 comp rx=0 tx=0).stop 2024-04-13T03:09:37.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 msgr2=0x7f229c1523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f229000c3c0 tx=0x7f229000c880 comp rx=0 tx=0).stop 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 shutdown_connections 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f228806c920 0x7f228806ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f229c0b54a0 0x7f229c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f229c0b1c30 0x7f229c1523c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 --2- 172.21.15.106:0/3422211160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f229c070bb0 0x7f229c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 >> 172.21.15.106:0/3422211160 conn(0x7f229c071870 msgr2=0x7f229c15ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 shutdown_connections 2024-04-13T03:09:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.829+0000 7f22a21b4640 1 -- 172.21.15.106:0/3422211160 wait complete. 2024-04-13T03:09:37.841 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:09:37.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.940+0000 7f86daa2d640 1 Processor -- start 2024-04-13T03:09:37.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.940+0000 7f86daa2d640 1 -- start start 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86daa2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d4162550 0x7f86d40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86daa2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86d40d1630 0x7f86d40d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86daa2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d40cfe70 con 0x7f86d415ea80 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86daa2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f86d40d1f80 con 0x7f86d4162550 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86daa2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f86d415ee60 con 0x7f86d40d1630 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d4162550 0x7f86d40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d4162550 0x7f86d40d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51972/0 (socket says 172.21.15.106:51972) 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 -- 172.21.15.106:0/3149684428 learned_addr learned my addr 172.21.15.106:0/3149684428 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 -- 172.21.15.106:0/3149684428 >> v2:172.21.15.114:3300/0 conn(0x7f86d40d1630 msgr2=0x7f86d40d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d8fca640 1 --2- 172.21.15.106:0/3149684428 >> v2:172.21.15.114:3300/0 conn(0x7f86d40d1630 0x7f86d40d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 --2- 172.21.15.106:0/3149684428 >> v2:172.21.15.114:3300/0 conn(0x7f86d40d1630 0x7f86d40d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 -- 172.21.15.106:0/3149684428 >> v1:172.21.15.114:6789/0 conn(0x7f86d415ea80 legacy=0x7f86d40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.941+0000 7f86d37fe640 1 -- 172.21.15.106:0/3149684428 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d4162af0 con 0x7f86d4162550 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86d37fe640 1 --2- 172.21.15.106:0/3149684428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d4162550 0x7f86d40d22b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86d408e0a0 tx=0x7f86c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a05b235efd8888c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86d2ffd640 1 -- 172.21.15.106:0/3149684428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8035020 con 0x7f86d4162550 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86d2ffd640 1 -- 172.21.15.106:0/3149684428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c8031870 con 0x7f86d4162550 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86d2ffd640 1 -- 172.21.15.106:0/3149684428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8031c40 con 0x7f86d4162550 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86daa2d640 1 -- 172.21.15.106:0/3149684428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d4162550 msgr2=0x7f86d40d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.942+0000 7f86daa2d640 1 --2- 172.21.15.106:0/3149684428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d4162550 0x7f86d40d22b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86d408e0a0 tx=0x7f86c802b910 comp rx=0 tx=0).stop 2024-04-13T03:09:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 -- 172.21.15.106:0/3149684428 shutdown_connections 2024-04-13T03:09:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 --2- 172.21.15.106:0/3149684428 >> v2:172.21.15.114:3300/0 conn(0x7f86d40d1630 0x7f86d40d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 --2- 172.21.15.106:0/3149684428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d4162550 0x7f86d40d22b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 -- 172.21.15.106:0/3149684428 >> 172.21.15.106:0/3149684428 conn(0x7f86d409da50 msgr2=0x7f86d40b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 -- 172.21.15.106:0/3149684428 shutdown_connections 2024-04-13T03:09:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.943+0000 7f86daa2d640 1 -- 172.21.15.106:0/3149684428 wait complete. 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.944+0000 7f86daa2d640 1 Processor -- start 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.944+0000 7f86daa2d640 1 -- start start 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d415ea80 0x7f86d40c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d4162550 0x7f86d415da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4163390 con 0x7f86d40d1630 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41630b0 con 0x7f86d415ea80 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f86d40d0320 con 0x7f86d4162550 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51974/0 (socket says 172.21.15.106:51974) 2024-04-13T03:09:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 -- 172.21.15.106:0/210487851 learned_addr learned my addr 172.21.15.106:0/210487851 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d37fe640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d415ea80 0x7f86d40c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d8fca640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d4162550 0x7f86d415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 -- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d4162550 msgr2=0x7f86d415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d4162550 0x7f86d415da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 -- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d415ea80 msgr2=0x7f86d40c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d415ea80 0x7f86d40c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 -- 172.21.15.106:0/210487851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d40c92b0 con 0x7f86d40d1630 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d3fff640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86c400c410 tx=0x7f86c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4017070 con 0x7f86d40d1630 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c40035f0 con 0x7f86d40d1630 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d406b980 con 0x7f86d40d1630 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003a20 con 0x7f86d40d1630 2024-04-13T03:09:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.945+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d406bf40 con 0x7f86d40d1630 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.946+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d4068a10 con 0x7f86d40d1630 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.947+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86c401a070 con 0x7f86d40d1630 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.947+0000 7f86d0ff9640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.947+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f86c408b340 con 0x7f86d40d1630 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.947+0000 7f86d37fe640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.948+0000 7f86d37fe640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86c8036000 tx=0x7f86c803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:37.952+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86c4059430 con 0x7f86d40d1630 2024-04-13T03:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.140+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86d40b6e00 con 0x7f86d40d1630 2024-04-13T03:09:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.141+0000 7f86d0ff9640 1 -- 172.21.15.106:0/210487851 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f86c4058d30 con 0x7f86d40d1630 2024-04-13T03:09:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 msgr2=0x7f86c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86c8036000 tx=0x7f86c803c040 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 msgr2=0x7f86d40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86c400c410 tx=0x7f86c400c8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 shutdown_connections 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.142+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d4162550 0x7f86d415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d415ea80 0x7f86d40c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 --2- 172.21.15.106:0/210487851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d40d1630 0x7f86d40b4d30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 >> 172.21.15.106:0/210487851 conn(0x7f86d409da50 msgr2=0x7f86d4161640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:38.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 shutdown_connections 2024-04-13T03:09:38.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:38.143+0000 7f86daa2d640 1 -- 172.21.15.106:0/210487851 wait complete. 2024-04-13T03:09:38.154 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-13T03:09:38.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:38.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-13T03:09:38.180 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[44671]: fuse finished with error 0 and tester_r 0 2024-04-13T03:09:38.182 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:09:44.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:09:44.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:44.284 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-13T03:09:44.306 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.452+0000 7f3fe9236640 1 Processor -- start 2024-04-13T03:09:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 -- start start 2024-04-13T03:09:44.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fe415e980 0x7f3fe4162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3fe415ed60 0x7f3fe40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe415fe40 con 0x7f3fe40cd380 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe41633f0 con 0x7f3fe415e980 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe9236640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe40cd780 con 0x7f3fe415ed60 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fe415e980 0x7f3fe4162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fe415e980 0x7f3fe4162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44816/0 (socket says 172.21.15.106:44816) 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.453+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/2525190840 learned_addr learned my addr 172.21.15.106:0/2525190840 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe259c640 1 --2- 172.21.15.106:0/2525190840 >> v2:172.21.15.114:3300/0 conn(0x7f3fe415ed60 0x7f3fe40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/2525190840 >> v2:172.21.15.114:3300/0 conn(0x7f3fe415ed60 msgr2=0x7f3fe40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe2d9d640 1 --2- 172.21.15.106:0/2525190840 >> v2:172.21.15.114:3300/0 conn(0x7f3fe415ed60 0x7f3fe40cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/2525190840 >> v1:172.21.15.114:6789/0 conn(0x7f3fe40cd380 legacy=0x7f3fe40d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/2525190840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe415f2c0 con 0x7f3fe415e980 2024-04-13T03:09:44.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.454+0000 7f3fe2d9d640 1 --2- 172.21.15.106:0/2525190840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe415e980 0x7f3fe4162eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40d2390 tx=0x7f3fdc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2092d3e08e546e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe1d9b640 1 -- 172.21.15.106:0/2525190840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc02cb20 con 0x7f3fe415e980 2024-04-13T03:09:44.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe1d9b640 1 -- 172.21.15.106:0/2525190840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc02ccc0 con 0x7f3fe415e980 2024-04-13T03:09:44.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe1d9b640 1 -- 172.21.15.106:0/2525190840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc034610 con 0x7f3fe415e980 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 -- 172.21.15.106:0/2525190840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe415e980 msgr2=0x7f3fe4162eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 --2- 172.21.15.106:0/2525190840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe415e980 0x7f3fe4162eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40d2390 tx=0x7f3fdc02bf80 comp rx=0 tx=0).stop 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 -- 172.21.15.106:0/2525190840 shutdown_connections 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 --2- 172.21.15.106:0/2525190840 >> v2:172.21.15.114:3300/0 conn(0x7f3fe415ed60 0x7f3fe40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 --2- 172.21.15.106:0/2525190840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe415e980 0x7f3fe4162eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 -- 172.21.15.106:0/2525190840 >> 172.21.15.106:0/2525190840 conn(0x7f3fe40d2550 msgr2=0x7f3fe40d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 -- 172.21.15.106:0/2525190840 shutdown_connections 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.455+0000 7f3fe9236640 1 -- 172.21.15.106:0/2525190840 wait complete. 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 Processor -- start 2024-04-13T03:09:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 -- start start 2024-04-13T03:09:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe415e980 0x7f3fe4153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3fe415ed60 0x7f3fe4155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe415fb60 con 0x7f3fe40cd380 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe415f880 con 0x7f3fe415e980 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.457+0000 7f3fe9236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe40d1e20 con 0x7f3fe415ed60 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44824/0 (socket says 172.21.15.106:44824) 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/4284939907 learned_addr learned my addr 172.21.15.106:0/4284939907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe359e640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3fe415ed60 0x7f3fe4155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe259c640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe415e980 0x7f3fe4153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3fe415ed60 msgr2=0x7f3fe4155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3fe415ed60 0x7f3fe4155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe415e980 msgr2=0x7f3fe4153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe415e980 0x7f3fe4153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 -- 172.21.15.106:0/4284939907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe40cc920 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe2d9d640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000f80 tx=0x7f3fdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc043020 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc03e450 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe4156440 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc03e840 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.458+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe4156ce0 con 0x7f3fe40cd380 2024-04-13T03:09:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.460+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe4069120 con 0x7f3fe40cd380 2024-04-13T03:09:44.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.460+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fdc046020 con 0x7f3fe40cd380 2024-04-13T03:09:44.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.460+0000 7f3fdb7fe640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.460+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3fdc0ae700 con 0x7f3fe40cd380 2024-04-13T03:09:44.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.460+0000 7f3fe259c640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.461+0000 7f3fe259c640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 0x7f3fc406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4153a40 tx=0x7f3fcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.465+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fdc04d030 con 0x7f3fe40cd380 2024-04-13T03:09:44.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.640+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fe40703d0 con 0x7f3fe40cd380 2024-04-13T03:09:44.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.641+0000 7f3fdb7fe640 1 -- 172.21.15.106:0/4284939907 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f3fdc07c1a0 con 0x7f3fe40cd380 2024-04-13T03:09:44.643 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:44.643 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:44.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 msgr2=0x7f3fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 0x7f3fc406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4153a40 tx=0x7f3fcc008040 comp rx=0 tx=0).stop 2024-04-13T03:09:44.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 msgr2=0x7f3fe40cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000f80 tx=0x7f3fdc004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:44.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 shutdown_connections 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3fe415ed60 0x7f3fe4155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe415e980 0x7f3fe4153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 --2- 172.21.15.106:0/4284939907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe40cd380 0x7f3fe40cc3e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 >> 172.21.15.106:0/4284939907 conn(0x7f3fe40d2550 msgr2=0x7f3fe41609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 shutdown_connections 2024-04-13T03:09:44.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.643+0000 7f3fe9236640 1 -- 172.21.15.106:0/4284939907 wait complete. 2024-04-13T03:09:44.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:44.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 Processor -- start 2024-04-13T03:09:44.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 -- start start 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f848815b800 0x7f84880cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f848815f1f0 0x7f848806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84880ccd20 con 0x7f8488072e80 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f848815f790 con 0x7f848815f1f0 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.755+0000 7f848cb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f848815fa70 con 0x7f848815b800 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f847ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f848815f1f0 0x7f848806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f8486d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8488072e80 0x7f84880cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56092/0 (socket says 172.21.15.106:56092) 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f8486d9d640 1 -- 172.21.15.106:0/3919944585 learned_addr learned my addr 172.21.15.106:0/3919944585 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f848659c640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f848815b800 0x7f84880cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f847ffff640 1 -- 172.21.15.106:0/3919944585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f848815b800 msgr2=0x7f84880cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f847ffff640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f848815b800 0x7f84880cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f847ffff640 1 -- 172.21.15.106:0/3919944585 >> v1:172.21.15.106:6789/0 conn(0x7f8488072e80 legacy=0x7f84880cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f8485d9b640 1 -- 172.21.15.106:0/3919944585 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056318910 0 0) 0x7f84880ccd20 con 0x7f8488072e80 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.756+0000 7f847ffff640 1 -- 172.21.15.106:0/3919944585 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848815fd50 con 0x7f848815f1f0 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f847ffff640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815f1f0 0x7f848806e5c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f847000ba40 tx=0x7f847002e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4e48341b962ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f8485d9b640 1 -- 172.21.15.106:0/3919944585 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847003b070 con 0x7f848815f1f0 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f8485d9b640 1 -- 172.21.15.106:0/3919944585 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847002ee30 con 0x7f848815f1f0 2024-04-13T03:09:44.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f8485d9b640 1 -- 172.21.15.106:0/3919944585 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470036640 con 0x7f848815f1f0 2024-04-13T03:09:44.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 -- 172.21.15.106:0/3919944585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815f1f0 msgr2=0x7f848806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815f1f0 0x7f848806e5c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f847000ba40 tx=0x7f847002e080 comp rx=0 tx=0).stop 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 -- 172.21.15.106:0/3919944585 shutdown_connections 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815f1f0 0x7f848806e5c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 --2- 172.21.15.106:0/3919944585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f848815b800 0x7f84880cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 -- 172.21.15.106:0/3919944585 >> 172.21.15.106:0/3919944585 conn(0x7f84880cdee0 msgr2=0x7f84880ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 -- 172.21.15.106:0/3919944585 shutdown_connections 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.757+0000 7f848cb1e640 1 -- 172.21.15.106:0/3919944585 wait complete. 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 Processor -- start 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 -- start start 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8488072e80 0x7f84881f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 0x7f84880ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f848806eb00 con 0x7f848815b800 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84880cf570 con 0x7f848815f1f0 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f848cb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cd8e0 con 0x7f8488072e80 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f847ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f847ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44842/0 (socket says 172.21.15.106:44842) 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.759+0000 7f847ffff640 1 -- 172.21.15.106:0/146752488 learned_addr learned my addr 172.21.15.106:0/146752488 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f8486d9d640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 0x7f84880ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f848659c640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8488072e80 0x7f84881f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 -- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8488072e80 msgr2=0x7f84881f0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8488072e80 0x7f84881f0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 -- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 msgr2=0x7f84880ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 0x7f84880ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 -- 172.21.15.106:0/146752488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848815bc80 con 0x7f848815b800 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f8486d9d640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 0x7f84880ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847ffff640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f847000bb60 tx=0x7f84700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470046070 con 0x7f848815b800 2024-04-13T03:09:44.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8470042670 con 0x7f848815b800 2024-04-13T03:09:44.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848815bfc0 con 0x7f848815b800 2024-04-13T03:09:44.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847003b040 con 0x7f848815b800 2024-04-13T03:09:44.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.760+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84880c92c0 con 0x7f848815b800 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.761+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8488069120 con 0x7f848815b800 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.761+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8470049070 con 0x7f848815b800 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.762+0000 7f847effd640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.762+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f84700b0800 con 0x7f848815b800 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.762+0000 7f848659c640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:44.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.762+0000 7f848659c640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 0x7f846806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8478006fd0 tx=0x7f8478008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:44.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.766+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f848815bfc0 con 0x7f848815b800 2024-04-13T03:09:44.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.940+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8488073ab0 con 0x7f848815b800 2024-04-13T03:09:44.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.941+0000 7f847effd640 1 -- 172.21.15.106:0/146752488 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f847007ea10 con 0x7f848815b800 2024-04-13T03:09:44.943 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 msgr2=0x7f846806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 0x7f846806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8478006fd0 tx=0x7f8478008040 comp rx=0 tx=0).stop 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 msgr2=0x7f848814de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f847000bb60 tx=0x7f84700046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 shutdown_connections 2024-04-13T03:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f846806c8f0 0x7f846806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f848815f1f0 0x7f84880ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f848815b800 0x7f848814de20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 --2- 172.21.15.106:0/146752488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8488072e80 0x7f84881f0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 >> 172.21.15.106:0/146752488 conn(0x7f84880cdee0 msgr2=0x7f84880b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 shutdown_connections 2024-04-13T03:09:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:44.944+0000 7f848cb1e640 1 -- 172.21.15.106:0/146752488 wait complete. 2024-04-13T03:09:44.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:09:45.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 Processor -- start 2024-04-13T03:09:45.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 -- start start 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc06fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ccc0706b0 con 0x7f1ccc145f60 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1ccc071750 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1cd0f15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc0701a0 con 0x7f1ccc06f800 2024-04-13T03:09:45.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.061+0000 7f1ccad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44858/0 (socket says 172.21.15.106:44858) 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 -- 172.21.15.106:0/4103345916 learned_addr learned my addr 172.21.15.106:0/4103345916 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1cc3fff640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc06fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 -- 172.21.15.106:0/4103345916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 msgr2=0x7f1ccc06fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc06fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 -- 172.21.15.106:0/4103345916 >> v1:172.21.15.114:6789/0 conn(0x7f1ccc145f60 legacy=0x7f1ccc0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 -- 172.21.15.106:0/4103345916 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc070990 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1cc3fff640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc06fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.062+0000 7f1ccad9d640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009e80 tx=0x7f1cc402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac78590b708edc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cc9d9b640 1 -- 172.21.15.106:0/4103345916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc402ca50 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cc9d9b640 1 -- 172.21.15.106:0/4103345916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc402cbf0 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cc9d9b640 1 -- 172.21.15.106:0/4103345916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40343f0 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 -- 172.21.15.106:0/4103345916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 msgr2=0x7f1ccc0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009e80 tx=0x7f1cc402beb0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 -- 172.21.15.106:0/4103345916 shutdown_connections 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc0ce0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/4103345916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc06fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 -- 172.21.15.106:0/4103345916 >> 172.21.15.106:0/4103345916 conn(0x7f1ccc0d1b60 msgr2=0x7f1ccc06f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 -- 172.21.15.106:0/4103345916 shutdown_connections 2024-04-13T03:09:45.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.063+0000 7f1cd0f15640 1 -- 172.21.15.106:0/4103345916 wait complete. 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.064+0000 7f1cd0f15640 1 Processor -- start 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 -- start start 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ccc145f60 0x7f1ccc15ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc070f50 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc0712c0 con 0x7f1ccc145f60 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cd0f15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc0d1190 con 0x7f1ccc06f800 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44870/0 (socket says 172.21.15.106:44870) 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cc3fff640 1 -- 172.21.15.106:0/3441930787 learned_addr learned my addr 172.21.15.106:0/3441930787 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1cca59c640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.065+0000 7f1ccad9d640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ccc145f60 0x7f1ccc15ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 -- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 msgr2=0x7f1ccc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 -- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ccc145f60 msgr2=0x7f1ccc15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ccc145f60 0x7f1ccc15ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 -- 172.21.15.106:0/3441930787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc0b5d60 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc3fff640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400d800 tx=0x7f1cb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4004030 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb40041d0 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc0b6040 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40096a0 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.066+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc13f330 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.067+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc069120 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.068+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cb4009840 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.068+0000 7f1cc2ffd640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.068+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1cb408c030 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.068+0000 7f1cca59c640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:45.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.068+0000 7f1cca59c640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 0x7f1cac06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc001780 tx=0x7f1cbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:45.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.072+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cb4059ad0 con 0x7f1ccc0cdc10 2024-04-13T03:09:45.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:45.285+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1ccc15fce0 con 0x7f1ccc0cdc10 2024-04-13T03:09:46.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.045+0000 7f1cc2ffd640 1 -- 172.21.15.106:0/3441930787 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f1cb405ddd0 con 0x7f1ccc0cdc10 2024-04-13T03:09:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 msgr2=0x7f1cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 0x7f1cac06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc001780 tx=0x7f1cbc009040 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 msgr2=0x7f1ccc15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400d800 tx=0x7f1cb400dcc0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 shutdown_connections 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1cac06c8f0 0x7f1cac06edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ccc145f60 0x7f1ccc15ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ccc0cdc10 0x7f1ccc15ea30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 --2- 172.21.15.106:0/3441930787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ccc06f800 0x7f1ccc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 >> 172.21.15.106:0/3441930787 conn(0x7f1ccc0d1b60 msgr2=0x7f1ccc0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 shutdown_connections 2024-04-13T03:09:46.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.048+0000 7f1cd0f15640 1 -- 172.21.15.106:0/3441930787 wait complete. 2024-04-13T03:09:46.063 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-13T03:09:46.063 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:09:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 Processor -- start 2024-04-13T03:09:46.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 -- start start 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff140b5ad0 0x7eff140b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff140cd240 con 0x7eff1415e820 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff140cd520 con 0x7eff140ce9e0 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.171+0000 7eff1a836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7eff140cd800 con 0x7eff140b5ad0 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff18dd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff140b5ad0 0x7eff140b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff13fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7eff1415e820 0x7eff140b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56106/0 (socket says 172.21.15.106:56106) 2024-04-13T03:09:46.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff13fff640 1 -- 172.21.15.106:0/4032813150 learned_addr learned my addr 172.21.15.106:0/4032813150 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:46.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff18dd3640 1 -- 172.21.15.106:0/4032813150 >> v1:172.21.15.114:6789/0 conn(0x7eff140ce9e0 legacy=0x7eff140d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:46.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff18dd3640 1 -- 172.21.15.106:0/4032813150 >> v1:172.21.15.106:6789/0 conn(0x7eff1415e820 legacy=0x7eff140b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff18dd3640 1 -- 172.21.15.106:0/4032813150 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff140cdae0 con 0x7eff140b5ad0 2024-04-13T03:09:46.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.172+0000 7eff12ffd640 1 -- 172.21.15.106:0/4032813150 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632611897 0 0) 0x7eff140cd240 con 0x7eff1415e820 2024-04-13T03:09:46.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff18dd3640 1 --2- 172.21.15.106:0/4032813150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140b7390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009dc0 tx=0x7eff0c02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1cd48eb51e7521 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff12ffd640 1 -- 172.21.15.106:0/4032813150 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c039070 con 0x7eff140b5ad0 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff12ffd640 1 -- 172.21.15.106:0/4032813150 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c02cbb0 con 0x7eff140b5ad0 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff12ffd640 1 -- 172.21.15.106:0/4032813150 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c034440 con 0x7eff140b5ad0 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 -- 172.21.15.106:0/4032813150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 msgr2=0x7eff140b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 --2- 172.21.15.106:0/4032813150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140b7390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009dc0 tx=0x7eff0c02be00 comp rx=0 tx=0).stop 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 -- 172.21.15.106:0/4032813150 shutdown_connections 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 --2- 172.21.15.106:0/4032813150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140b7390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 -- 172.21.15.106:0/4032813150 >> 172.21.15.106:0/4032813150 conn(0x7eff140d2510 msgr2=0x7eff140d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 -- 172.21.15.106:0/4032813150 shutdown_connections 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.173+0000 7eff1a836640 1 -- 172.21.15.106:0/4032813150 wait complete. 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.174+0000 7eff1a836640 1 Processor -- start 2024-04-13T03:09:46.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 -- start start 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 0x7eff1415efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff1415e820 0x7eff14160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff140ccf50 con 0x7eff140b5ad0 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7eff140ccdd0 con 0x7eff140ce9e0 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff1a836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7eff1415ec00 con 0x7eff1415e820 2024-04-13T03:09:46.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff13fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff13fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44892/0 (socket says 172.21.15.106:44892) 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.175+0000 7eff13fff640 1 -- 172.21.15.106:0/1150397493 learned_addr learned my addr 172.21.15.106:0/1150397493 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff18dd3640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff1415e820 0x7eff14160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff137fe640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 0x7eff1415efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 -- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff1415e820 msgr2=0x7eff14160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff1415e820 0x7eff14160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 -- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 msgr2=0x7eff1415efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 0x7eff1415efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 -- 172.21.15.106:0/1150397493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1415f920 con 0x7eff140b5ad0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff137fe640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 0x7eff1415efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff13fff640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efefc00c460 tx=0x7efefc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc018070 con 0x7eff140b5ad0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc007500 con 0x7eff140b5ad0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc004a20 con 0x7eff140b5ad0 2024-04-13T03:09:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1415fc00 con 0x7eff140b5ad0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.176+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff141601c0 con 0x7eff140b5ad0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.177+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff14069120 con 0x7eff140b5ad0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.177+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efefc01b070 con 0x7eff140b5ad0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.178+0000 7eff10ff9640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 0x7efef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.178+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efefc08b140 con 0x7eff140b5ad0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.178+0000 7eff137fe640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 0x7efef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:46.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.178+0000 7eff137fe640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 0x7efef406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff140b1b60 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:46.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.182+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efefc059230 con 0x7eff140b5ad0 2024-04-13T03:09:46.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:46.395+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7eff14073fa0 con 0x7eff140b5ad0 2024-04-13T03:09:47.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.055+0000 7eff10ff9640 1 -- 172.21.15.106:0/1150397493 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7efefc058b30 con 0x7eff140b5ad0 2024-04-13T03:09:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 msgr2=0x7efef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 0x7efef406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff140b1b60 tx=0x7eff04009040 comp rx=0 tx=0).stop 2024-04-13T03:09:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 msgr2=0x7eff140cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efefc00c460 tx=0x7efefc00c920 comp rx=0 tx=0).stop 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 shutdown_connections 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efef406c8f0 0x7efef406edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff1415e820 0x7eff14160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff140ce9e0 0x7eff1415efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 --2- 172.21.15.106:0/1150397493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff140b5ad0 0x7eff140cbbf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 >> 172.21.15.106:0/1150397493 conn(0x7eff140d2510 msgr2=0x7eff14067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 shutdown_connections 2024-04-13T03:09:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.057+0000 7eff1a836640 1 -- 172.21.15.106:0/1150397493 wait complete. 2024-04-13T03:09:47.070 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:09:47.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:47.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.170+0000 7ff7d7ffe640 1 Processor -- start 2024-04-13T03:09:47.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.170+0000 7ff7d7ffe640 1 -- start start 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7d015cb00 0x7ff7d0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00b81f0 0x7ff7d00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 0x7ff7d006f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d00b54c0 con 0x7ff7d015cb00 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d0071320 con 0x7ff7d00cd2f0 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d7ffe640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d0070d60 con 0x7ff7d00b81f0 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d5d9a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7d015cb00 0x7ff7d0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d5d9a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7d015cb00 0x7ff7d0160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44902/0 (socket says 172.21.15.106:44902) 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.171+0000 7ff7d5d9a640 1 -- 172.21.15.106:0/1516212559 learned_addr learned my addr 172.21.15.106:0/1516212559 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5599640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00b81f0 0x7ff7d00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d659b640 1 --2- 172.21.15.106:0/1516212559 >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 0x7ff7d006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 -- 172.21.15.106:0/1516212559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00b81f0 msgr2=0x7ff7d00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00b81f0 0x7ff7d00ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 -- 172.21.15.106:0/1516212559 >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 msgr2=0x7ff7d006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/1516212559 >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 0x7ff7d006f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 -- 172.21.15.106:0/1516212559 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0071040 con 0x7ff7d015cb00 2024-04-13T03:09:47.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d659b640 1 --2- 172.21.15.106:0/1516212559 >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 0x7ff7d006f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d0160e90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00b18a0 tx=0x7ff7cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1aeb951a560aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.172+0000 7ff7d4d98640 1 -- 172.21.15.106:0/1516212559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc035020 con 0x7ff7d015cb00 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d4d98640 1 -- 172.21.15.106:0/1516212559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc031870 con 0x7ff7d015cb00 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d4d98640 1 -- 172.21.15.106:0/1516212559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc031c40 con 0x7ff7d015cb00 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/1516212559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 msgr2=0x7ff7d0160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d0160e90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00b18a0 tx=0x7ff7cc02b910 comp rx=0 tx=0).stop 2024-04-13T03:09:47.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/1516212559 shutdown_connections 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/1516212559 >> v2:172.21.15.114:3300/0 conn(0x7ff7d00cd2f0 0x7ff7d006f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00b81f0 0x7ff7d00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/1516212559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d0160e90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/1516212559 >> 172.21.15.106:0/1516212559 conn(0x7ff7d00d2530 msgr2=0x7ff7d00d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/1516212559 shutdown_connections 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.173+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/1516212559 wait complete. 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.174+0000 7ff7d7ffe640 1 Processor -- start 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.174+0000 7ff7d7ffe640 1 -- start start 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7d00b81f0 0x7ff7d00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00cd2f0 0x7ff7d015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d006fe60 con 0x7ff7d015cb00 2024-04-13T03:09:47.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d00d07b0 con 0x7ff7d00b81f0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d7ffe640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d006ffe0 con 0x7ff7d00cd2f0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44912/0 (socket says 172.21.15.106:44912) 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 -- 172.21.15.106:0/2918221820 learned_addr learned my addr 172.21.15.106:0/2918221820 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d5599640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00cd2f0 0x7ff7d015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7d00b81f0 0x7ff7d00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 -- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00cd2f0 msgr2=0x7ff7d015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00cd2f0 0x7ff7d015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 -- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7d00b81f0 msgr2=0x7ff7d00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7d00b81f0 0x7ff7d00b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.175+0000 7ff7d659b640 1 -- 172.21.15.106:0/2918221820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d00b1330 con 0x7ff7d015cb00 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7d659b640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007ef0 tx=0x7ff7c800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c801a070 con 0x7ff7d015cb00 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c8003b90 con 0x7ff7d015cb00 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8004030 con 0x7ff7d015cb00 2024-04-13T03:09:47.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d006dd30 con 0x7ff7d015cb00 2024-04-13T03:09:47.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.176+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d006e2f0 con 0x7ff7d015cb00 2024-04-13T03:09:47.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.177+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d0068a10 con 0x7ff7d015cb00 2024-04-13T03:09:47.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.177+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7c801d070 con 0x7ff7d015cb00 2024-04-13T03:09:47.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.177+0000 7ff7c67fc640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 0x7ff7b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.178+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff7c808c050 con 0x7ff7d015cb00 2024-04-13T03:09:47.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.178+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 0x7ff7b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.178+0000 7ff7d5d9a640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 0x7ff7b406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.182+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7c8010070 con 0x7ff7d015cb00 2024-04-13T03:09:47.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.394+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7d00d3120 con 0x7ff7d015cb00 2024-04-13T03:09:47.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.394+0000 7ff7c67fc640 1 -- 172.21.15.106:0/2918221820 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7ff7c8059b80 con 0x7ff7d015cb00 2024-04-13T03:09:47.395 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:47.396 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:47.396 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:09:47.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 msgr2=0x7ff7b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 0x7ff7b406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc03c040 comp rx=0 tx=0).stop 2024-04-13T03:09:47.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 msgr2=0x7ff7d015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007ef0 tx=0x7ff7c800d4a0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 shutdown_connections 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff7b406c8f0 0x7ff7b406edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff7d00cd2f0 0x7ff7d015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7d015cb00 0x7ff7d015d810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 --2- 172.21.15.106:0/2918221820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7d00b81f0 0x7ff7d00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 >> 172.21.15.106:0/2918221820 conn(0x7ff7d00d2530 msgr2=0x7ff7d015bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 shutdown_connections 2024-04-13T03:09:47.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.396+0000 7ff7d7ffe640 1 -- 172.21.15.106:0/2918221820 wait complete. 2024-04-13T03:09:47.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:47.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.507+0000 7f5030088640 1 Processor -- start 2024-04-13T03:09:47.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.507+0000 7f5030088640 1 -- start start 2024-04-13T03:09:47.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f5030088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 0x7f50280d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f5030088640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50280cfd20 con 0x7f502815e9b0 2024-04-13T03:09:47.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f5030088640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5028162af0 con 0x7f50280b3d60 2024-04-13T03:09:47.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f5030088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5028162dd0 con 0x7f502815ed90 2024-04-13T03:09:47.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502de24640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f502815e9b0 0x7f50280d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56112/0 (socket says 172.21.15.106:56112) 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502de24640 1 -- 172.21.15.106:0/2488598800 learned_addr learned my addr 172.21.15.106:0/2488598800 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502d623640 1 --2- 172.21.15.106:0/2488598800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 0x7f50280d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623065322 0 0) 0x7f50280cfd20 con 0x7f502815e9b0 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5014003520 con 0x7f502815e9b0 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 801611589 0 0) 0x7f5014003520 con 0x7f502815e9b0 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502d623640 1 -- 172.21.15.106:0/2488598800 >> v1:172.21.15.114:6789/0 conn(0x7f50280b3d60 legacy=0x7f50280d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502d623640 1 -- 172.21.15.106:0/2488598800 >> v1:172.21.15.106:6789/0 conn(0x7f502815e9b0 legacy=0x7f50280d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.508+0000 7f502d623640 1 -- 172.21.15.106:0/2488598800 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50281630b0 con 0x7f502815ed90 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.509+0000 7f502d623640 1 --2- 172.21.15.106:0/2488598800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 0x7f50280d2300 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50180097c0 tx=0x7f501802b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0f82882a495f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.509+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018037070 con 0x7f502815ed90 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.509+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5018030510 con 0x7f502815ed90 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.509+0000 7f502ce22640 1 -- 172.21.15.106:0/2488598800 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024001690 con 0x7f502815ed90 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 -- 172.21.15.106:0/2488598800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 msgr2=0x7f50280d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 --2- 172.21.15.106:0/2488598800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 0x7f50280d2300 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50180097c0 tx=0x7f501802b6e0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 -- 172.21.15.106:0/2488598800 shutdown_connections 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 --2- 172.21.15.106:0/2488598800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815ed90 0x7f50280d2300 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 -- 172.21.15.106:0/2488598800 >> 172.21.15.106:0/2488598800 conn(0x7f502808bf10 msgr2=0x7f50280b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 -- 172.21.15.106:0/2488598800 shutdown_connections 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.510+0000 7f5030088640 1 -- 172.21.15.106:0/2488598800 wait complete. 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.511+0000 7f5030088640 1 Processor -- start 2024-04-13T03:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.511+0000 7f5030088640 1 -- start start 2024-04-13T03:09:47.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 0x7f502815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815e9b0 0x7f50280ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5028162800 con 0x7f5028161190 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5028162680 con 0x7f50280b3d60 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f50280d4d00 con 0x7f502815e9b0 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44920/0 (socket says 172.21.15.106:44920) 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 -- 172.21.15.106:0/1619402874 learned_addr learned my addr 172.21.15.106:0/1619402874 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502d623640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815e9b0 0x7f50280ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502de24640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 0x7f502815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 -- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815e9b0 msgr2=0x7f50280ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815e9b0 0x7f50280ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 -- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 msgr2=0x7f502815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 0x7f502815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 -- 172.21.15.106:0/1619402874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50280ccb80 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502de24640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 0x7f502815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f502e625640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f502000caf0 tx=0x7f50200020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020017070 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5020003da0 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502000a920 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.512+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50280cce60 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.513+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50280cd420 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.513+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50280d3ec0 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.514+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f502001a070 con 0x7f5028161190 2024-04-13T03:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.514+0000 7f501e7fc640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.514+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f502008b650 con 0x7f5028161190 2024-04-13T03:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.514+0000 7f502de24640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.515+0000 7f502de24640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.519+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50280d3ec0 con 0x7f5028161190 2024-04-13T03:09:47.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.694+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5028068a10 con 0x7f5028161190 2024-04-13T03:09:47.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.695+0000 7f501e7fc640 1 -- 172.21.15.106:0/1619402874 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f50200597d0 con 0x7f5028161190 2024-04-13T03:09:47.696 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:47.696 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:47.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 msgr2=0x7f500c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024009040 comp rx=0 tx=0).stop 2024-04-13T03:09:47.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 msgr2=0x7f5028161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f502000caf0 tx=0x7f50200020a0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 shutdown_connections 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f502815e9b0 0x7f50280ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5028161190 0x7f5028161620 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 --2- 172.21.15.106:0/1619402874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50280b3d60 0x7f502815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 >> 172.21.15.106:0/1619402874 conn(0x7f502808bf10 msgr2=0x7f502815bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 shutdown_connections 2024-04-13T03:09:47.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.697+0000 7f5030088640 1 -- 172.21.15.106:0/1619402874 wait complete. 2024-04-13T03:09:47.709 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:09:47.709 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:09:47.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.808+0000 7f58e8b1e640 1 Processor -- start 2024-04-13T03:09:47.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.808+0000 7f58e8b1e640 1 -- start start 2024-04-13T03:09:47.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58e415e9c0 0x7f58e40ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e41624b0 0x7f58e40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f58e40b6700 0x7f58e406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f58e40d27f0 con 0x7f58e415e9c0 2024-04-13T03:09:47.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f58e415eda0 con 0x7f58e40b6700 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e8b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58e4162c20 con 0x7f58e41624b0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58e415e9c0 0x7f58e40ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58e415e9c0 0x7f58e40ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44922/0 (socket says 172.21.15.106:44922) 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 -- 172.21.15.106:0/2603152827 learned_addr learned my addr 172.21.15.106:0/2603152827 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e1d9b640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e41624b0 0x7f58e40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e2d9d640 1 --2- 172.21.15.106:0/2603152827 >> v2:172.21.15.114:3300/0 conn(0x7f58e40b6700 0x7f58e406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 -- 172.21.15.106:0/2603152827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e41624b0 msgr2=0x7f58e40b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e41624b0 0x7f58e40b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 -- 172.21.15.106:0/2603152827 >> v2:172.21.15.114:3300/0 conn(0x7f58e40b6700 msgr2=0x7f58e406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 --2- 172.21.15.106:0/2603152827 >> v2:172.21.15.114:3300/0 conn(0x7f58e40b6700 0x7f58e406b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.809+0000 7f58e259c640 1 -- 172.21.15.106:0/2603152827 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e4162f00 con 0x7f58e415e9c0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.810+0000 7f58e259c640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e415e9c0 0x7f58e40ba290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58e40b72b0 tx=0x7f58d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc5c02738802d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.810+0000 7f58e159a640 1 -- 172.21.15.106:0/2603152827 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4035020 con 0x7f58e415e9c0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.810+0000 7f58e159a640 1 -- 172.21.15.106:0/2603152827 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4031870 con 0x7f58e415e9c0 2024-04-13T03:09:47.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.810+0000 7f58e159a640 1 -- 172.21.15.106:0/2603152827 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4031c40 con 0x7f58e415e9c0 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 -- 172.21.15.106:0/2603152827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e415e9c0 msgr2=0x7f58e40ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e415e9c0 0x7f58e40ba290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58e40b72b0 tx=0x7f58d402b910 comp rx=0 tx=0).stop 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 -- 172.21.15.106:0/2603152827 shutdown_connections 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/2603152827 >> v2:172.21.15.114:3300/0 conn(0x7f58e40b6700 0x7f58e406b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e41624b0 0x7f58e40b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/2603152827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e415e9c0 0x7f58e40ba290 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 -- 172.21.15.106:0/2603152827 >> 172.21.15.106:0/2603152827 conn(0x7f58e40d11c0 msgr2=0x7f58e40d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 -- 172.21.15.106:0/2603152827 shutdown_connections 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.811+0000 7f58e8b1e640 1 -- 172.21.15.106:0/2603152827 wait complete. 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.812+0000 7f58e8b1e640 1 Processor -- start 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.812+0000 7f58e8b1e640 1 -- start start 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e40b6700 0x7f58e4149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e415e9c0 0x7f58e415efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4162910 con 0x7f58e41624b0 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40d2a40 con 0x7f58e40b6700 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e8b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58e406be20 con 0x7f58e415e9c0 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44930/0 (socket says 172.21.15.106:44930) 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 -- 172.21.15.106:0/122483772 learned_addr learned my addr 172.21.15.106:0/122483772 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e259c640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e40b6700 0x7f58e4149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e1d9b640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e415e9c0 0x7f58e415efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 -- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e415e9c0 msgr2=0x7f58e415efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e415e9c0 0x7f58e415efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 -- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e40b6700 msgr2=0x7f58e4149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e40b6700 0x7f58e4149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.813+0000 7f58e2d9d640 1 -- 172.21.15.106:0/122483772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40cc020 con 0x7f58e41624b0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58e2d9d640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007d90 tx=0x7f58dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc014420 con 0x7f58e41624b0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58dc019070 con 0x7f58e41624b0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc014820 con 0x7f58e41624b0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e4161c10 con 0x7f58e41624b0 2024-04-13T03:09:47.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.814+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e40cc5a0 con 0x7f58e41624b0 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.815+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4068a10 con 0x7f58e41624b0 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.815+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58dc0149c0 con 0x7f58e41624b0 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.816+0000 7f58d2ffd640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 0x7f58c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.816+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58dc08c570 con 0x7f58e41624b0 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.816+0000 7f58e259c640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 0x7f58c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:47.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.816+0000 7f58e259c640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 0x7f58c406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f58d4036000 tx=0x7f58d403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:47.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.820+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58dc05a010 con 0x7f58e41624b0 2024-04-13T03:09:47.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:47.992+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58e40b8a30 con 0x7f58c406c8f0 2024-04-13T03:09:48.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.000+0000 7f58d2ffd640 1 -- 172.21.15.106:0/122483772 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f58d4099310 con 0x7f58c406c8f0 2024-04-13T03:09:48.005 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:48.007 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:09:48.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 msgr2=0x7f58c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 0x7f58c406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f58d4036000 tx=0x7f58d403c040 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 msgr2=0x7f58e415f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007d90 tx=0x7f58dc00e4b0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 shutdown_connections 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58c406c8f0 0x7f58c406edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e415e9c0 0x7f58e415efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58e41624b0 0x7f58e415f4f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 --2- 172.21.15.106:0/122483772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e40b6700 0x7f58e4149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.003+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 >> 172.21.15.106:0/122483772 conn(0x7f58e40d11c0 msgr2=0x7f58e40cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.004+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 shutdown_connections 2024-04-13T03:09:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.004+0000 7f58e8b1e640 1 -- 172.21.15.106:0/122483772 wait complete. 2024-04-13T03:09:48.016 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-13T03:09:46.367067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004868+0000","last_change":"2024-04-13T03:09:23.977064+0000","last_active":"2024-04-13T03:09:31.004868+0000","last_peered":"2024-04-13T03:09:31.004868+0000","last_clean":"2024-04-13T03:09:31.004868+0000","last_became_active":"2024-04-13T03:09:23.976781+0000","last_became_peered":"2024-04-13T03:09:23.976781+0000","last_unstale":"2024-04-13T03:09:31.004868+0000","last_undegraded":"2024-04-13T03:09:31.004868+0000","last_fullsized":"2024-04-13T03:09:31.004868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:14:27.702105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017061+0000","last_change":"2024-04-13T03:09:24.983007+0000","last_active":"2024-04-13T03:09:31.017061+0000","last_peered":"2024-04-13T03:09:31.017061+0000","last_clean":"2024-04-13T03:09:31.017061+0000","last_became_active":"2024-04-13T03:09:24.982432+0000","last_became_peered":"2024-04-13T03:09:24.982432+0000","last_unstale":"2024-04-13T03:09:31.017061+0000","last_undegraded":"2024-04-13T03:09:31.017061+0000","last_fullsized":"2024-04-13T03:09:31.017061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016588+0000","last_change":"2024-04-13T03:09:29.567663+0000","last_active":"2024-04-13T03:09:31.016588+0000","last_peered":"2024-04-13T03:09:31.016588+0000","last_clean":"2024-04-13T03:09:31.016588+0000","last_became_active":"2024-04-13T03:09:29.567314+0000","last_became_peered":"2024-04-13T03:09:29.567314+0000","last_unstale":"2024-04-13T03:09:31.016588+0000","last_undegraded":"2024-04-13T03:09:31.016588+0000","last_fullsized":"2024-04-13T03:09:31.016588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:55:26.111999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017190+0000","last_change":"2024-04-13T03:09:23.977858+0000","last_active":"2024-04-13T03:09:31.017190+0000","last_peered":"2024-04-13T03:09:31.017190+0000","last_clean":"2024-04-13T03:09:31.017190+0000","last_became_active":"2024-04-13T03:09:23.977595+0000","last_became_peered":"2024-04-13T03:09:23.977595+0000","last_unstale":"2024-04-13T03:09:31.017190+0000","last_undegraded":"2024-04-13T03:09:31.017190+0000","last_fullsized":"2024-04-13T03:09:31.017190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004745+0000","last_change":"2024-04-13T03:09:24.981595+0000","last_active":"2024-04-13T03:09:31.004745+0000","last_peered":"2024-04-13T03:09:31.004745+0000","last_clean":"2024-04-13T03:09:31.004745+0000","last_became_active":"2024-04-13T03:09:24.981249+0000","last_became_peered":"2024-04-13T03:09:24.981249+0000","last_unstale":"2024-04-13T03:09:31.004745+0000","last_undegraded":"2024-04-13T03:09:31.004745+0000","last_fullsized":"2024-04-13T03:09:31.004745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004491+0000","last_change":"2024-04-13T03:09:29.008503+0000","last_active":"2024-04-13T03:09:31.004491+0000","last_peered":"2024-04-13T03:09:31.004491+0000","last_clean":"2024-04-13T03:09:31.004491+0000","last_became_active":"2024-04-13T03:09:29.007985+0000","last_became_peered":"2024-04-13T03:09:29.007985+0000","last_unstale":"2024-04-13T03:09:31.004491+0000","last_undegraded":"2024-04-13T03:09:31.004491+0000","last_fullsized":"2024-04-13T03:09:31.004491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074031+0000","last_change":"2024-04-13T03:09:23.982379+0000","last_active":"2024-04-13T03:09:32.074031+0000","last_peered":"2024-04-13T03:09:32.074031+0000","last_clean":"2024-04-13T03:09:32.074031+0000","last_became_active":"2024-04-13T03:09:23.982090+0000","last_became_peered":"2024-04-13T03:09:23.982090+0000","last_unstale":"2024-04-13T03:09:32.074031+0000","last_undegraded":"2024-04-13T03:09:32.074031+0000","last_fullsized":"2024-04-13T03:09:32.074031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006164+0000","last_change":"2024-04-13T03:09:24.983652+0000","last_active":"2024-04-13T03:09:31.006164+0000","last_peered":"2024-04-13T03:09:31.006164+0000","last_clean":"2024-04-13T03:09:31.006164+0000","last_became_active":"2024-04-13T03:09:24.981620+0000","last_became_peered":"2024-04-13T03:09:24.981620+0000","last_unstale":"2024-04-13T03:09:31.006164+0000","last_undegraded":"2024-04-13T03:09:31.006164+0000","last_fullsized":"2024-04-13T03:09:31.006164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004691+0000","last_change":"2024-04-13T03:09:29.007274+0000","last_active":"2024-04-13T03:09:31.004691+0000","last_peered":"2024-04-13T03:09:31.004691+0000","last_clean":"2024-04-13T03:09:31.004691+0000","last_became_active":"2024-04-13T03:09:29.006867+0000","last_became_peered":"2024-04-13T03:09:29.006867+0000","last_unstale":"2024-04-13T03:09:31.004691+0000","last_undegraded":"2024-04-13T03:09:31.004691+0000","last_fullsized":"2024-04-13T03:09:31.004691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:32.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073387+0000","last_change":"2024-04-13T03:09:23.979669+0000","last_active":"2024-04-13T03:09:32.073387+0000","last_peered":"2024-04-13T03:09:32.073387+0000","last_clean":"2024-04-13T03:09:32.073387+0000","last_became_active":"2024-04-13T03:09:23.979303+0000","last_became_peered":"2024-04-13T03:09:23.979303+0000","last_unstale":"2024-04-13T03:09:32.073387+0000","last_undegraded":"2024-04-13T03:09:32.073387+0000","last_fullsized":"2024-04-13T03:09:32.073387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005531+0000","last_change":"2024-04-13T03:09:24.976796+0000","last_active":"2024-04-13T03:09:31.005531+0000","last_peered":"2024-04-13T03:09:31.005531+0000","last_clean":"2024-04-13T03:09:31.005531+0000","last_became_active":"2024-04-13T03:09:24.975367+0000","last_became_peered":"2024-04-13T03:09:24.975367+0000","last_unstale":"2024-04-13T03:09:31.005531+0000","last_undegraded":"2024-04-13T03:09:31.005531+0000","last_fullsized":"2024-04-13T03:09:31.005531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004563+0000","last_change":"2024-04-13T03:09:29.007374+0000","last_active":"2024-04-13T03:09:31.004563+0000","last_peered":"2024-04-13T03:09:31.004563+0000","last_clean":"2024-04-13T03:09:31.004563+0000","last_became_active":"2024-04-13T03:09:29.006740+0000","last_became_peered":"2024-04-13T03:09:29.006740+0000","last_unstale":"2024-04-13T03:09:31.004563+0000","last_undegraded":"2024-04-13T03:09:31.004563+0000","last_fullsized":"2024-04-13T03:09:31.004563+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073554+0000","last_change":"2024-04-13T03:09:24.988155+0000","last_active":"2024-04-13T03:09:32.073554+0000","last_peered":"2024-04-13T03:09:32.073554+0000","last_clean":"2024-04-13T03:09:32.073554+0000","last_became_active":"2024-04-13T03:09:24.987879+0000","last_became_peered":"2024-04-13T03:09:24.987879+0000","last_unstale":"2024-04-13T03:09:32.073554+0000","last_undegraded":"2024-04-13T03:09:32.073554+0000","last_fullsized":"2024-04-13T03:09:32.073554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072988+0000","last_change":"2024-04-13T03:09:23.977048+0000","last_active":"2024-04-13T03:09:32.072988+0000","last_peered":"2024-04-13T03:09:32.072988+0000","last_clean":"2024-04-13T03:09:32.072988+0000","last_became_active":"2024-04-13T03:09:23.976770+0000","last_became_peered":"2024-04-13T03:09:23.976770+0000","last_unstale":"2024-04-13T03:09:32.072988+0000","last_undegraded":"2024-04-13T03:09:32.072988+0000","last_fullsized":"2024-04-13T03:09:32.072988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005678+0000","last_change":"2024-04-13T03:09:29.009555+0000","last_active":"2024-04-13T03:09:31.005678+0000","last_peered":"2024-04-13T03:09:31.005678+0000","last_clean":"2024-04-13T03:09:31.005678+0000","last_became_active":"2024-04-13T03:09:29.009108+0000","last_became_peered":"2024-04-13T03:09:29.009108+0000","last_unstale":"2024-04-13T03:09:31.005678+0000","last_undegraded":"2024-04-13T03:09:31.005678+0000","last_fullsized":"2024-04-13T03:09:31.005678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016758+0000","last_change":"2024-04-13T03:09:24.973588+0000","last_active":"2024-04-13T03:09:31.016758+0000","last_peered":"2024-04-13T03:09:31.016758+0000","last_clean":"2024-04-13T03:09:31.016758+0000","last_became_active":"2024-04-13T03:09:24.973322+0000","last_became_peered":"2024-04-13T03:09:24.973322+0000","last_unstale":"2024-04-13T03:09:31.016758+0000","last_undegraded":"2024-04-13T03:09:31.016758+0000","last_fullsized":"2024-04-13T03:09:31.016758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552368+0000","last_change":"2024-04-13T03:09:23.971765+0000","last_active":"2024-04-13T03:09:31.552368+0000","last_peered":"2024-04-13T03:09:31.552368+0000","last_clean":"2024-04-13T03:09:31.552368+0000","last_became_active":"2024-04-13T03:09:23.971543+0000","last_became_peered":"2024-04-13T03:09:23.971543+0000","last_unstale":"2024-04-13T03:09:31.552368+0000","last_undegraded":"2024-04-13T03:09:31.552368+0000","last_fullsized":"2024-04-13T03:09:31.552368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005586+0000","last_change":"2024-04-13T03:09:29.008916+0000","last_active":"2024-04-13T03:09:31.005586+0000","last_peered":"2024-04-13T03:09:31.005586+0000","last_clean":"2024-04-13T03:09:31.005586+0000","last_became_active":"2024-04-13T03:09:29.008366+0000","last_became_peered":"2024-04-13T03:09:29.008366+0000","last_unstale":"2024-04-13T03:09:31.005586+0000","last_undegraded":"2024-04-13T03:09:31.005586+0000","last_fullsized":"2024-04-13T03:09:31.005586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016669+0000","last_change":"2024-04-13T03:09:24.975847+0000","last_active":"2024-04-13T03:09:31.016669+0000","last_peered":"2024-04-13T03:09:31.016669+0000","last_clean":"2024-04-13T03:09:31.016669+0000","last_became_active":"2024-04-13T03:09:24.975258+0000","last_became_peered":"2024-04-13T03:09:24.975258+0000","last_unstale":"2024-04-13T03:09:31.016669+0000","last_undegraded":"2024-04-13T03:09:31.016669+0000","last_fullsized":"2024-04-13T03:09:31.016669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552132+0000","last_change":"2024-04-13T03:09:23.977100+0000","last_active":"2024-04-13T03:09:31.552132+0000","last_peered":"2024-04-13T03:09:31.552132+0000","last_clean":"2024-04-13T03:09:31.552132+0000","last_became_active":"2024-04-13T03:09:23.976787+0000","last_became_peered":"2024-04-13T03:09:23.976787+0000","last_unstale":"2024-04-13T03:09:31.552132+0000","last_undegraded":"2024-04-13T03:09:31.552132+0000","last_fullsized":"2024-04-13T03:09:31.552132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552936+0000","last_change":"2024-04-13T03:09:29.567374+0000","last_active":"2024-04-13T03:09:31.552936+0000","last_peered":"2024-04-13T03:09:31.552936+0000","last_clean":"2024-04-13T03:09:31.552936+0000","last_became_active":"2024-04-13T03:09:29.566995+0000","last_became_peered":"2024-04-13T03:09:29.566995+0000","last_unstale":"2024-04-13T03:09:31.552936+0000","last_undegraded":"2024-04-13T03:09:31.552936+0000","last_fullsized":"2024-04-13T03:09:31.552936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004813+0000","last_change":"2024-04-13T03:09:24.972822+0000","last_active":"2024-04-13T03:09:31.004813+0000","last_peered":"2024-04-13T03:09:31.004813+0000","last_clean":"2024-04-13T03:09:31.004813+0000","last_became_active":"2024-04-13T03:09:24.972521+0000","last_became_peered":"2024-04-13T03:09:24.972521+0000","last_unstale":"2024-04-13T03:09:31.004813+0000","last_undegraded":"2024-04-13T03:09:31.004813+0000","last_fullsized":"2024-04-13T03:09:31.004813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073287+0000","last_change":"2024-04-13T03:09:23.979583+0000","last_active":"2024-04-13T03:09:32.073287+0000","last_peered":"2024-04-13T03:09:32.073287+0000","last_clean":"2024-04-13T03:09:32.073287+0000","last_became_active":"2024-04-13T03:09:23.979321+0000","last_became_peered":"2024-04-13T03:09:23.979321+0000","last_unstale":"2024-04-13T03:09:32.073287+0000","last_undegraded":"2024-04-13T03:09:32.073287+0000","last_fullsized":"2024-04-13T03:09:32.073287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:05.954316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006004+0000","last_change":"2024-04-13T03:09:29.009924+0000","last_active":"2024-04-13T03:09:31.006004+0000","last_peered":"2024-04-13T03:09:31.006004+0000","last_clean":"2024-04-13T03:09:31.006004+0000","last_became_active":"2024-04-13T03:09:29.009528+0000","last_became_peered":"2024-04-13T03:09:29.009528+0000","last_unstale":"2024-04-13T03:09:31.006004+0000","last_undegraded":"2024-04-13T03:09:31.006004+0000","last_fullsized":"2024-04-13T03:09:31.006004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004799+0000","last_change":"2024-04-13T03:09:24.973677+0000","last_active":"2024-04-13T03:09:31.004799+0000","last_peered":"2024-04-13T03:09:31.004799+0000","last_clean":"2024-04-13T03:09:31.004799+0000","last_became_active":"2024-04-13T03:09:24.973419+0000","last_became_peered":"2024-04-13T03:09:24.973419+0000","last_unstale":"2024-04-13T03:09:31.004799+0000","last_undegraded":"2024-04-13T03:09:31.004799+0000","last_fullsized":"2024-04-13T03:09:31.004799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074813+0000","last_change":"2024-04-13T03:09:23.977870+0000","last_active":"2024-04-13T03:09:32.074813+0000","last_peered":"2024-04-13T03:09:32.074813+0000","last_clean":"2024-04-13T03:09:32.074813+0000","last_became_active":"2024-04-13T03:09:23.977529+0000","last_became_peered":"2024-04-13T03:09:23.977529+0000","last_unstale":"2024-04-13T03:09:32.074813+0000","last_undegraded":"2024-04-13T03:09:32.074813+0000","last_fullsized":"2024-04-13T03:09:32.074813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004609+0000","last_change":"2024-04-13T03:09:29.566777+0000","last_active":"2024-04-13T03:09:31.004609+0000","last_peered":"2024-04-13T03:09:31.004609+0000","last_clean":"2024-04-13T03:09:31.004609+0000","last_became_active":"2024-04-13T03:09:29.566173+0000","last_became_peered":"2024-04-13T03:09:29.566173+0000","last_unstale":"2024-04-13T03:09:31.004609+0000","last_undegraded":"2024-04-13T03:09:31.004609+0000","last_fullsized":"2024-04-13T03:09:31.004609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017303+0000","last_change":"2024-04-13T03:09:24.981346+0000","last_active":"2024-04-13T03:09:31.017303+0000","last_peered":"2024-04-13T03:09:31.017303+0000","last_clean":"2024-04-13T03:09:31.017303+0000","last_became_active":"2024-04-13T03:09:24.981003+0000","last_became_peered":"2024-04-13T03:09:24.981003+0000","last_unstale":"2024-04-13T03:09:31.017303+0000","last_undegraded":"2024-04-13T03:09:31.017303+0000","last_fullsized":"2024-04-13T03:09:31.017303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552769+0000","last_change":"2024-04-13T03:09:23.973982+0000","last_active":"2024-04-13T03:09:31.552769+0000","last_peered":"2024-04-13T03:09:31.552769+0000","last_clean":"2024-04-13T03:09:31.552769+0000","last_became_active":"2024-04-13T03:09:23.973759+0000","last_became_peered":"2024-04-13T03:09:23.973759+0000","last_unstale":"2024-04-13T03:09:31.552769+0000","last_undegraded":"2024-04-13T03:09:31.552769+0000","last_fullsized":"2024-04-13T03:09:31.552769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016499+0000","last_change":"2024-04-13T03:09:29.567476+0000","last_active":"2024-04-13T03:09:31.016499+0000","last_peered":"2024-04-13T03:09:31.016499+0000","last_clean":"2024-04-13T03:09:31.016499+0000","last_became_active":"2024-04-13T03:09:29.567116+0000","last_became_peered":"2024-04-13T03:09:29.567116+0000","last_unstale":"2024-04-13T03:09:31.016499+0000","last_undegraded":"2024-04-13T03:09:31.016499+0000","last_fullsized":"2024-04-13T03:09:31.016499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:07.048787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552621+0000","last_change":"2024-04-13T03:09:24.974586+0000","last_active":"2024-04-13T03:09:31.552621+0000","last_peered":"2024-04-13T03:09:31.552621+0000","last_clean":"2024-04-13T03:09:31.552621+0000","last_became_active":"2024-04-13T03:09:24.974348+0000","last_became_peered":"2024-04-13T03:09:24.974348+0000","last_unstale":"2024-04-13T03:09:31.552621+0000","last_undegraded":"2024-04-13T03:09:31.552621+0000","last_fullsized":"2024-04-13T03:09:31.552621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073219+0000","last_change":"2024-04-13T03:09:23.979817+0000","last_active":"2024-04-13T03:09:32.073219+0000","last_peered":"2024-04-13T03:09:32.073219+0000","last_clean":"2024-04-13T03:09:32.073219+0000","last_became_active":"2024-04-13T03:09:23.979575+0000","last_became_peered":"2024-04-13T03:09:23.979575+0000","last_unstale":"2024-04-13T03:09:32.073219+0000","last_undegraded":"2024-04-13T03:09:32.073219+0000","last_fullsized":"2024-04-13T03:09:32.073219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:42.984781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004425+0000","last_change":"2024-04-13T03:09:29.567336+0000","last_active":"2024-04-13T03:09:31.004425+0000","last_peered":"2024-04-13T03:09:31.004425+0000","last_clean":"2024-04-13T03:09:31.004425+0000","last_became_active":"2024-04-13T03:09:29.566964+0000","last_became_peered":"2024-04-13T03:09:29.566964+0000","last_unstale":"2024-04-13T03:09:31.004425+0000","last_undegraded":"2024-04-13T03:09:31.004425+0000","last_fullsized":"2024-04-13T03:09:31.004425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005618+0000","last_change":"2024-04-13T03:09:24.976546+0000","last_active":"2024-04-13T03:09:31.005618+0000","last_peered":"2024-04-13T03:09:31.005618+0000","last_clean":"2024-04-13T03:09:31.005618+0000","last_became_active":"2024-04-13T03:09:24.974772+0000","last_became_peered":"2024-04-13T03:09:24.974772+0000","last_unstale":"2024-04-13T03:09:31.005618+0000","last_undegraded":"2024-04-13T03:09:31.005618+0000","last_fullsized":"2024-04-13T03:09:31.005618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552722+0000","last_change":"2024-04-13T03:09:23.966690+0000","last_active":"2024-04-13T03:09:31.552722+0000","last_peered":"2024-04-13T03:09:31.552722+0000","last_clean":"2024-04-13T03:09:31.552722+0000","last_became_active":"2024-04-13T03:09:23.966440+0000","last_became_peered":"2024-04-13T03:09:23.966440+0000","last_unstale":"2024-04-13T03:09:31.552722+0000","last_undegraded":"2024-04-13T03:09:31.552722+0000","last_fullsized":"2024-04-13T03:09:31.552722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552227+0000","last_change":"2024-04-13T03:09:29.013113+0000","last_active":"2024-04-13T03:09:31.552227+0000","last_peered":"2024-04-13T03:09:31.552227+0000","last_clean":"2024-04-13T03:09:31.552227+0000","last_became_active":"2024-04-13T03:09:29.012470+0000","last_became_peered":"2024-04-13T03:09:29.012470+0000","last_unstale":"2024-04-13T03:09:31.552227+0000","last_undegraded":"2024-04-13T03:09:31.552227+0000","last_fullsized":"2024-04-13T03:09:31.552227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005144+0000","last_change":"2024-04-13T03:09:24.973635+0000","last_active":"2024-04-13T03:09:31.005144+0000","last_peered":"2024-04-13T03:09:31.005144+0000","last_clean":"2024-04-13T03:09:31.005144+0000","last_became_active":"2024-04-13T03:09:24.973362+0000","last_became_peered":"2024-04-13T03:09:24.973362+0000","last_unstale":"2024-04-13T03:09:31.005144+0000","last_undegraded":"2024-04-13T03:09:31.005144+0000","last_fullsized":"2024-04-13T03:09:31.005144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:24.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006449+0000","last_change":"2024-04-13T03:09:23.968272+0000","last_active":"2024-04-13T03:09:31.006449+0000","last_peered":"2024-04-13T03:09:31.006449+0000","last_clean":"2024-04-13T03:09:31.006449+0000","last_became_active":"2024-04-13T03:09:23.967784+0000","last_became_peered":"2024-04-13T03:09:23.967784+0000","last_unstale":"2024-04-13T03:09:31.006449+0000","last_undegraded":"2024-04-13T03:09:31.006449+0000","last_fullsized":"2024-04-13T03:09:31.006449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552810+0000","last_change":"2024-04-13T03:09:29.008833+0000","last_active":"2024-04-13T03:09:31.552810+0000","last_peered":"2024-04-13T03:09:31.552810+0000","last_clean":"2024-04-13T03:09:31.552810+0000","last_became_active":"2024-04-13T03:09:29.008342+0000","last_became_peered":"2024-04-13T03:09:29.008342+0000","last_unstale":"2024-04-13T03:09:31.552810+0000","last_undegraded":"2024-04-13T03:09:31.552810+0000","last_fullsized":"2024-04-13T03:09:31.552810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552174+0000","last_change":"2024-04-13T03:09:24.975965+0000","last_active":"2024-04-13T03:09:31.552174+0000","last_peered":"2024-04-13T03:09:31.552174+0000","last_clean":"2024-04-13T03:09:31.552174+0000","last_became_active":"2024-04-13T03:09:24.975143+0000","last_became_peered":"2024-04-13T03:09:24.975143+0000","last_unstale":"2024-04-13T03:09:31.552174+0000","last_undegraded":"2024-04-13T03:09:31.552174+0000","last_fullsized":"2024-04-13T03:09:31.552174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.978132+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.976718+0000","last_became_peered":"2024-04-13T03:09:23.976718+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552953+0000","last_change":"2024-04-13T03:09:29.012295+0000","last_active":"2024-04-13T03:09:31.552953+0000","last_peered":"2024-04-13T03:09:31.552953+0000","last_clean":"2024-04-13T03:09:31.552953+0000","last_became_active":"2024-04-13T03:09:29.011908+0000","last_became_peered":"2024-04-13T03:09:29.011908+0000","last_unstale":"2024-04-13T03:09:31.552953+0000","last_undegraded":"2024-04-13T03:09:31.552953+0000","last_fullsized":"2024-04-13T03:09:31.552953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004144+0000","last_change":"2024-04-13T03:09:24.974975+0000","last_active":"2024-04-13T03:09:31.004144+0000","last_peered":"2024-04-13T03:09:31.004144+0000","last_clean":"2024-04-13T03:09:31.004144+0000","last_became_active":"2024-04-13T03:09:24.974656+0000","last_became_peered":"2024-04-13T03:09:24.974656+0000","last_unstale":"2024-04-13T03:09:31.004144+0000","last_undegraded":"2024-04-13T03:09:31.004144+0000","last_fullsized":"2024-04-13T03:09:31.004144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072979+0000","last_change":"2024-04-13T03:09:23.978671+0000","last_active":"2024-04-13T03:09:32.072979+0000","last_peered":"2024-04-13T03:09:32.072979+0000","last_clean":"2024-04-13T03:09:32.072979+0000","last_became_active":"2024-04-13T03:09:23.978358+0000","last_became_peered":"2024-04-13T03:09:23.978358+0000","last_unstale":"2024-04-13T03:09:32.072979+0000","last_undegraded":"2024-04-13T03:09:32.072979+0000","last_fullsized":"2024-04-13T03:09:32.072979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004701+0000","last_change":"2024-04-13T03:09:29.566364+0000","last_active":"2024-04-13T03:09:31.004701+0000","last_peered":"2024-04-13T03:09:31.004701+0000","last_clean":"2024-04-13T03:09:31.004701+0000","last_became_active":"2024-04-13T03:09:29.565976+0000","last_became_peered":"2024-04-13T03:09:29.565976+0000","last_unstale":"2024-04-13T03:09:31.004701+0000","last_undegraded":"2024-04-13T03:09:31.004701+0000","last_fullsized":"2024-04-13T03:09:31.004701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016580+0000","last_change":"2024-04-13T03:09:24.988431+0000","last_active":"2024-04-13T03:09:31.016580+0000","last_peered":"2024-04-13T03:09:31.016580+0000","last_clean":"2024-04-13T03:09:31.016580+0000","last_became_active":"2024-04-13T03:09:24.988179+0000","last_became_peered":"2024-04-13T03:09:24.988179+0000","last_unstale":"2024-04-13T03:09:31.016580+0000","last_undegraded":"2024-04-13T03:09:31.016580+0000","last_fullsized":"2024-04-13T03:09:31.016580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005560+0000","last_change":"2024-04-13T03:09:23.970960+0000","last_active":"2024-04-13T03:09:31.005560+0000","last_peered":"2024-04-13T03:09:31.005560+0000","last_clean":"2024-04-13T03:09:31.005560+0000","last_became_active":"2024-04-13T03:09:23.970615+0000","last_became_peered":"2024-04-13T03:09:23.970615+0000","last_unstale":"2024-04-13T03:09:31.005560+0000","last_undegraded":"2024-04-13T03:09:31.005560+0000","last_fullsized":"2024-04-13T03:09:31.005560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073625+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:09:32.073625+0000","last_peered":"2024-04-13T03:09:32.073625+0000","last_clean":"2024-04-13T03:09:32.073625+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:09:32.073625+0000","last_undegraded":"2024-04-13T03:09:32.073625+0000","last_fullsized":"2024-04-13T03:09:32.073625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074664+0000","last_change":"2024-04-13T03:09:29.570505+0000","last_active":"2024-04-13T03:09:32.074664+0000","last_peered":"2024-04-13T03:09:32.074664+0000","last_clean":"2024-04-13T03:09:32.074664+0000","last_became_active":"2024-04-13T03:09:29.569637+0000","last_became_peered":"2024-04-13T03:09:29.569637+0000","last_unstale":"2024-04-13T03:09:32.074664+0000","last_undegraded":"2024-04-13T03:09:32.074664+0000","last_fullsized":"2024-04-13T03:09:32.074664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073883+0000","last_change":"2024-04-13T03:09:24.990807+0000","last_active":"2024-04-13T03:09:32.073883+0000","last_peered":"2024-04-13T03:09:32.073883+0000","last_clean":"2024-04-13T03:09:32.073883+0000","last_became_active":"2024-04-13T03:09:24.990454+0000","last_became_peered":"2024-04-13T03:09:24.990454+0000","last_unstale":"2024-04-13T03:09:32.073883+0000","last_undegraded":"2024-04-13T03:09:32.073883+0000","last_fullsized":"2024-04-13T03:09:32.073883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072543+0000","last_change":"2024-04-13T03:09:23.974849+0000","last_active":"2024-04-13T03:09:32.072543+0000","last_peered":"2024-04-13T03:09:32.072543+0000","last_clean":"2024-04-13T03:09:32.072543+0000","last_became_active":"2024-04-13T03:09:23.974564+0000","last_became_peered":"2024-04-13T03:09:23.974564+0000","last_unstale":"2024-04-13T03:09:32.072543+0000","last_undegraded":"2024-04-13T03:09:32.072543+0000","last_fullsized":"2024-04-13T03:09:32.072543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073496+0000","last_change":"2024-04-13T03:09:29.567076+0000","last_active":"2024-04-13T03:09:32.073496+0000","last_peered":"2024-04-13T03:09:32.073496+0000","last_clean":"2024-04-13T03:09:32.073496+0000","last_became_active":"2024-04-13T03:09:29.566162+0000","last_became_peered":"2024-04-13T03:09:29.566162+0000","last_unstale":"2024-04-13T03:09:32.073496+0000","last_undegraded":"2024-04-13T03:09:32.073496+0000","last_fullsized":"2024-04-13T03:09:32.073496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006229+0000","last_change":"2024-04-13T03:09:24.986470+0000","last_active":"2024-04-13T03:09:31.006229+0000","last_peered":"2024-04-13T03:09:31.006229+0000","last_clean":"2024-04-13T03:09:31.006229+0000","last_became_active":"2024-04-13T03:09:24.986035+0000","last_became_peered":"2024-04-13T03:09:24.986035+0000","last_unstale":"2024-04-13T03:09:31.006229+0000","last_undegraded":"2024-04-13T03:09:31.006229+0000","last_fullsized":"2024-04-13T03:09:31.006229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552711+0000","last_change":"2024-04-13T03:09:23.966911+0000","last_active":"2024-04-13T03:09:31.552711+0000","last_peered":"2024-04-13T03:09:31.552711+0000","last_clean":"2024-04-13T03:09:31.552711+0000","last_became_active":"2024-04-13T03:09:23.966494+0000","last_became_peered":"2024-04-13T03:09:23.966494+0000","last_unstale":"2024-04-13T03:09:31.552711+0000","last_undegraded":"2024-04-13T03:09:31.552711+0000","last_fullsized":"2024-04-13T03:09:31.552711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004556+0000","last_change":"2024-04-13T03:09:29.008472+0000","last_active":"2024-04-13T03:09:31.004556+0000","last_peered":"2024-04-13T03:09:31.004556+0000","last_clean":"2024-04-13T03:09:31.004556+0000","last_became_active":"2024-04-13T03:09:29.008101+0000","last_became_peered":"2024-04-13T03:09:29.008101+0000","last_unstale":"2024-04-13T03:09:31.004556+0000","last_undegraded":"2024-04-13T03:09:31.004556+0000","last_fullsized":"2024-04-13T03:09:31.004556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074447+0000","last_change":"2024-04-13T03:09:24.978817+0000","last_active":"2024-04-13T03:09:32.074447+0000","last_peered":"2024-04-13T03:09:32.074447+0000","last_clean":"2024-04-13T03:09:32.074447+0000","last_became_active":"2024-04-13T03:09:24.978091+0000","last_became_peered":"2024-04-13T03:09:24.978091+0000","last_unstale":"2024-04-13T03:09:32.074447+0000","last_undegraded":"2024-04-13T03:09:32.074447+0000","last_fullsized":"2024-04-13T03:09:32.074447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:28.224846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072548+0000","last_change":"2024-04-13T03:09:23.978022+0000","last_active":"2024-04-13T03:09:32.072548+0000","last_peered":"2024-04-13T03:09:32.072548+0000","last_clean":"2024-04-13T03:09:32.072548+0000","last_became_active":"2024-04-13T03:09:23.976602+0000","last_became_peered":"2024-04-13T03:09:23.976602+0000","last_unstale":"2024-04-13T03:09:32.072548+0000","last_undegraded":"2024-04-13T03:09:32.072548+0000","last_fullsized":"2024-04-13T03:09:32.072548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005100+0000","last_change":"2024-04-13T03:09:29.567904+0000","last_active":"2024-04-13T03:09:31.005100+0000","last_peered":"2024-04-13T03:09:31.005100+0000","last_clean":"2024-04-13T03:09:31.005100+0000","last_became_active":"2024-04-13T03:09:29.567497+0000","last_became_peered":"2024-04-13T03:09:29.567497+0000","last_unstale":"2024-04-13T03:09:31.005100+0000","last_undegraded":"2024-04-13T03:09:31.005100+0000","last_fullsized":"2024-04-13T03:09:31.005100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074412+0000","last_change":"2024-04-13T03:09:24.978521+0000","last_active":"2024-04-13T03:09:32.074412+0000","last_peered":"2024-04-13T03:09:32.074412+0000","last_clean":"2024-04-13T03:09:32.074412+0000","last_became_active":"2024-04-13T03:09:24.978163+0000","last_became_peered":"2024-04-13T03:09:24.978163+0000","last_unstale":"2024-04-13T03:09:32.074412+0000","last_undegraded":"2024-04-13T03:09:32.074412+0000","last_fullsized":"2024-04-13T03:09:32.074412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006024+0000","last_change":"2024-04-13T03:09:23.966891+0000","last_active":"2024-04-13T03:09:31.006024+0000","last_peered":"2024-04-13T03:09:31.006024+0000","last_clean":"2024-04-13T03:09:31.006024+0000","last_became_active":"2024-04-13T03:09:23.966596+0000","last_became_peered":"2024-04-13T03:09:23.966596+0000","last_unstale":"2024-04-13T03:09:31.006024+0000","last_undegraded":"2024-04-13T03:09:31.006024+0000","last_fullsized":"2024-04-13T03:09:31.006024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:29.744523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004384+0000","last_change":"2024-04-13T03:09:29.007029+0000","last_active":"2024-04-13T03:09:31.004384+0000","last_peered":"2024-04-13T03:09:31.004384+0000","last_clean":"2024-04-13T03:09:31.004384+0000","last_became_active":"2024-04-13T03:09:29.006625+0000","last_became_peered":"2024-04-13T03:09:29.006625+0000","last_unstale":"2024-04-13T03:09:31.004384+0000","last_undegraded":"2024-04-13T03:09:31.004384+0000","last_fullsized":"2024-04-13T03:09:31.004384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074033+0000","last_change":"2024-04-13T03:09:24.978991+0000","last_active":"2024-04-13T03:09:32.074033+0000","last_peered":"2024-04-13T03:09:32.074033+0000","last_clean":"2024-04-13T03:09:32.074033+0000","last_became_active":"2024-04-13T03:09:24.978725+0000","last_became_peered":"2024-04-13T03:09:24.978725+0000","last_unstale":"2024-04-13T03:09:32.074033+0000","last_undegraded":"2024-04-13T03:09:32.074033+0000","last_fullsized":"2024-04-13T03:09:32.074033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.969378+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.969098+0000","last_became_peered":"2024-04-13T03:09:23.969098+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004694+0000","last_change":"2024-04-13T03:09:29.005610+0000","last_active":"2024-04-13T03:09:31.004694+0000","last_peered":"2024-04-13T03:09:31.004694+0000","last_clean":"2024-04-13T03:09:31.004694+0000","last_became_active":"2024-04-13T03:09:29.005192+0000","last_became_peered":"2024-04-13T03:09:29.005192+0000","last_unstale":"2024-04-13T03:09:31.004694+0000","last_undegraded":"2024-04-13T03:09:31.004694+0000","last_fullsized":"2024-04-13T03:09:31.004694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073718+0000","last_change":"2024-04-13T03:09:24.984278+0000","last_active":"2024-04-13T03:09:32.073718+0000","last_peered":"2024-04-13T03:09:32.073718+0000","last_clean":"2024-04-13T03:09:32.073718+0000","last_became_active":"2024-04-13T03:09:24.983967+0000","last_became_peered":"2024-04-13T03:09:24.983967+0000","last_unstale":"2024-04-13T03:09:32.073718+0000","last_undegraded":"2024-04-13T03:09:32.073718+0000","last_fullsized":"2024-04-13T03:09:32.073718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072991+0000","last_change":"2024-04-13T03:09:23.975431+0000","last_active":"2024-04-13T03:09:32.072991+0000","last_peered":"2024-04-13T03:09:32.072991+0000","last_clean":"2024-04-13T03:09:32.072991+0000","last_became_active":"2024-04-13T03:09:23.974255+0000","last_became_peered":"2024-04-13T03:09:23.974255+0000","last_unstale":"2024-04-13T03:09:32.072991+0000","last_undegraded":"2024-04-13T03:09:32.072991+0000","last_fullsized":"2024-04-13T03:09:32.072991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017273+0000","last_change":"2024-04-13T03:09:29.567710+0000","last_active":"2024-04-13T03:09:31.017273+0000","last_peered":"2024-04-13T03:09:31.017273+0000","last_clean":"2024-04-13T03:09:31.017273+0000","last_became_active":"2024-04-13T03:09:29.567230+0000","last_became_peered":"2024-04-13T03:09:29.567230+0000","last_unstale":"2024-04-13T03:09:31.017273+0000","last_undegraded":"2024-04-13T03:09:31.017273+0000","last_fullsized":"2024-04-13T03:09:31.017273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:23:03.577409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073579+0000","last_change":"2024-04-13T03:09:24.978027+0000","last_active":"2024-04-13T03:09:32.073579+0000","last_peered":"2024-04-13T03:09:32.073579+0000","last_clean":"2024-04-13T03:09:32.073579+0000","last_became_active":"2024-04-13T03:09:24.977709+0000","last_became_peered":"2024-04-13T03:09:24.977709+0000","last_unstale":"2024-04-13T03:09:32.073579+0000","last_undegraded":"2024-04-13T03:09:32.073579+0000","last_fullsized":"2024-04-13T03:09:32.073579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006349+0000","last_change":"2024-04-13T03:09:23.972192+0000","last_active":"2024-04-13T03:09:31.006349+0000","last_peered":"2024-04-13T03:09:31.006349+0000","last_clean":"2024-04-13T03:09:31.006349+0000","last_became_active":"2024-04-13T03:09:23.971856+0000","last_became_peered":"2024-04-13T03:09:23.971856+0000","last_unstale":"2024-04-13T03:09:31.006349+0000","last_undegraded":"2024-04-13T03:09:31.006349+0000","last_fullsized":"2024-04-13T03:09:31.006349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017103+0000","last_change":"2024-04-13T03:09:29.565473+0000","last_active":"2024-04-13T03:09:31.017103+0000","last_peered":"2024-04-13T03:09:31.017103+0000","last_clean":"2024-04-13T03:09:31.017103+0000","last_became_active":"2024-04-13T03:09:29.565099+0000","last_became_peered":"2024-04-13T03:09:29.565099+0000","last_unstale":"2024-04-13T03:09:31.017103+0000","last_undegraded":"2024-04-13T03:09:31.017103+0000","last_fullsized":"2024-04-13T03:09:31.017103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:49:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004475+0000","last_change":"2024-04-13T03:09:24.980510+0000","last_active":"2024-04-13T03:09:31.004475+0000","last_peered":"2024-04-13T03:09:31.004475+0000","last_clean":"2024-04-13T03:09:31.004475+0000","last_became_active":"2024-04-13T03:09:24.980226+0000","last_became_peered":"2024-04-13T03:09:24.980226+0000","last_unstale":"2024-04-13T03:09:31.004475+0000","last_undegraded":"2024-04-13T03:09:31.004475+0000","last_fullsized":"2024-04-13T03:09:31.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004618+0000","last_change":"2024-04-13T03:09:23.971205+0000","last_active":"2024-04-13T03:09:31.004618+0000","last_peered":"2024-04-13T03:09:31.004618+0000","last_clean":"2024-04-13T03:09:31.004618+0000","last_became_active":"2024-04-13T03:09:23.970845+0000","last_became_peered":"2024-04-13T03:09:23.970845+0000","last_unstale":"2024-04-13T03:09:31.004618+0000","last_undegraded":"2024-04-13T03:09:31.004618+0000","last_fullsized":"2024-04-13T03:09:31.004618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005480+0000","last_change":"2024-04-13T03:09:29.568847+0000","last_active":"2024-04-13T03:09:31.005480+0000","last_peered":"2024-04-13T03:09:31.005480+0000","last_clean":"2024-04-13T03:09:31.005480+0000","last_became_active":"2024-04-13T03:09:29.568286+0000","last_became_peered":"2024-04-13T03:09:29.568286+0000","last_unstale":"2024-04-13T03:09:31.005480+0000","last_undegraded":"2024-04-13T03:09:31.005480+0000","last_fullsized":"2024-04-13T03:09:31.005480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073615+0000","last_change":"2024-04-13T03:09:24.980651+0000","last_active":"2024-04-13T03:09:32.073615+0000","last_peered":"2024-04-13T03:09:32.073615+0000","last_clean":"2024-04-13T03:09:32.073615+0000","last_became_active":"2024-04-13T03:09:24.980390+0000","last_became_peered":"2024-04-13T03:09:24.980390+0000","last_unstale":"2024-04-13T03:09:32.073615+0000","last_undegraded":"2024-04-13T03:09:32.073615+0000","last_fullsized":"2024-04-13T03:09:32.073615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:34.382842+0000","last_change":"2024-04-13T03:09:23.977841+0000","last_active":"2024-04-13T03:09:34.382842+0000","last_peered":"2024-04-13T03:09:34.382842+0000","last_clean":"2024-04-13T03:09:34.382842+0000","last_became_active":"2024-04-13T03:09:23.977546+0000","last_became_peered":"2024-04-13T03:09:23.977546+0000","last_unstale":"2024-04-13T03:09:34.382842+0000","last_undegraded":"2024-04-13T03:09:34.382842+0000","last_fullsized":"2024-04-13T03:09:34.382842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552322+0000","last_change":"2024-04-13T03:09:29.012775+0000","last_active":"2024-04-13T03:09:31.552322+0000","last_peered":"2024-04-13T03:09:31.552322+0000","last_clean":"2024-04-13T03:09:31.552322+0000","last_became_active":"2024-04-13T03:09:29.011952+0000","last_became_peered":"2024-04-13T03:09:29.011952+0000","last_unstale":"2024-04-13T03:09:31.552322+0000","last_undegraded":"2024-04-13T03:09:31.552322+0000","last_fullsized":"2024-04-13T03:09:31.552322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005496+0000","last_change":"2024-04-13T03:09:24.980974+0000","last_active":"2024-04-13T03:09:31.005496+0000","last_peered":"2024-04-13T03:09:31.005496+0000","last_clean":"2024-04-13T03:09:31.005496+0000","last_became_active":"2024-04-13T03:09:24.980580+0000","last_became_peered":"2024-04-13T03:09:24.980580+0000","last_unstale":"2024-04-13T03:09:31.005496+0000","last_undegraded":"2024-04-13T03:09:31.005496+0000","last_fullsized":"2024-04-13T03:09:31.005496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552665+0000","last_change":"2024-04-13T03:09:23.970652+0000","last_active":"2024-04-13T03:09:31.552665+0000","last_peered":"2024-04-13T03:09:31.552665+0000","last_clean":"2024-04-13T03:09:31.552665+0000","last_became_active":"2024-04-13T03:09:23.970405+0000","last_became_peered":"2024-04-13T03:09:23.970405+0000","last_unstale":"2024-04-13T03:09:31.552665+0000","last_undegraded":"2024-04-13T03:09:31.552665+0000","last_fullsized":"2024-04-13T03:09:31.552665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004925+0000","last_change":"2024-04-13T03:09:29.009723+0000","last_active":"2024-04-13T03:09:31.004925+0000","last_peered":"2024-04-13T03:09:31.004925+0000","last_clean":"2024-04-13T03:09:31.004925+0000","last_became_active":"2024-04-13T03:09:29.009177+0000","last_became_peered":"2024-04-13T03:09:29.009177+0000","last_unstale":"2024-04-13T03:09:31.004925+0000","last_undegraded":"2024-04-13T03:09:31.004925+0000","last_fullsized":"2024-04-13T03:09:31.004925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004826+0000","last_change":"2024-04-13T03:09:24.976839+0000","last_active":"2024-04-13T03:09:31.004826+0000","last_peered":"2024-04-13T03:09:31.004826+0000","last_clean":"2024-04-13T03:09:31.004826+0000","last_became_active":"2024-04-13T03:09:24.976565+0000","last_became_peered":"2024-04-13T03:09:24.976565+0000","last_unstale":"2024-04-13T03:09:31.004826+0000","last_undegraded":"2024-04-13T03:09:31.004826+0000","last_fullsized":"2024-04-13T03:09:31.004826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005802+0000","last_change":"2024-04-13T03:09:23.974057+0000","last_active":"2024-04-13T03:09:31.005802+0000","last_peered":"2024-04-13T03:09:31.005802+0000","last_clean":"2024-04-13T03:09:31.005802+0000","last_became_active":"2024-04-13T03:09:23.973783+0000","last_became_peered":"2024-04-13T03:09:23.973783+0000","last_unstale":"2024-04-13T03:09:31.005802+0000","last_undegraded":"2024-04-13T03:09:31.005802+0000","last_fullsized":"2024-04-13T03:09:31.005802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016630+0000","last_change":"2024-04-13T03:09:29.008489+0000","last_active":"2024-04-13T03:09:31.016630+0000","last_peered":"2024-04-13T03:09:31.016630+0000","last_clean":"2024-04-13T03:09:31.016630+0000","last_became_active":"2024-04-13T03:09:29.008129+0000","last_became_peered":"2024-04-13T03:09:29.008129+0000","last_unstale":"2024-04-13T03:09:31.016630+0000","last_undegraded":"2024-04-13T03:09:31.016630+0000","last_fullsized":"2024-04-13T03:09:31.016630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:03.613440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005713+0000","last_change":"2024-04-13T03:09:24.988904+0000","last_active":"2024-04-13T03:09:31.005713+0000","last_peered":"2024-04-13T03:09:31.005713+0000","last_clean":"2024-04-13T03:09:31.005713+0000","last_became_active":"2024-04-13T03:09:24.988667+0000","last_became_peered":"2024-04-13T03:09:24.988667+0000","last_unstale":"2024-04-13T03:09:31.005713+0000","last_undegraded":"2024-04-13T03:09:31.005713+0000","last_fullsized":"2024-04-13T03:09:31.005713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074106+0000","last_change":"2024-04-13T03:09:23.975478+0000","last_active":"2024-04-13T03:09:32.074106+0000","last_peered":"2024-04-13T03:09:32.074106+0000","last_clean":"2024-04-13T03:09:32.074106+0000","last_became_active":"2024-04-13T03:09:23.975155+0000","last_became_peered":"2024-04-13T03:09:23.975155+0000","last_unstale":"2024-04-13T03:09:32.074106+0000","last_undegraded":"2024-04-13T03:09:32.074106+0000","last_fullsized":"2024-04-13T03:09:32.074106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016599+0000","last_change":"2024-04-13T03:09:29.565039+0000","last_active":"2024-04-13T03:09:31.016599+0000","last_peered":"2024-04-13T03:09:31.016599+0000","last_clean":"2024-04-13T03:09:31.016599+0000","last_became_active":"2024-04-13T03:09:29.564715+0000","last_became_peered":"2024-04-13T03:09:29.564715+0000","last_unstale":"2024-04-13T03:09:31.016599+0000","last_undegraded":"2024-04-13T03:09:31.016599+0000","last_fullsized":"2024-04-13T03:09:31.016599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:38.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004534+0000","last_change":"2024-04-13T03:09:29.567360+0000","last_active":"2024-04-13T03:09:31.004534+0000","last_peered":"2024-04-13T03:09:31.004534+0000","last_clean":"2024-04-13T03:09:31.004534+0000","last_became_active":"2024-04-13T03:09:29.567005+0000","last_became_peered":"2024-04-13T03:09:29.567005+0000","last_unstale":"2024-04-13T03:09:31.004534+0000","last_undegraded":"2024-04-13T03:09:31.004534+0000","last_fullsized":"2024-04-13T03:09:31.004534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552017+0000","last_change":"2024-04-13T03:09:24.980403+0000","last_active":"2024-04-13T03:09:31.552017+0000","last_peered":"2024-04-13T03:09:31.552017+0000","last_clean":"2024-04-13T03:09:31.552017+0000","last_became_active":"2024-04-13T03:09:24.980125+0000","last_became_peered":"2024-04-13T03:09:24.980125+0000","last_unstale":"2024-04-13T03:09:31.552017+0000","last_undegraded":"2024-04-13T03:09:31.552017+0000","last_fullsized":"2024-04-13T03:09:31.552017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006298+0000","last_change":"2024-04-13T03:09:23.966400+0000","last_active":"2024-04-13T03:09:31.006298+0000","last_peered":"2024-04-13T03:09:31.006298+0000","last_clean":"2024-04-13T03:09:31.006298+0000","last_became_active":"2024-04-13T03:09:23.966136+0000","last_became_peered":"2024-04-13T03:09:23.966136+0000","last_unstale":"2024-04-13T03:09:31.006298+0000","last_undegraded":"2024-04-13T03:09:31.006298+0000","last_fullsized":"2024-04-13T03:09:31.006298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074769+0000","last_change":"2024-04-13T03:09:29.008579+0000","last_active":"2024-04-13T03:09:32.074769+0000","last_peered":"2024-04-13T03:09:32.074769+0000","last_clean":"2024-04-13T03:09:32.074769+0000","last_became_active":"2024-04-13T03:09:29.008030+0000","last_became_peered":"2024-04-13T03:09:29.008030+0000","last_unstale":"2024-04-13T03:09:32.074769+0000","last_undegraded":"2024-04-13T03:09:32.074769+0000","last_fullsized":"2024-04-13T03:09:32.074769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016574+0000","last_change":"2024-04-13T03:09:24.972241+0000","last_active":"2024-04-13T03:09:31.016574+0000","last_peered":"2024-04-13T03:09:31.016574+0000","last_clean":"2024-04-13T03:09:31.016574+0000","last_became_active":"2024-04-13T03:09:24.971962+0000","last_became_peered":"2024-04-13T03:09:24.971962+0000","last_unstale":"2024-04-13T03:09:31.016574+0000","last_undegraded":"2024-04-13T03:09:31.016574+0000","last_fullsized":"2024-04-13T03:09:31.016574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016806+0000","last_change":"2024-04-13T03:09:23.977688+0000","last_active":"2024-04-13T03:09:31.016806+0000","last_peered":"2024-04-13T03:09:31.016806+0000","last_clean":"2024-04-13T03:09:31.016806+0000","last_became_active":"2024-04-13T03:09:23.977412+0000","last_became_peered":"2024-04-13T03:09:23.977412+0000","last_unstale":"2024-04-13T03:09:31.016806+0000","last_undegraded":"2024-04-13T03:09:31.016806+0000","last_fullsized":"2024-04-13T03:09:31.016806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072995+0000","last_change":"2024-04-13T03:09:23.973771+0000","last_active":"2024-04-13T03:09:32.072995+0000","last_peered":"2024-04-13T03:09:32.072995+0000","last_clean":"2024-04-13T03:09:32.072995+0000","last_became_active":"2024-04-13T03:09:23.973271+0000","last_became_peered":"2024-04-13T03:09:23.973271+0000","last_unstale":"2024-04-13T03:09:32.072995+0000","last_undegraded":"2024-04-13T03:09:32.072995+0000","last_fullsized":"2024-04-13T03:09:32.072995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004613+0000","last_change":"2024-04-13T03:09:24.974833+0000","last_active":"2024-04-13T03:09:31.004613+0000","last_peered":"2024-04-13T03:09:31.004613+0000","last_clean":"2024-04-13T03:09:31.004613+0000","last_became_active":"2024-04-13T03:09:24.974554+0000","last_became_peered":"2024-04-13T03:09:24.974554+0000","last_unstale":"2024-04-13T03:09:31.004613+0000","last_undegraded":"2024-04-13T03:09:31.004613+0000","last_fullsized":"2024-04-13T03:09:31.004613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072905+0000","last_change":"2024-04-13T03:09:29.004327+0000","last_active":"2024-04-13T03:09:32.072905+0000","last_peered":"2024-04-13T03:09:32.072905+0000","last_clean":"2024-04-13T03:09:32.072905+0000","last_became_active":"2024-04-13T03:09:29.003998+0000","last_became_peered":"2024-04-13T03:09:29.003998+0000","last_unstale":"2024-04-13T03:09:32.072905+0000","last_undegraded":"2024-04-13T03:09:32.072905+0000","last_fullsized":"2024-04-13T03:09:32.072905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:10:12.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016845+0000","last_change":"2024-04-13T03:09:23.976680+0000","last_active":"2024-04-13T03:09:31.016845+0000","last_peered":"2024-04-13T03:09:31.016845+0000","last_clean":"2024-04-13T03:09:31.016845+0000","last_became_active":"2024-04-13T03:09:23.976390+0000","last_became_peered":"2024-04-13T03:09:23.976390+0000","last_unstale":"2024-04-13T03:09:31.016845+0000","last_undegraded":"2024-04-13T03:09:31.016845+0000","last_fullsized":"2024-04-13T03:09:31.016845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004533+0000","last_change":"2024-04-13T03:09:24.971520+0000","last_active":"2024-04-13T03:09:31.004533+0000","last_peered":"2024-04-13T03:09:31.004533+0000","last_clean":"2024-04-13T03:09:31.004533+0000","last_became_active":"2024-04-13T03:09:24.971236+0000","last_became_peered":"2024-04-13T03:09:24.971236+0000","last_unstale":"2024-04-13T03:09:31.004533+0000","last_undegraded":"2024-04-13T03:09:31.004533+0000","last_fullsized":"2024-04-13T03:09:31.004533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:47.399993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017010+0000","last_change":"2024-04-13T03:09:29.008243+0000","last_active":"2024-04-13T03:09:31.017010+0000","last_peered":"2024-04-13T03:09:31.017010+0000","last_clean":"2024-04-13T03:09:31.017010+0000","last_became_active":"2024-04-13T03:09:29.007854+0000","last_became_peered":"2024-04-13T03:09:29.007854+0000","last_unstale":"2024-04-13T03:09:31.017010+0000","last_undegraded":"2024-04-13T03:09:31.017010+0000","last_fullsized":"2024-04-13T03:09:31.017010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:33.651724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073532+0000","last_change":"2024-04-13T03:09:24.984919+0000","last_active":"2024-04-13T03:09:32.073532+0000","last_peered":"2024-04-13T03:09:32.073532+0000","last_clean":"2024-04-13T03:09:32.073532+0000","last_became_active":"2024-04-13T03:09:24.984652+0000","last_became_peered":"2024-04-13T03:09:24.984652+0000","last_unstale":"2024-04-13T03:09:32.073532+0000","last_undegraded":"2024-04-13T03:09:32.073532+0000","last_fullsized":"2024-04-13T03:09:32.073532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072697+0000","last_change":"2024-04-13T03:09:24.980195+0000","last_active":"2024-04-13T03:09:32.072697+0000","last_peered":"2024-04-13T03:09:32.072697+0000","last_clean":"2024-04-13T03:09:32.072697+0000","last_became_active":"2024-04-13T03:09:24.979846+0000","last_became_peered":"2024-04-13T03:09:24.979846+0000","last_unstale":"2024-04-13T03:09:32.072697+0000","last_undegraded":"2024-04-13T03:09:32.072697+0000","last_fullsized":"2024-04-13T03:09:32.072697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006342+0000","last_change":"2024-04-13T03:09:24.979746+0000","last_active":"2024-04-13T03:09:31.006342+0000","last_peered":"2024-04-13T03:09:31.006342+0000","last_clean":"2024-04-13T03:09:31.006342+0000","last_became_active":"2024-04-13T03:09:24.979481+0000","last_became_peered":"2024-04-13T03:09:24.979481+0000","last_unstale":"2024-04-13T03:09:31.006342+0000","last_undegraded":"2024-04-13T03:09:31.006342+0000","last_fullsized":"2024-04-13T03:09:31.006342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004510+0000","last_change":"2024-04-13T03:09:24.974603+0000","last_active":"2024-04-13T03:09:31.004510+0000","last_peered":"2024-04-13T03:09:31.004510+0000","last_clean":"2024-04-13T03:09:31.004510+0000","last_became_active":"2024-04-13T03:09:24.974099+0000","last_became_peered":"2024-04-13T03:09:24.974099+0000","last_unstale":"2024-04-13T03:09:31.004510+0000","last_undegraded":"2024-04-13T03:09:31.004510+0000","last_fullsized":"2024-04-13T03:09:31.004510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017151+0000","last_change":"2024-04-13T03:09:24.977844+0000","last_active":"2024-04-13T03:09:31.017151+0000","last_peered":"2024-04-13T03:09:31.017151+0000","last_clean":"2024-04-13T03:09:31.017151+0000","last_became_active":"2024-04-13T03:09:24.977578+0000","last_became_peered":"2024-04-13T03:09:24.977578+0000","last_unstale":"2024-04-13T03:09:31.017151+0000","last_undegraded":"2024-04-13T03:09:31.017151+0000","last_fullsized":"2024-04-13T03:09:31.017151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552548+0000","last_change":"2024-04-13T03:09:24.981088+0000","last_active":"2024-04-13T03:09:31.552548+0000","last_peered":"2024-04-13T03:09:31.552548+0000","last_clean":"2024-04-13T03:09:31.552548+0000","last_became_active":"2024-04-13T03:09:24.980793+0000","last_became_peered":"2024-04-13T03:09:24.980793+0000","last_unstale":"2024-04-13T03:09:31.552548+0000","last_undegraded":"2024-04-13T03:09:31.552548+0000","last_fullsized":"2024-04-13T03:09:31.552548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074858+0000","last_change":"2024-04-13T03:09:24.990583+0000","last_active":"2024-04-13T03:09:32.074858+0000","last_peered":"2024-04-13T03:09:32.074858+0000","last_clean":"2024-04-13T03:09:32.074858+0000","last_became_active":"2024-04-13T03:09:24.990319+0000","last_became_peered":"2024-04-13T03:09:24.990319+0000","last_unstale":"2024-04-13T03:09:32.074858+0000","last_undegraded":"2024-04-13T03:09:32.074858+0000","last_fullsized":"2024-04-13T03:09:32.074858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017256+0000","last_change":"2024-04-13T03:09:24.975238+0000","last_active":"2024-04-13T03:09:31.017256+0000","last_peered":"2024-04-13T03:09:31.017256+0000","last_clean":"2024-04-13T03:09:31.017256+0000","last_became_active":"2024-04-13T03:09:24.974911+0000","last_became_peered":"2024-04-13T03:09:24.974911+0000","last_unstale":"2024-04-13T03:09:31.017256+0000","last_undegraded":"2024-04-13T03:09:31.017256+0000","last_fullsized":"2024-04-13T03:09:31.017256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005161+0000","last_change":"2024-04-13T03:09:24.976029+0000","last_active":"2024-04-13T03:09:31.005161+0000","last_peered":"2024-04-13T03:09:31.005161+0000","last_clean":"2024-04-13T03:09:31.005161+0000","last_became_active":"2024-04-13T03:09:24.975739+0000","last_became_peered":"2024-04-13T03:09:24.975739+0000","last_unstale":"2024-04-13T03:09:31.005161+0000","last_undegraded":"2024-04-13T03:09:31.005161+0000","last_fullsized":"2024-04-13T03:09:31.005161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016664+0000","last_change":"2024-04-13T03:09:24.975447+0000","last_active":"2024-04-13T03:09:31.016664+0000","last_peered":"2024-04-13T03:09:31.016664+0000","last_clean":"2024-04-13T03:09:31.016664+0000","last_became_active":"2024-04-13T03:09:24.975131+0000","last_became_peered":"2024-04-13T03:09:24.975131+0000","last_unstale":"2024-04-13T03:09:31.016664+0000","last_undegraded":"2024-04-13T03:09:31.016664+0000","last_fullsized":"2024-04-13T03:09:31.016664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006028+0000","last_change":"2024-04-13T03:09:24.988399+0000","last_active":"2024-04-13T03:09:31.006028+0000","last_peered":"2024-04-13T03:09:31.006028+0000","last_clean":"2024-04-13T03:09:31.006028+0000","last_became_active":"2024-04-13T03:09:24.988064+0000","last_became_peered":"2024-04-13T03:09:24.988064+0000","last_unstale":"2024-04-13T03:09:31.006028+0000","last_undegraded":"2024-04-13T03:09:31.006028+0000","last_fullsized":"2024-04-13T03:09:31.006028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004399+0000","last_change":"2024-04-13T03:09:24.980428+0000","last_active":"2024-04-13T03:09:31.004399+0000","last_peered":"2024-04-13T03:09:31.004399+0000","last_clean":"2024-04-13T03:09:31.004399+0000","last_became_active":"2024-04-13T03:09:24.980157+0000","last_became_peered":"2024-04-13T03:09:24.980157+0000","last_unstale":"2024-04-13T03:09:31.004399+0000","last_undegraded":"2024-04-13T03:09:31.004399+0000","last_fullsized":"2024-04-13T03:09:31.004399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074511+0000","last_change":"2024-04-13T03:09:24.978696+0000","last_active":"2024-04-13T03:09:32.074511+0000","last_peered":"2024-04-13T03:09:32.074511+0000","last_clean":"2024-04-13T03:09:32.074511+0000","last_became_active":"2024-04-13T03:09:24.978413+0000","last_became_peered":"2024-04-13T03:09:24.978413+0000","last_unstale":"2024-04-13T03:09:32.074511+0000","last_undegraded":"2024-04-13T03:09:32.074511+0000","last_fullsized":"2024-04-13T03:09:32.074511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074117+0000","last_change":"2024-04-13T03:09:24.979845+0000","last_active":"2024-04-13T03:09:32.074117+0000","last_peered":"2024-04-13T03:09:32.074117+0000","last_clean":"2024-04-13T03:09:32.074117+0000","last_became_active":"2024-04-13T03:09:24.979598+0000","last_became_peered":"2024-04-13T03:09:24.979598+0000","last_unstale":"2024-04-13T03:09:32.074117+0000","last_undegraded":"2024-04-13T03:09:32.074117+0000","last_fullsized":"2024-04-13T03:09:32.074117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073041+0000","last_change":"2024-04-13T03:09:24.978510+0000","last_active":"2024-04-13T03:09:32.073041+0000","last_peered":"2024-04-13T03:09:32.073041+0000","last_clean":"2024-04-13T03:09:32.073041+0000","last_became_active":"2024-04-13T03:09:24.978226+0000","last_became_peered":"2024-04-13T03:09:24.978226+0000","last_unstale":"2024-04-13T03:09:32.073041+0000","last_undegraded":"2024-04-13T03:09:32.073041+0000","last_fullsized":"2024-04-13T03:09:32.073041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006512+0000","last_change":"2024-04-13T03:09:24.986783+0000","last_active":"2024-04-13T03:09:31.006512+0000","last_peered":"2024-04-13T03:09:31.006512+0000","last_clean":"2024-04-13T03:09:31.006512+0000","last_became_active":"2024-04-13T03:09:24.986139+0000","last_became_peered":"2024-04-13T03:09:24.986139+0000","last_unstale":"2024-04-13T03:09:31.006512+0000","last_undegraded":"2024-04-13T03:09:31.006512+0000","last_fullsized":"2024-04-13T03:09:31.006512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005046+0000","last_change":"2024-04-13T03:09:24.974308+0000","last_active":"2024-04-13T03:09:31.005046+0000","last_peered":"2024-04-13T03:09:31.005046+0000","last_clean":"2024-04-13T03:09:31.005046+0000","last_became_active":"2024-04-13T03:09:24.973935+0000","last_became_peered":"2024-04-13T03:09:24.973935+0000","last_unstale":"2024-04-13T03:09:31.005046+0000","last_undegraded":"2024-04-13T03:09:31.005046+0000","last_fullsized":"2024-04-13T03:09:31.005046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552097+0000","last_change":"2024-04-13T03:09:24.971657+0000","last_active":"2024-04-13T03:09:31.552097+0000","last_peered":"2024-04-13T03:09:31.552097+0000","last_clean":"2024-04-13T03:09:31.552097+0000","last_became_active":"2024-04-13T03:09:24.971379+0000","last_became_peered":"2024-04-13T03:09:24.971379+0000","last_unstale":"2024-04-13T03:09:31.552097+0000","last_undegraded":"2024-04-13T03:09:31.552097+0000","last_fullsized":"2024-04-13T03:09:31.552097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004364+0000","last_change":"2024-04-13T03:09:24.973741+0000","last_active":"2024-04-13T03:09:31.004364+0000","last_peered":"2024-04-13T03:09:31.004364+0000","last_clean":"2024-04-13T03:09:31.004364+0000","last_became_active":"2024-04-13T03:09:24.973468+0000","last_became_peered":"2024-04-13T03:09:24.973468+0000","last_unstale":"2024-04-13T03:09:31.004364+0000","last_undegraded":"2024-04-13T03:09:31.004364+0000","last_fullsized":"2024-04-13T03:09:31.004364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:20.516278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016730+0000","last_change":"2024-04-13T03:09:24.982585+0000","last_active":"2024-04-13T03:09:31.016730+0000","last_peered":"2024-04-13T03:09:31.016730+0000","last_clean":"2024-04-13T03:09:31.016730+0000","last_became_active":"2024-04-13T03:09:24.982301+0000","last_became_peered":"2024-04-13T03:09:24.982301+0000","last_unstale":"2024-04-13T03:09:31.016730+0000","last_undegraded":"2024-04-13T03:09:31.016730+0000","last_fullsized":"2024-04-13T03:09:31.016730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005384+0000","last_change":"2024-04-13T03:09:24.987283+0000","last_active":"2024-04-13T03:09:31.005384+0000","last_peered":"2024-04-13T03:09:31.005384+0000","last_clean":"2024-04-13T03:09:31.005384+0000","last_became_active":"2024-04-13T03:09:24.983517+0000","last_became_peered":"2024-04-13T03:09:24.983517+0000","last_unstale":"2024-04-13T03:09:31.005384+0000","last_undegraded":"2024-04-13T03:09:31.005384+0000","last_fullsized":"2024-04-13T03:09:31.005384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:43.988718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074632+0000","last_change":"2024-04-13T03:09:24.994512+0000","last_active":"2024-04-13T03:09:32.074632+0000","last_peered":"2024-04-13T03:09:32.074632+0000","last_clean":"2024-04-13T03:09:32.074632+0000","last_became_active":"2024-04-13T03:09:24.994212+0000","last_became_peered":"2024-04-13T03:09:24.994212+0000","last_unstale":"2024-04-13T03:09:32.074632+0000","last_undegraded":"2024-04-13T03:09:32.074632+0000","last_fullsized":"2024-04-13T03:09:32.074632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074805+0000","last_change":"2024-04-13T03:09:24.990511+0000","last_active":"2024-04-13T03:09:32.074805+0000","last_peered":"2024-04-13T03:09:32.074805+0000","last_clean":"2024-04-13T03:09:32.074805+0000","last_became_active":"2024-04-13T03:09:24.990230+0000","last_became_peered":"2024-04-13T03:09:24.990230+0000","last_unstale":"2024-04-13T03:09:32.074805+0000","last_undegraded":"2024-04-13T03:09:32.074805+0000","last_fullsized":"2024-04-13T03:09:32.074805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074337+0000","last_change":"2024-04-13T03:09:24.991357+0000","last_active":"2024-04-13T03:09:32.074337+0000","last_peered":"2024-04-13T03:09:32.074337+0000","last_clean":"2024-04-13T03:09:32.074337+0000","last_became_active":"2024-04-13T03:09:24.990754+0000","last_became_peered":"2024-04-13T03:09:24.990754+0000","last_unstale":"2024-04-13T03:09:32.074337+0000","last_undegraded":"2024-04-13T03:09:32.074337+0000","last_fullsized":"2024-04-13T03:09:32.074337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016489+0000","last_change":"2024-04-13T03:09:24.984967+0000","last_active":"2024-04-13T03:09:31.016489+0000","last_peered":"2024-04-13T03:09:31.016489+0000","last_clean":"2024-04-13T03:09:31.016489+0000","last_became_active":"2024-04-13T03:09:24.984674+0000","last_became_peered":"2024-04-13T03:09:24.984674+0000","last_unstale":"2024-04-13T03:09:31.016489+0000","last_undegraded":"2024-04-13T03:09:31.016489+0000","last_fullsized":"2024-04-13T03:09:31.016489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073906+0000","last_change":"2024-04-13T03:09:24.976323+0000","last_active":"2024-04-13T03:09:32.073906+0000","last_peered":"2024-04-13T03:09:32.073906+0000","last_clean":"2024-04-13T03:09:32.073906+0000","last_became_active":"2024-04-13T03:09:24.976014+0000","last_became_peered":"2024-04-13T03:09:24.976014+0000","last_unstale":"2024-04-13T03:09:32.073906+0000","last_undegraded":"2024-04-13T03:09:32.073906+0000","last_fullsized":"2024-04-13T03:09:32.073906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005924+0000","last_change":"2024-04-13T03:09:24.987113+0000","last_active":"2024-04-13T03:09:31.005924+0000","last_peered":"2024-04-13T03:09:31.005924+0000","last_clean":"2024-04-13T03:09:31.005924+0000","last_became_active":"2024-04-13T03:09:24.986836+0000","last_became_peered":"2024-04-13T03:09:24.986836+0000","last_unstale":"2024-04-13T03:09:31.005924+0000","last_undegraded":"2024-04-13T03:09:31.005924+0000","last_fullsized":"2024-04-13T03:09:31.005924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073397+0000","last_change":"2024-04-13T03:09:24.979549+0000","last_active":"2024-04-13T03:09:32.073397+0000","last_peered":"2024-04-13T03:09:32.073397+0000","last_clean":"2024-04-13T03:09:32.073397+0000","last_became_active":"2024-04-13T03:09:24.979203+0000","last_became_peered":"2024-04-13T03:09:24.979203+0000","last_unstale":"2024-04-13T03:09:32.073397+0000","last_undegraded":"2024-04-13T03:09:32.073397+0000","last_fullsized":"2024-04-13T03:09:32.073397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006135+0000","last_change":"2024-04-13T03:09:24.986349+0000","last_active":"2024-04-13T03:09:31.006135+0000","last_peered":"2024-04-13T03:09:31.006135+0000","last_clean":"2024-04-13T03:09:31.006135+0000","last_became_active":"2024-04-13T03:09:24.985694+0000","last_became_peered":"2024-04-13T03:09:24.985694+0000","last_unstale":"2024-04-13T03:09:31.006135+0000","last_undegraded":"2024-04-13T03:09:31.006135+0000","last_fullsized":"2024-04-13T03:09:31.006135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004564+0000","last_change":"2024-04-13T03:09:24.973883+0000","last_active":"2024-04-13T03:09:31.004564+0000","last_peered":"2024-04-13T03:09:31.004564+0000","last_clean":"2024-04-13T03:09:31.004564+0000","last_became_active":"2024-04-13T03:09:24.973623+0000","last_became_peered":"2024-04-13T03:09:24.973623+0000","last_unstale":"2024-04-13T03:09:31.004564+0000","last_undegraded":"2024-04-13T03:09:31.004564+0000","last_fullsized":"2024-04-13T03:09:31.004564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552728+0000","last_change":"2024-04-13T03:09:24.980289+0000","last_active":"2024-04-13T03:09:31.552728+0000","last_peered":"2024-04-13T03:09:31.552728+0000","last_clean":"2024-04-13T03:09:31.552728+0000","last_became_active":"2024-04-13T03:09:24.980040+0000","last_became_peered":"2024-04-13T03:09:24.980040+0000","last_unstale":"2024-04-13T03:09:31.552728+0000","last_undegraded":"2024-04-13T03:09:31.552728+0000","last_fullsized":"2024-04-13T03:09:31.552728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074027+0000","last_change":"2024-04-13T03:09:24.985108+0000","last_active":"2024-04-13T03:09:32.074027+0000","last_peered":"2024-04-13T03:09:32.074027+0000","last_clean":"2024-04-13T03:09:32.074027+0000","last_became_active":"2024-04-13T03:09:24.984576+0000","last_became_peered":"2024-04-13T03:09:24.984576+0000","last_unstale":"2024-04-13T03:09:32.074027+0000","last_undegraded":"2024-04-13T03:09:32.074027+0000","last_fullsized":"2024-04-13T03:09:32.074027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:09:48.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:09:48.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.154+0000 7fc965f55640 1 Processor -- start 2024-04-13T03:09:48.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 -- start start 2024-04-13T03:09:48.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 0x7fc960160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc96006e480 0x7fc96015d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96015d690 con 0x7fc96015cc10 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc96015d970 con 0x7fc96006e480 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc965f55640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc960160410 con 0x7fc960156b50 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc95ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc96006e480 0x7fc96015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc95ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc96006e480 0x7fc96015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44934/0 (socket says 172.21.15.106:44934) 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.155+0000 7fc95ffff640 1 -- 172.21.15.106:0/1849544242 learned_addr learned my addr 172.21.15.106:0/1849544242 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95f7fe640 1 --2- 172.21.15.106:0/1849544242 >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 0x7fc960160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95ffff640 1 -- 172.21.15.106:0/1849544242 >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 msgr2=0x7fc960160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95ffff640 1 --2- 172.21.15.106:0/1849544242 >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 0x7fc960160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95ffff640 1 -- 172.21.15.106:0/1849544242 >> v1:172.21.15.114:6789/0 conn(0x7fc96015cc10 legacy=0x7fc96006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95ffff640 1 -- 172.21.15.106:0/1849544242 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601606f0 con 0x7fc96006e480 2024-04-13T03:09:48.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95f7fe640 1 --2- 172.21.15.106:0/1849544242 >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 0x7fc960160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:48.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95ffff640 1 --2- 172.21.15.106:0/1849544242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc96015d080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc96009e4d0 tx=0x7fc95802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3348807aed03c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.156+0000 7fc95e7fc640 1 -- 172.21.15.106:0/1849544242 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95802ca80 con 0x7fc96006e480 2024-04-13T03:09:48.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc95e7fc640 1 -- 172.21.15.106:0/1849544242 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95802cc20 con 0x7fc96006e480 2024-04-13T03:09:48.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc95e7fc640 1 -- 172.21.15.106:0/1849544242 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580343f0 con 0x7fc96006e480 2024-04-13T03:09:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 -- 172.21.15.106:0/1849544242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 msgr2=0x7fc96015d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 --2- 172.21.15.106:0/1849544242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc96015d080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc96009e4d0 tx=0x7fc95802bee0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 -- 172.21.15.106:0/1849544242 shutdown_connections 2024-04-13T03:09:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 --2- 172.21.15.106:0/1849544242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc96015d080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 --2- 172.21.15.106:0/1849544242 >> v2:172.21.15.114:3300/0 conn(0x7fc960156b50 0x7fc960160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 -- 172.21.15.106:0/1849544242 >> 172.21.15.106:0/1849544242 conn(0x7fc9600b1f10 msgr2=0x7fc9600b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 -- 172.21.15.106:0/1849544242 shutdown_connections 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.157+0000 7fc965f55640 1 -- 172.21.15.106:0/1849544242 wait complete. 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.158+0000 7fc965f55640 1 Processor -- start 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 -- start start 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc960156b50 0x7fc9600cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 0x7fc9600ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600afc20 con 0x7fc96006e480 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600afaa0 con 0x7fc96015cc10 2024-04-13T03:09:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc965f55640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc96006bcb0 con 0x7fc960156b50 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44938/0 (socket says 172.21.15.106:44938) 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc95f7fe640 1 -- 172.21.15.106:0/1242791657 learned_addr learned my addr 172.21.15.106:0/1242791657 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc95effd640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc960156b50 0x7fc9600cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.159+0000 7fc95ffff640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 0x7fc9600ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 -- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc960156b50 msgr2=0x7fc9600cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc960156b50 0x7fc9600cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 -- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 msgr2=0x7fc9600ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 0x7fc9600ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 -- 172.21.15.106:0/1242791657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601564a0 con 0x7fc96006e480 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95ffff640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 0x7fc9600ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc95f7fe640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c7f0 tx=0x7fc95000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950015070 con 0x7fc96006e480 2024-04-13T03:09:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc950007a30 con 0x7fc96006e480 2024-04-13T03:09:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9600ce9d0 con 0x7fc96006e480 2024-04-13T03:09:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950007e00 con 0x7fc96006e480 2024-04-13T03:09:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.160+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9600cef90 con 0x7fc96006e480 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.161+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc960068a10 con 0x7fc96006e480 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.161+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc950018070 con 0x7fc96006e480 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.162+0000 7fc947fff640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 0x7fc94c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.162+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc95008bd70 con 0x7fc96006e480 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.162+0000 7fc95effd640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 0x7fc94c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.162+0000 7fc95effd640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 0x7fc94c06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc9601f7c10 tx=0x7fc954009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.166+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc950059ef0 con 0x7fc96006e480 2024-04-13T03:09:48.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.337+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc96006faa0 con 0x7fc94c06c8f0 2024-04-13T03:09:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.344+0000 7fc947fff640 1 -- 172.21.15.106:0/1242791657 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fc9540053f0 con 0x7fc94c06c8f0 2024-04-13T03:09:48.349 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 msgr2=0x7fc94c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 0x7fc94c06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc9601f7c10 tx=0x7fc954009040 comp rx=0 tx=0).stop 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 msgr2=0x7fc960155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c7f0 tx=0x7fc95000cbf0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 shutdown_connections 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc94c06c8f0 0x7fc94c06edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc960156b50 0x7fc9600cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc96015cc10 0x7fc9600ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 --2- 172.21.15.106:0/1242791657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc96006e480 0x7fc960155f60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.347+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 >> 172.21.15.106:0/1242791657 conn(0x7fc9600b1f10 msgr2=0x7fc96006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.348+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 shutdown_connections 2024-04-13T03:09:48.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.348+0000 7fc965f55640 1 -- 172.21.15.106:0/1242791657 wait complete. 2024-04-13T03:09:48.359 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-13T03:09:46.367067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004868+0000","last_change":"2024-04-13T03:09:23.977064+0000","last_active":"2024-04-13T03:09:31.004868+0000","last_peered":"2024-04-13T03:09:31.004868+0000","last_clean":"2024-04-13T03:09:31.004868+0000","last_became_active":"2024-04-13T03:09:23.976781+0000","last_became_peered":"2024-04-13T03:09:23.976781+0000","last_unstale":"2024-04-13T03:09:31.004868+0000","last_undegraded":"2024-04-13T03:09:31.004868+0000","last_fullsized":"2024-04-13T03:09:31.004868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:14:27.702105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017061+0000","last_change":"2024-04-13T03:09:24.983007+0000","last_active":"2024-04-13T03:09:31.017061+0000","last_peered":"2024-04-13T03:09:31.017061+0000","last_clean":"2024-04-13T03:09:31.017061+0000","last_became_active":"2024-04-13T03:09:24.982432+0000","last_became_peered":"2024-04-13T03:09:24.982432+0000","last_unstale":"2024-04-13T03:09:31.017061+0000","last_undegraded":"2024-04-13T03:09:31.017061+0000","last_fullsized":"2024-04-13T03:09:31.017061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016588+0000","last_change":"2024-04-13T03:09:29.567663+0000","last_active":"2024-04-13T03:09:31.016588+0000","last_peered":"2024-04-13T03:09:31.016588+0000","last_clean":"2024-04-13T03:09:31.016588+0000","last_became_active":"2024-04-13T03:09:29.567314+0000","last_became_peered":"2024-04-13T03:09:29.567314+0000","last_unstale":"2024-04-13T03:09:31.016588+0000","last_undegraded":"2024-04-13T03:09:31.016588+0000","last_fullsized":"2024-04-13T03:09:31.016588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:55:26.111999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017190+0000","last_change":"2024-04-13T03:09:23.977858+0000","last_active":"2024-04-13T03:09:31.017190+0000","last_peered":"2024-04-13T03:09:31.017190+0000","last_clean":"2024-04-13T03:09:31.017190+0000","last_became_active":"2024-04-13T03:09:23.977595+0000","last_became_peered":"2024-04-13T03:09:23.977595+0000","last_unstale":"2024-04-13T03:09:31.017190+0000","last_undegraded":"2024-04-13T03:09:31.017190+0000","last_fullsized":"2024-04-13T03:09:31.017190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004745+0000","last_change":"2024-04-13T03:09:24.981595+0000","last_active":"2024-04-13T03:09:31.004745+0000","last_peered":"2024-04-13T03:09:31.004745+0000","last_clean":"2024-04-13T03:09:31.004745+0000","last_became_active":"2024-04-13T03:09:24.981249+0000","last_became_peered":"2024-04-13T03:09:24.981249+0000","last_unstale":"2024-04-13T03:09:31.004745+0000","last_undegraded":"2024-04-13T03:09:31.004745+0000","last_fullsized":"2024-04-13T03:09:31.004745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004491+0000","last_change":"2024-04-13T03:09:29.008503+0000","last_active":"2024-04-13T03:09:31.004491+0000","last_peered":"2024-04-13T03:09:31.004491+0000","last_clean":"2024-04-13T03:09:31.004491+0000","last_became_active":"2024-04-13T03:09:29.007985+0000","last_became_peered":"2024-04-13T03:09:29.007985+0000","last_unstale":"2024-04-13T03:09:31.004491+0000","last_undegraded":"2024-04-13T03:09:31.004491+0000","last_fullsized":"2024-04-13T03:09:31.004491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074031+0000","last_change":"2024-04-13T03:09:23.982379+0000","last_active":"2024-04-13T03:09:32.074031+0000","last_peered":"2024-04-13T03:09:32.074031+0000","last_clean":"2024-04-13T03:09:32.074031+0000","last_became_active":"2024-04-13T03:09:23.982090+0000","last_became_peered":"2024-04-13T03:09:23.982090+0000","last_unstale":"2024-04-13T03:09:32.074031+0000","last_undegraded":"2024-04-13T03:09:32.074031+0000","last_fullsized":"2024-04-13T03:09:32.074031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006164+0000","last_change":"2024-04-13T03:09:24.983652+0000","last_active":"2024-04-13T03:09:31.006164+0000","last_peered":"2024-04-13T03:09:31.006164+0000","last_clean":"2024-04-13T03:09:31.006164+0000","last_became_active":"2024-04-13T03:09:24.981620+0000","last_became_peered":"2024-04-13T03:09:24.981620+0000","last_unstale":"2024-04-13T03:09:31.006164+0000","last_undegraded":"2024-04-13T03:09:31.006164+0000","last_fullsized":"2024-04-13T03:09:31.006164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004691+0000","last_change":"2024-04-13T03:09:29.007274+0000","last_active":"2024-04-13T03:09:31.004691+0000","last_peered":"2024-04-13T03:09:31.004691+0000","last_clean":"2024-04-13T03:09:31.004691+0000","last_became_active":"2024-04-13T03:09:29.006867+0000","last_became_peered":"2024-04-13T03:09:29.006867+0000","last_unstale":"2024-04-13T03:09:31.004691+0000","last_undegraded":"2024-04-13T03:09:31.004691+0000","last_fullsized":"2024-04-13T03:09:31.004691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:32.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073387+0000","last_change":"2024-04-13T03:09:23.979669+0000","last_active":"2024-04-13T03:09:32.073387+0000","last_peered":"2024-04-13T03:09:32.073387+0000","last_clean":"2024-04-13T03:09:32.073387+0000","last_became_active":"2024-04-13T03:09:23.979303+0000","last_became_peered":"2024-04-13T03:09:23.979303+0000","last_unstale":"2024-04-13T03:09:32.073387+0000","last_undegraded":"2024-04-13T03:09:32.073387+0000","last_fullsized":"2024-04-13T03:09:32.073387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005531+0000","last_change":"2024-04-13T03:09:24.976796+0000","last_active":"2024-04-13T03:09:31.005531+0000","last_peered":"2024-04-13T03:09:31.005531+0000","last_clean":"2024-04-13T03:09:31.005531+0000","last_became_active":"2024-04-13T03:09:24.975367+0000","last_became_peered":"2024-04-13T03:09:24.975367+0000","last_unstale":"2024-04-13T03:09:31.005531+0000","last_undegraded":"2024-04-13T03:09:31.005531+0000","last_fullsized":"2024-04-13T03:09:31.005531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004563+0000","last_change":"2024-04-13T03:09:29.007374+0000","last_active":"2024-04-13T03:09:31.004563+0000","last_peered":"2024-04-13T03:09:31.004563+0000","last_clean":"2024-04-13T03:09:31.004563+0000","last_became_active":"2024-04-13T03:09:29.006740+0000","last_became_peered":"2024-04-13T03:09:29.006740+0000","last_unstale":"2024-04-13T03:09:31.004563+0000","last_undegraded":"2024-04-13T03:09:31.004563+0000","last_fullsized":"2024-04-13T03:09:31.004563+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073554+0000","last_change":"2024-04-13T03:09:24.988155+0000","last_active":"2024-04-13T03:09:32.073554+0000","last_peered":"2024-04-13T03:09:32.073554+0000","last_clean":"2024-04-13T03:09:32.073554+0000","last_became_active":"2024-04-13T03:09:24.987879+0000","last_became_peered":"2024-04-13T03:09:24.987879+0000","last_unstale":"2024-04-13T03:09:32.073554+0000","last_undegraded":"2024-04-13T03:09:32.073554+0000","last_fullsized":"2024-04-13T03:09:32.073554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072988+0000","last_change":"2024-04-13T03:09:23.977048+0000","last_active":"2024-04-13T03:09:32.072988+0000","last_peered":"2024-04-13T03:09:32.072988+0000","last_clean":"2024-04-13T03:09:32.072988+0000","last_became_active":"2024-04-13T03:09:23.976770+0000","last_became_peered":"2024-04-13T03:09:23.976770+0000","last_unstale":"2024-04-13T03:09:32.072988+0000","last_undegraded":"2024-04-13T03:09:32.072988+0000","last_fullsized":"2024-04-13T03:09:32.072988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005678+0000","last_change":"2024-04-13T03:09:29.009555+0000","last_active":"2024-04-13T03:09:31.005678+0000","last_peered":"2024-04-13T03:09:31.005678+0000","last_clean":"2024-04-13T03:09:31.005678+0000","last_became_active":"2024-04-13T03:09:29.009108+0000","last_became_peered":"2024-04-13T03:09:29.009108+0000","last_unstale":"2024-04-13T03:09:31.005678+0000","last_undegraded":"2024-04-13T03:09:31.005678+0000","last_fullsized":"2024-04-13T03:09:31.005678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016758+0000","last_change":"2024-04-13T03:09:24.973588+0000","last_active":"2024-04-13T03:09:31.016758+0000","last_peered":"2024-04-13T03:09:31.016758+0000","last_clean":"2024-04-13T03:09:31.016758+0000","last_became_active":"2024-04-13T03:09:24.973322+0000","last_became_peered":"2024-04-13T03:09:24.973322+0000","last_unstale":"2024-04-13T03:09:31.016758+0000","last_undegraded":"2024-04-13T03:09:31.016758+0000","last_fullsized":"2024-04-13T03:09:31.016758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552368+0000","last_change":"2024-04-13T03:09:23.971765+0000","last_active":"2024-04-13T03:09:31.552368+0000","last_peered":"2024-04-13T03:09:31.552368+0000","last_clean":"2024-04-13T03:09:31.552368+0000","last_became_active":"2024-04-13T03:09:23.971543+0000","last_became_peered":"2024-04-13T03:09:23.971543+0000","last_unstale":"2024-04-13T03:09:31.552368+0000","last_undegraded":"2024-04-13T03:09:31.552368+0000","last_fullsized":"2024-04-13T03:09:31.552368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005586+0000","last_change":"2024-04-13T03:09:29.008916+0000","last_active":"2024-04-13T03:09:31.005586+0000","last_peered":"2024-04-13T03:09:31.005586+0000","last_clean":"2024-04-13T03:09:31.005586+0000","last_became_active":"2024-04-13T03:09:29.008366+0000","last_became_peered":"2024-04-13T03:09:29.008366+0000","last_unstale":"2024-04-13T03:09:31.005586+0000","last_undegraded":"2024-04-13T03:09:31.005586+0000","last_fullsized":"2024-04-13T03:09:31.005586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016669+0000","last_change":"2024-04-13T03:09:24.975847+0000","last_active":"2024-04-13T03:09:31.016669+0000","last_peered":"2024-04-13T03:09:31.016669+0000","last_clean":"2024-04-13T03:09:31.016669+0000","last_became_active":"2024-04-13T03:09:24.975258+0000","last_became_peered":"2024-04-13T03:09:24.975258+0000","last_unstale":"2024-04-13T03:09:31.016669+0000","last_undegraded":"2024-04-13T03:09:31.016669+0000","last_fullsized":"2024-04-13T03:09:31.016669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552132+0000","last_change":"2024-04-13T03:09:23.977100+0000","last_active":"2024-04-13T03:09:31.552132+0000","last_peered":"2024-04-13T03:09:31.552132+0000","last_clean":"2024-04-13T03:09:31.552132+0000","last_became_active":"2024-04-13T03:09:23.976787+0000","last_became_peered":"2024-04-13T03:09:23.976787+0000","last_unstale":"2024-04-13T03:09:31.552132+0000","last_undegraded":"2024-04-13T03:09:31.552132+0000","last_fullsized":"2024-04-13T03:09:31.552132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552936+0000","last_change":"2024-04-13T03:09:29.567374+0000","last_active":"2024-04-13T03:09:31.552936+0000","last_peered":"2024-04-13T03:09:31.552936+0000","last_clean":"2024-04-13T03:09:31.552936+0000","last_became_active":"2024-04-13T03:09:29.566995+0000","last_became_peered":"2024-04-13T03:09:29.566995+0000","last_unstale":"2024-04-13T03:09:31.552936+0000","last_undegraded":"2024-04-13T03:09:31.552936+0000","last_fullsized":"2024-04-13T03:09:31.552936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004813+0000","last_change":"2024-04-13T03:09:24.972822+0000","last_active":"2024-04-13T03:09:31.004813+0000","last_peered":"2024-04-13T03:09:31.004813+0000","last_clean":"2024-04-13T03:09:31.004813+0000","last_became_active":"2024-04-13T03:09:24.972521+0000","last_became_peered":"2024-04-13T03:09:24.972521+0000","last_unstale":"2024-04-13T03:09:31.004813+0000","last_undegraded":"2024-04-13T03:09:31.004813+0000","last_fullsized":"2024-04-13T03:09:31.004813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073287+0000","last_change":"2024-04-13T03:09:23.979583+0000","last_active":"2024-04-13T03:09:32.073287+0000","last_peered":"2024-04-13T03:09:32.073287+0000","last_clean":"2024-04-13T03:09:32.073287+0000","last_became_active":"2024-04-13T03:09:23.979321+0000","last_became_peered":"2024-04-13T03:09:23.979321+0000","last_unstale":"2024-04-13T03:09:32.073287+0000","last_undegraded":"2024-04-13T03:09:32.073287+0000","last_fullsized":"2024-04-13T03:09:32.073287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:05.954316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006004+0000","last_change":"2024-04-13T03:09:29.009924+0000","last_active":"2024-04-13T03:09:31.006004+0000","last_peered":"2024-04-13T03:09:31.006004+0000","last_clean":"2024-04-13T03:09:31.006004+0000","last_became_active":"2024-04-13T03:09:29.009528+0000","last_became_peered":"2024-04-13T03:09:29.009528+0000","last_unstale":"2024-04-13T03:09:31.006004+0000","last_undegraded":"2024-04-13T03:09:31.006004+0000","last_fullsized":"2024-04-13T03:09:31.006004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004799+0000","last_change":"2024-04-13T03:09:24.973677+0000","last_active":"2024-04-13T03:09:31.004799+0000","last_peered":"2024-04-13T03:09:31.004799+0000","last_clean":"2024-04-13T03:09:31.004799+0000","last_became_active":"2024-04-13T03:09:24.973419+0000","last_became_peered":"2024-04-13T03:09:24.973419+0000","last_unstale":"2024-04-13T03:09:31.004799+0000","last_undegraded":"2024-04-13T03:09:31.004799+0000","last_fullsized":"2024-04-13T03:09:31.004799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074813+0000","last_change":"2024-04-13T03:09:23.977870+0000","last_active":"2024-04-13T03:09:32.074813+0000","last_peered":"2024-04-13T03:09:32.074813+0000","last_clean":"2024-04-13T03:09:32.074813+0000","last_became_active":"2024-04-13T03:09:23.977529+0000","last_became_peered":"2024-04-13T03:09:23.977529+0000","last_unstale":"2024-04-13T03:09:32.074813+0000","last_undegraded":"2024-04-13T03:09:32.074813+0000","last_fullsized":"2024-04-13T03:09:32.074813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004609+0000","last_change":"2024-04-13T03:09:29.566777+0000","last_active":"2024-04-13T03:09:31.004609+0000","last_peered":"2024-04-13T03:09:31.004609+0000","last_clean":"2024-04-13T03:09:31.004609+0000","last_became_active":"2024-04-13T03:09:29.566173+0000","last_became_peered":"2024-04-13T03:09:29.566173+0000","last_unstale":"2024-04-13T03:09:31.004609+0000","last_undegraded":"2024-04-13T03:09:31.004609+0000","last_fullsized":"2024-04-13T03:09:31.004609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017303+0000","last_change":"2024-04-13T03:09:24.981346+0000","last_active":"2024-04-13T03:09:31.017303+0000","last_peered":"2024-04-13T03:09:31.017303+0000","last_clean":"2024-04-13T03:09:31.017303+0000","last_became_active":"2024-04-13T03:09:24.981003+0000","last_became_peered":"2024-04-13T03:09:24.981003+0000","last_unstale":"2024-04-13T03:09:31.017303+0000","last_undegraded":"2024-04-13T03:09:31.017303+0000","last_fullsized":"2024-04-13T03:09:31.017303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552769+0000","last_change":"2024-04-13T03:09:23.973982+0000","last_active":"2024-04-13T03:09:31.552769+0000","last_peered":"2024-04-13T03:09:31.552769+0000","last_clean":"2024-04-13T03:09:31.552769+0000","last_became_active":"2024-04-13T03:09:23.973759+0000","last_became_peered":"2024-04-13T03:09:23.973759+0000","last_unstale":"2024-04-13T03:09:31.552769+0000","last_undegraded":"2024-04-13T03:09:31.552769+0000","last_fullsized":"2024-04-13T03:09:31.552769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016499+0000","last_change":"2024-04-13T03:09:29.567476+0000","last_active":"2024-04-13T03:09:31.016499+0000","last_peered":"2024-04-13T03:09:31.016499+0000","last_clean":"2024-04-13T03:09:31.016499+0000","last_became_active":"2024-04-13T03:09:29.567116+0000","last_became_peered":"2024-04-13T03:09:29.567116+0000","last_unstale":"2024-04-13T03:09:31.016499+0000","last_undegraded":"2024-04-13T03:09:31.016499+0000","last_fullsized":"2024-04-13T03:09:31.016499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:07.048787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552621+0000","last_change":"2024-04-13T03:09:24.974586+0000","last_active":"2024-04-13T03:09:31.552621+0000","last_peered":"2024-04-13T03:09:31.552621+0000","last_clean":"2024-04-13T03:09:31.552621+0000","last_became_active":"2024-04-13T03:09:24.974348+0000","last_became_peered":"2024-04-13T03:09:24.974348+0000","last_unstale":"2024-04-13T03:09:31.552621+0000","last_undegraded":"2024-04-13T03:09:31.552621+0000","last_fullsized":"2024-04-13T03:09:31.552621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073219+0000","last_change":"2024-04-13T03:09:23.979817+0000","last_active":"2024-04-13T03:09:32.073219+0000","last_peered":"2024-04-13T03:09:32.073219+0000","last_clean":"2024-04-13T03:09:32.073219+0000","last_became_active":"2024-04-13T03:09:23.979575+0000","last_became_peered":"2024-04-13T03:09:23.979575+0000","last_unstale":"2024-04-13T03:09:32.073219+0000","last_undegraded":"2024-04-13T03:09:32.073219+0000","last_fullsized":"2024-04-13T03:09:32.073219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:42.984781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004425+0000","last_change":"2024-04-13T03:09:29.567336+0000","last_active":"2024-04-13T03:09:31.004425+0000","last_peered":"2024-04-13T03:09:31.004425+0000","last_clean":"2024-04-13T03:09:31.004425+0000","last_became_active":"2024-04-13T03:09:29.566964+0000","last_became_peered":"2024-04-13T03:09:29.566964+0000","last_unstale":"2024-04-13T03:09:31.004425+0000","last_undegraded":"2024-04-13T03:09:31.004425+0000","last_fullsized":"2024-04-13T03:09:31.004425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005618+0000","last_change":"2024-04-13T03:09:24.976546+0000","last_active":"2024-04-13T03:09:31.005618+0000","last_peered":"2024-04-13T03:09:31.005618+0000","last_clean":"2024-04-13T03:09:31.005618+0000","last_became_active":"2024-04-13T03:09:24.974772+0000","last_became_peered":"2024-04-13T03:09:24.974772+0000","last_unstale":"2024-04-13T03:09:31.005618+0000","last_undegraded":"2024-04-13T03:09:31.005618+0000","last_fullsized":"2024-04-13T03:09:31.005618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552722+0000","last_change":"2024-04-13T03:09:23.966690+0000","last_active":"2024-04-13T03:09:31.552722+0000","last_peered":"2024-04-13T03:09:31.552722+0000","last_clean":"2024-04-13T03:09:31.552722+0000","last_became_active":"2024-04-13T03:09:23.966440+0000","last_became_peered":"2024-04-13T03:09:23.966440+0000","last_unstale":"2024-04-13T03:09:31.552722+0000","last_undegraded":"2024-04-13T03:09:31.552722+0000","last_fullsized":"2024-04-13T03:09:31.552722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552227+0000","last_change":"2024-04-13T03:09:29.013113+0000","last_active":"2024-04-13T03:09:31.552227+0000","last_peered":"2024-04-13T03:09:31.552227+0000","last_clean":"2024-04-13T03:09:31.552227+0000","last_became_active":"2024-04-13T03:09:29.012470+0000","last_became_peered":"2024-04-13T03:09:29.012470+0000","last_unstale":"2024-04-13T03:09:31.552227+0000","last_undegraded":"2024-04-13T03:09:31.552227+0000","last_fullsized":"2024-04-13T03:09:31.552227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005144+0000","last_change":"2024-04-13T03:09:24.973635+0000","last_active":"2024-04-13T03:09:31.005144+0000","last_peered":"2024-04-13T03:09:31.005144+0000","last_clean":"2024-04-13T03:09:31.005144+0000","last_became_active":"2024-04-13T03:09:24.973362+0000","last_became_peered":"2024-04-13T03:09:24.973362+0000","last_unstale":"2024-04-13T03:09:31.005144+0000","last_undegraded":"2024-04-13T03:09:31.005144+0000","last_fullsized":"2024-04-13T03:09:31.005144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:24.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006449+0000","last_change":"2024-04-13T03:09:23.968272+0000","last_active":"2024-04-13T03:09:31.006449+0000","last_peered":"2024-04-13T03:09:31.006449+0000","last_clean":"2024-04-13T03:09:31.006449+0000","last_became_active":"2024-04-13T03:09:23.967784+0000","last_became_peered":"2024-04-13T03:09:23.967784+0000","last_unstale":"2024-04-13T03:09:31.006449+0000","last_undegraded":"2024-04-13T03:09:31.006449+0000","last_fullsized":"2024-04-13T03:09:31.006449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552810+0000","last_change":"2024-04-13T03:09:29.008833+0000","last_active":"2024-04-13T03:09:31.552810+0000","last_peered":"2024-04-13T03:09:31.552810+0000","last_clean":"2024-04-13T03:09:31.552810+0000","last_became_active":"2024-04-13T03:09:29.008342+0000","last_became_peered":"2024-04-13T03:09:29.008342+0000","last_unstale":"2024-04-13T03:09:31.552810+0000","last_undegraded":"2024-04-13T03:09:31.552810+0000","last_fullsized":"2024-04-13T03:09:31.552810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552174+0000","last_change":"2024-04-13T03:09:24.975965+0000","last_active":"2024-04-13T03:09:31.552174+0000","last_peered":"2024-04-13T03:09:31.552174+0000","last_clean":"2024-04-13T03:09:31.552174+0000","last_became_active":"2024-04-13T03:09:24.975143+0000","last_became_peered":"2024-04-13T03:09:24.975143+0000","last_unstale":"2024-04-13T03:09:31.552174+0000","last_undegraded":"2024-04-13T03:09:31.552174+0000","last_fullsized":"2024-04-13T03:09:31.552174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.978132+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.976718+0000","last_became_peered":"2024-04-13T03:09:23.976718+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552953+0000","last_change":"2024-04-13T03:09:29.012295+0000","last_active":"2024-04-13T03:09:31.552953+0000","last_peered":"2024-04-13T03:09:31.552953+0000","last_clean":"2024-04-13T03:09:31.552953+0000","last_became_active":"2024-04-13T03:09:29.011908+0000","last_became_peered":"2024-04-13T03:09:29.011908+0000","last_unstale":"2024-04-13T03:09:31.552953+0000","last_undegraded":"2024-04-13T03:09:31.552953+0000","last_fullsized":"2024-04-13T03:09:31.552953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004144+0000","last_change":"2024-04-13T03:09:24.974975+0000","last_active":"2024-04-13T03:09:31.004144+0000","last_peered":"2024-04-13T03:09:31.004144+0000","last_clean":"2024-04-13T03:09:31.004144+0000","last_became_active":"2024-04-13T03:09:24.974656+0000","last_became_peered":"2024-04-13T03:09:24.974656+0000","last_unstale":"2024-04-13T03:09:31.004144+0000","last_undegraded":"2024-04-13T03:09:31.004144+0000","last_fullsized":"2024-04-13T03:09:31.004144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072979+0000","last_change":"2024-04-13T03:09:23.978671+0000","last_active":"2024-04-13T03:09:32.072979+0000","last_peered":"2024-04-13T03:09:32.072979+0000","last_clean":"2024-04-13T03:09:32.072979+0000","last_became_active":"2024-04-13T03:09:23.978358+0000","last_became_peered":"2024-04-13T03:09:23.978358+0000","last_unstale":"2024-04-13T03:09:32.072979+0000","last_undegraded":"2024-04-13T03:09:32.072979+0000","last_fullsized":"2024-04-13T03:09:32.072979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004701+0000","last_change":"2024-04-13T03:09:29.566364+0000","last_active":"2024-04-13T03:09:31.004701+0000","last_peered":"2024-04-13T03:09:31.004701+0000","last_clean":"2024-04-13T03:09:31.004701+0000","last_became_active":"2024-04-13T03:09:29.565976+0000","last_became_peered":"2024-04-13T03:09:29.565976+0000","last_unstale":"2024-04-13T03:09:31.004701+0000","last_undegraded":"2024-04-13T03:09:31.004701+0000","last_fullsized":"2024-04-13T03:09:31.004701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016580+0000","last_change":"2024-04-13T03:09:24.988431+0000","last_active":"2024-04-13T03:09:31.016580+0000","last_peered":"2024-04-13T03:09:31.016580+0000","last_clean":"2024-04-13T03:09:31.016580+0000","last_became_active":"2024-04-13T03:09:24.988179+0000","last_became_peered":"2024-04-13T03:09:24.988179+0000","last_unstale":"2024-04-13T03:09:31.016580+0000","last_undegraded":"2024-04-13T03:09:31.016580+0000","last_fullsized":"2024-04-13T03:09:31.016580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005560+0000","last_change":"2024-04-13T03:09:23.970960+0000","last_active":"2024-04-13T03:09:31.005560+0000","last_peered":"2024-04-13T03:09:31.005560+0000","last_clean":"2024-04-13T03:09:31.005560+0000","last_became_active":"2024-04-13T03:09:23.970615+0000","last_became_peered":"2024-04-13T03:09:23.970615+0000","last_unstale":"2024-04-13T03:09:31.005560+0000","last_undegraded":"2024-04-13T03:09:31.005560+0000","last_fullsized":"2024-04-13T03:09:31.005560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073625+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:09:32.073625+0000","last_peered":"2024-04-13T03:09:32.073625+0000","last_clean":"2024-04-13T03:09:32.073625+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:09:32.073625+0000","last_undegraded":"2024-04-13T03:09:32.073625+0000","last_fullsized":"2024-04-13T03:09:32.073625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074664+0000","last_change":"2024-04-13T03:09:29.570505+0000","last_active":"2024-04-13T03:09:32.074664+0000","last_peered":"2024-04-13T03:09:32.074664+0000","last_clean":"2024-04-13T03:09:32.074664+0000","last_became_active":"2024-04-13T03:09:29.569637+0000","last_became_peered":"2024-04-13T03:09:29.569637+0000","last_unstale":"2024-04-13T03:09:32.074664+0000","last_undegraded":"2024-04-13T03:09:32.074664+0000","last_fullsized":"2024-04-13T03:09:32.074664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073883+0000","last_change":"2024-04-13T03:09:24.990807+0000","last_active":"2024-04-13T03:09:32.073883+0000","last_peered":"2024-04-13T03:09:32.073883+0000","last_clean":"2024-04-13T03:09:32.073883+0000","last_became_active":"2024-04-13T03:09:24.990454+0000","last_became_peered":"2024-04-13T03:09:24.990454+0000","last_unstale":"2024-04-13T03:09:32.073883+0000","last_undegraded":"2024-04-13T03:09:32.073883+0000","last_fullsized":"2024-04-13T03:09:32.073883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072543+0000","last_change":"2024-04-13T03:09:23.974849+0000","last_active":"2024-04-13T03:09:32.072543+0000","last_peered":"2024-04-13T03:09:32.072543+0000","last_clean":"2024-04-13T03:09:32.072543+0000","last_became_active":"2024-04-13T03:09:23.974564+0000","last_became_peered":"2024-04-13T03:09:23.974564+0000","last_unstale":"2024-04-13T03:09:32.072543+0000","last_undegraded":"2024-04-13T03:09:32.072543+0000","last_fullsized":"2024-04-13T03:09:32.072543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073496+0000","last_change":"2024-04-13T03:09:29.567076+0000","last_active":"2024-04-13T03:09:32.073496+0000","last_peered":"2024-04-13T03:09:32.073496+0000","last_clean":"2024-04-13T03:09:32.073496+0000","last_became_active":"2024-04-13T03:09:29.566162+0000","last_became_peered":"2024-04-13T03:09:29.566162+0000","last_unstale":"2024-04-13T03:09:32.073496+0000","last_undegraded":"2024-04-13T03:09:32.073496+0000","last_fullsized":"2024-04-13T03:09:32.073496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006229+0000","last_change":"2024-04-13T03:09:24.986470+0000","last_active":"2024-04-13T03:09:31.006229+0000","last_peered":"2024-04-13T03:09:31.006229+0000","last_clean":"2024-04-13T03:09:31.006229+0000","last_became_active":"2024-04-13T03:09:24.986035+0000","last_became_peered":"2024-04-13T03:09:24.986035+0000","last_unstale":"2024-04-13T03:09:31.006229+0000","last_undegraded":"2024-04-13T03:09:31.006229+0000","last_fullsized":"2024-04-13T03:09:31.006229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552711+0000","last_change":"2024-04-13T03:09:23.966911+0000","last_active":"2024-04-13T03:09:31.552711+0000","last_peered":"2024-04-13T03:09:31.552711+0000","last_clean":"2024-04-13T03:09:31.552711+0000","last_became_active":"2024-04-13T03:09:23.966494+0000","last_became_peered":"2024-04-13T03:09:23.966494+0000","last_unstale":"2024-04-13T03:09:31.552711+0000","last_undegraded":"2024-04-13T03:09:31.552711+0000","last_fullsized":"2024-04-13T03:09:31.552711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004556+0000","last_change":"2024-04-13T03:09:29.008472+0000","last_active":"2024-04-13T03:09:31.004556+0000","last_peered":"2024-04-13T03:09:31.004556+0000","last_clean":"2024-04-13T03:09:31.004556+0000","last_became_active":"2024-04-13T03:09:29.008101+0000","last_became_peered":"2024-04-13T03:09:29.008101+0000","last_unstale":"2024-04-13T03:09:31.004556+0000","last_undegraded":"2024-04-13T03:09:31.004556+0000","last_fullsized":"2024-04-13T03:09:31.004556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074447+0000","last_change":"2024-04-13T03:09:24.978817+0000","last_active":"2024-04-13T03:09:32.074447+0000","last_peered":"2024-04-13T03:09:32.074447+0000","last_clean":"2024-04-13T03:09:32.074447+0000","last_became_active":"2024-04-13T03:09:24.978091+0000","last_became_peered":"2024-04-13T03:09:24.978091+0000","last_unstale":"2024-04-13T03:09:32.074447+0000","last_undegraded":"2024-04-13T03:09:32.074447+0000","last_fullsized":"2024-04-13T03:09:32.074447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:28.224846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072548+0000","last_change":"2024-04-13T03:09:23.978022+0000","last_active":"2024-04-13T03:09:32.072548+0000","last_peered":"2024-04-13T03:09:32.072548+0000","last_clean":"2024-04-13T03:09:32.072548+0000","last_became_active":"2024-04-13T03:09:23.976602+0000","last_became_peered":"2024-04-13T03:09:23.976602+0000","last_unstale":"2024-04-13T03:09:32.072548+0000","last_undegraded":"2024-04-13T03:09:32.072548+0000","last_fullsized":"2024-04-13T03:09:32.072548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005100+0000","last_change":"2024-04-13T03:09:29.567904+0000","last_active":"2024-04-13T03:09:31.005100+0000","last_peered":"2024-04-13T03:09:31.005100+0000","last_clean":"2024-04-13T03:09:31.005100+0000","last_became_active":"2024-04-13T03:09:29.567497+0000","last_became_peered":"2024-04-13T03:09:29.567497+0000","last_unstale":"2024-04-13T03:09:31.005100+0000","last_undegraded":"2024-04-13T03:09:31.005100+0000","last_fullsized":"2024-04-13T03:09:31.005100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074412+0000","last_change":"2024-04-13T03:09:24.978521+0000","last_active":"2024-04-13T03:09:32.074412+0000","last_peered":"2024-04-13T03:09:32.074412+0000","last_clean":"2024-04-13T03:09:32.074412+0000","last_became_active":"2024-04-13T03:09:24.978163+0000","last_became_peered":"2024-04-13T03:09:24.978163+0000","last_unstale":"2024-04-13T03:09:32.074412+0000","last_undegraded":"2024-04-13T03:09:32.074412+0000","last_fullsized":"2024-04-13T03:09:32.074412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006024+0000","last_change":"2024-04-13T03:09:23.966891+0000","last_active":"2024-04-13T03:09:31.006024+0000","last_peered":"2024-04-13T03:09:31.006024+0000","last_clean":"2024-04-13T03:09:31.006024+0000","last_became_active":"2024-04-13T03:09:23.966596+0000","last_became_peered":"2024-04-13T03:09:23.966596+0000","last_unstale":"2024-04-13T03:09:31.006024+0000","last_undegraded":"2024-04-13T03:09:31.006024+0000","last_fullsized":"2024-04-13T03:09:31.006024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:29.744523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004384+0000","last_change":"2024-04-13T03:09:29.007029+0000","last_active":"2024-04-13T03:09:31.004384+0000","last_peered":"2024-04-13T03:09:31.004384+0000","last_clean":"2024-04-13T03:09:31.004384+0000","last_became_active":"2024-04-13T03:09:29.006625+0000","last_became_peered":"2024-04-13T03:09:29.006625+0000","last_unstale":"2024-04-13T03:09:31.004384+0000","last_undegraded":"2024-04-13T03:09:31.004384+0000","last_fullsized":"2024-04-13T03:09:31.004384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074033+0000","last_change":"2024-04-13T03:09:24.978991+0000","last_active":"2024-04-13T03:09:32.074033+0000","last_peered":"2024-04-13T03:09:32.074033+0000","last_clean":"2024-04-13T03:09:32.074033+0000","last_became_active":"2024-04-13T03:09:24.978725+0000","last_became_peered":"2024-04-13T03:09:24.978725+0000","last_unstale":"2024-04-13T03:09:32.074033+0000","last_undegraded":"2024-04-13T03:09:32.074033+0000","last_fullsized":"2024-04-13T03:09:32.074033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.969378+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.969098+0000","last_became_peered":"2024-04-13T03:09:23.969098+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004694+0000","last_change":"2024-04-13T03:09:29.005610+0000","last_active":"2024-04-13T03:09:31.004694+0000","last_peered":"2024-04-13T03:09:31.004694+0000","last_clean":"2024-04-13T03:09:31.004694+0000","last_became_active":"2024-04-13T03:09:29.005192+0000","last_became_peered":"2024-04-13T03:09:29.005192+0000","last_unstale":"2024-04-13T03:09:31.004694+0000","last_undegraded":"2024-04-13T03:09:31.004694+0000","last_fullsized":"2024-04-13T03:09:31.004694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073718+0000","last_change":"2024-04-13T03:09:24.984278+0000","last_active":"2024-04-13T03:09:32.073718+0000","last_peered":"2024-04-13T03:09:32.073718+0000","last_clean":"2024-04-13T03:09:32.073718+0000","last_became_active":"2024-04-13T03:09:24.983967+0000","last_became_peered":"2024-04-13T03:09:24.983967+0000","last_unstale":"2024-04-13T03:09:32.073718+0000","last_undegraded":"2024-04-13T03:09:32.073718+0000","last_fullsized":"2024-04-13T03:09:32.073718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072991+0000","last_change":"2024-04-13T03:09:23.975431+0000","last_active":"2024-04-13T03:09:32.072991+0000","last_peered":"2024-04-13T03:09:32.072991+0000","last_clean":"2024-04-13T03:09:32.072991+0000","last_became_active":"2024-04-13T03:09:23.974255+0000","last_became_peered":"2024-04-13T03:09:23.974255+0000","last_unstale":"2024-04-13T03:09:32.072991+0000","last_undegraded":"2024-04-13T03:09:32.072991+0000","last_fullsized":"2024-04-13T03:09:32.072991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017273+0000","last_change":"2024-04-13T03:09:29.567710+0000","last_active":"2024-04-13T03:09:31.017273+0000","last_peered":"2024-04-13T03:09:31.017273+0000","last_clean":"2024-04-13T03:09:31.017273+0000","last_became_active":"2024-04-13T03:09:29.567230+0000","last_became_peered":"2024-04-13T03:09:29.567230+0000","last_unstale":"2024-04-13T03:09:31.017273+0000","last_undegraded":"2024-04-13T03:09:31.017273+0000","last_fullsized":"2024-04-13T03:09:31.017273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:23:03.577409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073579+0000","last_change":"2024-04-13T03:09:24.978027+0000","last_active":"2024-04-13T03:09:32.073579+0000","last_peered":"2024-04-13T03:09:32.073579+0000","last_clean":"2024-04-13T03:09:32.073579+0000","last_became_active":"2024-04-13T03:09:24.977709+0000","last_became_peered":"2024-04-13T03:09:24.977709+0000","last_unstale":"2024-04-13T03:09:32.073579+0000","last_undegraded":"2024-04-13T03:09:32.073579+0000","last_fullsized":"2024-04-13T03:09:32.073579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006349+0000","last_change":"2024-04-13T03:09:23.972192+0000","last_active":"2024-04-13T03:09:31.006349+0000","last_peered":"2024-04-13T03:09:31.006349+0000","last_clean":"2024-04-13T03:09:31.006349+0000","last_became_active":"2024-04-13T03:09:23.971856+0000","last_became_peered":"2024-04-13T03:09:23.971856+0000","last_unstale":"2024-04-13T03:09:31.006349+0000","last_undegraded":"2024-04-13T03:09:31.006349+0000","last_fullsized":"2024-04-13T03:09:31.006349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017103+0000","last_change":"2024-04-13T03:09:29.565473+0000","last_active":"2024-04-13T03:09:31.017103+0000","last_peered":"2024-04-13T03:09:31.017103+0000","last_clean":"2024-04-13T03:09:31.017103+0000","last_became_active":"2024-04-13T03:09:29.565099+0000","last_became_peered":"2024-04-13T03:09:29.565099+0000","last_unstale":"2024-04-13T03:09:31.017103+0000","last_undegraded":"2024-04-13T03:09:31.017103+0000","last_fullsized":"2024-04-13T03:09:31.017103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:49:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004475+0000","last_change":"2024-04-13T03:09:24.980510+0000","last_active":"2024-04-13T03:09:31.004475+0000","last_peered":"2024-04-13T03:09:31.004475+0000","last_clean":"2024-04-13T03:09:31.004475+0000","last_became_active":"2024-04-13T03:09:24.980226+0000","last_became_peered":"2024-04-13T03:09:24.980226+0000","last_unstale":"2024-04-13T03:09:31.004475+0000","last_undegraded":"2024-04-13T03:09:31.004475+0000","last_fullsized":"2024-04-13T03:09:31.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004618+0000","last_change":"2024-04-13T03:09:23.971205+0000","last_active":"2024-04-13T03:09:31.004618+0000","last_peered":"2024-04-13T03:09:31.004618+0000","last_clean":"2024-04-13T03:09:31.004618+0000","last_became_active":"2024-04-13T03:09:23.970845+0000","last_became_peered":"2024-04-13T03:09:23.970845+0000","last_unstale":"2024-04-13T03:09:31.004618+0000","last_undegraded":"2024-04-13T03:09:31.004618+0000","last_fullsized":"2024-04-13T03:09:31.004618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005480+0000","last_change":"2024-04-13T03:09:29.568847+0000","last_active":"2024-04-13T03:09:31.005480+0000","last_peered":"2024-04-13T03:09:31.005480+0000","last_clean":"2024-04-13T03:09:31.005480+0000","last_became_active":"2024-04-13T03:09:29.568286+0000","last_became_peered":"2024-04-13T03:09:29.568286+0000","last_unstale":"2024-04-13T03:09:31.005480+0000","last_undegraded":"2024-04-13T03:09:31.005480+0000","last_fullsized":"2024-04-13T03:09:31.005480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073615+0000","last_change":"2024-04-13T03:09:24.980651+0000","last_active":"2024-04-13T03:09:32.073615+0000","last_peered":"2024-04-13T03:09:32.073615+0000","last_clean":"2024-04-13T03:09:32.073615+0000","last_became_active":"2024-04-13T03:09:24.980390+0000","last_became_peered":"2024-04-13T03:09:24.980390+0000","last_unstale":"2024-04-13T03:09:32.073615+0000","last_undegraded":"2024-04-13T03:09:32.073615+0000","last_fullsized":"2024-04-13T03:09:32.073615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:34.382842+0000","last_change":"2024-04-13T03:09:23.977841+0000","last_active":"2024-04-13T03:09:34.382842+0000","last_peered":"2024-04-13T03:09:34.382842+0000","last_clean":"2024-04-13T03:09:34.382842+0000","last_became_active":"2024-04-13T03:09:23.977546+0000","last_became_peered":"2024-04-13T03:09:23.977546+0000","last_unstale":"2024-04-13T03:09:34.382842+0000","last_undegraded":"2024-04-13T03:09:34.382842+0000","last_fullsized":"2024-04-13T03:09:34.382842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552322+0000","last_change":"2024-04-13T03:09:29.012775+0000","last_active":"2024-04-13T03:09:31.552322+0000","last_peered":"2024-04-13T03:09:31.552322+0000","last_clean":"2024-04-13T03:09:31.552322+0000","last_became_active":"2024-04-13T03:09:29.011952+0000","last_became_peered":"2024-04-13T03:09:29.011952+0000","last_unstale":"2024-04-13T03:09:31.552322+0000","last_undegraded":"2024-04-13T03:09:31.552322+0000","last_fullsized":"2024-04-13T03:09:31.552322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005496+0000","last_change":"2024-04-13T03:09:24.980974+0000","last_active":"2024-04-13T03:09:31.005496+0000","last_peered":"2024-04-13T03:09:31.005496+0000","last_clean":"2024-04-13T03:09:31.005496+0000","last_became_active":"2024-04-13T03:09:24.980580+0000","last_became_peered":"2024-04-13T03:09:24.980580+0000","last_unstale":"2024-04-13T03:09:31.005496+0000","last_undegraded":"2024-04-13T03:09:31.005496+0000","last_fullsized":"2024-04-13T03:09:31.005496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552665+0000","last_change":"2024-04-13T03:09:23.970652+0000","last_active":"2024-04-13T03:09:31.552665+0000","last_peered":"2024-04-13T03:09:31.552665+0000","last_clean":"2024-04-13T03:09:31.552665+0000","last_became_active":"2024-04-13T03:09:23.970405+0000","last_became_peered":"2024-04-13T03:09:23.970405+0000","last_unstale":"2024-04-13T03:09:31.552665+0000","last_undegraded":"2024-04-13T03:09:31.552665+0000","last_fullsized":"2024-04-13T03:09:31.552665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004925+0000","last_change":"2024-04-13T03:09:29.009723+0000","last_active":"2024-04-13T03:09:31.004925+0000","last_peered":"2024-04-13T03:09:31.004925+0000","last_clean":"2024-04-13T03:09:31.004925+0000","last_became_active":"2024-04-13T03:09:29.009177+0000","last_became_peered":"2024-04-13T03:09:29.009177+0000","last_unstale":"2024-04-13T03:09:31.004925+0000","last_undegraded":"2024-04-13T03:09:31.004925+0000","last_fullsized":"2024-04-13T03:09:31.004925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004826+0000","last_change":"2024-04-13T03:09:24.976839+0000","last_active":"2024-04-13T03:09:31.004826+0000","last_peered":"2024-04-13T03:09:31.004826+0000","last_clean":"2024-04-13T03:09:31.004826+0000","last_became_active":"2024-04-13T03:09:24.976565+0000","last_became_peered":"2024-04-13T03:09:24.976565+0000","last_unstale":"2024-04-13T03:09:31.004826+0000","last_undegraded":"2024-04-13T03:09:31.004826+0000","last_fullsized":"2024-04-13T03:09:31.004826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005802+0000","last_change":"2024-04-13T03:09:23.974057+0000","last_active":"2024-04-13T03:09:31.005802+0000","last_peered":"2024-04-13T03:09:31.005802+0000","last_clean":"2024-04-13T03:09:31.005802+0000","last_became_active":"2024-04-13T03:09:23.973783+0000","last_became_peered":"2024-04-13T03:09:23.973783+0000","last_unstale":"2024-04-13T03:09:31.005802+0000","last_undegraded":"2024-04-13T03:09:31.005802+0000","last_fullsized":"2024-04-13T03:09:31.005802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016630+0000","last_change":"2024-04-13T03:09:29.008489+0000","last_active":"2024-04-13T03:09:31.016630+0000","last_peered":"2024-04-13T03:09:31.016630+0000","last_clean":"2024-04-13T03:09:31.016630+0000","last_became_active":"2024-04-13T03:09:29.008129+0000","last_became_peered":"2024-04-13T03:09:29.008129+0000","last_unstale":"2024-04-13T03:09:31.016630+0000","last_undegraded":"2024-04-13T03:09:31.016630+0000","last_fullsized":"2024-04-13T03:09:31.016630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:03.613440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005713+0000","last_change":"2024-04-13T03:09:24.988904+0000","last_active":"2024-04-13T03:09:31.005713+0000","last_peered":"2024-04-13T03:09:31.005713+0000","last_clean":"2024-04-13T03:09:31.005713+0000","last_became_active":"2024-04-13T03:09:24.988667+0000","last_became_peered":"2024-04-13T03:09:24.988667+0000","last_unstale":"2024-04-13T03:09:31.005713+0000","last_undegraded":"2024-04-13T03:09:31.005713+0000","last_fullsized":"2024-04-13T03:09:31.005713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074106+0000","last_change":"2024-04-13T03:09:23.975478+0000","last_active":"2024-04-13T03:09:32.074106+0000","last_peered":"2024-04-13T03:09:32.074106+0000","last_clean":"2024-04-13T03:09:32.074106+0000","last_became_active":"2024-04-13T03:09:23.975155+0000","last_became_peered":"2024-04-13T03:09:23.975155+0000","last_unstale":"2024-04-13T03:09:32.074106+0000","last_undegraded":"2024-04-13T03:09:32.074106+0000","last_fullsized":"2024-04-13T03:09:32.074106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016599+0000","last_change":"2024-04-13T03:09:29.565039+0000","last_active":"2024-04-13T03:09:31.016599+0000","last_peered":"2024-04-13T03:09:31.016599+0000","last_clean":"2024-04-13T03:09:31.016599+0000","last_became_active":"2024-04-13T03:09:29.564715+0000","last_became_peered":"2024-04-13T03:09:29.564715+0000","last_unstale":"2024-04-13T03:09:31.016599+0000","last_undegraded":"2024-04-13T03:09:31.016599+0000","last_fullsized":"2024-04-13T03:09:31.016599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:38.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004534+0000","last_change":"2024-04-13T03:09:29.567360+0000","last_active":"2024-04-13T03:09:31.004534+0000","last_peered":"2024-04-13T03:09:31.004534+0000","last_clean":"2024-04-13T03:09:31.004534+0000","last_became_active":"2024-04-13T03:09:29.567005+0000","last_became_peered":"2024-04-13T03:09:29.567005+0000","last_unstale":"2024-04-13T03:09:31.004534+0000","last_undegraded":"2024-04-13T03:09:31.004534+0000","last_fullsized":"2024-04-13T03:09:31.004534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552017+0000","last_change":"2024-04-13T03:09:24.980403+0000","last_active":"2024-04-13T03:09:31.552017+0000","last_peered":"2024-04-13T03:09:31.552017+0000","last_clean":"2024-04-13T03:09:31.552017+0000","last_became_active":"2024-04-13T03:09:24.980125+0000","last_became_peered":"2024-04-13T03:09:24.980125+0000","last_unstale":"2024-04-13T03:09:31.552017+0000","last_undegraded":"2024-04-13T03:09:31.552017+0000","last_fullsized":"2024-04-13T03:09:31.552017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006298+0000","last_change":"2024-04-13T03:09:23.966400+0000","last_active":"2024-04-13T03:09:31.006298+0000","last_peered":"2024-04-13T03:09:31.006298+0000","last_clean":"2024-04-13T03:09:31.006298+0000","last_became_active":"2024-04-13T03:09:23.966136+0000","last_became_peered":"2024-04-13T03:09:23.966136+0000","last_unstale":"2024-04-13T03:09:31.006298+0000","last_undegraded":"2024-04-13T03:09:31.006298+0000","last_fullsized":"2024-04-13T03:09:31.006298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074769+0000","last_change":"2024-04-13T03:09:29.008579+0000","last_active":"2024-04-13T03:09:32.074769+0000","last_peered":"2024-04-13T03:09:32.074769+0000","last_clean":"2024-04-13T03:09:32.074769+0000","last_became_active":"2024-04-13T03:09:29.008030+0000","last_became_peered":"2024-04-13T03:09:29.008030+0000","last_unstale":"2024-04-13T03:09:32.074769+0000","last_undegraded":"2024-04-13T03:09:32.074769+0000","last_fullsized":"2024-04-13T03:09:32.074769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016574+0000","last_change":"2024-04-13T03:09:24.972241+0000","last_active":"2024-04-13T03:09:31.016574+0000","last_peered":"2024-04-13T03:09:31.016574+0000","last_clean":"2024-04-13T03:09:31.016574+0000","last_became_active":"2024-04-13T03:09:24.971962+0000","last_became_peered":"2024-04-13T03:09:24.971962+0000","last_unstale":"2024-04-13T03:09:31.016574+0000","last_undegraded":"2024-04-13T03:09:31.016574+0000","last_fullsized":"2024-04-13T03:09:31.016574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016806+0000","last_change":"2024-04-13T03:09:23.977688+0000","last_active":"2024-04-13T03:09:31.016806+0000","last_peered":"2024-04-13T03:09:31.016806+0000","last_clean":"2024-04-13T03:09:31.016806+0000","last_became_active":"2024-04-13T03:09:23.977412+0000","last_became_peered":"2024-04-13T03:09:23.977412+0000","last_unstale":"2024-04-13T03:09:31.016806+0000","last_undegraded":"2024-04-13T03:09:31.016806+0000","last_fullsized":"2024-04-13T03:09:31.016806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072995+0000","last_change":"2024-04-13T03:09:23.973771+0000","last_active":"2024-04-13T03:09:32.072995+0000","last_peered":"2024-04-13T03:09:32.072995+0000","last_clean":"2024-04-13T03:09:32.072995+0000","last_became_active":"2024-04-13T03:09:23.973271+0000","last_became_peered":"2024-04-13T03:09:23.973271+0000","last_unstale":"2024-04-13T03:09:32.072995+0000","last_undegraded":"2024-04-13T03:09:32.072995+0000","last_fullsized":"2024-04-13T03:09:32.072995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004613+0000","last_change":"2024-04-13T03:09:24.974833+0000","last_active":"2024-04-13T03:09:31.004613+0000","last_peered":"2024-04-13T03:09:31.004613+0000","last_clean":"2024-04-13T03:09:31.004613+0000","last_became_active":"2024-04-13T03:09:24.974554+0000","last_became_peered":"2024-04-13T03:09:24.974554+0000","last_unstale":"2024-04-13T03:09:31.004613+0000","last_undegraded":"2024-04-13T03:09:31.004613+0000","last_fullsized":"2024-04-13T03:09:31.004613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072905+0000","last_change":"2024-04-13T03:09:29.004327+0000","last_active":"2024-04-13T03:09:32.072905+0000","last_peered":"2024-04-13T03:09:32.072905+0000","last_clean":"2024-04-13T03:09:32.072905+0000","last_became_active":"2024-04-13T03:09:29.003998+0000","last_became_peered":"2024-04-13T03:09:29.003998+0000","last_unstale":"2024-04-13T03:09:32.072905+0000","last_undegraded":"2024-04-13T03:09:32.072905+0000","last_fullsized":"2024-04-13T03:09:32.072905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:10:12.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016845+0000","last_change":"2024-04-13T03:09:23.976680+0000","last_active":"2024-04-13T03:09:31.016845+0000","last_peered":"2024-04-13T03:09:31.016845+0000","last_clean":"2024-04-13T03:09:31.016845+0000","last_became_active":"2024-04-13T03:09:23.976390+0000","last_became_peered":"2024-04-13T03:09:23.976390+0000","last_unstale":"2024-04-13T03:09:31.016845+0000","last_undegraded":"2024-04-13T03:09:31.016845+0000","last_fullsized":"2024-04-13T03:09:31.016845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004533+0000","last_change":"2024-04-13T03:09:24.971520+0000","last_active":"2024-04-13T03:09:31.004533+0000","last_peered":"2024-04-13T03:09:31.004533+0000","last_clean":"2024-04-13T03:09:31.004533+0000","last_became_active":"2024-04-13T03:09:24.971236+0000","last_became_peered":"2024-04-13T03:09:24.971236+0000","last_unstale":"2024-04-13T03:09:31.004533+0000","last_undegraded":"2024-04-13T03:09:31.004533+0000","last_fullsized":"2024-04-13T03:09:31.004533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:47.399993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017010+0000","last_change":"2024-04-13T03:09:29.008243+0000","last_active":"2024-04-13T03:09:31.017010+0000","last_peered":"2024-04-13T03:09:31.017010+0000","last_clean":"2024-04-13T03:09:31.017010+0000","last_became_active":"2024-04-13T03:09:29.007854+0000","last_became_peered":"2024-04-13T03:09:29.007854+0000","last_unstale":"2024-04-13T03:09:31.017010+0000","last_undegraded":"2024-04-13T03:09:31.017010+0000","last_fullsized":"2024-04-13T03:09:31.017010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:33.651724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073532+0000","last_change":"2024-04-13T03:09:24.984919+0000","last_active":"2024-04-13T03:09:32.073532+0000","last_peered":"2024-04-13T03:09:32.073532+0000","last_clean":"2024-04-13T03:09:32.073532+0000","last_became_active":"2024-04-13T03:09:24.984652+0000","last_became_peered":"2024-04-13T03:09:24.984652+0000","last_unstale":"2024-04-13T03:09:32.073532+0000","last_undegraded":"2024-04-13T03:09:32.073532+0000","last_fullsized":"2024-04-13T03:09:32.073532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072697+0000","last_change":"2024-04-13T03:09:24.980195+0000","last_active":"2024-04-13T03:09:32.072697+0000","last_peered":"2024-04-13T03:09:32.072697+0000","last_clean":"2024-04-13T03:09:32.072697+0000","last_became_active":"2024-04-13T03:09:24.979846+0000","last_became_peered":"2024-04-13T03:09:24.979846+0000","last_unstale":"2024-04-13T03:09:32.072697+0000","last_undegraded":"2024-04-13T03:09:32.072697+0000","last_fullsized":"2024-04-13T03:09:32.072697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006342+0000","last_change":"2024-04-13T03:09:24.979746+0000","last_active":"2024-04-13T03:09:31.006342+0000","last_peered":"2024-04-13T03:09:31.006342+0000","last_clean":"2024-04-13T03:09:31.006342+0000","last_became_active":"2024-04-13T03:09:24.979481+0000","last_became_peered":"2024-04-13T03:09:24.979481+0000","last_unstale":"2024-04-13T03:09:31.006342+0000","last_undegraded":"2024-04-13T03:09:31.006342+0000","last_fullsized":"2024-04-13T03:09:31.006342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004510+0000","last_change":"2024-04-13T03:09:24.974603+0000","last_active":"2024-04-13T03:09:31.004510+0000","last_peered":"2024-04-13T03:09:31.004510+0000","last_clean":"2024-04-13T03:09:31.004510+0000","last_became_active":"2024-04-13T03:09:24.974099+0000","last_became_peered":"2024-04-13T03:09:24.974099+0000","last_unstale":"2024-04-13T03:09:31.004510+0000","last_undegraded":"2024-04-13T03:09:31.004510+0000","last_fullsized":"2024-04-13T03:09:31.004510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017151+0000","last_change":"2024-04-13T03:09:24.977844+0000","last_active":"2024-04-13T03:09:31.017151+0000","last_peered":"2024-04-13T03:09:31.017151+0000","last_clean":"2024-04-13T03:09:31.017151+0000","last_became_active":"2024-04-13T03:09:24.977578+0000","last_became_peered":"2024-04-13T03:09:24.977578+0000","last_unstale":"2024-04-13T03:09:31.017151+0000","last_undegraded":"2024-04-13T03:09:31.017151+0000","last_fullsized":"2024-04-13T03:09:31.017151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552548+0000","last_change":"2024-04-13T03:09:24.981088+0000","last_active":"2024-04-13T03:09:31.552548+0000","last_peered":"2024-04-13T03:09:31.552548+0000","last_clean":"2024-04-13T03:09:31.552548+0000","last_became_active":"2024-04-13T03:09:24.980793+0000","last_became_peered":"2024-04-13T03:09:24.980793+0000","last_unstale":"2024-04-13T03:09:31.552548+0000","last_undegraded":"2024-04-13T03:09:31.552548+0000","last_fullsized":"2024-04-13T03:09:31.552548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074858+0000","last_change":"2024-04-13T03:09:24.990583+0000","last_active":"2024-04-13T03:09:32.074858+0000","last_peered":"2024-04-13T03:09:32.074858+0000","last_clean":"2024-04-13T03:09:32.074858+0000","last_became_active":"2024-04-13T03:09:24.990319+0000","last_became_peered":"2024-04-13T03:09:24.990319+0000","last_unstale":"2024-04-13T03:09:32.074858+0000","last_undegraded":"2024-04-13T03:09:32.074858+0000","last_fullsized":"2024-04-13T03:09:32.074858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017256+0000","last_change":"2024-04-13T03:09:24.975238+0000","last_active":"2024-04-13T03:09:31.017256+0000","last_peered":"2024-04-13T03:09:31.017256+0000","last_clean":"2024-04-13T03:09:31.017256+0000","last_became_active":"2024-04-13T03:09:24.974911+0000","last_became_peered":"2024-04-13T03:09:24.974911+0000","last_unstale":"2024-04-13T03:09:31.017256+0000","last_undegraded":"2024-04-13T03:09:31.017256+0000","last_fullsized":"2024-04-13T03:09:31.017256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005161+0000","last_change":"2024-04-13T03:09:24.976029+0000","last_active":"2024-04-13T03:09:31.005161+0000","last_peered":"2024-04-13T03:09:31.005161+0000","last_clean":"2024-04-13T03:09:31.005161+0000","last_became_active":"2024-04-13T03:09:24.975739+0000","last_became_peered":"2024-04-13T03:09:24.975739+0000","last_unstale":"2024-04-13T03:09:31.005161+0000","last_undegraded":"2024-04-13T03:09:31.005161+0000","last_fullsized":"2024-04-13T03:09:31.005161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016664+0000","last_change":"2024-04-13T03:09:24.975447+0000","last_active":"2024-04-13T03:09:31.016664+0000","last_peered":"2024-04-13T03:09:31.016664+0000","last_clean":"2024-04-13T03:09:31.016664+0000","last_became_active":"2024-04-13T03:09:24.975131+0000","last_became_peered":"2024-04-13T03:09:24.975131+0000","last_unstale":"2024-04-13T03:09:31.016664+0000","last_undegraded":"2024-04-13T03:09:31.016664+0000","last_fullsized":"2024-04-13T03:09:31.016664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006028+0000","last_change":"2024-04-13T03:09:24.988399+0000","last_active":"2024-04-13T03:09:31.006028+0000","last_peered":"2024-04-13T03:09:31.006028+0000","last_clean":"2024-04-13T03:09:31.006028+0000","last_became_active":"2024-04-13T03:09:24.988064+0000","last_became_peered":"2024-04-13T03:09:24.988064+0000","last_unstale":"2024-04-13T03:09:31.006028+0000","last_undegraded":"2024-04-13T03:09:31.006028+0000","last_fullsized":"2024-04-13T03:09:31.006028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004399+0000","last_change":"2024-04-13T03:09:24.980428+0000","last_active":"2024-04-13T03:09:31.004399+0000","last_peered":"2024-04-13T03:09:31.004399+0000","last_clean":"2024-04-13T03:09:31.004399+0000","last_became_active":"2024-04-13T03:09:24.980157+0000","last_became_peered":"2024-04-13T03:09:24.980157+0000","last_unstale":"2024-04-13T03:09:31.004399+0000","last_undegraded":"2024-04-13T03:09:31.004399+0000","last_fullsized":"2024-04-13T03:09:31.004399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074511+0000","last_change":"2024-04-13T03:09:24.978696+0000","last_active":"2024-04-13T03:09:32.074511+0000","last_peered":"2024-04-13T03:09:32.074511+0000","last_clean":"2024-04-13T03:09:32.074511+0000","last_became_active":"2024-04-13T03:09:24.978413+0000","last_became_peered":"2024-04-13T03:09:24.978413+0000","last_unstale":"2024-04-13T03:09:32.074511+0000","last_undegraded":"2024-04-13T03:09:32.074511+0000","last_fullsized":"2024-04-13T03:09:32.074511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074117+0000","last_change":"2024-04-13T03:09:24.979845+0000","last_active":"2024-04-13T03:09:32.074117+0000","last_peered":"2024-04-13T03:09:32.074117+0000","last_clean":"2024-04-13T03:09:32.074117+0000","last_became_active":"2024-04-13T03:09:24.979598+0000","last_became_peered":"2024-04-13T03:09:24.979598+0000","last_unstale":"2024-04-13T03:09:32.074117+0000","last_undegraded":"2024-04-13T03:09:32.074117+0000","last_fullsized":"2024-04-13T03:09:32.074117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073041+0000","last_change":"2024-04-13T03:09:24.978510+0000","last_active":"2024-04-13T03:09:32.073041+0000","last_peered":"2024-04-13T03:09:32.073041+0000","last_clean":"2024-04-13T03:09:32.073041+0000","last_became_active":"2024-04-13T03:09:24.978226+0000","last_became_peered":"2024-04-13T03:09:24.978226+0000","last_unstale":"2024-04-13T03:09:32.073041+0000","last_undegraded":"2024-04-13T03:09:32.073041+0000","last_fullsized":"2024-04-13T03:09:32.073041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006512+0000","last_change":"2024-04-13T03:09:24.986783+0000","last_active":"2024-04-13T03:09:31.006512+0000","last_peered":"2024-04-13T03:09:31.006512+0000","last_clean":"2024-04-13T03:09:31.006512+0000","last_became_active":"2024-04-13T03:09:24.986139+0000","last_became_peered":"2024-04-13T03:09:24.986139+0000","last_unstale":"2024-04-13T03:09:31.006512+0000","last_undegraded":"2024-04-13T03:09:31.006512+0000","last_fullsized":"2024-04-13T03:09:31.006512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005046+0000","last_change":"2024-04-13T03:09:24.974308+0000","last_active":"2024-04-13T03:09:31.005046+0000","last_peered":"2024-04-13T03:09:31.005046+0000","last_clean":"2024-04-13T03:09:31.005046+0000","last_became_active":"2024-04-13T03:09:24.973935+0000","last_became_peered":"2024-04-13T03:09:24.973935+0000","last_unstale":"2024-04-13T03:09:31.005046+0000","last_undegraded":"2024-04-13T03:09:31.005046+0000","last_fullsized":"2024-04-13T03:09:31.005046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552097+0000","last_change":"2024-04-13T03:09:24.971657+0000","last_active":"2024-04-13T03:09:31.552097+0000","last_peered":"2024-04-13T03:09:31.552097+0000","last_clean":"2024-04-13T03:09:31.552097+0000","last_became_active":"2024-04-13T03:09:24.971379+0000","last_became_peered":"2024-04-13T03:09:24.971379+0000","last_unstale":"2024-04-13T03:09:31.552097+0000","last_undegraded":"2024-04-13T03:09:31.552097+0000","last_fullsized":"2024-04-13T03:09:31.552097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004364+0000","last_change":"2024-04-13T03:09:24.973741+0000","last_active":"2024-04-13T03:09:31.004364+0000","last_peered":"2024-04-13T03:09:31.004364+0000","last_clean":"2024-04-13T03:09:31.004364+0000","last_became_active":"2024-04-13T03:09:24.973468+0000","last_became_peered":"2024-04-13T03:09:24.973468+0000","last_unstale":"2024-04-13T03:09:31.004364+0000","last_undegraded":"2024-04-13T03:09:31.004364+0000","last_fullsized":"2024-04-13T03:09:31.004364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:20.516278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016730+0000","last_change":"2024-04-13T03:09:24.982585+0000","last_active":"2024-04-13T03:09:31.016730+0000","last_peered":"2024-04-13T03:09:31.016730+0000","last_clean":"2024-04-13T03:09:31.016730+0000","last_became_active":"2024-04-13T03:09:24.982301+0000","last_became_peered":"2024-04-13T03:09:24.982301+0000","last_unstale":"2024-04-13T03:09:31.016730+0000","last_undegraded":"2024-04-13T03:09:31.016730+0000","last_fullsized":"2024-04-13T03:09:31.016730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005384+0000","last_change":"2024-04-13T03:09:24.987283+0000","last_active":"2024-04-13T03:09:31.005384+0000","last_peered":"2024-04-13T03:09:31.005384+0000","last_clean":"2024-04-13T03:09:31.005384+0000","last_became_active":"2024-04-13T03:09:24.983517+0000","last_became_peered":"2024-04-13T03:09:24.983517+0000","last_unstale":"2024-04-13T03:09:31.005384+0000","last_undegraded":"2024-04-13T03:09:31.005384+0000","last_fullsized":"2024-04-13T03:09:31.005384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:43.988718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074632+0000","last_change":"2024-04-13T03:09:24.994512+0000","last_active":"2024-04-13T03:09:32.074632+0000","last_peered":"2024-04-13T03:09:32.074632+0000","last_clean":"2024-04-13T03:09:32.074632+0000","last_became_active":"2024-04-13T03:09:24.994212+0000","last_became_peered":"2024-04-13T03:09:24.994212+0000","last_unstale":"2024-04-13T03:09:32.074632+0000","last_undegraded":"2024-04-13T03:09:32.074632+0000","last_fullsized":"2024-04-13T03:09:32.074632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074805+0000","last_change":"2024-04-13T03:09:24.990511+0000","last_active":"2024-04-13T03:09:32.074805+0000","last_peered":"2024-04-13T03:09:32.074805+0000","last_clean":"2024-04-13T03:09:32.074805+0000","last_became_active":"2024-04-13T03:09:24.990230+0000","last_became_peered":"2024-04-13T03:09:24.990230+0000","last_unstale":"2024-04-13T03:09:32.074805+0000","last_undegraded":"2024-04-13T03:09:32.074805+0000","last_fullsized":"2024-04-13T03:09:32.074805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074337+0000","last_change":"2024-04-13T03:09:24.991357+0000","last_active":"2024-04-13T03:09:32.074337+0000","last_peered":"2024-04-13T03:09:32.074337+0000","last_clean":"2024-04-13T03:09:32.074337+0000","last_became_active":"2024-04-13T03:09:24.990754+0000","last_became_peered":"2024-04-13T03:09:24.990754+0000","last_unstale":"2024-04-13T03:09:32.074337+0000","last_undegraded":"2024-04-13T03:09:32.074337+0000","last_fullsized":"2024-04-13T03:09:32.074337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016489+0000","last_change":"2024-04-13T03:09:24.984967+0000","last_active":"2024-04-13T03:09:31.016489+0000","last_peered":"2024-04-13T03:09:31.016489+0000","last_clean":"2024-04-13T03:09:31.016489+0000","last_became_active":"2024-04-13T03:09:24.984674+0000","last_became_peered":"2024-04-13T03:09:24.984674+0000","last_unstale":"2024-04-13T03:09:31.016489+0000","last_undegraded":"2024-04-13T03:09:31.016489+0000","last_fullsized":"2024-04-13T03:09:31.016489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073906+0000","last_change":"2024-04-13T03:09:24.976323+0000","last_active":"2024-04-13T03:09:32.073906+0000","last_peered":"2024-04-13T03:09:32.073906+0000","last_clean":"2024-04-13T03:09:32.073906+0000","last_became_active":"2024-04-13T03:09:24.976014+0000","last_became_peered":"2024-04-13T03:09:24.976014+0000","last_unstale":"2024-04-13T03:09:32.073906+0000","last_undegraded":"2024-04-13T03:09:32.073906+0000","last_fullsized":"2024-04-13T03:09:32.073906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005924+0000","last_change":"2024-04-13T03:09:24.987113+0000","last_active":"2024-04-13T03:09:31.005924+0000","last_peered":"2024-04-13T03:09:31.005924+0000","last_clean":"2024-04-13T03:09:31.005924+0000","last_became_active":"2024-04-13T03:09:24.986836+0000","last_became_peered":"2024-04-13T03:09:24.986836+0000","last_unstale":"2024-04-13T03:09:31.005924+0000","last_undegraded":"2024-04-13T03:09:31.005924+0000","last_fullsized":"2024-04-13T03:09:31.005924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073397+0000","last_change":"2024-04-13T03:09:24.979549+0000","last_active":"2024-04-13T03:09:32.073397+0000","last_peered":"2024-04-13T03:09:32.073397+0000","last_clean":"2024-04-13T03:09:32.073397+0000","last_became_active":"2024-04-13T03:09:24.979203+0000","last_became_peered":"2024-04-13T03:09:24.979203+0000","last_unstale":"2024-04-13T03:09:32.073397+0000","last_undegraded":"2024-04-13T03:09:32.073397+0000","last_fullsized":"2024-04-13T03:09:32.073397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006135+0000","last_change":"2024-04-13T03:09:24.986349+0000","last_active":"2024-04-13T03:09:31.006135+0000","last_peered":"2024-04-13T03:09:31.006135+0000","last_clean":"2024-04-13T03:09:31.006135+0000","last_became_active":"2024-04-13T03:09:24.985694+0000","last_became_peered":"2024-04-13T03:09:24.985694+0000","last_unstale":"2024-04-13T03:09:31.006135+0000","last_undegraded":"2024-04-13T03:09:31.006135+0000","last_fullsized":"2024-04-13T03:09:31.006135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004564+0000","last_change":"2024-04-13T03:09:24.973883+0000","last_active":"2024-04-13T03:09:31.004564+0000","last_peered":"2024-04-13T03:09:31.004564+0000","last_clean":"2024-04-13T03:09:31.004564+0000","last_became_active":"2024-04-13T03:09:24.973623+0000","last_became_peered":"2024-04-13T03:09:24.973623+0000","last_unstale":"2024-04-13T03:09:31.004564+0000","last_undegraded":"2024-04-13T03:09:31.004564+0000","last_fullsized":"2024-04-13T03:09:31.004564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552728+0000","last_change":"2024-04-13T03:09:24.980289+0000","last_active":"2024-04-13T03:09:31.552728+0000","last_peered":"2024-04-13T03:09:31.552728+0000","last_clean":"2024-04-13T03:09:31.552728+0000","last_became_active":"2024-04-13T03:09:24.980040+0000","last_became_peered":"2024-04-13T03:09:24.980040+0000","last_unstale":"2024-04-13T03:09:31.552728+0000","last_undegraded":"2024-04-13T03:09:31.552728+0000","last_fullsized":"2024-04-13T03:09:31.552728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074027+0000","last_change":"2024-04-13T03:09:24.985108+0000","last_active":"2024-04-13T03:09:32.074027+0000","last_peered":"2024-04-13T03:09:32.074027+0000","last_clean":"2024-04-13T03:09:32.074027+0000","last_became_active":"2024-04-13T03:09:24.984576+0000","last_became_peered":"2024-04-13T03:09:24.984576+0000","last_unstale":"2024-04-13T03:09:32.074027+0000","last_undegraded":"2024-04-13T03:09:32.074027+0000","last_fullsized":"2024-04-13T03:09:32.074027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:09:48.364 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:09:48.364 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-13T03:09:48.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-13T03:09:48.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-13T03:09:48.365 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-13T03:09:48.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.499+0000 7f8e4b488640 1 Processor -- start 2024-04-13T03:09:48.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.499+0000 7f8e4b488640 1 -- start start 2024-04-13T03:09:48.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e4b488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e4b488640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e440b68d0 0x7f8e440d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e4b488640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4415eda0 con 0x7f8e441624b0 2024-04-13T03:09:48.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e4b488640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8e440d27f0 con 0x7f8e440b68d0 2024-04-13T03:09:48.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e4b488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8e44162c20 con 0x7f8e4415e9c0 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e440b68d0 0x7f8e440d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e440b68d0 0x7f8e440d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44946/0 (socket says 172.21.15.106:44946) 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 -- 172.21.15.106:0/3896857231 learned_addr learned my addr 172.21.15.106:0/3896857231 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 -- 172.21.15.106:0/3896857231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 msgr2=0x7f8e440ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49224640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 -- 172.21.15.106:0/3896857231 >> v1:172.21.15.114:6789/0 conn(0x7f8e441624b0 legacy=0x7f8e440b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.500+0000 7f8e49a25640 1 -- 172.21.15.106:0/3896857231 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e44162f00 con 0x7f8e440b68d0 2024-04-13T03:09:48.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e49a25640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e440b68d0 0x7f8e440d2f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009e50 tx=0x7f8e3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378700aa22607bf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e3bfff640 1 -- 172.21.15.106:0/3896857231 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c02ca70 con 0x7f8e440b68d0 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e3bfff640 1 -- 172.21.15.106:0/3896857231 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c02cc10 con 0x7f8e440b68d0 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e3bfff640 1 -- 172.21.15.106:0/3896857231 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0343f0 con 0x7f8e440b68d0 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 -- 172.21.15.106:0/3896857231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e440b68d0 msgr2=0x7f8e440d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e440b68d0 0x7f8e440d2f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009e50 tx=0x7f8e3c02bed0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 -- 172.21.15.106:0/3896857231 shutdown_connections 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e440b68d0 0x7f8e440d2f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 --2- 172.21.15.106:0/3896857231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.501+0000 7f8e4b488640 1 -- 172.21.15.106:0/3896857231 >> 172.21.15.106:0/3896857231 conn(0x7f8e440d11c0 msgr2=0x7f8e440d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.502+0000 7f8e4b488640 1 -- 172.21.15.106:0/3896857231 shutdown_connections 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.502+0000 7f8e4b488640 1 -- 172.21.15.106:0/3896857231 wait complete. 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 Processor -- start 2024-04-13T03:09:48.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 -- start start 2024-04-13T03:09:48.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 0x7f8e44152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44162910 con 0x7f8e441624b0 2024-04-13T03:09:48.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8e440d2a40 con 0x7f8e440b68d0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e4b488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8e440d34a0 con 0x7f8e4415e9c0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.503+0000 7f8e49a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44960/0 (socket says 172.21.15.106:44960) 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 -- 172.21.15.106:0/1192666758 learned_addr learned my addr 172.21.15.106:0/1192666758 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49224640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 0x7f8e44152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e48a23640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 -- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 msgr2=0x7f8e440cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 -- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 msgr2=0x7f8e44152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 0x7f8e44152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 -- 172.21.15.106:0/1192666758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e440cc1f0 con 0x7f8e441624b0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49224640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 0x7f8e44152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e49a25640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009f70 tx=0x7f8e3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c043020 con 0x7f8e441624b0 2024-04-13T03:09:48.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c03e450 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c009510 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e440cc4d0 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.504+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e440cca90 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.505+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44068a10 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.506+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e3c046020 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.506+0000 7f8e39ffb640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 0x7f8e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.506+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8e3c0ae7e0 con 0x7f8e441624b0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.506+0000 7f8e49224640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 0x7f8e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.506+0000 7f8e49224640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 0x7f8e2806edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e40008890 tx=0x7f8e40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.510+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e3c04d030 con 0x7f8e441624b0 2024-04-13T03:09:48.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.733+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8e440cce60 con 0x7f8e441624b0 2024-04-13T03:09:48.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.736+0000 7f8e39ffb640 1 -- 172.21.15.106:0/1192666758 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7f8e3c07c8d0 con 0x7f8e441624b0 2024-04-13T03:09:48.739 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:48.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.738+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 msgr2=0x7f8e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.738+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 0x7f8e2806edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e40008890 tx=0x7f8e40008040 comp rx=0 tx=0).stop 2024-04-13T03:09:48.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.738+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 msgr2=0x7f8e440cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.738+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009f70 tx=0x7f8e3c004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 shutdown_connections 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e2806c8f0 0x7f8e2806edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e4415e9c0 0x7f8e440cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e441624b0 0x7f8e440cda40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 --2- 172.21.15.106:0/1192666758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e440b68d0 0x7f8e44152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 >> 172.21.15.106:0/1192666758 conn(0x7f8e440d11c0 msgr2=0x7f8e440d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 shutdown_connections 2024-04-13T03:09:48.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.739+0000 7f8e4b488640 1 -- 172.21.15.106:0/1192666758 wait complete. 2024-04-13T03:09:48.750 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":3543688068},{"type":"v1","addr":"172.21.15.106:6833","nonce":3543688068}]},"active_addr":"172.21.15.106:6833/3543688068","active_change":"2024-04-13T03:09:04.353183+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2333101872}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":201875833}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3056843921}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3672671668}]}]} 2024-04-13T03:09:48.753 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-13T03:09:48.753 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-13T03:09:48.754 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:48.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.853+0000 7f2efd91e640 1 Processor -- start 2024-04-13T03:09:48.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.853+0000 7f2efd91e640 1 -- start start 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2efd91e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ef815e9b0 0x7f2ef80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2efd91e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ef80b3d60 0x7f2ef80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2efd91e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef80cfd20 con 0x7f2ef815ed90 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2efd91e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef8162af0 con 0x7f2ef80b3d60 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2efd91e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef8162dd0 con 0x7f2ef815e9b0 2024-04-13T03:09:48.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ef80b3d60 0x7f2ef80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ef80b3d60 0x7f2ef80d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44964/0 (socket says 172.21.15.106:44964) 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1921031895 learned_addr learned my addr 172.21.15.106:0/1921031895 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef6ffd640 1 --2- 172.21.15.106:0/1921031895 >> v2:172.21.15.114:3300/0 conn(0x7f2ef815e9b0 0x7f2ef80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1921031895 >> v2:172.21.15.114:3300/0 conn(0x7f2ef815e9b0 msgr2=0x7f2ef80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 --2- 172.21.15.106:0/1921031895 >> v2:172.21.15.114:3300/0 conn(0x7f2ef815e9b0 0x7f2ef80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1921031895 >> v1:172.21.15.114:6789/0 conn(0x7f2ef815ed90 legacy=0x7f2ef80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:48.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.854+0000 7f2ef77fe640 1 -- 172.21.15.106:0/1921031895 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef81630b0 con 0x7f2ef80b3d60 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.855+0000 7f2ef77fe640 1 --2- 172.21.15.106:0/1921031895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef80b3d60 0x7f2ef80d0200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ef808e4e0 tx=0x7f2ef002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e917e9b21d8364af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.855+0000 7f2ef5ffb640 1 -- 172.21.15.106:0/1921031895 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef002cb20 con 0x7f2ef80b3d60 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.855+0000 7f2ef5ffb640 1 -- 172.21.15.106:0/1921031895 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef002ccc0 con 0x7f2ef80b3d60 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.855+0000 7f2ef5ffb640 1 -- 172.21.15.106:0/1921031895 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0034610 con 0x7f2ef80b3d60 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 -- 172.21.15.106:0/1921031895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef80b3d60 msgr2=0x7f2ef80d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 --2- 172.21.15.106:0/1921031895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef80b3d60 0x7f2ef80d0200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ef808e4e0 tx=0x7f2ef002bf80 comp rx=0 tx=0).stop 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 -- 172.21.15.106:0/1921031895 shutdown_connections 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 --2- 172.21.15.106:0/1921031895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef80b3d60 0x7f2ef80d0200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 --2- 172.21.15.106:0/1921031895 >> v2:172.21.15.114:3300/0 conn(0x7f2ef815e9b0 0x7f2ef80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 -- 172.21.15.106:0/1921031895 >> 172.21.15.106:0/1921031895 conn(0x7f2ef808bf10 msgr2=0x7f2ef80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 -- 172.21.15.106:0/1921031895 shutdown_connections 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.856+0000 7f2efd91e640 1 -- 172.21.15.106:0/1921031895 wait complete. 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.857+0000 7f2efd91e640 1 Processor -- start 2024-04-13T03:09:48.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.857+0000 7f2efd91e640 1 -- start start 2024-04-13T03:09:48.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ef80b3d60 0x7f2ef81e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef815b210 0x7f2ef8160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8162800 con 0x7f2ef815e9b0 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8162680 con 0x7f2ef815b210 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef80d4d00 con 0x7f2ef80b3d60 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44980/0 (socket says 172.21.15.106:44980) 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 -- 172.21.15.106:0/981828669 learned_addr learned my addr 172.21.15.106:0/981828669 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 -- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ef80b3d60 msgr2=0x7f2ef81e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ef80b3d60 0x7f2ef81e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 -- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef815b210 msgr2=0x7f2ef8160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef815b210 0x7f2ef8160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 -- 172.21.15.106:0/981828669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef80b6910 con 0x7f2ef815e9b0 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2ef67fc640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c410 tx=0x7f2eec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec017070 con 0x7f2ef815e9b0 2024-04-13T03:09:48.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eec005230 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef81e8590 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0045f0 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.858+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef80b6bd0 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.859+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef8068a10 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.860+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2eec01a070 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.860+0000 7f2edffff640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 0x7f2ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.860+0000 7f2ef6ffd640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 0x7f2ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.860+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2eec08b1c0 con 0x7f2ef815e9b0 2024-04-13T03:09:48.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.861+0000 7f2ef6ffd640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 0x7f2ee406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098f0 tx=0x7f2ee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:48.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:48.865+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2eec0592b0 con 0x7f2ef815e9b0 2024-04-13T03:09:49.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.044+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ef815c1d0 con 0x7f2ef815e9b0 2024-04-13T03:09:49.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.045+0000 7f2edffff640 1 -- 172.21.15.106:0/981828669 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f2eec058bb0 con 0x7f2ef815e9b0 2024-04-13T03:09:49.047 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:49.047 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 msgr2=0x7f2ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 0x7f2ee406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098f0 tx=0x7f2ee8008040 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 msgr2=0x7f2ef8160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c410 tx=0x7f2eec00c8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 shutdown_connections 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ee406c8f0 0x7f2ee406edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef815b210 0x7f2ef8160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ef815e9b0 0x7f2ef8160770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 --2- 172.21.15.106:0/981828669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ef80b3d60 0x7f2ef81e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 >> 172.21.15.106:0/981828669 conn(0x7f2ef808bf10 msgr2=0x7f2ef80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 shutdown_connections 2024-04-13T03:09:49.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.048+0000 7f2efd91e640 1 -- 172.21.15.106:0/981828669 wait complete. 2024-04-13T03:09:49.060 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-13T03:09:49.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:49.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.159+0000 7fe85bc76640 1 Processor -- start 2024-04-13T03:09:49.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.159+0000 7fe85bc76640 1 -- start start 2024-04-13T03:09:49.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85bc76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe8540cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85bc76640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 0x7fe854071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85bc76640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe85415f420 con 0x7fe85406e8e0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85bc76640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe85406ecc0 con 0x7fe854070c80 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85bc76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe85415f1e0 con 0x7fe85415ed80 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859a12640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe85406e8e0 0x7fe8540cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56118/0 (socket says 172.21.15.106:56118) 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859a12640 1 -- 172.21.15.106:0/1571377325 learned_addr learned my addr 172.21.15.106:0/1571377325 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859211640 1 --2- 172.21.15.106:0/1571377325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe8540cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85a213640 1 --2- 172.21.15.106:0/1571377325 >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 0x7fe854071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709575819 0 0) 0x7fe85415f420 con 0x7fe85406e8e0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe840003520 con 0x7fe85406e8e0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859211640 1 -- 172.21.15.106:0/1571377325 >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 msgr2=0x7fe854071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 329473900 0 0) 0x7fe840003520 con 0x7fe85406e8e0 2024-04-13T03:09:49.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859211640 1 --2- 172.21.15.106:0/1571377325 >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 0x7fe854071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859211640 1 -- 172.21.15.106:0/1571377325 >> v1:172.21.15.106:6789/0 conn(0x7fe85406e8e0 legacy=0x7fe8540cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe859211640 1 -- 172.21.15.106:0/1571377325 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85415f700 con 0x7fe85415ed80 2024-04-13T03:09:49.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.160+0000 7fe85a213640 1 --2- 172.21.15.106:0/1571377325 >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 0x7fe854071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.161+0000 7fe859211640 1 --2- 172.21.15.106:0/1571377325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe8540cd770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe8440097a0 tx=0x7fe84402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120c5193054e21a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.161+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844037070 con 0x7fe85415ed80 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.161+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe844030510 con 0x7fe85415ed80 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.161+0000 7fe858a10640 1 -- 172.21.15.106:0/1571377325 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850001670 con 0x7fe85415ed80 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 -- 172.21.15.106:0/1571377325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 msgr2=0x7fe8540cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 --2- 172.21.15.106:0/1571377325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe8540cd770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe8440097a0 tx=0x7fe84402b790 comp rx=0 tx=0).stop 2024-04-13T03:09:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 -- 172.21.15.106:0/1571377325 shutdown_connections 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 --2- 172.21.15.106:0/1571377325 >> v2:172.21.15.114:3300/0 conn(0x7fe854070c80 0x7fe854071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 --2- 172.21.15.106:0/1571377325 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe8540cd770 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 -- 172.21.15.106:0/1571377325 >> 172.21.15.106:0/1571377325 conn(0x7fe85406f180 msgr2=0x7fe85406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 -- 172.21.15.106:0/1571377325 shutdown_connections 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.162+0000 7fe85bc76640 1 -- 172.21.15.106:0/1571377325 wait complete. 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.163+0000 7fe85bc76640 1 Processor -- start 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.163+0000 7fe85bc76640 1 -- start start 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe85406e8e0 0x7fe8541f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe854150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe85415fcc0 con 0x7fe854070c80 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540707f0 con 0x7fe85406e8e0 2024-04-13T03:09:49.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe8540cf790 con 0x7fe85415ed80 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44992/0 (socket says 172.21.15.106:44992) 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 -- 172.21.15.106:0/886830245 learned_addr learned my addr 172.21.15.106:0/886830245 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85a213640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe854150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859a12640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe85406e8e0 0x7fe8541f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 -- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 msgr2=0x7fe854150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe854150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 -- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe85406e8e0 msgr2=0x7fe8541f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe85406e8e0 0x7fe8541f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 -- 172.21.15.106:0/886830245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8541f59b0 con 0x7fe854070c80 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe859211640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe844035040 tx=0x7fe8440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844041030 con 0x7fe854070c80 2024-04-13T03:09:49.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84403c670 con 0x7fe854070c80 2024-04-13T03:09:49.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844037040 con 0x7fe854070c80 2024-04-13T03:09:49.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541e9630 con 0x7fe854070c80 2024-04-13T03:09:49.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.164+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541e9e70 con 0x7fe854070c80 2024-04-13T03:09:49.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.165+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854068a10 con 0x7fe854070c80 2024-04-13T03:09:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.166+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe844044070 con 0x7fe854070c80 2024-04-13T03:09:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.166+0000 7fe84a7fc640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.166+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8440ad5b0 con 0x7fe854070c80 2024-04-13T03:09:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.166+0000 7fe859a12640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.167+0000 7fe859a12640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 0x7fe83806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe8500017e0 tx=0x7fe850001570 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.171+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8541e9630 con 0x7fe854070c80 2024-04-13T03:09:49.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.345+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8540cdaf0 con 0x7fe854070c80 2024-04-13T03:09:49.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.346+0000 7fe84a7fc640 1 -- 172.21.15.106:0/886830245 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fe84407b6a0 con 0x7fe854070c80 2024-04-13T03:09:49.348 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:49.348 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 msgr2=0x7fe83806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 0x7fe83806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe8500017e0 tx=0x7fe850001570 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 msgr2=0x7fe8540b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe844035040 tx=0x7fe8440046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 shutdown_connections 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe83806c8f0 0x7fe83806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe85415ed80 0x7fe854150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe854070c80 0x7fe8540b6510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 --2- 172.21.15.106:0/886830245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe85406e8e0 0x7fe8541f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 >> 172.21.15.106:0/886830245 conn(0x7fe85406f180 msgr2=0x7fe85406caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 shutdown_connections 2024-04-13T03:09:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.348+0000 7fe85bc76640 1 -- 172.21.15.106:0/886830245 wait complete. 2024-04-13T03:09:49.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-13T03:09:49.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-13T03:09:49.362 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-13T03:09:49.362 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-13T03:09:49.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-13T03:09:49.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-13T03:09:49.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-13T03:09:49.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.531+0000 7f4b26108640 1 Processor -- start 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.531+0000 7f4b26108640 1 -- start start 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b26108640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b2006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b26108640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b200b6300 con 0x7f4b200aff30 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b26108640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b200b5d40 con 0x7f4b200c3350 2024-04-13T03:09:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b26108640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200b6020 con 0x7f4b200b50c0 2024-04-13T03:09:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b1ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4b200aff30 0x7f4b200d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56122/0 (socket says 172.21.15.106:56122) 2024-04-13T03:09:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b1ffff640 1 -- 172.21.15.106:0/3451387582 learned_addr learned my addr 172.21.15.106:0/3451387582 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.532+0000 7f4b1effd640 1 --2- 172.21.15.106:0/3451387582 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b2006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151605070 0 0) 0x7f4b200b5d40 con 0x7f4b200c3350 2024-04-13T03:09:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b08003520 con 0x7f4b200c3350 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358547597 0 0) 0x7f4b200b6300 con 0x7f4b200aff30 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b200b5d40 con 0x7f4b200aff30 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1effd640 1 -- 172.21.15.106:0/3451387582 >> v1:172.21.15.114:6789/0 conn(0x7f4b200c3350 legacy=0x7f4b200c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1effd640 1 -- 172.21.15.106:0/3451387582 >> v1:172.21.15.106:6789/0 conn(0x7f4b200aff30 legacy=0x7f4b200d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7f4b1effd640 1 -- 172.21.15.106:0/3451387582 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200d0220 con 0x7f4b200b50c0 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.533+0000 7fa09575e640 1 Processor -- start 2024-04-13T03:09:49.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.534+0000 7fa09575e640 1 -- start start 2024-04-13T03:09:49.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.534+0000 7f4ce724b640 1 Processor -- start 2024-04-13T03:09:49.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.534+0000 7f734597a640 1 Processor -- start 2024-04-13T03:09:49.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa09575e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa0880e30b0 0x7fa088057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.534+0000 7f4ce724b640 1 -- start start 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa09575e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa08803e830 0x7fa088053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa09575e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0880582c0 con 0x7fa088054120 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa09575e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa088054550 con 0x7fa0880e30b0 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa09575e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa088058560 con 0x7fa08803e830 2024-04-13T03:09:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7f4b1effd640 1 --2- 172.21.15.106:0/3451387582 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b2006b8e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b140097d0 tx=0x7f4b1402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218dff4cdcb09754 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa874fe9640 1 Processor -- start 2024-04-13T03:09:49.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa08f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa088054120 0x7fa0880e6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56136/0 (socket says 172.21.15.106:56136) 2024-04-13T03:09:49.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa08f7fe640 1 -- 172.21.15.106:0/1027937121 learned_addr learned my addr 172.21.15.106:0/1027937121 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7f210d62d640 1 Processor -- start 2024-04-13T03:09:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7f210d62d640 1 -- start start 2024-04-13T03:09:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/1027937121 >> v2:172.21.15.114:3300/0 conn(0x7fa08803e830 0x7fa088053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14034070 con 0x7f4b200b50c0 2024-04-13T03:09:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3451387582 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14030880 con 0x7f4b200b50c0 2024-04-13T03:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f210d62d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21000df830 0x7f210004fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.534+0000 7f734597a640 1 -- start start 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f734597a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f73400c71f0 0x7f73400ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f734597a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f73400cf050 0x7f73400cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f734597a640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73400cf460 con 0x7f73400afb90 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f734597a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f73400aff70 con 0x7f73400cf050 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f734597a640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f73400c5eb0 con 0x7f73400c71f0 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f7344978640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f73400afb90 0x7f7340163b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53796/0 (socket says 172.21.15.106:53796) 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f7344978640 1 -- 172.21.15.106:0/1730452122 learned_addr learned my addr 172.21.15.106:0/1730452122 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f4ce6249640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4ce015b420 0x7f4ce00cfdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56142/0 (socket says 172.21.15.106:56142) 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f7345179640 1 --2- 172.21.15.106:0/1730452122 >> v2:172.21.15.106:3300/0 conn(0x7f73400cf050 0x7f73400cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.535+0000 7fa874fe9640 1 -- start start 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa874fe9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa8700b19d0 0x7fa8700cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa874fe9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8700d0440 con 0x7fa87006ca00 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa874fe9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8700d0720 con 0x7fa870073c20 2024-04-13T03:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa874fe9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa8700d1050 con 0x7fa8700b19d0 2024-04-13T03:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f7345179640 1 -- 172.21.15.106:0/1730452122 >> v2:172.21.15.114:3300/0 conn(0x7f73400c71f0 msgr2=0x7f73400ceb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T03:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f7345179640 1 --2- 172.21.15.106:0/1730452122 >> v2:172.21.15.114:3300/0 conn(0x7f73400c71f0 0x7f73400ceb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f210d62d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2100006250 0x7f2100006680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f210d62d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2100006c60 con 0x7f2100055d40 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f210d62d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2100006f40 con 0x7f2100006250 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f210d62d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2100007220 con 0x7f21000df830 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f21077fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2100006250 0x7f2100006680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f21077fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2100006250 0x7f2100006680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53112/0 (socket says 172.21.15.106:53112) 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f21077fe640 1 -- 172.21.15.106:0/1925239257 learned_addr learned my addr 172.21.15.106:0/1925239257 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7f21067fc640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21000df830 0x7f210004fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f21077fe640 1 -- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21000df830 msgr2=0x7f210004fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f21077fe640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21000df830 0x7f210004fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f21077fe640 1 -- 172.21.15.106:0/1925239257 >> v1:172.21.15.106:6789/0 conn(0x7f2100055d40 legacy=0x7f21000366d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 -- 172.21.15.106:0/3451387582 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 msgr2=0x7f4b2006b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 --2- 172.21.15.106:0/3451387582 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b2006b8e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b140097d0 tx=0x7f4b1402b9c0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa86dd9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa87006ca00 0x7fa8701636e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56162/0 (socket says 172.21.15.106:56162) 2024-04-13T03:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa86dd9b640 1 -- 172.21.15.106:0/970829995 learned_addr learned my addr 172.21.15.106:0/970829995 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/970829995 >> v2:172.21.15.114:3300/0 conn(0x7fa8700b19d0 0x7fa8700cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7fa08effd640 1 --2- 172.21.15.106:0/1027937121 >> v2:172.21.15.106:3300/0 conn(0x7fa0880e30b0 0x7fa088057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402291892 0 0) 0x7fa0880582c0 con 0x7fa088054120 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.536+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa078003520 con 0x7fa088054120 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1168074398 0 0) 0x7fa078003520 con 0x7fa088054120 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 >> v2:172.21.15.114:3300/0 conn(0x7fa08803e830 msgr2=0x7fa088053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 --2- 172.21.15.106:0/1027937121 >> v2:172.21.15.114:3300/0 conn(0x7fa08803e830 0x7fa088053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 >> v2:172.21.15.106:3300/0 conn(0x7fa0880e30b0 msgr2=0x7fa088057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 --2- 172.21.15.106:0/1027937121 >> v2:172.21.15.106:3300/0 conn(0x7fa0880e30b0 0x7fa088057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088058840 con 0x7fa088054120 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa084001690 con 0x7fa088054120 2024-04-13T03:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 >> v1:172.21.15.106:6789/0 conn(0x7fa088054120 legacy=0x7fa0880e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 0x7fa078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa078004800 0x7fa078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780051c0 0x7fa07800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa088054550 con 0x7fa0780039b0 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa088058560 con 0x7fa0780051c0 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa07800c040 con 0x7fa078004800 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/1027937121 >> v2:172.21.15.114:3300/0 conn(0x7fa08803e830 0x7fa088053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08f7fe640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780051c0 0x7fa07800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 -- 172.21.15.106:0/3451387582 shutdown_connections 2024-04-13T03:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 --2- 172.21.15.106:0/3451387582 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b2006b8e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 -- 172.21.15.106:0/3451387582 >> 172.21.15.106:0/3451387582 conn(0x7f4b20073100 msgr2=0x7f4b20073510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 -- 172.21.15.106:0/3451387582 shutdown_connections 2024-04-13T03:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08effd640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 0x7fa078003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa078004800 0x7fa078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832429134 0 0) 0x7fa8700d0440 con 0x7fa87006ca00 2024-04-13T03:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa858003520 con 0x7fa87006ca00 2024-04-13T03:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f7345179640 1 -- 172.21.15.106:0/1730452122 >> v1:172.21.15.114:6789/0 conn(0x7f73400afb90 legacy=0x7f7340163b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2389227646 0 0) 0x7fa858003520 con 0x7fa87006ca00 2024-04-13T03:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08effd640 1 -- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa078004800 msgr2=0x7fa078004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08effd640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa078004800 0x7fa078004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08effd640 1 -- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780051c0 msgr2=0x7fa07800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08effd640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780051c0 0x7fa07800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08effd640 1 -- 172.21.15.106:0/1027937121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088058840 con 0x7fa0780039b0 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f7345179640 1 -- 172.21.15.106:0/1730452122 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73400c6150 con 0x7f73400cf050 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 >> v2:172.21.15.114:3300/0 conn(0x7fa8700b19d0 msgr2=0x7fa8700cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 --2- 172.21.15.106:0/970829995 >> v2:172.21.15.114:3300/0 conn(0x7fa8700b19d0 0x7fa8700cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 >> v1:172.21.15.114:6789/0 conn(0x7fa870073c20 legacy=0x7fa87006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8700d0a00 con 0x7fa87006ca00 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa860001670 con 0x7fa87006ca00 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 >> v1:172.21.15.106:6789/0 conn(0x7fa87006ca00 legacy=0x7fa8701636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa858003a00 0x7fa858003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa858004800 0x7fa858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 0x7fa85800fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8700d1050 con 0x7fa858003a00 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580079a0 con 0x7fa8580053c0 2024-04-13T03:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa858007ba0 con 0x7fa858004800 2024-04-13T03:09:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f5eef888640 1 Processor -- start 2024-04-13T03:09:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f5eef888640 1 -- start start 2024-04-13T03:09:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7fa08f7fe640 1 -- 172.21.15.106:0/1027937121 reap_dead start 2024-04-13T03:09:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7ffb158d5640 1 Processor -- start 2024-04-13T03:09:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f21077fe640 1 -- 172.21.15.106:0/1925239257 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2100007500 con 0x7f2100006250 2024-04-13T03:09:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7fa08effd640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 0x7fa078003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa090050270 tx=0x7fa078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a3a140e8d23c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f7345179640 1 --2- 172.21.15.106:0/1730452122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400cf050 0x7f73400cb350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f73400b3200 tx=0x7f733802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad8316783c7f442 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7ffb158d5640 1 -- start start 2024-04-13T03:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f21077fe640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f2100006680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f210806b120 tx=0x7f21080a7450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac87c513591ab04d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce6249640 1 -- 172.21.15.106:0/2967902259 learned_addr learned my addr 172.21.15.106:0/2967902259 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f733f7fe640 1 -- 172.21.15.106:0/1730452122 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338039070 con 0x7f73400cf050 2024-04-13T03:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f733f7fe640 1 -- 172.21.15.106:0/1730452122 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733802cc00 con 0x7f73400cf050 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f733f7fe640 1 -- 172.21.15.106:0/1730452122 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338034440 con 0x7f73400cf050 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce724b640 1 --2- 172.21.15.106:0/2967902259 >> v2:172.21.15.114:3300/0 conn(0x7f4ce00d0940 0x7f4ce00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce724b640 1 --2- 172.21.15.106:0/2967902259 >> v2:172.21.15.106:3300/0 conn(0x7f4ce006f400 0x7f4ce006d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce006f890 con 0x7f4ce015b420 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce006fb70 con 0x7f4ce006f400 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce00cf960 con 0x7f4ce00d0940 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510992782 0 0) 0x7f4ce006f890 con 0x7f4ce015b420 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ccc003520 con 0x7f4ce015b420 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4086873521 0 0) 0x7f4ccc003520 con 0x7f4ce015b420 2024-04-13T03:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 >> v2:172.21.15.114:3300/0 conn(0x7f4ce00d0940 msgr2=0x7f4ce00d0d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T03:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900ad070 con 0x7fa0780039b0 2024-04-13T03:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 --2- 172.21.15.106:0/2967902259 >> v2:172.21.15.114:3300/0 conn(0x7f4ce00d0940 0x7f4ce00d0d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 >> v2:172.21.15.106:3300/0 conn(0x7f4ce006f400 msgr2=0x7f4ce006d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 --2- 172.21.15.106:0/2967902259 >> v2:172.21.15.106:3300/0 conn(0x7f4ce006f400 0x7f4ce006d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce00cf310 con 0x7f4ce015b420 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4cdc001670 con 0x7f4ce015b420 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.538+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 >> v1:172.21.15.106:6789/0 conn(0x7f4ce015b420 legacy=0x7f4ce00cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ccc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ccc004800 0x7f4ccc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ccc0053c0 0x7f4ccc00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ccc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00cf960 con 0x7f4ccc003a00 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc00c2e0 con 0x7f4ccc0053c0 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc00c4e0 con 0x7f4ccc004800 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 -- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ccc004800 msgr2=0x7f4ccc004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ccc004800 0x7f4ccc004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0900b1400 con 0x7fa0780039b0 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7fa08dffb640 1 -- 172.21.15.106:0/1027937121 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090050c40 con 0x7fa0780039b0 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 -- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ccc0053c0 msgr2=0x7f4ccc00ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ccc0053c0 0x7f4ccc00ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6249640 1 -- 172.21.15.106:0/2967902259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc003520 con 0x7f4ccc003a00 2024-04-13T03:09:49.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4ce6a4a640 1 -- 172.21.15.106:0/2967902259 reap_dead start 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86e59c640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa858003a00 0x7fa858003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f5eef888640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ee8073a10 0x7f5ee80ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f5eef888640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee80b01d0 con 0x7f5ee8072f20 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f5eef888640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee80b04b0 con 0x7f5ee80b5f10 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f5eef888640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee80ad0f0 con 0x7f5ee8073a10 2024-04-13T03:09:49.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f5eece23640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5ee80b5f10 0x7f5ee80adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53810/0 (socket says 172.21.15.106:53810) 2024-04-13T03:09:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5eece23640 1 -- 172.21.15.106:0/1743764944 learned_addr learned my addr 172.21.15.106:0/1743764944 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5eede25640 1 --2- 172.21.15.106:0/1743764944 >> v2:172.21.15.106:3300/0 conn(0x7f5ee8073a10 0x7f5ee80ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 0x7fa85800fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.537+0000 7f4b26108640 1 -- 172.21.15.106:0/3451387582 wait complete. 2024-04-13T03:09:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4b26108640 1 Processor -- start 2024-04-13T03:09:49.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.539+0000 7f4b26108640 1 -- start start 2024-04-13T03:09:49.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb158d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb10160910 0x7ffb10163330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5edffff640 1 -- 172.21.15.106:0/1743764944 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024015137 0 0) 0x7f5ee80b04b0 con 0x7f5ee80b5f10 2024-04-13T03:09:49.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5edffff640 1 -- 172.21.15.106:0/1743764944 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ed4003440 con 0x7f5ee80b5f10 2024-04-13T03:09:49.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb158d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100cc540 0x7ffb10072db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb158d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb100ccc00 con 0x7ffb100732f0 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb0effd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb10160910 0x7ffb10163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb0effd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb10160910 0x7ffb10163330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53166/0 (socket says 172.21.15.106:53166) 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb158d5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ffb100cc9c0 con 0x7ffb10160910 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb158d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffb10073740 con 0x7ffb100cc540 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb0effd640 1 -- 172.21.15.106:0/2054542901 learned_addr learned my addr 172.21.15.106:0/2054542901 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5eede25640 1 -- 172.21.15.106:0/1743764944 >> v1:172.21.15.114:6789/0 conn(0x7f5ee80b5f10 legacy=0x7f5ee80adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5eede25640 1 -- 172.21.15.106:0/1743764944 >> v1:172.21.15.106:6789/0 conn(0x7f5ee8072f20 legacy=0x7f5ee8073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f5eede25640 1 -- 172.21.15.106:0/1743764944 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee80ad3d0 con 0x7f5ee8073a10 2024-04-13T03:09:49.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86dd9b640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa858004800 0x7fa858004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f2105ffb640 1 -- 172.21.15.106:0/1925239257 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080b3070 con 0x7f2100006250 2024-04-13T03:09:49.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f2105ffb640 1 -- 172.21.15.106:0/1925239257 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21080ac7d0 con 0x7f2100006250 2024-04-13T03:09:49.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f2105ffb640 1 -- 172.21.15.106:0/1925239257 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080acc00 con 0x7f2100006250 2024-04-13T03:09:49.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7f4ce6249640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ccc003e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc007ca0 tx=0x7f4ccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be93d382bc05c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa09575e640 1 -- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 msgr2=0x7fa078003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa09575e640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 0x7fa078003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa090050270 tx=0x7fa078005830 comp rx=0 tx=0).stop 2024-04-13T03:09:49.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7ffb0e7fc640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100cc540 0x7ffb10072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa09575e640 1 -- 172.21.15.106:0/1027937121 shutdown_connections 2024-04-13T03:09:49.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa09575e640 1 --2- 172.21.15.106:0/1027937121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0780039b0 0x7fa078003dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa09575e640 1 -- 172.21.15.106:0/1027937121 >> 172.21.15.106:0/1027937121 conn(0x7fa0880559d0 msgr2=0x7fa088055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 0x7f4b2016cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b201607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7ffb0effd640 1 -- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100cc540 msgr2=0x7ffb10072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100cc540 0x7ffb10072db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7ffb0effd640 1 -- 172.21.15.106:0/2054542901 >> v1:172.21.15.106:6789/0 conn(0x7ffb100732f0 legacy=0x7ffb10166920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7ffb0effd640 1 -- 172.21.15.106:0/2054542901 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb100ccee0 con 0x7ffb10160910 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b200c3350 0x7f4b2006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 -- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa858004800 msgr2=0x7fa858004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa858004800 0x7fa858004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 -- 172.21.15.106:0/970829995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa858003a00 msgr2=0x7fa858003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa858003a00 0x7fa858003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.540+0000 7fa86ed9d640 1 -- 172.21.15.106:0/970829995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858003520 con 0x7fa8580053c0 2024-04-13T03:09:49.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 0x7f4b2016cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 0x7f4b2016cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53168/0 (socket says 172.21.15.106:53168) 2024-04-13T03:09:49.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b1f7fe640 1 -- 172.21.15.106:0/3495694086 learned_addr learned my addr 172.21.15.106:0/3495694086 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b0540 con 0x7f4b200c3350 2024-04-13T03:09:49.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b03c0 con 0x7f4b200aff30 2024-04-13T03:09:49.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200d0e50 con 0x7f4b200b50c0 2024-04-13T03:09:49.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f734597a640 1 -- 172.21.15.106:0/1730452122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400cf050 msgr2=0x7f73400cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f734597a640 1 --2- 172.21.15.106:0/1730452122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400cf050 0x7f73400cb350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f73400b3200 tx=0x7f733802be50 comp rx=0 tx=0).stop 2024-04-13T03:09:49.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa86e59c640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa858003a00 0x7fa858003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa86ed9d640 1 -- 172.21.15.106:0/970829995 reap_dead start 2024-04-13T03:09:49.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f5eede25640 1 --2- 172.21.15.106:0/1743764944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8073a10 0x7f5ee80ae5c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80090e0 tx=0x7f5ed802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd8af770e6e5444 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa09575e640 1 -- 172.21.15.106:0/1027937121 shutdown_connections 2024-04-13T03:09:49.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa09575e640 1 -- 172.21.15.106:0/1027937121 wait complete. 2024-04-13T03:09:49.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f210d62d640 1 -- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 msgr2=0x7f2100006680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f210d62d640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f2100006680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f210806b120 tx=0x7f21080a7450 comp rx=0 tx=0).stop 2024-04-13T03:09:49.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4b1ffff640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b200c3350 0x7f4b2006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 0x7fa85800fe10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a9a0 tx=0x7fa858005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d176f9676e0775c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 -- 172.21.15.106:0/1925239257 shutdown_connections 2024-04-13T03:09:49.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f2100006680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 --2- 172.21.15.106:0/1925239257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21000df830 0x7f210004fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 -- 172.21.15.106:0/1925239257 >> 172.21.15.106:0/1925239257 conn(0x7f210001e6f0 msgr2=0x7f2100055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 -- 172.21.15.106:0/3495694086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 msgr2=0x7f4b201607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b200b50c0 0x7f4b201607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 -- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b200c3350 msgr2=0x7f4b2006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b200c3350 0x7f4b2006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200c6090 con 0x7f4b200aff30 2024-04-13T03:09:49.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864034070 con 0x7fa8580053c0 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 -- 172.21.15.106:0/1730452122 shutdown_connections 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 --2- 172.21.15.106:0/1730452122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400cf050 0x7f73400cb350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 --2- 172.21.15.106:0/1730452122 >> v2:172.21.15.114:3300/0 conn(0x7f73400c71f0 0x7f73400ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 -- 172.21.15.106:0/1730452122 >> 172.21.15.106:0/1730452122 conn(0x7f73400ccfe0 msgr2=0x7f73400cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa864030650 con 0x7fa8580053c0 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007ce0 con 0x7f4ccc003a00 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cdc052070 con 0x7f4ccc003a00 2024-04-13T03:09:49.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.541+0000 7f4ce5247640 1 -- 172.21.15.106:0/2967902259 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005dd0 con 0x7f4ccc003a00 2024-04-13T03:09:49.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 -- 172.21.15.106:0/1730452122 shutdown_connections 2024-04-13T03:09:49.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f5edffff640 1 -- 172.21.15.106:0/1743764944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8035020 con 0x7f5ee8073a10 2024-04-13T03:09:49.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f5edffff640 1 -- 172.21.15.106:0/1743764944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8031830 con 0x7f5ee8073a10 2024-04-13T03:09:49.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b1f7fe640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 0x7f4b2016cd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c7a0 tx=0x7f4b1800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7fa86d59a640 1 -- 172.21.15.106:0/970829995 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680040a0 con 0x7fa8580053c0 2024-04-13T03:09:49.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18003b70 con 0x7f4b200aff30 2024-04-13T03:09:49.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 msgr2=0x7f4ccc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4ce724b640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ccc003e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc007ca0 tx=0x7f4ccc005830 comp rx=0 tx=0).stop 2024-04-13T03:09:49.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18003d10 con 0x7f4b200aff30 2024-04-13T03:09:49.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb10160910 0x7ffb10163330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4dc4cf543134a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 -- 172.21.15.106:0/1925239257 shutdown_connections 2024-04-13T03:09:49.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f210d62d640 1 -- 172.21.15.106:0/1925239257 wait complete. 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800a9f0 con 0x7f4b200aff30 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b200c6370 con 0x7f4b200aff30 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b200c5d40 con 0x7f4b200aff30 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa874fe9640 1 -- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 msgr2=0x7fa85800fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa874fe9640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 0x7fa85800fe10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a9a0 tx=0x7fa858005830 comp rx=0 tx=0).stop 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa874fe9640 1 -- 172.21.15.106:0/970829995 shutdown_connections 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa874fe9640 1 --2- 172.21.15.106:0/970829995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8580053c0 0x7fa85800fe10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa874fe9640 1 -- 172.21.15.106:0/970829995 >> 172.21.15.106:0/970829995 conn(0x7fa870073100 msgr2=0x7fa870073510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 shutdown_connections 2024-04-13T03:09:49.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4ce724b640 1 --2- 172.21.15.106:0/2967902259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ccc003e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 >> 172.21.15.106:0/2967902259 conn(0x7f4ce0070550 msgr2=0x7f4ce0070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5edffff640 1 -- 172.21.15.106:0/1743764944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8031c00 con 0x7f5ee8073a10 2024-04-13T03:09:49.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 shutdown_connections 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 -- 172.21.15.106:0/1743764944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8073a10 msgr2=0x7f5ee80ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 --2- 172.21.15.106:0/1743764944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8073a10 0x7f5ee80ae5c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80090e0 tx=0x7f5ed802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 -- 172.21.15.106:0/1743764944 shutdown_connections 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 --2- 172.21.15.106:0/1743764944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8073a10 0x7f5ee80ae5c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 -- 172.21.15.106:0/1743764944 >> 172.21.15.106:0/1743764944 conn(0x7f5ee80b4f70 msgr2=0x7f5ee80b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 -- 172.21.15.106:0/1743764944 shutdown_connections 2024-04-13T03:09:49.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f5eef888640 1 -- 172.21.15.106:0/1743764944 wait complete. 2024-04-13T03:09:49.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7fa09575e640 1 Processor -- start 2024-04-13T03:09:49.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.543+0000 7f4ce724b640 1 -- 172.21.15.106:0/2967902259 wait complete. 2024-04-13T03:09:49.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.542+0000 7f734597a640 1 -- 172.21.15.106:0/1730452122 wait complete. 2024-04-13T03:09:49.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7fa874fe9640 1 -- 172.21.15.106:0/970829995 shutdown_connections 2024-04-13T03:09:49.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7fa09575e640 1 -- start start 2024-04-13T03:09:49.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b00001030 con 0x7f4b200aff30 2024-04-13T03:09:49.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7fa874fe9640 1 -- 172.21.15.106:0/970829995 wait complete. 2024-04-13T03:09:49.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7f734597a640 1 Processor -- start 2024-04-13T03:09:49.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7ffb0dffb640 1 -- 172.21.15.106:0/2054542901 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb000317c0 con 0x7ffb10160910 2024-04-13T03:09:49.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7ffb0dffb640 1 -- 172.21.15.106:0/2054542901 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb00037070 con 0x7ffb10160910 2024-04-13T03:09:49.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7ffb0dffb640 1 -- 172.21.15.106:0/2054542901 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00031b80 con 0x7ffb10160910 2024-04-13T03:09:49.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7f210d62d640 1 Processor -- start 2024-04-13T03:09:49.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.544+0000 7f210d62d640 1 -- start start 2024-04-13T03:09:49.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f734597a640 1 -- start start 2024-04-13T03:09:49.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa09575e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780039b0 0x7fa0880509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa09575e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0880e6050 0x7fa08804f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa09575e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 0x7fa088052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f21000e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 0x7f21000d4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21000df830 0x7f21000544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f210004ffb0 con 0x7f21000df830 2024-04-13T03:09:49.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa09575e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa08803bfb0 con 0x7fa0880e6050 2024-04-13T03:09:49.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f5eef888640 1 Processor -- start 2024-04-13T03:09:49.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2100056120 con 0x7f2100006250 2024-04-13T03:09:49.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f210d62d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2100050130 con 0x7f2100055d40 2024-04-13T03:09:49.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f4ce724b640 1 Processor -- start 2024-04-13T03:09:49.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa08f7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 0x7fa088052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa08f7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 0x7fa088052490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49634/0 (socket says 172.21.15.106:49634) 2024-04-13T03:09:49.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa08f7fe640 1 -- 172.21.15.106:0/986163282 learned_addr learned my addr 172.21.15.106:0/986163282 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa0880063a0 con 0x7fa0780039b0 2024-04-13T03:09:49.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f5eef888640 1 -- start start 2024-04-13T03:09:49.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f4ce724b640 1 -- start start 2024-04-13T03:09:49.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7ffb158d5640 1 -- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb10160910 msgr2=0x7ffb10163330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa0880e3490 con 0x7fa088052050 2024-04-13T03:09:49.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7ffb158d5640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb10160910 0x7ffb10163330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0002b860 comp rx=0 tx=0).stop 2024-04-13T03:09:49.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f21067fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 0x7f21000d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08effd640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780039b0 0x7fa0880509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f21067fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 0x7f21000d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49638/0 (socket says 172.21.15.106:49638) 2024-04-13T03:09:49.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa874fe9640 1 Processor -- start 2024-04-13T03:09:49.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.545+0000 7f21067fc640 1 -- 172.21.15.106:0/2830037091 learned_addr learned my addr 172.21.15.106:0/2830037091 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f21077fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21000df830 0x7f21000544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0880e6050 0x7fa08804f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73400afb90 0x7f7340151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 0x7f73400b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73400cf050 0x7f734014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73400c69f0 con 0x7f73400c71f0 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f73400c6710 con 0x7f73400cf050 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f734597a640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f73400c6cd0 con 0x7f73400afb90 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f733ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 0x7f73400b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f733ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 0x7f73400b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45084/0 (socket says 172.21.15.106:45084) 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f733ffff640 1 -- 172.21.15.106:0/4027026633 learned_addr learned my addr 172.21.15.106:0/4027026633 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08f7fe640 1 -- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780039b0 msgr2=0x7fa0880509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08f7fe640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780039b0 0x7fa0880509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08f7fe640 1 -- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0880e6050 msgr2=0x7fa08804f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08f7fe640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0880e6050 0x7fa08804f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa08f7fe640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088051030 con 0x7fa088052050 2024-04-13T03:09:49.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b1800ab90 con 0x7f4b200aff30 2024-04-13T03:09:49.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7f2106ffd640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f21000e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f7345179640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73400cf050 0x7f734014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 -- 172.21.15.106:0/2054542901 shutdown_connections 2024-04-13T03:09:49.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100cc540 0x7ffb10072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 --2- 172.21.15.106:0/2054542901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb10160910 0x7ffb10163330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 -- 172.21.15.106:0/2054542901 >> 172.21.15.106:0/2054542901 conn(0x7ffb100cdef0 msgr2=0x7ffb100ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f7344978640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73400afb90 0x7f7340151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 -- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 msgr2=0x7f21000e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100006250 0x7f21000e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 -- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21000df830 msgr2=0x7f21000544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce724b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ce0147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 -- 172.21.15.106:0/2054542901 shutdown_connections 2024-04-13T03:09:49.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa08f7fe640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 0x7fa088052490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa088011e10 tx=0x7fa084007660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084007790 con 0x7fa088052050 2024-04-13T03:09:49.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa084018070 con 0x7fa088052050 2024-04-13T03:09:49.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08400fb10 con 0x7fa088052050 2024-04-13T03:09:49.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f733ffff640 1 -- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73400afb90 msgr2=0x7f7340151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f733ffff640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73400afb90 0x7f7340151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa088051310 con 0x7fa088052050 2024-04-13T03:09:49.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f733ffff640 1 -- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73400cf050 msgr2=0x7f734014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f733ffff640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73400cf050 0x7f734014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f5eef888640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 0x7f5ee8201e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce724b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ce0148c90 0x7f4ce013c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21000df830 0x7f21000544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21000e01b0 con 0x7f2100055d40 2024-04-13T03:09:49.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21077fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21000df830 0x7f21000544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f733ffff640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7340151ef0 con 0x7f73400c71f0 2024-04-13T03:09:49.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0880e64d0 con 0x7fa088052050 2024-04-13T03:09:49.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f5eef888640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5ee8073a10 0x7f5ee8203a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce724b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ce00b4a90 0x7f4ce00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f21067fc640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 0x7f21000d4230 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e8a0 tx=0x7f20f800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.546+0000 7fa874fe9640 1 -- start start 2024-04-13T03:09:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0880e6050 0x7fa08804f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce5a48640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ce0148c90 0x7f4ce013c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce5a48640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ce0148c90 0x7f4ce013c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49686/0 (socket says 172.21.15.106:49686) 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8580053c0 0x7fa870099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa87009a300 0x7fa87009a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa87009cce0 0x7fa87009d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8700cdf50 con 0x7fa87009cce0 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa8700d0010 con 0x7fa87009a300 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa874fe9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa8700ce0d0 con 0x7fa8580053c0 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa86e59c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8580053c0 0x7fa870099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa86e59c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8580053c0 0x7fa870099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49670/0 (socket says 172.21.15.106:49670) 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7fa86e59c640 1 -- 172.21.15.106:0/1267236520 learned_addr learned my addr 172.21.15.106:0/1267236520 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f5eef888640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ee80b5f10 0x7f5ee806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce5a48640 1 -- 172.21.15.106:0/785602560 learned_addr learned my addr 172.21.15.106:0/785602560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce6a4a640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ce00b4a90 0x7f4ce00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00d0d90 con 0x7f4ccc003a00 2024-04-13T03:09:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f5eece23640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5ee8073a10 0x7f5ee8203a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eef888640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee80afd80 con 0x7f5ee8072f20 2024-04-13T03:09:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa87009cce0 0x7fa87009d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86dd9b640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa87009a300 0x7fa87009a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00cefc0 con 0x7f4ce00b4a90 2024-04-13T03:09:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.547+0000 7ffb158d5640 1 -- 172.21.15.106:0/2054542901 wait complete. 2024-04-13T03:09:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce6249640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ce0147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f733ffff640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 0x7f73400b4d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f733400b450 tx=0x7f733400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce00d0f10 con 0x7f4ce0148c90 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce6249640 1 -- 172.21.15.106:0/785602560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ce0148c90 msgr2=0x7f4ce013c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce6249640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ce0148c90 0x7f4ce013c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce6249640 1 -- 172.21.15.106:0/785602560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ce00b4a90 msgr2=0x7f4ce00b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4ce6249640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ce00b4a90 0x7f4ce00b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4ce6249640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce013c6e0 con 0x7f4ccc003a00 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4b07fff640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c9c0 0x7f4b0c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4b1808bc80 con 0x7f4b200aff30 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4b07fff640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f4b0c076140 0x7f4b0c078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b1801ae50 con 0x7f4b0c076140 2024-04-13T03:09:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b1808e2e0 con 0x7f4b200aff30 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4b1effd640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c9c0 0x7f4b0c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4b1ffff640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f4b0c076140 0x7f4b0c078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 -- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8580053c0 msgr2=0x7fa870099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8580053c0 0x7fa870099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 -- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa87009a300 msgr2=0x7fa87009a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa87009a300 0x7fa87009a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7fa86ed9d640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87009d6a0 con 0x7fa87009cce0 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8017070 con 0x7f2100055d40 2024-04-13T03:09:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f800c940 con 0x7f2100055d40 2024-04-13T03:09:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800cd30 con 0x7f2100055d40 2024-04-13T03:09:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21000e2de0 con 0x7f2100055d40 2024-04-13T03:09:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f210004d210 con 0x7f2100055d40 2024-04-13T03:09:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7fa86ed9d640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa87009cce0 0x7fa87009d160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa870074160 tx=0x7fa864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0880529d0 con 0x7fa088052050 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eef888640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee80ad990 con 0x7f5ee80b5f10 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eef888640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee80aff00 con 0x7f5ee8073a10 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eed624640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 0x7f5ee8201e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eed624640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 0x7f5ee8201e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45102/0 (socket says 172.21.15.106:45102) 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eed624640 1 -- 172.21.15.106:0/3790415082 learned_addr learned my addr 172.21.15.106:0/3790415082 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 -- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5ee8073a10 msgr2=0x7f5ee8203a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5ee8073a10 0x7f5ee8203a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 -- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ee80b5f10 msgr2=0x7f5ee806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4ce6249640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ce0147d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc053000 tx=0x7f4ce00cf4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.548+0000 7f5eede25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ee80b5f10 0x7f5ee806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ee80b5f10 0x7f5ee806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee8203fa0 con 0x7f5ee8072f20 2024-04-13T03:09:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4b1ffff640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f4b0c076140 0x7f4b0c078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eede25640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ee80b5f10 0x7f5ee806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eece23640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5ee8073a10 0x7f5ee8203a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f4b1effd640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c9c0 0x7f4b0c06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b14036000 tx=0x7f4b14030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f5eed624640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 0x7f5ee8201e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b3f0 tx=0x7f5ee400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.549+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334015020 con 0x7f73400c71f0 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340118b0 con 0x7f73400c71f0 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334011ca0 con 0x7f73400c71f0 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc064070 con 0x7f4ccc003a00 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0049f0 con 0x7f4ccc003a00 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7ffb158d5640 1 Processor -- start 2024-04-13T03:09:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2100005810 con 0x7f2100055d40 2024-04-13T03:09:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa08400fcb0 con 0x7fa088052050 2024-04-13T03:09:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7ffb158d5640 1 -- start start 2024-04-13T03:09:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f734597a640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73400b4230 con 0x7f73400c71f0 2024-04-13T03:09:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7f734597a640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73400b47f0 con 0x7f73400c71f0 2024-04-13T03:09:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4015020 con 0x7f5ee8072f20 2024-04-13T03:09:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee4011830 con 0x7f5ee8072f20 2024-04-13T03:09:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4011c20 con 0x7f5ee8072f20 2024-04-13T03:09:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20f801a070 con 0x7f2100055d40 2024-04-13T03:09:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa0777fe640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa07c06c9c0 0x7fa07c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f734597a640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f734004fd50 con 0x7f73400c71f0 2024-04-13T03:09:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee808def0 con 0x7f5ee8072f20 2024-04-13T03:09:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f20ef7fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20f406c970 0x7f20f406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa08408bde0 con 0x7fa088052050 2024-04-13T03:09:49.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee808e4b0 con 0x7f5ee8072f20 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004dc0 con 0x7f4ccc003a00 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce013b580 con 0x7f4ccc003a00 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f20f808b5b0 con 0x7f2100055d40 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f20ef7fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f20f4076060 0x7f20f40784a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20f800be50 con 0x7f20f4076060 2024-04-13T03:09:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20f8092050 con 0x7f2100055d40 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f2106ffd640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20f406c970 0x7f20f406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864048070 con 0x7fa87009cce0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.550+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa864030450 con 0x7fa87009cce0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864034040 con 0x7fa87009cce0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87009d960 con 0x7fa87009cce0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87009df20 con 0x7fa87009cce0 2024-04-13T03:09:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100732f0 0x7ffb101621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb100cc540 0x7ffb101532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb10160910 0x7ffb101537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f21077fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f20f4076060 0x7f20f40784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100cd4a0 con 0x7ffb10160910 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa0777fe640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7fa07c0760c0 0x7fa07c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.551+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa08400f020 con 0x7fa07c0760c0 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7fa08effd640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa07c06c9c0 0x7fa07c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7fa07c0760c0 0x7fa07c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7fa08effd640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa07c06c9c0 0x7fa07c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bf90 tx=0x7fa0900ad000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce01472b0 con 0x7f4ccc003a00 2024-04-13T03:09:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b1801ae50 con 0x7f4b0c076140 2024-04-13T03:09:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7fa85cff9640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa848001030 con 0x7fa87009cce0 2024-04-13T03:09:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb0e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb100cc540 0x7ffb101532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7f2106ffd640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20f406c970 0x7f20f406ee30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001640 tx=0x7f20fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa084055ae0 con 0x7fa088052050 2024-04-13T03:09:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb0e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb100cc540 0x7ffb101532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53212/0 (socket says 172.21.15.106:53212) 2024-04-13T03:09:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb0e7fc640 1 -- 172.21.15.106:0/2972946309 learned_addr learned my addr 172.21.15.106:0/2972946309 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100cd810 con 0x7ffb100cc540 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.552+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffb10160cf0 con 0x7ffb100732f0 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb10160910 0x7ffb101537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100732f0 0x7ffb101621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7f21077fe640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f20f4076060 0x7f20f40784a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 -- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100732f0 msgr2=0x7ffb101621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100732f0 0x7ffb101621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 -- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb100cc540 msgr2=0x7ffb101532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb100cc540 0x7ffb101532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0f7fe640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101654e0 con 0x7ffb10160910 2024-04-13T03:09:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ee804fd50 con 0x7f5ee8072f20 2024-04-13T03:09:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7fa08e7fc640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7fa07c0760c0 0x7fa07c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7334018020 con 0x7f73400c71f0 2024-04-13T03:09:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20f800be50 con 0x7f20f4076060 2024-04-13T03:09:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ce008aec0 con 0x7f4ccc003a00 2024-04-13T03:09:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f733d7fa640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f732c06c940 0x7f732c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa08400f020 con 0x7fa07c0760c0 2024-04-13T03:09:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f733408c400 con 0x7f73400c71f0 2024-04-13T03:09:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ee4018020 con 0x7f5ee8072f20 2024-04-13T03:09:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7f5eddffb640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ecc06c940 0x7f5ecc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f733d7fa640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7f732c076040 0x7f732c078480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f732c078b50 con 0x7f732c076040 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.554+0000 7f7344978640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f732c06c940 0x7f732c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7f7344978640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f732c06c940 0x7f732c06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7330007c90 tx=0x7f7330007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.553+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb100732f0 0x7ffb101621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f7345179640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7f732c076040 0x7f732c078480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5ee40574b0 con 0x7f5ee8072f20 2024-04-13T03:09:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7f5eece23640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ecc06c940 0x7f5ecc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7ffb0f7fe640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb10160910 0x7ffb101537e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c780 tx=0x7ffb0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73340562b0 con 0x7f73400c71f0 2024-04-13T03:09:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f5eece23640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ecc06c940 0x7f5ecc06ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0000b60 tx=0x7f5ee0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.555+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa86404b070 con 0x7fa87009cce0 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa85effd640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa85006c9c0 0x7fa85006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8640ae390 con 0x7fa87009cce0 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa85effd640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7fa850076130 0x7fa850078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa864009fa0 con 0x7fa850076130 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8640ae780 con 0x7fa87009cce0 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08016020 con 0x7ffb10160910 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0800aa50 con 0x7ffb10160910 2024-04-13T03:09:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800ce30 con 0x7ffb10160910 2024-04-13T03:09:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cdc007070 con 0x7f4ccc003a00 2024-04-13T03:09:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f5eddffb640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7f5ecc076d10 0x7f5ecc0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ecc079880 con 0x7f5ecc076d10 2024-04-13T03:09:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ee4055e40 con 0x7f5ee8072f20 2024-04-13T03:09:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7f7345179640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7f732c076040 0x7f732c078480 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f5eede25640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7f5ecc076d10 0x7f5ecc0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f4cd6ffd640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cc406c9c0 0x7f4cc406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.556+0000 7fa86e59c640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa85006c9c0 0x7fa85006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7fa86e59c640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa85006c9c0 0x7fa85006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8680057c0 tx=0x7fa868003c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb101657c0 con 0x7ffb10160910 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb10165d80 con 0x7ffb10160910 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4cdc0cfdb0 con 0x7f4ccc003a00 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f4cd6ffd640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f4cc4076140 0x7f4cc4078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cdc05ae50 con 0x7f4cc4076140 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.557+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc0d01a0 con 0x7f4ccc003a00 2024-04-13T03:09:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c078b50 con 0x7f732c076040 2024-04-13T03:09:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f5eede25640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7f5ecc076d10 0x7f5ecc0791b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7fa86dd9b640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7fa850076130 0x7fa850078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f4ce5a48640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cc406c9c0 0x7f4cc406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f4ce6a4a640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f4cc4076140 0x7f4cc4078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f4ce6a4a640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f4cc4076140 0x7f4cc4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.558+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ecc079880 con 0x7f5ecc076d10 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.559+0000 7f4ce5a48640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cc406c9c0 0x7f4cc406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ce01467e0 tx=0x7f4cd0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.559+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb100c8340 con 0x7ffb10160910 2024-04-13T03:09:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.559+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cdc05ae50 con 0x7f4cc4076140 2024-04-13T03:09:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.560+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb08019020 con 0x7ffb10160910 2024-04-13T03:09:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.560+0000 7ffaf7fff640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffafc06c9c0 0x7ffafc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.560+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb0808bdb0 con 0x7ffb10160910 2024-04-13T03:09:49.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.560+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffafc06c9c0 0x7ffafc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.560+0000 7fa86dd9b640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7fa850076130 0x7fa850078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7ffaf7fff640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7ffafc076140 0x7ffafc078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7ffb0effd640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffafc06c9c0 0x7ffafc06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffb00036000 tx=0x7ffb0003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffafc078c50 con 0x7ffafc076140 2024-04-13T03:09:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb080913c0 con 0x7ffb10160910 2024-04-13T03:09:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa864009fa0 con 0x7fa850076130 2024-04-13T03:09:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.561+0000 7ffb0e7fc640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7ffafc076140 0x7ffafc078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.562+0000 7ffb0e7fc640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7ffafc076140 0x7ffafc078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.563+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffafc078c50 con 0x7ffafc076140 2024-04-13T03:09:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.580+0000 7f4b26108640 1 -- 172.21.15.106:0/3495694086 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b00002dd0 con 0x7f4b0c076140 2024-04-13T03:09:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.581+0000 7f4b07fff640 1 -- 172.21.15.106:0/3495694086 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4b00002dd0 con 0x7f4b0c076140 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.581+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f4b0c076140 msgr2=0x7f4b0c078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.581+0000 7f4b05ffb640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f4b0c076140 0x7f4b0c078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c9c0 msgr2=0x7f4b0c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c9c0 0x7f4b0c06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b14036000 tx=0x7f4b14030000 comp rx=0 tx=0).stop 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 msgr2=0x7f4b2016cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 --2- 172.21.15.106:0/3495694086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b200aff30 0x7f4b2016cd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c7a0 tx=0x7f4b1800cc60 comp rx=0 tx=0).stop 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b1ffff640 1 -- 172.21.15.106:0/3495694086 reap_dead start 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 shutdown_connections 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.582+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 >> 172.21.15.106:0/3495694086 conn(0x7f4b20073100 msgr2=0x7f4b2006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.583+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 shutdown_connections 2024-04-13T03:09:49.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.583+0000 7f4b05ffb640 1 -- 172.21.15.106:0/3495694086 wait complete. 2024-04-13T03:09:49.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.586+0000 7fa09575e640 1 -- 172.21.15.106:0/986163282 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0880057d0 con 0x7fa07c0760c0 2024-04-13T03:09:49.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.587+0000 7fa0777fe640 1 -- 172.21.15.106:0/986163282 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0880057d0 con 0x7fa07c0760c0 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7fa07c0760c0 msgr2=0x7fa07c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7fa07c0760c0 0x7fa07c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa07c06c9c0 msgr2=0x7fa07c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa07c06c9c0 0x7fa07c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bf90 tx=0x7fa0900ad000 comp rx=0 tx=0).stop 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 msgr2=0x7fa088052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 --2- 172.21.15.106:0/986163282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa088052050 0x7fa088052490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa088011e10 tx=0x7fa084007660 comp rx=0 tx=0).stop 2024-04-13T03:09:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21000ed4a0 con 0x7f20f4076060 2024-04-13T03:09:49.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa08f7fe640 1 -- 172.21.15.106:0/986163282 reap_dead start 2024-04-13T03:09:49.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 shutdown_connections 2024-04-13T03:09:49.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7f20ef7fe640 1 -- 172.21.15.106:0/2830037091 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21000ed4a0 con 0x7f20f4076060 2024-04-13T03:09:49.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.588+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 >> 172.21.15.106:0/986163282 conn(0x7fa0880559d0 msgr2=0x7fa088039a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 shutdown_connections 2024-04-13T03:09:49.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7fa0757fa640 1 -- 172.21.15.106:0/986163282 wait complete. 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f20f4076060 msgr2=0x7f20f40784a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f20f4076060 0x7f20f40784a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20f406c970 msgr2=0x7f20f406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20f406c970 0x7f20f406ee30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001640 tx=0x7f20fc009040 comp rx=0 tx=0).stop 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 msgr2=0x7f21000d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f734597a640 1 -- 172.21.15.106:0/4027026633 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f734004fd50 con 0x7f732c076040 2024-04-13T03:09:49.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.589+0000 7f210d62d640 1 --2- 172.21.15.106:0/2830037091 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2100055d40 0x7f21000d4230 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e8a0 tx=0x7f20f800ed60 comp rx=0 tx=0).stop 2024-04-13T03:09:49.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f21077fe640 1 -- 172.21.15.106:0/2830037091 reap_dead start 2024-04-13T03:09:49.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 shutdown_connections 2024-04-13T03:09:49.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 >> 172.21.15.106:0/2830037091 conn(0x7f210001e6f0 msgr2=0x7f210003a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 shutdown_connections 2024-04-13T03:09:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f210d62d640 1 -- 172.21.15.106:0/2830037091 wait complete. 2024-04-13T03:09:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.590+0000 7f733d7fa640 1 -- 172.21.15.106:0/4027026633 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f734004fd50 con 0x7f732c076040 2024-04-13T03:09:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7f732c076040 msgr2=0x7f732c078480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7f732c076040 0x7f732c078480 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ce008aec0 con 0x7f4cc4076140 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ee816b240 con 0x7f5ecc076d10 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f732c06c940 msgr2=0x7f732c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f732c06c940 0x7f732c06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7330007c90 tx=0x7f7330007be0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 msgr2=0x7f73400b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 --2- 172.21.15.106:0/4027026633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73400c71f0 0x7f73400b4d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f733400b450 tx=0x7f733400b910 comp rx=0 tx=0).stop 2024-04-13T03:09:49.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7345179640 1 -- 172.21.15.106:0/4027026633 reap_dead start 2024-04-13T03:09:49.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f4cd6ffd640 1 -- 172.21.15.106:0/785602560 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4cd8003e00 con 0x7f4cc4076140 2024-04-13T03:09:49.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f5eddffb640 1 -- 172.21.15.106:0/3790415082 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ee816b240 con 0x7f5ecc076d10 2024-04-13T03:09:49.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 shutdown_connections 2024-04-13T03:09:49.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.591+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 >> 172.21.15.106:0/4027026633 conn(0x7f73400ccfe0 msgr2=0x7f734006f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 shutdown_connections 2024-04-13T03:09:49.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f7326ffd640 1 -- 172.21.15.106:0/4027026633 wait complete. 2024-04-13T03:09:49.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7f5ecc076d10 msgr2=0x7f5ecc0791b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f5eef888640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7f5ecc076d10 0x7f5ecc0791b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f4cc4076140 msgr2=0x7f4cc4078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f4ce724b640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f4cc4076140 0x7f4cc4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cc406c9c0 msgr2=0x7f4cc406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.592+0000 7f4ce724b640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cc406c9c0 0x7f4cc406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ce01467e0 tx=0x7f4cd0009060 comp rx=0 tx=0).stop 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 msgr2=0x7f4ce0147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f4ce724b640 1 --2- 172.21.15.106:0/785602560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ccc003a00 0x7f4ce0147d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc053000 tx=0x7f4ce00cf4e0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f4ce6a4a640 1 -- 172.21.15.106:0/785602560 reap_dead start 2024-04-13T03:09:49.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ecc06c940 msgr2=0x7f5ecc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 shutdown_connections 2024-04-13T03:09:49.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 >> 172.21.15.106:0/785602560 conn(0x7f4ce0070550 msgr2=0x7f4ce0073540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ecc06c940 0x7f5ecc06ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0000b60 tx=0x7f5ee0009040 comp rx=0 tx=0).stop 2024-04-13T03:09:49.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 msgr2=0x7f5ee8201e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 --2- 172.21.15.106:0/3790415082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8072f20 0x7f5ee8201e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b3f0 tx=0x7f5ee400b8b0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eede25640 1 -- 172.21.15.106:0/3790415082 reap_dead start 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 shutdown_connections 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 >> 172.21.15.106:0/3790415082 conn(0x7f5ee80b4f70 msgr2=0x7f5ee806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 shutdown_connections 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7f5eef888640 1 -- 172.21.15.106:0/3790415082 wait complete. 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.593+0000 7ffb158d5640 1 -- 172.21.15.106:0/2972946309 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb10069120 con 0x7ffafc076140 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.594+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 shutdown_connections 2024-04-13T03:09:49.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.594+0000 7f4ce724b640 1 -- 172.21.15.106:0/785602560 wait complete. 2024-04-13T03:09:49.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.594+0000 7ffaf7fff640 1 -- 172.21.15.106:0/2972946309 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffb10069120 con 0x7ffafc076140 2024-04-13T03:09:49.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.594+0000 7fa85cff9640 1 -- 172.21.15.106:0/1267236520 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa848002dd0 con 0x7fa850076130 2024-04-13T03:09:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7fa85effd640 1 -- 172.21.15.106:0/1267236520 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa848002dd0 con 0x7fa850076130 2024-04-13T03:09:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7ffafc076140 msgr2=0x7ffafc078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7ffafc076140 0x7ffafc078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffafc06c9c0 msgr2=0x7ffafc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffafc06c9c0 0x7ffafc06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffb00036000 tx=0x7ffb0003d040 comp rx=0 tx=0).stop 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb10160910 msgr2=0x7ffb101537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffaf5ffb640 1 --2- 172.21.15.106:0/2972946309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb10160910 0x7ffb101537e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c780 tx=0x7ffb0800cc40 comp rx=0 tx=0).stop 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7fa850076130 msgr2=0x7fa850078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7fa874fe9640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7fa850076130 0x7fa850078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.595+0000 7ffb0f7fe640 1 -- 172.21.15.106:0/2972946309 reap_dead start 2024-04-13T03:09:49.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa85006c9c0 msgr2=0x7fa85006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 shutdown_connections 2024-04-13T03:09:49.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa85006c9c0 0x7fa85006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8680057c0 tx=0x7fa868003c60 comp rx=0 tx=0).stop 2024-04-13T03:09:49.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa87009cce0 msgr2=0x7fa87009d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 --2- 172.21.15.106:0/1267236520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa87009cce0 0x7fa87009d160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa870074160 tx=0x7fa864004730 comp rx=0 tx=0).stop 2024-04-13T03:09:49.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 >> 172.21.15.106:0/2972946309 conn(0x7ffb100cdef0 msgr2=0x7ffb10163190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 shutdown_connections 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7ffaf5ffb640 1 -- 172.21.15.106:0/2972946309 wait complete. 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa86ed9d640 1 -- 172.21.15.106:0/1267236520 reap_dead start 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 shutdown_connections 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 >> 172.21.15.106:0/1267236520 conn(0x7fa870073100 msgr2=0x7fa87006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.596+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 shutdown_connections 2024-04-13T03:09:49.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.597+0000 7fa874fe9640 1 -- 172.21.15.106:0/1267236520 wait complete. 2024-04-13T03:09:49.660 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:49.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-13T03:09:49.661 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:49.662 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:49.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-13T03:09:49.663 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-13T03:09:49.663 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:49.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-13T03:09:49.664 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:49.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-13T03:09:49.665 INFO:teuthology.orchestra.run.smithi106.stdout:55834574860 2024-04-13T03:09:49.666 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-13T03:09:49.666 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:49.667 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-13T03:09:49.667 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:49.668 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-13T03:09:49.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.867+0000 7f383ca88640 1 Processor -- start 2024-04-13T03:09:49.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.867+0000 7f383ca88640 1 -- start start 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.868+0000 7f383ca88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38380cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.868+0000 7f383ca88640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38380cd910 0x7f383806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.868+0000 7f383ca88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f383806bac0 con 0x7f38380b4e10 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.868+0000 7f383ca88640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f383806c3f0 con 0x7f38380cd910 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.868+0000 7f383ca88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f383806bda0 con 0x7f383815a7a0 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.869+0000 7f3836d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f38380b4e10 0x7f383806c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56188/0 (socket says 172.21.15.106:56188) 2024-04-13T03:09:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.869+0000 7f3836d9d640 1 -- 172.21.15.106:0/1609745966 learned_addr learned my addr 172.21.15.106:0/1609745966 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.869+0000 7f383659c640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38380cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.869+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347539965 0 0) 0x7f383806bac0 con 0x7f38380b4e10 2024-04-13T03:09:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.869+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3820003520 con 0x7f38380b4e10 2024-04-13T03:09:49.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f24c881b640 1 Processor -- start 2024-04-13T03:09:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f3835d9b640 1 --2- 172.21.15.106:0/1609745966 >> v2:172.21.15.114:3300/0 conn(0x7f38380cd910 0x7f383806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 724625885 0 0) 0x7f3820003520 con 0x7f38380b4e10 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 msgr2=0x7f38380cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38380cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 >> v2:172.21.15.114:3300/0 conn(0x7f38380cd910 msgr2=0x7f383806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 --2- 172.21.15.106:0/1609745966 >> v2:172.21.15.114:3300/0 conn(0x7f38380cd910 0x7f383806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383806c080 con 0x7f38380b4e10 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f382c001670 con 0x7f38380b4e10 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 >> v1:172.21.15.106:6789/0 conn(0x7f38380b4e10 legacy=0x7f383806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3820003f80 0x7f3820004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3820004d40 0x7f38200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f383806c3f0 con 0x7f3820003f80 2024-04-13T03:09:49.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f383806bda0 con 0x7f3820004d40 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f382000ba70 con 0x7f383815a7a0 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f3835d9b640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3820003f80 0x7f3820004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f3836d9d640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3820004d40 0x7f38200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f63c4a2d640 1 Processor -- start 2024-04-13T03:09:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f63c4a2d640 1 -- start start 2024-04-13T03:09:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 -- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3820004d40 msgr2=0x7f38200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3820004d40 0x7f38200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 -- 172.21.15.106:0/1609745966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3820003f80 msgr2=0x7f3820004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3820003f80 0x7f3820004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f383659c640 1 -- 172.21.15.106:0/1609745966 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3820003520 con 0x7f383815a7a0 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.870+0000 7f24c881b640 1 -- start start 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c881b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c881b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24c406c420 0x7f24c406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c881b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c40ccba0 con 0x7f24c415aad0 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c881b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f24c40cd1b0 con 0x7f24c406c420 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c881b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24c4073b20 con 0x7f24c415a6f0 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c2d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c2d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49736/0 (socket says 172.21.15.106:49736) 2024-04-13T03:09:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c2d9d640 1 -- 172.21.15.106:0/4177739617 learned_addr learned my addr 172.21.15.106:0/4177739617 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f24c359e640 1 --2- 172.21.15.106:0/4177739617 >> v2:172.21.15.114:3300/0 conn(0x7f24c406c420 0x7f24c406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f24c2d9d640 1 -- 172.21.15.106:0/4177739617 >> v2:172.21.15.114:3300/0 conn(0x7f24c406c420 msgr2=0x7f24c406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f24c2d9d640 1 --2- 172.21.15.106:0/4177739617 >> v2:172.21.15.114:3300/0 conn(0x7f24c406c420 0x7f24c406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f24c2d9d640 1 -- 172.21.15.106:0/4177739617 >> v1:172.21.15.114:6789/0 conn(0x7f24c415aad0 legacy=0x7f24c4072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f24c2d9d640 1 -- 172.21.15.106:0/4177739617 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c40cce40 con 0x7f24c415a6f0 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f1dcf355640 1 Processor -- start 2024-04-13T03:09:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f1dcf355640 1 -- start start 2024-04-13T03:09:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f42f63cf640 1 Processor -- start 2024-04-13T03:09:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.871+0000 7f42f63cf640 1 -- start start 2024-04-13T03:09:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f383659c640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38200039b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3830009fc0 tx=0x7f3820005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6360c547beee4b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63c4a2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f63b803e840 0x7f63b8053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383003c070 con 0x7f383815a7a0 2024-04-13T03:09:49.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3830037600 con 0x7f383815a7a0 2024-04-13T03:09:49.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f24c2d9d640 1 --2- 172.21.15.106:0/4177739617 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24c40cd330 tx=0x7f24bc02c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6786030d807a986 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f383559a640 1 -- 172.21.15.106:0/1609745966 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830009c10 con 0x7f383815a7a0 2024-04-13T03:09:49.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63be59c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f63b80e30c0 0x7f63b8057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53828/0 (socket says 172.21.15.106:53828) 2024-04-13T03:09:49.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63be59c640 1 -- 172.21.15.106:0/1796617084 learned_addr learned my addr 172.21.15.106:0/1796617084 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 0x7f63b80068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b8058570 con 0x7f63b80e30c0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f63b8058850 con 0x7f63b803e840 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f63b8058b30 con 0x7f63b8054130 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63bed9d640 1 --2- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 0x7f63b80068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63bd59a640 1 -- 172.21.15.106:0/1796617084 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294280525 0 0) 0x7f63b8058570 con 0x7f63b80e30c0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f63bd59a640 1 -- 172.21.15.106:0/1796617084 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63a8003520 con 0x7f63b80e30c0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f24c1d9b640 1 -- 172.21.15.106:0/4177739617 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0343f0 con 0x7f24c415a6f0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f24c1d9b640 1 -- 172.21.15.106:0/4177739617 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc034590 con 0x7f24c415a6f0 2024-04-13T03:09:49.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f24c1d9b640 1 -- 172.21.15.106:0/4177739617 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc034960 con 0x7f24c415a6f0 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f42f63cf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.872+0000 7f42f63cf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42f006ebb0 0x7f42f00cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42f63cf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f00ce860 con 0x7f42f015d520 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42f63cf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f42f01638c0 con 0x7f42f006ebb0 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42f63cf640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42f006fb10 con 0x7f42f015d140 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49744/0 (socket says 172.21.15.106:49744) 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 -- 172.21.15.106:0/2196823814 learned_addr learned my addr 172.21.15.106:0/2196823814 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 -- 172.21.15.106:0/2196823814 >> v2:172.21.15.106:3300/0 conn(0x7f42f006ebb0 msgr2=0x7f42f00cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 --2- 172.21.15.106:0/2196823814 >> v2:172.21.15.106:3300/0 conn(0x7f42f006ebb0 0x7f42f00cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 -- 172.21.15.106:0/2196823814 >> v1:172.21.15.106:6789/0 conn(0x7f42f015d520 legacy=0x7f42f006e4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcf355640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 0x7f1dc806f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcf355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 0x7f1dc80b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcf355640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc806f6b0 con 0x7f1dc80b32d0 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcf355640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc80b4670 con 0x7f1dc81660f0 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcf355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc806ba10 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcd0f1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 0x7f1dc806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcd8f2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 0x7f1dc80b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcd0f1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 0x7f1dc806f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53244/0 (socket says 172.21.15.106:53244) 2024-04-13T03:09:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f1dcd0f1640 1 -- 172.21.15.106:0/3185819906 learned_addr learned my addr 172.21.15.106:0/3185819906 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f383ca88640 1 -- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 msgr2=0x7f38200039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f383ca88640 1 --2- 172.21.15.106:0/1609745966 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383815a7a0 0x7f38200039b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3830009fc0 tx=0x7f3820005830 comp rx=0 tx=0).stop 2024-04-13T03:09:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f3836d9d640 1 -- 172.21.15.106:0/1609745966 reap_dead start 2024-04-13T03:09:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.873+0000 7f42effff640 1 -- 172.21.15.106:0/2196823814 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f0163af0 con 0x7f42f015d140 2024-04-13T03:09:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f63bed9d640 1 -- 172.21.15.106:0/1796617084 >> v2:172.21.15.114:3300/0 conn(0x7f63b803e840 msgr2=0x7f63b8053bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f63bed9d640 1 --2- 172.21.15.106:0/1796617084 >> v2:172.21.15.114:3300/0 conn(0x7f63b803e840 0x7f63b8053bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f63bed9d640 1 -- 172.21.15.106:0/1796617084 >> v1:172.21.15.114:6789/0 conn(0x7f63b80e30c0 legacy=0x7f63b8057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f63bed9d640 1 -- 172.21.15.106:0/1796617084 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b80063b0 con 0x7f63b8054130 2024-04-13T03:09:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f25097b4640 1 Processor -- start 2024-04-13T03:09:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f1dcd8f2640 1 -- 172.21.15.106:0/3185819906 >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 msgr2=0x7f1dc806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f1dcd8f2640 1 --2- 172.21.15.106:0/3185819906 >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 0x7f1dc806f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f1dcd8f2640 1 -- 172.21.15.106:0/3185819906 >> v1:172.21.15.106:6789/0 conn(0x7f1dc80b32d0 legacy=0x7f1dc80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f1dcd8f2640 1 -- 172.21.15.106:0/3185819906 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc806bcf0 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f42effff640 1 --2- 172.21.15.106:0/2196823814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009870 tx=0x7f42e002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a066b0c2eb4968 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 -- 172.21.15.106:0/4177739617 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 msgr2=0x7f24c40cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 --2- 172.21.15.106:0/4177739617 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24c40cd330 tx=0x7f24bc02c360 comp rx=0 tx=0).stop 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 -- 172.21.15.106:0/4177739617 shutdown_connections 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 --2- 172.21.15.106:0/4177739617 >> v2:172.21.15.114:3300/0 conn(0x7f24c406c420 0x7f24c406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 --2- 172.21.15.106:0/4177739617 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c415a6f0 0x7f24c40cd640 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24c40cd330 tx=0x7f24bc02c360 comp rx=0 tx=0).stop 2024-04-13T03:09:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f24c881b640 1 -- 172.21.15.106:0/4177739617 >> 172.21.15.106:0/4177739617 conn(0x7f24c4072100 msgr2=0x7f24c40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f383ca88640 1 -- 172.21.15.106:0/1609745966 shutdown_connections 2024-04-13T03:09:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.874+0000 7f383ca88640 1 -- 172.21.15.106:0/1609745966 >> 172.21.15.106:0/1609745966 conn(0x7f38380d02e0 msgr2=0x7f38380d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f383ca88640 1 -- 172.21.15.106:0/1609745966 shutdown_connections 2024-04-13T03:09:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7fbfef5c8640 1 Processor -- start 2024-04-13T03:09:49.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f42eeffd640 1 -- 172.21.15.106:0/2196823814 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0035020 con 0x7f42f015d140 2024-04-13T03:09:49.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f42eeffd640 1 -- 172.21.15.106:0/2196823814 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e0031640 con 0x7f42f015d140 2024-04-13T03:09:49.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f42eeffd640 1 -- 172.21.15.106:0/2196823814 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e002d040 con 0x7f42f015d140 2024-04-13T03:09:49.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f25097b4640 1 -- start start 2024-04-13T03:09:49.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f383ca88640 1 -- 172.21.15.106:0/1609745966 wait complete. 2024-04-13T03:09:49.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f24c881b640 1 -- 172.21.15.106:0/4177739617 shutdown_connections 2024-04-13T03:09:49.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f63bed9d640 1 --2- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 0x7f63b80068d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63c0065750 tx=0x7f63c00a71f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776b5a967464f5be server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f1dcd8f2640 1 --2- 172.21.15.106:0/3185819906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 0x7f1dc80b4130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009d90 tx=0x7f1dc002bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3daafff85635af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7fbfef5c8640 1 -- start start 2024-04-13T03:09:49.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f63bd59a640 1 -- 172.21.15.106:0/1796617084 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00b4070 con 0x7f63b8054130 2024-04-13T03:09:49.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f63bd59a640 1 -- 172.21.15.106:0/1796617084 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c00a7de0 con 0x7f63b8054130 2024-04-13T03:09:49.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f24c881b640 1 -- 172.21.15.106:0/4177739617 wait complete. 2024-04-13T03:09:49.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.875+0000 7f63bd59a640 1 -- 172.21.15.106:0/1796617084 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4001670 con 0x7f63b8054130 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 -- 172.21.15.106:0/2196823814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 msgr2=0x7f42f0163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 --2- 172.21.15.106:0/2196823814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009870 tx=0x7f42e002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 -- 172.21.15.106:0/2196823814 shutdown_connections 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 --2- 172.21.15.106:0/2196823814 >> v2:172.21.15.106:3300/0 conn(0x7f42f006ebb0 0x7f42f00cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 --2- 172.21.15.106:0/2196823814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f015d140 0x7f42f0163380 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009870 tx=0x7f42e002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f42f63cf640 1 -- 172.21.15.106:0/2196823814 >> 172.21.15.106:0/2196823814 conn(0x7f42f00b2640 msgr2=0x7f42f00b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25097b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 0x7f250406f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25097b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25040b32d0 0x7f25040cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25097b4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250406f6b0 con 0x7f25040cd580 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25097b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f250406ba10 con 0x7f25040b32d0 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25097b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f250406bcf0 con 0x7f25041660f0 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25040b32d0 0x7f25040cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25040b32d0 0x7f25040cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45160/0 (socket says 172.21.15.106:45160) 2024-04-13T03:09:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f25027fc640 1 -- 172.21.15.106:0/981353020 learned_addr learned my addr 172.21.15.106:0/981353020 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 msgr2=0x7f63b80068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 0x7f63b80068d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63c0065750 tx=0x7f63c00a71f0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f1dbffff640 1 -- 172.21.15.106:0/3185819906 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0039070 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f1dbffff640 1 -- 172.21.15.106:0/3185819906 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc002cb40 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.876+0000 7f1dbffff640 1 -- 172.21.15.106:0/3185819906 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0034440 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f2502ffd640 1 --2- 172.21.15.106:0/981353020 >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 0x7f250406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 shutdown_connections 2024-04-13T03:09:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/1796617084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b8054130 0x7f63b80068d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/1796617084 >> v2:172.21.15.114:3300/0 conn(0x7f63b803e840 0x7f63b8053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 >> 172.21.15.106:0/1796617084 conn(0x7f63b80559e0 msgr2=0x7f63b8055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f383ca88640 1 Processor -- start 2024-04-13T03:09:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f383ca88640 1 -- start start 2024-04-13T03:09:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 shutdown_connections 2024-04-13T03:09:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f63c4a2d640 1 -- 172.21.15.106:0/1796617084 wait complete. 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 -- 172.21.15.106:0/3185819906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 msgr2=0x7f1dc80b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 --2- 172.21.15.106:0/3185819906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 0x7f1dc80b4130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009d90 tx=0x7f1dc002bd90 comp rx=0 tx=0).stop 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f25027fc640 1 -- 172.21.15.106:0/981353020 >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 msgr2=0x7f250406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f25027fc640 1 --2- 172.21.15.106:0/981353020 >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 0x7f250406f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f25027fc640 1 -- 172.21.15.106:0/981353020 >> v1:172.21.15.114:6789/0 conn(0x7f25040cd580 legacy=0x7f25040b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f25027fc640 1 -- 172.21.15.106:0/981353020 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250406bfd0 con 0x7f25040b32d0 2024-04-13T03:09:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f2502ffd640 1 --2- 172.21.15.106:0/981353020 >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 0x7f250406f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 -- 172.21.15.106:0/3185819906 shutdown_connections 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 --2- 172.21.15.106:0/3185819906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc80b3ce0 0x7f1dc80b4130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 --2- 172.21.15.106:0/3185819906 >> v2:172.21.15.114:3300/0 conn(0x7f1dc81660f0 0x7f1dc806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 -- 172.21.15.106:0/3185819906 >> 172.21.15.106:0/3185819906 conn(0x7f1dc80b2600 msgr2=0x7f1dc80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfef5c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfef5c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe80d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfef5c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe815fa30 con 0x7fbfe80d46c0 2024-04-13T03:09:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfef5c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe80d5060 con 0x7fbfe815e9d0 2024-04-13T03:09:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfef5c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe8162520 con 0x7fbfe815f4c0 2024-04-13T03:09:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 -- 172.21.15.106:0/3185819906 shutdown_connections 2024-04-13T03:09:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f24c881b640 1 Processor -- start 2024-04-13T03:09:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f1dcf355640 1 -- 172.21.15.106:0/3185819906 wait complete. 2024-04-13T03:09:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7fbfed364640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7fbfed364640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53264/0 (socket says 172.21.15.106:53264) 2024-04-13T03:09:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7fbfecb63640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbfe80d46c0 0x7fbfe815edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56204/0 (socket says 172.21.15.106:56204) 2024-04-13T03:09:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7fbfecb63640 1 -- 172.21.15.106:0/2052025595 learned_addr learned my addr 172.21.15.106:0/2052025595 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f7f53680640 1 Processor -- start 2024-04-13T03:09:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f7f53680640 1 -- start start 2024-04-13T03:09:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 0x7f38381506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383814e800 0x7f38381ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 0x7f38381efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f383815e590 con 0x7f383814ec60 2024-04-13T03:09:49.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.877+0000 7f24c881b640 1 -- start start 2024-04-13T03:09:49.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f383815e410 con 0x7f383815a7a0 2024-04-13T03:09:49.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383ca88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3838072d40 con 0x7f383814e800 2024-04-13T03:09:49.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 0x7f38381506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f3836d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 0x7f38381efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 0x7f38381506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53272/0 (socket says 172.21.15.106:53272) 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f383659c640 1 -- 172.21.15.106:0/3727026122 learned_addr learned my addr 172.21.15.106:0/3727026122 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f3835d9b640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383814e800 0x7f38381ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 0x7f24c41f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 0x7f24c409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 0x7f24c41f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40736c0 con 0x7f24c415aad0 2024-04-13T03:09:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4073e00 con 0x7f24c406c420 2024-04-13T03:09:49.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c881b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24c4073840 con 0x7f24c40cb730 2024-04-13T03:09:49.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f63c4a2d640 1 Processor -- start 2024-04-13T03:09:49.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 -- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383814e800 msgr2=0x7f38381ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f63c4a2d640 1 -- start start 2024-04-13T03:09:49.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383814e800 0x7f38381ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f53680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 0x7f7f4c0b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f53680640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c070350 con 0x7f7f4c15caf0 2024-04-13T03:09:49.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f53680640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c0d05d0 con 0x7f7f4c06fea0 2024-04-13T03:09:49.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f53680640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c15dee0 con 0x7f7f4c0cd520 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f1dcf355640 1 Processor -- start 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f1dcf355640 1 -- start start 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f25027fc640 1 --2- 172.21.15.106:0/981353020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040b32d0 0x7f25040cd040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f25040b2c70 tx=0x7f24f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b1b71fd416134c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f2501ffb640 1 -- 172.21.15.106:0/981353020 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8035020 con 0x7f25040b32d0 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f2501ffb640 1 -- 172.21.15.106:0/981353020 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f8031870 con 0x7f25040b32d0 2024-04-13T03:09:49.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7f2501ffb640 1 -- 172.21.15.106:0/981353020 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8031c40 con 0x7f25040b32d0 2024-04-13T03:09:49.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 -- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 msgr2=0x7f38381506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 0x7f38381506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 -- 172.21.15.106:0/3727026122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38380b0c90 con 0x7f383814ec60 2024-04-13T03:09:49.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f3836d9d640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 0x7f38381efa90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f383815e370 tx=0x7f382c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f5141c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7f4c15caf0 0x7f7f4c0cce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56208/0 (socket says 172.21.15.106:56208) 2024-04-13T03:09:49.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 -- 172.21.15.106:0/981353020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040b32d0 msgr2=0x7f25040cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 --2- 172.21.15.106:0/981353020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040b32d0 0x7f25040cd040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f25040b2c70 tx=0x7f24f802b910 comp rx=0 tx=0).stop 2024-04-13T03:09:49.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f5141c640 1 -- 172.21.15.106:0/3388068798 learned_addr learned my addr 172.21.15.106:0/3388068798 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:49.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f7f50c1b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 0x7f7f4c0b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c359e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 0x7f24c41f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c2d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 0x7f24c41f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 -- 172.21.15.106:0/981353020 shutdown_connections 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 --2- 172.21.15.106:0/981353020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040b32d0 0x7f25040cd040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 --2- 172.21.15.106:0/981353020 >> v2:172.21.15.114:3300/0 conn(0x7f25041660f0 0x7f250406f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c359e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 0x7f24c41f9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49788/0 (socket says 172.21.15.106:49788) 2024-04-13T03:09:49.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c359e640 1 -- 172.21.15.106:0/2707020317 learned_addr learned my addr 172.21.15.106:0/2707020317 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7f24c259c640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 0x7f24c409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f42f63cf640 1 -- 172.21.15.106:0/2196823814 shutdown_connections 2024-04-13T03:09:49.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7fbfedb65640 1 --2- 172.21.15.106:0/2052025595 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe80d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.878+0000 7fbfedb65640 1 -- 172.21.15.106:0/2052025595 >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 msgr2=0x7fbfe8162f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7fbfedb65640 1 --2- 172.21.15.106:0/2052025595 >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7fbfedb65640 1 -- 172.21.15.106:0/2052025595 >> v1:172.21.15.106:6789/0 conn(0x7fbfe80d46c0 legacy=0x7fbfe815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:49.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7fbfedb65640 1 -- 172.21.15.106:0/2052025595 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe815fd10 con 0x7fbfe815f4c0 2024-04-13T03:09:49.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c020420 con 0x7f383814ec60 2024-04-13T03:09:49.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.879+0000 7fbfed364640 1 --2- 172.21.15.106:0/2052025595 >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 -- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 msgr2=0x7f24c41f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 -- 172.21.15.106:0/981353020 >> 172.21.15.106:0/981353020 conn(0x7f25040b2600 msgr2=0x7f25040d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 0x7f24c41f9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 -- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 msgr2=0x7f24c409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 0x7f24c409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 -- 172.21.15.106:0/2707020317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41f9900 con 0x7f24c406c420 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f383ca88640 1 -- 172.21.15.106:0/3727026122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38380b0f70 con 0x7f383814ec60 2024-04-13T03:09:49.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f383ca88640 1 -- 172.21.15.106:0/3727026122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383815bde0 con 0x7f383814ec60 2024-04-13T03:09:49.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f25097b4640 1 -- 172.21.15.106:0/981353020 shutdown_connections 2024-04-13T03:09:49.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382c01b030 con 0x7f383814ec60 2024-04-13T03:09:49.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c259c640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 0x7f24c409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:49.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f3ffff640 1 -- 172.21.15.106:0/3388068798 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188888472 0 0) 0x7f7f4c0d05d0 con 0x7f7f4c06fea0 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f63c4a2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b803e840 0x7f63b804f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f3ffff640 1 -- 172.21.15.106:0/3388068798 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f38003520 con 0x7f7f4c06fea0 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f50c1b640 1 -- 172.21.15.106:0/3388068798 >> v1:172.21.15.114:6789/0 conn(0x7f7f4c06fea0 legacy=0x7f7f4c0d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f50c1b640 1 -- 172.21.15.106:0/3388068798 >> v1:172.21.15.106:6789/0 conn(0x7f7f4c15caf0 legacy=0x7f7f4c0cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f3ffff640 1 -- 172.21.15.106:0/3388068798 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886882189 0 0) 0x7f7f4c070350 con 0x7f7f4c15caf0 2024-04-13T03:09:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f7f50c1b640 1 -- 172.21.15.106:0/3388068798 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0705f0 con 0x7f7f4c0cd520 2024-04-13T03:09:49.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7fbfedb65640 1 --2- 172.21.15.106:0/2052025595 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe80d4b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009d90 tx=0x7fbfe002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855b174fa944bfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 0x7f1dc8201270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63c4a2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 0x7f63b80525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 0x7f1dc82017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80cd750 con 0x7f1dc80b32d0 2024-04-13T03:09:49.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc806c2b0 con 0x7f1dc80b3ce0 2024-04-13T03:09:49.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63c4a2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 0x7f63b80df140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcf355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc81664d0 con 0x7f1dc81660f0 2024-04-13T03:09:49.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63bed9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 0x7f63b80df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63bdd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 0x7f63b80525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63bdd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 0x7f63b80525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53294/0 (socket says 172.21.15.106:53294) 2024-04-13T03:09:49.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63bdd9b640 1 -- 172.21.15.106:0/655546296 learned_addr learned my addr 172.21.15.106:0/655546296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c020820 con 0x7f383814ec60 2024-04-13T03:09:49.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8006e10 con 0x7f63b803e840 2024-04-13T03:09:49.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63b80e34a0 con 0x7f63b8054130 2024-04-13T03:09:49.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f63b8006f90 con 0x7f63b80e30c0 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f25097b4640 1 -- 172.21.15.106:0/981353020 wait complete. 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 -- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 msgr2=0x7f63b80525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 0x7f63b80525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 -- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b803e840 msgr2=0x7f63b804f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b803e840 0x7f63b804f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b80d94e0 con 0x7f63b80e30c0 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcd0f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcd0f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45202/0 (socket says 172.21.15.106:45202) 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcd0f1640 1 -- 172.21.15.106:0/942305768 learned_addr learned my addr 172.21.15.106:0/942305768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f1dcc8f0640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 0x7f1dc8201270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63bed9d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 0x7f63b80df140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63c0075b00 tx=0x7f63c0075c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00be070 con 0x7f63b80e30c0 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c00b9660 con 0x7f63b80e30c0 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f42f63cf640 1 -- 172.21.15.106:0/2196823814 wait complete. 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f42f63cf640 1 Processor -- start 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd8f2640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 0x7f1dc82017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd0f1640 1 -- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 msgr2=0x7f1dc82017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd0f1640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 0x7f1dc82017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd0f1640 1 -- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 msgr2=0x7f1dc8201270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd0f1640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 0x7f1dc8201270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd0f1640 1 -- 172.21.15.106:0/942305768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc8163430 con 0x7f1dc80b32d0 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f1dcd8f2640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 0x7f1dc82017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f1dcc8f0640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 0x7f1dc8201270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f7f50c1b640 1 --2- 172.21.15.106:0/3388068798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 0x7f7f4c0b80f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f480097d0 tx=0x7f7f4802b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e314eb6d588b61c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f7f3ffff640 1 -- 172.21.15.106:0/3388068798 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4802f810 con 0x7f7f4c0cd520 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f7f3ffff640 1 -- 172.21.15.106:0/3388068798 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4802f9b0 con 0x7f7f4c0cd520 2024-04-13T03:09:49.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f383ca88640 1 -- 172.21.15.106:0/3727026122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c005290 con 0x7f383814ec60 2024-04-13T03:09:49.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f1dcd0f1640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e8b0 tx=0x7f1dc400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00b4040 con 0x7f63b80e30c0 2024-04-13T03:09:49.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b8051190 con 0x7f63b80e30c0 2024-04-13T03:09:49.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b8051750 con 0x7f63b80e30c0 2024-04-13T03:09:49.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f7f53680640 1 -- 172.21.15.106:0/3388068798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 msgr2=0x7f7f4c0b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f7f53680640 1 --2- 172.21.15.106:0/3388068798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 0x7f7f4c0b80f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f480097d0 tx=0x7f7f4802b9f0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f42f63cf640 1 -- start start 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.880+0000 7f24c2d9d640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 0x7f24c41f93c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24bc032000 tx=0x7f24bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc043020 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f24c881b640 1 -- 172.21.15.106:0/2707020317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c40cbbb0 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f24c881b640 1 -- 172.21.15.106:0/2707020317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c409c510 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc03e660 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.881+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0343f0 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.882+0000 7f24c881b640 1 -- 172.21.15.106:0/2707020317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c404fdc0 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24bc046020 con 0x7f24c406c420 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f24ab7fe640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 0x7f24b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f7f53680640 1 -- 172.21.15.106:0/3388068798 shutdown_connections 2024-04-13T03:09:49.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f7f53680640 1 --2- 172.21.15.106:0/3388068798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c0cd520 0x7f7f4c0b80f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f7f53680640 1 -- 172.21.15.106:0/3388068798 >> 172.21.15.106:0/3388068798 conn(0x7f7f4c0d25b0 msgr2=0x7f7f4c0d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f7f53680640 1 -- 172.21.15.106:0/3388068798 shutdown_connections 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f25097b4640 1 Processor -- start 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.883+0000 7f25097b4640 1 -- start start 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7fbfdffff640 1 -- 172.21.15.106:0/2052025595 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0039070 con 0x7fbfe815f4c0 2024-04-13T03:09:49.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7fbfdffff640 1 -- 172.21.15.106:0/2052025595 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe002cbd0 con 0x7fbfe815f4c0 2024-04-13T03:09:49.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24bc0aea20 con 0x7f24c406c420 2024-04-13T03:09:49.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f7f53680640 1 -- 172.21.15.106:0/3388068798 wait complete. 2024-04-13T03:09:49.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f42f63cf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f42f63cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f015d520 0x7f42f0161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f382c02a480 con 0x7f383814ec60 2024-04-13T03:09:49.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25040b32d0 0x7f25041547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42effff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42effff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49830/0 (socket says 172.21.15.106:49830) 2024-04-13T03:09:49.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42effff640 1 -- 172.21.15.106:0/535925622 learned_addr learned my addr 172.21.15.106:0/535925622 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.884+0000 7f24c259c640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 0x7f24b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25041660f0 0x7f2504157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45218/0 (socket says 172.21.15.106:45218) 2024-04-13T03:09:49.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25027fc640 1 -- 172.21.15.106:0/2236174083 learned_addr learned my addr 172.21.15.106:0/2236174083 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2504166650 con 0x7f25040cd580 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25041664d0 con 0x7f25041660f0 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25040b43f0 con 0x7f25040b32d0 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f2502ffd640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25040b32d0 0x7f25041547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f25037fe640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25041660f0 0x7f2504157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f3826ffd640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 0x7f381806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 -- 172.21.15.106:0/2052025595 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 msgr2=0x7fbfe80d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/2052025595 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe80d4b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009d90 tx=0x7fbfe002be20 comp rx=0 tx=0).stop 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42ef7fe640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f015d520 0x7f42f0161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b80057e0 con 0x7f63b80e30c0 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 0x7f42f006bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42f00ceae0 con 0x7f42f006b8e0 2024-04-13T03:09:49.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01640b0 con 0x7f42f015d520 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42f0166970 con 0x7f42f006ebb0 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 -- 172.21.15.106:0/2052025595 shutdown_connections 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/2052025595 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe80d4b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/2052025595 >> v2:172.21.15.114:3300/0 conn(0x7fbfe815e9d0 0x7fbfe8162f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7fbfef5c8640 1 -- 172.21.15.106:0/2052025595 >> 172.21.15.106:0/2052025595 conn(0x7fbfe808bf10 msgr2=0x7fbfe80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f383659c640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 0x7f381806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f24c259c640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 0x7f24b006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24b4007fd0 tx=0x7f24b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.885+0000 7f42f496c640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 0x7f42f006bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42effff640 1 -- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f015d520 msgr2=0x7f42f0161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42effff640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f015d520 0x7f42f0161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42effff640 1 -- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 msgr2=0x7f42f006bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42effff640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 0x7f42f006bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42effff640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f0161e70 con 0x7f42f006ebb0 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f25027fc640 1 -- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25040b32d0 msgr2=0x7f25041547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f25027fc640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25040b32d0 0x7f25041547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f25027fc640 1 -- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25041660f0 msgr2=0x7f2504157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f25027fc640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25041660f0 0x7f2504157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f25027fc640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040ceee0 con 0x7f25040cd580 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7fbfef5c8640 1 -- 172.21.15.106:0/2052025595 shutdown_connections 2024-04-13T03:09:49.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7fbfef5c8640 1 -- 172.21.15.106:0/2052025595 wait complete. 2024-04-13T03:09:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f7f53680640 1 Processor -- start 2024-04-13T03:09:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63c00c1070 con 0x7f63b80e30c0 2024-04-13T03:09:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f382c093e80 con 0x7f383814ec60 2024-04-13T03:09:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f7f53680640 1 -- start start 2024-04-13T03:09:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f383659c640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 0x7f381806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3830000f80 tx=0x7f3830034000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f63aeffd640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 0x7f63a006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63c0129270 con 0x7f63b80e30c0 2024-04-13T03:09:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f42effff640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42e002be80 tx=0x7f42e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f25027fc640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009960 tx=0x7f24f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f63be59c640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 0x7f63a006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.886+0000 7f42f496c640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 0x7f42f006bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:49.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f63be59c640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 0x7f63a006eeb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005880 tx=0x7f63b40014b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f7f53680640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 0x7f7f4c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f7f53680640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 0x7f7f4c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.887+0000 7f7f53680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c15caf0 0x7f7f4c1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f53680640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c070e90 con 0x7f7f4c0cd520 2024-04-13T03:09:49.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7fbfef5c8640 1 Processor -- start 2024-04-13T03:09:49.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f5141c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 0x7f7f4c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f50c1b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 0x7f7f4c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f5141c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 0x7f7f4c0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53338/0 (socket says 172.21.15.106:53338) 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f5141c640 1 -- 172.21.15.106:0/473559464 learned_addr learned my addr 172.21.15.106:0/473559464 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c070bb0 con 0x7f7f4c06fea0 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d11e0 con 0x7f7f4c15caf0 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7fbfef5c8640 1 -- start start 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f7f51c1d640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c15caf0 0x7f7f4c1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24bc04d030 con 0x7f24c406c420 2024-04-13T03:09:49.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7fbfef5c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.888+0000 7fbfef5c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 0x7fbfe815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfef5c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfef5c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe81634b0 con 0x7fbfe815e9d0 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfef5c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8162ae0 con 0x7fbfe80d46c0 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53340/0 (socket says 172.21.15.106:53340) 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 -- 172.21.15.106:0/1608274327 learned_addr learned my addr 172.21.15.106:0/1608274327 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfef5c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe806ef40 con 0x7fbfe815f4c0 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfecb63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 0x7fbfe815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 -- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c15caf0 msgr2=0x7f7f4c1f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfedb65640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8035040 con 0x7f25040cd580 2024-04-13T03:09:49.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25040cf1c0 con 0x7f25040cd580 2024-04-13T03:09:49.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 -- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 msgr2=0x7fbfe815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4017070 con 0x7f1dc80b32d0 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 -- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 msgr2=0x7fbfe815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 0x7fbfe815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25040cf780 con 0x7f25040cd580 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f8045480 con 0x7f25040cd580 2024-04-13T03:09:49.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8045870 con 0x7f25040cd580 2024-04-13T03:09:49.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f382c061900 con 0x7f383814ec60 2024-04-13T03:09:49.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7fbfed364640 1 -- 172.21.15.106:0/1608274327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe80b4ff0 con 0x7fbfe80d46c0 2024-04-13T03:09:49.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7f1dcf355640 1 -- 172.21.15.106:0/942305768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc8162390 con 0x7f1dc80b32d0 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c15caf0 0x7f7f4c1f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 -- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 msgr2=0x7f7f4c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 0x7f7f4c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0caf10 con 0x7f7f4c0cd520 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.889+0000 7f7f50c1b640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 0x7f7f4c0c7e80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c073920 tx=0x7f7f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48037d50 con 0x7f7f4c0cd520 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfecb63640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 0x7fbfe815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:49.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c0cac30 con 0x7f7f4c0cd520 2024-04-13T03:09:49.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7f1dcf355640 1 -- 172.21.15.106:0/942305768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc8162950 con 0x7f1dc80b32d0 2024-04-13T03:09:49.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfed364640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800feb0 tx=0x7fbfd8012430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c06d690 con 0x7f7f4c0cd520 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd801e020 con 0x7fbfe80d46c0 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd801a420 con 0x7fbfe80d46c0 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd801a7f0 con 0x7fbfe80d46c0 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.890+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe80b52d0 con 0x7fbfe80d46c0 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe80b5890 con 0x7fbfe80d46c0 2024-04-13T03:09:49.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f48042460 con 0x7f7f4c0cd520 2024-04-13T03:09:49.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48042850 con 0x7f7f4c0cd520 2024-04-13T03:09:49.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc400c940 con 0x7f1dc80b32d0 2024-04-13T03:09:49.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cd10 con 0x7f1dc80b32d0 2024-04-13T03:09:49.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.891+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4005290 con 0x7f25040cd580 2024-04-13T03:09:49.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.892+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c005290 con 0x7f7f4c0cd520 2024-04-13T03:09:49.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.892+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc8068a10 con 0x7f1dc80b32d0 2024-04-13T03:09:49.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe8069120 con 0x7fbfe80d46c0 2024-04-13T03:09:49.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e003ac60 con 0x7f42f006ebb0 2024-04-13T03:09:49.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e003ae00 con 0x7f42f006ebb0 2024-04-13T03:09:49.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24f8038020 con 0x7f25040cd580 2024-04-13T03:09:49.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0045670 con 0x7f42f006ebb0 2024-04-13T03:09:49.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f0162150 con 0x7f42f006ebb0 2024-04-13T03:09:49.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dc401a070 con 0x7f1dc80b32d0 2024-04-13T03:09:49.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63c00c8030 con 0x7f63b80e30c0 2024-04-13T03:09:49.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f24eb7fe640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 0x7f24f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.893+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24f80ae3c0 con 0x7f25040cd580 2024-04-13T03:09:49.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f0162710 con 0x7f42f006ebb0 2024-04-13T03:09:49.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f2502ffd640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 0x7f24f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f1dbdffb640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 0x7f1dac06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1dc408c5d0 con 0x7f1dc80b32d0 2024-04-13T03:09:49.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.894+0000 7f1dcc8f0640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 0x7f1dac06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f48049060 con 0x7f7f4c0cd520 2024-04-13T03:09:49.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfd8021020 con 0x7fbfe80d46c0 2024-04-13T03:09:49.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f0202410 con 0x7f42f006ebb0 2024-04-13T03:09:49.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7f1dcc8f0640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 0x7f1dac06ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dc81642c0 tx=0x7f1db8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7f7f3dffb640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 0x7f7f3006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7fbfddffb640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 0x7fbfcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.895+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7f480ae340 con 0x7f7f4c0cd520 2024-04-13T03:09:49.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7f7f5141c640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 0x7f7f3006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7fbfecb63640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 0x7fbfcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbfd8060250 con 0x7fbfe80d46c0 2024-04-13T03:09:49.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7f7f5141c640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 0x7f7f3006ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42e0038020 con 0x7f42f006ebb0 2024-04-13T03:09:49.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.896+0000 7f2502ffd640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 0x7f24f006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b840 tx=0x7f24f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.897+0000 7fbfecb63640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 0x7fbfcc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8156ee0 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.897+0000 7f42ecff9640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 0x7f42dc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.897+0000 7f42ef7fe640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 0x7f42dc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.897+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f42e00adbf0 con 0x7f42f006ebb0 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.897+0000 7f42ef7fe640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 0x7f42dc06ee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b52b0 tx=0x7f42e4009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.901+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfd8028020 con 0x7fbfe80d46c0 2024-04-13T03:09:49.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.901+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1dc405a730 con 0x7f1dc80b32d0 2024-04-13T03:09:49.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.901+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42f00ce860 con 0x7f42f006ebb0 2024-04-13T03:09:49.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.908+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24f807c520 con 0x7f25040cd580 2024-04-13T03:09:49.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:49.917+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f48033030 con 0x7f7f4c0cd520 2024-04-13T03:09:50.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.296+0000 7f24c881b640 1 -- 172.21.15.106:0/2707020317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f24c406ce00 con 0x7f24c406c420 2024-04-13T03:09:50.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.296+0000 7f24ab7fe640 1 -- 172.21.15.106:0/2707020317 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24c41f9900 con 0x7f24c406c420 2024-04-13T03:09:50.297 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.299+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 msgr2=0x7f24b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.299+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 0x7f24b006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24b4007fd0 tx=0x7f24b4009040 comp rx=0 tx=0).stop 2024-04-13T03:09:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 msgr2=0x7f24c41f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 0x7f24c41f93c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24bc032000 tx=0x7f24bc004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:50.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 shutdown_connections 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24b006c920 0x7f24b006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c40cb730 0x7f24c41f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24c415aad0 0x7f24c409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 --2- 172.21.15.106:0/2707020317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c406c420 0x7f24c41f93c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 >> 172.21.15.106:0/2707020317 conn(0x7f24c4072100 msgr2=0x7f24c40d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 shutdown_connections 2024-04-13T03:09:50.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.300+0000 7f24a97fa640 1 -- 172.21.15.106:0/2707020317 wait complete. 2024-04-13T03:09:50.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.309+0000 7f383ca88640 1 -- 172.21.15.106:0/3727026122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f381c005850 con 0x7f383814ec60 2024-04-13T03:09:50.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.309+0000 7f3826ffd640 1 -- 172.21.15.106:0/3727026122 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f382c065c00 con 0x7f383814ec60 2024-04-13T03:09:50.310 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.312+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 msgr2=0x7f381806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 0x7f381806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3830000f80 tx=0x7f3830034000 comp rx=0 tx=0).stop 2024-04-13T03:09:50.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 msgr2=0x7f38381efa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 0x7f38381efa90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f383815e370 tx=0x7f382c00ead0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 shutdown_connections 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f381806c9c0 0x7f381806ee80 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3830000f80 tx=0x7f3830034000 comp rx=0 tx=0).stop 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f383814e800 0x7f38381ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f383814ec60 0x7f38381efa90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.313+0000 7f3824ff9640 1 --2- 172.21.15.106:0/3727026122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f383815a7a0 0x7f38381506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.314+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 >> 172.21.15.106:0/3727026122 conn(0x7f38380d02e0 msgr2=0x7f383806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.315+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 shutdown_connections 2024-04-13T03:09:50.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.315+0000 7f3824ff9640 1 -- 172.21.15.106:0/3727026122 wait complete. 2024-04-13T03:09:50.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.316+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f63b80050a0 con 0x7f63b80e30c0 2024-04-13T03:09:50.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.316+0000 7f63aeffd640 1 -- 172.21.15.106:0/655546296 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63c00f7340 con 0x7f63b80e30c0 2024-04-13T03:09:50.317 INFO:teuthology.orchestra.run.smithi106.stdout:55834574857 2024-04-13T03:09:50.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-13T03:09:50.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.319+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 msgr2=0x7f63a006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.319+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 0x7f63a006eeb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005880 tx=0x7f63b40014b0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 msgr2=0x7f63b80df140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 0x7f63b80df140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63c0075b00 tx=0x7f63c0075c20 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 shutdown_connections 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f63a006c9f0 0x7f63a006eeb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63b80e30c0 0x7f63b80df140 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63b8054130 0x7f63b80525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 --2- 172.21.15.106:0/655546296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b803e840 0x7f63b804f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 >> 172.21.15.106:0/655546296 conn(0x7f63b80559e0 msgr2=0x7f63b803d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 shutdown_connections 2024-04-13T03:09:50.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7f63c4a2d640 1 -- 172.21.15.106:0/655546296 wait complete. 2024-04-13T03:09:50.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.320+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbfe815f1c0 con 0x7fbfe80d46c0 2024-04-13T03:09:50.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.321+0000 7fbfddffb640 1 -- 172.21.15.106:0/1608274327 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfd8028020 con 0x7fbfe80d46c0 2024-04-13T03:09:50.322 INFO:teuthology.orchestra.run.smithi106.stdout:55834574857 2024-04-13T03:09:50.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.322+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1dc8162040 con 0x7f1dc80b32d0 2024-04-13T03:09:50.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.323+0000 7f1dbdffb640 1 -- 172.21.15.106:0/942305768 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1dc405a030 con 0x7f1dc80b32d0 2024-04-13T03:09:50.324 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:50.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 msgr2=0x7fbfcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 0x7fbfcc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8156ee0 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2024-04-13T03:09:50.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 msgr2=0x7fbfe8154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800feb0 tx=0x7fbfd8012430 comp rx=0 tx=0).stop 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 shutdown_connections 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfcc06c8f0 0x7fbfcc06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbfe815f4c0 0x7fbfe815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfe815e9d0 0x7fbfe815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 --2- 172.21.15.106:0/1608274327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfe80d46c0 0x7fbfe8154780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.324+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 >> 172.21.15.106:0/1608274327 conn(0x7fbfe808bf10 msgr2=0x7fbfe8160080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.325+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f24e4005850 con 0x7f25040cd580 2024-04-13T03:09:50.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.325+0000 7f24eb7fe640 1 -- 172.21.15.106:0/2236174083 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24f807be20 con 0x7f25040cd580 2024-04-13T03:09:50.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.325+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 shutdown_connections 2024-04-13T03:09:50.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.325+0000 7fbfef5c8640 1 -- 172.21.15.106:0/1608274327 wait complete. 2024-04-13T03:09:50.327 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 msgr2=0x7f1dac06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 0x7f1dac06ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dc81642c0 tx=0x7f1db8009040 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 msgr2=0x7f1dc8157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e8b0 tx=0x7f1dc400ed70 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 shutdown_connections 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1dac06c9c0 0x7f1dac06ee80 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dc81642c0 tx=0x7f1db8009040 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dc81660f0 0x7f1dc82017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dc80b3ce0 0x7f1dc8201270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 --2- 172.21.15.106:0/942305768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc80b32d0 0x7f1dc8157640 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e8b0 tx=0x7f1dc400ed70 comp rx=0 tx=0).stop 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 >> 172.21.15.106:0/942305768 conn(0x7f1dc80b2600 msgr2=0x7f1dc80b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 shutdown_connections 2024-04-13T03:09:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f1db37fe640 1 -- 172.21.15.106:0/942305768 wait complete. 2024-04-13T03:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.327+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f42f0069120 con 0x7f42f006ebb0 2024-04-13T03:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.328+0000 7f42ecff9640 1 -- 172.21.15.106:0/535925622 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42e007bd50 con 0x7f42f006ebb0 2024-04-13T03:09:50.330 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 msgr2=0x7f24f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 0x7f24f006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b840 tx=0x7f24f4008040 comp rx=0 tx=0).stop 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 msgr2=0x7f2504160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009960 tx=0x7f24f80046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 shutdown_connections 2024-04-13T03:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24f006c8f0 0x7f24f006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25041660f0 0x7f2504157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25040cd580 0x7f2504160160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 --2- 172.21.15.106:0/2236174083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25040b32d0 0x7f25041547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.329+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 >> 172.21.15.106:0/2236174083 conn(0x7f25040b2600 msgr2=0x7f25040b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.330+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 shutdown_connections 2024-04-13T03:09:50.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.330+0000 7f25097b4640 1 -- 172.21.15.106:0/2236174083 wait complete. 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 msgr2=0x7f42dc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 0x7f42dc06ee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b52b0 tx=0x7f42e4009060 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 msgr2=0x7f42f0159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42e002be80 tx=0x7f42e00046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 shutdown_connections 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.331+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42dc06c970 0x7f42dc06ee30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42f006b8e0 0x7f42f006bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f015d520 0x7f42f0161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 --2- 172.21.15.106:0/535925622 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42f006ebb0 0x7f42f0159a90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 >> 172.21.15.106:0/535925622 conn(0x7f42f00b2640 msgr2=0x7f42f00b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 shutdown_connections 2024-04-13T03:09:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.332+0000 7f42f63cf640 1 -- 172.21.15.106:0/535925622 wait complete. 2024-04-13T03:09:50.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-13T03:09:50.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.333+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7f2c005850 con 0x7f7f4c0cd520 2024-04-13T03:09:50.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.334+0000 7f7f3dffb640 1 -- 172.21.15.106:0/473559464 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f480093b0 con 0x7f7f4c0cd520 2024-04-13T03:09:50.335 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 msgr2=0x7f7f3006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 0x7f7f3006ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007240 comp rx=0 tx=0).stop 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 msgr2=0x7f7f4c0c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 0x7f7f4c0c7e80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c073920 tx=0x7f7f480046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 shutdown_connections 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c9c0 0x7f7f3006ee80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f4c15caf0 0x7f7f4c1f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f4c0cd520 0x7f7f4c0c7e80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 --2- 172.21.15.106:0/473559464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f4c06fea0 0x7f7f4c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.337+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 >> 172.21.15.106:0/473559464 conn(0x7f7f4c0d25b0 msgr2=0x7f7f4c15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.338+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 shutdown_connections 2024-04-13T03:09:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:50.338+0000 7f7f53680640 1 -- 172.21.15.106:0/473559464 wait complete. 2024-04-13T03:09:50.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-13T03:09:50.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-13T03:09:50.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2024-04-13T03:09:50.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-13T03:09:50.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-04-13T03:09:50.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-13T03:09:51.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-13T03:09:51.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-13T03:09:51.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-13T03:09:51.347 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-13T03:09:51.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-13T03:09:51.350 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-13T03:09:51.352 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-13T03:09:51.353 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.483+0000 7f2c3898c640 1 Processor -- start 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.483+0000 7f2c3898c640 1 -- start start 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c3898c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 0x7f2c340cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c3898c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c340cd910 con 0x7f2c3406cce0 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c3898c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3406f600 con 0x7f2c3406f090 2024-04-13T03:09:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c3898c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2c340b35d0 con 0x7f2c3415a7a0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c32d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2c3406cce0 0x7f2c3415e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56222/0 (socket says 172.21.15.106:56222) 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1563758358 learned_addr learned my addr 172.21.15.106:0/1563758358 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.485+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166428744 0 0) 0x7f2c2c001490 con 0x7f2c3406cce0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c1c003520 con 0x7f2c3406cce0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2695954667 0 0) 0x7f2c1c003520 con 0x7f2c3406cce0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 msgr2=0x7f2c340cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 0x7f2c340cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 >> v1:172.21.15.114:6789/0 conn(0x7f2c3406f090 legacy=0x7f2c3406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.486+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3406baf0 con 0x7f2c3406cce0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2c2c001bc0 con 0x7f2c3406cce0 2024-04-13T03:09:51.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 >> v1:172.21.15.106:6789/0 conn(0x7f2c3406cce0 legacy=0x7f2c3415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c1c0037a0 0x7f2c1c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c1c00b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 0x7f2c1c00d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c00de90 con 0x7f2c1c004200 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c00e040 con 0x7f2c1c0037a0 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.487+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c00e240 con 0x7f2c3415a7a0 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c3259c640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c1c0037a0 0x7f2c1c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c1c00b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c32d9d640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 0x7f2c1c00d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 -- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 msgr2=0x7f2c1c00d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c3415a7a0 0x7f2c1c00d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 -- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c1c0037a0 msgr2=0x7f2c1c003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c1c0037a0 0x7f2c1c003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.488+0000 7f2c31d9b640 1 -- 172.21.15.106:0/1563758358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c003520 con 0x7f2c1c004200 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.490+0000 7f2c31d9b640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c1c00b010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001650 tx=0x7f2c1c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ede50914f7e71ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.490+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24001220 con 0x7f2c1c004200 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.491+0000 7f2c3159a640 1 -- 172.21.15.106:0/1563758358 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c24005290 con 0x7f2c1c004200 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.492+0000 7f2c3898c640 1 -- 172.21.15.106:0/1563758358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 msgr2=0x7f2c1c00b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.492+0000 7f2c3898c640 1 --2- 172.21.15.106:0/1563758358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c1c00b010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001650 tx=0x7f2c1c004820 comp rx=0 tx=0).stop 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.492+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1563758358 reap_dead start 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.492+0000 7f2c3898c640 1 -- 172.21.15.106:0/1563758358 shutdown_connections 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.492+0000 7f2c3898c640 1 -- 172.21.15.106:0/1563758358 >> 172.21.15.106:0/1563758358 conn(0x7f2c340b2600 msgr2=0x7f2c340d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.493+0000 7f2c3898c640 1 -- 172.21.15.106:0/1563758358 shutdown_connections 2024-04-13T03:09:51.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.493+0000 7f2c3898c640 1 -- 172.21.15.106:0/1563758358 wait complete. 2024-04-13T03:09:51.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.495+0000 7f2c3898c640 1 Processor -- start 2024-04-13T03:09:51.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.495+0000 7f2c3898c640 1 -- start start 2024-04-13T03:09:51.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c3415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c340ae8c0 0x7f2c340aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406d170 con 0x7f2c1c004200 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406c0b0 con 0x7f2c340acad0 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c3898c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3415ab80 con 0x7f2c340ae8c0 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c32d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c32d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53360/0 (socket says 172.21.15.106:53360) 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1424149210 learned_addr learned my addr 172.21.15.106:0/1424149210 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.496+0000 7f2c31d9b640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c340ae8c0 0x7f2c340aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c340ae8c0 msgr2=0x7f2c340aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c340ae8c0 0x7f2c340aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 msgr2=0x7f2c3415aed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c3415aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 -- 172.21.15.106:0/1424149210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c340ad820 con 0x7f2c340acad0 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.497+0000 7f2c32d9d640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0013d0 tx=0x7f2c2c00e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.498+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c007aa0 con 0x7f2c340acad0 2024-04-13T03:09:51.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.498+0000 7f2c3898c640 1 -- 172.21.15.106:0/1424149210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c340adb00 con 0x7f2c340acad0 2024-04-13T03:09:51.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.498+0000 7f2c3898c640 1 -- 172.21.15.106:0/1424149210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c340ae0c0 con 0x7f2c340acad0 2024-04-13T03:09:51.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.501+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c018070 con 0x7f2c340acad0 2024-04-13T03:09:51.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.501+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c013420 con 0x7f2c340acad0 2024-04-13T03:09:51.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.501+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c2c0135e0 con 0x7f2c340acad0 2024-04-13T03:09:51.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.502+0000 7f2c22ffd640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 0x7f2c1c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.502+0000 7f2c3259c640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 0x7f2c1c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.503+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c2c08c230 con 0x7f2c340acad0 2024-04-13T03:09:51.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.503+0000 7f2c3259c640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 0x7f2c1c0709b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c28004700 tx=0x7f2c28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.504+0000 7f2c3898c640 1 -- 172.21.15.106:0/1424149210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18005290 con 0x7f2c340acad0 2024-04-13T03:09:51.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.509+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c340cd910 con 0x7f2c340acad0 2024-04-13T03:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.534+0000 7f81fb27c640 1 Processor -- start 2024-04-13T03:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.534+0000 7f81fb27c640 1 -- start start 2024-04-13T03:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81fb27c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f81f406d990 0x7f81f40ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81fb27c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f40d1510 con 0x7f81f40cc990 2024-04-13T03:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81fb27c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f40cd500 con 0x7f81f4162d00 2024-04-13T03:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81fb27c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f81f40cd7e0 con 0x7f81f406d990 2024-04-13T03:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81f9a79640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f81f40cc990 0x7f81f406d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56232/0 (socket says 172.21.15.106:56232) 2024-04-13T03:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81f9a79640 1 -- 172.21.15.106:0/3255849882 learned_addr learned my addr 172.21.15.106:0/3255849882 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.536+0000 7f81faa7b640 1 --2- 172.21.15.106:0/3255849882 >> v2:172.21.15.114:3300/0 conn(0x7f81f406d990 0x7f81f40ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f81f9278640 1 -- 172.21.15.106:0/3255849882 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770746934 0 0) 0x7f81f40d1510 con 0x7f81f40cc990 2024-04-13T03:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f81f9278640 1 -- 172.21.15.106:0/3255849882 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81e0003520 con 0x7f81f40cc990 2024-04-13T03:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f9a9b6d5640 1 Processor -- start 2024-04-13T03:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f9a9b6d5640 1 -- start start 2024-04-13T03:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f81faa7b640 1 -- 172.21.15.106:0/3255849882 >> v1:172.21.15.114:6789/0 conn(0x7f81f4162d00 legacy=0x7f81f40ce5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T03:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f81faa7b640 1 -- 172.21.15.106:0/3255849882 >> v1:172.21.15.106:6789/0 conn(0x7f81f40cc990 legacy=0x7f81f406d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.537+0000 7f81faa7b640 1 -- 172.21.15.106:0/3255849882 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f40cdac0 con 0x7f81f406d990 2024-04-13T03:09:51.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f9a9b6d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 0x7f9a940704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f9a9b6d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f81faa7b640 1 --2- 172.21.15.106:0/3255849882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f406d990 0x7f81f40ccdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009d90 tx=0x7f81ec02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29bd48b36ceb8c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f9a9b6d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9406ee40 con 0x7f9a94162cf0 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f9a9b6d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9a941677f0 con 0x7f9a94167280 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.538+0000 7f9a9b6d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9406d280 con 0x7f9a94072570 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a98c70640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 0x7f9a940704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a99c72640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a98c70640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 0x7f9a940704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45274/0 (socket says 172.21.15.106:45274) 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a98c70640 1 -- 172.21.15.106:0/4201945545 learned_addr learned my addr 172.21.15.106:0/4201945545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a99c72640 1 -- 172.21.15.106:0/4201945545 >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 msgr2=0x7f9a940704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a99c72640 1 --2- 172.21.15.106:0/4201945545 >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 0x7f9a940704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a99c72640 1 -- 172.21.15.106:0/4201945545 >> v1:172.21.15.106:6789/0 conn(0x7f9a94162cf0 legacy=0x7f9a9406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f9a99c72640 1 -- 172.21.15.106:0/4201945545 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9406f120 con 0x7f9a94072570 2024-04-13T03:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f81f9278640 1 -- 172.21.15.106:0/3255849882 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec039070 con 0x7f81f406d990 2024-04-13T03:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f81f9278640 1 -- 172.21.15.106:0/3255849882 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec02cc60 con 0x7f81f406d990 2024-04-13T03:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f371dc7c640 1 Processor -- start 2024-04-13T03:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 -- 172.21.15.106:0/3255849882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f406d990 msgr2=0x7f81f40ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 --2- 172.21.15.106:0/3255849882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f406d990 0x7f81f40ccdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009d90 tx=0x7f81ec02beb0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f8631bdc640 1 Processor -- start 2024-04-13T03:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f8631bdc640 1 -- start start 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f3b0cc15640 1 Processor -- start 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f3b0cc15640 1 -- start start 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 -- 172.21.15.106:0/3255849882 shutdown_connections 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 --2- 172.21.15.106:0/3255849882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f406d990 0x7f81f40ccdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 -- 172.21.15.106:0/3255849882 >> 172.21.15.106:0/3255849882 conn(0x7f81f40cedc0 msgr2=0x7f81f40cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 -- 172.21.15.106:0/3255849882 shutdown_connections 2024-04-13T03:09:51.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.539+0000 7f371dc7c640 1 -- start start 2024-04-13T03:09:51.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.540+0000 7f81fb27c640 1 -- 172.21.15.106:0/3255849882 wait complete. 2024-04-13T03:09:51.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371dc7c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37180cfac0 0x7f371806e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371dc7c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37180afaa0 con 0x7f37180cd0a0 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371dc7c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37181666b0 con 0x7f3718162c10 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371dc7c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3718166990 con 0x7f37180cfac0 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f9a99c72640 1 --2- 172.21.15.106:0/4201945545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a940709e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0097a0 tx=0x7f9a8c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aae05478302e340 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371cc7a640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3718162c10 0x7f37180cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53868/0 (socket says 172.21.15.106:53868) 2024-04-13T03:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f371cc7a640 1 -- 172.21.15.106:0/881805149 learned_addr learned my addr 172.21.15.106:0/881805149 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f9a8bfff640 1 -- 172.21.15.106:0/4201945545 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c038070 con 0x7f9a94072570 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f8631bdc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f862c0c77a0 0x7f862c0d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f8631bdc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f862c073460 0x7f862c0d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f8631bdc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0cfd00 con 0x7f862c0b5ff0 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f8631bdc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f862c0d38f0 con 0x7f862c0c77a0 2024-04-13T03:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f8631bdc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f862c0d3bd0 con 0x7f862c073460 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f9a8bfff640 1 -- 172.21.15.106:0/4201945545 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c031700 con 0x7f9a94072570 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f3b0cc15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f3b0cc15640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3b080d0490 0x7f3b08167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f3b0cc15640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b080b3910 con 0x7f3b080c7560 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f3b0cc15640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3b081679a0 con 0x7f3b080d0490 2024-04-13T03:09:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.541+0000 7f3b0cc15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b080b36c0 con 0x7f3b08162dd0 2024-04-13T03:09:51.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f862bfff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f862c0b5ff0 0x7f862c0d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53874/0 (socket says 172.21.15.106:53874) 2024-04-13T03:09:51.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f862bfff640 1 -- 172.21.15.106:0/2582777371 learned_addr learned my addr 172.21.15.106:0/2582777371 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:51.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633889757 0 0) 0x7f37181666b0 con 0x7f3718162c10 2024-04-13T03:09:51.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3704003520 con 0x7f3718162c10 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49884/0 (socket says 172.21.15.106:49884) 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 -- 172.21.15.106:0/2107434855 learned_addr learned my addr 172.21.15.106:0/2107434855 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b07fff640 1 --2- 172.21.15.106:0/2107434855 >> v2:172.21.15.114:3300/0 conn(0x7f3b080d0490 0x7f3b08167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633692646 0 0) 0x7f37180afaa0 con 0x7f37180cd0a0 2024-04-13T03:09:51.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37181666b0 con 0x7f37180cd0a0 2024-04-13T03:09:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f9c1c236640 1 Processor -- start 2024-04-13T03:09:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f81fb27c640 1 Processor -- start 2024-04-13T03:09:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f9c1c236640 1 -- start start 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.1 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4060750104 0 0) 0x7f3704003520 con 0x7f3718162c10 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37180cfac0 msgr2=0x7f371806e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37180cfac0 0x7f371806e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f81fb27c640 1 -- start start 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f9a8bfff640 1 -- 172.21.15.106:0/4201945545 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c031ad0 con 0x7f9a94072570 2024-04-13T03:09:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/4201945545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 msgr2=0x7f9a940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/4201945545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a940709e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0097a0 tx=0x7f9a8c02b7a0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 >> v1:172.21.15.106:6789/0 conn(0x7f37180cd0a0 legacy=0x7f371806dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718166c70 con 0x7f3718162c10 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.1 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f37100019f0 con 0x7f3718162c10 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 >> v1:172.21.15.114:6789/0 conn(0x7f3718162c10 legacy=0x7f37180cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3704003b70 0x7f3704003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3704004800 0x7f3704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f370400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3718166990 con 0x7f3704003b70 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f370400c110 con 0x7f3704005210 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f370400c310 con 0x7f3704004800 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 -- 172.21.15.106:0/2107434855 >> v2:172.21.15.114:3300/0 conn(0x7f3b080d0490 msgr2=0x7f3b08167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862b7fe640 1 --2- 172.21.15.106:0/2582777371 >> v2:172.21.15.106:3300/0 conn(0x7f862c0c77a0 0x7f862c0d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593325433 0 0) 0x7f862c0cfd00 con 0x7f862c0b5ff0 2024-04-13T03:09:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8614003520 con 0x7f862c0b5ff0 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862affd640 1 --2- 172.21.15.106:0/2582777371 >> v2:172.21.15.114:3300/0 conn(0x7f862c073460 0x7f862c0d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/4201945545 shutdown_connections 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/4201945545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a940709e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/4201945545 >> v2:172.21.15.106:3300/0 conn(0x7f9a94167280 0x7f9a940704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/4201945545 >> 172.21.15.106:0/4201945545 conn(0x7f9a94071870 msgr2=0x7f9a94071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/4201945545 shutdown_connections 2024-04-13T03:09:51.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/4201945545 wait complete. 2024-04-13T03:09:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 --2- 172.21.15.106:0/2107434855 >> v2:172.21.15.114:3300/0 conn(0x7f3b080d0490 0x7f3b08167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 -- 172.21.15.106:0/2107434855 >> v1:172.21.15.106:6789/0 conn(0x7f3b080c7560 legacy=0x7f3b080cfd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T03:09:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862b7fe640 1 -- 172.21.15.106:0/2582777371 >> v2:172.21.15.114:3300/0 conn(0x7f862c073460 msgr2=0x7f862c0d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 <== mon.1 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1363716887 0 0) 0x7f8614003520 con 0x7f862c0b5ff0 2024-04-13T03:09:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862b7fe640 1 --2- 172.21.15.106:0/2582777371 >> v2:172.21.15.114:3300/0 conn(0x7f862c073460 0x7f862c0d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862b7fe640 1 -- 172.21.15.106:0/2582777371 >> v1:172.21.15.114:6789/0 conn(0x7f862c0b5ff0 legacy=0x7f862c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f862b7fe640 1 -- 172.21.15.106:0/2582777371 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0d3eb0 con 0x7f862c0c77a0 2024-04-13T03:09:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.543+0000 7f9c1c236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c0c0e3160 0x7f9c0c0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.542+0000 7f3b077fe640 1 -- 172.21.15.106:0/2107434855 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080b3bf0 con 0x7f3b08162dd0 2024-04-13T03:09:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f3b077fe640 1 --2- 172.21.15.106:0/2107434855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009870 tx=0x7f3afc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97717602d6320cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 0x7f81f41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f9c1c236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c058810 con 0x7f9c0c053cb0 2024-04-13T03:09:51.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f9c197d1640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9c0c03c0a0 0x7f9c0c058050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:53890/0 (socket says 172.21.15.106:53890) 2024-04-13T03:09:51.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f9c197d1640 1 -- 172.21.15.106:0/2011494257 learned_addr learned my addr 172.21.15.106:0/2011494257 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:51.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c058af0 con 0x7f9c0c03c0a0 2024-04-13T03:09:51.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c007b70 con 0x7f9c0c0e3160 2024-04-13T03:09:51.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f3b067fc640 1 -- 172.21.15.106:0/2107434855 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0317c0 con 0x7f3b08162dd0 2024-04-13T03:09:51.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7fbb448d5640 1 Processor -- start 2024-04-13T03:09:51.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7fbb448d5640 1 -- start start 2024-04-13T03:09:51.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81f40cc990 0x7f81f41fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 0x7f81f41f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40b51d0 con 0x7f81f40cc990 2024-04-13T03:09:51.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40ce080 con 0x7f81f4162d00 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fb27c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f81f40b5350 con 0x7f81f406d990 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fa27a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 0x7f81f41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fa27a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 0x7f81f41524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49902/0 (socket says 172.21.15.106:49902) 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f81fa27a640 1 -- 172.21.15.106:0/1756713906 learned_addr learned my addr 172.21.15.106:0/1756713906 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 0x7f81f41f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81f9a79640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81f40cc990 0x7f81f41fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 -- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 msgr2=0x7f81f41524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 0x7f81f41524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f9c19fd2640 1 --2- 172.21.15.106:0/2011494257 >> v2:172.21.15.106:3300/0 conn(0x7f9c0c0e3160 0x7f9c0c0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 -- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81f40cc990 msgr2=0x7f81f41fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f3b067fc640 1 -- 172.21.15.106:0/2107434855 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc037070 con 0x7f3b08162dd0 2024-04-13T03:09:51.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 -- 172.21.15.106:0/2107434855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 msgr2=0x7f3b080b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f370400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371cc7a640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3704003b70 0x7f3704003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 -- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3704004800 msgr2=0x7f3704004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3704004800 0x7f3704004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 -- 172.21.15.106:0/881805149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3704003b70 msgr2=0x7f3704003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3704003b70 0x7f3704003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.544+0000 7f371d47b640 1 -- 172.21.15.106:0/881805149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718166c70 con 0x7f3704005210 2024-04-13T03:09:51.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b067fc640 1 -- 172.21.15.106:0/2107434855 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0317c0 con 0x7f3b08162dd0 2024-04-13T03:09:51.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/2107434855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009870 tx=0x7f3afc02b860 comp rx=0 tx=0).stop 2024-04-13T03:09:51.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81f40cc990 0x7f81f41fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 -- 172.21.15.106:0/1756713906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f4153430 con 0x7f81f4162d00 2024-04-13T03:09:51.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 -- 172.21.15.106:0/2107434855 shutdown_connections 2024-04-13T03:09:51.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/2107434855 >> v2:172.21.15.114:3300/0 conn(0x7f3b080d0490 0x7f3b08167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/2107434855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08162dd0 0x7f3b080b6a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 -- 172.21.15.106:0/2107434855 >> 172.21.15.106:0/2107434855 conn(0x7f3b0809d8d0 msgr2=0x7f3b080b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f9c18fd0640 1 -- 172.21.15.106:0/2011494257 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558851921 0 0) 0x7f9c0c058af0 con 0x7f9c0c03c0a0 2024-04-13T03:09:51.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f9c18fd0640 1 -- 172.21.15.106:0/2011494257 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c00003520 con 0x7f9c0c03c0a0 2024-04-13T03:09:51.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81faa7b640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 0x7f81f41f8710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81ec033000 tx=0x7f81ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f3b0cc15640 1 -- 172.21.15.106:0/2107434855 shutdown_connections 2024-04-13T03:09:51.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec044020 con 0x7f81f4162d00 2024-04-13T03:09:51.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec032870 con 0x7f81f4162d00 2024-04-13T03:09:51.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec039040 con 0x7f81f4162d00 2024-04-13T03:09:51.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb448d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 0x7fbb38006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb448d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb38054560 0x7fbb3804f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb448d5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb38005ec0 con 0x7fbb38007460 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb448d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbb380e7b90 con 0x7fbb38054560 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb448d5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbb38006a90 con 0x7fbb38048100 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb38054560 0x7fbb3804f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb38054560 0x7fbb3804f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45322/0 (socket says 172.21.15.106:45322) 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 -- 172.21.15.106:0/740824450 learned_addr learned my addr 172.21.15.106:0/740824450 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/740824450 >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 0x7fbb38006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f9a9b6d5640 1 Processor -- start 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.545+0000 7f9a9b6d5640 1 -- start start 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a941fb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 0x7f9a940d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 0x7f9a941fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9406e750 con 0x7f9a94167280 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9406e100 con 0x7f9a94162cf0 2024-04-13T03:09:51.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a9b6d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9406e8d0 con 0x7f9a94072570 2024-04-13T03:09:51.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a98c70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 0x7f9a940d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a99c72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 0x7f9a941fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a99c72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 0x7f9a941fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45326/0 (socket says 172.21.15.106:45326) 2024-04-13T03:09:51.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a99c72640 1 -- 172.21.15.106:0/89399009 learned_addr learned my addr 172.21.15.106:0/89399009 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f4152b90 con 0x7f81f4162d00 2024-04-13T03:09:51.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 -- 172.21.15.106:0/740824450 >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 msgr2=0x7fbb38006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 --2- 172.21.15.106:0/740824450 >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 0x7fbb38006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 -- 172.21.15.106:0/740824450 >> v1:172.21.15.114:6789/0 conn(0x7fbb38007460 legacy=0x7fbb38053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:51.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f4153150 con 0x7f81f4162d00 2024-04-13T03:09:51.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f371d47b640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f370400b9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37180cfea0 tx=0x7f3704005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c768d6d24a5b211 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7fbb3ed9d640 1 -- 172.21.15.106:0/740824450 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb38053280 con 0x7fbb38054560 2024-04-13T03:09:51.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f3b0cc15640 1 -- 172.21.15.106:0/2107434855 wait complete. 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9c19fd2640 1 -- 172.21.15.106:0/2011494257 >> v1:172.21.15.114:6789/0 conn(0x7f9c0c03c0a0 legacy=0x7f9c0c058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9c19fd2640 1 -- 172.21.15.106:0/2011494257 >> v1:172.21.15.106:6789/0 conn(0x7f9c0c053cb0 legacy=0x7f9c0c054100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9c19fd2640 1 -- 172.21.15.106:0/2011494257 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c006360 con 0x7f9c0c0e3160 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.546+0000 7f9a99471640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a941fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 -- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 msgr2=0x7f9a941fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a941fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 -- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 msgr2=0x7f9a941fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 0x7f9a941fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 -- 172.21.15.106:0/89399009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941650a0 con 0x7f9a94162cf0 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/740824450 >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 0x7fbb38006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700037070 con 0x7f3704005210 2024-04-13T03:09:51.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9a98c70640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 0x7f9a940d1cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000efc0 tx=0x7f9a9000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000cda0 con 0x7f9a94162cf0 2024-04-13T03:09:51.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f37177fe640 1 -- 172.21.15.106:0/881805149 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3700032600 con 0x7f3704005210 2024-04-13T03:09:51.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f9c19fd2640 1 --2- 172.21.15.106:0/2011494257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c0e7780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c14065750 tx=0x7f9c140a73d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9cacce85b9487c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f862b7fe640 1 --2- 172.21.15.106:0/2582777371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c0c77a0 0x7f862c0d42c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ba600 tx=0x7f861c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd5e3e16970f4f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f404fdc0 con 0x7f81f4162d00 2024-04-13T03:09:51.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7fbb3ed9d640 1 --2- 172.21.15.106:0/740824450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38054560 0x7fbb3804f930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006c10 tx=0x7fbb3402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c576ceeae0591e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a94165380 con 0x7f9a94162cf0 2024-04-13T03:09:51.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f371dc7c640 1 -- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 msgr2=0x7f370400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f371dc7c640 1 --2- 172.21.15.106:0/881805149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f370400b9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37180cfea0 tx=0x7f3704005830 comp rx=0 tx=0).stop 2024-04-13T03:09:51.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f371d47b640 1 -- 172.21.15.106:0/881805149 reap_dead start 2024-04-13T03:09:51.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a94165940 con 0x7f9a94162cf0 2024-04-13T03:09:51.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f371dc7c640 1 -- 172.21.15.106:0/881805149 shutdown_connections 2024-04-13T03:09:51.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f371dc7c640 1 -- 172.21.15.106:0/881805149 >> 172.21.15.106:0/881805149 conn(0x7f37180b1d00 msgr2=0x7f37180b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90019070 con 0x7f9a94162cf0 2024-04-13T03:09:51.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.547+0000 7f3b0cc15640 1 Processor -- start 2024-04-13T03:09:51.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f3b0cc15640 1 -- start start 2024-04-13T03:09:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90014680 con 0x7f9a94162cf0 2024-04-13T03:09:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9c18fd0640 1 -- 172.21.15.106:0/2011494257 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140b3070 con 0x7f9c0c0e3160 2024-04-13T03:09:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9c18fd0640 1 -- 172.21.15.106:0/2011494257 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c140ac7d0 con 0x7f9c0c0e3160 2024-04-13T03:09:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f9c18fd0640 1 -- 172.21.15.106:0/2011494257 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140acc40 con 0x7f9c0c0e3160 2024-04-13T03:09:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81ec005d10 con 0x7f81f4162d00 2024-04-13T03:09:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b080c7560 0x7f3b0815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 0x7f3b08161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 0x7f3b080cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080b6570 con 0x7f3b080d0490 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080b5fb0 con 0x7f3b080c7560 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b0cc15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b081631b0 con 0x7f3b08161750 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 0x7f3b08161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 0x7f3b080cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 0x7f3b08161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45336/0 (socket says 172.21.15.106:45336) 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c038070 con 0x7f862c0c77a0 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c031540 con 0x7f862c0c77a0 2024-04-13T03:09:51.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7f862a7fc640 1 -- 172.21.15.106:0/2582777371 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c02f050 con 0x7f862c0c77a0 2024-04-13T03:09:51.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7fbb3d59a640 1 -- 172.21.15.106:0/740824450 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34034070 con 0x7fbb38054560 2024-04-13T03:09:51.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7fbb3d59a640 1 -- 172.21.15.106:0/740824450 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb34030860 con 0x7fbb38054560 2024-04-13T03:09:51.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.548+0000 7fbb3d59a640 1 -- 172.21.15.106:0/740824450 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34030c30 con 0x7fbb38054560 2024-04-13T03:09:51.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b06ffd640 1 -- 172.21.15.106:0/752697680 learned_addr learned my addr 172.21.15.106:0/752697680 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f81eaffd640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 0x7f81d806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f81ec0ae8b0 con 0x7f81f4162d00 2024-04-13T03:09:51.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f81fa27a640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 0x7f81d806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f371dc7c640 1 -- 172.21.15.106:0/881805149 shutdown_connections 2024-04-13T03:09:51.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0e3160 msgr2=0x7f9c0c0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9c1c236640 1 --2- 172.21.15.106:0/2011494257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c0e7780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c14065750 tx=0x7f9c140a73d0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f8631bdc640 1 -- 172.21.15.106:0/2582777371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c0c77a0 msgr2=0x7f862c0d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f8631bdc640 1 --2- 172.21.15.106:0/2582777371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c0c77a0 0x7f862c0d42c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ba600 tx=0x7f861c02b860 comp rx=0 tx=0).stop 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 -- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b080c7560 msgr2=0x7f3b0815e3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b080c7560 0x7f3b0815e3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 -- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 msgr2=0x7f3b08161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 0x7f3b08161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f3b07fff640 1 -- 172.21.15.106:0/752697680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080cd020 con 0x7f3b08161750 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f3b07fff640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 0x7f3b080cc140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3af800e7f0 tx=0x7f3af800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f8631bdc640 1 -- 172.21.15.106:0/2582777371 shutdown_connections 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f81fa27a640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 0x7f81d806eeb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81e40099a0 tx=0x7f81e40090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 shutdown_connections 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9c1c236640 1 --2- 172.21.15.106:0/2011494257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c0e7780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017070 con 0x7f3b08161750 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af800c730 con 0x7f3b08161750 2024-04-13T03:09:51.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af800cb40 con 0x7f3b08161750 2024-04-13T03:09:51.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 >> 172.21.15.106:0/2011494257 conn(0x7f9c0c058ee0 msgr2=0x7f9c0c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a90005d10 con 0x7f9a94162cf0 2024-04-13T03:09:51.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a74005290 con 0x7f9a94162cf0 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f9a89ffb640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 0x7f9a7806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9a9008c300 con 0x7f9a94162cf0 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f8631bdc640 1 --2- 172.21.15.106:0/2582777371 >> v2:172.21.15.114:3300/0 conn(0x7f862c073460 0x7f862c0d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f8631bdc640 1 --2- 172.21.15.106:0/2582777371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c0c77a0 0x7f862c0d42c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.550+0000 7f8631bdc640 1 -- 172.21.15.106:0/2582777371 >> 172.21.15.106:0/2582777371 conn(0x7f862c08bf10 msgr2=0x7f862c09d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 -- 172.21.15.106:0/740824450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38054560 msgr2=0x7fbb3804f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 --2- 172.21.15.106:0/740824450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38054560 0x7fbb3804f930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006c10 tx=0x7fbb3402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.549+0000 7f371dc7c640 1 -- 172.21.15.106:0/881805149 wait complete. 2024-04-13T03:09:51.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 shutdown_connections 2024-04-13T03:09:51.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f9a99471640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 0x7f9a7806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 -- 172.21.15.106:0/740824450 shutdown_connections 2024-04-13T03:09:51.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 --2- 172.21.15.106:0/740824450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38054560 0x7fbb3804f930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 --2- 172.21.15.106:0/740824450 >> v2:172.21.15.114:3300/0 conn(0x7fbb38048100 0x7fbb38006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 -- 172.21.15.106:0/740824450 >> 172.21.15.106:0/740824450 conn(0x7fbb38009fe0 msgr2=0x7fbb3800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f8631bdc640 1 -- 172.21.15.106:0/2582777371 shutdown_connections 2024-04-13T03:09:51.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f8631bdc640 1 -- 172.21.15.106:0/2582777371 wait complete. 2024-04-13T03:09:51.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f371dc7c640 1 Processor -- start 2024-04-13T03:09:51.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 -- 172.21.15.106:0/740824450 shutdown_connections 2024-04-13T03:09:51.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7f9c1c236640 1 -- 172.21.15.106:0/2011494257 wait complete. 2024-04-13T03:09:51.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.551+0000 7fbb448d5640 1 -- 172.21.15.106:0/740824450 wait complete. 2024-04-13T03:09:51.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.552+0000 7f9a99471640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 0x7f9a7806eeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9a84007fd0 tx=0x7f9a84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.552+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b080cd300 con 0x7f3b08161750 2024-04-13T03:09:51.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.552+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b080c8720 con 0x7f3b08161750 2024-04-13T03:09:51.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.552+0000 7f371dc7c640 1 -- start start 2024-04-13T03:09:51.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f9c1c236640 1 Processor -- start 2024-04-13T03:09:51.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f9c1c236640 1 -- start start 2024-04-13T03:09:51.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f8631bdc640 1 Processor -- start 2024-04-13T03:09:51.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7fbb448d5640 1 Processor -- start 2024-04-13T03:09:51.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7fbb448d5640 1 -- start start 2024-04-13T03:09:51.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b080c8920 con 0x7f3b08161750 2024-04-13T03:09:51.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f8631bdc640 1 -- start start 2024-04-13T03:09:51.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f371dc7c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f37180995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.553+0000 7f371dc7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371dc7c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f371808cda0 0x7f3718165920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371dc7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f371806c7c0 con 0x7f371808c920 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45352/0 (socket says 172.21.15.106:45352) 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3718167470 con 0x7f3704005210 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 -- 172.21.15.106:0/1933841444 learned_addr learned my addr 172.21.15.106:0/1933841444 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f371806cb30 con 0x7f371808cda0 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371cc7a640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f37180995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3af801a070 con 0x7f3b08161750 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 0x7fbb380d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 0x7fbb38036ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38006720 con 0x7fbb38048100 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38006440 con 0x7fbb38007460 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb448d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3804fe70 con 0x7fbb38054560 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb3dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb3dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45354/0 (socket says 172.21.15.106:45354) 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7fbb3dd9b640 1 -- 172.21.15.106:0/2364642455 learned_addr learned my addr 172.21.15.106:0/2364642455 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 0x7fbb380d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3ed9d640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 0x7fbb38036ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 0x7f9c0c0e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c051be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c19fd2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c19fd2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49942/0 (socket says 172.21.15.106:49942) 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c19fd2640 1 -- 172.21.15.106:0/3431444255 learned_addr learned my addr 172.21.15.106:0/3431444255 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0ee050 con 0x7f9c0c053cb0 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0e3540 con 0x7f9c0c0e3160 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0ee1d0 con 0x7f9c0c03c0a0 2024-04-13T03:09:51.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 0x7f9c0c0e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 0x7f862c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f862c0c77a0 0x7f862c0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f371d47b640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f371808cda0 0x7f3718165920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 -- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f371808cda0 msgr2=0x7f3718165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f371808cda0 0x7f3718165920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 -- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 msgr2=0x7f37180995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f37180995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371808d370 con 0x7f371808c920 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.554+0000 7f3717fff640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f370800c580 tx=0x7f370800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d1650 con 0x7f862c073460 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d14d0 con 0x7f862c0b5ff0 2024-04-13T03:09:51.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f3aeffff640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 0x7f3af406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3af808bfe0 con 0x7f3b08161750 2024-04-13T03:09:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f8631bdc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0d17d0 con 0x7f862c0c77a0 2024-04-13T03:09:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708017070 con 0x7f371808c920 2024-04-13T03:09:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37080035f0 con 0x7f371808c920 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3dd9b640 1 -- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 msgr2=0x7fbb38036ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3dd9b640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 0x7fbb38036ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3dd9b640 1 -- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 msgr2=0x7fbb380d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3dd9b640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 0x7fbb380d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3dd9b640 1 -- 172.21.15.106:0/2364642455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb38035490 con 0x7fbb38048100 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 0x7fbb380d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7fbb3ed9d640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 0x7fbb38036ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 -- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 msgr2=0x7f9c0c0d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 -- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0e3160 msgr2=0x7f9c0c051be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c051be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c0d8440 con 0x7f9c0c053cb0 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c19fd2640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f3b077fe640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 0x7f3af406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371808d650 con 0x7f371808c920 2024-04-13T03:09:51.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371808dc10 con 0x7f371808c920 2024-04-13T03:09:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f862affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f862affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53458/0 (socket says 172.21.15.106:53458) 2024-04-13T03:09:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f862affd640 1 -- 172.21.15.106:0/1031834123 learned_addr learned my addr 172.21.15.106:0/1031834123 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.555+0000 7f9c197d1640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 0x7f9c0c0e2370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c460 tx=0x7f9c1000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81ec04e070 con 0x7f81f4162d00 2024-04-13T03:09:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 -- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f862c0c77a0 msgr2=0x7f862c0caa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T03:09:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f862c0c77a0 0x7f862c0caa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 -- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 msgr2=0x7f862c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f3b077fe640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 0x7f3af406ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3afc036000 tx=0x7f3afc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862b7fe640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 0x7f862c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37080039e0 con 0x7f371808c920 2024-04-13T03:09:51.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7fbb3dd9b640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb40050fc0 tx=0x7fbb400694f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 0x7f862c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 -- 172.21.15.106:0/1031834123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c160080 con 0x7f862c0b5ff0 2024-04-13T03:09:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862b7fe640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 0x7f862c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f862affd640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f862400efc0 tx=0x7f862400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc005290 con 0x7f371808c920 2024-04-13T03:09:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40074480 con 0x7fbb38048100 2024-04-13T03:09:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.556+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10017070 con 0x7f9c0c053cb0 2024-04-13T03:09:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c10005ce0 con 0x7f9c0c053cb0 2024-04-13T03:09:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007270 con 0x7f9c0c053cb0 2024-04-13T03:09:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c0d8720 con 0x7f9c0c053cb0 2024-04-13T03:09:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb40074640 con 0x7fbb38048100 2024-04-13T03:09:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb38035770 con 0x7fbb38048100 2024-04-13T03:09:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f370801a070 con 0x7f371808c920 2024-04-13T03:09:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.557+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c0d8ce0 con 0x7f9c0c053cb0 2024-04-13T03:09:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb38035d30 con 0x7fbb38048100 2024-04-13T03:09:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f37157fa640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 0x7f36f806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f370808b160 con 0x7f371808c920 2024-04-13T03:09:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f371cc7a640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 0x7f36f806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.559+0000 7f371cc7a640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 0x7f36f806ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3710001370 tx=0x7f3710009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.559+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c1001a070 con 0x7f9c0c053cb0 2024-04-13T03:09:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40074ab0 con 0x7fbb38048100 2024-04-13T03:09:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.559+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c005390 con 0x7f9c0c053cb0 2024-04-13T03:09:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.559+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb380053c0 con 0x7fbb38048100 2024-04-13T03:09:51.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.559+0000 7f9c0a7fc640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b080c8920 con 0x7f3b08161750 2024-04-13T03:09:51.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624018020 con 0x7f862c0b5ff0 2024-04-13T03:09:51.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8624014480 con 0x7f862c0b5ff0 2024-04-13T03:09:51.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.558+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624014830 con 0x7f862c0b5ff0 2024-04-13T03:09:51.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a90010040 con 0x7f9a94162cf0 2024-04-13T03:09:51.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f9c19fd2640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c164dd0 con 0x7f862c0b5ff0 2024-04-13T03:09:51.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c165390 con 0x7f862c0b5ff0 2024-04-13T03:09:51.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7f9c19fd2640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9c1407ad60 tx=0x7f9c1406f030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.560+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb40093400 con 0x7fbb38048100 2024-04-13T03:09:51.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.561+0000 7fbb2effd640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 0x7fbb2006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.561+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 0x7fbb2006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.562+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c04fdc0 con 0x7f862c0b5ff0 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.562+0000 7fbb3e59c640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 0x7fbb2006ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb3804b250 tx=0x7fbb30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.562+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8624005d10 con 0x7f862c0b5ff0 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.562+0000 7f8613fff640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 0x7f861806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.563+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8624058750 con 0x7f862c0b5ff0 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.563+0000 7f862b7fe640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 0x7f861806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:51.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.563+0000 7f862b7fe640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 0x7f861806ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f861c035000 tx=0x7f861c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:51.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.566+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb400d2bd0 con 0x7fbb38048100 2024-04-13T03:09:51.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.566+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3708058c70 con 0x7f371808c920 2024-04-13T03:09:51.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.567+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9c100569c0 con 0x7f9c0c053cb0 2024-04-13T03:09:51.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.568+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8624022030 con 0x7f862c0b5ff0 2024-04-13T03:09:51.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.577+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb4008f030 con 0x7fbb38048100 2024-04-13T03:09:51.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.585+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c10058920 con 0x7f9c0c053cb0 2024-04-13T03:09:51.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.925+0000 7f2c3898c640 1 -- 172.21.15.106:0/1424149210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2c18005580 con 0x7f2c340acad0 2024-04-13T03:09:51.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.926+0000 7f2c22ffd640 1 -- 172.21.15.106:0/1424149210 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2c2c05a390 con 0x7f2c340acad0 2024-04-13T03:09:51.927 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.929+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 msgr2=0x7f2c1c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.929+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 0x7f2c1c0709b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c28004700 tx=0x7f2c28008040 comp rx=0 tx=0).stop 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.929+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 msgr2=0x7f2c340acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.929+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0013d0 tx=0x7f2c2c00e530 comp rx=0 tx=0).stop 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 shutdown_connections 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c1c06e4f0 0x7f2c1c0709b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c340ae8c0 0x7f2c340aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c340acad0 0x7f2c340acf50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 --2- 172.21.15.106:0/1424149210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c1c004200 0x7f2c3415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 >> 172.21.15.106:0/1424149210 conn(0x7f2c340b2600 msgr2=0x7f2c340b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 shutdown_connections 2024-04-13T03:09:51.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.930+0000 7f2c20ff9640 1 -- 172.21.15.106:0/1424149210 wait complete. 2024-04-13T03:09:51.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-13T03:09:51.948 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:51.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.959+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f81f40b37d0 con 0x7f81f4162d00 2024-04-13T03:09:51.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.959+0000 7f81eaffd640 1 -- 172.21.15.106:0/1756713906 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f81ec07c980 con 0x7f81f4162d00 2024-04-13T03:09:51.960 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.961+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 msgr2=0x7f81d806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.961+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 0x7f81d806eeb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81e40099a0 tx=0x7f81e40090e0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 msgr2=0x7f81f41f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 0x7f81f41f8710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81ec033000 tx=0x7f81ec004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 shutdown_connections 2024-04-13T03:09:51.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81d806c9f0 0x7f81d806eeb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f4162d00 0x7f81f41f8710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81f40cc990 0x7f81f41fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 --2- 172.21.15.106:0/1756713906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81f406d990 0x7f81f41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 >> 172.21.15.106:0/1756713906 conn(0x7f81f40cedc0 msgr2=0x7f81f40b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 shutdown_connections 2024-04-13T03:09:51.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.962+0000 7f81fb27c640 1 -- 172.21.15.106:0/1756713906 wait complete. 2024-04-13T03:09:51.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.966+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3b0804fdc0 con 0x7f3b08161750 2024-04-13T03:09:51.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.967+0000 7f3aeffff640 1 -- 172.21.15.106:0/752697680 <== mon.2 v2:172.21.15.114:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3af80599d0 con 0x7f3b08161750 2024-04-13T03:09:51.968 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.969+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 msgr2=0x7f3af406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 0x7f3af406ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3afc036000 tx=0x7f3afc008030 comp rx=0 tx=0).stop 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 msgr2=0x7f3b080cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 0x7f3b080cc140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3af800e7f0 tx=0x7f3af800ecb0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 shutdown_connections 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af406c940 0x7f3af406ee00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b08161750 0x7f3b080cc140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b080d0490 0x7f3b08161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 --2- 172.21.15.106:0/752697680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b080c7560 0x7f3b0815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 >> 172.21.15.106:0/752697680 conn(0x7f3b0809d8d0 msgr2=0x7f3b080b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 shutdown_connections 2024-04-13T03:09:51.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.970+0000 7f3b0cc15640 1 -- 172.21.15.106:0/752697680 wait complete. 2024-04-13T03:09:51.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-13T03:09:51.975 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:51.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.984+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9a74005580 con 0x7f9a94162cf0 2024-04-13T03:09:51.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.984+0000 7f9a89ffb640 1 -- 172.21.15.106:0/89399009 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a9005a460 con 0x7f9a94162cf0 2024-04-13T03:09:51.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.984+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f36fc005580 con 0x7f371808c920 2024-04-13T03:09:51.986 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:51.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-04-13T03:09:51.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.985+0000 7f37157fa640 1 -- 172.21.15.106:0/1933841444 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f370805cf70 con 0x7f371808c920 2024-04-13T03:09:51.987 INFO:teuthology.orchestra.run.smithi106.stdout:55834574858 2024-04-13T03:09:51.987 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:51.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.987+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 msgr2=0x7f36f806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.987+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 0x7f36f806ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3710001370 tx=0x7f3710009040 comp rx=0 tx=0).stop 2024-04-13T03:09:51.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.987+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 msgr2=0x7f37181653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.987+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f370800c580 tx=0x7f370800ca40 comp rx=0 tx=0).stop 2024-04-13T03:09:51.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 msgr2=0x7f9a7806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 0x7f9a7806eeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9a84007fd0 tx=0x7f9a84009040 comp rx=0 tx=0).stop 2024-04-13T03:09:51.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 msgr2=0x7f9a940d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 0x7f9a940d1cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000efc0 tx=0x7f9a9000c490 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 shutdown_connections 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a7806c9f0 0x7f9a7806eeb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 shutdown_connections 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f36f806c9c0 0x7f36f806ee80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f371808cda0 0x7f3718165920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f371808c920 0x7f37181653e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 --2- 172.21.15.106:0/1933841444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704005210 0x7f37180995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 >> 172.21.15.106:0/1933841444 conn(0x7f37180b1d00 msgr2=0x7f3718073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94167280 0x7f9a941fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a94162cf0 0x7f9a940d1cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 --2- 172.21.15.106:0/89399009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a94072570 0x7f9a941fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 >> 172.21.15.106:0/89399009 conn(0x7f9a94071870 msgr2=0x7f9a9406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 shutdown_connections 2024-04-13T03:09:51.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f371dc7c640 1 -- 172.21.15.106:0/1933841444 wait complete. 2024-04-13T03:09:51.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 shutdown_connections 2024-04-13T03:09:51.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.988+0000 7f9a9b6d5640 1 -- 172.21.15.106:0/89399009 wait complete. 2024-04-13T03:09:51.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.989+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f862c0cb400 con 0x7f862c0b5ff0 2024-04-13T03:09:51.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.990+0000 7f8613fff640 1 -- 172.21.15.106:0/1031834123 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8624022030 con 0x7f862c0b5ff0 2024-04-13T03:09:51.993 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:51.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.990+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbb38052850 con 0x7fbb38048100 2024-04-13T03:09:51.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.991+0000 7fbb2effd640 1 -- 172.21.15.106:0/2364642455 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbb400d5230 con 0x7fbb38048100 2024-04-13T03:09:51.994 INFO:teuthology.orchestra.run.smithi106.stdout:55834574860 2024-04-13T03:09:51.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 msgr2=0x7f861806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 0x7f861806ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f861c035000 tx=0x7f861c008030 comp rx=0 tx=0).stop 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 msgr2=0x7f862c164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f862400efc0 tx=0x7f862400c490 comp rx=0 tx=0).stop 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 shutdown_connections 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f861806c940 0x7f861806ee00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f862c0c77a0 0x7f862c0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f862c0b5ff0 0x7f862c164740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 --2- 172.21.15.106:0/1031834123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f862c073460 0x7f862c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 >> 172.21.15.106:0/1031834123 conn(0x7f862c08bf10 msgr2=0x7f862c0d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 shutdown_connections 2024-04-13T03:09:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.992+0000 7f8631bdc640 1 -- 172.21.15.106:0/1031834123 wait complete. 2024-04-13T03:09:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 msgr2=0x7fbb2006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 0x7fbb2006ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb3804b250 tx=0x7fbb30008040 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 msgr2=0x7fbb380392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb40050fc0 tx=0x7fbb400694f0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 shutdown_connections 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb2006c940 0x7fbb2006ee00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb38054560 0x7fbb38036ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb38048100 0x7fbb380392a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 --2- 172.21.15.106:0/2364642455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb38007460 0x7fbb380d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.993+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 >> 172.21.15.106:0/2364642455 conn(0x7fbb38009fe0 msgr2=0x7fbb3804d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.994+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 shutdown_connections 2024-04-13T03:09:52.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:51.994+0000 7fbb448d5640 1 -- 172.21.15.106:0/2364642455 wait complete. 2024-04-13T03:09:52.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.000+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9c0c03e4a0 con 0x7f9c0c053cb0 2024-04-13T03:09:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.001+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/3431444255 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c1005cc20 con 0x7f9c0c053cb0 2024-04-13T03:09:52.002 INFO:teuthology.orchestra.run.smithi106.stdout:55834574859 2024-04-13T03:09:52.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.003+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 msgr2=0x7f9bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.003+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9c1407ad60 tx=0x7f9c1406f030 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 msgr2=0x7f9c0c0e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 0x7f9c0c0e2370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c460 tx=0x7f9c1000c920 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 shutdown_connections 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0e3160 0x7f9c0c051be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c053cb0 0x7f9c0c0e2370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 --2- 172.21.15.106:0/3431444255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c03c0a0 0x7f9c0c0d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 >> 172.21.15.106:0/3431444255 conn(0x7f9c0c058ee0 msgr2=0x7f9c0c057c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 shutdown_connections 2024-04-13T03:09:52.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.004+0000 7f9c1c236640 1 -- 172.21.15.106:0/3431444255 wait complete. 2024-04-13T03:09:52.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-13T03:09:52.006 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:52.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-13T03:09:52.008 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:52.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-13T03:09:52.009 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:52.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2024-04-13T03:09:52.010 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:52.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-13T03:09:52.017 DEBUG:teuthology.parallel:result is None 2024-04-13T03:09:52.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-13T03:09:52.017 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:09:52.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.155+0000 7f4e5b708640 1 Processor -- start 2024-04-13T03:09:52.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.155+0000 7f4e5b708640 1 -- start start 2024-04-13T03:09:52.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e5b708640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e541622d0 0x7f4e540d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e5b708640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4e540d1c00 0x7f4e540b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e5b708640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540d3530 con 0x7f4e540cc6f0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e5b708640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4e541626b0 con 0x7f4e541622d0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e5b708640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4e540d0b60 con 0x7f4e540d1c00 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e594a4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e541622d0 0x7f4e540d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e59ca5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4e540cc6f0 0x7f4e540ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56280/0 (socket says 172.21.15.106:56280) 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e59ca5640 1 -- 172.21.15.106:0/4055043989 learned_addr learned my addr 172.21.15.106:0/4055043989 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e58ca3640 1 --2- 172.21.15.106:0/4055043989 >> v2:172.21.15.114:3300/0 conn(0x7f4e540d1c00 0x7f4e540b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e4bfff640 1 -- 172.21.15.106:0/4055043989 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396985833 0 0) 0x7f4e540d3530 con 0x7f4e540cc6f0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e4bfff640 1 -- 172.21.15.106:0/4055043989 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e40003520 con 0x7f4e540cc6f0 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e594a4640 1 -- 172.21.15.106:0/4055043989 >> v2:172.21.15.114:3300/0 conn(0x7f4e540d1c00 msgr2=0x7f4e540b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.156+0000 7f4e594a4640 1 --2- 172.21.15.106:0/4055043989 >> v2:172.21.15.114:3300/0 conn(0x7f4e540d1c00 0x7f4e540b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e594a4640 1 -- 172.21.15.106:0/4055043989 >> v1:172.21.15.106:6789/0 conn(0x7f4e540cc6f0 legacy=0x7f4e540ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e594a4640 1 -- 172.21.15.106:0/4055043989 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540d0e40 con 0x7f4e541622d0 2024-04-13T03:09:52.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e594a4640 1 --2- 172.21.15.106:0/4055043989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e541622d0 0x7f4e540d04d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e54073520 tx=0x7f4e5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52389cc0bcd068e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e4bfff640 1 -- 172.21.15.106:0/4055043989 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50039070 con 0x7f4e541622d0 2024-04-13T03:09:52.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e4bfff640 1 -- 172.21.15.106:0/4055043989 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5002cc80 con 0x7f4e541622d0 2024-04-13T03:09:52.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.157+0000 7f4e4bfff640 1 -- 172.21.15.106:0/4055043989 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50034660 con 0x7f4e541622d0 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 -- 172.21.15.106:0/4055043989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e541622d0 msgr2=0x7f4e540d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 --2- 172.21.15.106:0/4055043989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e541622d0 0x7f4e540d04d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e54073520 tx=0x7f4e5002bed0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 -- 172.21.15.106:0/4055043989 shutdown_connections 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 --2- 172.21.15.106:0/4055043989 >> v2:172.21.15.114:3300/0 conn(0x7f4e540d1c00 0x7f4e540b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 --2- 172.21.15.106:0/4055043989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e541622d0 0x7f4e540d04d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 -- 172.21.15.106:0/4055043989 >> 172.21.15.106:0/4055043989 conn(0x7f4e540d25d0 msgr2=0x7f4e540d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 -- 172.21.15.106:0/4055043989 shutdown_connections 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.158+0000 7f4e5b708640 1 -- 172.21.15.106:0/4055043989 wait complete. 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.159+0000 7f4e5b708640 1 Processor -- start 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.159+0000 7f4e5b708640 1 -- start start 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e540d1c00 0x7f4e5415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e541622d0 0x7f4e54147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540d16e0 con 0x7f4e540cc6f0 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540d1400 con 0x7f4e540d1c00 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e5b708640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540b7eb0 con 0x7f4e541622d0 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45386/0 (socket says 172.21.15.106:45386) 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 -- 172.21.15.106:0/3819375265 learned_addr learned my addr 172.21.15.106:0/3819375265 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e58ca3640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e540d1c00 0x7f4e5415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e59ca5640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e541622d0 0x7f4e54147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 -- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e541622d0 msgr2=0x7f4e54147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e541622d0 0x7f4e54147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 -- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e540d1c00 msgr2=0x7f4e5415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e540d1c00 0x7f4e5415a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 -- 172.21.15.106:0/3819375265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5415a5f0 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e594a4640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e50037040 tx=0x7f4e50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.160+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50039050 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.161+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e50045070 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.161+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5415a890 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.161+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5003e850 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.161+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5415ae50 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.162+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54068a10 con 0x7f4e540cc6f0 2024-04-13T03:09:52.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.162+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e5003e9f0 con 0x7f4e540cc6f0 2024-04-13T03:09:52.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.162+0000 7f4e49ffb640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 0x7f4e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.163+0000 7f4e58ca3640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 0x7f4e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.163+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e500aeb60 con 0x7f4e540cc6f0 2024-04-13T03:09:52.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.163+0000 7f4e58ca3640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 0x7f4e3806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e541f8240 tx=0x7f4e44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.167+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e5007cce0 con 0x7f4e540cc6f0 2024-04-13T03:09:52.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.339+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e540ceb80 con 0x7f4e3806c8f0 2024-04-13T03:09:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.346+0000 7f4e49ffb640 1 -- 172.21.15.106:0/3819375265 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f4e540ceb80 con 0x7f4e3806c8f0 2024-04-13T03:09:52.351 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:52.353 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:09:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 msgr2=0x7f4e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 0x7f4e3806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e541f8240 tx=0x7f4e44008040 comp rx=0 tx=0).stop 2024-04-13T03:09:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 msgr2=0x7f4e54157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e50037040 tx=0x7f4e50004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 shutdown_connections 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e3806c8f0 0x7f4e3806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e541622d0 0x7f4e54147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e540d1c00 0x7f4e5415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 --2- 172.21.15.106:0/3819375265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e540cc6f0 0x7f4e54157910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 >> 172.21.15.106:0/3819375265 conn(0x7f4e540d25d0 msgr2=0x7f4e540cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 shutdown_connections 2024-04-13T03:09:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.350+0000 7f4e5b708640 1 -- 172.21.15.106:0/3819375265 wait complete. 2024-04-13T03:09:52.362 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-13T03:09:50.367978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002870"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004868+0000","last_change":"2024-04-13T03:09:23.977064+0000","last_active":"2024-04-13T03:09:31.004868+0000","last_peered":"2024-04-13T03:09:31.004868+0000","last_clean":"2024-04-13T03:09:31.004868+0000","last_became_active":"2024-04-13T03:09:23.976781+0000","last_became_peered":"2024-04-13T03:09:23.976781+0000","last_unstale":"2024-04-13T03:09:31.004868+0000","last_undegraded":"2024-04-13T03:09:31.004868+0000","last_fullsized":"2024-04-13T03:09:31.004868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:14:27.702105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017061+0000","last_change":"2024-04-13T03:09:24.983007+0000","last_active":"2024-04-13T03:09:31.017061+0000","last_peered":"2024-04-13T03:09:31.017061+0000","last_clean":"2024-04-13T03:09:31.017061+0000","last_became_active":"2024-04-13T03:09:24.982432+0000","last_became_peered":"2024-04-13T03:09:24.982432+0000","last_unstale":"2024-04-13T03:09:31.017061+0000","last_undegraded":"2024-04-13T03:09:31.017061+0000","last_fullsized":"2024-04-13T03:09:31.017061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016588+0000","last_change":"2024-04-13T03:09:29.567663+0000","last_active":"2024-04-13T03:09:31.016588+0000","last_peered":"2024-04-13T03:09:31.016588+0000","last_clean":"2024-04-13T03:09:31.016588+0000","last_became_active":"2024-04-13T03:09:29.567314+0000","last_became_peered":"2024-04-13T03:09:29.567314+0000","last_unstale":"2024-04-13T03:09:31.016588+0000","last_undegraded":"2024-04-13T03:09:31.016588+0000","last_fullsized":"2024-04-13T03:09:31.016588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:55:26.111999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017190+0000","last_change":"2024-04-13T03:09:23.977858+0000","last_active":"2024-04-13T03:09:31.017190+0000","last_peered":"2024-04-13T03:09:31.017190+0000","last_clean":"2024-04-13T03:09:31.017190+0000","last_became_active":"2024-04-13T03:09:23.977595+0000","last_became_peered":"2024-04-13T03:09:23.977595+0000","last_unstale":"2024-04-13T03:09:31.017190+0000","last_undegraded":"2024-04-13T03:09:31.017190+0000","last_fullsized":"2024-04-13T03:09:31.017190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004745+0000","last_change":"2024-04-13T03:09:24.981595+0000","last_active":"2024-04-13T03:09:31.004745+0000","last_peered":"2024-04-13T03:09:31.004745+0000","last_clean":"2024-04-13T03:09:31.004745+0000","last_became_active":"2024-04-13T03:09:24.981249+0000","last_became_peered":"2024-04-13T03:09:24.981249+0000","last_unstale":"2024-04-13T03:09:31.004745+0000","last_undegraded":"2024-04-13T03:09:31.004745+0000","last_fullsized":"2024-04-13T03:09:31.004745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004491+0000","last_change":"2024-04-13T03:09:29.008503+0000","last_active":"2024-04-13T03:09:31.004491+0000","last_peered":"2024-04-13T03:09:31.004491+0000","last_clean":"2024-04-13T03:09:31.004491+0000","last_became_active":"2024-04-13T03:09:29.007985+0000","last_became_peered":"2024-04-13T03:09:29.007985+0000","last_unstale":"2024-04-13T03:09:31.004491+0000","last_undegraded":"2024-04-13T03:09:31.004491+0000","last_fullsized":"2024-04-13T03:09:31.004491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074031+0000","last_change":"2024-04-13T03:09:23.982379+0000","last_active":"2024-04-13T03:09:32.074031+0000","last_peered":"2024-04-13T03:09:32.074031+0000","last_clean":"2024-04-13T03:09:32.074031+0000","last_became_active":"2024-04-13T03:09:23.982090+0000","last_became_peered":"2024-04-13T03:09:23.982090+0000","last_unstale":"2024-04-13T03:09:32.074031+0000","last_undegraded":"2024-04-13T03:09:32.074031+0000","last_fullsized":"2024-04-13T03:09:32.074031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006164+0000","last_change":"2024-04-13T03:09:24.983652+0000","last_active":"2024-04-13T03:09:31.006164+0000","last_peered":"2024-04-13T03:09:31.006164+0000","last_clean":"2024-04-13T03:09:31.006164+0000","last_became_active":"2024-04-13T03:09:24.981620+0000","last_became_peered":"2024-04-13T03:09:24.981620+0000","last_unstale":"2024-04-13T03:09:31.006164+0000","last_undegraded":"2024-04-13T03:09:31.006164+0000","last_fullsized":"2024-04-13T03:09:31.006164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004691+0000","last_change":"2024-04-13T03:09:29.007274+0000","last_active":"2024-04-13T03:09:31.004691+0000","last_peered":"2024-04-13T03:09:31.004691+0000","last_clean":"2024-04-13T03:09:31.004691+0000","last_became_active":"2024-04-13T03:09:29.006867+0000","last_became_peered":"2024-04-13T03:09:29.006867+0000","last_unstale":"2024-04-13T03:09:31.004691+0000","last_undegraded":"2024-04-13T03:09:31.004691+0000","last_fullsized":"2024-04-13T03:09:31.004691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:32.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073387+0000","last_change":"2024-04-13T03:09:23.979669+0000","last_active":"2024-04-13T03:09:32.073387+0000","last_peered":"2024-04-13T03:09:32.073387+0000","last_clean":"2024-04-13T03:09:32.073387+0000","last_became_active":"2024-04-13T03:09:23.979303+0000","last_became_peered":"2024-04-13T03:09:23.979303+0000","last_unstale":"2024-04-13T03:09:32.073387+0000","last_undegraded":"2024-04-13T03:09:32.073387+0000","last_fullsized":"2024-04-13T03:09:32.073387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005531+0000","last_change":"2024-04-13T03:09:24.976796+0000","last_active":"2024-04-13T03:09:31.005531+0000","last_peered":"2024-04-13T03:09:31.005531+0000","last_clean":"2024-04-13T03:09:31.005531+0000","last_became_active":"2024-04-13T03:09:24.975367+0000","last_became_peered":"2024-04-13T03:09:24.975367+0000","last_unstale":"2024-04-13T03:09:31.005531+0000","last_undegraded":"2024-04-13T03:09:31.005531+0000","last_fullsized":"2024-04-13T03:09:31.005531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004563+0000","last_change":"2024-04-13T03:09:29.007374+0000","last_active":"2024-04-13T03:09:31.004563+0000","last_peered":"2024-04-13T03:09:31.004563+0000","last_clean":"2024-04-13T03:09:31.004563+0000","last_became_active":"2024-04-13T03:09:29.006740+0000","last_became_peered":"2024-04-13T03:09:29.006740+0000","last_unstale":"2024-04-13T03:09:31.004563+0000","last_undegraded":"2024-04-13T03:09:31.004563+0000","last_fullsized":"2024-04-13T03:09:31.004563+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073554+0000","last_change":"2024-04-13T03:09:24.988155+0000","last_active":"2024-04-13T03:09:32.073554+0000","last_peered":"2024-04-13T03:09:32.073554+0000","last_clean":"2024-04-13T03:09:32.073554+0000","last_became_active":"2024-04-13T03:09:24.987879+0000","last_became_peered":"2024-04-13T03:09:24.987879+0000","last_unstale":"2024-04-13T03:09:32.073554+0000","last_undegraded":"2024-04-13T03:09:32.073554+0000","last_fullsized":"2024-04-13T03:09:32.073554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072988+0000","last_change":"2024-04-13T03:09:23.977048+0000","last_active":"2024-04-13T03:09:32.072988+0000","last_peered":"2024-04-13T03:09:32.072988+0000","last_clean":"2024-04-13T03:09:32.072988+0000","last_became_active":"2024-04-13T03:09:23.976770+0000","last_became_peered":"2024-04-13T03:09:23.976770+0000","last_unstale":"2024-04-13T03:09:32.072988+0000","last_undegraded":"2024-04-13T03:09:32.072988+0000","last_fullsized":"2024-04-13T03:09:32.072988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005678+0000","last_change":"2024-04-13T03:09:29.009555+0000","last_active":"2024-04-13T03:09:31.005678+0000","last_peered":"2024-04-13T03:09:31.005678+0000","last_clean":"2024-04-13T03:09:31.005678+0000","last_became_active":"2024-04-13T03:09:29.009108+0000","last_became_peered":"2024-04-13T03:09:29.009108+0000","last_unstale":"2024-04-13T03:09:31.005678+0000","last_undegraded":"2024-04-13T03:09:31.005678+0000","last_fullsized":"2024-04-13T03:09:31.005678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016758+0000","last_change":"2024-04-13T03:09:24.973588+0000","last_active":"2024-04-13T03:09:31.016758+0000","last_peered":"2024-04-13T03:09:31.016758+0000","last_clean":"2024-04-13T03:09:31.016758+0000","last_became_active":"2024-04-13T03:09:24.973322+0000","last_became_peered":"2024-04-13T03:09:24.973322+0000","last_unstale":"2024-04-13T03:09:31.016758+0000","last_undegraded":"2024-04-13T03:09:31.016758+0000","last_fullsized":"2024-04-13T03:09:31.016758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552368+0000","last_change":"2024-04-13T03:09:23.971765+0000","last_active":"2024-04-13T03:09:31.552368+0000","last_peered":"2024-04-13T03:09:31.552368+0000","last_clean":"2024-04-13T03:09:31.552368+0000","last_became_active":"2024-04-13T03:09:23.971543+0000","last_became_peered":"2024-04-13T03:09:23.971543+0000","last_unstale":"2024-04-13T03:09:31.552368+0000","last_undegraded":"2024-04-13T03:09:31.552368+0000","last_fullsized":"2024-04-13T03:09:31.552368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005586+0000","last_change":"2024-04-13T03:09:29.008916+0000","last_active":"2024-04-13T03:09:31.005586+0000","last_peered":"2024-04-13T03:09:31.005586+0000","last_clean":"2024-04-13T03:09:31.005586+0000","last_became_active":"2024-04-13T03:09:29.008366+0000","last_became_peered":"2024-04-13T03:09:29.008366+0000","last_unstale":"2024-04-13T03:09:31.005586+0000","last_undegraded":"2024-04-13T03:09:31.005586+0000","last_fullsized":"2024-04-13T03:09:31.005586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016669+0000","last_change":"2024-04-13T03:09:24.975847+0000","last_active":"2024-04-13T03:09:31.016669+0000","last_peered":"2024-04-13T03:09:31.016669+0000","last_clean":"2024-04-13T03:09:31.016669+0000","last_became_active":"2024-04-13T03:09:24.975258+0000","last_became_peered":"2024-04-13T03:09:24.975258+0000","last_unstale":"2024-04-13T03:09:31.016669+0000","last_undegraded":"2024-04-13T03:09:31.016669+0000","last_fullsized":"2024-04-13T03:09:31.016669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552132+0000","last_change":"2024-04-13T03:09:23.977100+0000","last_active":"2024-04-13T03:09:31.552132+0000","last_peered":"2024-04-13T03:09:31.552132+0000","last_clean":"2024-04-13T03:09:31.552132+0000","last_became_active":"2024-04-13T03:09:23.976787+0000","last_became_peered":"2024-04-13T03:09:23.976787+0000","last_unstale":"2024-04-13T03:09:31.552132+0000","last_undegraded":"2024-04-13T03:09:31.552132+0000","last_fullsized":"2024-04-13T03:09:31.552132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552936+0000","last_change":"2024-04-13T03:09:29.567374+0000","last_active":"2024-04-13T03:09:31.552936+0000","last_peered":"2024-04-13T03:09:31.552936+0000","last_clean":"2024-04-13T03:09:31.552936+0000","last_became_active":"2024-04-13T03:09:29.566995+0000","last_became_peered":"2024-04-13T03:09:29.566995+0000","last_unstale":"2024-04-13T03:09:31.552936+0000","last_undegraded":"2024-04-13T03:09:31.552936+0000","last_fullsized":"2024-04-13T03:09:31.552936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004813+0000","last_change":"2024-04-13T03:09:24.972822+0000","last_active":"2024-04-13T03:09:31.004813+0000","last_peered":"2024-04-13T03:09:31.004813+0000","last_clean":"2024-04-13T03:09:31.004813+0000","last_became_active":"2024-04-13T03:09:24.972521+0000","last_became_peered":"2024-04-13T03:09:24.972521+0000","last_unstale":"2024-04-13T03:09:31.004813+0000","last_undegraded":"2024-04-13T03:09:31.004813+0000","last_fullsized":"2024-04-13T03:09:31.004813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073287+0000","last_change":"2024-04-13T03:09:23.979583+0000","last_active":"2024-04-13T03:09:32.073287+0000","last_peered":"2024-04-13T03:09:32.073287+0000","last_clean":"2024-04-13T03:09:32.073287+0000","last_became_active":"2024-04-13T03:09:23.979321+0000","last_became_peered":"2024-04-13T03:09:23.979321+0000","last_unstale":"2024-04-13T03:09:32.073287+0000","last_undegraded":"2024-04-13T03:09:32.073287+0000","last_fullsized":"2024-04-13T03:09:32.073287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:05.954316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006004+0000","last_change":"2024-04-13T03:09:29.009924+0000","last_active":"2024-04-13T03:09:31.006004+0000","last_peered":"2024-04-13T03:09:31.006004+0000","last_clean":"2024-04-13T03:09:31.006004+0000","last_became_active":"2024-04-13T03:09:29.009528+0000","last_became_peered":"2024-04-13T03:09:29.009528+0000","last_unstale":"2024-04-13T03:09:31.006004+0000","last_undegraded":"2024-04-13T03:09:31.006004+0000","last_fullsized":"2024-04-13T03:09:31.006004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004799+0000","last_change":"2024-04-13T03:09:24.973677+0000","last_active":"2024-04-13T03:09:31.004799+0000","last_peered":"2024-04-13T03:09:31.004799+0000","last_clean":"2024-04-13T03:09:31.004799+0000","last_became_active":"2024-04-13T03:09:24.973419+0000","last_became_peered":"2024-04-13T03:09:24.973419+0000","last_unstale":"2024-04-13T03:09:31.004799+0000","last_undegraded":"2024-04-13T03:09:31.004799+0000","last_fullsized":"2024-04-13T03:09:31.004799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074813+0000","last_change":"2024-04-13T03:09:23.977870+0000","last_active":"2024-04-13T03:09:32.074813+0000","last_peered":"2024-04-13T03:09:32.074813+0000","last_clean":"2024-04-13T03:09:32.074813+0000","last_became_active":"2024-04-13T03:09:23.977529+0000","last_became_peered":"2024-04-13T03:09:23.977529+0000","last_unstale":"2024-04-13T03:09:32.074813+0000","last_undegraded":"2024-04-13T03:09:32.074813+0000","last_fullsized":"2024-04-13T03:09:32.074813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004609+0000","last_change":"2024-04-13T03:09:29.566777+0000","last_active":"2024-04-13T03:09:31.004609+0000","last_peered":"2024-04-13T03:09:31.004609+0000","last_clean":"2024-04-13T03:09:31.004609+0000","last_became_active":"2024-04-13T03:09:29.566173+0000","last_became_peered":"2024-04-13T03:09:29.566173+0000","last_unstale":"2024-04-13T03:09:31.004609+0000","last_undegraded":"2024-04-13T03:09:31.004609+0000","last_fullsized":"2024-04-13T03:09:31.004609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017303+0000","last_change":"2024-04-13T03:09:24.981346+0000","last_active":"2024-04-13T03:09:31.017303+0000","last_peered":"2024-04-13T03:09:31.017303+0000","last_clean":"2024-04-13T03:09:31.017303+0000","last_became_active":"2024-04-13T03:09:24.981003+0000","last_became_peered":"2024-04-13T03:09:24.981003+0000","last_unstale":"2024-04-13T03:09:31.017303+0000","last_undegraded":"2024-04-13T03:09:31.017303+0000","last_fullsized":"2024-04-13T03:09:31.017303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552769+0000","last_change":"2024-04-13T03:09:23.973982+0000","last_active":"2024-04-13T03:09:31.552769+0000","last_peered":"2024-04-13T03:09:31.552769+0000","last_clean":"2024-04-13T03:09:31.552769+0000","last_became_active":"2024-04-13T03:09:23.973759+0000","last_became_peered":"2024-04-13T03:09:23.973759+0000","last_unstale":"2024-04-13T03:09:31.552769+0000","last_undegraded":"2024-04-13T03:09:31.552769+0000","last_fullsized":"2024-04-13T03:09:31.552769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016499+0000","last_change":"2024-04-13T03:09:29.567476+0000","last_active":"2024-04-13T03:09:31.016499+0000","last_peered":"2024-04-13T03:09:31.016499+0000","last_clean":"2024-04-13T03:09:31.016499+0000","last_became_active":"2024-04-13T03:09:29.567116+0000","last_became_peered":"2024-04-13T03:09:29.567116+0000","last_unstale":"2024-04-13T03:09:31.016499+0000","last_undegraded":"2024-04-13T03:09:31.016499+0000","last_fullsized":"2024-04-13T03:09:31.016499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:07.048787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552621+0000","last_change":"2024-04-13T03:09:24.974586+0000","last_active":"2024-04-13T03:09:31.552621+0000","last_peered":"2024-04-13T03:09:31.552621+0000","last_clean":"2024-04-13T03:09:31.552621+0000","last_became_active":"2024-04-13T03:09:24.974348+0000","last_became_peered":"2024-04-13T03:09:24.974348+0000","last_unstale":"2024-04-13T03:09:31.552621+0000","last_undegraded":"2024-04-13T03:09:31.552621+0000","last_fullsized":"2024-04-13T03:09:31.552621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073219+0000","last_change":"2024-04-13T03:09:23.979817+0000","last_active":"2024-04-13T03:09:32.073219+0000","last_peered":"2024-04-13T03:09:32.073219+0000","last_clean":"2024-04-13T03:09:32.073219+0000","last_became_active":"2024-04-13T03:09:23.979575+0000","last_became_peered":"2024-04-13T03:09:23.979575+0000","last_unstale":"2024-04-13T03:09:32.073219+0000","last_undegraded":"2024-04-13T03:09:32.073219+0000","last_fullsized":"2024-04-13T03:09:32.073219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:42.984781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004425+0000","last_change":"2024-04-13T03:09:29.567336+0000","last_active":"2024-04-13T03:09:31.004425+0000","last_peered":"2024-04-13T03:09:31.004425+0000","last_clean":"2024-04-13T03:09:31.004425+0000","last_became_active":"2024-04-13T03:09:29.566964+0000","last_became_peered":"2024-04-13T03:09:29.566964+0000","last_unstale":"2024-04-13T03:09:31.004425+0000","last_undegraded":"2024-04-13T03:09:31.004425+0000","last_fullsized":"2024-04-13T03:09:31.004425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005618+0000","last_change":"2024-04-13T03:09:24.976546+0000","last_active":"2024-04-13T03:09:31.005618+0000","last_peered":"2024-04-13T03:09:31.005618+0000","last_clean":"2024-04-13T03:09:31.005618+0000","last_became_active":"2024-04-13T03:09:24.974772+0000","last_became_peered":"2024-04-13T03:09:24.974772+0000","last_unstale":"2024-04-13T03:09:31.005618+0000","last_undegraded":"2024-04-13T03:09:31.005618+0000","last_fullsized":"2024-04-13T03:09:31.005618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552722+0000","last_change":"2024-04-13T03:09:23.966690+0000","last_active":"2024-04-13T03:09:31.552722+0000","last_peered":"2024-04-13T03:09:31.552722+0000","last_clean":"2024-04-13T03:09:31.552722+0000","last_became_active":"2024-04-13T03:09:23.966440+0000","last_became_peered":"2024-04-13T03:09:23.966440+0000","last_unstale":"2024-04-13T03:09:31.552722+0000","last_undegraded":"2024-04-13T03:09:31.552722+0000","last_fullsized":"2024-04-13T03:09:31.552722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552227+0000","last_change":"2024-04-13T03:09:29.013113+0000","last_active":"2024-04-13T03:09:31.552227+0000","last_peered":"2024-04-13T03:09:31.552227+0000","last_clean":"2024-04-13T03:09:31.552227+0000","last_became_active":"2024-04-13T03:09:29.012470+0000","last_became_peered":"2024-04-13T03:09:29.012470+0000","last_unstale":"2024-04-13T03:09:31.552227+0000","last_undegraded":"2024-04-13T03:09:31.552227+0000","last_fullsized":"2024-04-13T03:09:31.552227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005144+0000","last_change":"2024-04-13T03:09:24.973635+0000","last_active":"2024-04-13T03:09:31.005144+0000","last_peered":"2024-04-13T03:09:31.005144+0000","last_clean":"2024-04-13T03:09:31.005144+0000","last_became_active":"2024-04-13T03:09:24.973362+0000","last_became_peered":"2024-04-13T03:09:24.973362+0000","last_unstale":"2024-04-13T03:09:31.005144+0000","last_undegraded":"2024-04-13T03:09:31.005144+0000","last_fullsized":"2024-04-13T03:09:31.005144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:24.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006449+0000","last_change":"2024-04-13T03:09:23.968272+0000","last_active":"2024-04-13T03:09:31.006449+0000","last_peered":"2024-04-13T03:09:31.006449+0000","last_clean":"2024-04-13T03:09:31.006449+0000","last_became_active":"2024-04-13T03:09:23.967784+0000","last_became_peered":"2024-04-13T03:09:23.967784+0000","last_unstale":"2024-04-13T03:09:31.006449+0000","last_undegraded":"2024-04-13T03:09:31.006449+0000","last_fullsized":"2024-04-13T03:09:31.006449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552810+0000","last_change":"2024-04-13T03:09:29.008833+0000","last_active":"2024-04-13T03:09:31.552810+0000","last_peered":"2024-04-13T03:09:31.552810+0000","last_clean":"2024-04-13T03:09:31.552810+0000","last_became_active":"2024-04-13T03:09:29.008342+0000","last_became_peered":"2024-04-13T03:09:29.008342+0000","last_unstale":"2024-04-13T03:09:31.552810+0000","last_undegraded":"2024-04-13T03:09:31.552810+0000","last_fullsized":"2024-04-13T03:09:31.552810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552174+0000","last_change":"2024-04-13T03:09:24.975965+0000","last_active":"2024-04-13T03:09:31.552174+0000","last_peered":"2024-04-13T03:09:31.552174+0000","last_clean":"2024-04-13T03:09:31.552174+0000","last_became_active":"2024-04-13T03:09:24.975143+0000","last_became_peered":"2024-04-13T03:09:24.975143+0000","last_unstale":"2024-04-13T03:09:31.552174+0000","last_undegraded":"2024-04-13T03:09:31.552174+0000","last_fullsized":"2024-04-13T03:09:31.552174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.978132+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.976718+0000","last_became_peered":"2024-04-13T03:09:23.976718+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552953+0000","last_change":"2024-04-13T03:09:29.012295+0000","last_active":"2024-04-13T03:09:31.552953+0000","last_peered":"2024-04-13T03:09:31.552953+0000","last_clean":"2024-04-13T03:09:31.552953+0000","last_became_active":"2024-04-13T03:09:29.011908+0000","last_became_peered":"2024-04-13T03:09:29.011908+0000","last_unstale":"2024-04-13T03:09:31.552953+0000","last_undegraded":"2024-04-13T03:09:31.552953+0000","last_fullsized":"2024-04-13T03:09:31.552953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004144+0000","last_change":"2024-04-13T03:09:24.974975+0000","last_active":"2024-04-13T03:09:31.004144+0000","last_peered":"2024-04-13T03:09:31.004144+0000","last_clean":"2024-04-13T03:09:31.004144+0000","last_became_active":"2024-04-13T03:09:24.974656+0000","last_became_peered":"2024-04-13T03:09:24.974656+0000","last_unstale":"2024-04-13T03:09:31.004144+0000","last_undegraded":"2024-04-13T03:09:31.004144+0000","last_fullsized":"2024-04-13T03:09:31.004144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072979+0000","last_change":"2024-04-13T03:09:23.978671+0000","last_active":"2024-04-13T03:09:32.072979+0000","last_peered":"2024-04-13T03:09:32.072979+0000","last_clean":"2024-04-13T03:09:32.072979+0000","last_became_active":"2024-04-13T03:09:23.978358+0000","last_became_peered":"2024-04-13T03:09:23.978358+0000","last_unstale":"2024-04-13T03:09:32.072979+0000","last_undegraded":"2024-04-13T03:09:32.072979+0000","last_fullsized":"2024-04-13T03:09:32.072979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004701+0000","last_change":"2024-04-13T03:09:29.566364+0000","last_active":"2024-04-13T03:09:31.004701+0000","last_peered":"2024-04-13T03:09:31.004701+0000","last_clean":"2024-04-13T03:09:31.004701+0000","last_became_active":"2024-04-13T03:09:29.565976+0000","last_became_peered":"2024-04-13T03:09:29.565976+0000","last_unstale":"2024-04-13T03:09:31.004701+0000","last_undegraded":"2024-04-13T03:09:31.004701+0000","last_fullsized":"2024-04-13T03:09:31.004701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016580+0000","last_change":"2024-04-13T03:09:24.988431+0000","last_active":"2024-04-13T03:09:31.016580+0000","last_peered":"2024-04-13T03:09:31.016580+0000","last_clean":"2024-04-13T03:09:31.016580+0000","last_became_active":"2024-04-13T03:09:24.988179+0000","last_became_peered":"2024-04-13T03:09:24.988179+0000","last_unstale":"2024-04-13T03:09:31.016580+0000","last_undegraded":"2024-04-13T03:09:31.016580+0000","last_fullsized":"2024-04-13T03:09:31.016580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005560+0000","last_change":"2024-04-13T03:09:23.970960+0000","last_active":"2024-04-13T03:09:31.005560+0000","last_peered":"2024-04-13T03:09:31.005560+0000","last_clean":"2024-04-13T03:09:31.005560+0000","last_became_active":"2024-04-13T03:09:23.970615+0000","last_became_peered":"2024-04-13T03:09:23.970615+0000","last_unstale":"2024-04-13T03:09:31.005560+0000","last_undegraded":"2024-04-13T03:09:31.005560+0000","last_fullsized":"2024-04-13T03:09:31.005560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073625+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:09:32.073625+0000","last_peered":"2024-04-13T03:09:32.073625+0000","last_clean":"2024-04-13T03:09:32.073625+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:09:32.073625+0000","last_undegraded":"2024-04-13T03:09:32.073625+0000","last_fullsized":"2024-04-13T03:09:32.073625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074664+0000","last_change":"2024-04-13T03:09:29.570505+0000","last_active":"2024-04-13T03:09:32.074664+0000","last_peered":"2024-04-13T03:09:32.074664+0000","last_clean":"2024-04-13T03:09:32.074664+0000","last_became_active":"2024-04-13T03:09:29.569637+0000","last_became_peered":"2024-04-13T03:09:29.569637+0000","last_unstale":"2024-04-13T03:09:32.074664+0000","last_undegraded":"2024-04-13T03:09:32.074664+0000","last_fullsized":"2024-04-13T03:09:32.074664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073883+0000","last_change":"2024-04-13T03:09:24.990807+0000","last_active":"2024-04-13T03:09:32.073883+0000","last_peered":"2024-04-13T03:09:32.073883+0000","last_clean":"2024-04-13T03:09:32.073883+0000","last_became_active":"2024-04-13T03:09:24.990454+0000","last_became_peered":"2024-04-13T03:09:24.990454+0000","last_unstale":"2024-04-13T03:09:32.073883+0000","last_undegraded":"2024-04-13T03:09:32.073883+0000","last_fullsized":"2024-04-13T03:09:32.073883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072543+0000","last_change":"2024-04-13T03:09:23.974849+0000","last_active":"2024-04-13T03:09:32.072543+0000","last_peered":"2024-04-13T03:09:32.072543+0000","last_clean":"2024-04-13T03:09:32.072543+0000","last_became_active":"2024-04-13T03:09:23.974564+0000","last_became_peered":"2024-04-13T03:09:23.974564+0000","last_unstale":"2024-04-13T03:09:32.072543+0000","last_undegraded":"2024-04-13T03:09:32.072543+0000","last_fullsized":"2024-04-13T03:09:32.072543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073496+0000","last_change":"2024-04-13T03:09:29.567076+0000","last_active":"2024-04-13T03:09:32.073496+0000","last_peered":"2024-04-13T03:09:32.073496+0000","last_clean":"2024-04-13T03:09:32.073496+0000","last_became_active":"2024-04-13T03:09:29.566162+0000","last_became_peered":"2024-04-13T03:09:29.566162+0000","last_unstale":"2024-04-13T03:09:32.073496+0000","last_undegraded":"2024-04-13T03:09:32.073496+0000","last_fullsized":"2024-04-13T03:09:32.073496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006229+0000","last_change":"2024-04-13T03:09:24.986470+0000","last_active":"2024-04-13T03:09:31.006229+0000","last_peered":"2024-04-13T03:09:31.006229+0000","last_clean":"2024-04-13T03:09:31.006229+0000","last_became_active":"2024-04-13T03:09:24.986035+0000","last_became_peered":"2024-04-13T03:09:24.986035+0000","last_unstale":"2024-04-13T03:09:31.006229+0000","last_undegraded":"2024-04-13T03:09:31.006229+0000","last_fullsized":"2024-04-13T03:09:31.006229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552711+0000","last_change":"2024-04-13T03:09:23.966911+0000","last_active":"2024-04-13T03:09:31.552711+0000","last_peered":"2024-04-13T03:09:31.552711+0000","last_clean":"2024-04-13T03:09:31.552711+0000","last_became_active":"2024-04-13T03:09:23.966494+0000","last_became_peered":"2024-04-13T03:09:23.966494+0000","last_unstale":"2024-04-13T03:09:31.552711+0000","last_undegraded":"2024-04-13T03:09:31.552711+0000","last_fullsized":"2024-04-13T03:09:31.552711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004556+0000","last_change":"2024-04-13T03:09:29.008472+0000","last_active":"2024-04-13T03:09:31.004556+0000","last_peered":"2024-04-13T03:09:31.004556+0000","last_clean":"2024-04-13T03:09:31.004556+0000","last_became_active":"2024-04-13T03:09:29.008101+0000","last_became_peered":"2024-04-13T03:09:29.008101+0000","last_unstale":"2024-04-13T03:09:31.004556+0000","last_undegraded":"2024-04-13T03:09:31.004556+0000","last_fullsized":"2024-04-13T03:09:31.004556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074447+0000","last_change":"2024-04-13T03:09:24.978817+0000","last_active":"2024-04-13T03:09:32.074447+0000","last_peered":"2024-04-13T03:09:32.074447+0000","last_clean":"2024-04-13T03:09:32.074447+0000","last_became_active":"2024-04-13T03:09:24.978091+0000","last_became_peered":"2024-04-13T03:09:24.978091+0000","last_unstale":"2024-04-13T03:09:32.074447+0000","last_undegraded":"2024-04-13T03:09:32.074447+0000","last_fullsized":"2024-04-13T03:09:32.074447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:28.224846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072548+0000","last_change":"2024-04-13T03:09:23.978022+0000","last_active":"2024-04-13T03:09:32.072548+0000","last_peered":"2024-04-13T03:09:32.072548+0000","last_clean":"2024-04-13T03:09:32.072548+0000","last_became_active":"2024-04-13T03:09:23.976602+0000","last_became_peered":"2024-04-13T03:09:23.976602+0000","last_unstale":"2024-04-13T03:09:32.072548+0000","last_undegraded":"2024-04-13T03:09:32.072548+0000","last_fullsized":"2024-04-13T03:09:32.072548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005100+0000","last_change":"2024-04-13T03:09:29.567904+0000","last_active":"2024-04-13T03:09:31.005100+0000","last_peered":"2024-04-13T03:09:31.005100+0000","last_clean":"2024-04-13T03:09:31.005100+0000","last_became_active":"2024-04-13T03:09:29.567497+0000","last_became_peered":"2024-04-13T03:09:29.567497+0000","last_unstale":"2024-04-13T03:09:31.005100+0000","last_undegraded":"2024-04-13T03:09:31.005100+0000","last_fullsized":"2024-04-13T03:09:31.005100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074412+0000","last_change":"2024-04-13T03:09:24.978521+0000","last_active":"2024-04-13T03:09:32.074412+0000","last_peered":"2024-04-13T03:09:32.074412+0000","last_clean":"2024-04-13T03:09:32.074412+0000","last_became_active":"2024-04-13T03:09:24.978163+0000","last_became_peered":"2024-04-13T03:09:24.978163+0000","last_unstale":"2024-04-13T03:09:32.074412+0000","last_undegraded":"2024-04-13T03:09:32.074412+0000","last_fullsized":"2024-04-13T03:09:32.074412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006024+0000","last_change":"2024-04-13T03:09:23.966891+0000","last_active":"2024-04-13T03:09:31.006024+0000","last_peered":"2024-04-13T03:09:31.006024+0000","last_clean":"2024-04-13T03:09:31.006024+0000","last_became_active":"2024-04-13T03:09:23.966596+0000","last_became_peered":"2024-04-13T03:09:23.966596+0000","last_unstale":"2024-04-13T03:09:31.006024+0000","last_undegraded":"2024-04-13T03:09:31.006024+0000","last_fullsized":"2024-04-13T03:09:31.006024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:29.744523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004384+0000","last_change":"2024-04-13T03:09:29.007029+0000","last_active":"2024-04-13T03:09:31.004384+0000","last_peered":"2024-04-13T03:09:31.004384+0000","last_clean":"2024-04-13T03:09:31.004384+0000","last_became_active":"2024-04-13T03:09:29.006625+0000","last_became_peered":"2024-04-13T03:09:29.006625+0000","last_unstale":"2024-04-13T03:09:31.004384+0000","last_undegraded":"2024-04-13T03:09:31.004384+0000","last_fullsized":"2024-04-13T03:09:31.004384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074033+0000","last_change":"2024-04-13T03:09:24.978991+0000","last_active":"2024-04-13T03:09:32.074033+0000","last_peered":"2024-04-13T03:09:32.074033+0000","last_clean":"2024-04-13T03:09:32.074033+0000","last_became_active":"2024-04-13T03:09:24.978725+0000","last_became_peered":"2024-04-13T03:09:24.978725+0000","last_unstale":"2024-04-13T03:09:32.074033+0000","last_undegraded":"2024-04-13T03:09:32.074033+0000","last_fullsized":"2024-04-13T03:09:32.074033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.969378+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.969098+0000","last_became_peered":"2024-04-13T03:09:23.969098+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004694+0000","last_change":"2024-04-13T03:09:29.005610+0000","last_active":"2024-04-13T03:09:31.004694+0000","last_peered":"2024-04-13T03:09:31.004694+0000","last_clean":"2024-04-13T03:09:31.004694+0000","last_became_active":"2024-04-13T03:09:29.005192+0000","last_became_peered":"2024-04-13T03:09:29.005192+0000","last_unstale":"2024-04-13T03:09:31.004694+0000","last_undegraded":"2024-04-13T03:09:31.004694+0000","last_fullsized":"2024-04-13T03:09:31.004694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073718+0000","last_change":"2024-04-13T03:09:24.984278+0000","last_active":"2024-04-13T03:09:32.073718+0000","last_peered":"2024-04-13T03:09:32.073718+0000","last_clean":"2024-04-13T03:09:32.073718+0000","last_became_active":"2024-04-13T03:09:24.983967+0000","last_became_peered":"2024-04-13T03:09:24.983967+0000","last_unstale":"2024-04-13T03:09:32.073718+0000","last_undegraded":"2024-04-13T03:09:32.073718+0000","last_fullsized":"2024-04-13T03:09:32.073718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072991+0000","last_change":"2024-04-13T03:09:23.975431+0000","last_active":"2024-04-13T03:09:32.072991+0000","last_peered":"2024-04-13T03:09:32.072991+0000","last_clean":"2024-04-13T03:09:32.072991+0000","last_became_active":"2024-04-13T03:09:23.974255+0000","last_became_peered":"2024-04-13T03:09:23.974255+0000","last_unstale":"2024-04-13T03:09:32.072991+0000","last_undegraded":"2024-04-13T03:09:32.072991+0000","last_fullsized":"2024-04-13T03:09:32.072991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017273+0000","last_change":"2024-04-13T03:09:29.567710+0000","last_active":"2024-04-13T03:09:31.017273+0000","last_peered":"2024-04-13T03:09:31.017273+0000","last_clean":"2024-04-13T03:09:31.017273+0000","last_became_active":"2024-04-13T03:09:29.567230+0000","last_became_peered":"2024-04-13T03:09:29.567230+0000","last_unstale":"2024-04-13T03:09:31.017273+0000","last_undegraded":"2024-04-13T03:09:31.017273+0000","last_fullsized":"2024-04-13T03:09:31.017273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:23:03.577409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073579+0000","last_change":"2024-04-13T03:09:24.978027+0000","last_active":"2024-04-13T03:09:32.073579+0000","last_peered":"2024-04-13T03:09:32.073579+0000","last_clean":"2024-04-13T03:09:32.073579+0000","last_became_active":"2024-04-13T03:09:24.977709+0000","last_became_peered":"2024-04-13T03:09:24.977709+0000","last_unstale":"2024-04-13T03:09:32.073579+0000","last_undegraded":"2024-04-13T03:09:32.073579+0000","last_fullsized":"2024-04-13T03:09:32.073579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006349+0000","last_change":"2024-04-13T03:09:23.972192+0000","last_active":"2024-04-13T03:09:31.006349+0000","last_peered":"2024-04-13T03:09:31.006349+0000","last_clean":"2024-04-13T03:09:31.006349+0000","last_became_active":"2024-04-13T03:09:23.971856+0000","last_became_peered":"2024-04-13T03:09:23.971856+0000","last_unstale":"2024-04-13T03:09:31.006349+0000","last_undegraded":"2024-04-13T03:09:31.006349+0000","last_fullsized":"2024-04-13T03:09:31.006349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017103+0000","last_change":"2024-04-13T03:09:29.565473+0000","last_active":"2024-04-13T03:09:31.017103+0000","last_peered":"2024-04-13T03:09:31.017103+0000","last_clean":"2024-04-13T03:09:31.017103+0000","last_became_active":"2024-04-13T03:09:29.565099+0000","last_became_peered":"2024-04-13T03:09:29.565099+0000","last_unstale":"2024-04-13T03:09:31.017103+0000","last_undegraded":"2024-04-13T03:09:31.017103+0000","last_fullsized":"2024-04-13T03:09:31.017103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:49:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004475+0000","last_change":"2024-04-13T03:09:24.980510+0000","last_active":"2024-04-13T03:09:31.004475+0000","last_peered":"2024-04-13T03:09:31.004475+0000","last_clean":"2024-04-13T03:09:31.004475+0000","last_became_active":"2024-04-13T03:09:24.980226+0000","last_became_peered":"2024-04-13T03:09:24.980226+0000","last_unstale":"2024-04-13T03:09:31.004475+0000","last_undegraded":"2024-04-13T03:09:31.004475+0000","last_fullsized":"2024-04-13T03:09:31.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004618+0000","last_change":"2024-04-13T03:09:23.971205+0000","last_active":"2024-04-13T03:09:31.004618+0000","last_peered":"2024-04-13T03:09:31.004618+0000","last_clean":"2024-04-13T03:09:31.004618+0000","last_became_active":"2024-04-13T03:09:23.970845+0000","last_became_peered":"2024-04-13T03:09:23.970845+0000","last_unstale":"2024-04-13T03:09:31.004618+0000","last_undegraded":"2024-04-13T03:09:31.004618+0000","last_fullsized":"2024-04-13T03:09:31.004618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005480+0000","last_change":"2024-04-13T03:09:29.568847+0000","last_active":"2024-04-13T03:09:31.005480+0000","last_peered":"2024-04-13T03:09:31.005480+0000","last_clean":"2024-04-13T03:09:31.005480+0000","last_became_active":"2024-04-13T03:09:29.568286+0000","last_became_peered":"2024-04-13T03:09:29.568286+0000","last_unstale":"2024-04-13T03:09:31.005480+0000","last_undegraded":"2024-04-13T03:09:31.005480+0000","last_fullsized":"2024-04-13T03:09:31.005480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073615+0000","last_change":"2024-04-13T03:09:24.980651+0000","last_active":"2024-04-13T03:09:32.073615+0000","last_peered":"2024-04-13T03:09:32.073615+0000","last_clean":"2024-04-13T03:09:32.073615+0000","last_became_active":"2024-04-13T03:09:24.980390+0000","last_became_peered":"2024-04-13T03:09:24.980390+0000","last_unstale":"2024-04-13T03:09:32.073615+0000","last_undegraded":"2024-04-13T03:09:32.073615+0000","last_fullsized":"2024-04-13T03:09:32.073615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:34.382842+0000","last_change":"2024-04-13T03:09:23.977841+0000","last_active":"2024-04-13T03:09:34.382842+0000","last_peered":"2024-04-13T03:09:34.382842+0000","last_clean":"2024-04-13T03:09:34.382842+0000","last_became_active":"2024-04-13T03:09:23.977546+0000","last_became_peered":"2024-04-13T03:09:23.977546+0000","last_unstale":"2024-04-13T03:09:34.382842+0000","last_undegraded":"2024-04-13T03:09:34.382842+0000","last_fullsized":"2024-04-13T03:09:34.382842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552322+0000","last_change":"2024-04-13T03:09:29.012775+0000","last_active":"2024-04-13T03:09:31.552322+0000","last_peered":"2024-04-13T03:09:31.552322+0000","last_clean":"2024-04-13T03:09:31.552322+0000","last_became_active":"2024-04-13T03:09:29.011952+0000","last_became_peered":"2024-04-13T03:09:29.011952+0000","last_unstale":"2024-04-13T03:09:31.552322+0000","last_undegraded":"2024-04-13T03:09:31.552322+0000","last_fullsized":"2024-04-13T03:09:31.552322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005496+0000","last_change":"2024-04-13T03:09:24.980974+0000","last_active":"2024-04-13T03:09:31.005496+0000","last_peered":"2024-04-13T03:09:31.005496+0000","last_clean":"2024-04-13T03:09:31.005496+0000","last_became_active":"2024-04-13T03:09:24.980580+0000","last_became_peered":"2024-04-13T03:09:24.980580+0000","last_unstale":"2024-04-13T03:09:31.005496+0000","last_undegraded":"2024-04-13T03:09:31.005496+0000","last_fullsized":"2024-04-13T03:09:31.005496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552665+0000","last_change":"2024-04-13T03:09:23.970652+0000","last_active":"2024-04-13T03:09:31.552665+0000","last_peered":"2024-04-13T03:09:31.552665+0000","last_clean":"2024-04-13T03:09:31.552665+0000","last_became_active":"2024-04-13T03:09:23.970405+0000","last_became_peered":"2024-04-13T03:09:23.970405+0000","last_unstale":"2024-04-13T03:09:31.552665+0000","last_undegraded":"2024-04-13T03:09:31.552665+0000","last_fullsized":"2024-04-13T03:09:31.552665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004925+0000","last_change":"2024-04-13T03:09:29.009723+0000","last_active":"2024-04-13T03:09:31.004925+0000","last_peered":"2024-04-13T03:09:31.004925+0000","last_clean":"2024-04-13T03:09:31.004925+0000","last_became_active":"2024-04-13T03:09:29.009177+0000","last_became_peered":"2024-04-13T03:09:29.009177+0000","last_unstale":"2024-04-13T03:09:31.004925+0000","last_undegraded":"2024-04-13T03:09:31.004925+0000","last_fullsized":"2024-04-13T03:09:31.004925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004826+0000","last_change":"2024-04-13T03:09:24.976839+0000","last_active":"2024-04-13T03:09:31.004826+0000","last_peered":"2024-04-13T03:09:31.004826+0000","last_clean":"2024-04-13T03:09:31.004826+0000","last_became_active":"2024-04-13T03:09:24.976565+0000","last_became_peered":"2024-04-13T03:09:24.976565+0000","last_unstale":"2024-04-13T03:09:31.004826+0000","last_undegraded":"2024-04-13T03:09:31.004826+0000","last_fullsized":"2024-04-13T03:09:31.004826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005802+0000","last_change":"2024-04-13T03:09:23.974057+0000","last_active":"2024-04-13T03:09:31.005802+0000","last_peered":"2024-04-13T03:09:31.005802+0000","last_clean":"2024-04-13T03:09:31.005802+0000","last_became_active":"2024-04-13T03:09:23.973783+0000","last_became_peered":"2024-04-13T03:09:23.973783+0000","last_unstale":"2024-04-13T03:09:31.005802+0000","last_undegraded":"2024-04-13T03:09:31.005802+0000","last_fullsized":"2024-04-13T03:09:31.005802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016630+0000","last_change":"2024-04-13T03:09:29.008489+0000","last_active":"2024-04-13T03:09:31.016630+0000","last_peered":"2024-04-13T03:09:31.016630+0000","last_clean":"2024-04-13T03:09:31.016630+0000","last_became_active":"2024-04-13T03:09:29.008129+0000","last_became_peered":"2024-04-13T03:09:29.008129+0000","last_unstale":"2024-04-13T03:09:31.016630+0000","last_undegraded":"2024-04-13T03:09:31.016630+0000","last_fullsized":"2024-04-13T03:09:31.016630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:03.613440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005713+0000","last_change":"2024-04-13T03:09:24.988904+0000","last_active":"2024-04-13T03:09:31.005713+0000","last_peered":"2024-04-13T03:09:31.005713+0000","last_clean":"2024-04-13T03:09:31.005713+0000","last_became_active":"2024-04-13T03:09:24.988667+0000","last_became_peered":"2024-04-13T03:09:24.988667+0000","last_unstale":"2024-04-13T03:09:31.005713+0000","last_undegraded":"2024-04-13T03:09:31.005713+0000","last_fullsized":"2024-04-13T03:09:31.005713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074106+0000","last_change":"2024-04-13T03:09:23.975478+0000","last_active":"2024-04-13T03:09:32.074106+0000","last_peered":"2024-04-13T03:09:32.074106+0000","last_clean":"2024-04-13T03:09:32.074106+0000","last_became_active":"2024-04-13T03:09:23.975155+0000","last_became_peered":"2024-04-13T03:09:23.975155+0000","last_unstale":"2024-04-13T03:09:32.074106+0000","last_undegraded":"2024-04-13T03:09:32.074106+0000","last_fullsized":"2024-04-13T03:09:32.074106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016599+0000","last_change":"2024-04-13T03:09:29.565039+0000","last_active":"2024-04-13T03:09:31.016599+0000","last_peered":"2024-04-13T03:09:31.016599+0000","last_clean":"2024-04-13T03:09:31.016599+0000","last_became_active":"2024-04-13T03:09:29.564715+0000","last_became_peered":"2024-04-13T03:09:29.564715+0000","last_unstale":"2024-04-13T03:09:31.016599+0000","last_undegraded":"2024-04-13T03:09:31.016599+0000","last_fullsized":"2024-04-13T03:09:31.016599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:38.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004534+0000","last_change":"2024-04-13T03:09:29.567360+0000","last_active":"2024-04-13T03:09:31.004534+0000","last_peered":"2024-04-13T03:09:31.004534+0000","last_clean":"2024-04-13T03:09:31.004534+0000","last_became_active":"2024-04-13T03:09:29.567005+0000","last_became_peered":"2024-04-13T03:09:29.567005+0000","last_unstale":"2024-04-13T03:09:31.004534+0000","last_undegraded":"2024-04-13T03:09:31.004534+0000","last_fullsized":"2024-04-13T03:09:31.004534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552017+0000","last_change":"2024-04-13T03:09:24.980403+0000","last_active":"2024-04-13T03:09:31.552017+0000","last_peered":"2024-04-13T03:09:31.552017+0000","last_clean":"2024-04-13T03:09:31.552017+0000","last_became_active":"2024-04-13T03:09:24.980125+0000","last_became_peered":"2024-04-13T03:09:24.980125+0000","last_unstale":"2024-04-13T03:09:31.552017+0000","last_undegraded":"2024-04-13T03:09:31.552017+0000","last_fullsized":"2024-04-13T03:09:31.552017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006298+0000","last_change":"2024-04-13T03:09:23.966400+0000","last_active":"2024-04-13T03:09:31.006298+0000","last_peered":"2024-04-13T03:09:31.006298+0000","last_clean":"2024-04-13T03:09:31.006298+0000","last_became_active":"2024-04-13T03:09:23.966136+0000","last_became_peered":"2024-04-13T03:09:23.966136+0000","last_unstale":"2024-04-13T03:09:31.006298+0000","last_undegraded":"2024-04-13T03:09:31.006298+0000","last_fullsized":"2024-04-13T03:09:31.006298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074769+0000","last_change":"2024-04-13T03:09:29.008579+0000","last_active":"2024-04-13T03:09:32.074769+0000","last_peered":"2024-04-13T03:09:32.074769+0000","last_clean":"2024-04-13T03:09:32.074769+0000","last_became_active":"2024-04-13T03:09:29.008030+0000","last_became_peered":"2024-04-13T03:09:29.008030+0000","last_unstale":"2024-04-13T03:09:32.074769+0000","last_undegraded":"2024-04-13T03:09:32.074769+0000","last_fullsized":"2024-04-13T03:09:32.074769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016574+0000","last_change":"2024-04-13T03:09:24.972241+0000","last_active":"2024-04-13T03:09:31.016574+0000","last_peered":"2024-04-13T03:09:31.016574+0000","last_clean":"2024-04-13T03:09:31.016574+0000","last_became_active":"2024-04-13T03:09:24.971962+0000","last_became_peered":"2024-04-13T03:09:24.971962+0000","last_unstale":"2024-04-13T03:09:31.016574+0000","last_undegraded":"2024-04-13T03:09:31.016574+0000","last_fullsized":"2024-04-13T03:09:31.016574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016806+0000","last_change":"2024-04-13T03:09:23.977688+0000","last_active":"2024-04-13T03:09:31.016806+0000","last_peered":"2024-04-13T03:09:31.016806+0000","last_clean":"2024-04-13T03:09:31.016806+0000","last_became_active":"2024-04-13T03:09:23.977412+0000","last_became_peered":"2024-04-13T03:09:23.977412+0000","last_unstale":"2024-04-13T03:09:31.016806+0000","last_undegraded":"2024-04-13T03:09:31.016806+0000","last_fullsized":"2024-04-13T03:09:31.016806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072995+0000","last_change":"2024-04-13T03:09:23.973771+0000","last_active":"2024-04-13T03:09:32.072995+0000","last_peered":"2024-04-13T03:09:32.072995+0000","last_clean":"2024-04-13T03:09:32.072995+0000","last_became_active":"2024-04-13T03:09:23.973271+0000","last_became_peered":"2024-04-13T03:09:23.973271+0000","last_unstale":"2024-04-13T03:09:32.072995+0000","last_undegraded":"2024-04-13T03:09:32.072995+0000","last_fullsized":"2024-04-13T03:09:32.072995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004613+0000","last_change":"2024-04-13T03:09:24.974833+0000","last_active":"2024-04-13T03:09:31.004613+0000","last_peered":"2024-04-13T03:09:31.004613+0000","last_clean":"2024-04-13T03:09:31.004613+0000","last_became_active":"2024-04-13T03:09:24.974554+0000","last_became_peered":"2024-04-13T03:09:24.974554+0000","last_unstale":"2024-04-13T03:09:31.004613+0000","last_undegraded":"2024-04-13T03:09:31.004613+0000","last_fullsized":"2024-04-13T03:09:31.004613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072905+0000","last_change":"2024-04-13T03:09:29.004327+0000","last_active":"2024-04-13T03:09:32.072905+0000","last_peered":"2024-04-13T03:09:32.072905+0000","last_clean":"2024-04-13T03:09:32.072905+0000","last_became_active":"2024-04-13T03:09:29.003998+0000","last_became_peered":"2024-04-13T03:09:29.003998+0000","last_unstale":"2024-04-13T03:09:32.072905+0000","last_undegraded":"2024-04-13T03:09:32.072905+0000","last_fullsized":"2024-04-13T03:09:32.072905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:10:12.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016845+0000","last_change":"2024-04-13T03:09:23.976680+0000","last_active":"2024-04-13T03:09:31.016845+0000","last_peered":"2024-04-13T03:09:31.016845+0000","last_clean":"2024-04-13T03:09:31.016845+0000","last_became_active":"2024-04-13T03:09:23.976390+0000","last_became_peered":"2024-04-13T03:09:23.976390+0000","last_unstale":"2024-04-13T03:09:31.016845+0000","last_undegraded":"2024-04-13T03:09:31.016845+0000","last_fullsized":"2024-04-13T03:09:31.016845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004533+0000","last_change":"2024-04-13T03:09:24.971520+0000","last_active":"2024-04-13T03:09:31.004533+0000","last_peered":"2024-04-13T03:09:31.004533+0000","last_clean":"2024-04-13T03:09:31.004533+0000","last_became_active":"2024-04-13T03:09:24.971236+0000","last_became_peered":"2024-04-13T03:09:24.971236+0000","last_unstale":"2024-04-13T03:09:31.004533+0000","last_undegraded":"2024-04-13T03:09:31.004533+0000","last_fullsized":"2024-04-13T03:09:31.004533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:47.399993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017010+0000","last_change":"2024-04-13T03:09:29.008243+0000","last_active":"2024-04-13T03:09:31.017010+0000","last_peered":"2024-04-13T03:09:31.017010+0000","last_clean":"2024-04-13T03:09:31.017010+0000","last_became_active":"2024-04-13T03:09:29.007854+0000","last_became_peered":"2024-04-13T03:09:29.007854+0000","last_unstale":"2024-04-13T03:09:31.017010+0000","last_undegraded":"2024-04-13T03:09:31.017010+0000","last_fullsized":"2024-04-13T03:09:31.017010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:33.651724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073532+0000","last_change":"2024-04-13T03:09:24.984919+0000","last_active":"2024-04-13T03:09:32.073532+0000","last_peered":"2024-04-13T03:09:32.073532+0000","last_clean":"2024-04-13T03:09:32.073532+0000","last_became_active":"2024-04-13T03:09:24.984652+0000","last_became_peered":"2024-04-13T03:09:24.984652+0000","last_unstale":"2024-04-13T03:09:32.073532+0000","last_undegraded":"2024-04-13T03:09:32.073532+0000","last_fullsized":"2024-04-13T03:09:32.073532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072697+0000","last_change":"2024-04-13T03:09:24.980195+0000","last_active":"2024-04-13T03:09:32.072697+0000","last_peered":"2024-04-13T03:09:32.072697+0000","last_clean":"2024-04-13T03:09:32.072697+0000","last_became_active":"2024-04-13T03:09:24.979846+0000","last_became_peered":"2024-04-13T03:09:24.979846+0000","last_unstale":"2024-04-13T03:09:32.072697+0000","last_undegraded":"2024-04-13T03:09:32.072697+0000","last_fullsized":"2024-04-13T03:09:32.072697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006342+0000","last_change":"2024-04-13T03:09:24.979746+0000","last_active":"2024-04-13T03:09:31.006342+0000","last_peered":"2024-04-13T03:09:31.006342+0000","last_clean":"2024-04-13T03:09:31.006342+0000","last_became_active":"2024-04-13T03:09:24.979481+0000","last_became_peered":"2024-04-13T03:09:24.979481+0000","last_unstale":"2024-04-13T03:09:31.006342+0000","last_undegraded":"2024-04-13T03:09:31.006342+0000","last_fullsized":"2024-04-13T03:09:31.006342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004510+0000","last_change":"2024-04-13T03:09:24.974603+0000","last_active":"2024-04-13T03:09:31.004510+0000","last_peered":"2024-04-13T03:09:31.004510+0000","last_clean":"2024-04-13T03:09:31.004510+0000","last_became_active":"2024-04-13T03:09:24.974099+0000","last_became_peered":"2024-04-13T03:09:24.974099+0000","last_unstale":"2024-04-13T03:09:31.004510+0000","last_undegraded":"2024-04-13T03:09:31.004510+0000","last_fullsized":"2024-04-13T03:09:31.004510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017151+0000","last_change":"2024-04-13T03:09:24.977844+0000","last_active":"2024-04-13T03:09:31.017151+0000","last_peered":"2024-04-13T03:09:31.017151+0000","last_clean":"2024-04-13T03:09:31.017151+0000","last_became_active":"2024-04-13T03:09:24.977578+0000","last_became_peered":"2024-04-13T03:09:24.977578+0000","last_unstale":"2024-04-13T03:09:31.017151+0000","last_undegraded":"2024-04-13T03:09:31.017151+0000","last_fullsized":"2024-04-13T03:09:31.017151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552548+0000","last_change":"2024-04-13T03:09:24.981088+0000","last_active":"2024-04-13T03:09:31.552548+0000","last_peered":"2024-04-13T03:09:31.552548+0000","last_clean":"2024-04-13T03:09:31.552548+0000","last_became_active":"2024-04-13T03:09:24.980793+0000","last_became_peered":"2024-04-13T03:09:24.980793+0000","last_unstale":"2024-04-13T03:09:31.552548+0000","last_undegraded":"2024-04-13T03:09:31.552548+0000","last_fullsized":"2024-04-13T03:09:31.552548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074858+0000","last_change":"2024-04-13T03:09:24.990583+0000","last_active":"2024-04-13T03:09:32.074858+0000","last_peered":"2024-04-13T03:09:32.074858+0000","last_clean":"2024-04-13T03:09:32.074858+0000","last_became_active":"2024-04-13T03:09:24.990319+0000","last_became_peered":"2024-04-13T03:09:24.990319+0000","last_unstale":"2024-04-13T03:09:32.074858+0000","last_undegraded":"2024-04-13T03:09:32.074858+0000","last_fullsized":"2024-04-13T03:09:32.074858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017256+0000","last_change":"2024-04-13T03:09:24.975238+0000","last_active":"2024-04-13T03:09:31.017256+0000","last_peered":"2024-04-13T03:09:31.017256+0000","last_clean":"2024-04-13T03:09:31.017256+0000","last_became_active":"2024-04-13T03:09:24.974911+0000","last_became_peered":"2024-04-13T03:09:24.974911+0000","last_unstale":"2024-04-13T03:09:31.017256+0000","last_undegraded":"2024-04-13T03:09:31.017256+0000","last_fullsized":"2024-04-13T03:09:31.017256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005161+0000","last_change":"2024-04-13T03:09:24.976029+0000","last_active":"2024-04-13T03:09:31.005161+0000","last_peered":"2024-04-13T03:09:31.005161+0000","last_clean":"2024-04-13T03:09:31.005161+0000","last_became_active":"2024-04-13T03:09:24.975739+0000","last_became_peered":"2024-04-13T03:09:24.975739+0000","last_unstale":"2024-04-13T03:09:31.005161+0000","last_undegraded":"2024-04-13T03:09:31.005161+0000","last_fullsized":"2024-04-13T03:09:31.005161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016664+0000","last_change":"2024-04-13T03:09:24.975447+0000","last_active":"2024-04-13T03:09:31.016664+0000","last_peered":"2024-04-13T03:09:31.016664+0000","last_clean":"2024-04-13T03:09:31.016664+0000","last_became_active":"2024-04-13T03:09:24.975131+0000","last_became_peered":"2024-04-13T03:09:24.975131+0000","last_unstale":"2024-04-13T03:09:31.016664+0000","last_undegraded":"2024-04-13T03:09:31.016664+0000","last_fullsized":"2024-04-13T03:09:31.016664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006028+0000","last_change":"2024-04-13T03:09:24.988399+0000","last_active":"2024-04-13T03:09:31.006028+0000","last_peered":"2024-04-13T03:09:31.006028+0000","last_clean":"2024-04-13T03:09:31.006028+0000","last_became_active":"2024-04-13T03:09:24.988064+0000","last_became_peered":"2024-04-13T03:09:24.988064+0000","last_unstale":"2024-04-13T03:09:31.006028+0000","last_undegraded":"2024-04-13T03:09:31.006028+0000","last_fullsized":"2024-04-13T03:09:31.006028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004399+0000","last_change":"2024-04-13T03:09:24.980428+0000","last_active":"2024-04-13T03:09:31.004399+0000","last_peered":"2024-04-13T03:09:31.004399+0000","last_clean":"2024-04-13T03:09:31.004399+0000","last_became_active":"2024-04-13T03:09:24.980157+0000","last_became_peered":"2024-04-13T03:09:24.980157+0000","last_unstale":"2024-04-13T03:09:31.004399+0000","last_undegraded":"2024-04-13T03:09:31.004399+0000","last_fullsized":"2024-04-13T03:09:31.004399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074511+0000","last_change":"2024-04-13T03:09:24.978696+0000","last_active":"2024-04-13T03:09:32.074511+0000","last_peered":"2024-04-13T03:09:32.074511+0000","last_clean":"2024-04-13T03:09:32.074511+0000","last_became_active":"2024-04-13T03:09:24.978413+0000","last_became_peered":"2024-04-13T03:09:24.978413+0000","last_unstale":"2024-04-13T03:09:32.074511+0000","last_undegraded":"2024-04-13T03:09:32.074511+0000","last_fullsized":"2024-04-13T03:09:32.074511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074117+0000","last_change":"2024-04-13T03:09:24.979845+0000","last_active":"2024-04-13T03:09:32.074117+0000","last_peered":"2024-04-13T03:09:32.074117+0000","last_clean":"2024-04-13T03:09:32.074117+0000","last_became_active":"2024-04-13T03:09:24.979598+0000","last_became_peered":"2024-04-13T03:09:24.979598+0000","last_unstale":"2024-04-13T03:09:32.074117+0000","last_undegraded":"2024-04-13T03:09:32.074117+0000","last_fullsized":"2024-04-13T03:09:32.074117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073041+0000","last_change":"2024-04-13T03:09:24.978510+0000","last_active":"2024-04-13T03:09:32.073041+0000","last_peered":"2024-04-13T03:09:32.073041+0000","last_clean":"2024-04-13T03:09:32.073041+0000","last_became_active":"2024-04-13T03:09:24.978226+0000","last_became_peered":"2024-04-13T03:09:24.978226+0000","last_unstale":"2024-04-13T03:09:32.073041+0000","last_undegraded":"2024-04-13T03:09:32.073041+0000","last_fullsized":"2024-04-13T03:09:32.073041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006512+0000","last_change":"2024-04-13T03:09:24.986783+0000","last_active":"2024-04-13T03:09:31.006512+0000","last_peered":"2024-04-13T03:09:31.006512+0000","last_clean":"2024-04-13T03:09:31.006512+0000","last_became_active":"2024-04-13T03:09:24.986139+0000","last_became_peered":"2024-04-13T03:09:24.986139+0000","last_unstale":"2024-04-13T03:09:31.006512+0000","last_undegraded":"2024-04-13T03:09:31.006512+0000","last_fullsized":"2024-04-13T03:09:31.006512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005046+0000","last_change":"2024-04-13T03:09:24.974308+0000","last_active":"2024-04-13T03:09:31.005046+0000","last_peered":"2024-04-13T03:09:31.005046+0000","last_clean":"2024-04-13T03:09:31.005046+0000","last_became_active":"2024-04-13T03:09:24.973935+0000","last_became_peered":"2024-04-13T03:09:24.973935+0000","last_unstale":"2024-04-13T03:09:31.005046+0000","last_undegraded":"2024-04-13T03:09:31.005046+0000","last_fullsized":"2024-04-13T03:09:31.005046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552097+0000","last_change":"2024-04-13T03:09:24.971657+0000","last_active":"2024-04-13T03:09:31.552097+0000","last_peered":"2024-04-13T03:09:31.552097+0000","last_clean":"2024-04-13T03:09:31.552097+0000","last_became_active":"2024-04-13T03:09:24.971379+0000","last_became_peered":"2024-04-13T03:09:24.971379+0000","last_unstale":"2024-04-13T03:09:31.552097+0000","last_undegraded":"2024-04-13T03:09:31.552097+0000","last_fullsized":"2024-04-13T03:09:31.552097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004364+0000","last_change":"2024-04-13T03:09:24.973741+0000","last_active":"2024-04-13T03:09:31.004364+0000","last_peered":"2024-04-13T03:09:31.004364+0000","last_clean":"2024-04-13T03:09:31.004364+0000","last_became_active":"2024-04-13T03:09:24.973468+0000","last_became_peered":"2024-04-13T03:09:24.973468+0000","last_unstale":"2024-04-13T03:09:31.004364+0000","last_undegraded":"2024-04-13T03:09:31.004364+0000","last_fullsized":"2024-04-13T03:09:31.004364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:20.516278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016730+0000","last_change":"2024-04-13T03:09:24.982585+0000","last_active":"2024-04-13T03:09:31.016730+0000","last_peered":"2024-04-13T03:09:31.016730+0000","last_clean":"2024-04-13T03:09:31.016730+0000","last_became_active":"2024-04-13T03:09:24.982301+0000","last_became_peered":"2024-04-13T03:09:24.982301+0000","last_unstale":"2024-04-13T03:09:31.016730+0000","last_undegraded":"2024-04-13T03:09:31.016730+0000","last_fullsized":"2024-04-13T03:09:31.016730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005384+0000","last_change":"2024-04-13T03:09:24.987283+0000","last_active":"2024-04-13T03:09:31.005384+0000","last_peered":"2024-04-13T03:09:31.005384+0000","last_clean":"2024-04-13T03:09:31.005384+0000","last_became_active":"2024-04-13T03:09:24.983517+0000","last_became_peered":"2024-04-13T03:09:24.983517+0000","last_unstale":"2024-04-13T03:09:31.005384+0000","last_undegraded":"2024-04-13T03:09:31.005384+0000","last_fullsized":"2024-04-13T03:09:31.005384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:43.988718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074632+0000","last_change":"2024-04-13T03:09:24.994512+0000","last_active":"2024-04-13T03:09:32.074632+0000","last_peered":"2024-04-13T03:09:32.074632+0000","last_clean":"2024-04-13T03:09:32.074632+0000","last_became_active":"2024-04-13T03:09:24.994212+0000","last_became_peered":"2024-04-13T03:09:24.994212+0000","last_unstale":"2024-04-13T03:09:32.074632+0000","last_undegraded":"2024-04-13T03:09:32.074632+0000","last_fullsized":"2024-04-13T03:09:32.074632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074805+0000","last_change":"2024-04-13T03:09:24.990511+0000","last_active":"2024-04-13T03:09:32.074805+0000","last_peered":"2024-04-13T03:09:32.074805+0000","last_clean":"2024-04-13T03:09:32.074805+0000","last_became_active":"2024-04-13T03:09:24.990230+0000","last_became_peered":"2024-04-13T03:09:24.990230+0000","last_unstale":"2024-04-13T03:09:32.074805+0000","last_undegraded":"2024-04-13T03:09:32.074805+0000","last_fullsized":"2024-04-13T03:09:32.074805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074337+0000","last_change":"2024-04-13T03:09:24.991357+0000","last_active":"2024-04-13T03:09:32.074337+0000","last_peered":"2024-04-13T03:09:32.074337+0000","last_clean":"2024-04-13T03:09:32.074337+0000","last_became_active":"2024-04-13T03:09:24.990754+0000","last_became_peered":"2024-04-13T03:09:24.990754+0000","last_unstale":"2024-04-13T03:09:32.074337+0000","last_undegraded":"2024-04-13T03:09:32.074337+0000","last_fullsized":"2024-04-13T03:09:32.074337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016489+0000","last_change":"2024-04-13T03:09:24.984967+0000","last_active":"2024-04-13T03:09:31.016489+0000","last_peered":"2024-04-13T03:09:31.016489+0000","last_clean":"2024-04-13T03:09:31.016489+0000","last_became_active":"2024-04-13T03:09:24.984674+0000","last_became_peered":"2024-04-13T03:09:24.984674+0000","last_unstale":"2024-04-13T03:09:31.016489+0000","last_undegraded":"2024-04-13T03:09:31.016489+0000","last_fullsized":"2024-04-13T03:09:31.016489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073906+0000","last_change":"2024-04-13T03:09:24.976323+0000","last_active":"2024-04-13T03:09:32.073906+0000","last_peered":"2024-04-13T03:09:32.073906+0000","last_clean":"2024-04-13T03:09:32.073906+0000","last_became_active":"2024-04-13T03:09:24.976014+0000","last_became_peered":"2024-04-13T03:09:24.976014+0000","last_unstale":"2024-04-13T03:09:32.073906+0000","last_undegraded":"2024-04-13T03:09:32.073906+0000","last_fullsized":"2024-04-13T03:09:32.073906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005924+0000","last_change":"2024-04-13T03:09:24.987113+0000","last_active":"2024-04-13T03:09:31.005924+0000","last_peered":"2024-04-13T03:09:31.005924+0000","last_clean":"2024-04-13T03:09:31.005924+0000","last_became_active":"2024-04-13T03:09:24.986836+0000","last_became_peered":"2024-04-13T03:09:24.986836+0000","last_unstale":"2024-04-13T03:09:31.005924+0000","last_undegraded":"2024-04-13T03:09:31.005924+0000","last_fullsized":"2024-04-13T03:09:31.005924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073397+0000","last_change":"2024-04-13T03:09:24.979549+0000","last_active":"2024-04-13T03:09:32.073397+0000","last_peered":"2024-04-13T03:09:32.073397+0000","last_clean":"2024-04-13T03:09:32.073397+0000","last_became_active":"2024-04-13T03:09:24.979203+0000","last_became_peered":"2024-04-13T03:09:24.979203+0000","last_unstale":"2024-04-13T03:09:32.073397+0000","last_undegraded":"2024-04-13T03:09:32.073397+0000","last_fullsized":"2024-04-13T03:09:32.073397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006135+0000","last_change":"2024-04-13T03:09:24.986349+0000","last_active":"2024-04-13T03:09:31.006135+0000","last_peered":"2024-04-13T03:09:31.006135+0000","last_clean":"2024-04-13T03:09:31.006135+0000","last_became_active":"2024-04-13T03:09:24.985694+0000","last_became_peered":"2024-04-13T03:09:24.985694+0000","last_unstale":"2024-04-13T03:09:31.006135+0000","last_undegraded":"2024-04-13T03:09:31.006135+0000","last_fullsized":"2024-04-13T03:09:31.006135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004564+0000","last_change":"2024-04-13T03:09:24.973883+0000","last_active":"2024-04-13T03:09:31.004564+0000","last_peered":"2024-04-13T03:09:31.004564+0000","last_clean":"2024-04-13T03:09:31.004564+0000","last_became_active":"2024-04-13T03:09:24.973623+0000","last_became_peered":"2024-04-13T03:09:24.973623+0000","last_unstale":"2024-04-13T03:09:31.004564+0000","last_undegraded":"2024-04-13T03:09:31.004564+0000","last_fullsized":"2024-04-13T03:09:31.004564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552728+0000","last_change":"2024-04-13T03:09:24.980289+0000","last_active":"2024-04-13T03:09:31.552728+0000","last_peered":"2024-04-13T03:09:31.552728+0000","last_clean":"2024-04-13T03:09:31.552728+0000","last_became_active":"2024-04-13T03:09:24.980040+0000","last_became_peered":"2024-04-13T03:09:24.980040+0000","last_unstale":"2024-04-13T03:09:31.552728+0000","last_undegraded":"2024-04-13T03:09:31.552728+0000","last_fullsized":"2024-04-13T03:09:31.552728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074027+0000","last_change":"2024-04-13T03:09:24.985108+0000","last_active":"2024-04-13T03:09:32.074027+0000","last_peered":"2024-04-13T03:09:32.074027+0000","last_clean":"2024-04-13T03:09:32.074027+0000","last_became_active":"2024-04-13T03:09:24.984576+0000","last_became_peered":"2024-04-13T03:09:24.984576+0000","last_unstale":"2024-04-13T03:09:32.074027+0000","last_undegraded":"2024-04-13T03:09:32.074027+0000","last_fullsized":"2024-04-13T03:09:32.074027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:09:52.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:09:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.500+0000 7f09b015e640 1 Processor -- start 2024-04-13T03:09:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.500+0000 7f09b015e640 1 -- start start 2024-04-13T03:09:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09b015e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09a815ebf0 0x7f09a806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09b015e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 0x7f09a806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09b015e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a80ce6f0 con 0x7f09a80b5bf0 2024-04-13T03:09:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09b015e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09a815fa70 con 0x7f09a815ebf0 2024-04-13T03:09:52.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09b015e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f09a815f420 con 0x7f09a80ce1f0 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09a815ebf0 0x7f09a806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09ae6fb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09a80b5bf0 0x7f09a806e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56290/0 (socket says 172.21.15.106:56290) 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09a815ebf0 0x7f09a806f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45396/0 (socket says 172.21.15.106:45396) 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09ae6fb640 1 -- 172.21.15.106:0/3151158415 learned_addr learned my addr 172.21.15.106:0/3151158415 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09ad6f9640 1 --2- 172.21.15.106:0/3151158415 >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 0x7f09a806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 -- 172.21.15.106:0/3151158415 >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 msgr2=0x7f09a806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 --2- 172.21.15.106:0/3151158415 >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 0x7f09a806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 -- 172.21.15.106:0/3151158415 >> v1:172.21.15.106:6789/0 conn(0x7f09a80b5bf0 legacy=0x7f09a806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09adefa640 1 -- 172.21.15.106:0/3151158415 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a815f700 con 0x7f09a815ebf0 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.501+0000 7f09acef8640 1 -- 172.21.15.106:0/3151158415 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645613560 0 0) 0x7f09a80ce6f0 con 0x7f09a80b5bf0 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.502+0000 7f09ad6f9640 1 --2- 172.21.15.106:0/3151158415 >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 0x7f09a806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:52.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.502+0000 7f09adefa640 1 --2- 172.21.15.106:0/3151158415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a806f780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09a80d0c20 tx=0x7f09a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c215c20b3cbf03c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.502+0000 7f09acef8640 1 -- 172.21.15.106:0/3151158415 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40315a0 con 0x7f09a815ebf0 2024-04-13T03:09:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.502+0000 7f09acef8640 1 -- 172.21.15.106:0/3151158415 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a4037070 con 0x7f09a815ebf0 2024-04-13T03:09:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.502+0000 7f09acef8640 1 -- 172.21.15.106:0/3151158415 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4031960 con 0x7f09a815ebf0 2024-04-13T03:09:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 -- 172.21.15.106:0/3151158415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 msgr2=0x7f09a806f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 --2- 172.21.15.106:0/3151158415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a806f780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09a80d0c20 tx=0x7f09a402b840 comp rx=0 tx=0).stop 2024-04-13T03:09:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 -- 172.21.15.106:0/3151158415 shutdown_connections 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 --2- 172.21.15.106:0/3151158415 >> v2:172.21.15.114:3300/0 conn(0x7f09a80ce1f0 0x7f09a806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 --2- 172.21.15.106:0/3151158415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a806f780 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 -- 172.21.15.106:0/3151158415 >> 172.21.15.106:0/3151158415 conn(0x7f09a80cd6d0 msgr2=0x7f09a80cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 -- 172.21.15.106:0/3151158415 shutdown_connections 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.503+0000 7f09b015e640 1 -- 172.21.15.106:0/3151158415 wait complete. 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.504+0000 7f09b015e640 1 Processor -- start 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.504+0000 7f09b015e640 1 -- start start 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 0x7f09a806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 0x7f09a814a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09a80b0650 con 0x7f09a815ebf0 2024-04-13T03:09:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09a80b04d0 con 0x7f09a80ce1f0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09b015e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f09a815efd0 con 0x7f09a80b5bf0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45412/0 (socket says 172.21.15.106:45412) 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 -- 172.21.15.106:0/4171389451 learned_addr learned my addr 172.21.15.106:0/4171389451 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09adefa640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 0x7f09a806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ad6f9640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 0x7f09a814a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 -- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 msgr2=0x7f09a806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 0x7f09a806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 -- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 msgr2=0x7f09a814a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 0x7f09a814a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 -- 172.21.15.106:0/4171389451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a814ac20 con 0x7f09a815ebf0 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09adefa640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 0x7f09a806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ad6f9640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 0x7f09a814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.505+0000 7f09ae6fb640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09a000c730 tx=0x7f09a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.506+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0018070 con 0x7f09a815ebf0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.506+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a000ce40 con 0x7f09a815ebf0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.506+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a806baf0 con 0x7f09a815ebf0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.506+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000a860 con 0x7f09a815ebf0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.506+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a806c0b0 con 0x7f09a815ebf0 2024-04-13T03:09:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.507+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a806f780 con 0x7f09a815ebf0 2024-04-13T03:09:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.507+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09a001b070 con 0x7f09a815ebf0 2024-04-13T03:09:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.507+0000 7f099e7fc640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.508+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09a008b490 con 0x7f09a815ebf0 2024-04-13T03:09:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.508+0000 7f09adefa640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.508+0000 7f09adefa640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 0x7f098c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09a4036000 tx=0x7f09a403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.512+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09a806f780 con 0x7f09a815ebf0 2024-04-13T03:09:52.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.685+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09a8068a10 con 0x7f098c06c8f0 2024-04-13T03:09:52.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.693+0000 7f099e7fc640 1 -- 172.21.15.106:0/4171389451 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f09a8068a10 con 0x7f098c06c8f0 2024-04-13T03:09:52.697 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:52.699 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 msgr2=0x7f098c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 0x7f098c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09a4036000 tx=0x7f09a403d040 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 msgr2=0x7f09a80adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09a000c730 tx=0x7f09a000cbf0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 shutdown_connections 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a815ebf0 0x7f09a80adcc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09a80ce1f0 0x7f09a814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 --2- 172.21.15.106:0/4171389451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09a80b5bf0 0x7f09a806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 >> 172.21.15.106:0/4171389451 conn(0x7f09a80cd6d0 msgr2=0x7f09a80b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 shutdown_connections 2024-04-13T03:09:52.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.696+0000 7f09b015e640 1 -- 172.21.15.106:0/4171389451 wait complete. 2024-04-13T03:09:52.708 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-13T03:09:52.368600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002885"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004868+0000","last_change":"2024-04-13T03:09:23.977064+0000","last_active":"2024-04-13T03:09:31.004868+0000","last_peered":"2024-04-13T03:09:31.004868+0000","last_clean":"2024-04-13T03:09:31.004868+0000","last_became_active":"2024-04-13T03:09:23.976781+0000","last_became_peered":"2024-04-13T03:09:23.976781+0000","last_unstale":"2024-04-13T03:09:31.004868+0000","last_undegraded":"2024-04-13T03:09:31.004868+0000","last_fullsized":"2024-04-13T03:09:31.004868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:14:27.702105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017061+0000","last_change":"2024-04-13T03:09:24.983007+0000","last_active":"2024-04-13T03:09:31.017061+0000","last_peered":"2024-04-13T03:09:31.017061+0000","last_clean":"2024-04-13T03:09:31.017061+0000","last_became_active":"2024-04-13T03:09:24.982432+0000","last_became_peered":"2024-04-13T03:09:24.982432+0000","last_unstale":"2024-04-13T03:09:31.017061+0000","last_undegraded":"2024-04-13T03:09:31.017061+0000","last_fullsized":"2024-04-13T03:09:31.017061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016588+0000","last_change":"2024-04-13T03:09:29.567663+0000","last_active":"2024-04-13T03:09:31.016588+0000","last_peered":"2024-04-13T03:09:31.016588+0000","last_clean":"2024-04-13T03:09:31.016588+0000","last_became_active":"2024-04-13T03:09:29.567314+0000","last_became_peered":"2024-04-13T03:09:29.567314+0000","last_unstale":"2024-04-13T03:09:31.016588+0000","last_undegraded":"2024-04-13T03:09:31.016588+0000","last_fullsized":"2024-04-13T03:09:31.016588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:55:26.111999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017190+0000","last_change":"2024-04-13T03:09:23.977858+0000","last_active":"2024-04-13T03:09:31.017190+0000","last_peered":"2024-04-13T03:09:31.017190+0000","last_clean":"2024-04-13T03:09:31.017190+0000","last_became_active":"2024-04-13T03:09:23.977595+0000","last_became_peered":"2024-04-13T03:09:23.977595+0000","last_unstale":"2024-04-13T03:09:31.017190+0000","last_undegraded":"2024-04-13T03:09:31.017190+0000","last_fullsized":"2024-04-13T03:09:31.017190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004745+0000","last_change":"2024-04-13T03:09:24.981595+0000","last_active":"2024-04-13T03:09:31.004745+0000","last_peered":"2024-04-13T03:09:31.004745+0000","last_clean":"2024-04-13T03:09:31.004745+0000","last_became_active":"2024-04-13T03:09:24.981249+0000","last_became_peered":"2024-04-13T03:09:24.981249+0000","last_unstale":"2024-04-13T03:09:31.004745+0000","last_undegraded":"2024-04-13T03:09:31.004745+0000","last_fullsized":"2024-04-13T03:09:31.004745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004491+0000","last_change":"2024-04-13T03:09:29.008503+0000","last_active":"2024-04-13T03:09:31.004491+0000","last_peered":"2024-04-13T03:09:31.004491+0000","last_clean":"2024-04-13T03:09:31.004491+0000","last_became_active":"2024-04-13T03:09:29.007985+0000","last_became_peered":"2024-04-13T03:09:29.007985+0000","last_unstale":"2024-04-13T03:09:31.004491+0000","last_undegraded":"2024-04-13T03:09:31.004491+0000","last_fullsized":"2024-04-13T03:09:31.004491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074031+0000","last_change":"2024-04-13T03:09:23.982379+0000","last_active":"2024-04-13T03:09:32.074031+0000","last_peered":"2024-04-13T03:09:32.074031+0000","last_clean":"2024-04-13T03:09:32.074031+0000","last_became_active":"2024-04-13T03:09:23.982090+0000","last_became_peered":"2024-04-13T03:09:23.982090+0000","last_unstale":"2024-04-13T03:09:32.074031+0000","last_undegraded":"2024-04-13T03:09:32.074031+0000","last_fullsized":"2024-04-13T03:09:32.074031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006164+0000","last_change":"2024-04-13T03:09:24.983652+0000","last_active":"2024-04-13T03:09:31.006164+0000","last_peered":"2024-04-13T03:09:31.006164+0000","last_clean":"2024-04-13T03:09:31.006164+0000","last_became_active":"2024-04-13T03:09:24.981620+0000","last_became_peered":"2024-04-13T03:09:24.981620+0000","last_unstale":"2024-04-13T03:09:31.006164+0000","last_undegraded":"2024-04-13T03:09:31.006164+0000","last_fullsized":"2024-04-13T03:09:31.006164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004691+0000","last_change":"2024-04-13T03:09:29.007274+0000","last_active":"2024-04-13T03:09:31.004691+0000","last_peered":"2024-04-13T03:09:31.004691+0000","last_clean":"2024-04-13T03:09:31.004691+0000","last_became_active":"2024-04-13T03:09:29.006867+0000","last_became_peered":"2024-04-13T03:09:29.006867+0000","last_unstale":"2024-04-13T03:09:31.004691+0000","last_undegraded":"2024-04-13T03:09:31.004691+0000","last_fullsized":"2024-04-13T03:09:31.004691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:32.259081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073387+0000","last_change":"2024-04-13T03:09:23.979669+0000","last_active":"2024-04-13T03:09:32.073387+0000","last_peered":"2024-04-13T03:09:32.073387+0000","last_clean":"2024-04-13T03:09:32.073387+0000","last_became_active":"2024-04-13T03:09:23.979303+0000","last_became_peered":"2024-04-13T03:09:23.979303+0000","last_unstale":"2024-04-13T03:09:32.073387+0000","last_undegraded":"2024-04-13T03:09:32.073387+0000","last_fullsized":"2024-04-13T03:09:32.073387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005531+0000","last_change":"2024-04-13T03:09:24.976796+0000","last_active":"2024-04-13T03:09:31.005531+0000","last_peered":"2024-04-13T03:09:31.005531+0000","last_clean":"2024-04-13T03:09:31.005531+0000","last_became_active":"2024-04-13T03:09:24.975367+0000","last_became_peered":"2024-04-13T03:09:24.975367+0000","last_unstale":"2024-04-13T03:09:31.005531+0000","last_undegraded":"2024-04-13T03:09:31.005531+0000","last_fullsized":"2024-04-13T03:09:31.005531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004563+0000","last_change":"2024-04-13T03:09:29.007374+0000","last_active":"2024-04-13T03:09:31.004563+0000","last_peered":"2024-04-13T03:09:31.004563+0000","last_clean":"2024-04-13T03:09:31.004563+0000","last_became_active":"2024-04-13T03:09:29.006740+0000","last_became_peered":"2024-04-13T03:09:29.006740+0000","last_unstale":"2024-04-13T03:09:31.004563+0000","last_undegraded":"2024-04-13T03:09:31.004563+0000","last_fullsized":"2024-04-13T03:09:31.004563+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073554+0000","last_change":"2024-04-13T03:09:24.988155+0000","last_active":"2024-04-13T03:09:32.073554+0000","last_peered":"2024-04-13T03:09:32.073554+0000","last_clean":"2024-04-13T03:09:32.073554+0000","last_became_active":"2024-04-13T03:09:24.987879+0000","last_became_peered":"2024-04-13T03:09:24.987879+0000","last_unstale":"2024-04-13T03:09:32.073554+0000","last_undegraded":"2024-04-13T03:09:32.073554+0000","last_fullsized":"2024-04-13T03:09:32.073554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072988+0000","last_change":"2024-04-13T03:09:23.977048+0000","last_active":"2024-04-13T03:09:32.072988+0000","last_peered":"2024-04-13T03:09:32.072988+0000","last_clean":"2024-04-13T03:09:32.072988+0000","last_became_active":"2024-04-13T03:09:23.976770+0000","last_became_peered":"2024-04-13T03:09:23.976770+0000","last_unstale":"2024-04-13T03:09:32.072988+0000","last_undegraded":"2024-04-13T03:09:32.072988+0000","last_fullsized":"2024-04-13T03:09:32.072988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005678+0000","last_change":"2024-04-13T03:09:29.009555+0000","last_active":"2024-04-13T03:09:31.005678+0000","last_peered":"2024-04-13T03:09:31.005678+0000","last_clean":"2024-04-13T03:09:31.005678+0000","last_became_active":"2024-04-13T03:09:29.009108+0000","last_became_peered":"2024-04-13T03:09:29.009108+0000","last_unstale":"2024-04-13T03:09:31.005678+0000","last_undegraded":"2024-04-13T03:09:31.005678+0000","last_fullsized":"2024-04-13T03:09:31.005678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016758+0000","last_change":"2024-04-13T03:09:24.973588+0000","last_active":"2024-04-13T03:09:31.016758+0000","last_peered":"2024-04-13T03:09:31.016758+0000","last_clean":"2024-04-13T03:09:31.016758+0000","last_became_active":"2024-04-13T03:09:24.973322+0000","last_became_peered":"2024-04-13T03:09:24.973322+0000","last_unstale":"2024-04-13T03:09:31.016758+0000","last_undegraded":"2024-04-13T03:09:31.016758+0000","last_fullsized":"2024-04-13T03:09:31.016758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552368+0000","last_change":"2024-04-13T03:09:23.971765+0000","last_active":"2024-04-13T03:09:31.552368+0000","last_peered":"2024-04-13T03:09:31.552368+0000","last_clean":"2024-04-13T03:09:31.552368+0000","last_became_active":"2024-04-13T03:09:23.971543+0000","last_became_peered":"2024-04-13T03:09:23.971543+0000","last_unstale":"2024-04-13T03:09:31.552368+0000","last_undegraded":"2024-04-13T03:09:31.552368+0000","last_fullsized":"2024-04-13T03:09:31.552368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005586+0000","last_change":"2024-04-13T03:09:29.008916+0000","last_active":"2024-04-13T03:09:31.005586+0000","last_peered":"2024-04-13T03:09:31.005586+0000","last_clean":"2024-04-13T03:09:31.005586+0000","last_became_active":"2024-04-13T03:09:29.008366+0000","last_became_peered":"2024-04-13T03:09:29.008366+0000","last_unstale":"2024-04-13T03:09:31.005586+0000","last_undegraded":"2024-04-13T03:09:31.005586+0000","last_fullsized":"2024-04-13T03:09:31.005586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016669+0000","last_change":"2024-04-13T03:09:24.975847+0000","last_active":"2024-04-13T03:09:31.016669+0000","last_peered":"2024-04-13T03:09:31.016669+0000","last_clean":"2024-04-13T03:09:31.016669+0000","last_became_active":"2024-04-13T03:09:24.975258+0000","last_became_peered":"2024-04-13T03:09:24.975258+0000","last_unstale":"2024-04-13T03:09:31.016669+0000","last_undegraded":"2024-04-13T03:09:31.016669+0000","last_fullsized":"2024-04-13T03:09:31.016669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552132+0000","last_change":"2024-04-13T03:09:23.977100+0000","last_active":"2024-04-13T03:09:31.552132+0000","last_peered":"2024-04-13T03:09:31.552132+0000","last_clean":"2024-04-13T03:09:31.552132+0000","last_became_active":"2024-04-13T03:09:23.976787+0000","last_became_peered":"2024-04-13T03:09:23.976787+0000","last_unstale":"2024-04-13T03:09:31.552132+0000","last_undegraded":"2024-04-13T03:09:31.552132+0000","last_fullsized":"2024-04-13T03:09:31.552132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552936+0000","last_change":"2024-04-13T03:09:29.567374+0000","last_active":"2024-04-13T03:09:31.552936+0000","last_peered":"2024-04-13T03:09:31.552936+0000","last_clean":"2024-04-13T03:09:31.552936+0000","last_became_active":"2024-04-13T03:09:29.566995+0000","last_became_peered":"2024-04-13T03:09:29.566995+0000","last_unstale":"2024-04-13T03:09:31.552936+0000","last_undegraded":"2024-04-13T03:09:31.552936+0000","last_fullsized":"2024-04-13T03:09:31.552936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004813+0000","last_change":"2024-04-13T03:09:24.972822+0000","last_active":"2024-04-13T03:09:31.004813+0000","last_peered":"2024-04-13T03:09:31.004813+0000","last_clean":"2024-04-13T03:09:31.004813+0000","last_became_active":"2024-04-13T03:09:24.972521+0000","last_became_peered":"2024-04-13T03:09:24.972521+0000","last_unstale":"2024-04-13T03:09:31.004813+0000","last_undegraded":"2024-04-13T03:09:31.004813+0000","last_fullsized":"2024-04-13T03:09:31.004813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073287+0000","last_change":"2024-04-13T03:09:23.979583+0000","last_active":"2024-04-13T03:09:32.073287+0000","last_peered":"2024-04-13T03:09:32.073287+0000","last_clean":"2024-04-13T03:09:32.073287+0000","last_became_active":"2024-04-13T03:09:23.979321+0000","last_became_peered":"2024-04-13T03:09:23.979321+0000","last_unstale":"2024-04-13T03:09:32.073287+0000","last_undegraded":"2024-04-13T03:09:32.073287+0000","last_fullsized":"2024-04-13T03:09:32.073287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:05.954316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006004+0000","last_change":"2024-04-13T03:09:29.009924+0000","last_active":"2024-04-13T03:09:31.006004+0000","last_peered":"2024-04-13T03:09:31.006004+0000","last_clean":"2024-04-13T03:09:31.006004+0000","last_became_active":"2024-04-13T03:09:29.009528+0000","last_became_peered":"2024-04-13T03:09:29.009528+0000","last_unstale":"2024-04-13T03:09:31.006004+0000","last_undegraded":"2024-04-13T03:09:31.006004+0000","last_fullsized":"2024-04-13T03:09:31.006004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004799+0000","last_change":"2024-04-13T03:09:24.973677+0000","last_active":"2024-04-13T03:09:31.004799+0000","last_peered":"2024-04-13T03:09:31.004799+0000","last_clean":"2024-04-13T03:09:31.004799+0000","last_became_active":"2024-04-13T03:09:24.973419+0000","last_became_peered":"2024-04-13T03:09:24.973419+0000","last_unstale":"2024-04-13T03:09:31.004799+0000","last_undegraded":"2024-04-13T03:09:31.004799+0000","last_fullsized":"2024-04-13T03:09:31.004799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074813+0000","last_change":"2024-04-13T03:09:23.977870+0000","last_active":"2024-04-13T03:09:32.074813+0000","last_peered":"2024-04-13T03:09:32.074813+0000","last_clean":"2024-04-13T03:09:32.074813+0000","last_became_active":"2024-04-13T03:09:23.977529+0000","last_became_peered":"2024-04-13T03:09:23.977529+0000","last_unstale":"2024-04-13T03:09:32.074813+0000","last_undegraded":"2024-04-13T03:09:32.074813+0000","last_fullsized":"2024-04-13T03:09:32.074813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004609+0000","last_change":"2024-04-13T03:09:29.566777+0000","last_active":"2024-04-13T03:09:31.004609+0000","last_peered":"2024-04-13T03:09:31.004609+0000","last_clean":"2024-04-13T03:09:31.004609+0000","last_became_active":"2024-04-13T03:09:29.566173+0000","last_became_peered":"2024-04-13T03:09:29.566173+0000","last_unstale":"2024-04-13T03:09:31.004609+0000","last_undegraded":"2024-04-13T03:09:31.004609+0000","last_fullsized":"2024-04-13T03:09:31.004609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017303+0000","last_change":"2024-04-13T03:09:24.981346+0000","last_active":"2024-04-13T03:09:31.017303+0000","last_peered":"2024-04-13T03:09:31.017303+0000","last_clean":"2024-04-13T03:09:31.017303+0000","last_became_active":"2024-04-13T03:09:24.981003+0000","last_became_peered":"2024-04-13T03:09:24.981003+0000","last_unstale":"2024-04-13T03:09:31.017303+0000","last_undegraded":"2024-04-13T03:09:31.017303+0000","last_fullsized":"2024-04-13T03:09:31.017303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552769+0000","last_change":"2024-04-13T03:09:23.973982+0000","last_active":"2024-04-13T03:09:31.552769+0000","last_peered":"2024-04-13T03:09:31.552769+0000","last_clean":"2024-04-13T03:09:31.552769+0000","last_became_active":"2024-04-13T03:09:23.973759+0000","last_became_peered":"2024-04-13T03:09:23.973759+0000","last_unstale":"2024-04-13T03:09:31.552769+0000","last_undegraded":"2024-04-13T03:09:31.552769+0000","last_fullsized":"2024-04-13T03:09:31.552769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:31.768176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016499+0000","last_change":"2024-04-13T03:09:29.567476+0000","last_active":"2024-04-13T03:09:31.016499+0000","last_peered":"2024-04-13T03:09:31.016499+0000","last_clean":"2024-04-13T03:09:31.016499+0000","last_became_active":"2024-04-13T03:09:29.567116+0000","last_became_peered":"2024-04-13T03:09:29.567116+0000","last_unstale":"2024-04-13T03:09:31.016499+0000","last_undegraded":"2024-04-13T03:09:31.016499+0000","last_fullsized":"2024-04-13T03:09:31.016499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:07.048787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552621+0000","last_change":"2024-04-13T03:09:24.974586+0000","last_active":"2024-04-13T03:09:31.552621+0000","last_peered":"2024-04-13T03:09:31.552621+0000","last_clean":"2024-04-13T03:09:31.552621+0000","last_became_active":"2024-04-13T03:09:24.974348+0000","last_became_peered":"2024-04-13T03:09:24.974348+0000","last_unstale":"2024-04-13T03:09:31.552621+0000","last_undegraded":"2024-04-13T03:09:31.552621+0000","last_fullsized":"2024-04-13T03:09:31.552621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073219+0000","last_change":"2024-04-13T03:09:23.979817+0000","last_active":"2024-04-13T03:09:32.073219+0000","last_peered":"2024-04-13T03:09:32.073219+0000","last_clean":"2024-04-13T03:09:32.073219+0000","last_became_active":"2024-04-13T03:09:23.979575+0000","last_became_peered":"2024-04-13T03:09:23.979575+0000","last_unstale":"2024-04-13T03:09:32.073219+0000","last_undegraded":"2024-04-13T03:09:32.073219+0000","last_fullsized":"2024-04-13T03:09:32.073219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:42.984781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004425+0000","last_change":"2024-04-13T03:09:29.567336+0000","last_active":"2024-04-13T03:09:31.004425+0000","last_peered":"2024-04-13T03:09:31.004425+0000","last_clean":"2024-04-13T03:09:31.004425+0000","last_became_active":"2024-04-13T03:09:29.566964+0000","last_became_peered":"2024-04-13T03:09:29.566964+0000","last_unstale":"2024-04-13T03:09:31.004425+0000","last_undegraded":"2024-04-13T03:09:31.004425+0000","last_fullsized":"2024-04-13T03:09:31.004425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005618+0000","last_change":"2024-04-13T03:09:24.976546+0000","last_active":"2024-04-13T03:09:31.005618+0000","last_peered":"2024-04-13T03:09:31.005618+0000","last_clean":"2024-04-13T03:09:31.005618+0000","last_became_active":"2024-04-13T03:09:24.974772+0000","last_became_peered":"2024-04-13T03:09:24.974772+0000","last_unstale":"2024-04-13T03:09:31.005618+0000","last_undegraded":"2024-04-13T03:09:31.005618+0000","last_fullsized":"2024-04-13T03:09:31.005618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552722+0000","last_change":"2024-04-13T03:09:23.966690+0000","last_active":"2024-04-13T03:09:31.552722+0000","last_peered":"2024-04-13T03:09:31.552722+0000","last_clean":"2024-04-13T03:09:31.552722+0000","last_became_active":"2024-04-13T03:09:23.966440+0000","last_became_peered":"2024-04-13T03:09:23.966440+0000","last_unstale":"2024-04-13T03:09:31.552722+0000","last_undegraded":"2024-04-13T03:09:31.552722+0000","last_fullsized":"2024-04-13T03:09:31.552722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552227+0000","last_change":"2024-04-13T03:09:29.013113+0000","last_active":"2024-04-13T03:09:31.552227+0000","last_peered":"2024-04-13T03:09:31.552227+0000","last_clean":"2024-04-13T03:09:31.552227+0000","last_became_active":"2024-04-13T03:09:29.012470+0000","last_became_peered":"2024-04-13T03:09:29.012470+0000","last_unstale":"2024-04-13T03:09:31.552227+0000","last_undegraded":"2024-04-13T03:09:31.552227+0000","last_fullsized":"2024-04-13T03:09:31.552227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:25.229254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005144+0000","last_change":"2024-04-13T03:09:24.973635+0000","last_active":"2024-04-13T03:09:31.005144+0000","last_peered":"2024-04-13T03:09:31.005144+0000","last_clean":"2024-04-13T03:09:31.005144+0000","last_became_active":"2024-04-13T03:09:24.973362+0000","last_became_peered":"2024-04-13T03:09:24.973362+0000","last_unstale":"2024-04-13T03:09:31.005144+0000","last_undegraded":"2024-04-13T03:09:31.005144+0000","last_fullsized":"2024-04-13T03:09:31.005144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:24.434188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006449+0000","last_change":"2024-04-13T03:09:23.968272+0000","last_active":"2024-04-13T03:09:31.006449+0000","last_peered":"2024-04-13T03:09:31.006449+0000","last_clean":"2024-04-13T03:09:31.006449+0000","last_became_active":"2024-04-13T03:09:23.967784+0000","last_became_peered":"2024-04-13T03:09:23.967784+0000","last_unstale":"2024-04-13T03:09:31.006449+0000","last_undegraded":"2024-04-13T03:09:31.006449+0000","last_fullsized":"2024-04-13T03:09:31.006449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552810+0000","last_change":"2024-04-13T03:09:29.008833+0000","last_active":"2024-04-13T03:09:31.552810+0000","last_peered":"2024-04-13T03:09:31.552810+0000","last_clean":"2024-04-13T03:09:31.552810+0000","last_became_active":"2024-04-13T03:09:29.008342+0000","last_became_peered":"2024-04-13T03:09:29.008342+0000","last_unstale":"2024-04-13T03:09:31.552810+0000","last_undegraded":"2024-04-13T03:09:31.552810+0000","last_fullsized":"2024-04-13T03:09:31.552810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552174+0000","last_change":"2024-04-13T03:09:24.975965+0000","last_active":"2024-04-13T03:09:31.552174+0000","last_peered":"2024-04-13T03:09:31.552174+0000","last_clean":"2024-04-13T03:09:31.552174+0000","last_became_active":"2024-04-13T03:09:24.975143+0000","last_became_peered":"2024-04-13T03:09:24.975143+0000","last_unstale":"2024-04-13T03:09:31.552174+0000","last_undegraded":"2024-04-13T03:09:31.552174+0000","last_fullsized":"2024-04-13T03:09:31.552174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.978132+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.976718+0000","last_became_peered":"2024-04-13T03:09:23.976718+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552953+0000","last_change":"2024-04-13T03:09:29.012295+0000","last_active":"2024-04-13T03:09:31.552953+0000","last_peered":"2024-04-13T03:09:31.552953+0000","last_clean":"2024-04-13T03:09:31.552953+0000","last_became_active":"2024-04-13T03:09:29.011908+0000","last_became_peered":"2024-04-13T03:09:29.011908+0000","last_unstale":"2024-04-13T03:09:31.552953+0000","last_undegraded":"2024-04-13T03:09:31.552953+0000","last_fullsized":"2024-04-13T03:09:31.552953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004144+0000","last_change":"2024-04-13T03:09:24.974975+0000","last_active":"2024-04-13T03:09:31.004144+0000","last_peered":"2024-04-13T03:09:31.004144+0000","last_clean":"2024-04-13T03:09:31.004144+0000","last_became_active":"2024-04-13T03:09:24.974656+0000","last_became_peered":"2024-04-13T03:09:24.974656+0000","last_unstale":"2024-04-13T03:09:31.004144+0000","last_undegraded":"2024-04-13T03:09:31.004144+0000","last_fullsized":"2024-04-13T03:09:31.004144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072979+0000","last_change":"2024-04-13T03:09:23.978671+0000","last_active":"2024-04-13T03:09:32.072979+0000","last_peered":"2024-04-13T03:09:32.072979+0000","last_clean":"2024-04-13T03:09:32.072979+0000","last_became_active":"2024-04-13T03:09:23.978358+0000","last_became_peered":"2024-04-13T03:09:23.978358+0000","last_unstale":"2024-04-13T03:09:32.072979+0000","last_undegraded":"2024-04-13T03:09:32.072979+0000","last_fullsized":"2024-04-13T03:09:32.072979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004701+0000","last_change":"2024-04-13T03:09:29.566364+0000","last_active":"2024-04-13T03:09:31.004701+0000","last_peered":"2024-04-13T03:09:31.004701+0000","last_clean":"2024-04-13T03:09:31.004701+0000","last_became_active":"2024-04-13T03:09:29.565976+0000","last_became_peered":"2024-04-13T03:09:29.565976+0000","last_unstale":"2024-04-13T03:09:31.004701+0000","last_undegraded":"2024-04-13T03:09:31.004701+0000","last_fullsized":"2024-04-13T03:09:31.004701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016580+0000","last_change":"2024-04-13T03:09:24.988431+0000","last_active":"2024-04-13T03:09:31.016580+0000","last_peered":"2024-04-13T03:09:31.016580+0000","last_clean":"2024-04-13T03:09:31.016580+0000","last_became_active":"2024-04-13T03:09:24.988179+0000","last_became_peered":"2024-04-13T03:09:24.988179+0000","last_unstale":"2024-04-13T03:09:31.016580+0000","last_undegraded":"2024-04-13T03:09:31.016580+0000","last_fullsized":"2024-04-13T03:09:31.016580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005560+0000","last_change":"2024-04-13T03:09:23.970960+0000","last_active":"2024-04-13T03:09:31.005560+0000","last_peered":"2024-04-13T03:09:31.005560+0000","last_clean":"2024-04-13T03:09:31.005560+0000","last_became_active":"2024-04-13T03:09:23.970615+0000","last_became_peered":"2024-04-13T03:09:23.970615+0000","last_unstale":"2024-04-13T03:09:31.005560+0000","last_undegraded":"2024-04-13T03:09:31.005560+0000","last_fullsized":"2024-04-13T03:09:31.005560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073625+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:09:32.073625+0000","last_peered":"2024-04-13T03:09:32.073625+0000","last_clean":"2024-04-13T03:09:32.073625+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:09:32.073625+0000","last_undegraded":"2024-04-13T03:09:32.073625+0000","last_fullsized":"2024-04-13T03:09:32.073625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074664+0000","last_change":"2024-04-13T03:09:29.570505+0000","last_active":"2024-04-13T03:09:32.074664+0000","last_peered":"2024-04-13T03:09:32.074664+0000","last_clean":"2024-04-13T03:09:32.074664+0000","last_became_active":"2024-04-13T03:09:29.569637+0000","last_became_peered":"2024-04-13T03:09:29.569637+0000","last_unstale":"2024-04-13T03:09:32.074664+0000","last_undegraded":"2024-04-13T03:09:32.074664+0000","last_fullsized":"2024-04-13T03:09:32.074664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:17.308576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073883+0000","last_change":"2024-04-13T03:09:24.990807+0000","last_active":"2024-04-13T03:09:32.073883+0000","last_peered":"2024-04-13T03:09:32.073883+0000","last_clean":"2024-04-13T03:09:32.073883+0000","last_became_active":"2024-04-13T03:09:24.990454+0000","last_became_peered":"2024-04-13T03:09:24.990454+0000","last_unstale":"2024-04-13T03:09:32.073883+0000","last_undegraded":"2024-04-13T03:09:32.073883+0000","last_fullsized":"2024-04-13T03:09:32.073883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:23.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072543+0000","last_change":"2024-04-13T03:09:23.974849+0000","last_active":"2024-04-13T03:09:32.072543+0000","last_peered":"2024-04-13T03:09:32.072543+0000","last_clean":"2024-04-13T03:09:32.072543+0000","last_became_active":"2024-04-13T03:09:23.974564+0000","last_became_peered":"2024-04-13T03:09:23.974564+0000","last_unstale":"2024-04-13T03:09:32.072543+0000","last_undegraded":"2024-04-13T03:09:32.072543+0000","last_fullsized":"2024-04-13T03:09:32.072543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073496+0000","last_change":"2024-04-13T03:09:29.567076+0000","last_active":"2024-04-13T03:09:32.073496+0000","last_peered":"2024-04-13T03:09:32.073496+0000","last_clean":"2024-04-13T03:09:32.073496+0000","last_became_active":"2024-04-13T03:09:29.566162+0000","last_became_peered":"2024-04-13T03:09:29.566162+0000","last_unstale":"2024-04-13T03:09:32.073496+0000","last_undegraded":"2024-04-13T03:09:32.073496+0000","last_fullsized":"2024-04-13T03:09:32.073496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006229+0000","last_change":"2024-04-13T03:09:24.986470+0000","last_active":"2024-04-13T03:09:31.006229+0000","last_peered":"2024-04-13T03:09:31.006229+0000","last_clean":"2024-04-13T03:09:31.006229+0000","last_became_active":"2024-04-13T03:09:24.986035+0000","last_became_peered":"2024-04-13T03:09:24.986035+0000","last_unstale":"2024-04-13T03:09:31.006229+0000","last_undegraded":"2024-04-13T03:09:31.006229+0000","last_fullsized":"2024-04-13T03:09:31.006229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552711+0000","last_change":"2024-04-13T03:09:23.966911+0000","last_active":"2024-04-13T03:09:31.552711+0000","last_peered":"2024-04-13T03:09:31.552711+0000","last_clean":"2024-04-13T03:09:31.552711+0000","last_became_active":"2024-04-13T03:09:23.966494+0000","last_became_peered":"2024-04-13T03:09:23.966494+0000","last_unstale":"2024-04-13T03:09:31.552711+0000","last_undegraded":"2024-04-13T03:09:31.552711+0000","last_fullsized":"2024-04-13T03:09:31.552711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:52.132186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004556+0000","last_change":"2024-04-13T03:09:29.008472+0000","last_active":"2024-04-13T03:09:31.004556+0000","last_peered":"2024-04-13T03:09:31.004556+0000","last_clean":"2024-04-13T03:09:31.004556+0000","last_became_active":"2024-04-13T03:09:29.008101+0000","last_became_peered":"2024-04-13T03:09:29.008101+0000","last_unstale":"2024-04-13T03:09:31.004556+0000","last_undegraded":"2024-04-13T03:09:31.004556+0000","last_fullsized":"2024-04-13T03:09:31.004556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:48.677621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074447+0000","last_change":"2024-04-13T03:09:24.978817+0000","last_active":"2024-04-13T03:09:32.074447+0000","last_peered":"2024-04-13T03:09:32.074447+0000","last_clean":"2024-04-13T03:09:32.074447+0000","last_became_active":"2024-04-13T03:09:24.978091+0000","last_became_peered":"2024-04-13T03:09:24.978091+0000","last_unstale":"2024-04-13T03:09:32.074447+0000","last_undegraded":"2024-04-13T03:09:32.074447+0000","last_fullsized":"2024-04-13T03:09:32.074447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:43:28.224846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072548+0000","last_change":"2024-04-13T03:09:23.978022+0000","last_active":"2024-04-13T03:09:32.072548+0000","last_peered":"2024-04-13T03:09:32.072548+0000","last_clean":"2024-04-13T03:09:32.072548+0000","last_became_active":"2024-04-13T03:09:23.976602+0000","last_became_peered":"2024-04-13T03:09:23.976602+0000","last_unstale":"2024-04-13T03:09:32.072548+0000","last_undegraded":"2024-04-13T03:09:32.072548+0000","last_fullsized":"2024-04-13T03:09:32.072548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005100+0000","last_change":"2024-04-13T03:09:29.567904+0000","last_active":"2024-04-13T03:09:31.005100+0000","last_peered":"2024-04-13T03:09:31.005100+0000","last_clean":"2024-04-13T03:09:31.005100+0000","last_became_active":"2024-04-13T03:09:29.567497+0000","last_became_peered":"2024-04-13T03:09:29.567497+0000","last_unstale":"2024-04-13T03:09:31.005100+0000","last_undegraded":"2024-04-13T03:09:31.005100+0000","last_fullsized":"2024-04-13T03:09:31.005100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074412+0000","last_change":"2024-04-13T03:09:24.978521+0000","last_active":"2024-04-13T03:09:32.074412+0000","last_peered":"2024-04-13T03:09:32.074412+0000","last_clean":"2024-04-13T03:09:32.074412+0000","last_became_active":"2024-04-13T03:09:24.978163+0000","last_became_peered":"2024-04-13T03:09:24.978163+0000","last_unstale":"2024-04-13T03:09:32.074412+0000","last_undegraded":"2024-04-13T03:09:32.074412+0000","last_fullsized":"2024-04-13T03:09:32.074412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006024+0000","last_change":"2024-04-13T03:09:23.966891+0000","last_active":"2024-04-13T03:09:31.006024+0000","last_peered":"2024-04-13T03:09:31.006024+0000","last_clean":"2024-04-13T03:09:31.006024+0000","last_became_active":"2024-04-13T03:09:23.966596+0000","last_became_peered":"2024-04-13T03:09:23.966596+0000","last_unstale":"2024-04-13T03:09:31.006024+0000","last_undegraded":"2024-04-13T03:09:31.006024+0000","last_fullsized":"2024-04-13T03:09:31.006024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:29.744523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004384+0000","last_change":"2024-04-13T03:09:29.007029+0000","last_active":"2024-04-13T03:09:31.004384+0000","last_peered":"2024-04-13T03:09:31.004384+0000","last_clean":"2024-04-13T03:09:31.004384+0000","last_became_active":"2024-04-13T03:09:29.006625+0000","last_became_peered":"2024-04-13T03:09:29.006625+0000","last_unstale":"2024-04-13T03:09:31.004384+0000","last_undegraded":"2024-04-13T03:09:31.004384+0000","last_fullsized":"2024-04-13T03:09:31.004384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:34.498611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074033+0000","last_change":"2024-04-13T03:09:24.978991+0000","last_active":"2024-04-13T03:09:32.074033+0000","last_peered":"2024-04-13T03:09:32.074033+0000","last_clean":"2024-04-13T03:09:32.074033+0000","last_became_active":"2024-04-13T03:09:24.978725+0000","last_became_peered":"2024-04-13T03:09:24.978725+0000","last_unstale":"2024-04-13T03:09:32.074033+0000","last_undegraded":"2024-04-13T03:09:32.074033+0000","last_fullsized":"2024-04-13T03:09:32.074033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072533+0000","last_change":"2024-04-13T03:09:23.969378+0000","last_active":"2024-04-13T03:09:32.072533+0000","last_peered":"2024-04-13T03:09:32.072533+0000","last_clean":"2024-04-13T03:09:32.072533+0000","last_became_active":"2024-04-13T03:09:23.969098+0000","last_became_peered":"2024-04-13T03:09:23.969098+0000","last_unstale":"2024-04-13T03:09:32.072533+0000","last_undegraded":"2024-04-13T03:09:32.072533+0000","last_fullsized":"2024-04-13T03:09:32.072533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:20.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004694+0000","last_change":"2024-04-13T03:09:29.005610+0000","last_active":"2024-04-13T03:09:31.004694+0000","last_peered":"2024-04-13T03:09:31.004694+0000","last_clean":"2024-04-13T03:09:31.004694+0000","last_became_active":"2024-04-13T03:09:29.005192+0000","last_became_peered":"2024-04-13T03:09:29.005192+0000","last_unstale":"2024-04-13T03:09:31.004694+0000","last_undegraded":"2024-04-13T03:09:31.004694+0000","last_fullsized":"2024-04-13T03:09:31.004694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:33.876868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073718+0000","last_change":"2024-04-13T03:09:24.984278+0000","last_active":"2024-04-13T03:09:32.073718+0000","last_peered":"2024-04-13T03:09:32.073718+0000","last_clean":"2024-04-13T03:09:32.073718+0000","last_became_active":"2024-04-13T03:09:24.983967+0000","last_became_peered":"2024-04-13T03:09:24.983967+0000","last_unstale":"2024-04-13T03:09:32.073718+0000","last_undegraded":"2024-04-13T03:09:32.073718+0000","last_fullsized":"2024-04-13T03:09:32.073718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072991+0000","last_change":"2024-04-13T03:09:23.975431+0000","last_active":"2024-04-13T03:09:32.072991+0000","last_peered":"2024-04-13T03:09:32.072991+0000","last_clean":"2024-04-13T03:09:32.072991+0000","last_became_active":"2024-04-13T03:09:23.974255+0000","last_became_peered":"2024-04-13T03:09:23.974255+0000","last_unstale":"2024-04-13T03:09:32.072991+0000","last_undegraded":"2024-04-13T03:09:32.072991+0000","last_fullsized":"2024-04-13T03:09:32.072991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017273+0000","last_change":"2024-04-13T03:09:29.567710+0000","last_active":"2024-04-13T03:09:31.017273+0000","last_peered":"2024-04-13T03:09:31.017273+0000","last_clean":"2024-04-13T03:09:31.017273+0000","last_became_active":"2024-04-13T03:09:29.567230+0000","last_became_peered":"2024-04-13T03:09:29.567230+0000","last_unstale":"2024-04-13T03:09:31.017273+0000","last_undegraded":"2024-04-13T03:09:31.017273+0000","last_fullsized":"2024-04-13T03:09:31.017273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:23:03.577409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073579+0000","last_change":"2024-04-13T03:09:24.978027+0000","last_active":"2024-04-13T03:09:32.073579+0000","last_peered":"2024-04-13T03:09:32.073579+0000","last_clean":"2024-04-13T03:09:32.073579+0000","last_became_active":"2024-04-13T03:09:24.977709+0000","last_became_peered":"2024-04-13T03:09:24.977709+0000","last_unstale":"2024-04-13T03:09:32.073579+0000","last_undegraded":"2024-04-13T03:09:32.073579+0000","last_fullsized":"2024-04-13T03:09:32.073579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006349+0000","last_change":"2024-04-13T03:09:23.972192+0000","last_active":"2024-04-13T03:09:31.006349+0000","last_peered":"2024-04-13T03:09:31.006349+0000","last_clean":"2024-04-13T03:09:31.006349+0000","last_became_active":"2024-04-13T03:09:23.971856+0000","last_became_peered":"2024-04-13T03:09:23.971856+0000","last_unstale":"2024-04-13T03:09:31.006349+0000","last_undegraded":"2024-04-13T03:09:31.006349+0000","last_fullsized":"2024-04-13T03:09:31.006349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017103+0000","last_change":"2024-04-13T03:09:29.565473+0000","last_active":"2024-04-13T03:09:31.017103+0000","last_peered":"2024-04-13T03:09:31.017103+0000","last_clean":"2024-04-13T03:09:31.017103+0000","last_became_active":"2024-04-13T03:09:29.565099+0000","last_became_peered":"2024-04-13T03:09:29.565099+0000","last_unstale":"2024-04-13T03:09:31.017103+0000","last_undegraded":"2024-04-13T03:09:31.017103+0000","last_fullsized":"2024-04-13T03:09:31.017103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:49:49.717261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004475+0000","last_change":"2024-04-13T03:09:24.980510+0000","last_active":"2024-04-13T03:09:31.004475+0000","last_peered":"2024-04-13T03:09:31.004475+0000","last_clean":"2024-04-13T03:09:31.004475+0000","last_became_active":"2024-04-13T03:09:24.980226+0000","last_became_peered":"2024-04-13T03:09:24.980226+0000","last_unstale":"2024-04-13T03:09:31.004475+0000","last_undegraded":"2024-04-13T03:09:31.004475+0000","last_fullsized":"2024-04-13T03:09:31.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:31.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004618+0000","last_change":"2024-04-13T03:09:23.971205+0000","last_active":"2024-04-13T03:09:31.004618+0000","last_peered":"2024-04-13T03:09:31.004618+0000","last_clean":"2024-04-13T03:09:31.004618+0000","last_became_active":"2024-04-13T03:09:23.970845+0000","last_became_peered":"2024-04-13T03:09:23.970845+0000","last_unstale":"2024-04-13T03:09:31.004618+0000","last_undegraded":"2024-04-13T03:09:31.004618+0000","last_fullsized":"2024-04-13T03:09:31.004618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005480+0000","last_change":"2024-04-13T03:09:29.568847+0000","last_active":"2024-04-13T03:09:31.005480+0000","last_peered":"2024-04-13T03:09:31.005480+0000","last_clean":"2024-04-13T03:09:31.005480+0000","last_became_active":"2024-04-13T03:09:29.568286+0000","last_became_peered":"2024-04-13T03:09:29.568286+0000","last_unstale":"2024-04-13T03:09:31.005480+0000","last_undegraded":"2024-04-13T03:09:31.005480+0000","last_fullsized":"2024-04-13T03:09:31.005480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:24.042160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073615+0000","last_change":"2024-04-13T03:09:24.980651+0000","last_active":"2024-04-13T03:09:32.073615+0000","last_peered":"2024-04-13T03:09:32.073615+0000","last_clean":"2024-04-13T03:09:32.073615+0000","last_became_active":"2024-04-13T03:09:24.980390+0000","last_became_peered":"2024-04-13T03:09:24.980390+0000","last_unstale":"2024-04-13T03:09:32.073615+0000","last_undegraded":"2024-04-13T03:09:32.073615+0000","last_fullsized":"2024-04-13T03:09:32.073615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:34.382842+0000","last_change":"2024-04-13T03:09:23.977841+0000","last_active":"2024-04-13T03:09:34.382842+0000","last_peered":"2024-04-13T03:09:34.382842+0000","last_clean":"2024-04-13T03:09:34.382842+0000","last_became_active":"2024-04-13T03:09:23.977546+0000","last_became_peered":"2024-04-13T03:09:23.977546+0000","last_unstale":"2024-04-13T03:09:34.382842+0000","last_undegraded":"2024-04-13T03:09:34.382842+0000","last_fullsized":"2024-04-13T03:09:34.382842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552322+0000","last_change":"2024-04-13T03:09:29.012775+0000","last_active":"2024-04-13T03:09:31.552322+0000","last_peered":"2024-04-13T03:09:31.552322+0000","last_clean":"2024-04-13T03:09:31.552322+0000","last_became_active":"2024-04-13T03:09:29.011952+0000","last_became_peered":"2024-04-13T03:09:29.011952+0000","last_unstale":"2024-04-13T03:09:31.552322+0000","last_undegraded":"2024-04-13T03:09:31.552322+0000","last_fullsized":"2024-04-13T03:09:31.552322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005496+0000","last_change":"2024-04-13T03:09:24.980974+0000","last_active":"2024-04-13T03:09:31.005496+0000","last_peered":"2024-04-13T03:09:31.005496+0000","last_clean":"2024-04-13T03:09:31.005496+0000","last_became_active":"2024-04-13T03:09:24.980580+0000","last_became_peered":"2024-04-13T03:09:24.980580+0000","last_unstale":"2024-04-13T03:09:31.005496+0000","last_undegraded":"2024-04-13T03:09:31.005496+0000","last_fullsized":"2024-04-13T03:09:31.005496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552665+0000","last_change":"2024-04-13T03:09:23.970652+0000","last_active":"2024-04-13T03:09:31.552665+0000","last_peered":"2024-04-13T03:09:31.552665+0000","last_clean":"2024-04-13T03:09:31.552665+0000","last_became_active":"2024-04-13T03:09:23.970405+0000","last_became_peered":"2024-04-13T03:09:23.970405+0000","last_unstale":"2024-04-13T03:09:31.552665+0000","last_undegraded":"2024-04-13T03:09:31.552665+0000","last_fullsized":"2024-04-13T03:09:31.552665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:23.430251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004925+0000","last_change":"2024-04-13T03:09:29.009723+0000","last_active":"2024-04-13T03:09:31.004925+0000","last_peered":"2024-04-13T03:09:31.004925+0000","last_clean":"2024-04-13T03:09:31.004925+0000","last_became_active":"2024-04-13T03:09:29.009177+0000","last_became_peered":"2024-04-13T03:09:29.009177+0000","last_unstale":"2024-04-13T03:09:31.004925+0000","last_undegraded":"2024-04-13T03:09:31.004925+0000","last_fullsized":"2024-04-13T03:09:31.004925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:16:33.394306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004826+0000","last_change":"2024-04-13T03:09:24.976839+0000","last_active":"2024-04-13T03:09:31.004826+0000","last_peered":"2024-04-13T03:09:31.004826+0000","last_clean":"2024-04-13T03:09:31.004826+0000","last_became_active":"2024-04-13T03:09:24.976565+0000","last_became_peered":"2024-04-13T03:09:24.976565+0000","last_unstale":"2024-04-13T03:09:31.004826+0000","last_undegraded":"2024-04-13T03:09:31.004826+0000","last_fullsized":"2024-04-13T03:09:31.004826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005802+0000","last_change":"2024-04-13T03:09:23.974057+0000","last_active":"2024-04-13T03:09:31.005802+0000","last_peered":"2024-04-13T03:09:31.005802+0000","last_clean":"2024-04-13T03:09:31.005802+0000","last_became_active":"2024-04-13T03:09:23.973783+0000","last_became_peered":"2024-04-13T03:09:23.973783+0000","last_unstale":"2024-04-13T03:09:31.005802+0000","last_undegraded":"2024-04-13T03:09:31.005802+0000","last_fullsized":"2024-04-13T03:09:31.005802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:47:45.441208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016630+0000","last_change":"2024-04-13T03:09:29.008489+0000","last_active":"2024-04-13T03:09:31.016630+0000","last_peered":"2024-04-13T03:09:31.016630+0000","last_clean":"2024-04-13T03:09:31.016630+0000","last_became_active":"2024-04-13T03:09:29.008129+0000","last_became_peered":"2024-04-13T03:09:29.008129+0000","last_unstale":"2024-04-13T03:09:31.016630+0000","last_undegraded":"2024-04-13T03:09:31.016630+0000","last_fullsized":"2024-04-13T03:09:31.016630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:03.613440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005713+0000","last_change":"2024-04-13T03:09:24.988904+0000","last_active":"2024-04-13T03:09:31.005713+0000","last_peered":"2024-04-13T03:09:31.005713+0000","last_clean":"2024-04-13T03:09:31.005713+0000","last_became_active":"2024-04-13T03:09:24.988667+0000","last_became_peered":"2024-04-13T03:09:24.988667+0000","last_unstale":"2024-04-13T03:09:31.005713+0000","last_undegraded":"2024-04-13T03:09:31.005713+0000","last_fullsized":"2024-04-13T03:09:31.005713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:08.519926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074106+0000","last_change":"2024-04-13T03:09:23.975478+0000","last_active":"2024-04-13T03:09:32.074106+0000","last_peered":"2024-04-13T03:09:32.074106+0000","last_clean":"2024-04-13T03:09:32.074106+0000","last_became_active":"2024-04-13T03:09:23.975155+0000","last_became_peered":"2024-04-13T03:09:23.975155+0000","last_unstale":"2024-04-13T03:09:32.074106+0000","last_undegraded":"2024-04-13T03:09:32.074106+0000","last_fullsized":"2024-04-13T03:09:32.074106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:46.396056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016599+0000","last_change":"2024-04-13T03:09:29.565039+0000","last_active":"2024-04-13T03:09:31.016599+0000","last_peered":"2024-04-13T03:09:31.016599+0000","last_clean":"2024-04-13T03:09:31.016599+0000","last_became_active":"2024-04-13T03:09:29.564715+0000","last_became_peered":"2024-04-13T03:09:29.564715+0000","last_unstale":"2024-04-13T03:09:31.016599+0000","last_undegraded":"2024-04-13T03:09:31.016599+0000","last_fullsized":"2024-04-13T03:09:31.016599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:38.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004534+0000","last_change":"2024-04-13T03:09:29.567360+0000","last_active":"2024-04-13T03:09:31.004534+0000","last_peered":"2024-04-13T03:09:31.004534+0000","last_clean":"2024-04-13T03:09:31.004534+0000","last_became_active":"2024-04-13T03:09:29.567005+0000","last_became_peered":"2024-04-13T03:09:29.567005+0000","last_unstale":"2024-04-13T03:09:31.004534+0000","last_undegraded":"2024-04-13T03:09:31.004534+0000","last_fullsized":"2024-04-13T03:09:31.004534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:27:54.857624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552017+0000","last_change":"2024-04-13T03:09:24.980403+0000","last_active":"2024-04-13T03:09:31.552017+0000","last_peered":"2024-04-13T03:09:31.552017+0000","last_clean":"2024-04-13T03:09:31.552017+0000","last_became_active":"2024-04-13T03:09:24.980125+0000","last_became_peered":"2024-04-13T03:09:24.980125+0000","last_unstale":"2024-04-13T03:09:31.552017+0000","last_undegraded":"2024-04-13T03:09:31.552017+0000","last_fullsized":"2024-04-13T03:09:31.552017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006298+0000","last_change":"2024-04-13T03:09:23.966400+0000","last_active":"2024-04-13T03:09:31.006298+0000","last_peered":"2024-04-13T03:09:31.006298+0000","last_clean":"2024-04-13T03:09:31.006298+0000","last_became_active":"2024-04-13T03:09:23.966136+0000","last_became_peered":"2024-04-13T03:09:23.966136+0000","last_unstale":"2024-04-13T03:09:31.006298+0000","last_undegraded":"2024-04-13T03:09:31.006298+0000","last_fullsized":"2024-04-13T03:09:31.006298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074769+0000","last_change":"2024-04-13T03:09:29.008579+0000","last_active":"2024-04-13T03:09:32.074769+0000","last_peered":"2024-04-13T03:09:32.074769+0000","last_clean":"2024-04-13T03:09:32.074769+0000","last_became_active":"2024-04-13T03:09:29.008030+0000","last_became_peered":"2024-04-13T03:09:29.008030+0000","last_unstale":"2024-04-13T03:09:32.074769+0000","last_undegraded":"2024-04-13T03:09:32.074769+0000","last_fullsized":"2024-04-13T03:09:32.074769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:10.900787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016574+0000","last_change":"2024-04-13T03:09:24.972241+0000","last_active":"2024-04-13T03:09:31.016574+0000","last_peered":"2024-04-13T03:09:31.016574+0000","last_clean":"2024-04-13T03:09:31.016574+0000","last_became_active":"2024-04-13T03:09:24.971962+0000","last_became_peered":"2024-04-13T03:09:24.971962+0000","last_unstale":"2024-04-13T03:09:31.016574+0000","last_undegraded":"2024-04-13T03:09:31.016574+0000","last_fullsized":"2024-04-13T03:09:31.016574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016806+0000","last_change":"2024-04-13T03:09:23.977688+0000","last_active":"2024-04-13T03:09:31.016806+0000","last_peered":"2024-04-13T03:09:31.016806+0000","last_clean":"2024-04-13T03:09:31.016806+0000","last_became_active":"2024-04-13T03:09:23.977412+0000","last_became_peered":"2024-04-13T03:09:23.977412+0000","last_unstale":"2024-04-13T03:09:31.016806+0000","last_undegraded":"2024-04-13T03:09:31.016806+0000","last_fullsized":"2024-04-13T03:09:31.016806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:57.708019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072995+0000","last_change":"2024-04-13T03:09:23.973771+0000","last_active":"2024-04-13T03:09:32.072995+0000","last_peered":"2024-04-13T03:09:32.072995+0000","last_clean":"2024-04-13T03:09:32.072995+0000","last_became_active":"2024-04-13T03:09:23.973271+0000","last_became_peered":"2024-04-13T03:09:23.973271+0000","last_unstale":"2024-04-13T03:09:32.072995+0000","last_undegraded":"2024-04-13T03:09:32.072995+0000","last_fullsized":"2024-04-13T03:09:32.072995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:53:45.546212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004613+0000","last_change":"2024-04-13T03:09:24.974833+0000","last_active":"2024-04-13T03:09:31.004613+0000","last_peered":"2024-04-13T03:09:31.004613+0000","last_clean":"2024-04-13T03:09:31.004613+0000","last_became_active":"2024-04-13T03:09:24.974554+0000","last_became_peered":"2024-04-13T03:09:24.974554+0000","last_unstale":"2024-04-13T03:09:31.004613+0000","last_undegraded":"2024-04-13T03:09:31.004613+0000","last_fullsized":"2024-04-13T03:09:31.004613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072905+0000","last_change":"2024-04-13T03:09:29.004327+0000","last_active":"2024-04-13T03:09:32.072905+0000","last_peered":"2024-04-13T03:09:32.072905+0000","last_clean":"2024-04-13T03:09:32.072905+0000","last_became_active":"2024-04-13T03:09:29.003998+0000","last_became_peered":"2024-04-13T03:09:29.003998+0000","last_unstale":"2024-04-13T03:09:32.072905+0000","last_undegraded":"2024-04-13T03:09:32.072905+0000","last_fullsized":"2024-04-13T03:09:32.072905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:10:12.310827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016845+0000","last_change":"2024-04-13T03:09:23.976680+0000","last_active":"2024-04-13T03:09:31.016845+0000","last_peered":"2024-04-13T03:09:31.016845+0000","last_clean":"2024-04-13T03:09:31.016845+0000","last_became_active":"2024-04-13T03:09:23.976390+0000","last_became_peered":"2024-04-13T03:09:23.976390+0000","last_unstale":"2024-04-13T03:09:31.016845+0000","last_undegraded":"2024-04-13T03:09:31.016845+0000","last_fullsized":"2024-04-13T03:09:31.016845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:22.947669+0000","last_clean_scrub_stamp":"2024-04-13T03:09:22.947669+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:30.868347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004533+0000","last_change":"2024-04-13T03:09:24.971520+0000","last_active":"2024-04-13T03:09:31.004533+0000","last_peered":"2024-04-13T03:09:31.004533+0000","last_clean":"2024-04-13T03:09:31.004533+0000","last_became_active":"2024-04-13T03:09:24.971236+0000","last_became_peered":"2024-04-13T03:09:24.971236+0000","last_unstale":"2024-04-13T03:09:31.004533+0000","last_undegraded":"2024-04-13T03:09:31.004533+0000","last_fullsized":"2024-04-13T03:09:31.004533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:46:47.399993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017010+0000","last_change":"2024-04-13T03:09:29.008243+0000","last_active":"2024-04-13T03:09:31.017010+0000","last_peered":"2024-04-13T03:09:31.017010+0000","last_clean":"2024-04-13T03:09:31.017010+0000","last_became_active":"2024-04-13T03:09:29.007854+0000","last_became_peered":"2024-04-13T03:09:29.007854+0000","last_unstale":"2024-04-13T03:09:31.017010+0000","last_undegraded":"2024-04-13T03:09:31.017010+0000","last_fullsized":"2024-04-13T03:09:31.017010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:27.985841+0000","last_clean_scrub_stamp":"2024-04-13T03:09:27.985841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:33.651724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073532+0000","last_change":"2024-04-13T03:09:24.984919+0000","last_active":"2024-04-13T03:09:32.073532+0000","last_peered":"2024-04-13T03:09:32.073532+0000","last_clean":"2024-04-13T03:09:32.073532+0000","last_became_active":"2024-04-13T03:09:24.984652+0000","last_became_peered":"2024-04-13T03:09:24.984652+0000","last_unstale":"2024-04-13T03:09:32.073532+0000","last_undegraded":"2024-04-13T03:09:32.073532+0000","last_fullsized":"2024-04-13T03:09:32.073532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.072697+0000","last_change":"2024-04-13T03:09:24.980195+0000","last_active":"2024-04-13T03:09:32.072697+0000","last_peered":"2024-04-13T03:09:32.072697+0000","last_clean":"2024-04-13T03:09:32.072697+0000","last_became_active":"2024-04-13T03:09:24.979846+0000","last_became_peered":"2024-04-13T03:09:24.979846+0000","last_unstale":"2024-04-13T03:09:32.072697+0000","last_undegraded":"2024-04-13T03:09:32.072697+0000","last_fullsized":"2024-04-13T03:09:32.072697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:32.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006342+0000","last_change":"2024-04-13T03:09:24.979746+0000","last_active":"2024-04-13T03:09:31.006342+0000","last_peered":"2024-04-13T03:09:31.006342+0000","last_clean":"2024-04-13T03:09:31.006342+0000","last_became_active":"2024-04-13T03:09:24.979481+0000","last_became_peered":"2024-04-13T03:09:24.979481+0000","last_unstale":"2024-04-13T03:09:31.006342+0000","last_undegraded":"2024-04-13T03:09:31.006342+0000","last_fullsized":"2024-04-13T03:09:31.006342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:32.106183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004510+0000","last_change":"2024-04-13T03:09:24.974603+0000","last_active":"2024-04-13T03:09:31.004510+0000","last_peered":"2024-04-13T03:09:31.004510+0000","last_clean":"2024-04-13T03:09:31.004510+0000","last_became_active":"2024-04-13T03:09:24.974099+0000","last_became_peered":"2024-04-13T03:09:24.974099+0000","last_unstale":"2024-04-13T03:09:31.004510+0000","last_undegraded":"2024-04-13T03:09:31.004510+0000","last_fullsized":"2024-04-13T03:09:31.004510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017151+0000","last_change":"2024-04-13T03:09:24.977844+0000","last_active":"2024-04-13T03:09:31.017151+0000","last_peered":"2024-04-13T03:09:31.017151+0000","last_clean":"2024-04-13T03:09:31.017151+0000","last_became_active":"2024-04-13T03:09:24.977578+0000","last_became_peered":"2024-04-13T03:09:24.977578+0000","last_unstale":"2024-04-13T03:09:31.017151+0000","last_undegraded":"2024-04-13T03:09:31.017151+0000","last_fullsized":"2024-04-13T03:09:31.017151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:21:59.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552548+0000","last_change":"2024-04-13T03:09:24.981088+0000","last_active":"2024-04-13T03:09:31.552548+0000","last_peered":"2024-04-13T03:09:31.552548+0000","last_clean":"2024-04-13T03:09:31.552548+0000","last_became_active":"2024-04-13T03:09:24.980793+0000","last_became_peered":"2024-04-13T03:09:24.980793+0000","last_unstale":"2024-04-13T03:09:31.552548+0000","last_undegraded":"2024-04-13T03:09:31.552548+0000","last_fullsized":"2024-04-13T03:09:31.552548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:25.492972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074858+0000","last_change":"2024-04-13T03:09:24.990583+0000","last_active":"2024-04-13T03:09:32.074858+0000","last_peered":"2024-04-13T03:09:32.074858+0000","last_clean":"2024-04-13T03:09:32.074858+0000","last_became_active":"2024-04-13T03:09:24.990319+0000","last_became_peered":"2024-04-13T03:09:24.990319+0000","last_unstale":"2024-04-13T03:09:32.074858+0000","last_undegraded":"2024-04-13T03:09:32.074858+0000","last_fullsized":"2024-04-13T03:09:32.074858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.017256+0000","last_change":"2024-04-13T03:09:24.975238+0000","last_active":"2024-04-13T03:09:31.017256+0000","last_peered":"2024-04-13T03:09:31.017256+0000","last_clean":"2024-04-13T03:09:31.017256+0000","last_became_active":"2024-04-13T03:09:24.974911+0000","last_became_peered":"2024-04-13T03:09:24.974911+0000","last_unstale":"2024-04-13T03:09:31.017256+0000","last_undegraded":"2024-04-13T03:09:31.017256+0000","last_fullsized":"2024-04-13T03:09:31.017256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005161+0000","last_change":"2024-04-13T03:09:24.976029+0000","last_active":"2024-04-13T03:09:31.005161+0000","last_peered":"2024-04-13T03:09:31.005161+0000","last_clean":"2024-04-13T03:09:31.005161+0000","last_became_active":"2024-04-13T03:09:24.975739+0000","last_became_peered":"2024-04-13T03:09:24.975739+0000","last_unstale":"2024-04-13T03:09:31.005161+0000","last_undegraded":"2024-04-13T03:09:31.005161+0000","last_fullsized":"2024-04-13T03:09:31.005161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:50.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016664+0000","last_change":"2024-04-13T03:09:24.975447+0000","last_active":"2024-04-13T03:09:31.016664+0000","last_peered":"2024-04-13T03:09:31.016664+0000","last_clean":"2024-04-13T03:09:31.016664+0000","last_became_active":"2024-04-13T03:09:24.975131+0000","last_became_peered":"2024-04-13T03:09:24.975131+0000","last_unstale":"2024-04-13T03:09:31.016664+0000","last_undegraded":"2024-04-13T03:09:31.016664+0000","last_fullsized":"2024-04-13T03:09:31.016664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006028+0000","last_change":"2024-04-13T03:09:24.988399+0000","last_active":"2024-04-13T03:09:31.006028+0000","last_peered":"2024-04-13T03:09:31.006028+0000","last_clean":"2024-04-13T03:09:31.006028+0000","last_became_active":"2024-04-13T03:09:24.988064+0000","last_became_peered":"2024-04-13T03:09:24.988064+0000","last_unstale":"2024-04-13T03:09:31.006028+0000","last_undegraded":"2024-04-13T03:09:31.006028+0000","last_fullsized":"2024-04-13T03:09:31.006028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004399+0000","last_change":"2024-04-13T03:09:24.980428+0000","last_active":"2024-04-13T03:09:31.004399+0000","last_peered":"2024-04-13T03:09:31.004399+0000","last_clean":"2024-04-13T03:09:31.004399+0000","last_became_active":"2024-04-13T03:09:24.980157+0000","last_became_peered":"2024-04-13T03:09:24.980157+0000","last_unstale":"2024-04-13T03:09:31.004399+0000","last_undegraded":"2024-04-13T03:09:31.004399+0000","last_fullsized":"2024-04-13T03:09:31.004399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074511+0000","last_change":"2024-04-13T03:09:24.978696+0000","last_active":"2024-04-13T03:09:32.074511+0000","last_peered":"2024-04-13T03:09:32.074511+0000","last_clean":"2024-04-13T03:09:32.074511+0000","last_became_active":"2024-04-13T03:09:24.978413+0000","last_became_peered":"2024-04-13T03:09:24.978413+0000","last_unstale":"2024-04-13T03:09:32.074511+0000","last_undegraded":"2024-04-13T03:09:32.074511+0000","last_fullsized":"2024-04-13T03:09:32.074511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074117+0000","last_change":"2024-04-13T03:09:24.979845+0000","last_active":"2024-04-13T03:09:32.074117+0000","last_peered":"2024-04-13T03:09:32.074117+0000","last_clean":"2024-04-13T03:09:32.074117+0000","last_became_active":"2024-04-13T03:09:24.979598+0000","last_became_peered":"2024-04-13T03:09:24.979598+0000","last_unstale":"2024-04-13T03:09:32.074117+0000","last_undegraded":"2024-04-13T03:09:32.074117+0000","last_fullsized":"2024-04-13T03:09:32.074117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:29:30.464376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073041+0000","last_change":"2024-04-13T03:09:24.978510+0000","last_active":"2024-04-13T03:09:32.073041+0000","last_peered":"2024-04-13T03:09:32.073041+0000","last_clean":"2024-04-13T03:09:32.073041+0000","last_became_active":"2024-04-13T03:09:24.978226+0000","last_became_peered":"2024-04-13T03:09:24.978226+0000","last_unstale":"2024-04-13T03:09:32.073041+0000","last_undegraded":"2024-04-13T03:09:32.073041+0000","last_fullsized":"2024-04-13T03:09:32.073041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:16:53.136123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006512+0000","last_change":"2024-04-13T03:09:24.986783+0000","last_active":"2024-04-13T03:09:31.006512+0000","last_peered":"2024-04-13T03:09:31.006512+0000","last_clean":"2024-04-13T03:09:31.006512+0000","last_became_active":"2024-04-13T03:09:24.986139+0000","last_became_peered":"2024-04-13T03:09:24.986139+0000","last_unstale":"2024-04-13T03:09:31.006512+0000","last_undegraded":"2024-04-13T03:09:31.006512+0000","last_fullsized":"2024-04-13T03:09:31.006512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005046+0000","last_change":"2024-04-13T03:09:24.974308+0000","last_active":"2024-04-13T03:09:31.005046+0000","last_peered":"2024-04-13T03:09:31.005046+0000","last_clean":"2024-04-13T03:09:31.005046+0000","last_became_active":"2024-04-13T03:09:24.973935+0000","last_became_peered":"2024-04-13T03:09:24.973935+0000","last_unstale":"2024-04-13T03:09:31.005046+0000","last_undegraded":"2024-04-13T03:09:31.005046+0000","last_fullsized":"2024-04-13T03:09:31.005046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552097+0000","last_change":"2024-04-13T03:09:24.971657+0000","last_active":"2024-04-13T03:09:31.552097+0000","last_peered":"2024-04-13T03:09:31.552097+0000","last_clean":"2024-04-13T03:09:31.552097+0000","last_became_active":"2024-04-13T03:09:24.971379+0000","last_became_peered":"2024-04-13T03:09:24.971379+0000","last_unstale":"2024-04-13T03:09:31.552097+0000","last_undegraded":"2024-04-13T03:09:31.552097+0000","last_fullsized":"2024-04-13T03:09:31.552097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004364+0000","last_change":"2024-04-13T03:09:24.973741+0000","last_active":"2024-04-13T03:09:31.004364+0000","last_peered":"2024-04-13T03:09:31.004364+0000","last_clean":"2024-04-13T03:09:31.004364+0000","last_became_active":"2024-04-13T03:09:24.973468+0000","last_became_peered":"2024-04-13T03:09:24.973468+0000","last_unstale":"2024-04-13T03:09:31.004364+0000","last_undegraded":"2024-04-13T03:09:31.004364+0000","last_fullsized":"2024-04-13T03:09:31.004364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:20.516278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016730+0000","last_change":"2024-04-13T03:09:24.982585+0000","last_active":"2024-04-13T03:09:31.016730+0000","last_peered":"2024-04-13T03:09:31.016730+0000","last_clean":"2024-04-13T03:09:31.016730+0000","last_became_active":"2024-04-13T03:09:24.982301+0000","last_became_peered":"2024-04-13T03:09:24.982301+0000","last_unstale":"2024-04-13T03:09:31.016730+0000","last_undegraded":"2024-04-13T03:09:31.016730+0000","last_fullsized":"2024-04-13T03:09:31.016730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:23.915576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005384+0000","last_change":"2024-04-13T03:09:24.987283+0000","last_active":"2024-04-13T03:09:31.005384+0000","last_peered":"2024-04-13T03:09:31.005384+0000","last_clean":"2024-04-13T03:09:31.005384+0000","last_became_active":"2024-04-13T03:09:24.983517+0000","last_became_peered":"2024-04-13T03:09:24.983517+0000","last_unstale":"2024-04-13T03:09:31.005384+0000","last_undegraded":"2024-04-13T03:09:31.005384+0000","last_fullsized":"2024-04-13T03:09:31.005384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:43.988718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074632+0000","last_change":"2024-04-13T03:09:24.994512+0000","last_active":"2024-04-13T03:09:32.074632+0000","last_peered":"2024-04-13T03:09:32.074632+0000","last_clean":"2024-04-13T03:09:32.074632+0000","last_became_active":"2024-04-13T03:09:24.994212+0000","last_became_peered":"2024-04-13T03:09:24.994212+0000","last_unstale":"2024-04-13T03:09:32.074632+0000","last_undegraded":"2024-04-13T03:09:32.074632+0000","last_fullsized":"2024-04-13T03:09:32.074632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:15:21.107329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074805+0000","last_change":"2024-04-13T03:09:24.990511+0000","last_active":"2024-04-13T03:09:32.074805+0000","last_peered":"2024-04-13T03:09:32.074805+0000","last_clean":"2024-04-13T03:09:32.074805+0000","last_became_active":"2024-04-13T03:09:24.990230+0000","last_became_peered":"2024-04-13T03:09:24.990230+0000","last_unstale":"2024-04-13T03:09:32.074805+0000","last_undegraded":"2024-04-13T03:09:32.074805+0000","last_fullsized":"2024-04-13T03:09:32.074805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:07.721717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074337+0000","last_change":"2024-04-13T03:09:24.991357+0000","last_active":"2024-04-13T03:09:32.074337+0000","last_peered":"2024-04-13T03:09:32.074337+0000","last_clean":"2024-04-13T03:09:32.074337+0000","last_became_active":"2024-04-13T03:09:24.990754+0000","last_became_peered":"2024-04-13T03:09:24.990754+0000","last_unstale":"2024-04-13T03:09:32.074337+0000","last_undegraded":"2024-04-13T03:09:32.074337+0000","last_fullsized":"2024-04-13T03:09:32.074337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:06.866553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.016489+0000","last_change":"2024-04-13T03:09:24.984967+0000","last_active":"2024-04-13T03:09:31.016489+0000","last_peered":"2024-04-13T03:09:31.016489+0000","last_clean":"2024-04-13T03:09:31.016489+0000","last_became_active":"2024-04-13T03:09:24.984674+0000","last_became_peered":"2024-04-13T03:09:24.984674+0000","last_unstale":"2024-04-13T03:09:31.016489+0000","last_undegraded":"2024-04-13T03:09:31.016489+0000","last_fullsized":"2024-04-13T03:09:31.016489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073906+0000","last_change":"2024-04-13T03:09:24.976323+0000","last_active":"2024-04-13T03:09:32.073906+0000","last_peered":"2024-04-13T03:09:32.073906+0000","last_clean":"2024-04-13T03:09:32.073906+0000","last_became_active":"2024-04-13T03:09:24.976014+0000","last_became_peered":"2024-04-13T03:09:24.976014+0000","last_unstale":"2024-04-13T03:09:32.073906+0000","last_undegraded":"2024-04-13T03:09:32.073906+0000","last_fullsized":"2024-04-13T03:09:32.073906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:37:12.028164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.005924+0000","last_change":"2024-04-13T03:09:24.987113+0000","last_active":"2024-04-13T03:09:31.005924+0000","last_peered":"2024-04-13T03:09:31.005924+0000","last_clean":"2024-04-13T03:09:31.005924+0000","last_became_active":"2024-04-13T03:09:24.986836+0000","last_became_peered":"2024-04-13T03:09:24.986836+0000","last_unstale":"2024-04-13T03:09:31.005924+0000","last_undegraded":"2024-04-13T03:09:31.005924+0000","last_fullsized":"2024-04-13T03:09:31.005924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.073397+0000","last_change":"2024-04-13T03:09:24.979549+0000","last_active":"2024-04-13T03:09:32.073397+0000","last_peered":"2024-04-13T03:09:32.073397+0000","last_clean":"2024-04-13T03:09:32.073397+0000","last_became_active":"2024-04-13T03:09:24.979203+0000","last_became_peered":"2024-04-13T03:09:24.979203+0000","last_unstale":"2024-04-13T03:09:32.073397+0000","last_undegraded":"2024-04-13T03:09:32.073397+0000","last_fullsized":"2024-04-13T03:09:32.073397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:54:37.345914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.006135+0000","last_change":"2024-04-13T03:09:24.986349+0000","last_active":"2024-04-13T03:09:31.006135+0000","last_peered":"2024-04-13T03:09:31.006135+0000","last_clean":"2024-04-13T03:09:31.006135+0000","last_became_active":"2024-04-13T03:09:24.985694+0000","last_became_peered":"2024-04-13T03:09:24.985694+0000","last_unstale":"2024-04-13T03:09:31.006135+0000","last_undegraded":"2024-04-13T03:09:31.006135+0000","last_fullsized":"2024-04-13T03:09:31.006135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:35.343303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.004564+0000","last_change":"2024-04-13T03:09:24.973883+0000","last_active":"2024-04-13T03:09:31.004564+0000","last_peered":"2024-04-13T03:09:31.004564+0000","last_clean":"2024-04-13T03:09:31.004564+0000","last_became_active":"2024-04-13T03:09:24.973623+0000","last_became_peered":"2024-04-13T03:09:24.973623+0000","last_unstale":"2024-04-13T03:09:31.004564+0000","last_undegraded":"2024-04-13T03:09:31.004564+0000","last_fullsized":"2024-04-13T03:09:31.004564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:07:35.429902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:31.552728+0000","last_change":"2024-04-13T03:09:24.980289+0000","last_active":"2024-04-13T03:09:31.552728+0000","last_peered":"2024-04-13T03:09:31.552728+0000","last_clean":"2024-04-13T03:09:31.552728+0000","last_became_active":"2024-04-13T03:09:24.980040+0000","last_became_peered":"2024-04-13T03:09:24.980040+0000","last_unstale":"2024-04-13T03:09:31.552728+0000","last_undegraded":"2024-04-13T03:09:31.552728+0000","last_fullsized":"2024-04-13T03:09:31.552728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:52.873491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-13T03:09:32.074027+0000","last_change":"2024-04-13T03:09:24.985108+0000","last_active":"2024-04-13T03:09:32.074027+0000","last_peered":"2024-04-13T03:09:32.074027+0000","last_clean":"2024-04-13T03:09:32.074027+0000","last_became_active":"2024-04-13T03:09:24.984576+0000","last_became_peered":"2024-04-13T03:09:24.984576+0000","last_unstale":"2024-04-13T03:09:32.074027+0000","last_undegraded":"2024-04-13T03:09:32.074027+0000","last_fullsized":"2024-04-13T03:09:32.074027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:23.951606+0000","last_clean_scrub_stamp":"2024-04-13T03:09:23.951606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:13.274342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:09:52.713 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-13T03:09:52.713 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-13T03:09:52.713 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-13T03:09:52.713 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:09:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.846+0000 7feed8f8c640 1 Processor -- start 2024-04-13T03:09:52.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.846+0000 7feed8f8c640 1 -- start start 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed8f8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feed415bbf0 0x7feed40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed8f8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed40b5560 con 0x7feed40b5150 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed8f8c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed4159580 con 0x7feed41591a0 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed8f8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feed40b3ab0 con 0x7feed415bbf0 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed1d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feed415bbf0 0x7feed40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feed40b5150 0x7feed406ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54922/0 (socket says 172.21.15.106:54922) 2024-04-13T03:09:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed2d9d640 1 -- 172.21.15.106:0/1587549629 learned_addr learned my addr 172.21.15.106:0/1587549629 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed1d9b640 1 -- 172.21.15.106:0/1587549629 >> v1:172.21.15.114:6789/0 conn(0x7feed41591a0 legacy=0x7feed40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed1d9b640 1 -- 172.21.15.106:0/1587549629 >> v1:172.21.15.106:6789/0 conn(0x7feed40b5150 legacy=0x7feed406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed159a640 1 -- 172.21.15.106:0/1587549629 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292128472 0 0) 0x7feecc001490 con 0x7feed40b5150 2024-04-13T03:09:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.847+0000 7feed1d9b640 1 -- 172.21.15.106:0/1587549629 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed40b3d90 con 0x7feed415bbf0 2024-04-13T03:09:52.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.848+0000 7feed1d9b640 1 --2- 172.21.15.106:0/1587549629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed415bbf0 0x7feed40b4c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feec80097a0 tx=0x7feec802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b9f706df2d118e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.848+0000 7feed159a640 1 -- 172.21.15.106:0/1587549629 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8034070 con 0x7feed415bbf0 2024-04-13T03:09:52.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.848+0000 7feed159a640 1 -- 172.21.15.106:0/1587549629 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8030610 con 0x7feed415bbf0 2024-04-13T03:09:52.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.848+0000 7feed159a640 1 -- 172.21.15.106:0/1587549629 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80309e0 con 0x7feed415bbf0 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 -- 172.21.15.106:0/1587549629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed415bbf0 msgr2=0x7feed40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 --2- 172.21.15.106:0/1587549629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed415bbf0 0x7feed40b4c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feec80097a0 tx=0x7feec802b960 comp rx=0 tx=0).stop 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 -- 172.21.15.106:0/1587549629 shutdown_connections 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 --2- 172.21.15.106:0/1587549629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed415bbf0 0x7feed40b4c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 -- 172.21.15.106:0/1587549629 >> 172.21.15.106:0/1587549629 conn(0x7feed40d0340 msgr2=0x7feed40d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 -- 172.21.15.106:0/1587549629 shutdown_connections 2024-04-13T03:09:52.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.849+0000 7feed8f8c640 1 -- 172.21.15.106:0/1587549629 wait complete. 2024-04-13T03:09:52.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.850+0000 7feed8f8c640 1 Processor -- start 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.850+0000 7feed8f8c640 1 -- start start 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feed41591a0 0x7feed406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 0x7feed41f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feed40b4630 con 0x7feed40b5150 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feed40b4350 con 0x7feed41591a0 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed8f8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feed406e4e0 con 0x7feed415bbf0 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52898/0 (socket says 172.21.15.106:52898) 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 -- 172.21.15.106:0/2931524018 learned_addr learned my addr 172.21.15.106:0/2931524018 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed1d9b640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feed41591a0 0x7feed406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed2d9d640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 0x7feed41f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 -- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 msgr2=0x7feed41f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 0x7feed41f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 -- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feed41591a0 msgr2=0x7feed406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feed41591a0 0x7feed406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 -- 172.21.15.106:0/2931524018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41f6ba0 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed2d9d640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 0x7feed41f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feed259c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feec400c480 tx=0x7feec400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40052d0 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.851+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4005470 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.852+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41f6e80 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.852+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4012870 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.852+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41e54f0 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.852+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed4068a10 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.853+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feec4012a10 con 0x7feed40b5150 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.853+0000 7feec2ffd640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 0x7feeb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:52.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.853+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feec408bae0 con 0x7feed40b5150 2024-04-13T03:09:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.853+0000 7feed1d9b640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 0x7feeb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:52.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.854+0000 7feed1d9b640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 0x7feeb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:52.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:52.858+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feec4059c60 con 0x7feed40b5150 2024-04-13T03:09:53.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.071+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7feed40d10b0 con 0x7feed40b5150 2024-04-13T03:09:53.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.071+0000 7feec2ffd640 1 -- 172.21.15.106:0/2931524018 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7feec4059560 con 0x7feed40b5150 2024-04-13T03:09:53.073 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:53.073 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 msgr2=0x7feeb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 0x7feeb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec803b040 comp rx=0 tx=0).stop 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 msgr2=0x7feed41e5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feec400c480 tx=0x7feec400c940 comp rx=0 tx=0).stop 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 shutdown_connections 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feeb406c8f0 0x7feeb406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feed415bbf0 0x7feed41f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feed41591a0 0x7feed406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 --2- 172.21.15.106:0/2931524018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed40b5150 0x7feed41e5cf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.073+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 >> 172.21.15.106:0/2931524018 conn(0x7feed40d0340 msgr2=0x7feed415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.074+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 shutdown_connections 2024-04-13T03:09:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.074+0000 7feed8f8c640 1 -- 172.21.15.106:0/2931524018 wait complete. 2024-04-13T03:09:53.085 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-13T03:09:53.086 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:53.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.186+0000 7f8a2c199640 1 Processor -- start 2024-04-13T03:09:53.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 -- start start 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8a241622f0 0x7f8a240cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a240d0010 con 0x7f8a240b7a60 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8a240d36b0 con 0x7f8a241622f0 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.187+0000 7f8a2c199640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8a240d3990 con 0x7f8a24161f10 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47158/0 (socket says 172.21.15.106:47158) 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a2a736640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8a240b7a60 0x7f8a240b7eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:50432/0 (socket says 172.21.15.106:50432) 2024-04-13T03:09:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29734640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8a241622f0 0x7f8a240cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a2a736640 1 -- 172.21.15.106:0/712925227 learned_addr learned my addr 172.21.15.106:0/712925227 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a28f33640 1 -- 172.21.15.106:0/712925227 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355541877 0 0) 0x7f8a240d0010 con 0x7f8a240b7a60 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a28f33640 1 -- 172.21.15.106:0/712925227 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a10003520 con 0x7f8a240b7a60 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 -- 172.21.15.106:0/712925227 >> v2:172.21.15.114:3300/0 conn(0x7f8a241622f0 msgr2=0x7f8a240cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 --2- 172.21.15.106:0/712925227 >> v2:172.21.15.114:3300/0 conn(0x7f8a241622f0 0x7f8a240cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 -- 172.21.15.106:0/712925227 >> v1:172.21.15.114:6789/0 conn(0x7f8a240b7a60 legacy=0x7f8a240b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.188+0000 7f8a29f35640 1 -- 172.21.15.106:0/712925227 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240d3c70 con 0x7f8a24161f10 2024-04-13T03:09:53.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a29f35640 1 --2- 172.21.15.106:0/712925227 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a200097a0 tx=0x7f8a2002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d981f39fdf3ebbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a28f33640 1 -- 172.21.15.106:0/712925227 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20034070 con 0x7f8a24161f10 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a28f33640 1 -- 172.21.15.106:0/712925227 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20030610 con 0x7f8a24161f10 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a28f33640 1 -- 172.21.15.106:0/712925227 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200309e0 con 0x7f8a24161f10 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a2c199640 1 -- 172.21.15.106:0/712925227 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 msgr2=0x7f8a240cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a2c199640 1 --2- 172.21.15.106:0/712925227 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a200097a0 tx=0x7f8a2002b960 comp rx=0 tx=0).stop 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a2c199640 1 -- 172.21.15.106:0/712925227 shutdown_connections 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.189+0000 7f8a2c199640 1 --2- 172.21.15.106:0/712925227 >> v2:172.21.15.114:3300/0 conn(0x7f8a241622f0 0x7f8a240cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.190+0000 7f8a2c199640 1 --2- 172.21.15.106:0/712925227 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a240cf170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.190+0000 7f8a2c199640 1 -- 172.21.15.106:0/712925227 >> 172.21.15.106:0/712925227 conn(0x7f8a240d4780 msgr2=0x7f8a240d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.190+0000 7f8a2c199640 1 -- 172.21.15.106:0/712925227 shutdown_connections 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.190+0000 7f8a2c199640 1 -- 172.21.15.106:0/712925227 wait complete. 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 Processor -- start 2024-04-13T03:09:53.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 -- start start 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a240b7a60 0x7f8a240cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a2415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b85c0 con 0x7f8a241622f0 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240d4230 con 0x7f8a240b7a60 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.191+0000 7f8a2c199640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8a240b8740 con 0x7f8a24161f10 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a29f35640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a240b7a60 0x7f8a240cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52914/0 (socket says 172.21.15.106:52914) 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 -- 172.21.15.106:0/579450569 learned_addr learned my addr 172.21.15.106:0/579450569 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a29734640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a2415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 -- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 msgr2=0x7f8a2415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a2415b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 -- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a240b7a60 msgr2=0x7f8a240cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a240b7a60 0x7f8a240cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 -- 172.21.15.106:0/579450569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a24158f20 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2a736640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00cb40 tx=0x7f8a1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c017070 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c00aca0 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241500a0 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c012620 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.192+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a240cc110 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.193+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a24069120 con 0x7f8a241622f0 2024-04-13T03:09:53.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.194+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a1c01a070 con 0x7f8a241622f0 2024-04-13T03:09:53.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.194+0000 7f8a1a7fc640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.194+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8a1c08bc90 con 0x7f8a241622f0 2024-04-13T03:09:53.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.194+0000 7f8a29f35640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.195+0000 7f8a29f35640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 0x7f8a0806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a20036000 tx=0x7f8a2003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.199+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a1c059740 con 0x7f8a241622f0 2024-04-13T03:09:53.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.374+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a240b7640 con 0x7f8a241622f0 2024-04-13T03:09:53.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.375+0000 7f8a1a7fc640 1 -- 172.21.15.106:0/579450569 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f8a1c05da40 con 0x7f8a241622f0 2024-04-13T03:09:53.379 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:53.379 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 msgr2=0x7f8a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 0x7f8a0806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a20036000 tx=0x7f8a2003b040 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 msgr2=0x7f8a2414fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00cb40 tx=0x7f8a1c00a060 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 shutdown_connections 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8a0806c8f0 0x7f8a0806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8a24161f10 0x7f8a2415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a241622f0 0x7f8a2414fb60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 --2- 172.21.15.106:0/579450569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a240b7a60 0x7f8a240cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 >> 172.21.15.106:0/579450569 conn(0x7f8a240d4780 msgr2=0x7f8a240b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 shutdown_connections 2024-04-13T03:09:53.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.377+0000 7f8a2c199640 1 -- 172.21.15.106:0/579450569 wait complete. 2024-04-13T03:09:53.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:09:53.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:53.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.491+0000 7f20d617b640 1 Processor -- start 2024-04-13T03:09:53.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.491+0000 7f20d617b640 1 -- start start 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20d617b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20d006e740 0x7f20d015c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20d617b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20d015c7a0 0x7f20d00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20d617b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d006d180 con 0x7f20d01592b0 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20d617b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f20d00ce180 con 0x7f20d006e740 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20d617b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f20d006ec90 con 0x7f20d015c7a0 2024-04-13T03:09:53.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20d006e740 0x7f20d015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20cf7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f20d01592b0 0x7f20d015bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54934/0 (socket says 172.21.15.106:54934) 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20d006e740 0x7f20d015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52924/0 (socket says 172.21.15.106:52924) 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20cf7fe640 1 -- 172.21.15.106:0/2675455807 learned_addr learned my addr 172.21.15.106:0/2675455807 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20cffff640 1 --2- 172.21.15.106:0/2675455807 >> v2:172.21.15.114:3300/0 conn(0x7f20d015c7a0 0x7f20d00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 -- 172.21.15.106:0/2675455807 >> v2:172.21.15.114:3300/0 conn(0x7f20d015c7a0 msgr2=0x7f20d00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 --2- 172.21.15.106:0/2675455807 >> v2:172.21.15.114:3300/0 conn(0x7f20d015c7a0 0x7f20d00ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 -- 172.21.15.106:0/2675455807 >> v1:172.21.15.106:6789/0 conn(0x7f20d01592b0 legacy=0x7f20d015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ceffd640 1 -- 172.21.15.106:0/2675455807 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d006ef70 con 0x7f20d006e740 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.492+0000 7f20ce7fc640 1 -- 172.21.15.106:0/2675455807 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670019552 0 0) 0x7f20d006d180 con 0x7f20d01592b0 2024-04-13T03:09:53.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20ceffd640 1 --2- 172.21.15.106:0/2675455807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d015c260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20d00b3070 tx=0x7f20c002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a485d15608d03acf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20ce7fc640 1 -- 172.21.15.106:0/2675455807 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c003b070 con 0x7f20d006e740 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20ce7fc640 1 -- 172.21.15.106:0/2675455807 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c002ee50 con 0x7f20d006e740 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20ce7fc640 1 -- 172.21.15.106:0/2675455807 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0036660 con 0x7f20d006e740 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20d617b640 1 -- 172.21.15.106:0/2675455807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 msgr2=0x7f20d015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.493+0000 7f20d617b640 1 --2- 172.21.15.106:0/2675455807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d015c260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20d00b3070 tx=0x7f20c002e0a0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 -- 172.21.15.106:0/2675455807 shutdown_connections 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 --2- 172.21.15.106:0/2675455807 >> v2:172.21.15.114:3300/0 conn(0x7f20d015c7a0 0x7f20d00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 --2- 172.21.15.106:0/2675455807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d015c260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 -- 172.21.15.106:0/2675455807 >> 172.21.15.106:0/2675455807 conn(0x7f20d00b2600 msgr2=0x7f20d00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 -- 172.21.15.106:0/2675455807 shutdown_connections 2024-04-13T03:09:53.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.494+0000 7f20d617b640 1 -- 172.21.15.106:0/2675455807 wait complete. 2024-04-13T03:09:53.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.495+0000 7f20d617b640 1 Processor -- start 2024-04-13T03:09:53.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.495+0000 7f20d617b640 1 -- start start 2024-04-13T03:09:53.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.495+0000 7f20d617b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20d01592b0 0x7f20d0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20d015c7a0 0x7f20d015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20d00ceb60 con 0x7f20d006e740 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20d006f530 con 0x7f20d01592b0 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20d00cece0 con 0x7f20d015c7a0 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52940/0 (socket says 172.21.15.106:52940) 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 -- 172.21.15.106:0/3196171059 learned_addr learned my addr 172.21.15.106:0/3196171059 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cffff640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20d015c7a0 0x7f20d015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20ceffd640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20d01592b0 0x7f20d0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 -- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20d015c7a0 msgr2=0x7f20d015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20d015c7a0 0x7f20d015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 -- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20d01592b0 msgr2=0x7f20d0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20d01592b0 0x7f20d0156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 -- 172.21.15.106:0/3196171059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d01f3c20 con 0x7f20d006e740 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20cf7fe640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c4d0 tx=0x7f20c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4018070 con 0x7f20d006e740 2024-04-13T03:09:53.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4007500 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004a20 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d0156cc0 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.496+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d0155490 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.497+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d0068a10 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.498+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20c4004bc0 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.498+0000 7f20b7fff640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 0x7f20bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.498+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f20c408b150 con 0x7f20d006e740 2024-04-13T03:09:53.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.498+0000 7f20ceffd640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 0x7f20bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.498+0000 7f20ceffd640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 0x7f20bc06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20c0038000 tx=0x7f20c0032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.502+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20c40591c0 con 0x7f20d006e740 2024-04-13T03:09:53.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.678+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20d0070050 con 0x7f20d006e740 2024-04-13T03:09:53.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.678+0000 7f20b7fff640 1 -- 172.21.15.106:0/3196171059 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f20c4058ac0 con 0x7f20d006e740 2024-04-13T03:09:53.680 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 msgr2=0x7f20bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 0x7f20bc06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20c0038000 tx=0x7f20c0032000 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 msgr2=0x7f20d01f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c4d0 tx=0x7f20c400c990 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 shutdown_connections 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20bc06c920 0x7f20bc06ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20d015c7a0 0x7f20d015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20d01592b0 0x7f20d0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 --2- 172.21.15.106:0/3196171059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20d006e740 0x7f20d01f36e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.680+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 >> 172.21.15.106:0/3196171059 conn(0x7f20d00b2600 msgr2=0x7f20d00b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.681+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 shutdown_connections 2024-04-13T03:09:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.681+0000 7f20d617b640 1 -- 172.21.15.106:0/3196171059 wait complete. 2024-04-13T03:09:53.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:09:53.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.794+0000 7f40419e0640 1 Processor -- start 2024-04-13T03:09:53.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.794+0000 7f40419e0640 1 -- start start 2024-04-13T03:09:53.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f40419e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f403c0ba170 0x7f403c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f40419e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0cfda0 con 0x7f403c0cf480 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f40419e0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c15f600 con 0x7f403c0d3220 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f40419e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f403c0d2d90 con 0x7f403c0ba170 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f403c0cf480 0x7f403c15ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54944/0 (socket says 172.21.15.106:54944) 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403affd640 1 -- 172.21.15.106:0/3023250696 learned_addr learned my addr 172.21.15.106:0/3023250696 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403a7fc640 1 --2- 172.21.15.106:0/3023250696 >> v2:172.21.15.106:3300/0 conn(0x7f403c0ba170 0x7f403c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403a7fc640 1 -- 172.21.15.106:0/3023250696 >> v1:172.21.15.114:6789/0 conn(0x7f403c0d3220 legacy=0x7f403c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403a7fc640 1 -- 172.21.15.106:0/3023250696 >> v1:172.21.15.106:6789/0 conn(0x7f403c0cf480 legacy=0x7f403c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f403a7fc640 1 -- 172.21.15.106:0/3023250696 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c15f8e0 con 0x7f403c0ba170 2024-04-13T03:09:53.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.795+0000 7f4039ffb640 1 -- 172.21.15.106:0/3023250696 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415183256 0 0) 0x7f403c0cfda0 con 0x7f403c0cf480 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.796+0000 7f403a7fc640 1 --2- 172.21.15.106:0/3023250696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0ba170 0x7f403c0cf860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40300097a0 tx=0x7f403002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef741be3a39f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.796+0000 7f4039ffb640 1 -- 172.21.15.106:0/3023250696 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030034070 con 0x7f403c0ba170 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.796+0000 7f4039ffb640 1 -- 172.21.15.106:0/3023250696 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4030030610 con 0x7f403c0ba170 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.796+0000 7f4039ffb640 1 -- 172.21.15.106:0/3023250696 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300309e0 con 0x7f403c0ba170 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 -- 172.21.15.106:0/3023250696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0ba170 msgr2=0x7f403c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 --2- 172.21.15.106:0/3023250696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0ba170 0x7f403c0cf860 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40300097a0 tx=0x7f403002b960 comp rx=0 tx=0).stop 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 -- 172.21.15.106:0/3023250696 shutdown_connections 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 --2- 172.21.15.106:0/3023250696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0ba170 0x7f403c0cf860 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 -- 172.21.15.106:0/3023250696 >> 172.21.15.106:0/3023250696 conn(0x7f403c0d47d0 msgr2=0x7f403c0d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 -- 172.21.15.106:0/3023250696 shutdown_connections 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.797+0000 7f40419e0640 1 -- 172.21.15.106:0/3023250696 wait complete. 2024-04-13T03:09:53.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.798+0000 7f40419e0640 1 Processor -- start 2024-04-13T03:09:53.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.798+0000 7f40419e0640 1 -- start start 2024-04-13T03:09:53.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f403c0ba170 0x7f403c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f403c0cf480 0x7f403c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1592f0 con 0x7f403c0d3220 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f403c159170 con 0x7f403c0ba170 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f40419e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0d3de0 con 0x7f403c0cf480 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52958/0 (socket says 172.21.15.106:52958) 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 -- 172.21.15.106:0/1966622386 learned_addr learned my addr 172.21.15.106:0/1966622386 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403affd640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f403c0ba170 0x7f403c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403a7fc640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f403c0cf480 0x7f403c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 -- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f403c0cf480 msgr2=0x7f403c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f403c0cf480 0x7f403c0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 -- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f403c0ba170 msgr2=0x7f403c159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f403c0ba170 0x7f403c159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 -- 172.21.15.106:0/1966622386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0c94d0 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.799+0000 7f403b7fe640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f403400c390 tx=0x7f403400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.800+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034017070 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.800+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340052d0 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.800+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034003810 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.800+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0c97b0 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.800+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c0c9d70 con 0x7f403c0d3220 2024-04-13T03:09:53.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.801+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0ccaf0 con 0x7f403c0d3220 2024-04-13T03:09:53.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.801+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f403401a070 con 0x7f403c0d3220 2024-04-13T03:09:53.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.802+0000 7f4023fff640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:53.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.802+0000 7f403affd640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:53.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.802+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f403408b0d0 con 0x7f403c0d3220 2024-04-13T03:09:53.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.802+0000 7f403affd640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 0x7f402806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f402c005fd0 tx=0x7f402c007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:53.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.806+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4034059250 con 0x7f403c0d3220 2024-04-13T03:09:53.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.981+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f403c068a10 con 0x7f403c0d3220 2024-04-13T03:09:53.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.982+0000 7f4023fff640 1 -- 172.21.15.106:0/1966622386 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f4034058b50 con 0x7f403c0d3220 2024-04-13T03:09:53.983 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:53.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 msgr2=0x7f402806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 0x7f402806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f402c005fd0 tx=0x7f402c007f10 comp rx=0 tx=0).stop 2024-04-13T03:09:53.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 msgr2=0x7f403c157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:53.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f403400c390 tx=0x7f403400c850 comp rx=0 tx=0).stop 2024-04-13T03:09:53.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 shutdown_connections 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f402806c8f0 0x7f402806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f403c0cf480 0x7f403c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f403c0d3220 0x7f403c157620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 --2- 172.21.15.106:0/1966622386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f403c0ba170 0x7f403c159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 >> 172.21.15.106:0/1966622386 conn(0x7f403c0d47d0 msgr2=0x7f403c161720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 shutdown_connections 2024-04-13T03:09:53.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:53.984+0000 7f40419e0640 1 -- 172.21.15.106:0/1966622386 wait complete. 2024-04-13T03:09:53.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:09:53.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:09:53.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:09:54.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.140+0000 7f3448208640 1 Processor -- start 2024-04-13T03:09:54.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.140+0000 7f3448208640 1 -- start start 2024-04-13T03:09:54.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3448208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f34400ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3448208640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f344015eca0 0x7f34400b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3448208640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34400b5060 con 0x7f344015e8c0 2024-04-13T03:09:54.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3448208640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f34400b5340 con 0x7f344015eca0 2024-04-13T03:09:54.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3448208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f344006c500 con 0x7f3440073520 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3445fa4640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f344015e8c0 0x7f3440072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54946/0 (socket says 172.21.15.106:54946) 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3445fa4640 1 -- 172.21.15.106:0/3036559625 learned_addr learned my addr 172.21.15.106:0/3036559625 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f34467a5640 1 --2- 172.21.15.106:0/3036559625 >> v2:172.21.15.114:3300/0 conn(0x7f344015eca0 0x7f34400b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f34457a3640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f34400ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267651498 0 0) 0x7f34400b5060 con 0x7f344015e8c0 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.141+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f342c003520 con 0x7f344015e8c0 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2058981618 0 0) 0x7f342c003520 con 0x7f344015e8c0 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 msgr2=0x7f34400ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f34400ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 >> v2:172.21.15.114:3300/0 conn(0x7f344015eca0 msgr2=0x7f34400b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 --2- 172.21.15.106:0/3036559625 >> v2:172.21.15.114:3300/0 conn(0x7f344015eca0 0x7f34400b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344006c7e0 con 0x7f344015e8c0 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f343c0019f0 con 0x7f344015e8c0 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 >> v1:172.21.15.106:6789/0 conn(0x7f344015e8c0 legacy=0x7f3440072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f342c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f342c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f342c004d40 0x7f342c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34400b5340 con 0x7f342c0039b0 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f344006c500 con 0x7f342c004d40 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f342c00b9c0 con 0x7f3440073520 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f342c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f34467a5640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f342c004d40 0x7f342c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f34457a3640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f342c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 -- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 msgr2=0x7f342c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f342c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 -- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f342c004d40 msgr2=0x7f342c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f342c004d40 0x7f342c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f3445fa4640 1 -- 172.21.15.106:0/3036559625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c003520 con 0x7f342c0039b0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.142+0000 7f34457a3640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3440073520 0x7f342c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3445fa4640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f342c003dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f343c00a880 tx=0x7f342c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6c3ecb916d0993 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00a4a0 con 0x7f342c0039b0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c034650 con 0x7f342c0039b0 2024-04-13T03:09:54.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3444fa2640 1 -- 172.21.15.106:0/3036559625 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c034a40 con 0x7f342c0039b0 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 -- 172.21.15.106:0/3036559625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 msgr2=0x7f342c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 --2- 172.21.15.106:0/3036559625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f342c003dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f343c00a880 tx=0x7f342c005830 comp rx=0 tx=0).stop 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f34467a5640 1 -- 172.21.15.106:0/3036559625 reap_dead start 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 -- 172.21.15.106:0/3036559625 shutdown_connections 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 -- 172.21.15.106:0/3036559625 >> 172.21.15.106:0/3036559625 conn(0x7f344006f0d0 msgr2=0x7f344006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 -- 172.21.15.106:0/3036559625 shutdown_connections 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.143+0000 7f3448208640 1 -- 172.21.15.106:0/3036559625 wait complete. 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 Processor -- start 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 -- start start 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f344015f0e0 0x7f344015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f344015ff30 0x7f34401603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34400ce5e0 con 0x7f342c0039b0 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f34400ce460 con 0x7f344015ff30 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3448208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34400b4970 con 0x7f344015f0e0 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3445fa4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3445fa4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52982/0 (socket says 172.21.15.106:52982) 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.145+0000 7f3445fa4640 1 -- 172.21.15.106:0/2650166111 learned_addr learned my addr 172.21.15.106:0/2650166111 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f34457a3640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f344015f0e0 0x7f344015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f34467a5640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f344015ff30 0x7f34401603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 -- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f344015f0e0 msgr2=0x7f344015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f344015f0e0 0x7f344015d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 -- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f344015ff30 msgr2=0x7f34401603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f344015ff30 0x7f34401603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 -- 172.21.15.106:0/2650166111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344015f650 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3445fa4640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f343c02cc90 tx=0x7f344006c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c045070 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c03a040 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401608f0 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c034a20 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.146+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34400cabc0 con 0x7f342c0039b0 2024-04-13T03:09:54.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.147+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3440068a10 con 0x7f342c0039b0 2024-04-13T03:09:54.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.148+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f343c048070 con 0x7f342c0039b0 2024-04-13T03:09:54.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.148+0000 7f34367fc640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.148+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f343c0adba0 con 0x7f342c0039b0 2024-04-13T03:09:54.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.148+0000 7f34457a3640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.148+0000 7f34457a3640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 0x7f342406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34400b0930 tx=0x7f3430008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.152+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f343c07b6d0 con 0x7f342c0039b0 2024-04-13T03:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.328+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3440073dd0 con 0x7f342c0039b0 2024-04-13T03:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.328+0000 7f34367fc640 1 -- 172.21.15.106:0/2650166111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f343c0ad0a0 con 0x7f342c0039b0 2024-04-13T03:09:54.330 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 msgr2=0x7f342406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 0x7f342406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34400b0930 tx=0x7f3430008040 comp rx=0 tx=0).stop 2024-04-13T03:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 msgr2=0x7f3440161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f343c02cc90 tx=0x7f344006c9b0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 shutdown_connections 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f342406c8f0 0x7f342406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f344015f0e0 0x7f344015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f344015ff30 0x7f34401603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 --2- 172.21.15.106:0/2650166111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f342c0039b0 0x7f3440161d80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 >> 172.21.15.106:0/2650166111 conn(0x7f344006f0d0 msgr2=0x7f344015b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.330+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 shutdown_connections 2024-04-13T03:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.331+0000 7f3448208640 1 -- 172.21.15.106:0/2650166111 wait complete. 2024-04-13T03:09:54.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:09:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 Processor -- start 2024-04-13T03:09:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 -- start start 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7f500aeae0 0x7f7f50169840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f501632d0 con 0x7f7f5006fe60 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f500b3fc0 con 0x7f7f5015ea80 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.445+0000 7f7f549bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7f500b42a0 con 0x7f7f500aeae0 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7f500aeae0 0x7f7f50169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7f500aeae0 0x7f7f50169840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52988/0 (socket says 172.21.15.106:52988) 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4dd9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7f5006fe60 0x7f7f50162bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54952/0 (socket says 172.21.15.106:54952) 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 -- 172.21.15.106:0/1799349313 learned_addr learned my addr 172.21.15.106:0/1799349313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 -- 172.21.15.106:0/1799349313 >> v1:172.21.15.114:6789/0 conn(0x7f7f5015ea80 legacy=0x7f7f501624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 -- 172.21.15.106:0/1799349313 >> v1:172.21.15.106:6789/0 conn(0x7f7f5006fe60 legacy=0x7f7f50162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.446+0000 7f7f4ed9d640 1 -- 172.21.15.106:0/1799349313 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f500b4580 con 0x7f7f500aeae0 2024-04-13T03:09:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f4ed9d640 1 --2- 172.21.15.106:0/1799349313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f50169840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009870 tx=0x7f7f4802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee4c1cc1d4c1c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f4d59a640 1 -- 172.21.15.106:0/1799349313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4802f840 con 0x7f7f500aeae0 2024-04-13T03:09:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f4d59a640 1 -- 172.21.15.106:0/1799349313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4802f9e0 con 0x7f7f500aeae0 2024-04-13T03:09:54.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f4d59a640 1 -- 172.21.15.106:0/1799349313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4802fdb0 con 0x7f7f500aeae0 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 -- 172.21.15.106:0/1799349313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 msgr2=0x7f7f50169840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 --2- 172.21.15.106:0/1799349313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f50169840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009870 tx=0x7f7f4802ba20 comp rx=0 tx=0).stop 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 -- 172.21.15.106:0/1799349313 shutdown_connections 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 --2- 172.21.15.106:0/1799349313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f50169840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 -- 172.21.15.106:0/1799349313 >> 172.21.15.106:0/1799349313 conn(0x7f7f500b2600 msgr2=0x7f7f500d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 -- 172.21.15.106:0/1799349313 shutdown_connections 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.447+0000 7f7f549bb640 1 -- 172.21.15.106:0/1799349313 wait complete. 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.449+0000 7f7f549bb640 1 Processor -- start 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.449+0000 7f7f549bb640 1 -- start start 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.449+0000 7f7f549bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f5006fe60 0x7f7f500cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.449+0000 7f7f549bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f5015ea80 0x7f7f5006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50169d80 con 0x7f7f500aeae0 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500b3c90 con 0x7f7f5006fe60 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f50169f00 con 0x7f7f5015ea80 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52990/0 (socket says 172.21.15.106:52990) 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 -- 172.21.15.106:0/4074042605 learned_addr learned my addr 172.21.15.106:0/4074042605 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:54.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4ed9d640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f5015ea80 0x7f7f5006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4e59c640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f5006fe60 0x7f7f500cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 -- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f5015ea80 msgr2=0x7f7f5006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f5015ea80 0x7f7f5006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 -- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f5006fe60 msgr2=0x7f7f500cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f5006fe60 0x7f7f500cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 -- 172.21.15.106:0/4074042605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f50161920 con 0x7f7f500aeae0 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f4dd9b640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c4d0 tx=0x7f7f4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44018070 con 0x7f7f500aeae0 2024-04-13T03:09:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f44007480 con 0x7f7f500aeae0 2024-04-13T03:09:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440049e0 con 0x7f7f500aeae0 2024-04-13T03:09:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f50161c00 con 0x7f7f500aeae0 2024-04-13T03:09:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.450+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f500cfaf0 con 0x7f7f500aeae0 2024-04-13T03:09:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.452+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50068a10 con 0x7f7f500aeae0 2024-04-13T03:09:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.452+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f4401b070 con 0x7f7f500aeae0 2024-04-13T03:09:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.452+0000 7f7f3effd640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 0x7f7f3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.452+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7f4408b1a0 con 0x7f7f500aeae0 2024-04-13T03:09:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.452+0000 7f7f4e59c640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 0x7f7f3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.453+0000 7f7f4e59c640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 0x7f7f3006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f40007fd0 tx=0x7f7f40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.457+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f44007670 con 0x7f7f500aeae0 2024-04-13T03:09:54.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.631+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f5006e2a0 con 0x7f7f500aeae0 2024-04-13T03:09:54.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.631+0000 7f7f3effd640 1 -- 172.21.15.106:0/4074042605 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f7f44059320 con 0x7f7f500aeae0 2024-04-13T03:09:54.632 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 msgr2=0x7f7f3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 0x7f7f3006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f40007fd0 tx=0x7f7f40009040 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 msgr2=0x7f7f5006bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c4d0 tx=0x7f7f4400c990 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 shutdown_connections 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f3006c8f0 0x7f7f3006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f5015ea80 0x7f7f5006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f500aeae0 0x7f7f5006bc80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 --2- 172.21.15.106:0/4074042605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f5006fe60 0x7f7f500cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 >> 172.21.15.106:0/4074042605 conn(0x7f7f500b2600 msgr2=0x7f7f5006eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 shutdown_connections 2024-04-13T03:09:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.633+0000 7f7f549bb640 1 -- 172.21.15.106:0/4074042605 wait complete. 2024-04-13T03:09:54.645 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.744+0000 7f439b9cc640 1 Processor -- start 2024-04-13T03:09:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.744+0000 7f439b9cc640 1 -- start start 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f439b9cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43940d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f439b9cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f43940d06c0 0x7f43940b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f439b9cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43940b5d60 con 0x7f43940b5940 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f439b9cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f43940d1be0 con 0x7f43940d06c0 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f439b9cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f43940d0b40 con 0x7f43940ce580 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f4399f69640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f43940b5940 0x7f43941627f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54956/0 (socket says 172.21.15.106:54956) 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f4399f69640 1 -- 172.21.15.106:0/3225569956 learned_addr learned my addr 172.21.15.106:0/3225569956 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f4399768640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43940d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f4398f67640 1 --2- 172.21.15.106:0/3225569956 >> v2:172.21.15.114:3300/0 conn(0x7f43940d06c0 0x7f43940b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289265405 0 0) 0x7f43940b5d60 con 0x7f43940b5940 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4380003520 con 0x7f43940b5940 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2924415491 0 0) 0x7f4380003520 con 0x7f43940b5940 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 msgr2=0x7f43940d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43940d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 >> v2:172.21.15.114:3300/0 conn(0x7f43940d06c0 msgr2=0x7f43940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 --2- 172.21.15.106:0/3225569956 >> v2:172.21.15.114:3300/0 conn(0x7f43940d06c0 0x7f43940b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43940d0e20 con 0x7f43940b5940 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f438c001a10 con 0x7f43940b5940 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 >> v1:172.21.15.106:6789/0 conn(0x7f43940b5940 legacy=0x7f43941627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43800039b0 0x7f4380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4380004800 0x7f4380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43940d1be0 con 0x7f4380004800 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43940d0b40 con 0x7f43800039b0 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.745+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f438000b9c0 con 0x7f43940ce580 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4380004800 0x7f4380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4399768640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43800039b0 0x7f4380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4399f69640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 -- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 msgr2=0x7f43800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940ce580 0x7f43800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 -- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43800039b0 msgr2=0x7f4380003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43800039b0 0x7f4380003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4398f67640 1 -- 172.21.15.106:0/3225569956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4380003520 con 0x7f4380004800 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.746+0000 7f4399768640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43800039b0 0x7f4380003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f4398f67640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4380004800 0x7f4380004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4384009ac0 tx=0x7f4380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ddeda902c5de96 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384039070 con 0x7f4380004800 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438402c710 con 0x7f4380004800 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f438bfff640 1 -- 172.21.15.106:0/3225569956 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384009690 con 0x7f4380004800 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 -- 172.21.15.106:0/3225569956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4380004800 msgr2=0x7f4380004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3225569956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4380004800 0x7f4380004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4384009ac0 tx=0x7f4380005830 comp rx=0 tx=0).stop 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f4399f69640 1 -- 172.21.15.106:0/3225569956 reap_dead start 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 -- 172.21.15.106:0/3225569956 shutdown_connections 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 -- 172.21.15.106:0/3225569956 >> 172.21.15.106:0/3225569956 conn(0x7f43940d25b0 msgr2=0x7f43940d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 -- 172.21.15.106:0/3225569956 shutdown_connections 2024-04-13T03:09:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.747+0000 7f439b9cc640 1 -- 172.21.15.106:0/3225569956 wait complete. 2024-04-13T03:09:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 Processor -- start 2024-04-13T03:09:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 -- start start 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4380004800 0x7f43941f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940b78e0 0x7f43940b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43940d16c0 con 0x7f43941f2840 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43940d13e0 con 0x7f4380004800 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f439b9cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f43940ce960 con 0x7f43940b78e0 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f4398f67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f4398f67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53004/0 (socket says 172.21.15.106:53004) 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.749+0000 7f4398f67640 1 -- 172.21.15.106:0/3434219750 learned_addr learned my addr 172.21.15.106:0/3434219750 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4399f69640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940b78e0 0x7f43940b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4399768640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4380004800 0x7f43941f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 -- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940b78e0 msgr2=0x7f43940b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940b78e0 0x7f43940b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 -- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4380004800 msgr2=0x7f43941f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4380004800 0x7f43941f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 -- 172.21.15.106:0/3434219750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941f5b90 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4398f67640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4384037040 tx=0x7f4384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384042020 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438403a040 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43941f5dc0 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384039070 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.750+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43941f6370 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.751+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4394074750 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.752+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4384045020 con 0x7f43941f2840 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.752+0000 7f4389ffb640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 0x7f437806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.752+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f43840ad3a0 con 0x7f43941f2840 2024-04-13T03:09:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.752+0000 7f4399768640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 0x7f437806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.753+0000 7f4399768640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 0x7f437806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4390009b40 tx=0x7f4390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.756+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f438404c070 con 0x7f43941f2840 2024-04-13T03:09:54.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.968+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43941f4a30 con 0x7f43941f2840 2024-04-13T03:09:54.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.969+0000 7f4389ffb640 1 -- 172.21.15.106:0/3434219750 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f438407b520 con 0x7f43941f2840 2024-04-13T03:09:54.970 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:54.971 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:54.971 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 msgr2=0x7f437806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 0x7f437806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4390009b40 tx=0x7f4390008040 comp rx=0 tx=0).stop 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 msgr2=0x7f43941f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4384037040 tx=0x7f4384004730 comp rx=0 tx=0).stop 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 shutdown_connections 2024-04-13T03:09:54.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f437806c8f0 0x7f437806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f43940b78e0 0x7f43940b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43941f2840 0x7f43941f44f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 --2- 172.21.15.106:0/3434219750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4380004800 0x7f43941f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 >> 172.21.15.106:0/3434219750 conn(0x7f43940d25b0 msgr2=0x7f43940ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 shutdown_connections 2024-04-13T03:09:54.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:54.971+0000 7f439b9cc640 1 -- 172.21.15.106:0/3434219750 wait complete. 2024-04-13T03:09:54.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:55.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.082+0000 7fd18831e640 1 Processor -- start 2024-04-13T03:09:55.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.082+0000 7fd18831e640 1 -- start start 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd18831e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd1800ce580 0x7fd1800d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd18831e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800d0b40 con 0x7fd1800d06c0 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd18831e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800d0e20 con 0x7fd1800b5b10 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd18831e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd1800d1be0 con 0x7fd1800ce580 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd1800ce580 0x7fd1800d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd1800ce580 0x7fd1800d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53008/0 (socket says 172.21.15.106:53008) 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 -- 172.21.15.106:0/3741029363 learned_addr learned my addr 172.21.15.106:0/3741029363 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1858b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd1800d06c0 0x7fd1800b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54962/0 (socket says 172.21.15.106:54962) 2024-04-13T03:09:55.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 -- 172.21.15.106:0/3741029363 >> v1:172.21.15.114:6789/0 conn(0x7fd1800b5b10 legacy=0x7fd18015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 -- 172.21.15.106:0/3741029363 >> v1:172.21.15.106:6789/0 conn(0x7fd1800d06c0 legacy=0x7fd1800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1860ba640 1 -- 172.21.15.106:0/3741029363 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1800d1100 con 0x7fd1800ce580 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.083+0000 7fd1850b8640 1 -- 172.21.15.106:0/3741029363 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509902239 0 0) 0x7fd1800d0b40 con 0x7fd1800d06c0 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.084+0000 7fd1860ba640 1 --2- 172.21.15.106:0/3741029363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800ce580 0x7fd1800d0180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e113f9b0489144c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.084+0000 7fd1850b8640 1 -- 172.21.15.106:0/3741029363 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c02f840 con 0x7fd1800ce580 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.084+0000 7fd1850b8640 1 -- 172.21.15.106:0/3741029363 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c02f9e0 con 0x7fd1800ce580 2024-04-13T03:09:55.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.084+0000 7fd1850b8640 1 -- 172.21.15.106:0/3741029363 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c02fdb0 con 0x7fd1800ce580 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 -- 172.21.15.106:0/3741029363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800ce580 msgr2=0x7fd1800d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 --2- 172.21.15.106:0/3741029363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800ce580 0x7fd1800d0180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c02ba20 comp rx=0 tx=0).stop 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 -- 172.21.15.106:0/3741029363 shutdown_connections 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 --2- 172.21.15.106:0/3741029363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800ce580 0x7fd1800d0180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 -- 172.21.15.106:0/3741029363 >> 172.21.15.106:0/3741029363 conn(0x7fd1800d25b0 msgr2=0x7fd1800d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 -- 172.21.15.106:0/3741029363 shutdown_connections 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.085+0000 7fd18831e640 1 -- 172.21.15.106:0/3741029363 wait complete. 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 Processor -- start 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 -- start start 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd1800ce580 0x7fd18014db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1800d06c0 0x7fd1801f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800ce960 con 0x7fd1800b5b10 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800d16c0 con 0x7fd1800d06c0 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.086+0000 7fd18831e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd1800cdc10 con 0x7fd1800ce580 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53012/0 (socket says 172.21.15.106:53012) 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 -- 172.21.15.106:0/440267391 learned_addr learned my addr 172.21.15.106:0/440267391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:55.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 -- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd1800ce580 msgr2=0x7fd18014db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd1800ce580 0x7fd18014db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 -- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1800d06c0 msgr2=0x7fd1801f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1868bb640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1800d06c0 0x7fd1801f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1800d06c0 0x7fd1801f7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 -- 172.21.15.106:0/440267391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1801f87e0 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd1860ba640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0090b0 tx=0x7fd17c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c047070 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c037cc0 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801f8ac0 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004270 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.087+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801589d0 con 0x7fd1800b5b10 2024-04-13T03:09:55.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.088+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd180068a10 con 0x7fd1800b5b10 2024-04-13T03:09:55.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.089+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd17c04a070 con 0x7fd1800b5b10 2024-04-13T03:09:55.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.089+0000 7fd176ffd640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.089+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd17c0ae060 con 0x7fd1800b5b10 2024-04-13T03:09:55.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.089+0000 7fd1858b9640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.089+0000 7fd1858b9640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 0x7fd16406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd1700079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.093+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd17c033030 con 0x7fd1800b5b10 2024-04-13T03:09:55.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.272+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1800b20d0 con 0x7fd1800b5b10 2024-04-13T03:09:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.274+0000 7fd176ffd640 1 -- 172.21.15.106:0/440267391 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fd17c07c1e0 con 0x7fd1800b5b10 2024-04-13T03:09:55.275 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:55.275 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 msgr2=0x7fd16406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 0x7fd16406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd1700079c0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 msgr2=0x7fd18015ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0090b0 tx=0x7fd17c004730 comp rx=0 tx=0).stop 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 shutdown_connections 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd1800ce580 0x7fd18014db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1800d06c0 0x7fd1801f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 --2- 172.21.15.106:0/440267391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1800b5b10 0x7fd18015ae60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 >> 172.21.15.106:0/440267391 conn(0x7fd1800d25b0 msgr2=0x7fd1800b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 shutdown_connections 2024-04-13T03:09:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.276+0000 7fd18831e640 1 -- 172.21.15.106:0/440267391 wait complete. 2024-04-13T03:09:55.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:09:55.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:09:55.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 Processor -- start 2024-04-13T03:09:55.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 -- start start 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f731015ed90 0x7f73100d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73100cfd20 con 0x7f73100b3d60 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7310162af0 con 0x7f731015e9b0 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.392+0000 7f73156c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7310162dd0 con 0x7f731015ed90 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f731015ed90 0x7f73100d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f73100b3d60 0x7f73100d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54968/0 (socket says 172.21.15.106:54968) 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730f7fe640 1 -- 172.21.15.106:0/3147631914 learned_addr learned my addr 172.21.15.106:0/3147631914 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730e7fc640 1 -- 172.21.15.106:0/3147631914 >> v1:172.21.15.114:6789/0 conn(0x7f731015e9b0 legacy=0x7f73100d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730e7fc640 1 -- 172.21.15.106:0/3147631914 >> v1:172.21.15.106:6789/0 conn(0x7f73100b3d60 legacy=0x7f73100d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730e7fc640 1 -- 172.21.15.106:0/3147631914 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101630b0 con 0x7f731015ed90 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.393+0000 7f730dffb640 1 -- 172.21.15.106:0/3147631914 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145431469 0 0) 0x7f73100cfd20 con 0x7f73100b3d60 2024-04-13T03:09:55.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f730e7fc640 1 --2- 172.21.15.106:0/3147631914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015ed90 0x7f73100d2300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7304009de0 tx=0x7f730402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa454836ac0678d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f730dffb640 1 -- 172.21.15.106:0/3147631914 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304039070 con 0x7f731015ed90 2024-04-13T03:09:55.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f730dffb640 1 -- 172.21.15.106:0/3147631914 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730402cdc0 con 0x7f731015ed90 2024-04-13T03:09:55.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f730dffb640 1 -- 172.21.15.106:0/3147631914 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304034660 con 0x7f731015ed90 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 -- 172.21.15.106:0/3147631914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015ed90 msgr2=0x7f73100d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 --2- 172.21.15.106:0/3147631914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015ed90 0x7f73100d2300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7304009de0 tx=0x7f730402c010 comp rx=0 tx=0).stop 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 -- 172.21.15.106:0/3147631914 shutdown_connections 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 --2- 172.21.15.106:0/3147631914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015ed90 0x7f73100d2300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 -- 172.21.15.106:0/3147631914 >> 172.21.15.106:0/3147631914 conn(0x7f731008bf10 msgr2=0x7f73100b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 -- 172.21.15.106:0/3147631914 shutdown_connections 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.394+0000 7f73156c8640 1 -- 172.21.15.106:0/3147631914 wait complete. 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 Processor -- start 2024-04-13T03:09:55.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 -- start start 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73100b3d60 0x7f731015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73100ceab0 0x7f73100ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7310162800 con 0x7f731015e9b0 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7310162680 con 0x7f73100b3d60 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f73156c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f73100d4d00 con 0x7f73100ceab0 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f730e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f730e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53034/0 (socket says 172.21.15.106:53034) 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.396+0000 7f730e7fc640 1 -- 172.21.15.106:0/563764190 learned_addr learned my addr 172.21.15.106:0/563764190 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730effd640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73100b3d60 0x7f731015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730f7fe640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73100ceab0 0x7f73100ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 -- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73100ceab0 msgr2=0x7f73100ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73100ceab0 0x7f73100ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 -- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73100b3d60 msgr2=0x7f731015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73100b3d60 0x7f731015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 -- 172.21.15.106:0/563764190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101f48a0 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f730e7fc640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7304037040 tx=0x7f7304004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304039050 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730403a040 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101f4b80 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304003990 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.397+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7310161aa0 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.398+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7310068a10 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.398+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7304047030 con 0x7f731015e9b0 2024-04-13T03:09:55.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.399+0000 7f72f77fe640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 0x7f72fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.399+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f73040ae2f0 con 0x7f731015e9b0 2024-04-13T03:09:55.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.399+0000 7f730effd640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 0x7f72fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.399+0000 7f730effd640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 0x7f72fc06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7308000f60 tx=0x7f7308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.403+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f730404e020 con 0x7f731015e9b0 2024-04-13T03:09:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.581+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73100d53a0 con 0x7f731015e9b0 2024-04-13T03:09:55.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.581+0000 7f72f77fe640 1 -- 172.21.15.106:0/563764190 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f730407c470 con 0x7f731015e9b0 2024-04-13T03:09:55.583 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 msgr2=0x7f72fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 0x7f72fc06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7308000f60 tx=0x7f7308009040 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 msgr2=0x7f73100cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7304037040 tx=0x7f7304004a40 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 shutdown_connections 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f72fc06c920 0x7f72fc06ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73100ceab0 0x7f73100ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f731015e9b0 0x7f73100cdc20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 --2- 172.21.15.106:0/563764190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73100b3d60 0x7f731015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.583+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 >> 172.21.15.106:0/563764190 conn(0x7f731008bf10 msgr2=0x7f73100b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.584+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 shutdown_connections 2024-04-13T03:09:55.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.584+0000 7f73156c8640 1 -- 172.21.15.106:0/563764190 wait complete. 2024-04-13T03:09:55.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:09:55.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.697+0000 7f96f90c8640 1 Processor -- start 2024-04-13T03:09:55.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.697+0000 7f96f90c8640 1 -- start start 2024-04-13T03:09:55.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f90c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96f40cd210 0x7f96f40b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f90c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40d0da0 con 0x7f96f415c960 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f90c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40d1080 con 0x7f96f40ccdd0 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f90c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f96f40d0b50 con 0x7f96f40cd210 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f96f415c960 0x7f96f40b17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54970/0 (socket says 172.21.15.106:54970) 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f2d9d640 1 -- 172.21.15.106:0/2091128719 learned_addr learned my addr 172.21.15.106:0/2091128719 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.698+0000 7f96f359e640 1 --2- 172.21.15.106:0/2091128719 >> v2:172.21.15.106:3300/0 conn(0x7f96f40cd210 0x7f96f40b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f359e640 1 -- 172.21.15.106:0/2091128719 >> v1:172.21.15.114:6789/0 conn(0x7f96f40ccdd0 legacy=0x7f96f40b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f359e640 1 -- 172.21.15.106:0/2091128719 >> v1:172.21.15.106:6789/0 conn(0x7f96f415c960 legacy=0x7f96f40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f1d9b640 1 -- 172.21.15.106:0/2091128719 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915789584 0 0) 0x7f96f40d0da0 con 0x7f96f415c960 2024-04-13T03:09:55.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f359e640 1 -- 172.21.15.106:0/2091128719 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f40d1360 con 0x7f96f40cd210 2024-04-13T03:09:55.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f359e640 1 --2- 172.21.15.106:0/2091128719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f40cd210 0x7f96f40b5ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0097a0 tx=0x7f96ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f3303433f73648 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f1d9b640 1 -- 172.21.15.106:0/2091128719 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec034070 con 0x7f96f40cd210 2024-04-13T03:09:55.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f1d9b640 1 -- 172.21.15.106:0/2091128719 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec030610 con 0x7f96f40cd210 2024-04-13T03:09:55.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.699+0000 7f96f1d9b640 1 -- 172.21.15.106:0/2091128719 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0309e0 con 0x7f96f40cd210 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 -- 172.21.15.106:0/2091128719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f40cd210 msgr2=0x7f96f40b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 --2- 172.21.15.106:0/2091128719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f40cd210 0x7f96f40b5ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0097a0 tx=0x7f96ec02b960 comp rx=0 tx=0).stop 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 -- 172.21.15.106:0/2091128719 shutdown_connections 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 --2- 172.21.15.106:0/2091128719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f40cd210 0x7f96f40b5ad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 -- 172.21.15.106:0/2091128719 >> 172.21.15.106:0/2091128719 conn(0x7f96f40ceff0 msgr2=0x7f96f40cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 -- 172.21.15.106:0/2091128719 shutdown_connections 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.700+0000 7f96f90c8640 1 -- 172.21.15.106:0/2091128719 wait complete. 2024-04-13T03:09:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.701+0000 7f96f90c8640 1 Processor -- start 2024-04-13T03:09:55.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.701+0000 7f96f90c8640 1 -- start start 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96f40ccdd0 0x7f96f4158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f40cd210 0x7f96f41f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40cd660 con 0x7f96f415c960 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40d19b0 con 0x7f96f40cd210 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f90c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96f415cd40 con 0x7f96f40ccdd0 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53054/0 (socket says 172.21.15.106:53054) 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f359e640 1 -- 172.21.15.106:0/294639628 learned_addr learned my addr 172.21.15.106:0/294639628 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f259c640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f40cd210 0x7f96f41f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.702+0000 7f96f2d9d640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96f40ccdd0 0x7f96f4158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 -- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96f40ccdd0 msgr2=0x7f96f4158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96f40ccdd0 0x7f96f4158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 -- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f40cd210 msgr2=0x7f96f41f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f40cd210 0x7f96f41f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 -- 172.21.15.106:0/294639628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f40caa40 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f359e640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0098c0 tx=0x7f96ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec049070 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec039780 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f40cad20 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec034040 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.703+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f41f3c10 con 0x7f96f415c960 2024-04-13T03:09:55.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.704+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f4069120 con 0x7f96f415c960 2024-04-13T03:09:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.704+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ec04c070 con 0x7f96f415c960 2024-04-13T03:09:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.705+0000 7f96db7fe640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 0x7f96e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.705+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96ec0adc40 con 0x7f96f415c960 2024-04-13T03:09:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.705+0000 7f96f2d9d640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 0x7f96e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.705+0000 7f96f2d9d640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 0x7f96e006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96e40012e0 tx=0x7f96e4001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:55.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.709+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96ec07b890 con 0x7f96f415c960 2024-04-13T03:09:55.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.883+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96f415a770 con 0x7f96f415c960 2024-04-13T03:09:55.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.884+0000 7f96db7fe640 1 -- 172.21.15.106:0/294639628 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f96ec07fb90 con 0x7f96f415c960 2024-04-13T03:09:55.885 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.885+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 msgr2=0x7f96e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.885+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 0x7f96e006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96e40012e0 tx=0x7f96e4001630 comp rx=0 tx=0).stop 2024-04-13T03:09:55.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 msgr2=0x7f96f415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:55.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0098c0 tx=0x7f96ec0046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 shutdown_connections 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96e006c8f0 0x7f96e006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96f415c960 0x7f96f415e3b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f40cd210 0x7f96f41f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 --2- 172.21.15.106:0/294639628 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96f40ccdd0 0x7f96f4158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 >> 172.21.15.106:0/294639628 conn(0x7f96f40ceff0 msgr2=0x7f96f415b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 shutdown_connections 2024-04-13T03:09:55.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:55.886+0000 7f96f90c8640 1 -- 172.21.15.106:0/294639628 wait complete. 2024-04-13T03:09:55.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:09:56.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 Processor -- start 2024-04-13T03:09:56.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 -- start start 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b680d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b680b3d60 0x7f8b680d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b680cfd20 con 0x7f8b6815ed90 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8b68162af0 con 0x7f8b680b3d60 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.002+0000 7f8b6d60f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8b68162dd0 con 0x7f8b6815e9b0 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b680b3d60 0x7f8b680d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b680b3d60 0x7f8b680d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53070/0 (socket says 172.21.15.106:53070) 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 -- 172.21.15.106:0/4068851416 learned_addr learned my addr 172.21.15.106:0/4068851416 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b5ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8b6815ed90 0x7f8b680d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54984/0 (socket says 172.21.15.106:54984) 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b66ffd640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b680d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 -- 172.21.15.106:0/4068851416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 msgr2=0x7f8b680d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b680d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 -- 172.21.15.106:0/4068851416 >> v1:172.21.15.106:6789/0 conn(0x7f8b6815ed90 legacy=0x7f8b680d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.003+0000 7f8b677fe640 1 -- 172.21.15.106:0/4068851416 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681630b0 con 0x7f8b680b3d60 2024-04-13T03:09:56.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b677fe640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b680d0200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009e50 tx=0x7f8b6002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd7249fc3f0f20f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b667fc640 1 -- 172.21.15.106:0/4068851416 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6002cb90 con 0x7f8b680b3d60 2024-04-13T03:09:56.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b667fc640 1 -- 172.21.15.106:0/4068851416 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6002cd30 con 0x7f8b680b3d60 2024-04-13T03:09:56.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b667fc640 1 -- 172.21.15.106:0/4068851416 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60034610 con 0x7f8b680b3d60 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 -- 172.21.15.106:0/4068851416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 msgr2=0x7f8b680d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b680d0200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009e50 tx=0x7f8b6002bff0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 -- 172.21.15.106:0/4068851416 shutdown_connections 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b680d0200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/4068851416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b680d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 -- 172.21.15.106:0/4068851416 >> 172.21.15.106:0/4068851416 conn(0x7f8b6808bf10 msgr2=0x7f8b680b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 -- 172.21.15.106:0/4068851416 shutdown_connections 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.004+0000 7f8b6d60f640 1 -- 172.21.15.106:0/4068851416 wait complete. 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.005+0000 7f8b6d60f640 1 Processor -- start 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 -- start start 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b68159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b680b6d70 0x7f8b68159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68162800 con 0x7f8b680b3d60 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68162680 con 0x7f8b680b6d70 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b6d60f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8b680d4d00 con 0x7f8b6815e9b0 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b66ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b66ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53080/0 (socket says 172.21.15.106:53080) 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b66ffd640 1 -- 172.21.15.106:0/2422309783 learned_addr learned my addr 172.21.15.106:0/2422309783 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.006+0000 7f8b677fe640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b680b6d70 0x7f8b68159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b5ffff640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b68159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 -- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 msgr2=0x7f8b68159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b68159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 -- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b680b6d70 msgr2=0x7f8b68159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b680b6d70 0x7f8b68159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 -- 172.21.15.106:0/2422309783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b680b5cd0 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b66ffd640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c7f0 tx=0x7f8b5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58015070 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b58004240 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58007620 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b680b5fb0 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.007+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b680b6570 con 0x7f8b680b3d60 2024-04-13T03:09:56.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.008+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68068a10 con 0x7f8b680b3d60 2024-04-13T03:09:56.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.008+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b580079d0 con 0x7f8b680b3d60 2024-04-13T03:09:56.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.009+0000 7f8b5f7fe640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 0x7f8b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.009+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b5808c320 con 0x7f8b680b3d60 2024-04-13T03:09:56.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.009+0000 7f8b5ffff640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 0x7f8b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.009+0000 7f8b5ffff640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 0x7f8b4806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001020 tx=0x7f8b50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.013+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b5805a410 con 0x7f8b680b3d60 2024-04-13T03:09:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.189+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b680d53a0 con 0x7f8b680b3d60 2024-04-13T03:09:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.189+0000 7f8b5f7fe640 1 -- 172.21.15.106:0/2422309783 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f8b58059d10 con 0x7f8b680b3d60 2024-04-13T03:09:56.190 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:09:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 msgr2=0x7f8b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 0x7f8b4806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001020 tx=0x7f8b50009040 comp rx=0 tx=0).stop 2024-04-13T03:09:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 msgr2=0x7f8b6815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c7f0 tx=0x7f8b5800ccb0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 shutdown_connections 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b4806c8f0 0x7f8b4806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b6815e9b0 0x7f8b68159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b680b6d70 0x7f8b68159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 --2- 172.21.15.106:0/2422309783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b680b3d60 0x7f8b6815afb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 >> 172.21.15.106:0/2422309783 conn(0x7f8b6808bf10 msgr2=0x7f8b680b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 shutdown_connections 2024-04-13T03:09:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.191+0000 7f8b6d60f640 1 -- 172.21.15.106:0/2422309783 wait complete. 2024-04-13T03:09:56.203 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:09:56.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.304+0000 7f69016ad640 1 Processor -- start 2024-04-13T03:09:56.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.304+0000 7f69016ad640 1 -- start start 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f69016ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68fc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f69016ad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc0b5250 con 0x7f68fc0b5c60 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f69016ad640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc06d170 con 0x7f68fc0cf780 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f69016ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc06d450 con 0x7f68fc15e9c0 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f68fa7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f68fc0b5c60 0x7f68fc0b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54998/0 (socket says 172.21.15.106:54998) 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.305+0000 7f68fa7fc640 1 -- 172.21.15.106:0/2356395 learned_addr learned my addr 172.21.15.106:0/2356395 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68fc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 77455633 0 0) 0x7f68fc0b5250 con 0x7f68fc0b5c60 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68e4003520 con 0x7f68fc0b5c60 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 870450556 0 0) 0x7f68e4003520 con 0x7f68fc0b5c60 2024-04-13T03:09:56.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 msgr2=0x7f68fc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68fc06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 >> v1:172.21.15.114:6789/0 conn(0x7f68fc0cf780 legacy=0x7f68fc0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc06d730 con 0x7f68fc0b5c60 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f68f0001670 con 0x7f68fc0b5c60 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 >> v1:172.21.15.106:6789/0 conn(0x7f68fc0b5c60 legacy=0x7f68fc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40039e0 0x7f68e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc06d450 con 0x7f68e40039e0 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68e400baf0 con 0x7f68e4004800 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68e400bcf0 con 0x7f68fc15e9c0 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40039e0 0x7f68e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68fa7fc640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68fb7fe640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 -- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 msgr2=0x7f68e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc15e9c0 0x7f68e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 -- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e4004800 msgr2=0x7f68e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.306+0000 7f68faffd640 1 -- 172.21.15.106:0/2356395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e4003520 con 0x7f68e40039e0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.307+0000 7f68faffd640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40039e0 0x7f68e4003e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fc0 tx=0x7f68e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b021a9d2251595 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.307+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec039070 con 0x7f68e40039e0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.307+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ec02cc20 con 0x7f68e40039e0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.307+0000 7f68f9ffb640 1 -- 172.21.15.106:0/2356395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec009c10 con 0x7f68e40039e0 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 -- 172.21.15.106:0/2356395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40039e0 msgr2=0x7f68e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 --2- 172.21.15.106:0/2356395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40039e0 0x7f68e4003e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fc0 tx=0x7f68e4005830 comp rx=0 tx=0).stop 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f68fb7fe640 1 -- 172.21.15.106:0/2356395 reap_dead start 2024-04-13T03:09:56.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 -- 172.21.15.106:0/2356395 shutdown_connections 2024-04-13T03:09:56.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 -- 172.21.15.106:0/2356395 >> 172.21.15.106:0/2356395 conn(0x7f68fc0d1480 msgr2=0x7f68fc0d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 -- 172.21.15.106:0/2356395 shutdown_connections 2024-04-13T03:09:56.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.308+0000 7f69016ad640 1 -- 172.21.15.106:0/2356395 wait complete. 2024-04-13T03:09:56.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.309+0000 7f69016ad640 1 Processor -- start 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.309+0000 7f69016ad640 1 -- start start 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 0x7f68fc158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc161900 0x7f68fc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc06cca0 con 0x7f68fc155490 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc15eda0 con 0x7f68e40039e0 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0b60c0 con 0x7f68fc161900 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53094/0 (socket says 172.21.15.106:53094) 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 -- 172.21.15.106:0/2200567844 learned_addr learned my addr 172.21.15.106:0/2200567844 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fa7fc640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc161900 0x7f68fc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68faffd640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 0x7f68fc158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 -- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc161900 msgr2=0x7f68fc1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc161900 0x7f68fc1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 -- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 msgr2=0x7f68fc158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 0x7f68fc158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 -- 172.21.15.106:0/2200567844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc158ea0 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68faffd640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 0x7f68fc158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68fb7fe640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0cfc10 tx=0x7f68f400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40053a0 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4018070 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003830 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.310+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc159180 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.311+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc159740 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.311+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc069120 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.312+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68f40039d0 con 0x7f68fc155490 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.312+0000 7f68e37fe640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 0x7f68e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.312+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68f408b8b0 con 0x7f68fc155490 2024-04-13T03:09:56.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.312+0000 7f68faffd640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 0x7f68e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.313+0000 7f68faffd640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 0x7f68e806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68ec036000 tx=0x7f68ec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.316+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68f4059a30 con 0x7f68fc155490 2024-04-13T03:09:56.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.490+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68fc0731d0 con 0x7f68fc155490 2024-04-13T03:09:56.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.491+0000 7f68e37fe640 1 -- 172.21.15.106:0/2200567844 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f68f4059330 con 0x7f68fc155490 2024-04-13T03:09:56.492 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 msgr2=0x7f68e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 0x7f68e806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68ec036000 tx=0x7f68ec02e000 comp rx=0 tx=0).stop 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 msgr2=0x7f68fc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0cfc10 tx=0x7f68f400f920 comp rx=0 tx=0).stop 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 shutdown_connections 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e806c8f0 0x7f68e806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68fc161900 0x7f68fc1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68fc155490 0x7f68fc0b2440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 --2- 172.21.15.106:0/2200567844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40039e0 0x7f68fc158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.492+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 >> 172.21.15.106:0/2200567844 conn(0x7f68fc0d1480 msgr2=0x7f68fc15e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.493+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 shutdown_connections 2024-04-13T03:09:56.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.493+0000 7f69016ad640 1 -- 172.21.15.106:0/2200567844 wait complete. 2024-04-13T03:09:56.505 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.604+0000 7f74d88b7640 1 Processor -- start 2024-04-13T03:09:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.604+0000 7f74d88b7640 1 -- start start 2024-04-13T03:09:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d88b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74d415ca40 0x7f74d40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d88b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d40d02a0 con 0x7f74d415bf50 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d88b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d40cfa40 con 0x7f74d40cf5e0 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d88b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f74d4070bd0 con 0x7f74d415ca40 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74d415ca40 0x7f74d40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f74d415bf50 0x7f74d4071060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55002/0 (socket says 172.21.15.106:55002) 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74d415ca40 0x7f74d40cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53110/0 (socket says 172.21.15.106:53110) 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d259c640 1 -- 172.21.15.106:0/686647555 learned_addr learned my addr 172.21.15.106:0/686647555 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d2d9d640 1 -- 172.21.15.106:0/686647555 >> v1:172.21.15.114:6789/0 conn(0x7f74d40cf5e0 legacy=0x7f74d415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d2d9d640 1 -- 172.21.15.106:0/686647555 >> v1:172.21.15.106:6789/0 conn(0x7f74d415bf50 legacy=0x7f74d4071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d2d9d640 1 -- 172.21.15.106:0/686647555 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d40cfce0 con 0x7f74d415ca40 2024-04-13T03:09:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.605+0000 7f74d159a640 1 -- 172.21.15.106:0/686647555 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005638746 0 0) 0x7f74d40d02a0 con 0x7f74d415bf50 2024-04-13T03:09:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.606+0000 7f74d2d9d640 1 --2- 172.21.15.106:0/686647555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d40cdec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f74d4070d50 tx=0x7f74c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b674b765f798fdd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.606+0000 7f74d159a640 1 -- 172.21.15.106:0/686647555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4035020 con 0x7f74d415ca40 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.606+0000 7f74d159a640 1 -- 172.21.15.106:0/686647555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4031830 con 0x7f74d415ca40 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.606+0000 7f74d159a640 1 -- 172.21.15.106:0/686647555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4031c00 con 0x7f74d415ca40 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 -- 172.21.15.106:0/686647555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 msgr2=0x7f74d40cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 --2- 172.21.15.106:0/686647555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d40cdec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f74d4070d50 tx=0x7f74c402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 -- 172.21.15.106:0/686647555 shutdown_connections 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 --2- 172.21.15.106:0/686647555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d40cdec0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 -- 172.21.15.106:0/686647555 >> 172.21.15.106:0/686647555 conn(0x7f74d4072ea0 msgr2=0x7f74d40732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 -- 172.21.15.106:0/686647555 shutdown_connections 2024-04-13T03:09:56.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.607+0000 7f74d88b7640 1 -- 172.21.15.106:0/686647555 wait complete. 2024-04-13T03:09:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.608+0000 7f74d88b7640 1 Processor -- start 2024-04-13T03:09:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.608+0000 7f74d88b7640 1 -- start start 2024-04-13T03:09:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74d40cf5e0 0x7f74d4140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74d415bf50 0x7f74d406df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74d40ce400 con 0x7f74d415ca40 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4070860 con 0x7f74d415bf50 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d88b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74d40ce580 con 0x7f74d40cf5e0 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53120/0 (socket says 172.21.15.106:53120) 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 -- 172.21.15.106:0/2640887131 learned_addr learned my addr 172.21.15.106:0/2640887131 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d1d9b640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74d415bf50 0x7f74d406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d259c640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74d40cf5e0 0x7f74d4140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 -- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74d40cf5e0 msgr2=0x7f74d4140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74d40cf5e0 0x7f74d4140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 -- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74d415bf50 msgr2=0x7f74d406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74d415bf50 0x7f74d406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.609+0000 7f74d2d9d640 1 -- 172.21.15.106:0/2640887131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d415a540 con 0x7f74d415ca40 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74d2d9d640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009890 tx=0x7f74c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c403a930 con 0x7f74d415ca40 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c403aad0 con 0x7f74d415ca40 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d415a820 con 0x7f74d415ca40 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40443f0 con 0x7f74d415ca40 2024-04-13T03:09:56.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.610+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d415ade0 con 0x7f74d415ca40 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.611+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d4069120 con 0x7f74d415ca40 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.611+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74c4038020 con 0x7f74d415ca40 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.611+0000 7f74caffd640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 0x7f74b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.612+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f74c40adb40 con 0x7f74d415ca40 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.612+0000 7f74d259c640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 0x7f74b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.612+0000 7f74d259c640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 0x7f74b406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0012e0 tx=0x7f74cc001700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.616+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74c403f070 con 0x7f74d415ca40 2024-04-13T03:09:56.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.828+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74d4071860 con 0x7f74d415ca40 2024-04-13T03:09:56.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.828+0000 7f74caffd640 1 -- 172.21.15.106:0/2640887131 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f74c407b5e0 con 0x7f74d415ca40 2024-04-13T03:09:56.829 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:56.830 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:56.830 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 msgr2=0x7f74b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 0x7f74b406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0012e0 tx=0x7f74cc001700 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 msgr2=0x7f74d41e8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009890 tx=0x7f74c40046c0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 shutdown_connections 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.830+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b406c8f0 0x7f74b406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d415ca40 0x7f74d41e8250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74d415bf50 0x7f74d406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 --2- 172.21.15.106:0/2640887131 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74d40cf5e0 0x7f74d4140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 >> 172.21.15.106:0/2640887131 conn(0x7f74d4072ea0 msgr2=0x7f74d40b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 shutdown_connections 2024-04-13T03:09:56.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.831+0000 7f74d88b7640 1 -- 172.21.15.106:0/2640887131 wait complete. 2024-04-13T03:09:56.843 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:09:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.941+0000 7f1f871cc640 1 Processor -- start 2024-04-13T03:09:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.941+0000 7f1f871cc640 1 -- start start 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f871cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1f800734c0 0x7f1f80169600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f871cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f800ccd00 con 0x7f1f8015eeb0 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f871cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f800ccfe0 con 0x7f1f8015c3f0 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f871cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8006ec10 con 0x7f1f800734c0 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f7ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1f8015eeb0 0x7f1f80072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55014/0 (socket says 172.21.15.106:55014) 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f7ffff640 1 -- 172.21.15.106:0/3701917264 learned_addr learned my addr 172.21.15.106:0/3701917264 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.942+0000 7f1f85769640 1 --2- 172.21.15.106:0/3701917264 >> v2:172.21.15.114:3300/0 conn(0x7f1f800734c0 0x7f1f80169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498404030 0 0) 0x7f1f800ccd00 con 0x7f1f8015eeb0 2024-04-13T03:09:56.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f68003520 con 0x7f1f8015eeb0 2024-04-13T03:09:56.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f85769640 1 -- 172.21.15.106:0/3701917264 >> v1:172.21.15.114:6789/0 conn(0x7f1f8015c3f0 legacy=0x7f1f800cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f85769640 1 -- 172.21.15.106:0/3701917264 >> v1:172.21.15.106:6789/0 conn(0x7f1f8015eeb0 legacy=0x7f1f80072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3543397650 0 0) 0x7f1f68003520 con 0x7f1f8015eeb0 2024-04-13T03:09:56.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.943+0000 7f1f85769640 1 -- 172.21.15.106:0/3701917264 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f800cd2c0 con 0x7f1f800734c0 2024-04-13T03:09:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f85769640 1 --2- 172.21.15.106:0/3701917264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f800734c0 0x7f1f80169600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009d90 tx=0x7f1f7802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c16e212444ce574e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78039070 con 0x7f1f800734c0 2024-04-13T03:09:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7802c940 con 0x7f1f800734c0 2024-04-13T03:09:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f7f7fe640 1 -- 172.21.15.106:0/3701917264 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74001670 con 0x7f1f800734c0 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 -- 172.21.15.106:0/3701917264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f800734c0 msgr2=0x7f1f80169600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 --2- 172.21.15.106:0/3701917264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f800734c0 0x7f1f80169600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009d90 tx=0x7f1f7802be10 comp rx=0 tx=0).stop 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 -- 172.21.15.106:0/3701917264 shutdown_connections 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 --2- 172.21.15.106:0/3701917264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f800734c0 0x7f1f80169600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 -- 172.21.15.106:0/3701917264 >> 172.21.15.106:0/3701917264 conn(0x7f1f800cdef0 msgr2=0x7f1f800ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 -- 172.21.15.106:0/3701917264 shutdown_connections 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.944+0000 7f1f871cc640 1 -- 172.21.15.106:0/3701917264 wait complete. 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.945+0000 7f1f871cc640 1 Processor -- start 2024-04-13T03:09:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 -- start start 2024-04-13T03:09:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1f8015c3f0 0x7f1f800c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 0x7f1f800c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8015c7d0 con 0x7f1f800734c0 2024-04-13T03:09:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cd880 con 0x7f1f8015eeb0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f871cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8015c950 con 0x7f1f8015c3f0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.946+0000 7f1f84f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53126/0 (socket says 172.21.15.106:53126) 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 -- 172.21.15.106:0/1438805555 learned_addr learned my addr 172.21.15.106:0/1438805555 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f7ffff640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1f8015c3f0 0x7f1f800c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f85769640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 0x7f1f800c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 -- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1f8015c3f0 msgr2=0x7f1f800c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1f8015c3f0 0x7f1f800c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 -- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 msgr2=0x7f1f800c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 0x7f1f800c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 -- 172.21.15.106:0/1438805555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8015b340 con 0x7f1f800734c0 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f85769640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 0x7f1f800c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f84f68640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c450 tx=0x7f1f7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700052d0 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f70005470 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70012870 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8015ad80 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.947+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f80157380 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.948+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f80069120 con 0x7f1f800734c0 2024-04-13T03:09:56.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.948+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f70012a10 con 0x7f1f800734c0 2024-04-13T03:09:56.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.949+0000 7f1f7d7fa640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:56.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.949+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1f7008ba00 con 0x7f1f800734c0 2024-04-13T03:09:56.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.949+0000 7f1f7ffff640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:56.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.949+0000 7f1f7ffff640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006e030 tx=0x7f1f74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:56.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:56.953+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f70059b80 con 0x7f1f800734c0 2024-04-13T03:09:57.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.136+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f8006d8a0 con 0x7f1f800734c0 2024-04-13T03:09:57.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.137+0000 7f1f7d7fa640 1 -- 172.21.15.106:0/1438805555 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f1f70059480 con 0x7f1f800734c0 2024-04-13T03:09:57.139 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:09:31.000854+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 msgr2=0x7f1f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006e030 tx=0x7f1f74009040 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 msgr2=0x7f1f8015b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c450 tx=0x7f1f7000c910 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 shutdown_connections 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1f8015c3f0 0x7f1f800c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f8015eeb0 0x7f1f800c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 --2- 172.21.15.106:0/1438805555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f800734c0 0x7f1f8015b670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 >> 172.21.15.106:0/1438805555 conn(0x7f1f800cdef0 msgr2=0x7f1f8006e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:57.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 shutdown_connections 2024-04-13T03:09:57.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.140+0000 7f1f871cc640 1 -- 172.21.15.106:0/1438805555 wait complete. 2024-04-13T03:09:57.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:09:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.251+0000 7f846ea2d640 1 Processor -- start 2024-04-13T03:09:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.251+0000 7f846ea2d640 1 -- start start 2024-04-13T03:09:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846ea2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846ea2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84680cf5a0 0x7f84680cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846ea2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8468070470 con 0x7f8468070090 2024-04-13T03:09:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846ea2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f84680ceb50 con 0x7f84680cf5a0 2024-04-13T03:09:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846ea2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84680cf110 con 0x7f84680b0390 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f84677fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f846cfca640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84680cf5a0 0x7f84680cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f8467fff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8468070090 0x7f8468160570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:50500/0 (socket says 172.21.15.106:50500) 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f8467fff640 1 -- 172.21.15.106:0/2741301608 learned_addr learned my addr 172.21.15.106:0/2741301608 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.252+0000 7f84677fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47350/0 (socket says 172.21.15.106:47350) 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f8466ffd640 1 -- 172.21.15.106:0/2741301608 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031616359 0 0) 0x7f8468070470 con 0x7f8468070090 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f8466ffd640 1 -- 172.21.15.106:0/2741301608 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8450003520 con 0x7f8468070090 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f846cfca640 1 -- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 msgr2=0x7f8468160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f846cfca640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f846cfca640 1 -- 172.21.15.106:0/2741301608 >> v1:172.21.15.114:6789/0 conn(0x7f8468070090 legacy=0x7f8468160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f846cfca640 1 -- 172.21.15.106:0/2741301608 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846815da00 con 0x7f84680cf5a0 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.253+0000 7f84677fe640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:09:57.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846cfca640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f84680cfa30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d90 tx=0x7f846002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33abe662e695f9eb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:57.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f8466ffd640 1 -- 172.21.15.106:0/2741301608 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460039070 con 0x7f84680cf5a0 2024-04-13T03:09:57.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f8466ffd640 1 -- 172.21.15.106:0/2741301608 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846002cbc0 con 0x7f84680cf5a0 2024-04-13T03:09:57.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f8466ffd640 1 -- 172.21.15.106:0/2741301608 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460034440 con 0x7f84680cf5a0 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 -- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 msgr2=0x7f84680cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f84680cfa30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d90 tx=0x7f846002be10 comp rx=0 tx=0).stop 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 -- 172.21.15.106:0/2741301608 shutdown_connections 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f84680cfa30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 --2- 172.21.15.106:0/2741301608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f8468160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 -- 172.21.15.106:0/2741301608 >> 172.21.15.106:0/2741301608 conn(0x7f84680cd6d0 msgr2=0x7f84680cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 -- 172.21.15.106:0/2741301608 shutdown_connections 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.254+0000 7f846ea2d640 1 -- 172.21.15.106:0/2741301608 wait complete. 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.255+0000 7f846ea2d640 1 Processor -- start 2024-04-13T03:09:57.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 -- start start 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f84681e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f846809a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f846806e560 con 0x7f8468070090 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84680ceda0 con 0x7f84680cf5a0 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f846ea2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84680cd220 con 0x7f84680b0390 2024-04-13T03:09:57.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f8467fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f8467fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53136/0 (socket says 172.21.15.106:53136) 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.256+0000 7f8467fff640 1 -- 172.21.15.106:0/361052493 learned_addr learned my addr 172.21.15.106:0/361052493 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f846cfca640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f846809a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f84677fe640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f84681e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 -- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 msgr2=0x7f84681e6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f84681e6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 -- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 msgr2=0x7f846809a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f846809a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 -- 172.21.15.106:0/361052493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846809aec0 con 0x7f8468070090 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f846cfca640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f846809a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8467fff640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f845800c390 tx=0x7f845800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458017070 con 0x7f8468070090 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8458005ce0 con 0x7f8468070090 2024-04-13T03:09:57.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846809b160 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580072f0 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.257+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846809b720 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.258+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8468068a10 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.259+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f845801a070 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.259+0000 7f8464ff9640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.259+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f845808b2b0 con 0x7f8468070090 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.259+0000 7f84677fe640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.259+0000 7f84677fe640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 0x7f845406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84681520d0 tx=0x7f845c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:57.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.263+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84580593a0 con 0x7f8468070090 2024-04-13T03:09:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.475+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84680d0840 con 0x7f8468070090 2024-04-13T03:09:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.475+0000 7f8464ff9640 1 -- 172.21.15.106:0/361052493 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f8458058ca0 con 0x7f8468070090 2024-04-13T03:09:57.476 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:09:57.477 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:09:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:09:57.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 msgr2=0x7f845406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 0x7f845406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84681520d0 tx=0x7f845c008040 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 msgr2=0x7f846809c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f845800c390 tx=0x7f845800c850 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 shutdown_connections 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84680b0390 0x7f84681e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84680cf5a0 0x7f846809a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 --2- 172.21.15.106:0/361052493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8468070090 0x7f846809c200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.477+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 >> 172.21.15.106:0/361052493 conn(0x7f84680cd6d0 msgr2=0x7f846806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.478+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 shutdown_connections 2024-04-13T03:09:57.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.478+0000 7f846ea2d640 1 -- 172.21.15.106:0/361052493 wait complete. 2024-04-13T03:09:57.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:09:24.350589+0000', 'modified': '2024-04-13T03:09:46.396950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6835/4025109601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 4025109601}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 4025109601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-04-13T03:09:57.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:09:57.892 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:09:57.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 Processor -- start 2024-04-13T03:09:57.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 -- start start 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f690415e930 0x7f69040b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 0x7f69040cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690406f200 con 0x7f690406ed70 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f690406e280 con 0x7f690415e930 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.994+0000 7f6908ade640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f690406e560 con 0x7f69040d09f0 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f690415e930 0x7f69040b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f690415e930 0x7f69040b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53142/0 (socket says 172.21.15.106:53142) 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 -- 172.21.15.106:0/2428397410 learned_addr learned my addr 172.21.15.106:0/2428397410 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f6902d9d640 1 --2- 172.21.15.106:0/2428397410 >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 0x7f69040cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 -- 172.21.15.106:0/2428397410 >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 msgr2=0x7f69040cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 --2- 172.21.15.106:0/2428397410 >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 0x7f69040cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 -- 172.21.15.106:0/2428397410 >> v1:172.21.15.114:6789/0 conn(0x7f690406ed70 legacy=0x7f69040d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f690259c640 1 -- 172.21.15.106:0/2428397410 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69040cccf0 con 0x7f690415e930 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.995+0000 7f6902d9d640 1 --2- 172.21.15.106:0/2428397410 >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 0x7f69040cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f690259c640 1 --2- 172.21.15.106:0/2428397410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040b5350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69040c9560 tx=0x7f68fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa6610b97c1f0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f690159a640 1 -- 172.21.15.106:0/2428397410 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc035020 con 0x7f690415e930 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f690159a640 1 -- 172.21.15.106:0/2428397410 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc031870 con 0x7f690415e930 2024-04-13T03:09:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f690159a640 1 -- 172.21.15.106:0/2428397410 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc031c40 con 0x7f690415e930 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f6908ade640 1 -- 172.21.15.106:0/2428397410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 msgr2=0x7f69040b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.996+0000 7f6908ade640 1 --2- 172.21.15.106:0/2428397410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040b5350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69040c9560 tx=0x7f68fc02b910 comp rx=0 tx=0).stop 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 -- 172.21.15.106:0/2428397410 shutdown_connections 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 --2- 172.21.15.106:0/2428397410 >> v2:172.21.15.114:3300/0 conn(0x7f69040d09f0 0x7f69040cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 --2- 172.21.15.106:0/2428397410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040b5350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 -- 172.21.15.106:0/2428397410 >> 172.21.15.106:0/2428397410 conn(0x7f69040cef80 msgr2=0x7f69040cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 -- 172.21.15.106:0/2428397410 shutdown_connections 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.997+0000 7f6908ade640 1 -- 172.21.15.106:0/2428397410 wait complete. 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.998+0000 7f6908ade640 1 Processor -- start 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.998+0000 7f6908ade640 1 -- start start 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 0x7f69040cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69040d09f0 0x7f69040c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69040ce830 con 0x7f690415e930 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69040cd2b0 con 0x7f690406ed70 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f69040ce9b0 con 0x7f69040d09f0 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53152/0 (socket says 172.21.15.106:53152) 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 -- 172.21.15.106:0/2338557427 learned_addr learned my addr 172.21.15.106:0/2338557427 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f690259c640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 0x7f69040cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6901d9b640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69040d09f0 0x7f69040c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 -- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69040d09f0 msgr2=0x7f69040c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69040d09f0 0x7f69040c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 -- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 msgr2=0x7f69040cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 0x7f69040cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 -- 172.21.15.106:0/2338557427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69041f4f50 con 0x7f690415e930 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f690259c640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 0x7f69040cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6902d9d640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c7f0 tx=0x7f68f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f68f2ffd640 1 -- 172.21.15.106:0/2338557427 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4015070 con 0x7f690415e930 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f68f2ffd640 1 -- 172.21.15.106:0/2338557427 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4007820 con 0x7f690415e930 2024-04-13T03:09:58.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690415fa00 con 0x7f690415e930 2024-04-13T03:09:58.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f68f2ffd640 1 -- 172.21.15.106:0/2338557427 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4007c50 con 0x7f690415e930 2024-04-13T03:09:58.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:57.999+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690415ffc0 con 0x7f690415e930 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.001+0000 7f68f2ffd640 1 -- 172.21.15.106:0/2338557427 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68f4018070 con 0x7f690415e930 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.001+0000 7f68f2ffd640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 0x7f68e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.001+0000 7f68f2ffd640 1 -- 172.21.15.106:0/2338557427 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68f408c000 con 0x7f690415e930 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.001+0000 7f690259c640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 0x7f68e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.002+0000 7f690259c640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 0x7f68e406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68fc036000 tx=0x7f68fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.002+0000 7f690b9ba740 1 Processor -- start 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.002+0000 7f690b9ba740 1 -- start start 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645deffad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 0x5645df02ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5645def65990 con 0x5645deffb2c0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645def0c620 con 0x5645df027de0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5645def61950 con 0x5645df0281c0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53154/0 (socket says 172.21.15.106:53154) 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 -- 172.21.15.106:0/3349107899 learned_addr learned my addr 172.21.15.106:0/3349107899 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f690259c640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645deffad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 0x5645df02ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 -- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 msgr2=0x5645df02ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 0x5645df02ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 -- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 msgr2=0x5645deffad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645deffad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6902d9d640 1 -- 172.21.15.106:0/3349107899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645dec782d0 con 0x5645deffb2c0 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.003+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 0x5645df02ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f6902d9d640 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68f408a190 tx=0x7f68f405de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7c8c60e1d8f62f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f68e3fff640 1 -- 172.21.15.106:0/3349107899 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f401f040 con 0x5645deffb2c0 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f68e3fff640 1 -- 172.21.15.106:0/3349107899 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4056060 con 0x5645deffb2c0 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f68e3fff640 1 -- 172.21.15.106:0/3349107899 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4060290 con 0x5645deffb2c0 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 -- 172.21.15.106:0/3349107899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 msgr2=0x5645df02b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68f408a190 tx=0x7f68f405de60 comp rx=0 tx=0).stop 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 -- 172.21.15.106:0/3349107899 shutdown_connections 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df0281c0 0x5645df02ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645deffb2c0 0x5645df02b0b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3349107899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645deffad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 -- 172.21.15.106:0/3349107899 >> 172.21.15.106:0/3349107899 conn(0x5645deffcff0 msgr2=0x5645df045b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 -- 172.21.15.106:0/3349107899 shutdown_connections 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.004+0000 7f690b9ba740 1 -- 172.21.15.106:0/3349107899 wait complete. 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.005+0000 7f690b9ba740 1 -- start start 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.005+0000 7f690b9ba740 1 Processor -- start 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df027de0 0x5645df025bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df02cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5645defff590 con 0x5645df031bc0 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645def1f9e0 con 0x5645df0281c0 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5645deffe980 con 0x5645df027de0 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690b9ba740 10 client.0 _ll_register_callbacks cb 0x5645def5b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:09:58.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53158/0 (socket says 172.21.15.106:53158) 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 -- 172.21.15.106:0/4185232667 learned_addr learned my addr 172.21.15.106:0/4185232667 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6901d9b640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df02cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f690259c640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df027de0 0x5645df025bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 -- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df027de0 msgr2=0x5645df025bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df027de0 0x5645df025bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 -- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 msgr2=0x5645df02cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df02cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.006+0000 7f6902d9d640 1 -- 172.21.15.106:0/4185232667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ded426b0 con 0x5645df031bc0 2024-04-13T03:09:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f6902d9d640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68f40596f0 tx=0x7f68f401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53639f56158cca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4088060 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68f4083b50 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f4056060 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f68f401f040 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 10 client.4869 fetch_fsmap learned FSMap version 9 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5645df02fc40 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4056060 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f68f4055740 con 0x5645df031bc0 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-04-13T03:09:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 10 client.4869 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 20 client.4869 populate_metadata read hostname 'smithi106' 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df059380 0x5645df059760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 4 client.4869 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 --> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645def733b0 con 0x5645df059380 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.007+0000 7f690259c640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df059380 0x5645df059760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.008+0000 7f68e17fa640 10 client.4869 ms_handle_connect on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.008+0000 7f690259c640 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df059380 0x5645df059760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc66d68f95b2d41 server_cookie=3927ecb3f3ab45f5 in_seq=0 out_seq=0 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.008+0000 7f68e17fa640 1 -- 172.21.15.106:0/4185232667 <== mds.0 v2:172.21.15.114:6834/4025109601 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645def733b0 con 0x5645df059380 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.008+0000 7f68e17fa640 10 client.4869 handle_command_reply: tid=0 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.008+0000 7f690b9ba740 1 client.4869 shutdown 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 msgr2=0x5645df032040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68f40596f0 tx=0x7f68f401c070 comp rx=0 tx=0).stop 2024-04-13T03:09:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 shutdown_connections 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df059380 0x5645df059760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df031bc0 0x5645df032040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df02cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 --2- 172.21.15.106:0/4185232667 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df027de0 0x5645df025bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f68e17fa640 0 client.4869 ms_handle_reset on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 >> 172.21.15.106:0/4185232667 conn(0x5645deffb2c0 msgr2=0x5645df02ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 shutdown_connections 2024-04-13T03:09:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 1 -- 172.21.15.106:0/4185232667 wait complete. 2024-04-13T03:09:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.009+0000 7f690b9ba740 20 client.4869 trim_cache size 0 max 16384 2024-04-13T03:09:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.027+0000 7f690b9ba740 1 Processor -- start 2024-04-13T03:09:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.027+0000 7f690b9ba740 1 -- start start 2024-04-13T03:09:58.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df058a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df02c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6904201290 con 0x5645df027de0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69042005f0 con 0x5645df0281c0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6904200c40 con 0x5645df031bc0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53164/0 (socket says 172.21.15.106:53164) 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 -- 172.21.15.106:0/3313346610 learned_addr learned my addr 172.21.15.106:0/3313346610 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 -- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 msgr2=0x5645df02c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f6902d9d640 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df02c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df058a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df02c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 -- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 msgr2=0x5645df058a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df058a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 -- 172.21.15.106:0/3313346610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645df0014f0 con 0x5645df027de0 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f690259c640 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0090b0 tx=0x7f68fc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bffc2d698ae502 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.028+0000 7f68e37fe640 1 -- 172.21.15.106:0/3313346610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc035040 con 0x5645df027de0 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f68e37fe640 1 -- 172.21.15.106:0/3313346610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc053640 con 0x5645df027de0 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f68e37fe640 1 -- 172.21.15.106:0/3313346610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc051410 con 0x5645df027de0 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 -- 172.21.15.106:0/3313346610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 msgr2=0x5645df058500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0090b0 tx=0x7f68fc030000 comp rx=0 tx=0).stop 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 -- 172.21.15.106:0/3313346610 shutdown_connections 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df02c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df0281c0 0x5645df058a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3313346610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df027de0 0x5645df058500 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 -- 172.21.15.106:0/3313346610 >> 172.21.15.106:0/3313346610 conn(0x5645deffb2c0 msgr2=0x5645df02ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 -- 172.21.15.106:0/3313346610 shutdown_connections 2024-04-13T03:09:58.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.029+0000 7f690b9ba740 1 -- 172.21.15.106:0/3313346610 wait complete. 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.030+0000 7f690b9ba740 1 -- start start 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.030+0000 7f690b9ba740 1 Processor -- start 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645df032210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df058a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5645defff590 con 0x5645df0281c0 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645deffe980 con 0x5645df027de0 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5645def1f9e0 con 0x5645df031bc0 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 10 client.0 _ll_register_callbacks cb 0x7f68f4072070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53168/0 (socket says 172.21.15.106:53168) 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 -- 172.21.15.106:0/3166250465 learned_addr learned my addr 172.21.15.106:0/3166250465 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6902d9d640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df058a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690259c640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645df032210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 -- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 msgr2=0x5645df058a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df058a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 -- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 msgr2=0x5645df032210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645df032210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 -- 172.21.15.106:0/3166250465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645df024920 con 0x5645df0281c0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f6901d9b640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68f8012540 tx=0x7f68f8012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4b28d88da45d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f68e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8014870 con 0x5645df0281c0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68f8014a10 con 0x5645df0281c0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68fc053640 con 0x5645df0281c0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f68f8025050 con 0x5645df0281c0 2024-04-13T03:09:58.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 10 client.4875 fetch_fsmap learned FSMap version 9 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.031+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5645df024ee0 con 0x5645df0281c0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc053640 con 0x5645df0281c0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f68f8028020 con 0x5645df0281c0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 10 client.4875 fetch_fsmap finished waiting for FSMap version 9 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 10 client.4875 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 20 client.4875 populate_metadata read hostname 'smithi106' 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df0c04b0 0x5645df0c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 4 client.4875 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 --> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] -- command(tid 0: {"prefix": "status"}) -- 0x5645def6d220 con 0x5645df0c04b0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690259c640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df0c04b0 0x5645df0c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f68e17fa640 10 client.4875 ms_handle_connect on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.032+0000 7f690259c640 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df0c04b0 0x5645df0c0890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7166257e8cb84ea1 server_cookie=46743735d5d9a4d4 in_seq=0 out_seq=0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f68e17fa640 1 -- 172.21.15.106:0/3166250465 <== mds.0 v2:172.21.15.114:6834/4025109601 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5645def6d220 con 0x5645df0c04b0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f68e17fa640 10 client.4875 handle_command_reply: tid=0 2024-04-13T03:09:58.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 client.4875 shutdown 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 msgr2=0x5645df058500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68f8012540 tx=0x7f68f8012a00 comp rx=0 tx=0).stop 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 shutdown_connections 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:6834/4025109601,v1:172.21.15.114:6835/4025109601] conn(0x5645df0c04b0 0x5645df0c0890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645df031bc0 0x5645df058a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645df0281c0 0x5645df058500 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 --2- 172.21.15.106:0/3166250465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645df027de0 0x5645df032210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f68e17fa640 0 client.4875 ms_handle_reset on v2:172.21.15.114:6834/4025109601 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 >> 172.21.15.106:0/3166250465 conn(0x5645deffb2c0 msgr2=0x5645df05fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 shutdown_connections 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 1 -- 172.21.15.106:0/3166250465 wait complete. 2024-04-13T03:09:58.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f690b9ba740 20 client.4875 trim_cache size 0 max 16384 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4293, 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 33.059905037, 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 9, 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 25, 2024-04-13T03:09:58.042 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 25, 2024-04-13T03:09:58.043 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 40.084030519999999 2024-04-13T03:09:58.043 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:09:58.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 msgr2=0x7f68e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 0x7f68e406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68fc036000 tx=0x7f68fc008030 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 msgr2=0x7f69040cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c7f0 tx=0x7f68f400ccb0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 shutdown_connections 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68e406c8f0 0x7f68e406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69040d09f0 0x7f69040c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690415e930 0x7f69040cb0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 --2- 172.21.15.106:0/2338557427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690406ed70 0x7f69040cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 >> 172.21.15.106:0/2338557427 conn(0x7f69040cef80 msgr2=0x7f69040b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 shutdown_connections 2024-04-13T03:09:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.033+0000 7f6908ade640 1 -- 172.21.15.106:0/2338557427 wait complete. 2024-04-13T03:09:58.046 INFO:teuthology.run_tasks:Running task sequential... 2024-04-13T03:09:58.055 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-13T03:09:58.056 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-13T03:09:58.056 INFO:teuthology.task.print:Enabling mgr modules 2024-04-13T03:09:58.056 INFO:teuthology.run_tasks:Running task kclient... 2024-04-13T03:09:58.080 INFO:tasks.kclient:Mounting kernel clients... 2024-04-13T03:09:58.080 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-13T03:09:58.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.081 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:09:58.095 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-13T03:09:58.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T03:09:58.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-13T03:09:58.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.259 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:09:58.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.274 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-13T03:09:58.342 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2024-04-13T03:09:58.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:09:58.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.344 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:09:58.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-04-13T03:09:58.436 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2024-04-13T03:09:58.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:09:58.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.437 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:09:58.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-04-13T03:09:58.564 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2024-04-13T03:09:58.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:09:58.566 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-13T03:09:58.566 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T03:09:58.567 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:09:58.567 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:09:58.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.667+0000 7f1e27515640 1 Processor -- start 2024-04-13T03:09:58.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.667+0000 7f1e27515640 1 -- start start 2024-04-13T03:09:58.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e27515640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1e2015b270 0x7f1e2015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e27515640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e20158c50 con 0x7f1e200b32d0 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e27515640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e200b3800 con 0x7f1e20158870 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e27515640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2015ea90 con 0x7f1e2015b270 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e24ab0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1e200b32d0 0x7f1e2015ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55026/0 (socket says 172.21.15.106:55026) 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e24ab0640 1 -- 172.21.15.106:0/2311313678 learned_addr learned my addr 172.21.15.106:0/2311313678 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e25ab2640 1 --2- 172.21.15.106:0/2311313678 >> v2:172.21.15.114:3300/0 conn(0x7f1e2015b270 0x7f1e2015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576168190 0 0) 0x7f1e20158c50 con 0x7f1e200b32d0 2024-04-13T03:09:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e0c003520 con 0x7f1e200b32d0 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1995185405 0 0) 0x7f1e0c003520 con 0x7f1e200b32d0 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 >> v2:172.21.15.114:3300/0 conn(0x7f1e2015b270 msgr2=0x7f1e2015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 --2- 172.21.15.106:0/2311313678 >> v2:172.21.15.114:3300/0 conn(0x7f1e2015b270 0x7f1e2015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 >> v1:172.21.15.114:6789/0 conn(0x7f1e20158870 legacy=0x7f1e2006faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e200b3ae0 con 0x7f1e200b32d0 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f1e10001670 con 0x7f1e200b32d0 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 >> v1:172.21.15.106:6789/0 conn(0x7f1e200b32d0 legacy=0x7f1e2015ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.668+0000 7f1e17fff640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e17fff640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e0c004800 0x7f1e0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e17fff640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0051c0 0x7f1e0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2015ea90 con 0x7f1e0c0039b0 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c00c060 con 0x7f1e0c0051c0 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c00c260 con 0x7f1e0c004800 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e25ab2640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0051c0 0x7f1e0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e24ab0640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e0c004800 0x7f1e0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 -- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e0c004800 msgr2=0x7f1e0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e0c004800 0x7f1e0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 -- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0051c0 msgr2=0x7f1e0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:09:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0051c0 0x7f1e0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e25ab2640 1 -- 172.21.15.106:0/2311313678 reap_dead start 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.669+0000 7f1e252b1640 1 -- 172.21.15.106:0/2311313678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c003520 con 0x7f1e0c0039b0 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e252b1640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c001750 tx=0x7f1e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fe6c6523065ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c001370 con 0x7f1e0c0039b0 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0052d0 con 0x7f1e0c0039b0 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e17fff640 1 -- 172.21.15.106:0/2311313678 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003830 con 0x7f1e0c0039b0 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 -- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 msgr2=0x7f1e0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c001750 tx=0x7f1e0c005830 comp rx=0 tx=0).stop 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 -- 172.21.15.106:0/2311313678 shutdown_connections 2024-04-13T03:09:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 --2- 172.21.15.106:0/2311313678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 -- 172.21.15.106:0/2311313678 >> 172.21.15.106:0/2311313678 conn(0x7f1e200d0340 msgr2=0x7f1e200d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 -- 172.21.15.106:0/2311313678 shutdown_connections 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.670+0000 7f1e27515640 1 -- 172.21.15.106:0/2311313678 wait complete. 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 Processor -- start 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 -- start start 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e200cfe10 0x7f1e20145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 0x7f1e2006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2015b870 con 0x7f1e0c0039b0 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2015b6f0 con 0x7f1e2014f1b0 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e27515640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2015b9f0 con 0x7f1e200cfe10 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e252b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e252b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53184/0 (socket says 172.21.15.106:53184) 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e252b1640 1 -- 172.21.15.106:0/3968048641 learned_addr learned my addr 172.21.15.106:0/3968048641 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e24ab0640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e200cfe10 0x7f1e20145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.672+0000 7f1e25ab2640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 0x7f1e2006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 -- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e200cfe10 msgr2=0x7f1e20145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e200cfe10 0x7f1e20145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 -- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 msgr2=0x7f1e2006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 0x7f1e2006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 -- 172.21.15.106:0/3968048641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e20146140 con 0x7f1e0c0039b0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e25ab2640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 0x7f1e2006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e252b1640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034040 tx=0x7f1e1c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c041070 con 0x7f1e0c0039b0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c005480 con 0x7f1e0c0039b0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e20150690 con 0x7f1e0c0039b0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c007a30 con 0x7f1e0c0039b0 2024-04-13T03:09:58.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.673+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e200cf330 con 0x7f1e0c0039b0 2024-04-13T03:09:58.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.674+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e200ccf50 con 0x7f1e0c0039b0 2024-04-13T03:09:58.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.675+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e1c044070 con 0x7f1e0c0039b0 2024-04-13T03:09:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.675+0000 7f1e15ffb640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 0x7f1e0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:09:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.675+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1e1c0ad820 con 0x7f1e0c0039b0 2024-04-13T03:09:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.675+0000 7f1e24ab0640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 0x7f1e0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:09:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.675+0000 7f1e24ab0640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 0x7f1e0406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b1cf0 tx=0x7f1e100078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:09:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.679+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e200ccf50 con 0x7f1e0c0039b0 2024-04-13T03:09:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.890+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e20068a10 con 0x7f1e0c0039b0 2024-04-13T03:09:58.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.890+0000 7f1e15ffb640 1 -- 172.21.15.106:0/3968048641 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1e1c07b2c0 con 0x7f1e0c0039b0 2024-04-13T03:09:58.892 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T03:09:58.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 msgr2=0x7f1e0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 0x7f1e0406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b1cf0 tx=0x7f1e100078f0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 msgr2=0x7f1e20150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034040 tx=0x7f1e1c001120 comp rx=0 tx=0).stop 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 shutdown_connections 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1e0406c8f0 0x7f1e0406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1e200cfe10 0x7f1e20145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e2014f1b0 0x7f1e2006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 --2- 172.21.15.106:0/3968048641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c0039b0 0x7f1e20150040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:09:58.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 >> 172.21.15.106:0/3968048641 conn(0x7f1e200d0340 msgr2=0x7f1e2006ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:09:58.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 shutdown_connections 2024-04-13T03:09:58.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:58.892+0000 7f1e27515640 1 -- 172.21.15.106:0/3968048641 wait complete. 2024-04-13T03:09:58.904 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-13T03:09:58.904 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:09:58.904 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:09:58.904 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:09:58.904 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:09:58.905 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-13T03:09:58.905 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T03:09:58.905 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:09:58.905 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:09:58.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:58.905 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:58.920 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42550sec preferred_lft 42550sec 2024-04-13T03:09:58.921 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:09:58.922 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:09:58.922 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-13T03:09:58.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-13T03:09:58.923 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:59.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:59.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.068 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:09:59.094 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:09:59.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.096 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:09:59.151 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:09:59.152 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:09:59.152 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:09:59.152 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:09:59.152 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:09:59.152 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:59.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:59.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:59.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.287 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:09:59.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.302 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-13T03:09:59.356 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:59.356 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:59.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:09:59.357 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-13T03:09:59.357 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:59.478 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-13T03:09:59.478 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-13T03:09:59.479 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:59.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:59.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:59.603 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:09:59.603 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:09:59.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:09:59.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-13T03:09:59.603 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:09:59.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:09:59.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:09:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:09:59.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:09:59.684 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:09:59.698 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:09:59.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:09:59.698 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:09:59.751 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:09:59.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:09:59.836 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:09:59.836 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:09:59.940 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:09:59.940 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:09:59.940 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:09:59.940 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:09:59.941 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:09:59.943 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:09:59.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:09:59.943 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:09:59.969 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:00.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:00.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:00.065 INFO:teuthology.orchestra.run.smithi106.stdout:client4884 2024-04-13T03:10:00.065 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:00.065 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:00.080 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:00.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:00.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:00.176 INFO:teuthology.orchestra.run.smithi106.stdout:client4884 2024-04-13T03:10:00.177 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:00.232 INFO:teuthology.orchestra.run.smithi106.stdout:client4884 2024-04-13T03:10:00.232 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7ff0368-2781-4238-b477-50485b08a223.client4884/status') 2024-04-13T03:10:00.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:00.294 INFO:teuthology.orchestra.run.smithi106.stdout:instance: client.4884 (3)192.168.0.1:0/2081358964 2024-04-13T03:10:00.294 INFO:teuthology.orchestra.run.smithi106.stdout:blocklisted: false 2024-04-13T03:10:00.294 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2024-04-13T03:10:00.294 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2024-04-13T03:10:00.294 INFO:teuthology.orchestra.run.smithi106.stderr:69 bytes copied, 2.0248e-05 s, 3.4 MB/s 2024-04-13T03:10:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:00.301 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-13T03:10:00.301 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T03:10:00.302 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-13T03:10:00.302 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:10:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.404+0000 7f69e8b1e640 1 Processor -- start 2024-04-13T03:10:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 -- start start 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f69e41591a0 0x7f69e415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f69e40b5320 0x7f69e406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e4159580 con 0x7f69e40b32d0 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f69e40b3ab0 con 0x7f69e41591a0 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.405+0000 7f69e8b1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f69e40b3d90 con 0x7f69e40b5320 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f69e41591a0 0x7f69e415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f69e41591a0 0x7f69e415bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53190/0 (socket says 172.21.15.106:53190) 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 -- 172.21.15.106:0/324012504 learned_addr learned my addr 172.21.15.106:0/324012504 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e2d9d640 1 --2- 172.21.15.106:0/324012504 >> v2:172.21.15.114:3300/0 conn(0x7f69e40b5320 0x7f69e406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 -- 172.21.15.106:0/324012504 >> v2:172.21.15.114:3300/0 conn(0x7f69e40b5320 msgr2=0x7f69e406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 --2- 172.21.15.106:0/324012504 >> v2:172.21.15.114:3300/0 conn(0x7f69e40b5320 0x7f69e406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 -- 172.21.15.106:0/324012504 >> v1:172.21.15.114:6789/0 conn(0x7f69e40b32d0 legacy=0x7f69e40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.406+0000 7f69e259c640 1 -- 172.21.15.106:0/324012504 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40b4070 con 0x7f69e41591a0 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e259c640 1 --2- 172.21.15.106:0/324012504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e41591a0 0x7f69e415bbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69e406fe80 tx=0x7f69d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b524ad7221656e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e159a640 1 -- 172.21.15.106:0/324012504 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4034070 con 0x7f69e41591a0 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e159a640 1 -- 172.21.15.106:0/324012504 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d4030860 con 0x7f69e41591a0 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e159a640 1 -- 172.21.15.106:0/324012504 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4030c30 con 0x7f69e41591a0 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 -- 172.21.15.106:0/324012504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e41591a0 msgr2=0x7f69e415bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/324012504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e41591a0 0x7f69e415bbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69e406fe80 tx=0x7f69d402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 -- 172.21.15.106:0/324012504 shutdown_connections 2024-04-13T03:10:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/324012504 >> v2:172.21.15.114:3300/0 conn(0x7f69e40b5320 0x7f69e406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/324012504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e41591a0 0x7f69e415bbf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.407+0000 7f69e8b1e640 1 -- 172.21.15.106:0/324012504 >> 172.21.15.106:0/324012504 conn(0x7f69e40d0340 msgr2=0x7f69e40d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.408+0000 7f69e8b1e640 1 -- 172.21.15.106:0/324012504 shutdown_connections 2024-04-13T03:10:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.408+0000 7f69e8b1e640 1 -- 172.21.15.106:0/324012504 wait complete. 2024-04-13T03:10:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.409+0000 7f69e8b1e640 1 Processor -- start 2024-04-13T03:10:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.409+0000 7f69e8b1e640 1 -- start start 2024-04-13T03:10:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.409+0000 7f69e8b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.409+0000 7f69e8b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 0x7f69e4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.409+0000 7f69e8b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 0x7f69e4157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e8b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b3760 con 0x7f69e40b32d0 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e8b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b4630 con 0x7f69e40b5320 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e8b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f69e406e310 con 0x7f69e41591a0 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53194/0 (socket says 172.21.15.106:53194) 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 -- 172.21.15.106:0/2557443081 learned_addr learned my addr 172.21.15.106:0/2557443081 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e2d9d640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 0x7f69e4157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e1d9b640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 0x7f69e4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 -- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 msgr2=0x7f69e4157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 0x7f69e4157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 -- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 msgr2=0x7f69e4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 0x7f69e4156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 -- 172.21.15.106:0/2557443081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e41581d0 con 0x7f69e40b32d0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e2d9d640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 0x7f69e4157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e1d9b640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 0x7f69e4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e259c640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4034040 con 0x7f69e40b32d0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d4039d90 con 0x7f69e40b32d0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4043610 con 0x7f69e40b32d0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41e6a60 con 0x7f69e40b32d0 2024-04-13T03:10:00.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.410+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e41e7020 con 0x7f69e40b32d0 2024-04-13T03:10:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.411+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e4068a10 con 0x7f69e40b32d0 2024-04-13T03:10:00.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.412+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69d4037070 con 0x7f69e40b32d0 2024-04-13T03:10:00.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.412+0000 7f69d2ffd640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 0x7f69c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:00.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.412+0000 7f69e1d9b640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 0x7f69c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:00.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.412+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f69d40ae000 con 0x7f69e40b32d0 2024-04-13T03:10:00.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.413+0000 7f69e1d9b640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 0x7f69c406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f69e415af80 tx=0x7f69d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.417+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69d403e070 con 0x7f69e40b32d0 2024-04-13T03:10:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.638+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f69e40d10b0 con 0x7f69e40b32d0 2024-04-13T03:10:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.639+0000 7f69d2ffd640 1 -- 172.21.15.106:0/2557443081 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f69d407c0f0 con 0x7f69e40b32d0 2024-04-13T03:10:00.640 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.640+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 msgr2=0x7f69c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.640+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 0x7f69c406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f69e415af80 tx=0x7f69d8009040 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.640+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 msgr2=0x7f69e40ab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.640+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d40046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 shutdown_connections 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69c406c8f0 0x7f69c406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69e41591a0 0x7f69e4157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69e40b5320 0x7f69e4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 --2- 172.21.15.106:0/2557443081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69e40b32d0 0x7f69e40ab2d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 >> 172.21.15.106:0/2557443081 conn(0x7f69e40d0340 msgr2=0x7f69e40ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 shutdown_connections 2024-04-13T03:10:00.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00.641+0000 7f69e8b1e640 1 -- 172.21.15.106:0/2557443081 wait complete. 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:10:00.653 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-13T03:10:00.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:00.653 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:10:00.668 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:00.668 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:00.668 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:00.668 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.668 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-04-13T03:10:00.669 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link tentative 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link tentative 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:00.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:00.671 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:10:00.735 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:10:00.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:00.736 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:10:00.792 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:10:00.792 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:10:00.792 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:10:00.792 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:10:00.792 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:10:00.793 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:10:00.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:00.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:00.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:00.927 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:10:00.942 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:10:00.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:00.942 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-13T03:10:00.997 INFO:teuthology.orchestra.run.smithi106.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-13T03:10:00.998 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:00.998 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:10:00.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:10:00.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-13T03:10:00.998 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:10:01.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:01.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:01.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-13T03:10:01.145 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:01.145 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:01.145 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:01.145 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9e:9a:42:3d:6f:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9c9a:42ff:fe3d:6f45/64 scope link 2024-04-13T03:10:01.146 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:01.147 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-13T03:10:01.214 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:01.214 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:01.214 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:01.214 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9e:9a:42:3d:6f:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9c9a:42ff:fe3d:6f45/64 scope link 2024-04-13T03:10:01.215 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:01.216 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-13T03:10:01.217 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:10:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:01.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.382 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:01.383 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:10:01.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.1 up 2024-04-13T03:10:01.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.1 master ceph-brx 2024-04-13T03:10:01.383 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:10:01.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:01.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:01.463 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-13T03:10:01.478 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-13T03:10:01.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:01.478 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-13T03:10:01.532 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-13T03:10:01.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:01.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:01.617 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:01.617 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:01.625 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:01.625 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:01.625 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:10:01.625 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 2@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:10:01.626 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:10:01.627 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:10:01.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:01.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-13T03:10:01.659 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:01.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:01.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.754 INFO:teuthology.orchestra.run.smithi106.stdout:client4893 2024-04-13T03:10:01.755 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:01.755 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:01.770 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:01.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.866 INFO:teuthology.orchestra.run.smithi106.stdout:client4893 2024-04-13T03:10:01.866 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:01.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:01.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.921 INFO:teuthology.orchestra.run.smithi106.stdout:client4893 2024-04-13T03:10:01.921 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7ff0368-2781-4238-b477-50485b08a223.client4893/status') 2024-04-13T03:10:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:01.981 INFO:teuthology.orchestra.run.smithi106.stdout:instance: client.4893 (3)192.168.0.2:0/3287276388 2024-04-13T03:10:01.982 INFO:teuthology.orchestra.run.smithi106.stdout:blocklisted: false 2024-04-13T03:10:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2024-04-13T03:10:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2024-04-13T03:10:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:69 bytes copied, 2.0923e-05 s, 3.3 MB/s 2024-04-13T03:10:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:01.988 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-13T03:10:01.988 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T03:10:01.989 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-13T03:10:01.989 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:10:02.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.087+0000 7f808ede9640 1 Processor -- start 2024-04-13T03:10:02.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.087+0000 7f808ede9640 1 -- start start 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808ede9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80880b53c0 0x7f80880b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808ede9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f80880d1460 0x7f80880ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808ede9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880b61a0 con 0x7f80880c9140 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808ede9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d0fd0 con 0x7f80880b53c0 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808ede9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d0100 con 0x7f80880d1460 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80880b53c0 0x7f80880b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80880b53c0 0x7f80880b7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53208/0 (socket says 172.21.15.106:53208) 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 -- 172.21.15.106:0/1628923028 learned_addr learned my addr 172.21.15.106:0/1628923028 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808d386640 1 --2- 172.21.15.106:0/1628923028 >> v2:172.21.15.114:3300/0 conn(0x7f80880d1460 0x7f80880ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 -- 172.21.15.106:0/1628923028 >> v2:172.21.15.114:3300/0 conn(0x7f80880d1460 msgr2=0x7f80880ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 --2- 172.21.15.106:0/1628923028 >> v2:172.21.15.114:3300/0 conn(0x7f80880d1460 0x7f80880ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 -- 172.21.15.106:0/1628923028 >> v1:172.21.15.114:6789/0 conn(0x7f80880c9140 legacy=0x7f80880b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.088+0000 7f808cb85640 1 -- 172.21.15.106:0/1628923028 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880d03c0 con 0x7f80880b53c0 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808cb85640 1 --2- 172.21.15.106:0/1628923028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80880b7630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ed65e107c500f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f80877fe640 1 -- 172.21.15.106:0/1628923028 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c035020 con 0x7f80880b53c0 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f80877fe640 1 -- 172.21.15.106:0/1628923028 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f807c031870 con 0x7f80880b53c0 2024-04-13T03:10:02.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f80877fe640 1 -- 172.21.15.106:0/1628923028 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c031c40 con 0x7f80880b53c0 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 -- 172.21.15.106:0/1628923028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 msgr2=0x7f80880b7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 --2- 172.21.15.106:0/1628923028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80880b7630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c02b910 comp rx=0 tx=0).stop 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 -- 172.21.15.106:0/1628923028 shutdown_connections 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 --2- 172.21.15.106:0/1628923028 >> v2:172.21.15.114:3300/0 conn(0x7f80880d1460 0x7f80880ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 --2- 172.21.15.106:0/1628923028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80880b7630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.089+0000 7f808ede9640 1 -- 172.21.15.106:0/1628923028 >> 172.21.15.106:0/1628923028 conn(0x7f80880d2530 msgr2=0x7f80880d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.090+0000 7f808ede9640 1 -- 172.21.15.106:0/1628923028 shutdown_connections 2024-04-13T03:10:02.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.090+0000 7f808ede9640 1 -- 172.21.15.106:0/1628923028 wait complete. 2024-04-13T03:10:02.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.091+0000 7f808ede9640 1 Processor -- start 2024-04-13T03:10:02.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.091+0000 7f808ede9640 1 -- start start 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 0x7f8088158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 0x7f8088155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80880d0c60 con 0x7f80880b53c0 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f80880d0980 con 0x7f80880d1460 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f808815edb0 con 0x7f80880c9140 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53210/0 (socket says 172.21.15.106:53210) 2024-04-13T03:10:02.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 -- 172.21.15.106:0/4062262731 learned_addr learned my addr 172.21.15.106:0/4062262731 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f8087fff640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 0x7f8088158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808d386640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 0x7f8088155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 -- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 msgr2=0x7f8088158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 0x7f8088158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 -- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 msgr2=0x7f8088155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 0x7f8088155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 -- 172.21.15.106:0/4062262731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088158650 con 0x7f80880b53c0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808d386640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 0x7f8088155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f8087fff640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 0x7f8088158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808cb85640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c040070 con 0x7f80880b53c0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f807c030d00 con 0x7f80880b53c0 2024-04-13T03:10:02.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c035040 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80881e9b50 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.092+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881ea110 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.093+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8088068a10 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.094+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f807c04b060 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.094+0000 7f80857fa640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.094+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f807c0adc30 con 0x7f80880b53c0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.094+0000 7f8087fff640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.095+0000 7f8087fff640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 0x7f806c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80880ce020 tx=0x7f8074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.099+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f807c02f320 con 0x7f80880b53c0 2024-04-13T03:10:02.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.312+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80880b7800 con 0x7f80880b53c0 2024-04-13T03:10:02.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.312+0000 7f80857fa640 1 -- 172.21.15.106:0/4062262731 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f807c07bdb0 con 0x7f80880b53c0 2024-04-13T03:10:02.315 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.314+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 msgr2=0x7f806c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.314+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 0x7f806c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80880ce020 tx=0x7f8074009040 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.314+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 msgr2=0x7f80881e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.314+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 shutdown_connections 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f806c06c8f0 0x7f806c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f80880c9140 0x7f8088158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80880d1460 0x7f8088155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 --2- 172.21.15.106:0/4062262731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80880b53c0 0x7f80881e9570 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 >> 172.21.15.106:0/4062262731 conn(0x7f80880d2530 msgr2=0x7f80880c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 shutdown_connections 2024-04-13T03:10:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:02.315+0000 7f808ede9640 1 -- 172.21.15.106:0/4062262731 wait complete. 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-13T03:10:02.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T03:10:02.328 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:10:02.328 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-13T03:10:02.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:02.328 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-04-13T03:10:02.346 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42657sec preferred_lft 42657sec 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link 2024-04-13T03:10:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:02.348 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-13T03:10:02.348 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:02.348 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:02.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-04-13T03:10:02.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-04-13T03:10:02.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-04-13T03:10:02.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-13T03:10:02.349 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:02.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:02.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:02.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:02.636 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:10:02.663 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-04-13T03:10:02.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:02.664 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-04-13T03:10:02.731 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-04-13T03:10:02.731 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:10:02.732 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:10:02.732 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:10:02.732 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:10:02.732 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:02.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:03.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:03.050 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:10:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:03.066 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-04-13T03:10:03.121 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:03.122 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:03.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-13T03:10:03.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-13T03:10:03.122 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:03.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:03.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.247 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-13T03:10:03.247 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:03.247 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:03.247 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-13T03:10:03.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-13T03:10:03.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-13T03:10:03.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-13T03:10:03.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-13T03:10:03.248 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.392 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:03.392 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:03.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-04-13T03:10:03.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-04-13T03:10:03.392 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:03.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:03.474 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-13T03:10:03.496 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-13T03:10:03.496 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:03.496 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-13T03:10:03.551 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-13T03:10:03.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:03.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:03.637 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:03.637 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:03.751 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:03.751 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:03.751 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:10:03.751 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 1@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:10:03.751 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:10:03.754 INFO:teuthology.orchestra.run.smithi114.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-13T03:10:03.754 INFO:teuthology.orchestra.run.smithi114.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:10:03.754 INFO:teuthology.orchestra.run.smithi114.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:10:03.755 INFO:teuthology.orchestra.run.smithi114.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:10:03.755 INFO:teuthology.orchestra.run.smithi114.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:10:03.755 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:10:03.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:03.755 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-13T03:10:03.789 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.886 INFO:teuthology.orchestra.run.smithi114.stdout:client4717 2024-04-13T03:10:03.886 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:03.886 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:03.901 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:03.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:03.997 INFO:teuthology.orchestra.run.smithi114.stdout:client4717 2024-04-13T03:10:03.997 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:04.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:04.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:04.052 INFO:teuthology.orchestra.run.smithi114.stdout:client4717 2024-04-13T03:10:04.052 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7ff0368-2781-4238-b477-50485b08a223.client4717/status') 2024-04-13T03:10:04.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.4717 (3)192.168.0.1:0/235844876 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stderr:68 bytes copied, 3.8736e-05 s, 1.8 MB/s 2024-04-13T03:10:04.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:04.121 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-13T03:10:04.121 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T03:10:04.121 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-13T03:10:04.121 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:10:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.236+0000 7f84668c8640 1 Processor -- start 2024-04-13T03:10:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.236+0000 7f84668c8640 1 -- start start 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f84668c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f846015e5c0 0x7f84600cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f84668c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 0x7f84600d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f84668c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84600ba680 con 0x7f84600d2810 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f84668c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f84600cfb00 con 0x7f846015e5c0 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f84668c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f84600ba0c0 con 0x7f846015e9a0 2024-04-13T03:10:04.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f845ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f846015e5c0 0x7f84600cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f845ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f846015e5c0 0x7f84600cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54062/0 (socket says 172.21.15.106:54062) 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.237+0000 7f845ffff640 1 -- 172.21.15.106:0/3594155353 learned_addr learned my addr 172.21.15.106:0/3594155353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845f7fe640 1 --2- 172.21.15.106:0/3594155353 >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 0x7f84600d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845ffff640 1 -- 172.21.15.106:0/3594155353 >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 msgr2=0x7f84600d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845ffff640 1 --2- 172.21.15.106:0/3594155353 >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 0x7f84600d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845ffff640 1 -- 172.21.15.106:0/3594155353 >> v1:172.21.15.114:6789/0 conn(0x7f84600d2810 legacy=0x7f84601623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845ffff640 1 -- 172.21.15.106:0/3594155353 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84600ba3a0 con 0x7f846015e5c0 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845f7fe640 1 --2- 172.21.15.106:0/3594155353 >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 0x7f84600d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.238+0000 7f845ffff640 1 --2- 172.21.15.106:0/3594155353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f84600cf010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f84600b9de0 tx=0x7f845002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ee5aa9994d57db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f845effd640 1 -- 172.21.15.106:0/3594155353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450035020 con 0x7f846015e5c0 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f845effd640 1 -- 172.21.15.106:0/3594155353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8450031870 con 0x7f846015e5c0 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f845effd640 1 -- 172.21.15.106:0/3594155353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450031c40 con 0x7f846015e5c0 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 -- 172.21.15.106:0/3594155353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 msgr2=0x7f84600cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 --2- 172.21.15.106:0/3594155353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f84600cf010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f84600b9de0 tx=0x7f845002b910 comp rx=0 tx=0).stop 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 -- 172.21.15.106:0/3594155353 shutdown_connections 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 --2- 172.21.15.106:0/3594155353 >> v2:172.21.15.114:3300/0 conn(0x7f846015e9a0 0x7f84600d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 --2- 172.21.15.106:0/3594155353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f84600cf010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 -- 172.21.15.106:0/3594155353 >> 172.21.15.106:0/3594155353 conn(0x7f84600d4700 msgr2=0x7f84600d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 -- 172.21.15.106:0/3594155353 shutdown_connections 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.239+0000 7f84668c8640 1 -- 172.21.15.106:0/3594155353 wait complete. 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.240+0000 7f84668c8640 1 Processor -- start 2024-04-13T03:10:04.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 -- start start 2024-04-13T03:10:04.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84600d2810 0x7f84601e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f846015e9a0 0x7f84601612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f846006ca70 con 0x7f846015e5c0 2024-04-13T03:10:04.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f846006c8f0 con 0x7f84600d2810 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.241+0000 7f84668c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f846006cbf0 con 0x7f846015e9a0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54068/0 (socket says 172.21.15.106:54068) 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 -- 172.21.15.106:0/549224144 learned_addr learned my addr 172.21.15.106:0/549224144 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845ffff640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84600d2810 0x7f84601e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f8464e65640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f846015e9a0 0x7f84601612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 -- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f846015e9a0 msgr2=0x7f84601612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f846015e9a0 0x7f84601612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 -- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84600d2810 msgr2=0x7f84601e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84600d2810 0x7f84601e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 -- 172.21.15.106:0/549224144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84601e6fc0 con 0x7f846015e5c0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845f7fe640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f845400c7f0 tx=0x7f845400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:04.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454015070 con 0x7f846015e5c0 2024-04-13T03:10:04.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454007820 con 0x7f846015e5c0 2024-04-13T03:10:04.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84601e72a0 con 0x7f846015e5c0 2024-04-13T03:10:04.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454007bd0 con 0x7f846015e5c0 2024-04-13T03:10:04.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.242+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84601e7860 con 0x7f846015e5c0 2024-04-13T03:10:04.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.244+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8460068a10 con 0x7f846015e5c0 2024-04-13T03:10:04.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.244+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8454018070 con 0x7f846015e5c0 2024-04-13T03:10:04.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.244+0000 7f845cff9640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 0x7f844c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:04.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.244+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f845408bec0 con 0x7f846015e5c0 2024-04-13T03:10:04.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.244+0000 7f845ffff640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 0x7f844c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:04.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.245+0000 7f845ffff640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 0x7f844c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8450036000 tx=0x7f845003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:04.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.249+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84540599f0 con 0x7f846015e5c0 2024-04-13T03:10:04.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.461+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84600b3650 con 0x7f846015e5c0 2024-04-13T03:10:04.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.462+0000 7f845cff9640 1 -- 172.21.15.106:0/549224144 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f845405dcf0 con 0x7f846015e5c0 2024-04-13T03:10:04.464 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 msgr2=0x7f844c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 0x7f844c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8450036000 tx=0x7f845003c040 comp rx=0 tx=0).stop 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 msgr2=0x7f846014ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f845400c7f0 tx=0x7f845400ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 shutdown_connections 2024-04-13T03:10:04.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f844c06c8f0 0x7f844c06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f846015e9a0 0x7f84601612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f846015e5c0 0x7f846014ebd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 --2- 172.21.15.106:0/549224144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84600d2810 0x7f84601e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 >> 172.21.15.106:0/549224144 conn(0x7f84600d4700 msgr2=0x7f84600b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 shutdown_connections 2024-04-13T03:10:04.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:04.464+0000 7f84668c8640 1 -- 172.21.15.106:0/549224144 wait complete. 2024-04-13T03:10:04.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-13T03:10:04.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:10:04.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:10:04.477 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-13T03:10:04.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.477 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:04.493 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42655sec preferred_lft 42655sec 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:04.494 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether a6:a4:b2:f9:ae:64 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::2cad:7cff:feb0:eda4/64 scope link 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether a6:a4:b2:f9:ae:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::a4a4:b2ff:fef9:ae64/64 scope link tentative 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.495 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:10:04.559 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-04-13T03:10:04.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.561 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-04-13T03:10:04.617 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-04-13T03:10:04.617 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:10:04.617 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:10:04.618 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:10:04.618 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:10:04.618 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:04.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:04.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:04.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.752 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:10:04.766 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-13T03:10:04.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.767 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-04-13T03:10:04.822 INFO:teuthology.orchestra.run.smithi114.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-13T03:10:04.822 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:04.823 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:04.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-13T03:10:04.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-13T03:10:04.823 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:04.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.945 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.971 INFO:teuthology.orchestra.run.smithi114.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:04.972 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether de:7d:6b:c1:47:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-13T03:10:04.972 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-13T03:10:04.972 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.972 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::dc7d:6bff:fec1:478c/64 scope link 2024-04-13T03:10:04.972 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:04.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:04.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether de:7d:6b:c1:47:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-13T03:10:05.040 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:05.041 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::dc7d:6bff:fec1:478c/64 scope link 2024-04-13T03:10:05.041 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:05.042 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-13T03:10:05.042 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:05.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:05.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:05.210 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:05.210 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-04-13T03:10:05.210 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.1 up 2024-04-13T03:10:05.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.1 master ceph-brx 2024-04-13T03:10:05.211 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-04-13T03:10:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:05.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:05.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:05.291 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-13T03:10:05.305 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-13T03:10:05.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:05.306 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-13T03:10:05.361 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-13T03:10:05.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:05.361 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:05.445 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:05.445 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:05.452 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:05.452 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:05.452 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:10:05.452 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 3@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:10:05.453 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run.smithi114.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run.smithi114.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run.smithi114.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run.smithi114.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run.smithi114.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:10:05.455 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:10:05.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:05.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-13T03:10:05.488 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:05.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:05.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:05.583 INFO:teuthology.orchestra.run.smithi114.stdout:client4908 2024-04-13T03:10:05.583 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:05.584 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:05.599 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:05.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:05.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:05.923 INFO:teuthology.orchestra.run.smithi114.stdout:client4908 2024-04-13T03:10:05.924 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:05.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:05.981 INFO:teuthology.orchestra.run.smithi114.stdout:client4908 2024-04-13T03:10:05.982 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7ff0368-2781-4238-b477-50485b08a223.client4908/status') 2024-04-13T03:10:06.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:06.044 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.4908 (3)172.21.15.114:0/87667927 2024-04-13T03:10:06.044 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2024-04-13T03:10:06.044 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2024-04-13T03:10:06.044 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2024-04-13T03:10:06.044 INFO:teuthology.orchestra.run.smithi114.stderr:69 bytes copied, 2.1291e-05 s, 3.2 MB/s 2024-04-13T03:10:06.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-13T03:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:06.051 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-13T03:10:06.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.215+0000 7f9fa537b640 1 Processor -- start 2024-04-13T03:10:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.215+0000 7f9fa537b640 1 -- start start 2024-04-13T03:10:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9fa537b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fa00b3d60 0x7f9fa00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9fa537b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa00cfd20 con 0x7f9fa015ed90 2024-04-13T03:10:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9fa537b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa0162af0 con 0x7f9fa015e9b0 2024-04-13T03:10:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9fa537b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa0162dd0 con 0x7f9fa00b3d60 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fa00b3d60 0x7f9fa00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fa00b3d60 0x7f9fa00d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54082/0 (socket says 172.21.15.106:54082) 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/817080288 learned_addr learned my addr 172.21.15.106:0/817080288 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/817080288 >> v1:172.21.15.114:6789/0 conn(0x7f9fa015e9b0 legacy=0x7f9fa00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/817080288 >> v1:172.21.15.106:6789/0 conn(0x7f9fa015ed90 legacy=0x7f9fa00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9dffb640 1 -- 172.21.15.106:0/817080288 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923618255 0 0) 0x7f9fa00cfd20 con 0x7f9fa015ed90 2024-04-13T03:10:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.216+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/817080288 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa01630b0 con 0x7f9fa00b3d60 2024-04-13T03:10:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.217+0000 7f9f9f7fe640 1 --2- 172.21.15.106:0/817080288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa00d2300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009e50 tx=0x7f9f9802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afe874d2cd13940 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.217+0000 7f9f9dffb640 1 -- 172.21.15.106:0/817080288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9802cbb0 con 0x7f9fa00b3d60 2024-04-13T03:10:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.217+0000 7f9f9dffb640 1 -- 172.21.15.106:0/817080288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9802cd50 con 0x7f9fa00b3d60 2024-04-13T03:10:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.217+0000 7f9f9dffb640 1 -- 172.21.15.106:0/817080288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98034610 con 0x7f9fa00b3d60 2024-04-13T03:10:06.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 -- 172.21.15.106:0/817080288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 msgr2=0x7f9fa00d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 --2- 172.21.15.106:0/817080288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa00d2300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009e50 tx=0x7f9f9802c010 comp rx=0 tx=0).stop 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 -- 172.21.15.106:0/817080288 shutdown_connections 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 --2- 172.21.15.106:0/817080288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa00d2300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 -- 172.21.15.106:0/817080288 >> 172.21.15.106:0/817080288 conn(0x7f9fa008bf10 msgr2=0x7f9fa00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 -- 172.21.15.106:0/817080288 shutdown_connections 2024-04-13T03:10:06.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.218+0000 7f9fa537b640 1 -- 172.21.15.106:0/817080288 wait complete. 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.219+0000 7f9fa537b640 1 Processor -- start 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.219+0000 7f9fa537b640 1 -- start start 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa015e9b0 0x7f9fa0159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa015b640 0x7f9fa015bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0162800 con 0x7f9fa00b3d60 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0162680 con 0x7f9fa015b640 2024-04-13T03:10:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9fa537b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa00d4d00 con 0x7f9fa015e9b0 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54088/0 (socket says 172.21.15.106:54088) 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 -- 172.21.15.106:0/1537952905 learned_addr learned my addr 172.21.15.106:0/1537952905 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9f7fe640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa015b640 0x7f9fa015bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9e7fc640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa015e9b0 0x7f9fa0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 -- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa015e9b0 msgr2=0x7f9fa0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa015e9b0 0x7f9fa0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 -- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa015b640 msgr2=0x7f9fa015bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa015b640 0x7f9fa015bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 -- 172.21.15.106:0/1537952905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa01f8b00 con 0x7f9fa00b3d60 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.220+0000 7f9f9effd640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c460 tx=0x7f9f9000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.221+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90016020 con 0x7f9fa00b3d60 2024-04-13T03:10:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.221+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f90012870 con 0x7f9fa00b3d60 2024-04-13T03:10:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.221+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa006d9c0 con 0x7f9fa00b3d60 2024-04-13T03:10:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.221+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90012c60 con 0x7f9fa00b3d60 2024-04-13T03:10:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.221+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa006df80 con 0x7f9fa00b3d60 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.222+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0068a10 con 0x7f9fa00b3d60 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.222+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f90019020 con 0x7f9fa00b3d60 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.222+0000 7f9f877fe640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.223+0000 7f9f9e7fc640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.223+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9f9008bdb0 con 0x7f9fa00b3d60 2024-04-13T03:10:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.223+0000 7f9f9e7fc640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015a490 tx=0x7f9f94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.228+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f90020020 con 0x7f9fa00b3d60 2024-04-13T03:10:06.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.404+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fa015c010 con 0x7f9fa00b3d60 2024-04-13T03:10:06.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.406+0000 7f9f877fe640 1 -- 172.21.15.106:0/1537952905 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f9f90059820 con 0x7f9fa00b3d60 2024-04-13T03:10:06.408 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:06.408 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 msgr2=0x7f9f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015a490 tx=0x7f9f94009040 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 msgr2=0x7f9fa006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c460 tx=0x7f9f9000c920 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 shutdown_connections 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa015e9b0 0x7f9fa0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa015b640 0x7f9fa015bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.407+0000 7f9fa537b640 1 --2- 172.21.15.106:0/1537952905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fa00b3d60 0x7f9fa006e4a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.408+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 >> 172.21.15.106:0/1537952905 conn(0x7f9fa008bf10 msgr2=0x7f9fa00b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:06.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.408+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 shutdown_connections 2024-04-13T03:10:06.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.408+0000 7f9fa537b640 1 -- 172.21.15.106:0/1537952905 wait complete. 2024-04-13T03:10:06.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:06.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:06.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.523+0000 7f519935e640 1 Processor -- start 2024-04-13T03:10:06.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.523+0000 7f519935e640 1 -- start start 2024-04-13T03:10:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f519935e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f519415ed90 0x7f51940d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f519935e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f51940d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f519935e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51940cfd20 con 0x7f519415e9b0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f519935e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5194162af0 con 0x7f519415ed90 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f519935e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5194162dd0 con 0x7f51940b3d60 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5192ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f519415e9b0 0x7f51940d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42610/0 (socket says 172.21.15.106:42610) 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5192ffd640 1 -- 172.21.15.106:0/1128046890 learned_addr learned my addr 172.21.15.106:0/1128046890 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220779127 0 0) 0x7f51940cfd20 con 0x7f519415e9b0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f51927fc640 1 --2- 172.21.15.106:0/1128046890 >> v2:172.21.15.114:3300/0 conn(0x7f519415ed90 0x7f51940d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f517c003520 con 0x7f519415e9b0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f51937fe640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f51940d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278573534 0 0) 0x7f517c003520 con 0x7f519415e9b0 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 msgr2=0x7f51940d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f51940d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 >> v2:172.21.15.114:3300/0 conn(0x7f519415ed90 msgr2=0x7f51940d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 --2- 172.21.15.106:0/1128046890 >> v2:172.21.15.114:3300/0 conn(0x7f519415ed90 0x7f51940d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51941630b0 con 0x7f519415e9b0 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5184001670 con 0x7f519415e9b0 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 >> v1:172.21.15.106:6789/0 conn(0x7f519415e9b0 legacy=0x7f51940d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f517c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f517c003f30 0x7f517c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f517c004d40 0x7f517c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5194162af0 con 0x7f517c004d40 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5194162dd0 con 0x7f517c003f30 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.524+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f517c00b9f0 con 0x7f51940b3d60 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f517c004d40 0x7f517c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f5192ffd640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f517c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51927fc640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f517c003f30 0x7f517c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 -- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 msgr2=0x7f517c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b3d60 0x7f517c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 -- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f517c003f30 msgr2=0x7f517c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f517c003f30 0x7f517c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 -- 172.21.15.106:0/1128046890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c003520 con 0x7f517c004d40 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.525+0000 7f51937fe640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f517c004d40 0x7f517c0051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f518c009f90 tx=0x7f517c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bd02f7632a4693 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c039070 con 0x7f517c004d40 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c02cbd0 con 0x7f517c004d40 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f5191ffb640 1 -- 172.21.15.106:0/1128046890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c009c10 con 0x7f517c004d40 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 -- 172.21.15.106:0/1128046890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f517c004d40 msgr2=0x7f517c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 --2- 172.21.15.106:0/1128046890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f517c004d40 0x7f517c0051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f518c009f90 tx=0x7f517c005830 comp rx=0 tx=0).stop 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f51937fe640 1 -- 172.21.15.106:0/1128046890 reap_dead start 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 -- 172.21.15.106:0/1128046890 shutdown_connections 2024-04-13T03:10:06.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 -- 172.21.15.106:0/1128046890 >> 172.21.15.106:0/1128046890 conn(0x7f519408bf10 msgr2=0x7f51940b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:06.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 -- 172.21.15.106:0/1128046890 shutdown_connections 2024-04-13T03:10:06.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.526+0000 7f519935e640 1 -- 172.21.15.106:0/1128046890 wait complete. 2024-04-13T03:10:06.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 Processor -- start 2024-04-13T03:10:06.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 -- start start 2024-04-13T03:10:06.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f519406e210 0x7f51940b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b6b60 0x7f51940b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5194162800 con 0x7f51940b7890 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5194162680 con 0x7f519406e210 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f519935e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f51940d4d00 con 0x7f51940b6b60 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f51937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f51937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54104/0 (socket says 172.21.15.106:54104) 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f51937fe640 1 -- 172.21.15.106:0/2605975668 learned_addr learned my addr 172.21.15.106:0/2605975668 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f51927fc640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b6b60 0x7f51940b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.528+0000 7f5192ffd640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f519406e210 0x7f51940b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 -- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b6b60 msgr2=0x7f51940b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b6b60 0x7f51940b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 -- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f519406e210 msgr2=0x7f51940b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f519406e210 0x7f51940b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 -- 172.21.15.106:0/2605975668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519406b8e0 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f51937fe640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e230 tx=0x7f518c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c039030 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c03a040 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51940b6fe0 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c009c10 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.529+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519406bba0 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.530+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5194068a10 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.530+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f518c045030 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.531+0000 7f517b7fe640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.531+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f518c0ada50 con 0x7f51940b7890 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.531+0000 7f5192ffd640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.531+0000 7f5192ffd640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 0x7f518006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5184004840 tx=0x7f5184007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.536+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f518c04c060 con 0x7f51940b7890 2024-04-13T03:10:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.709+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51940b7fb0 con 0x7f51940b7890 2024-04-13T03:10:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.709+0000 7f517b7fe640 1 -- 172.21.15.106:0/2605975668 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f518c07bbd0 con 0x7f51940b7890 2024-04-13T03:10:06.711 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 msgr2=0x7f518006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 0x7f518006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5184004840 tx=0x7f5184007970 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 msgr2=0x7f51941f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e230 tx=0x7f518c009750 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 shutdown_connections 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f518006c8f0 0x7f518006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51940b6b60 0x7f51940b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51940b7890 0x7f51941f8010 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 --2- 172.21.15.106:0/2605975668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f519406e210 0x7f51940b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 >> 172.21.15.106:0/2605975668 conn(0x7f519408bf10 msgr2=0x7f51940d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 shutdown_connections 2024-04-13T03:10:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.711+0000 7f519935e640 1 -- 172.21.15.106:0/2605975668 wait complete. 2024-04-13T03:10:06.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:06.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.826+0000 7f42ee2c8640 1 Processor -- start 2024-04-13T03:10:06.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.826+0000 7f42ee2c8640 1 -- start start 2024-04-13T03:10:06.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ee2c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ee2c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f42e80cef00 0x7f42e80aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ee2c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e806ebb0 con 0x7f42e806e6b0 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ee2c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f42e815ba00 con 0x7f42e80cef00 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ee2c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42e806ee90 con 0x7f42e815b620 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f42e80cef00 0x7f42e80aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f42e80cef00 0x7f42e80aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:45620/0 (socket says 172.21.15.106:45620) 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:38544/0 (socket says 172.21.15.106:38544) 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 -- 172.21.15.106:0/1026641684 learned_addr learned my addr 172.21.15.106:0/1026641684 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42e77fe640 1 --1- 172.21.15.106:0/1026641684 >> v1:172.21.15.114:6789/0 conn(0x7f42e806e6b0 0x7f42e80ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:44354/0 (socket says 172.21.15.106:44354) 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 -- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 msgr2=0x7f42e806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 -- 172.21.15.106:0/1026641684 >> v1:172.21.15.114:6789/0 conn(0x7f42e806e6b0 legacy=0x7f42e80ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42ec865640 1 -- 172.21.15.106:0/1026641684 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e806f170 con 0x7f42e80cef00 2024-04-13T03:10:06.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42e6ffd640 1 -- 172.21.15.106:0/1026641684 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878236461 0 0) 0x7f42e806ebb0 con 0x7f42e806e6b0 2024-04-13T03:10:06.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.827+0000 7f42e7fff640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.828+0000 7f42ec865640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e80aebd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009e50 tx=0x7f42e002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aacbec69b9164ba server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.828+0000 7f42e6ffd640 1 -- 172.21.15.106:0/1026641684 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e002cb00 con 0x7f42e80cef00 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.828+0000 7f42e6ffd640 1 -- 172.21.15.106:0/1026641684 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e002cca0 con 0x7f42e80cef00 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.828+0000 7f42e6ffd640 1 -- 172.21.15.106:0/1026641684 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0034610 con 0x7f42e80cef00 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 -- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 msgr2=0x7f42e80aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e80aebd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009e50 tx=0x7f42e002bf60 comp rx=0 tx=0).stop 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 -- 172.21.15.106:0/1026641684 shutdown_connections 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e80aebd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/1026641684 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 -- 172.21.15.106:0/1026641684 >> 172.21.15.106:0/1026641684 conn(0x7f42e80cd6a0 msgr2=0x7f42e80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 -- 172.21.15.106:0/1026641684 shutdown_connections 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.829+0000 7f42ee2c8640 1 -- 172.21.15.106:0/1026641684 wait complete. 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.830+0000 7f42ee2c8640 1 Processor -- start 2024-04-13T03:10:06.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.830+0000 7f42ee2c8640 1 -- start start 2024-04-13T03:10:06.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e81e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e81e5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80cf350 con 0x7f42e806e6b0 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42e806f730 con 0x7f42e80cef00 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ee2c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80cf4d0 con 0x7f42e815b620 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54116/0 (socket says 172.21.15.106:54116) 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 -- 172.21.15.106:0/179511728 learned_addr learned my addr 172.21.15.106:0/179511728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e77fe640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e81e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42ec865640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e81e5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 -- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 msgr2=0x7f42e81e5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e81e5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 -- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 msgr2=0x7f42e81e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:06.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e81e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.831+0000 7f42e7fff640 1 -- 172.21.15.106:0/179511728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81e4aa0 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42e7fff640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42d800ebe0 tx=0x7f42d800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8017070 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d800cc30 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8013470 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81e4d80 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.832+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81e5340 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.833+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e8068a10 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.833+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42d801a070 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.833+0000 7f42e4ff9640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 0x7f42d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.834+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f42d808be00 con 0x7f42e806e6b0 2024-04-13T03:10:06.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.834+0000 7f42e77fe640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 0x7f42d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:06.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.834+0000 7f42e77fe640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 0x7f42d406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42e81561f0 tx=0x7f42dc007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:06.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:06.839+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42d80597f0 con 0x7f42e806e6b0 2024-04-13T03:10:07.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.012+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42e806d280 con 0x7f42e806e6b0 2024-04-13T03:10:07.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.013+0000 7f42e4ff9640 1 -- 172.21.15.106:0/179511728 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f42d805daf0 con 0x7f42e806e6b0 2024-04-13T03:10:07.014 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 msgr2=0x7f42d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 0x7f42d406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42e81561f0 tx=0x7f42dc007e40 comp rx=0 tx=0).stop 2024-04-13T03:10:07.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 msgr2=0x7f42e8144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42d800ebe0 tx=0x7f42d800c520 comp rx=0 tx=0).stop 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 shutdown_connections 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f42d406c8f0 0x7f42d406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42e815b620 0x7f42e81e5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42e80cef00 0x7f42e81e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 --2- 172.21.15.106:0/179511728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e806e6b0 0x7f42e8144320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 >> 172.21.15.106:0/179511728 conn(0x7f42e80cd6a0 msgr2=0x7f42e8155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 shutdown_connections 2024-04-13T03:10:07.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.015+0000 7f42ee2c8640 1 -- 172.21.15.106:0/179511728 wait complete. 2024-04-13T03:10:07.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:07.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.129+0000 7fcf34436640 1 Processor -- start 2024-04-13T03:10:07.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.129+0000 7fcf34436640 1 -- start start 2024-04-13T03:10:07.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf34436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf2c1697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf34436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c0b5cf0 con 0x7fcf2c15ee40 2024-04-13T03:10:07.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf34436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c0b5fd0 con 0x7fcf2c0b1c30 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf34436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c0b62b0 con 0x7fcf2c070bb0 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf319d1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcf2c15ee40 0x7fcf2c0704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42612/0 (socket says 172.21.15.106:42612) 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf319d1640 1 -- 172.21.15.106:0/3119691626 learned_addr learned my addr 172.21.15.106:0/3119691626 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf329d3640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf2c1697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013028741 0 0) 0x7fcf2c0b5cf0 con 0x7fcf2c15ee40 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf18003520 con 0x7fcf2c15ee40 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1934256114 0 0) 0x7fcf18003520 con 0x7fcf2c15ee40 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 msgr2=0x7fcf2c1697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf2c1697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 >> v1:172.21.15.114:6789/0 conn(0x7fcf2c0b1c30 legacy=0x7fcf2c0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c0b6590 con 0x7fcf2c15ee40 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fcf1c001670 con 0x7fcf2c15ee40 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 >> v1:172.21.15.106:6789/0 conn(0x7fcf2c15ee40 legacy=0x7fcf2c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf180039e0 0x7fcf18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf18004800 0x7fcf18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c0b62b0 con 0x7fcf180039e0 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1800baa0 con 0x7fcf18004800 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.130+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1800bca0 con 0x7fcf2c070bb0 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf180039e0 0x7fcf18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf319d1640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf18004800 0x7fcf18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf329d3640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 -- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 msgr2=0x7fcf180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c070bb0 0x7fcf180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 -- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf18004800 msgr2=0x7fcf18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf18004800 0x7fcf18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.131+0000 7fcf321d2640 1 -- 172.21.15.106:0/3119691626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf18003520 con 0x7fcf180039e0 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf321d2640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf180039e0 0x7fcf18003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001750 tx=0x7fcf18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6131042e2d7c452 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28001370 con 0x7fcf180039e0 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf280052d0 con 0x7fcf180039e0 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf311d0640 1 -- 172.21.15.106:0/3119691626 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28003830 con 0x7fcf180039e0 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 -- 172.21.15.106:0/3119691626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf180039e0 msgr2=0x7fcf18003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 --2- 172.21.15.106:0/3119691626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf180039e0 0x7fcf18003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001750 tx=0x7fcf18005830 comp rx=0 tx=0).stop 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf329d3640 1 -- 172.21.15.106:0/3119691626 reap_dead start 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 -- 172.21.15.106:0/3119691626 shutdown_connections 2024-04-13T03:10:07.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 -- 172.21.15.106:0/3119691626 >> 172.21.15.106:0/3119691626 conn(0x7fcf2c071870 msgr2=0x7fcf2c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 -- 172.21.15.106:0/3119691626 shutdown_connections 2024-04-13T03:10:07.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.132+0000 7fcf34436640 1 -- 172.21.15.106:0/3119691626 wait complete. 2024-04-13T03:10:07.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.133+0000 7fcf34436640 1 Processor -- start 2024-04-13T03:10:07.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 -- start start 2024-04-13T03:10:07.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf180039e0 0x7fcf2c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c156f50 0x7fcf2c1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1677b0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c06e880 con 0x7fcf180039e0 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf34436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c167930 con 0x7fcf2c156f50 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf329d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf329d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54130/0 (socket says 172.21.15.106:54130) 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf329d3640 1 -- 172.21.15.106:0/4130985112 learned_addr learned my addr 172.21.15.106:0/4130985112 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf319d1640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c156f50 0x7fcf2c1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.134+0000 7fcf321d2640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf180039e0 0x7fcf2c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 -- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c156f50 msgr2=0x7fcf2c1f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c156f50 0x7fcf2c1f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 -- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf180039e0 msgr2=0x7fcf2c156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf180039e0 0x7fcf2c156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 -- 172.21.15.106:0/4130985112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c157510 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf329d3640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c15c9a0 tx=0x7fcf2400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24015430 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf240155f0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240159e0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1577f0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.135+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c157db0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.136+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c068a10 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.136+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf24015b80 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.137+0000 7fcf22ffd640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 0x7fcf1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.137+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcf2408cde0 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.137+0000 7fcf321d2640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 0x7fcf1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.137+0000 7fcf321d2640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 0x7fcf1006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcf28033000 tx=0x7fcf28009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.141+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf24019030 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.317+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf2c15f870 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.317+0000 7fcf22ffd640 1 -- 172.21.15.106:0/4130985112 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fcf24010040 con 0x7fcf2c1f53f0 2024-04-13T03:10:07.318 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 msgr2=0x7fcf1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 0x7fcf1006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcf28033000 tx=0x7fcf28009030 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 msgr2=0x7fcf2c1f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c15c9a0 tx=0x7fcf2400f580 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 shutdown_connections 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf1006c8f0 0x7fcf1006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcf2c156f50 0x7fcf2c1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf2c1f53f0 0x7fcf2c1f5860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 --2- 172.21.15.106:0/4130985112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf180039e0 0x7fcf2c156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 >> 172.21.15.106:0/4130985112 conn(0x7fcf2c071870 msgr2=0x7fcf2c158920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 shutdown_connections 2024-04-13T03:10:07.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.319+0000 7fcf34436640 1 -- 172.21.15.106:0/4130985112 wait complete. 2024-04-13T03:10:07.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:07.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.434+0000 7f3828cad640 1 Processor -- start 2024-04-13T03:10:07.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.434+0000 7f3828cad640 1 -- start start 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f3828cad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38240af800 0x7f382415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f3828cad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382406f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f3828cad640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382415c200 con 0x7f382415e830 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f3828cad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f382406fba0 con 0x7f38240af800 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f3828cad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38240b5090 con 0x7f382406f200 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38240af800 0x7f382415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38240af800 0x7f382415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54140/0 (socket says 172.21.15.106:54140) 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 -- 172.21.15.106:0/626565167 learned_addr learned my addr 172.21.15.106:0/626565167 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f381bfff640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382406f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 -- 172.21.15.106:0/626565167 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 msgr2=0x7f382406f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382406f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 -- 172.21.15.106:0/626565167 >> v1:172.21.15.114:6789/0 conn(0x7f382415e830 legacy=0x7f382415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.435+0000 7f382259c640 1 -- 172.21.15.106:0/626565167 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38240b3ff0 con 0x7f38240af800 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.436+0000 7f382259c640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38240af800 0x7f382415bcc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f381400bb10 tx=0x7f381402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4742c4c944010ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.436+0000 7f3821d9b640 1 -- 172.21.15.106:0/626565167 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381403b070 con 0x7f38240af800 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.436+0000 7f3821d9b640 1 -- 172.21.15.106:0/626565167 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3814036440 con 0x7f38240af800 2024-04-13T03:10:07.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.436+0000 7f3821d9b640 1 -- 172.21.15.106:0/626565167 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814036810 con 0x7f38240af800 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 -- 172.21.15.106:0/626565167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38240af800 msgr2=0x7f382415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38240af800 0x7f382415bcc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f381400bb10 tx=0x7f381402e0c0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 -- 172.21.15.106:0/626565167 shutdown_connections 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382406f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 --2- 172.21.15.106:0/626565167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38240af800 0x7f382415bcc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 -- 172.21.15.106:0/626565167 >> 172.21.15.106:0/626565167 conn(0x7f38240b2600 msgr2=0x7f38240d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 -- 172.21.15.106:0/626565167 shutdown_connections 2024-04-13T03:10:07.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.437+0000 7f3828cad640 1 -- 172.21.15.106:0/626565167 wait complete. 2024-04-13T03:10:07.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.438+0000 7f3828cad640 1 Processor -- start 2024-04-13T03:10:07.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.438+0000 7f3828cad640 1 -- start start 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382414adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38240af800 0x7f3824153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38240b4890 con 0x7f382415e830 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38240b45b0 con 0x7f38240af800 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38240b4b70 con 0x7f382406f200 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54150/0 (socket says 172.21.15.106:54150) 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 -- 172.21.15.106:0/3184681193 learned_addr learned my addr 172.21.15.106:0/3184681193 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:07.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f381bfff640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38240af800 0x7f3824153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f382259c640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382414adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 -- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 msgr2=0x7f382414adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382414adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 -- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38240af800 msgr2=0x7f3824153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38240af800 0x7f3824153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 -- 172.21.15.106:0/3184681193 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382414b2f0 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3822d9d640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c7b0 tx=0x7f381c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c004b60 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c004d00 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c00aa90 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3824153d00 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.439+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382414f420 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.440+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3824068a10 con 0x7f382415e830 2024-04-13T03:10:07.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.441+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f381c00ac30 con 0x7f382415e830 2024-04-13T03:10:07.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.441+0000 7f381affd640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.441+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f381c08c220 con 0x7f382415e830 2024-04-13T03:10:07.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.441+0000 7f382259c640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.442+0000 7f382259c640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 0x7f380406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3814038000 tx=0x7f3814032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.445+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f381c020070 con 0x7f382415e830 2024-04-13T03:10:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.621+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f382415e610 con 0x7f382415e830 2024-04-13T03:10:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.621+0000 7f381affd640 1 -- 172.21.15.106:0/3184681193 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f381c05a250 con 0x7f382415e830 2024-04-13T03:10:07.622 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 msgr2=0x7f380406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 0x7f380406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3814038000 tx=0x7f3814032000 comp rx=0 tx=0).stop 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 msgr2=0x7f3824151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c7b0 tx=0x7f381c00cc70 comp rx=0 tx=0).stop 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 shutdown_connections 2024-04-13T03:10:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f382415e830 0x7f3824151710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38240af800 0x7f3824153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 --2- 172.21.15.106:0/3184681193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f382406f200 0x7f382414adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 >> 172.21.15.106:0/3184681193 conn(0x7f38240b2600 msgr2=0x7f382406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 shutdown_connections 2024-04-13T03:10:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.623+0000 7f3828cad640 1 -- 172.21.15.106:0/3184681193 wait complete. 2024-04-13T03:10:07.635 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:07.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.735+0000 7f0f92008640 1 Processor -- start 2024-04-13T03:10:07.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.735+0000 7f0f92008640 1 -- start start 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f92008640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f8c1622d0 0x7f0f8c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f92008640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f92008640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8c0d3530 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f92008640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8c1626b0 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f92008640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8c0d0b60 con 0x7f0f8c1622d0 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54152/0 (socket says 172.21.15.106:54152) 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 -- 172.21.15.106:0/3859679178 learned_addr learned my addr 172.21.15.106:0/3859679178 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:07.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8b7fe640 1 --2- 172.21.15.106:0/3859679178 >> v2:172.21.15.114:3300/0 conn(0x7f0f8c1622d0 0x7f0f8c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 -- 172.21.15.106:0/3859679178 >> v2:172.21.15.114:3300/0 conn(0x7f0f8c1622d0 msgr2=0x7f0f8c0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 --2- 172.21.15.106:0/3859679178 >> v2:172.21.15.114:3300/0 conn(0x7f0f8c1622d0 0x7f0f8c0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 -- 172.21.15.106:0/3859679178 >> v1:172.21.15.114:6789/0 conn(0x7f0f8c0cc6f0 legacy=0x7f0f8c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.736+0000 7f0f8affd640 1 -- 172.21.15.106:0/3859679178 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c0d0e40 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f8affd640 1 --2- 172.21.15.106:0/3859679178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0b5200 tx=0x7f0f8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0091638091bbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f8a7fc640 1 -- 172.21.15.106:0/3859679178 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035020 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f8a7fc640 1 -- 172.21.15.106:0/3859679178 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80031870 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f8a7fc640 1 -- 172.21.15.106:0/3859679178 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80031c40 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f92008640 1 -- 172.21.15.106:0/3859679178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0d1c00 msgr2=0x7f0f8c0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.737+0000 7f0f92008640 1 --2- 172.21.15.106:0/3859679178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0b5200 tx=0x7f0f8002b910 comp rx=0 tx=0).stop 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 -- 172.21.15.106:0/3859679178 shutdown_connections 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 --2- 172.21.15.106:0/3859679178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0d1c00 0x7f0f8c0b7970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 --2- 172.21.15.106:0/3859679178 >> v2:172.21.15.114:3300/0 conn(0x7f0f8c1622d0 0x7f0f8c0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 -- 172.21.15.106:0/3859679178 >> 172.21.15.106:0/3859679178 conn(0x7f0f8c0d25d0 msgr2=0x7f0f8c0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 -- 172.21.15.106:0/3859679178 shutdown_connections 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.738+0000 7f0f92008640 1 -- 172.21.15.106:0/3859679178 wait complete. 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.739+0000 7f0f92008640 1 Processor -- start 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.739+0000 7f0f92008640 1 -- start start 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f8c0d1c00 0x7f0f8c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f8c1622d0 0x7f0f8c0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c0d16e0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c0d1400 con 0x7f0f8c1622d0 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8c0b7eb0 con 0x7f0f8c0d1c00 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54166/0 (socket says 172.21.15.106:54166) 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 -- 172.21.15.106:0/1658857780 learned_addr learned my addr 172.21.15.106:0/1658857780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8affd640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f8c0d1c00 0x7f0f8c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8bfff640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f8c1622d0 0x7f0f8c0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 -- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f8c0d1c00 msgr2=0x7f0f8c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f8c0d1c00 0x7f0f8c0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 -- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f8c1622d0 msgr2=0x7f0f8c0c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f8c1622d0 0x7f0f8c0c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 -- 172.21.15.106:0/1658857780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c1f53d0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f8b7fe640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d5f0 tx=0x7f0f7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0125f0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c019070 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c0c9da0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0129d0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.740+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c0ca360 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.741+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8c068a10 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.742+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f7c012b70 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.742+0000 7f0f73fff640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 0x7f0f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.742+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0f7c08c1d0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.742+0000 7f0f8affd640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 0x7f0f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:07.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.743+0000 7f0f8affd640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 0x7f0f7806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0f80036000 tx=0x7f0f8003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:07.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.746+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f7c059bb0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.921+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f8c068140 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.922+0000 7f0f73fff640 1 -- 172.21.15.106:0/1658857780 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f0f7c05deb0 con 0x7f0f8c0cc6f0 2024-04-13T03:10:07.923 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:07.923 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 msgr2=0x7f0f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 0x7f0f7806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0f80036000 tx=0x7f0f8003c040 comp rx=0 tx=0).stop 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 msgr2=0x7f0f8c0c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d5f0 tx=0x7f0f7c00dab0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 shutdown_connections 2024-04-13T03:10:07.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0f7806c920 0x7f0f7806ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f8c0d1c00 0x7f0f8c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f8c1622d0 0x7f0f8c0c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 --2- 172.21.15.106:0/1658857780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f8c0cc6f0 0x7f0f8c0c96a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 >> 172.21.15.106:0/1658857780 conn(0x7f0f8c0d25d0 msgr2=0x7f0f8c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 shutdown_connections 2024-04-13T03:10:07.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:07.924+0000 7f0f92008640 1 -- 172.21.15.106:0/1658857780 wait complete. 2024-04-13T03:10:07.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:07.937 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.039+0000 7f1880c0f640 1 Processor -- start 2024-04-13T03:10:08.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.039+0000 7f1880c0f640 1 -- start start 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f1880c0f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c15b4e0 0x7f187c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f1880c0f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f187c0b5390 0x7f187c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f1880c0f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187c0b3690 con 0x7f187c0b4730 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f1880c0f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f187c0b3970 con 0x7f187c0b5390 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f1880c0f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f187c0b3c50 con 0x7f187c15b4e0 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f187c0b5390 0x7f187c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f187c0b5390 0x7f187c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54174/0 (socket says 172.21.15.106:54174) 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 -- 172.21.15.106:0/1018235572 learned_addr learned my addr 172.21.15.106:0/1018235572 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187a59c640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c15b4e0 0x7f187c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 -- 172.21.15.106:0/1018235572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c15b4e0 msgr2=0x7f187c15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:08.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c15b4e0 0x7f187c15fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 -- 172.21.15.106:0/1018235572 >> v1:172.21.15.114:6789/0 conn(0x7f187c0b4730 legacy=0x7f187c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.040+0000 7f187ad9d640 1 -- 172.21.15.106:0/1018235572 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c0b3f30 con 0x7f187c0b5390 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f187ad9d640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c06dd30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1874009e50 tx=0x7f187402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7222f1765fab2d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1879d9b640 1 -- 172.21.15.106:0/1018235572 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187402cb00 con 0x7f187c0b5390 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1879d9b640 1 -- 172.21.15.106:0/1018235572 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187402cca0 con 0x7f187c0b5390 2024-04-13T03:10:08.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1879d9b640 1 -- 172.21.15.106:0/1018235572 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874034610 con 0x7f187c0b5390 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 -- 172.21.15.106:0/1018235572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 msgr2=0x7f187c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c06dd30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1874009e50 tx=0x7f187402bf60 comp rx=0 tx=0).stop 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 -- 172.21.15.106:0/1018235572 shutdown_connections 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c06dd30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 --2- 172.21.15.106:0/1018235572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c15b4e0 0x7f187c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.041+0000 7f1880c0f640 1 -- 172.21.15.106:0/1018235572 >> 172.21.15.106:0/1018235572 conn(0x7f187c0b2600 msgr2=0x7f187c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.042+0000 7f1880c0f640 1 -- 172.21.15.106:0/1018235572 shutdown_connections 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.042+0000 7f1880c0f640 1 -- 172.21.15.106:0/1018235572 wait complete. 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.043+0000 7f1880c0f640 1 Processor -- start 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.043+0000 7f1880c0f640 1 -- start start 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 0x7f187c0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f187c15b4e0 0x7f187c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187c06d0c0 con 0x7f187c0b5390 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0b32d0 con 0x7f187c15b4e0 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f187c154cc0 con 0x7f187c0b4730 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54178/0 (socket says 172.21.15.106:54178) 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 -- 172.21.15.106:0/3545615890 learned_addr learned my addr 172.21.15.106:0/3545615890 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f187a59c640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 0x7f187c0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f187ad9d640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f187c15b4e0 0x7f187c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 -- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 msgr2=0x7f187c0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 0x7f187c0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 -- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f187c15b4e0 msgr2=0x7f187c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f187c15b4e0 0x7f187c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 -- 172.21.15.106:0/3545615890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c0ce4f0 con 0x7f187c0b5390 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f187a59c640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 0x7f187c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1873fff640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f186400c400 tx=0x7f186400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640052d0 con 0x7f187c0b5390 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864005490 con 0x7f187c0b5390 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c150340 con 0x7f187c0b5390 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864007260 con 0x7f187c0b5390 2024-04-13T03:10:08.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.044+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c154f10 con 0x7f187c0b5390 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.045+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c068a10 con 0x7f187c0b5390 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.046+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1864007400 con 0x7f187c0b5390 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.046+0000 7f1872ffd640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 0x7f185c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.046+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f186408b450 con 0x7f187c0b5390 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.046+0000 7f187a59c640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 0x7f185c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.047+0000 7f187a59c640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 0x7f185c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f187c0ce910 tx=0x7f186c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.050+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1864059510 con 0x7f187c0b5390 2024-04-13T03:10:08.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.226+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f187c0af9c0 con 0x7f187c0b5390 2024-04-13T03:10:08.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.226+0000 7f1872ffd640 1 -- 172.21.15.106:0/3545615890 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1864058e10 con 0x7f187c0b5390 2024-04-13T03:10:08.227 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:08.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 msgr2=0x7f185c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 0x7f185c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f187c0ce910 tx=0x7f186c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:08.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 msgr2=0x7f187c14fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f186400c400 tx=0x7f186400c8c0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 shutdown_connections 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185c06c8f0 0x7f185c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f187c15b4e0 0x7f187c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c0b5390 0x7f187c14fe00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 --2- 172.21.15.106:0/3545615890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f187c0b4730 0x7f187c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 >> 172.21.15.106:0/3545615890 conn(0x7f187c0b2600 msgr2=0x7f187c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 shutdown_connections 2024-04-13T03:10:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.228+0000 7f1880c0f640 1 -- 172.21.15.106:0/3545615890 wait complete. 2024-04-13T03:10:08.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:08.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.343+0000 7fdd3431e640 1 Processor -- start 2024-04-13T03:10:08.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.343+0000 7fdd3431e640 1 -- start start 2024-04-13T03:10:08.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 0x7fdd2c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b51c0 0x7fdd2c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c0b3690 con 0x7fdd2c0b4730 2024-04-13T03:10:08.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c0b3970 con 0x7fdd2c15b4e0 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd3431e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c0b3c50 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54184/0 (socket says 172.21.15.106:54184) 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 -- 172.21.15.106:0/1896866608 learned_addr learned my addr 172.21.15.106:0/1896866608 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd320ba640 1 --2- 172.21.15.106:0/1896866608 >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 0x7fdd2c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd328bb640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b51c0 0x7fdd2c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 -- 172.21.15.106:0/1896866608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b51c0 msgr2=0x7fdd2c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b51c0 0x7fdd2c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 -- 172.21.15.106:0/1896866608 >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 msgr2=0x7fdd2c15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 --2- 172.21.15.106:0/1896866608 >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 0x7fdd2c15fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd318b9640 1 -- 172.21.15.106:0/1896866608 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0b3f30 con 0x7fdd2c0b4730 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.344+0000 7fdd320ba640 1 --2- 172.21.15.106:0/1896866608 >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 0x7fdd2c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.345+0000 7fdd318b9640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b2ee0 tx=0x7fdd1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e18783a589962fe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.345+0000 7fdd310b8640 1 -- 172.21.15.106:0/1896866608 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c035020 con 0x7fdd2c0b4730 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.345+0000 7fdd310b8640 1 -- 172.21.15.106:0/1896866608 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c031870 con 0x7fdd2c0b4730 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.345+0000 7fdd310b8640 1 -- 172.21.15.106:0/1896866608 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031c40 con 0x7fdd2c0b4730 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 -- 172.21.15.106:0/1896866608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b4730 msgr2=0x7fdd2c0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b2ee0 tx=0x7fdd1c02b910 comp rx=0 tx=0).stop 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 -- 172.21.15.106:0/1896866608 shutdown_connections 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b51c0 0x7fdd2c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 --2- 172.21.15.106:0/1896866608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b4730 0x7fdd2c0b4c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 --2- 172.21.15.106:0/1896866608 >> v2:172.21.15.114:3300/0 conn(0x7fdd2c15b4e0 0x7fdd2c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 -- 172.21.15.106:0/1896866608 >> 172.21.15.106:0/1896866608 conn(0x7fdd2c0b2600 msgr2=0x7fdd2c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 -- 172.21.15.106:0/1896866608 shutdown_connections 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.346+0000 7fdd3431e640 1 -- 172.21.15.106:0/1896866608 wait complete. 2024-04-13T03:10:08.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.347+0000 7fdd3431e640 1 Processor -- start 2024-04-13T03:10:08.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.347+0000 7fdd3431e640 1 -- start start 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b4730 0x7fdd2c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd2c15b4e0 0x7fdd2c1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c06d0c0 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0b32d0 con 0x7fdd2c15b4e0 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c1f7140 con 0x7fdd2c0b4730 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54194/0 (socket says 172.21.15.106:54194) 2024-04-13T03:10:08.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 -- 172.21.15.106:0/4268401099 learned_addr learned my addr 172.21.15.106:0/4268401099 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 -- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b4730 msgr2=0x7fdd2c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b4730 0x7fdd2c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 -- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd2c15b4e0 msgr2=0x7fdd2c1f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd2c15b4e0 0x7fdd2c1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 -- 172.21.15.106:0/4268401099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c1f6e90 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd318b9640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c02b950 tx=0x7fdd1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c040070 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c030c80 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c149420 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c035040 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.348+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c148940 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.349+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c068a10 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.350+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd1c04b060 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.350+0000 7fdd22ffd640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 0x7fdd1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.350+0000 7fdd320ba640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 0x7fdd1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.350+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdd1c0adcc0 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.351+0000 7fdd320ba640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 0x7fdd1006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006fd0 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.355+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd1c02f320 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.529+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd2c0af9c0 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.530+0000 7fdd22ffd640 1 -- 172.21.15.106:0/4268401099 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdd1c07bcf0 con 0x7fdd2c0b51c0 2024-04-13T03:10:08.531 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 msgr2=0x7fdd1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 0x7fdd1006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006fd0 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 msgr2=0x7fdd2c1f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c02b950 tx=0x7fdd1c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 shutdown_connections 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd1006c8f0 0x7fdd1006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd2c15b4e0 0x7fdd2c1f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2c0b51c0 0x7fdd2c1f6410 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 --2- 172.21.15.106:0/4268401099 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd2c0b4730 0x7fdd2c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 >> 172.21.15.106:0/4268401099 conn(0x7fdd2c0b2600 msgr2=0x7fdd2c15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 shutdown_connections 2024-04-13T03:10:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.532+0000 7fdd3431e640 1 -- 172.21.15.106:0/4268401099 wait complete. 2024-04-13T03:10:08.543 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:08.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 Processor -- start 2024-04-13T03:10:08.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 -- start start 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 0x7f759c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f759c06ebf0 0x7f759c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759c15d690 con 0x7f759c15d110 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f759c06b8e0 con 0x7f759c06ebf0 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.646+0000 7f75a2f5e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f759c06bea0 con 0x7f759c156b40 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f7593fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f759c06ebf0 0x7f759c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f75a14fb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f759c15d110 0x7f759c06c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42622/0 (socket says 172.21.15.106:42622) 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f75a14fb640 1 -- 172.21.15.106:0/2950395023 learned_addr learned my addr 172.21.15.106:0/2950395023 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f75a0cfa640 1 --2- 172.21.15.106:0/2950395023 >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 0x7f759c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f7593fff640 1 -- 172.21.15.106:0/2950395023 >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 msgr2=0x7f759c06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f7593fff640 1 --2- 172.21.15.106:0/2950395023 >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 0x7f759c06e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f7593fff640 1 -- 172.21.15.106:0/2950395023 >> v1:172.21.15.106:6789/0 conn(0x7f759c15d110 legacy=0x7f759c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f7593fff640 1 -- 172.21.15.106:0/2950395023 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c15d970 con 0x7f759c06ebf0 2024-04-13T03:10:08.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.647+0000 7f75a0cfa640 1 --2- 172.21.15.106:0/2950395023 >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 0x7f759c06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f7593fff640 1 --2- 172.21.15.106:0/2950395023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c15cbd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f759c070380 tx=0x7f758c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330598881cac66f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75937fe640 1 -- 172.21.15.106:0/2950395023 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c035020 con 0x7f759c06ebf0 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75937fe640 1 -- 172.21.15.106:0/2950395023 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758c031830 con 0x7f759c06ebf0 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75937fe640 1 -- 172.21.15.106:0/2950395023 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c031c00 con 0x7f759c06ebf0 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 -- 172.21.15.106:0/2950395023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 msgr2=0x7f759c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/2950395023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c15cbd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f759c070380 tx=0x7f758c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 -- 172.21.15.106:0/2950395023 shutdown_connections 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/2950395023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c15cbd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/2950395023 >> v2:172.21.15.114:3300/0 conn(0x7f759c156b40 0x7f759c06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 -- 172.21.15.106:0/2950395023 >> 172.21.15.106:0/2950395023 conn(0x7f759c0cd6a0 msgr2=0x7f759c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 -- 172.21.15.106:0/2950395023 shutdown_connections 2024-04-13T03:10:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.648+0000 7f75a2f5e640 1 -- 172.21.15.106:0/2950395023 wait complete. 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.649+0000 7f75a2f5e640 1 Processor -- start 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.649+0000 7f75a2f5e640 1 -- start start 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f759c156b40 0x7f759c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f759c15d110 0x7f759c1f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06d880 con 0x7f759c06ebf0 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f759c156f20 con 0x7f759c156b40 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a2f5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f759c06da00 con 0x7f759c15d110 2024-04-13T03:10:08.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a0cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a0cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54220/0 (socket says 172.21.15.106:54220) 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.650+0000 7f75a0cfa640 1 -- 172.21.15.106:0/3484201257 learned_addr learned my addr 172.21.15.106:0/3484201257 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a14fb640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f759c15d110 0x7f759c1f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f7593fff640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f759c156b40 0x7f759c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 -- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f759c15d110 msgr2=0x7f759c1f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f759c15d110 0x7f759c1f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 -- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f759c156b40 msgr2=0x7f759c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f759c156b40 0x7f759c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 -- 172.21.15.106:0/3484201257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c1e7000 con 0x7f759c06ebf0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a0cfa640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f759800c7f0 tx=0x7f759800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598015070 con 0x7f759c06ebf0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7598007a30 con 0x7f759c06ebf0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c14edc0 con 0x7f759c06ebf0 2024-04-13T03:10:08.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598007e00 con 0x7f759c06ebf0 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.651+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1e5fe0 con 0x7f759c06ebf0 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.652+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c15edf0 con 0x7f759c06ebf0 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.652+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7598018070 con 0x7f759c06ebf0 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.653+0000 7f75917fa640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 0x7f758806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.653+0000 7f7593fff640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 0x7f758806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.653+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f759808bf90 con 0x7f759c06ebf0 2024-04-13T03:10:08.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.653+0000 7f7593fff640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 0x7f758806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f758c036000 tx=0x7f758c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.657+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f759c15edf0 con 0x7f759c06ebf0 2024-04-13T03:10:08.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.832+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f759c0ce300 con 0x7f759c06ebf0 2024-04-13T03:10:08.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.832+0000 7f75917fa640 1 -- 172.21.15.106:0/3484201257 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f7598059f40 con 0x7f759c06ebf0 2024-04-13T03:10:08.833 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:08.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 msgr2=0x7f758806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 0x7f758806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f758c036000 tx=0x7f758c008030 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 msgr2=0x7f759c1e6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f759800c7f0 tx=0x7f759800cbf0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 shutdown_connections 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f758806c8f0 0x7f758806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f759c15d110 0x7f759c1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f759c156b40 0x7f759c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 --2- 172.21.15.106:0/3484201257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c06ebf0 0x7f759c1e6ac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 >> 172.21.15.106:0/3484201257 conn(0x7f759c0cd6a0 msgr2=0x7f759c06f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 shutdown_connections 2024-04-13T03:10:08.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.834+0000 7f75a2f5e640 1 -- 172.21.15.106:0/3484201257 wait complete. 2024-04-13T03:10:08.846 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:08.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.947+0000 7f87bd4c8640 1 Processor -- start 2024-04-13T03:10:08.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.947+0000 7f87bd4c8640 1 -- start start 2024-04-13T03:10:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87bd4c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87b815ed90 0x7f87b80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87bd4c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 0x7f87b80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87bd4c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b80cfd20 con 0x7f87b815e9b0 2024-04-13T03:10:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87bd4c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f87b8162af0 con 0x7f87b815ed90 2024-04-13T03:10:08.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87bd4c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f87b8162dd0 con 0x7f87b80b3d60 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87affff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87b815ed90 0x7f87b80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87b6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f87b815e9b0 0x7f87b80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42626/0 (socket says 172.21.15.106:42626) 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87b6ffd640 1 -- 172.21.15.106:0/2563583999 learned_addr learned my addr 172.21.15.106:0/2563583999 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87b77fe640 1 --2- 172.21.15.106:0/2563583999 >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 0x7f87b80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87affff640 1 -- 172.21.15.106:0/2563583999 >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 msgr2=0x7f87b80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87affff640 1 --2- 172.21.15.106:0/2563583999 >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 0x7f87b80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87affff640 1 -- 172.21.15.106:0/2563583999 >> v1:172.21.15.106:6789/0 conn(0x7f87b815e9b0 legacy=0x7f87b80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87affff640 1 -- 172.21.15.106:0/2563583999 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b81630b0 con 0x7f87b815ed90 2024-04-13T03:10:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.948+0000 7f87b77fe640 1 --2- 172.21.15.106:0/2563583999 >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 0x7f87b80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:08.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.949+0000 7f87affff640 1 --2- 172.21.15.106:0/2563583999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815ed90 0x7f87b80d2300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cb7b0 tx=0x7f87a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ffd34df916b810 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.949+0000 7f87b67fc640 1 -- 172.21.15.106:0/2563583999 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0034070 con 0x7f87b815ed90 2024-04-13T03:10:08.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.949+0000 7f87b67fc640 1 -- 172.21.15.106:0/2563583999 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a0030610 con 0x7f87b815ed90 2024-04-13T03:10:08.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.949+0000 7f87b67fc640 1 -- 172.21.15.106:0/2563583999 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a00309e0 con 0x7f87b815ed90 2024-04-13T03:10:08.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2563583999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815ed90 msgr2=0x7f87b80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2563583999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815ed90 0x7f87b80d2300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cb7b0 tx=0x7f87a002b960 comp rx=0 tx=0).stop 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2563583999 shutdown_connections 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2563583999 >> v2:172.21.15.114:3300/0 conn(0x7f87b80b3d60 0x7f87b80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2563583999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815ed90 0x7f87b80d2300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2563583999 >> 172.21.15.106:0/2563583999 conn(0x7f87b808bf10 msgr2=0x7f87b80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2563583999 shutdown_connections 2024-04-13T03:10:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.950+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2563583999 wait complete. 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.951+0000 7f87bd4c8640 1 Processor -- start 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.951+0000 7f87bd4c8640 1 -- start start 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87b80b3d60 0x7f87b815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b815f6c0 0x7f87b8161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8162800 con 0x7f87b815e9b0 2024-04-13T03:10:08.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8162680 con 0x7f87b815f6c0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f87b80d4d00 con 0x7f87b80b3d60 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54236/0 (socket says 172.21.15.106:54236) 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 -- 172.21.15.106:0/2682614002 learned_addr learned my addr 172.21.15.106:0/2682614002 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 -- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87b80b3d60 msgr2=0x7f87b815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87b80b3d60 0x7f87b815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 -- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b815f6c0 msgr2=0x7f87b8161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b815f6c0 0x7f87b8161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 -- 172.21.15.106:0/2682614002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b80c91c0 con 0x7f87b815e9b0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87affff640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009880 tx=0x7f87a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0034040 con 0x7f87b815e9b0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a003aca0 con 0x7f87b815e9b0 2024-04-13T03:10:08.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0005bd0 con 0x7f87b815e9b0 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b80c94a0 con 0x7f87b815e9b0 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.952+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b80c9a60 con 0x7f87b815e9b0 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.953+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b80d1b40 con 0x7f87b815e9b0 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.954+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87a0037070 con 0x7f87b815e9b0 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.954+0000 7f87af7fe640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:08.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.954+0000 7f87b6ffd640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:08.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.954+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f87a00adee0 con 0x7f87b815e9b0 2024-04-13T03:10:08.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.955+0000 7f87b6ffd640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 0x7f879806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000f60 tx=0x7f87a8007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:08.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:08.959+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87b80d1b40 con 0x7f87b815e9b0 2024-04-13T03:10:09.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.139+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f87b80d1d40 con 0x7f87b815e9b0 2024-04-13T03:10:09.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.139+0000 7f87af7fe640 1 -- 172.21.15.106:0/2682614002 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f87b80d1d40 con 0x7f87b815e9b0 2024-04-13T03:10:09.140 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:09.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 msgr2=0x7f879806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 0x7f879806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000f60 tx=0x7f87a8007a80 comp rx=0 tx=0).stop 2024-04-13T03:10:09.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 msgr2=0x7f87b8161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009880 tx=0x7f87a00046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 shutdown_connections 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f879806c8f0 0x7f879806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b815f6c0 0x7f87b8161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b815e9b0 0x7f87b8161610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 --2- 172.21.15.106:0/2682614002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87b80b3d60 0x7f87b815f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 >> 172.21.15.106:0/2682614002 conn(0x7f87b808bf10 msgr2=0x7f87b80b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 shutdown_connections 2024-04-13T03:10:09.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.141+0000 7f87bd4c8640 1 -- 172.21.15.106:0/2682614002 wait complete. 2024-04-13T03:10:09.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:09.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.252+0000 7f67becc8640 1 Processor -- start 2024-04-13T03:10:09.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.252+0000 7f67becc8640 1 -- start start 2024-04-13T03:10:09.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67becc8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67b80b68d0 0x7f67b80ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67becc8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b81624b0 0x7f67b80d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67becc8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b815eda0 con 0x7f67b815e9c0 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67becc8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f67b80d27f0 con 0x7f67b80b68d0 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67becc8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f67b8162c20 con 0x7f67b81624b0 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67b80b68d0 0x7f67b80ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f67b80b68d0 0x7f67b80ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54242/0 (socket says 172.21.15.106:54242) 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 -- 172.21.15.106:0/2806223941 learned_addr learned my addr 172.21.15.106:0/2806223941 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 -- 172.21.15.106:0/2806223941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b81624b0 msgr2=0x7f67b80d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 --2- 172.21.15.106:0/2806223941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b81624b0 0x7f67b80d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 -- 172.21.15.106:0/2806223941 >> v1:172.21.15.114:6789/0 conn(0x7f67b815e9c0 legacy=0x7f67b80b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.253+0000 7f67b7fff640 1 -- 172.21.15.106:0/2806223941 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b8162f00 con 0x7f67b80b68d0 2024-04-13T03:10:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67b7fff640 1 --2- 172.21.15.106:0/2806223941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b80b68d0 0x7f67b80ba290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009870 tx=0x7f67a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff1638e21bad03d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67b77fe640 1 -- 172.21.15.106:0/2806223941 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4035020 con 0x7f67b80b68d0 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67b77fe640 1 -- 172.21.15.106:0/2806223941 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a4031640 con 0x7f67b80b68d0 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67b77fe640 1 -- 172.21.15.106:0/2806223941 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a402d040 con 0x7f67b80b68d0 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67becc8640 1 -- 172.21.15.106:0/2806223941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b80b68d0 msgr2=0x7f67b80ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.254+0000 7f67becc8640 1 --2- 172.21.15.106:0/2806223941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b80b68d0 0x7f67b80ba290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009870 tx=0x7f67a402b8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 -- 172.21.15.106:0/2806223941 shutdown_connections 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 --2- 172.21.15.106:0/2806223941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b81624b0 0x7f67b80d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 --2- 172.21.15.106:0/2806223941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b80b68d0 0x7f67b80ba290 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 -- 172.21.15.106:0/2806223941 >> 172.21.15.106:0/2806223941 conn(0x7f67b80d11c0 msgr2=0x7f67b80d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 -- 172.21.15.106:0/2806223941 shutdown_connections 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.255+0000 7f67becc8640 1 -- 172.21.15.106:0/2806223941 wait complete. 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.256+0000 7f67becc8640 1 Processor -- start 2024-04-13T03:10:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.256+0000 7f67becc8640 1 -- start start 2024-04-13T03:10:09.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b80b68d0 0x7f67b8160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67b815e9c0 0x7f67b80cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8162940 con 0x7f67b81624b0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67b80d2a40 con 0x7f67b815e9c0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f67b80d34a0 con 0x7f67b80b68d0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54252/0 (socket says 172.21.15.106:54252) 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 -- 172.21.15.106:0/1721173059 learned_addr learned my addr 172.21.15.106:0/1721173059 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bca64640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b80b68d0 0x7f67b8160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67b7fff640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67b815e9c0 0x7f67b80cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 -- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b80b68d0 msgr2=0x7f67b8160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b80b68d0 0x7f67b8160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 -- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67b815e9c0 msgr2=0x7f67b80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67b815e9c0 0x7f67b80cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 -- 172.21.15.106:0/1721173059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b8160b20 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67bd265640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67b000b9e0 tx=0x7f67b000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0019070 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b000cac0 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0014440 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b8160e00 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.257+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81613c0 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.258+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b8068a10 con 0x7f67b81624b0 2024-04-13T03:10:09.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.259+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67b001c070 con 0x7f67b81624b0 2024-04-13T03:10:09.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.259+0000 7f67b57fa640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.259+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f67b008bb80 con 0x7f67b81624b0 2024-04-13T03:10:09.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.259+0000 7f67bca64640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.260+0000 7f67bca64640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 0x7f679c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67b815c490 tx=0x7f67ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.264+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67b0059c40 con 0x7f67b81624b0 2024-04-13T03:10:09.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.438+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67b80c99a0 con 0x7f67b81624b0 2024-04-13T03:10:09.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.440+0000 7f67b57fa640 1 -- 172.21.15.106:0/1721173059 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f67b0059540 con 0x7f67b81624b0 2024-04-13T03:10:09.441 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:09.441 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 msgr2=0x7f679c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 0x7f679c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67b815c490 tx=0x7f67ac009040 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 msgr2=0x7f67b80cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67b000b9e0 tx=0x7f67b000bea0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 shutdown_connections 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f679c06c8f0 0x7f679c06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67b81624b0 0x7f67b80cf4e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67b815e9c0 0x7f67b80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 --2- 172.21.15.106:0/1721173059 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f67b80b68d0 0x7f67b8160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 >> 172.21.15.106:0/1721173059 conn(0x7f67b80d11c0 msgr2=0x7f67b80b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 shutdown_connections 2024-04-13T03:10:09.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.442+0000 7f67becc8640 1 -- 172.21.15.106:0/1721173059 wait complete. 2024-04-13T03:10:09.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:09.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.556+0000 7fec8c4c4640 1 Processor -- start 2024-04-13T03:10:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.556+0000 7fec8c4c4640 1 -- start start 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8c4c4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fec840cf480 0x7fec8415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8c4c4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec840d3220 0x7fec840d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8c4c4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8415f380 con 0x7fec840ba170 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8c4c4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fec840d2d90 con 0x7fec840d3220 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8c4c4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fec8415f600 con 0x7fec840cf480 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8aa61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec840d3220 0x7fec840d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8aa61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec840d3220 0x7fec840d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54256/0 (socket says 172.21.15.106:54256) 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8aa61640 1 -- 172.21.15.106:0/3907423920 learned_addr learned my addr 172.21.15.106:0/3907423920 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.557+0000 7fec8a260640 1 --2- 172.21.15.106:0/3907423920 >> v2:172.21.15.114:3300/0 conn(0x7fec840cf480 0x7fec8415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8aa61640 1 -- 172.21.15.106:0/3907423920 >> v2:172.21.15.114:3300/0 conn(0x7fec840cf480 msgr2=0x7fec8415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8aa61640 1 --2- 172.21.15.106:0/3907423920 >> v2:172.21.15.114:3300/0 conn(0x7fec840cf480 0x7fec8415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8aa61640 1 -- 172.21.15.106:0/3907423920 >> v1:172.21.15.114:6789/0 conn(0x7fec840ba170 legacy=0x7fec840cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8aa61640 1 -- 172.21.15.106:0/3907423920 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8415f8e0 con 0x7fec840d3220 2024-04-13T03:10:09.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8aa61640 1 --2- 172.21.15.106:0/3907423920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840d3220 0x7fec840d36d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec8406e580 tx=0x7fec7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c7e08353acf82a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.558+0000 7fec8925e640 1 -- 172.21.15.106:0/3907423920 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c034070 con 0x7fec840d3220 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8925e640 1 -- 172.21.15.106:0/3907423920 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c030860 con 0x7fec840d3220 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8925e640 1 -- 172.21.15.106:0/3907423920 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c030c30 con 0x7fec840d3220 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 -- 172.21.15.106:0/3907423920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840d3220 msgr2=0x7fec840d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/3907423920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840d3220 0x7fec840d36d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec8406e580 tx=0x7fec7c02b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 -- 172.21.15.106:0/3907423920 shutdown_connections 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/3907423920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840d3220 0x7fec840d36d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/3907423920 >> v2:172.21.15.114:3300/0 conn(0x7fec840cf480 0x7fec8415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 -- 172.21.15.106:0/3907423920 >> 172.21.15.106:0/3907423920 conn(0x7fec840d47d0 msgr2=0x7fec840d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 -- 172.21.15.106:0/3907423920 shutdown_connections 2024-04-13T03:10:09.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.559+0000 7fec8c4c4640 1 -- 172.21.15.106:0/3907423920 wait complete. 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.560+0000 7fec8c4c4640 1 Processor -- start 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.560+0000 7fec8c4c4640 1 -- start start 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec840cf480 0x7fec841f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 0x7fec840ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec841592f0 con 0x7fec840ba170 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec84159170 con 0x7fec840cf480 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8c4c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec840d3c10 con 0x7fec840d3220 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54262/0 (socket says 172.21.15.106:54262) 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 -- 172.21.15.106:0/2103110546 learned_addr learned my addr 172.21.15.106:0/2103110546 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8aa61640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 0x7fec840ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec89a5f640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec840cf480 0x7fec841f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 -- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 msgr2=0x7fec840ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 0x7fec840ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 -- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec840cf480 msgr2=0x7fec841f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec840cf480 0x7fec841f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8a260640 1 -- 172.21.15.106:0/2103110546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec841e8150 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.561+0000 7fec8aa61640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 0x7fec840ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec8a260640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec80007ef0 tx=0x7fec8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80014620 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec80019070 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec840cb8a0 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80014a00 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.562+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec840cbe60 con 0x7fec840ba170 2024-04-13T03:10:09.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.563+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec84068a10 con 0x7fec840ba170 2024-04-13T03:10:09.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.563+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec80014ba0 con 0x7fec840ba170 2024-04-13T03:10:09.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.563+0000 7fec7affd640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 0x7fec6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.564+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fec8008c840 con 0x7fec840ba170 2024-04-13T03:10:09.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.564+0000 7fec89a5f640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 0x7fec6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.564+0000 7fec89a5f640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 0x7fec6806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec8406e150 tx=0x7fec74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.569+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec8005a870 con 0x7fec840ba170 2024-04-13T03:10:09.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.744+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec84156410 con 0x7fec840ba170 2024-04-13T03:10:09.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.745+0000 7fec7affd640 1 -- 172.21.15.106:0/2103110546 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fec8005a170 con 0x7fec840ba170 2024-04-13T03:10:09.746 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.746+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 msgr2=0x7fec6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.746+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 0x7fec6806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec8406e150 tx=0x7fec74009040 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 msgr2=0x7fec841e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec80007ef0 tx=0x7fec8000e4b0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 shutdown_connections 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec6806c8f0 0x7fec6806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec840d3220 0x7fec840ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec840cf480 0x7fec841f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 --2- 172.21.15.106:0/2103110546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec840ba170 0x7fec841e7ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 >> 172.21.15.106:0/2103110546 conn(0x7fec840d47d0 msgr2=0x7fec841617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 shutdown_connections 2024-04-13T03:10:09.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.747+0000 7fec8c4c4640 1 -- 172.21.15.106:0/2103110546 wait complete. 2024-04-13T03:10:09.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.860+0000 7fb385088640 1 Processor -- start 2024-04-13T03:10:09.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.860+0000 7fb385088640 1 -- start start 2024-04-13T03:10:09.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb385088640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb3800b80f0 0x7fb38006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb385088640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3800d05d0 con 0x7fb38015caf0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb385088640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3800705f0 con 0x7fb3800cd520 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb385088640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb38015dee0 con 0x7fb3800b80f0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb37ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb38015caf0 0x7fb3800cce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42642/0 (socket says 172.21.15.106:42642) 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.861+0000 7fb37ed9d640 1 -- 172.21.15.106:0/2630641418 learned_addr learned my addr 172.21.15.106:0/2630641418 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37f59e640 1 --2- 172.21.15.106:0/2630641418 >> v2:172.21.15.114:3300/0 conn(0x7fb3800b80f0 0x7fb38006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397117750 0 0) 0x7fb3800d05d0 con 0x7fb38015caf0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb368003520 con 0x7fb38015caf0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253893040 0 0) 0x7fb3800705f0 con 0x7fb3800cd520 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3800d05d0 con 0x7fb3800cd520 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2078952483 0 0) 0x7fb368003520 con 0x7fb38015caf0 2024-04-13T03:10:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 >> v2:172.21.15.114:3300/0 conn(0x7fb3800b80f0 msgr2=0x7fb38006fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 --2- 172.21.15.106:0/2630641418 >> v2:172.21.15.114:3300/0 conn(0x7fb3800b80f0 0x7fb38006fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 >> v1:172.21.15.114:6789/0 conn(0x7fb3800cd520 legacy=0x7fb3800d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3800708d0 con 0x7fb38015caf0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb374001670 con 0x7fb38015caf0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 >> v1:172.21.15.106:6789/0 conn(0x7fb38015caf0 legacy=0x7fb3800cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 0x7fb368003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3680051c0 0x7fb36800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb38015dee0 con 0x7fb368003b40 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb36800bfb0 con 0x7fb368004800 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb36800c1b0 con 0x7fb3680051c0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 0x7fb368003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37e59c640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37f59e640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3680051c0 0x7fb36800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 -- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3680051c0 msgr2=0x7fb36800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3680051c0 0x7fb36800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 -- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368004800 msgr2=0x7fb368004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37ed9d640 1 -- 172.21.15.106:0/2630641418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb368003520 con 0x7fb368003b40 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.862+0000 7fb37f59e640 1 -- 172.21.15.106:0/2630641418 reap_dead start 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.863+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 0x7fb368003f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de28d25d9b765d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.863+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374007ce0 con 0x7fb368003b40 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.863+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb374030070 con 0x7fb368003b40 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.863+0000 7fb37dd9b640 1 -- 172.21.15.106:0/2630641418 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374005d50 con 0x7fb368003b40 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 -- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 msgr2=0x7fb368003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 0x7fb368003f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb368005830 comp rx=0 tx=0).stop 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 -- 172.21.15.106:0/2630641418 shutdown_connections 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 --2- 172.21.15.106:0/2630641418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb368003b40 0x7fb368003f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 -- 172.21.15.106:0/2630641418 >> 172.21.15.106:0/2630641418 conn(0x7fb3800d25b0 msgr2=0x7fb3800d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:09.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 -- 172.21.15.106:0/2630641418 shutdown_connections 2024-04-13T03:10:09.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.864+0000 7fb385088640 1 -- 172.21.15.106:0/2630641418 wait complete. 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.865+0000 7fb385088640 1 Processor -- start 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.865+0000 7fb385088640 1 -- start start 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368003b40 0x7fb38014bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb38014c8b0 0x7fb3800c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb38006fea0 con 0x7fb38014c430 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb380070e90 con 0x7fb368003b40 2024-04-13T03:10:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb385088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb380070020 con 0x7fb38014c8b0 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54280/0 (socket says 172.21.15.106:54280) 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 -- 172.21.15.106:0/2063029806 learned_addr learned my addr 172.21.15.106:0/2063029806 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368003b40 0x7fb38014bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37f59e640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb38014c8b0 0x7fb3800c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 -- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb38014c8b0 msgr2=0x7fb3800c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb38014c8b0 0x7fb3800c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 -- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368003b40 msgr2=0x7fb38014bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.866+0000 7fb37e59c640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368003b40 0x7fb38014bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb37e59c640 1 -- 172.21.15.106:0/2063029806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3800c7aa0 con 0x7fb38014c430 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb37e59c640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb37000f7a0 tx=0x7fb37000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001a070 con 0x7fb38014c430 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37000fe50 con 0x7fb38014c430 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3800c7d80 con 0x7fb38014c430 2024-04-13T03:10:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007050 con 0x7fb38014c430 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.867+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3800c8340 con 0x7fb38014c430 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.868+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb380069120 con 0x7fb38014c430 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.868+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb37001d070 con 0x7fb38014c430 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.869+0000 7fb3677fe640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 0x7fb36c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.869+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 0x7fb36c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.869+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb37008b600 con 0x7fb38014c430 2024-04-13T03:10:09.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.869+0000 7fb37ed9d640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 0x7fb36c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb374007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:09.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:09.873+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb370059070 con 0x7fb38014c430 2024-04-13T03:10:10.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.050+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3800b7b60 con 0x7fb38014c430 2024-04-13T03:10:10.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.050+0000 7fb3677fe640 1 -- 172.21.15.106:0/2063029806 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fb37005d370 con 0x7fb38014c430 2024-04-13T03:10:10.051 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 msgr2=0x7fb36c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 0x7fb36c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb374007000 comp rx=0 tx=0).stop 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 msgr2=0x7fb3800c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb37000f7a0 tx=0x7fb37000fc60 comp rx=0 tx=0).stop 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 shutdown_connections 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb36c06c8f0 0x7fb36c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb38014c8b0 0x7fb3800c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb38014c430 0x7fb3800c8b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 --2- 172.21.15.106:0/2063029806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb368003b40 0x7fb38014bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 >> 172.21.15.106:0/2063029806 conn(0x7fb3800d25b0 msgr2=0x7fb3800b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 shutdown_connections 2024-04-13T03:10:10.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.052+0000 7fb385088640 1 -- 172.21.15.106:0/2063029806 wait complete. 2024-04-13T03:10:10.064 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:10.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.166+0000 7fba83ade640 1 Processor -- start 2024-04-13T03:10:10.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.166+0000 7fba83ade640 1 -- start start 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba83ade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba83ade640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba7c0709e0 0x7fba7c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba83ade640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c0b5cf0 con 0x7fba7c0b1c30 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba83ade640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0b5fd0 con 0x7fba7c0709e0 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba83ade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c0b62b0 con 0x7fba7c15ee40 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba7c0709e0 0x7fba7c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba7c0709e0 0x7fba7c06d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54282/0 (socket says 172.21.15.106:54282) 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 -- 172.21.15.106:0/8389675 learned_addr learned my addr 172.21.15.106:0/8389675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8187a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fba7c0b1c30 0x7fba7c0b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42650/0 (socket says 172.21.15.106:42650) 2024-04-13T03:10:10.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba81079640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 -- 172.21.15.106:0/8389675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 msgr2=0x7fba7c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 -- 172.21.15.106:0/8389675 >> v1:172.21.15.106:6789/0 conn(0x7fba7c0b1c30 legacy=0x7fba7c0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba80878640 1 -- 172.21.15.106:0/8389675 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778122310 0 0) 0x7fba7c0b5cf0 con 0x7fba7c0b1c30 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.167+0000 7fba8207b640 1 -- 172.21.15.106:0/8389675 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0b6590 con 0x7fba7c0709e0 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.168+0000 7fba8207b640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c06d660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba74009e80 tx=0x7fba7402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c03210092cc2036 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.168+0000 7fba80878640 1 -- 172.21.15.106:0/8389675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7402caf0 con 0x7fba7c0709e0 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.168+0000 7fba80878640 1 -- 172.21.15.106:0/8389675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7402cc90 con 0x7fba7c0709e0 2024-04-13T03:10:10.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.168+0000 7fba80878640 1 -- 172.21.15.106:0/8389675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74034610 con 0x7fba7c0709e0 2024-04-13T03:10:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 -- 172.21.15.106:0/8389675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 msgr2=0x7fba7c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c06d660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba74009e80 tx=0x7fba7402bf50 comp rx=0 tx=0).stop 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 -- 172.21.15.106:0/8389675 shutdown_connections 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c06d660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 --2- 172.21.15.106:0/8389675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 -- 172.21.15.106:0/8389675 >> 172.21.15.106:0/8389675 conn(0x7fba7c071870 msgr2=0x7fba7c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 -- 172.21.15.106:0/8389675 shutdown_connections 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.169+0000 7fba83ade640 1 -- 172.21.15.106:0/8389675 wait complete. 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 Processor -- start 2024-04-13T03:10:10.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 -- start start 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba7c0b1c30 0x7fba7c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c06df30 con 0x7fba7c0709e0 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c06ddb0 con 0x7fba7c0b1c30 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.170+0000 7fba83ade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c06e0b0 con 0x7fba7c15ee40 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54288/0 (socket says 172.21.15.106:54288) 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 -- 172.21.15.106:0/3091879756 learned_addr learned my addr 172.21.15.106:0/3091879756 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:10.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 -- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 msgr2=0x7fba7c149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 -- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba7c0b1c30 msgr2=0x7fba7c1f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba7c0b1c30 0x7fba7c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 -- 172.21.15.106:0/3091879756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0cde00 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba8187a640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c4d0 tx=0x7fba7800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78018070 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba78007500 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78004a20 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c0ce0e0 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.171+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c0ce6a0 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.172+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c068a10 con 0x7fba7c0709e0 2024-04-13T03:10:10.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.172+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba7801b070 con 0x7fba7c0709e0 2024-04-13T03:10:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.173+0000 7fba727fc640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 0x7fba6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.173+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fba7808b200 con 0x7fba7c0709e0 2024-04-13T03:10:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.173+0000 7fba81079640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 0x7fba6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.173+0000 7fba81079640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 0x7fba6006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.177+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba780592c0 con 0x7fba7c0709e0 2024-04-13T03:10:10.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.357+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba7c0711f0 con 0x7fba7c0709e0 2024-04-13T03:10:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.358+0000 7fba727fc640 1 -- 172.21.15.106:0/3091879756 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fba78058bc0 con 0x7fba7c0709e0 2024-04-13T03:10:10.359 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 msgr2=0x7fba6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 0x7fba6006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 msgr2=0x7fba7c0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c4d0 tx=0x7fba7800c990 comp rx=0 tx=0).stop 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 shutdown_connections 2024-04-13T03:10:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba6006c8f0 0x7fba6006edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba7c15ee40 0x7fba7c149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba7c0b1c30 0x7fba7c1f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 --2- 172.21.15.106:0/3091879756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba7c0709e0 0x7fba7c0cba90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 >> 172.21.15.106:0/3091879756 conn(0x7fba7c071870 msgr2=0x7fba7c15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 shutdown_connections 2024-04-13T03:10:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.360+0000 7fba83ade640 1 -- 172.21.15.106:0/3091879756 wait complete. 2024-04-13T03:10:10.372 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:10.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.474+0000 7f634515e640 1 Processor -- start 2024-04-13T03:10:10.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.474+0000 7f634515e640 1 -- start start 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f634515e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f63400ce9e0 0x7f63400b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f634515e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63400d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f634515e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63400cd240 con 0x7f634015e820 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f634515e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f63400cd520 con 0x7f63400ce9e0 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f634515e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f63400cd800 con 0x7f63400b5ad0 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f633ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f634015e820 0x7f63400b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42666/0 (socket says 172.21.15.106:42666) 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f633ed9d640 1 -- 172.21.15.106:0/3627629110 learned_addr learned my addr 172.21.15.106:0/3627629110 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f633f59e640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63400d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f633e59c640 1 --2- 172.21.15.106:0/3627629110 >> v2:172.21.15.114:3300/0 conn(0x7f63400ce9e0 0x7f63400b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.475+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451794736 0 0) 0x7f63400cd240 con 0x7f634015e820 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6328003520 con 0x7f634015e820 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1121248903 0 0) 0x7f6328003520 con 0x7f634015e820 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 msgr2=0x7f63400d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63400d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 >> v2:172.21.15.114:3300/0 conn(0x7f63400ce9e0 msgr2=0x7f63400b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 --2- 172.21.15.106:0/3627629110 >> v2:172.21.15.114:3300/0 conn(0x7f63400ce9e0 0x7f63400b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63400cdae0 con 0x7f634015e820 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f63300019f0 con 0x7f634015e820 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 >> v1:172.21.15.106:6789/0 conn(0x7f634015e820 legacy=0x7f63400b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63400cd520 con 0x7f63280039b0 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63400cd800 con 0x7f6328004800 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f632800ba10 con 0x7f63400b5ad0 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633f59e640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633e59c640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 -- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 msgr2=0x7f63280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f63400b5ad0 0x7f63280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 -- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6328004800 msgr2=0x7f6328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633ed9d640 1 -- 172.21.15.106:0/3627629110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6328003520 con 0x7f63280039b0 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.476+0000 7f633e59c640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f6328003dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f633000a890 tx=0x7f6328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f51352a3fa9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633000a480 con 0x7f63280039b0 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6330034630 con 0x7f63280039b0 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f633dd9b640 1 -- 172.21.15.106:0/3627629110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330034a00 con 0x7f63280039b0 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f634515e640 1 -- 172.21.15.106:0/3627629110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 msgr2=0x7f6328003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.477+0000 7f634515e640 1 --2- 172.21.15.106:0/3627629110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f6328003dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f633000a890 tx=0x7f6328005830 comp rx=0 tx=0).stop 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.478+0000 7f633f59e640 1 -- 172.21.15.106:0/3627629110 reap_dead start 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.478+0000 7f634515e640 1 -- 172.21.15.106:0/3627629110 shutdown_connections 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.478+0000 7f634515e640 1 -- 172.21.15.106:0/3627629110 >> 172.21.15.106:0/3627629110 conn(0x7f63400d2510 msgr2=0x7f63400d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.478+0000 7f634515e640 1 -- 172.21.15.106:0/3627629110 shutdown_connections 2024-04-13T03:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.478+0000 7f634515e640 1 -- 172.21.15.106:0/3627629110 wait complete. 2024-04-13T03:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.479+0000 7f634515e640 1 Processor -- start 2024-04-13T03:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.479+0000 7f634515e640 1 -- start start 2024-04-13T03:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6340070ea0 0x7f634015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63400c89a0 0x7f63400c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63400ccf50 con 0x7f63280039b0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63400ccdd0 con 0x7f63400c89a0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f634015ec00 con 0x7f6340070ea0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54314/0 (socket says 172.21.15.106:54314) 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 -- 172.21.15.106:0/3071388073 learned_addr learned my addr 172.21.15.106:0/3071388073 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 -- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6340070ea0 msgr2=0x7f634015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6340070ea0 0x7f634015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 -- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63400c89a0 msgr2=0x7f63400c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63400c89a0 0x7f63400c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 -- 172.21.15.106:0/3071388073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634006d5f0 con 0x7f63280039b0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f633ed9d640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f633000a830 tx=0x7f63400cd410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633000a480 con 0x7f63280039b0 2024-04-13T03:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6330044070 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634006f9f0 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.480+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330034c10 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.481+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634006ffb0 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.481+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634006f810 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.482+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6330049460 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.482+0000 7f63277fe640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.482+0000 7f633e59c640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.482+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f63300ae4f0 con 0x7f63280039b0 2024-04-13T03:10:10.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.483+0000 7f633e59c640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 0x7f632c06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63401ea640 tx=0x7f6334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.487+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f634006f810 con 0x7f63280039b0 2024-04-13T03:10:10.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.659+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6340069120 con 0x7f63280039b0 2024-04-13T03:10:10.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.659+0000 7f63277fe640 1 -- 172.21.15.106:0/3071388073 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f633007c520 con 0x7f63280039b0 2024-04-13T03:10:10.661 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 msgr2=0x7f632c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 0x7f632c06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63401ea640 tx=0x7f6334008040 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 msgr2=0x7f63400cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f633000a830 tx=0x7f63400cd410 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 shutdown_connections 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6340070ea0 0x7f634015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63400c89a0 0x7f63400c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 --2- 172.21.15.106:0/3071388073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63280039b0 0x7f63400cc100 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 >> 172.21.15.106:0/3071388073 conn(0x7f63400d2510 msgr2=0x7f6340160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 shutdown_connections 2024-04-13T03:10:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.661+0000 7f634515e640 1 -- 172.21.15.106:0/3071388073 wait complete. 2024-04-13T03:10:10.673 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.774+0000 7fa2a4fbb640 1 Processor -- start 2024-04-13T03:10:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.774+0000 7fa2a4fbb640 1 -- start start 2024-04-13T03:10:10.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa2a4fbb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa2a015e980 0x7fa2a0162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa2a4fbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 0x7fa2a00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa2a4fbb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a015fe40 con 0x7fa2a00cd380 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa2a4fbb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a00cd7f0 con 0x7fa2a015e980 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa2a4fbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a01633f0 con 0x7fa2a015ed60 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa2a00cd380 0x7fa2a00d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42678/0 (socket says 172.21.15.106:42678) 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29ed9d640 1 -- 172.21.15.106:0/3240754960 learned_addr learned my addr 172.21.15.106:0/3240754960 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29dd9b640 1 --2- 172.21.15.106:0/3240754960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 0x7fa2a00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29e59c640 1 --2- 172.21.15.106:0/3240754960 >> v2:172.21.15.114:3300/0 conn(0x7fa2a015e980 0x7fa2a0162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123833177 0 0) 0x7fa2a015fe40 con 0x7fa2a00cd380 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.775+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa288003520 con 0x7fa2a00cd380 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114030987 0 0) 0x7fa288003520 con 0x7fa2a00cd380 2024-04-13T03:10:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29dd9b640 1 -- 172.21.15.106:0/3240754960 >> v2:172.21.15.114:3300/0 conn(0x7fa2a015e980 msgr2=0x7fa2a0162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29dd9b640 1 --2- 172.21.15.106:0/3240754960 >> v2:172.21.15.114:3300/0 conn(0x7fa2a015e980 0x7fa2a0162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29dd9b640 1 -- 172.21.15.106:0/3240754960 >> v1:172.21.15.106:6789/0 conn(0x7fa2a00cd380 legacy=0x7fa2a00d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29dd9b640 1 -- 172.21.15.106:0/3240754960 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a015f2c0 con 0x7fa2a015ed60 2024-04-13T03:10:10.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.776+0000 7fa29dd9b640 1 --2- 172.21.15.106:0/3240754960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 0x7fa2a00cce40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097a0 tx=0x7fa29002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417beda85870f454 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290038070 con 0x7fa2a015ed60 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290031500 con 0x7fa2a015ed60 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa29d59a640 1 -- 172.21.15.106:0/3240754960 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980019f0 con 0x7fa2a015ed60 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3240754960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 msgr2=0x7fa2a00cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3240754960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 0x7fa2a00cce40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097a0 tx=0x7fa29002b820 comp rx=0 tx=0).stop 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3240754960 shutdown_connections 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3240754960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015ed60 0x7fa2a00cce40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3240754960 >> v2:172.21.15.114:3300/0 conn(0x7fa2a015e980 0x7fa2a0162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.777+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3240754960 >> 172.21.15.106:0/3240754960 conn(0x7fa2a00d2550 msgr2=0x7fa2a00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.778+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3240754960 shutdown_connections 2024-04-13T03:10:10.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.778+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3240754960 wait complete. 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.779+0000 7fa2a4fbb640 1 Processor -- start 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.779+0000 7fa2a4fbb640 1 -- start start 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2a00cd380 0x7fa2a00c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015e980 0x7fa2a01f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a015fb60 con 0x7fa2a015ed60 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a015f880 con 0x7fa2a00cd380 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa2a4fbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a00d1e20 con 0x7fa2a015e980 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54318/0 (socket says 172.21.15.106:54318) 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 -- 172.21.15.106:0/3042371802 learned_addr learned my addr 172.21.15.106:0/3042371802 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29e59c640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2a00cd380 0x7fa2a00c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29dd9b640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015e980 0x7fa2a01f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 -- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015e980 msgr2=0x7fa2a01f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015e980 0x7fa2a01f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 -- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2a00cd380 msgr2=0x7fa2a00c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2a00cd380 0x7fa2a00c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 -- 172.21.15.106:0/3042371802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00cbc30 con 0x7fa2a015ed60 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.780+0000 7fa29ed9d640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa298001890 tx=0x7fa29800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.781+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298012400 con 0x7fa2a015ed60 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.781+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2980125c0 con 0x7fa2a015ed60 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.781+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980129b0 con 0x7fa2a015ed60 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.781+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a00cbf10 con 0x7fa2a015ed60 2024-04-13T03:10:10.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.781+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a00cc4d0 con 0x7fa2a015ed60 2024-04-13T03:10:10.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.782+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a0069120 con 0x7fa2a015ed60 2024-04-13T03:10:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.782+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa298012b50 con 0x7fa2a015ed60 2024-04-13T03:10:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.783+0000 7fa28effd640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.783+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa29808c750 con 0x7fa2a015ed60 2024-04-13T03:10:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.783+0000 7fa29e59c640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:10.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.783+0000 7fa29e59c640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 0x7fa28006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa294009b40 tx=0x7fa294009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:10.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.787+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa29805a0a0 con 0x7fa2a015ed60 2024-04-13T03:10:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.965+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2a00703d0 con 0x7fa2a015ed60 2024-04-13T03:10:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.966+0000 7fa28effd640 1 -- 172.21.15.106:0/3042371802 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fa29805e3a0 con 0x7fa2a015ed60 2024-04-13T03:10:10.968 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:10.968 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 msgr2=0x7fa28006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 0x7fa28006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa294009b40 tx=0x7fa294009190 comp rx=0 tx=0).stop 2024-04-13T03:10:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 msgr2=0x7fa2a01f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa298001890 tx=0x7fa29800a430 comp rx=0 tx=0).stop 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 shutdown_connections 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa28006c8f0 0x7fa28006edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2a015e980 0x7fa2a01f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a015ed60 0x7fa2a01f85a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 --2- 172.21.15.106:0/3042371802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2a00cd380 0x7fa2a00c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 >> 172.21.15.106:0/3042371802 conn(0x7fa2a00d2550 msgr2=0x7fa2a015c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 shutdown_connections 2024-04-13T03:10:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:10.968+0000 7fa2a4fbb640 1 -- 172.21.15.106:0/3042371802 wait complete. 2024-04-13T03:10:10.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:10.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:11.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.083+0000 7f61428b7640 1 Processor -- start 2024-04-13T03:10:11.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.083+0000 7f61428b7640 1 -- start start 2024-04-13T03:10:11.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f61428b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f613c06e740 0x7f613c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f61428b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f613c15c7a0 0x7f613c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f61428b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613c06d180 con 0x7f613c1592b0 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f61428b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f613c0ce180 con 0x7f613c06e740 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f61428b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f613c06ec90 con 0x7f613c15c7a0 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f613b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f613c06e740 0x7f613c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f613b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f613c06e740 0x7f613c15c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54326/0 (socket says 172.21.15.106:54326) 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f613b7fe640 1 -- 172.21.15.106:0/1948248505 learned_addr learned my addr 172.21.15.106:0/1948248505 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.084+0000 7f6140e54640 1 --2- 172.21.15.106:0/1948248505 >> v2:172.21.15.114:3300/0 conn(0x7f613c15c7a0 0x7f613c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.085+0000 7f613b7fe640 1 -- 172.21.15.106:0/1948248505 >> v2:172.21.15.114:3300/0 conn(0x7f613c15c7a0 msgr2=0x7f613c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.085+0000 7f613b7fe640 1 --2- 172.21.15.106:0/1948248505 >> v2:172.21.15.114:3300/0 conn(0x7f613c15c7a0 0x7f613c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.085+0000 7f613b7fe640 1 -- 172.21.15.106:0/1948248505 >> v1:172.21.15.114:6789/0 conn(0x7f613c1592b0 legacy=0x7f613c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.085+0000 7f613b7fe640 1 -- 172.21.15.106:0/1948248505 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c06ef70 con 0x7f613c06e740 2024-04-13T03:10:11.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.085+0000 7f613b7fe640 1 --2- 172.21.15.106:0/1948248505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c06e740 0x7f613c15c260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f613c0b3070 tx=0x7f613002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b1ab6a5a1626f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f613affd640 1 -- 172.21.15.106:0/1948248505 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130034070 con 0x7f613c06e740 2024-04-13T03:10:11.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f613affd640 1 -- 172.21.15.106:0/1948248505 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6130030860 con 0x7f613c06e740 2024-04-13T03:10:11.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f613affd640 1 -- 172.21.15.106:0/1948248505 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130030c30 con 0x7f613c06e740 2024-04-13T03:10:11.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 -- 172.21.15.106:0/1948248505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c06e740 msgr2=0x7f613c15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 --2- 172.21.15.106:0/1948248505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c06e740 0x7f613c15c260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f613c0b3070 tx=0x7f613002b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 -- 172.21.15.106:0/1948248505 shutdown_connections 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 --2- 172.21.15.106:0/1948248505 >> v2:172.21.15.114:3300/0 conn(0x7f613c15c7a0 0x7f613c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 --2- 172.21.15.106:0/1948248505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c06e740 0x7f613c15c260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 -- 172.21.15.106:0/1948248505 >> 172.21.15.106:0/1948248505 conn(0x7f613c0b2600 msgr2=0x7f613c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 -- 172.21.15.106:0/1948248505 shutdown_connections 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.086+0000 7f61428b7640 1 -- 172.21.15.106:0/1948248505 wait complete. 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.087+0000 7f61428b7640 1 Processor -- start 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.087+0000 7f61428b7640 1 -- start start 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 0x7f613c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f613c1592b0 0x7f613c167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f613c0ceb60 con 0x7f613c15c7a0 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f613c06f530 con 0x7f613c06e740 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f61428b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f613c0cece0 con 0x7f613c1592b0 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54334/0 (socket says 172.21.15.106:54334) 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 -- 172.21.15.106:0/2200866355 learned_addr learned my addr 172.21.15.106:0/2200866355 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f613b7fe640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f613c1592b0 0x7f613c167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f613bfff640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 0x7f613c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 -- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f613c1592b0 msgr2=0x7f613c167230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f613c1592b0 0x7f613c167230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 -- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 msgr2=0x7f613c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 0x7f613c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.088+0000 7f6140e54640 1 -- 172.21.15.106:0/2200866355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c167cb0 con 0x7f613c15c7a0 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f613bfff640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 0x7f613c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f6140e54640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f613400efc0 tx=0x7f613400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134018020 con 0x7f613c15c7a0 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6134014480 con 0x7f613c15c7a0 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c15cc50 con 0x7f613c15c7a0 2024-04-13T03:10:11.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134009e60 con 0x7f613c15c7a0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.089+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c15d4f0 con 0x7f613c15c7a0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.090+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613c068a10 con 0x7f613c15c7a0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.090+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6134005d10 con 0x7f613c15c7a0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.091+0000 7f6138ff9640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 0x7f612806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.091+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f613408cbd0 con 0x7f613c15c7a0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.091+0000 7f613bfff640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 0x7f612806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.091+0000 7f613bfff640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 0x7f612806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f612c001700 tx=0x7f612c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.095+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6134022030 con 0x7f613c15c7a0 2024-04-13T03:10:11.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.270+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f613c070050 con 0x7f613c15c7a0 2024-04-13T03:10:11.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.270+0000 7f6138ff9640 1 -- 172.21.15.106:0/2200866355 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6134010040 con 0x7f613c15c7a0 2024-04-13T03:10:11.271 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:11.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 msgr2=0x7f612806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 0x7f612806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f612c001700 tx=0x7f612c009040 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 msgr2=0x7f613c167770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f613400efc0 tx=0x7f613400c490 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 shutdown_connections 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f612806c8f0 0x7f612806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f613c1592b0 0x7f613c167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f613c15c7a0 0x7f613c167770 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 --2- 172.21.15.106:0/2200866355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613c06e740 0x7f613c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 >> 172.21.15.106:0/2200866355 conn(0x7f613c0b2600 msgr2=0x7f613c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 shutdown_connections 2024-04-13T03:10:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.272+0000 7f61428b7640 1 -- 172.21.15.106:0/2200866355 wait complete. 2024-04-13T03:10:11.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:11.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.415+0000 7fa79b6de640 1 Processor -- start 2024-04-13T03:10:11.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.415+0000 7fa79b6de640 1 -- start start 2024-04-13T03:10:11.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa79b6de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7940b5390 0x7fa79415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa79b6de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7940b3690 con 0x7fa7940b4730 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa79b6de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7940b3970 con 0x7fa79415b4e0 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa79b6de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa7940b3c50 con 0x7fa7940b5390 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa799c7b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa7940b4730 0x7fa79406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42684/0 (socket says 172.21.15.106:42684) 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa799c7b640 1 -- 172.21.15.106:0/769633582 learned_addr learned my addr 172.21.15.106:0/769633582 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa798c79640 1 --2- 172.21.15.106:0/769633582 >> v2:172.21.15.114:3300/0 conn(0x7fa7940b5390 0x7fa79415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 55128877 0 0) 0x7fa7940b3690 con 0x7fa7940b4730 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa780003520 con 0x7fa7940b4730 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231533455 0 0) 0x7fa7940b3970 con 0x7fa79415b4e0 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7940b3690 con 0x7fa79415b4e0 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3814026909 0 0) 0x7fa780003520 con 0x7fa7940b4730 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa798c79640 1 -- 172.21.15.106:0/769633582 >> v1:172.21.15.114:6789/0 conn(0x7fa79415b4e0 legacy=0x7fa7940b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa798c79640 1 -- 172.21.15.106:0/769633582 >> v1:172.21.15.106:6789/0 conn(0x7fa7940b4730 legacy=0x7fa79406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.416+0000 7fa798c79640 1 -- 172.21.15.106:0/769633582 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7940b3f30 con 0x7fa7940b5390 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.417+0000 7fa798c79640 1 --2- 172.21.15.106:0/769633582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7940b5390 0x7fa79415fb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7840097a0 tx=0x7fa78402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478ea9c0b01be324 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.417+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784038070 con 0x7fa7940b5390 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.417+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa784031500 con 0x7fa7940b5390 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.417+0000 7fa78bfff640 1 -- 172.21.15.106:0/769633582 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0019f0 con 0x7fa7940b5390 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 -- 172.21.15.106:0/769633582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7940b5390 msgr2=0x7fa79415fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 --2- 172.21.15.106:0/769633582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7940b5390 0x7fa79415fb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7840097a0 tx=0x7fa78402b820 comp rx=0 tx=0).stop 2024-04-13T03:10:11.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 -- 172.21.15.106:0/769633582 shutdown_connections 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 --2- 172.21.15.106:0/769633582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7940b5390 0x7fa79415fb60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 -- 172.21.15.106:0/769633582 >> 172.21.15.106:0/769633582 conn(0x7fa7940b2600 msgr2=0x7fa7940d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 -- 172.21.15.106:0/769633582 shutdown_connections 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.418+0000 7fa79b6de640 1 -- 172.21.15.106:0/769633582 wait complete. 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.419+0000 7fa79b6de640 1 Processor -- start 2024-04-13T03:10:11.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.419+0000 7fa79b6de640 1 -- start start 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7940b4730 0x7fa794145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa79415b4e0 0x7fa7940a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa79406d0c0 con 0x7fa7940b5390 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940b32d0 con 0x7fa79415b4e0 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79b6de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa794147e30 con 0x7fa7940b4730 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54340/0 (socket says 172.21.15.106:54340) 2024-04-13T03:10:11.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 -- 172.21.15.106:0/2026413253 learned_addr learned my addr 172.21.15.106:0/2026413253 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa79947a640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7940b4730 0x7fa794145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa799c7b640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa79415b4e0 0x7fa7940a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 -- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7940b4730 msgr2=0x7fa794145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7940b4730 0x7fa794145230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 -- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa79415b4e0 msgr2=0x7fa7940a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa79415b4e0 0x7fa7940a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 -- 172.21.15.106:0/2026413253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa794148110 con 0x7fa7940b5390 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.420+0000 7fa798c79640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa7840098c0 tx=0x7fa7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.421+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784042070 con 0x7fa7940b5390 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.421+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78403d870 con 0x7fa7940b5390 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.421+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784038040 con 0x7fa7940b5390 2024-04-13T03:10:11.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.421+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7941483f0 con 0x7fa7940b5390 2024-04-13T03:10:11.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.421+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7940ae560 con 0x7fa7940b5390 2024-04-13T03:10:11.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.422+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa794068a10 con 0x7fa7940b5390 2024-04-13T03:10:11.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.422+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa784045070 con 0x7fa7940b5390 2024-04-13T03:10:11.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.423+0000 7fa789ffb640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 0x7fa77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.423+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa7840ad960 con 0x7fa7940b5390 2024-04-13T03:10:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.423+0000 7fa79947a640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 0x7fa77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.423+0000 7fa79947a640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 0x7fa77806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7900011e0 tx=0x7fa790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.427+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa784030070 con 0x7fa7940b5390 2024-04-13T03:10:11.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.603+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7940af9c0 con 0x7fa7940b5390 2024-04-13T03:10:11.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.603+0000 7fa789ffb640 1 -- 172.21.15.106:0/2026413253 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa78407bab0 con 0x7fa7940b5390 2024-04-13T03:10:11.604 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:11.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 msgr2=0x7fa77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 0x7fa77806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7900011e0 tx=0x7fa790009040 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 msgr2=0x7fa7940ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa7840098c0 tx=0x7fa7840046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 shutdown_connections 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa77806c8f0 0x7fa77806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa79415b4e0 0x7fa7940a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7940b5390 0x7fa7940ae020 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 --2- 172.21.15.106:0/2026413253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7940b4730 0x7fa794145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 >> 172.21.15.106:0/2026413253 conn(0x7fa7940b2600 msgr2=0x7fa7940afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 shutdown_connections 2024-04-13T03:10:11.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.605+0000 7fa79b6de640 1 -- 172.21.15.106:0/2026413253 wait complete. 2024-04-13T03:10:11.617 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:11.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.719+0000 7f66a7955640 1 Processor -- start 2024-04-13T03:10:11.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.719+0000 7f66a7955640 1 -- start start 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a7955640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f66a015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a7955640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a015ddd0 con 0x7f66a00704a0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a7955640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a015d4d0 con 0x7f66a00d1c30 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a7955640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66a015e100 con 0x7f66a00cfea0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a4ef0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f66a00704a0 0x7f66a00d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42700/0 (socket says 172.21.15.106:42700) 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a4ef0640 1 -- 172.21.15.106:0/250094296 learned_addr learned my addr 172.21.15.106:0/250094296 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f66a56f1640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f66a015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3077103974 0 0) 0x7f66a015ddd0 con 0x7f66a00704a0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f668c003520 con 0x7f66a00704a0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4048343420 0 0) 0x7f668c003520 con 0x7f66a00704a0 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 msgr2=0x7f66a015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f66a015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 >> v1:172.21.15.114:6789/0 conn(0x7f66a00d1c30 legacy=0x7f66a015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a015d7b0 con 0x7f66a00704a0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.720+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6690001670 con 0x7f66a00704a0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 >> v1:172.21.15.106:6789/0 conn(0x7f66a00704a0 legacy=0x7f66a00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668c0039e0 0x7f668c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f668c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c004d40 0x7f668c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66a015e100 con 0x7f668c0039e0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f668c00ba70 con 0x7f668c004d40 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f668c00bc70 con 0x7f66a00cfea0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668c0039e0 0x7f668c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a4ef0640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f668c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a5ef2640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c004d40 0x7f668c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 -- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 msgr2=0x7f668c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:11.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a00cfea0 0x7f668c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 -- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c004d40 msgr2=0x7f668c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c004d40 0x7f668c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.721+0000 7f66a56f1640 1 -- 172.21.15.106:0/250094296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c003520 con 0x7f668c0039e0 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a56f1640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668c0039e0 0x7f668c003dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f669c009ac0 tx=0x7f668c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb49741139caccc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c039070 con 0x7f668c0039e0 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669c02c6b0 con 0x7f668c0039e0 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f6697fff640 1 -- 172.21.15.106:0/250094296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c009690 con 0x7f668c0039e0 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 -- 172.21.15.106:0/250094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668c0039e0 msgr2=0x7f668c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 --2- 172.21.15.106:0/250094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668c0039e0 0x7f668c003dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f669c009ac0 tx=0x7f668c005830 comp rx=0 tx=0).stop 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a5ef2640 1 -- 172.21.15.106:0/250094296 reap_dead start 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 -- 172.21.15.106:0/250094296 shutdown_connections 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 -- 172.21.15.106:0/250094296 >> 172.21.15.106:0/250094296 conn(0x7f66a0071870 msgr2=0x7f66a0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 -- 172.21.15.106:0/250094296 shutdown_connections 2024-04-13T03:10:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.722+0000 7f66a7955640 1 -- 172.21.15.106:0/250094296 wait complete. 2024-04-13T03:10:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 Processor -- start 2024-04-13T03:10:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 -- start start 2024-04-13T03:10:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c0039e0 0x7f66a01569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a0154270 0x7f66a01f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66a015cd90 con 0x7f66a01f6260 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66a015cc10 con 0x7f668c0039e0 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a7955640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66a015cf10 con 0x7f66a0154270 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a5ef2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a5ef2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54362/0 (socket says 172.21.15.106:54362) 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.724+0000 7f66a5ef2640 1 -- 172.21.15.106:0/2958279389 learned_addr learned my addr 172.21.15.106:0/2958279389 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a56f1640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c0039e0 0x7f66a01569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a4ef0640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a0154270 0x7f66a01f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 -- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a0154270 msgr2=0x7f66a01f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a0154270 0x7f66a01f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 -- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c0039e0 msgr2=0x7f66a01569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c0039e0 0x7f66a01569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 -- 172.21.15.106:0/2958279389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01f6770 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a5ef2640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66a00d2010 tx=0x7f669800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698019070 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698006d60 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698003bb0 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01f6a50 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.725+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a0150730 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.726+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a0068a10 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.726+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f669801c070 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.727+0000 7f6695ffb640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 0x7f668406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.727+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f669808b270 con 0x7f66a01f6260 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.727+0000 7f66a56f1640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 0x7f668406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.727+0000 7f66a56f1640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 0x7f668406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f669c036000 tx=0x7f669c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.731+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6698059330 con 0x7f66a01f6260 2024-04-13T03:10:11.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.905+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66a00d0de0 con 0x7f66a01f6260 2024-04-13T03:10:11.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.906+0000 7f6695ffb640 1 -- 172.21.15.106:0/2958279389 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f6698058c30 con 0x7f66a01f6260 2024-04-13T03:10:11.907 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 msgr2=0x7f668406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 0x7f668406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f669c036000 tx=0x7f669c030000 comp rx=0 tx=0).stop 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 msgr2=0x7f66a0152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66a00d2010 tx=0x7f669800ff70 comp rx=0 tx=0).stop 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 shutdown_connections 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f668406c920 0x7f668406ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66a0154270 0x7f66a01f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01f6260 0x7f66a0152d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 --2- 172.21.15.106:0/2958279389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668c0039e0 0x7f66a01569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:11.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 >> 172.21.15.106:0/2958279389 conn(0x7f66a0071870 msgr2=0x7f66a00d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:11.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 shutdown_connections 2024-04-13T03:10:11.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:11.908+0000 7f66a7955640 1 -- 172.21.15.106:0/2958279389 wait complete. 2024-04-13T03:10:11.920 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:12.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.023+0000 7f050cd1e640 1 Processor -- start 2024-04-13T03:10:12.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.023+0000 7f050cd1e640 1 -- start start 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050cd1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f05080b3d60 0x7f05080d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050cd1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05080cfd20 con 0x7f050815e9b0 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050cd1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0508162af0 con 0x7f050815ed90 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050cd1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0508162dd0 con 0x7f05080b3d60 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f050815e9b0 0x7f05080d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42706/0 (socket says 172.21.15.106:42706) 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.024+0000 7f050659c640 1 -- 172.21.15.106:0/984670380 learned_addr learned my addr 172.21.15.106:0/984670380 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f0506d9d640 1 --2- 172.21.15.106:0/984670380 >> v2:172.21.15.114:3300/0 conn(0x7f05080b3d60 0x7f05080d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286986616 0 0) 0x7f05080cfd20 con 0x7f050815e9b0 2024-04-13T03:10:12.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f0003520 con 0x7f050815e9b0 2024-04-13T03:10:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f0506d9d640 1 -- 172.21.15.106:0/984670380 >> v1:172.21.15.114:6789/0 conn(0x7f050815ed90 legacy=0x7f05080d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f0506d9d640 1 -- 172.21.15.106:0/984670380 >> v1:172.21.15.106:6789/0 conn(0x7f050815e9b0 legacy=0x7f05080d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2839908179 0 0) 0x7f04f0003520 con 0x7f050815e9b0 2024-04-13T03:10:12.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.025+0000 7f0506d9d640 1 -- 172.21.15.106:0/984670380 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081630b0 con 0x7f05080b3d60 2024-04-13T03:10:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f0506d9d640 1 --2- 172.21.15.106:0/984670380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05080b3d60 0x7f05080d2300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05000097a0 tx=0x7f050002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6642f735d3dd871a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80019f0 con 0x7f05080b3d60 2024-04-13T03:10:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500037070 con 0x7f05080b3d60 2024-04-13T03:10:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050559a640 1 -- 172.21.15.106:0/984670380 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500031790 con 0x7f05080b3d60 2024-04-13T03:10:12.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050cd1e640 1 -- 172.21.15.106:0/984670380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05080b3d60 msgr2=0x7f05080d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050cd1e640 1 --2- 172.21.15.106:0/984670380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05080b3d60 0x7f05080d2300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05000097a0 tx=0x7f050002b8b0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050cd1e640 1 -- 172.21.15.106:0/984670380 shutdown_connections 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050cd1e640 1 --2- 172.21.15.106:0/984670380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05080b3d60 0x7f05080d2300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.026+0000 7f050cd1e640 1 -- 172.21.15.106:0/984670380 >> 172.21.15.106:0/984670380 conn(0x7f050808bf10 msgr2=0x7f05080b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.027+0000 7f050cd1e640 1 -- 172.21.15.106:0/984670380 shutdown_connections 2024-04-13T03:10:12.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.027+0000 7f050cd1e640 1 -- 172.21.15.106:0/984670380 wait complete. 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.028+0000 7f050cd1e640 1 Processor -- start 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.028+0000 7f050cd1e640 1 -- start start 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f050815e9b0 0x7f05081f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 0x7f050814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0508162800 con 0x7f05080b3d60 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0508162680 con 0x7f050815e9b0 2024-04-13T03:10:12.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f05080d4d00 con 0x7f050814d6a0 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54376/0 (socket says 172.21.15.106:54376) 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 -- 172.21.15.106:0/3106482300 learned_addr learned my addr 172.21.15.106:0/3106482300 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f0505d9b640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f050815e9b0 0x7f05081f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f0506d9d640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 0x7f050814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 -- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 msgr2=0x7f050814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 0x7f050814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 -- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f050815e9b0 msgr2=0x7f05081f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f050815e9b0 0x7f05081f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 -- 172.21.15.106:0/3106482300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050806ef70 con 0x7f05080b3d60 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f0506d9d640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 0x7f050814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050659c640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04f8001880 tx=0x7f04f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8017070 con 0x7f05080b3d60 2024-04-13T03:10:12.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8012400 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050806ded0 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8012830 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.029+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050806e490 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.030+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508068a10 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.031+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04f801a070 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.031+0000 7f04f6ffd640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.031+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f04f808bf50 con 0x7f05080b3d60 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.031+0000 7f0505d9b640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.032+0000 7f0505d9b640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 0x7f04e806ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f050815c3a0 tx=0x7f04fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.036+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04f8059f80 con 0x7f05080b3d60 2024-04-13T03:10:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.208+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f05080d53a0 con 0x7f05080b3d60 2024-04-13T03:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.209+0000 7f04f6ffd640 1 -- 172.21.15.106:0/3106482300 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f04f8059880 con 0x7f05080b3d60 2024-04-13T03:10:12.210 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.210+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 msgr2=0x7f04e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.210+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 0x7f04e806ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f050815c3a0 tx=0x7f04fc009040 comp rx=0 tx=0).stop 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.210+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 msgr2=0x7f050806d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.210+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04f8001880 tx=0x7f04f800a430 comp rx=0 tx=0).stop 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 shutdown_connections 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f050814d6a0 0x7f050814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f050815e9b0 0x7f05081f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 --2- 172.21.15.106:0/3106482300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05080b3d60 0x7f050806d6c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 >> 172.21.15.106:0/3106482300 conn(0x7f050808bf10 msgr2=0x7f050815c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 shutdown_connections 2024-04-13T03:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.211+0000 7f050cd1e640 1 -- 172.21.15.106:0/3106482300 wait complete. 2024-04-13T03:10:12.232 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:10:12.233 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:12.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.332+0000 7f74c9e20640 1 Processor -- start 2024-04-13T03:10:12.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.332+0000 7f74c9e20640 1 -- start start 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.332+0000 7f74c9e20640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c40b68d0 0x7f74c40ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c9e20640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74c41624b0 0x7f74c40d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c9e20640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c415eda0 con 0x7f74c415e9c0 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c9e20640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f74c40d27f0 con 0x7f74c41624b0 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c9e20640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74c4162c20 con 0x7f74c40b68d0 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74c41624b0 0x7f74c40d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74c41624b0 0x7f74c40d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54378/0 (socket says 172.21.15.106:54378) 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 -- 172.21.15.106:0/767254296 learned_addr learned my addr 172.21.15.106:0/767254296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c2ffd640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c40b68d0 0x7f74c40ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 -- 172.21.15.106:0/767254296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c40b68d0 msgr2=0x7f74c40ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:12.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c40b68d0 0x7f74c40ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 -- 172.21.15.106:0/767254296 >> v1:172.21.15.114:6789/0 conn(0x7f74c415e9c0 legacy=0x7f74c40b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.333+0000 7f74c3fff640 1 -- 172.21.15.106:0/767254296 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c4162f00 con 0x7f74c41624b0 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c3fff640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c41624b0 0x7f74c40d2f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009e50 tx=0x7f74bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf9e5bacccd93fa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c27fc640 1 -- 172.21.15.106:0/767254296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc02cb00 con 0x7f74c41624b0 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c27fc640 1 -- 172.21.15.106:0/767254296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74bc02cca0 con 0x7f74c41624b0 2024-04-13T03:10:12.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c27fc640 1 -- 172.21.15.106:0/767254296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc034610 con 0x7f74c41624b0 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 -- 172.21.15.106:0/767254296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c41624b0 msgr2=0x7f74c40d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c41624b0 0x7f74c40d2f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009e50 tx=0x7f74bc02bf60 comp rx=0 tx=0).stop 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 -- 172.21.15.106:0/767254296 shutdown_connections 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c41624b0 0x7f74c40d2f60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 --2- 172.21.15.106:0/767254296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c40b68d0 0x7f74c40ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 -- 172.21.15.106:0/767254296 >> 172.21.15.106:0/767254296 conn(0x7f74c40d11c0 msgr2=0x7f74c40d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 -- 172.21.15.106:0/767254296 shutdown_connections 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.334+0000 7f74c9e20640 1 -- 172.21.15.106:0/767254296 wait complete. 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 Processor -- start 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 -- start start 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c415e9c0 0x7f74c40ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 0x7f74c40cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4162940 con 0x7f74c40b68d0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74c40d2a40 con 0x7f74c415e9c0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c9e20640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74c40d34a0 con 0x7f74c41624b0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.336+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54392/0 (socket says 172.21.15.106:54392) 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 -- 172.21.15.106:0/2149569844 learned_addr learned my addr 172.21.15.106:0/2149569844 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c2ffd640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c415e9c0 0x7f74c40ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c3fff640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 0x7f74c40cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 -- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 msgr2=0x7f74c40cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 0x7f74c40cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 -- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c415e9c0 msgr2=0x7f74c40ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c415e9c0 0x7f74c40ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 -- 172.21.15.106:0/2149569844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c4161bf0 con 0x7f74c40b68d0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c3fff640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 0x7f74c40cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c37fe640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c430 tx=0x7f74b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40052d0 con 0x7f74c40b68d0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b4005490 con 0x7f74c40b68d0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c40c8270 con 0x7f74c40b68d0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4007260 con 0x7f74c40b68d0 2024-04-13T03:10:12.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.337+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c40c8830 con 0x7f74c40b68d0 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.338+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c4068a10 con 0x7f74c40b68d0 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.338+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74b4007400 con 0x7f74c40b68d0 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.339+0000 7f74abfff640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.339+0000 7f74c2ffd640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.339+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f74b408b3d0 con 0x7f74c40b68d0 2024-04-13T03:10:12.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.339+0000 7f74c2ffd640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74c41f8170 tx=0x7f74b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.343+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74b4058db0 con 0x7f74c40b68d0 2024-04-13T03:10:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.521+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74c40d5430 con 0x7f74c40b68d0 2024-04-13T03:10:12.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.522+0000 7f74abfff640 1 -- 172.21.15.106:0/2149569844 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f74b405d0b0 con 0x7f74c40b68d0 2024-04-13T03:10:12.523 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:12.523 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 msgr2=0x7f74b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74c41f8170 tx=0x7f74b8008040 comp rx=0 tx=0).stop 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 msgr2=0x7f74c40b7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c430 tx=0x7f74b400c8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 shutdown_connections 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c41624b0 0x7f74c40cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c415e9c0 0x7f74c40ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 --2- 172.21.15.106:0/2149569844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c40b68d0 0x7f74c40b7e80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 >> 172.21.15.106:0/2149569844 conn(0x7f74c40d11c0 msgr2=0x7f74c40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 shutdown_connections 2024-04-13T03:10:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.524+0000 7f74c9e20640 1 -- 172.21.15.106:0/2149569844 wait complete. 2024-04-13T03:10:12.536 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:12.537 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:12.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.639+0000 7f9ecc088640 1 Processor -- start 2024-04-13T03:10:12.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.639+0000 7f9ecc088640 1 -- start start 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ecc088640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ec415e8c0 0x7f9ec40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ecc088640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9ec415eca0 0x7f9ec4072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ecc088640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec40b5060 con 0x7f9ec4073350 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ecc088640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec4073780 con 0x7f9ec415e8c0 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ecc088640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec40b5340 con 0x7f9ec415eca0 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ec415e8c0 0x7f9ec40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9eca625640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9ec4073350 0x7f9ec40b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42712/0 (socket says 172.21.15.106:42712) 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ec415e8c0 0x7f9ec40ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54402/0 (socket says 172.21.15.106:54402) 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9eca625640 1 -- 172.21.15.106:0/1384938166 learned_addr learned my addr 172.21.15.106:0/1384938166 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9623640 1 --2- 172.21.15.106:0/1384938166 >> v2:172.21.15.114:3300/0 conn(0x7f9ec415eca0 0x7f9ec4072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 -- 172.21.15.106:0/1384938166 >> v2:172.21.15.114:3300/0 conn(0x7f9ec415eca0 msgr2=0x7f9ec4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 --2- 172.21.15.106:0/1384938166 >> v2:172.21.15.114:3300/0 conn(0x7f9ec415eca0 0x7f9ec4072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 -- 172.21.15.106:0/1384938166 >> v1:172.21.15.106:6789/0 conn(0x7f9ec4073350 legacy=0x7f9ec40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec9e24640 1 -- 172.21.15.106:0/1384938166 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec406c500 con 0x7f9ec415e8c0 2024-04-13T03:10:12.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.640+0000 7f9ec8e22640 1 -- 172.21.15.106:0/1384938166 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068489818 0 0) 0x7f9ec40b5060 con 0x7f9ec4073350 2024-04-13T03:10:12.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.641+0000 7f9ec9e24640 1 --2- 172.21.15.106:0/1384938166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec415e8c0 0x7f9ec40ce250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4072ae0 tx=0x7f9ec002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2123be9cdf0d37d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.641+0000 7f9ec8e22640 1 -- 172.21.15.106:0/1384938166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0039070 con 0x7f9ec415e8c0 2024-04-13T03:10:12.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.641+0000 7f9ec8e22640 1 -- 172.21.15.106:0/1384938166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec002cd00 con 0x7f9ec415e8c0 2024-04-13T03:10:12.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.641+0000 7f9ec8e22640 1 -- 172.21.15.106:0/1384938166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0034660 con 0x7f9ec415e8c0 2024-04-13T03:10:12.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 -- 172.21.15.106:0/1384938166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec415e8c0 msgr2=0x7f9ec40ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 --2- 172.21.15.106:0/1384938166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec415e8c0 0x7f9ec40ce250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4072ae0 tx=0x7f9ec002bf50 comp rx=0 tx=0).stop 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 -- 172.21.15.106:0/1384938166 shutdown_connections 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 --2- 172.21.15.106:0/1384938166 >> v2:172.21.15.114:3300/0 conn(0x7f9ec415eca0 0x7f9ec4072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 --2- 172.21.15.106:0/1384938166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec415e8c0 0x7f9ec40ce250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 -- 172.21.15.106:0/1384938166 >> 172.21.15.106:0/1384938166 conn(0x7f9ec406f0d0 msgr2=0x7f9ec406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 -- 172.21.15.106:0/1384938166 shutdown_connections 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.642+0000 7f9ecc088640 1 -- 172.21.15.106:0/1384938166 wait complete. 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.643+0000 7f9ecc088640 1 Processor -- start 2024-04-13T03:10:12.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.643+0000 7f9ecc088640 1 -- start start 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec415e8c0 0x7f9ec41e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ec415eca0 0x7f9ec4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40b4cc0 con 0x7f9ec4073350 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40b4b40 con 0x7f9ec415e8c0 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ecc088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec40b4e40 con 0x7f9ec415eca0 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54408/0 (socket says 172.21.15.106:54408) 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 -- 172.21.15.106:0/973281825 learned_addr learned my addr 172.21.15.106:0/973281825 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 -- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ec415eca0 msgr2=0x7f9ec4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ec415eca0 0x7f9ec4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 -- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec415e8c0 msgr2=0x7f9ec41e9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec415e8c0 0x7f9ec41e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 -- 172.21.15.106:0/973281825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41ea2e0 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.644+0000 7f9ec9e24640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0037040 tx=0x7f9ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0039050 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0045070 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41ea5c0 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec003e850 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec4140850 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.645+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec4068a10 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.646+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec003e9f0 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.646+0000 7f9eba7fc640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.646+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9ec00aecc0 con 0x7f9ec4073350 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.646+0000 7f9ec9623640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.647+0000 7f9ec9623640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 0x7f9ea806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4006fd0 tx=0x7f9eb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.651+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ec007ce10 con 0x7f9ec4073350 2024-04-13T03:10:12.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.825+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ec40cdbf0 con 0x7f9ec4073350 2024-04-13T03:10:12.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.825+0000 7f9eba7fc640 1 -- 172.21.15.106:0/973281825 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9ec007c710 con 0x7f9ec4073350 2024-04-13T03:10:12.826 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:12.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 msgr2=0x7f9ea806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 0x7f9ea806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4006fd0 tx=0x7f9eb4008040 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 msgr2=0x7f9ec40ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0037040 tx=0x7f9ec0004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 shutdown_connections 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ea806c920 0x7f9ea806ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ec415eca0 0x7f9ec4142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec415e8c0 0x7f9ec41e9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 --2- 172.21.15.106:0/973281825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ec4073350 0x7f9ec40ca900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 >> 172.21.15.106:0/973281825 conn(0x7f9ec406f0d0 msgr2=0x7f9ec4162c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 shutdown_connections 2024-04-13T03:10:12.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.827+0000 7f9ecc088640 1 -- 172.21.15.106:0/973281825 wait complete. 2024-04-13T03:10:12.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 Processor -- start 2024-04-13T03:10:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 -- start start 2024-04-13T03:10:12.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f177c15e9b0 0x7f177c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177c0cfd20 con 0x7f177c15ed90 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f177c162af0 con 0x7f177c15e9b0 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.942+0000 7f17819f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f177c162dd0 con 0x7f177c0b3d60 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f177c15e9b0 0x7f177c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:51972/0 (socket says 172.21.15.106:51972) 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f177c15e9b0 0x7f177c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:33518/0 (socket says 172.21.15.106:33518) 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177b7fe640 1 -- 172.21.15.106:0/2166558062 learned_addr learned my addr 172.21.15.106:0/2166558062 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f1773fff640 1 --1- 172.21.15.106:0/2166558062 >> v1:172.21.15.114:6789/0 conn(0x7f177c15ed90 0x7f177c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42752/0 (socket says 172.21.15.106:42752) 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 -- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 msgr2=0x7f177c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:12.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 -- 172.21.15.106:0/2166558062 >> v1:172.21.15.114:6789/0 conn(0x7f177c15ed90 legacy=0x7f177c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177affd640 1 -- 172.21.15.106:0/2166558062 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1630b0 con 0x7f177c15e9b0 2024-04-13T03:10:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.943+0000 7f177b7fe640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.944+0000 7f177affd640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0d2300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097a0 tx=0x7f176c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e60b6b6c589ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.944+0000 7f177a7fc640 1 -- 172.21.15.106:0/2166558062 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c034070 con 0x7f177c15e9b0 2024-04-13T03:10:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.944+0000 7f177a7fc640 1 -- 172.21.15.106:0/2166558062 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c030610 con 0x7f177c15e9b0 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.944+0000 7f177a7fc640 1 -- 172.21.15.106:0/2166558062 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0309e0 con 0x7f177c15e9b0 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 -- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 msgr2=0x7f177c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0d2300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097a0 tx=0x7f176c02b960 comp rx=0 tx=0).stop 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 -- 172.21.15.106:0/2166558062 shutdown_connections 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0b3d60 0x7f177c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 --2- 172.21.15.106:0/2166558062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0d2300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 -- 172.21.15.106:0/2166558062 >> 172.21.15.106:0/2166558062 conn(0x7f177c08bf10 msgr2=0x7f177c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 -- 172.21.15.106:0/2166558062 shutdown_connections 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.945+0000 7f17819f7640 1 -- 172.21.15.106:0/2166558062 wait complete. 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.946+0000 7f17819f7640 1 Processor -- start 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.946+0000 7f17819f7640 1 -- start start 2024-04-13T03:10:12.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 0x7f177c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f177c162800 con 0x7f177c0b3d60 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f177c162680 con 0x7f177c15e9b0 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f17819f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0d4d00 con 0x7f177c0cd1f0 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46274/0 (socket says 172.21.15.106:46274) 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 -- 172.21.15.106:0/338608656 learned_addr learned my addr 172.21.15.106:0/338608656 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177b7fe640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 0x7f177c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f1773fff640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 -- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 msgr2=0x7f177c15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 0x7f177c15b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 -- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 msgr2=0x7f177c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 -- 172.21.15.106:0/338608656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0cd700 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177b7fe640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 0x7f177c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f1773fff640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f177affd640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f176c0098c0 tx=0x7f176c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.947+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c034040 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.948+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c030410 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.948+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c0cd9e0 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.948+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c039c30 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.948+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c0cdfa0 con 0x7f177c0b3d60 2024-04-13T03:10:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.949+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c068a10 con 0x7f177c0b3d60 2024-04-13T03:10:12.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.949+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f176c037070 con 0x7f177c0b3d60 2024-04-13T03:10:12.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.949+0000 7f1778ff9640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 0x7f175c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:12.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.949+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f176c0ade30 con 0x7f177c0b3d60 2024-04-13T03:10:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.949+0000 7f1773fff640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 0x7f175c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.950+0000 7f1773fff640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 0x7f175c06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f177c06f160 tx=0x7f1764009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:12.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:12.954+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f176c07bef0 con 0x7f177c0b3d60 2024-04-13T03:10:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.130+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f177c0b2d90 con 0x7f177c0b3d60 2024-04-13T03:10:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.130+0000 7f1778ff9640 1 -- 172.21.15.106:0/338608656 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f176c0820c0 con 0x7f177c0b3d60 2024-04-13T03:10:13.131 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:13.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 msgr2=0x7f175c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 0x7f175c06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f177c06f160 tx=0x7f1764009040 comp rx=0 tx=0).stop 2024-04-13T03:10:13.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 msgr2=0x7f177c159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f176c0098c0 tx=0x7f176c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 shutdown_connections 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f175c06c920 0x7f175c06ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f177c0cd1f0 0x7f177c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f177c15e9b0 0x7f177c0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 --2- 172.21.15.106:0/338608656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f177c0b3d60 0x7f177c159b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 >> 172.21.15.106:0/338608656 conn(0x7f177c08bf10 msgr2=0x7f177c161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 shutdown_connections 2024-04-13T03:10:13.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.132+0000 7f17819f7640 1 -- 172.21.15.106:0/338608656 wait complete. 2024-04-13T03:10:13.144 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:13.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.247+0000 7f49f4820640 1 Processor -- start 2024-04-13T03:10:13.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.247+0000 7f49f4820640 1 -- start start 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b1c30 0x7f49f015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49f00b54a0 0x7f49f00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f49f00709e0 0x7f49f0162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f49f00b5cf0 con 0x7f49f00b54a0 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f49f00b5fd0 con 0x7f49f00709e0 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49f4820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00b62b0 con 0x7f49f00b1c30 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49f00b54a0 0x7f49f00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49f00b54a0 0x7f49f00704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46290/0 (socket says 172.21.15.106:46290) 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 -- 172.21.15.106:0/466087315 learned_addr learned my addr 172.21.15.106:0/466087315 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 -- 172.21.15.106:0/466087315 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b1c30 msgr2=0x7f49f015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:13.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 --2- 172.21.15.106:0/466087315 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b1c30 0x7f49f015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49edd9b640 1 -- 172.21.15.106:0/466087315 >> v2:172.21.15.114:3300/0 conn(0x7f49f00709e0 msgr2=0x7f49f0162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:13.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.248+0000 7f49eed9d640 1 --2- 172.21.15.106:0/466087315 >> v2:172.21.15.114:3300/0 conn(0x7f49f00709e0 0x7f49f0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49edd9b640 1 --2- 172.21.15.106:0/466087315 >> v2:172.21.15.114:3300/0 conn(0x7f49f00709e0 0x7f49f0162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49edd9b640 1 -- 172.21.15.106:0/466087315 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00b6590 con 0x7f49f00b54a0 2024-04-13T03:10:13.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49edd9b640 1 --2- 172.21.15.106:0/466087315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b54a0 0x7f49f00704a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009870 tx=0x7f49e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb1794c3a6100d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49ed59a640 1 -- 172.21.15.106:0/466087315 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4035020 con 0x7f49f00b54a0 2024-04-13T03:10:13.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49ed59a640 1 -- 172.21.15.106:0/466087315 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4031640 con 0x7f49f00b54a0 2024-04-13T03:10:13.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.249+0000 7f49ed59a640 1 -- 172.21.15.106:0/466087315 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e402d040 con 0x7f49f00b54a0 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 -- 172.21.15.106:0/466087315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b54a0 msgr2=0x7f49f00704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 --2- 172.21.15.106:0/466087315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b54a0 0x7f49f00704a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009870 tx=0x7f49e402b8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 -- 172.21.15.106:0/466087315 shutdown_connections 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 --2- 172.21.15.106:0/466087315 >> v2:172.21.15.114:3300/0 conn(0x7f49f00709e0 0x7f49f0162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 --2- 172.21.15.106:0/466087315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b54a0 0x7f49f00704a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 --2- 172.21.15.106:0/466087315 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b1c30 0x7f49f015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 -- 172.21.15.106:0/466087315 >> 172.21.15.106:0/466087315 conn(0x7f49f0071870 msgr2=0x7f49f0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 -- 172.21.15.106:0/466087315 shutdown_connections 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.250+0000 7f49f4820640 1 -- 172.21.15.106:0/466087315 wait complete. 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.251+0000 7f49f4820640 1 Processor -- start 2024-04-13T03:10:13.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.251+0000 7f49f4820640 1 -- start start 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f00709e0 0x7f49f0152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b54a0 0x7f49f00cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0162af0 con 0x7f49f00b1c30 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0162970 con 0x7f49f00709e0 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49f4820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f49f0162c70 con 0x7f49f00b54a0 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46292/0 (socket says 172.21.15.106:46292) 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 -- 172.21.15.106:0/1233402096 learned_addr learned my addr 172.21.15.106:0/1233402096 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49eed9d640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b54a0 0x7f49f00cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49ee59c640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f00709e0 0x7f49f0152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 -- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b54a0 msgr2=0x7f49f00cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b54a0 0x7f49f00cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 -- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f00709e0 msgr2=0x7f49f0152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f00709e0 0x7f49f0152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.252+0000 7f49edd9b640 1 -- 172.21.15.106:0/1233402096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00a93e0 con 0x7f49f00b1c30 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49edd9b640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4040070 con 0x7f49f00b1c30 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4039bf0 con 0x7f49f00b1c30 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f00a9680 con 0x7f49f00b1c30 2024-04-13T03:10:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4035040 con 0x7f49f00b1c30 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.253+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f00a9c40 con 0x7f49f00b1c30 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.254+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f0068a10 con 0x7f49f00b1c30 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.254+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49e4038020 con 0x7f49f00b1c30 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.255+0000 7f49deffd640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 0x7f49d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.255+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f49e40ad920 con 0x7f49f00b1c30 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.255+0000 7f49ee59c640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 0x7f49d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.255+0000 7f49ee59c640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 0x7f49d006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009990 tx=0x7f49e0009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.259+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49e407b300 con 0x7f49f00b1c30 2024-04-13T03:10:13.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.432+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f49f00692c0 con 0x7f49f00b1c30 2024-04-13T03:10:13.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.433+0000 7f49deffd640 1 -- 172.21.15.106:0/1233402096 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f49e407f600 con 0x7f49f00b1c30 2024-04-13T03:10:13.434 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 msgr2=0x7f49d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 0x7f49d006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009990 tx=0x7f49e0009110 comp rx=0 tx=0).stop 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 msgr2=0x7f49f00cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e40046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 shutdown_connections 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f49d006c8f0 0x7f49d006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f49f00b54a0 0x7f49f00cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49f00b1c30 0x7f49f00cdcc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 --2- 172.21.15.106:0/1233402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f00709e0 0x7f49f0152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 >> 172.21.15.106:0/1233402096 conn(0x7f49f0071870 msgr2=0x7f49f006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 shutdown_connections 2024-04-13T03:10:13.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.435+0000 7f49f4820640 1 -- 172.21.15.106:0/1233402096 wait complete. 2024-04-13T03:10:13.447 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:13.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.549+0000 7fa3556c8640 1 Processor -- start 2024-04-13T03:10:13.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.549+0000 7fa3556c8640 1 -- start start 2024-04-13T03:10:13.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa3556c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa3556c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa3500b3d60 0x7fa3500d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa3556c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3500cfd20 con 0x7fa35015ed90 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa3556c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa350162af0 con 0x7fa3500b3d60 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa3556c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa350162dd0 con 0x7fa35015e9b0 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34f7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa3500b3d60 0x7fa3500d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34effd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34effd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:52000/0 (socket says 172.21.15.106:52000) 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34f7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa3500b3d60 0x7fa3500d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:33540/0 (socket says 172.21.15.106:33540) 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34effd640 1 -- 172.21.15.106:0/78185414 learned_addr learned my addr 172.21.15.106:0/78185414 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.550+0000 7fa34e7fc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa35015ed90 0x7fa3500d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42766/0 (socket says 172.21.15.106:42766) 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34f7fe640 1 -- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 msgr2=0x7fa3500d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34f7fe640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34f7fe640 1 -- 172.21.15.106:0/78185414 >> v1:172.21.15.114:6789/0 conn(0x7fa35015ed90 legacy=0x7fa3500d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34f7fe640 1 -- 172.21.15.106:0/78185414 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501630b0 con 0x7fa3500b3d60 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34effd640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.551+0000 7fa34f7fe640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa3500d0200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3480097a0 tx=0x7fa34802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4226ecca8a6d5c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa34dffb640 1 -- 172.21.15.106:0/78185414 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348034070 con 0x7fa3500b3d60 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa34dffb640 1 -- 172.21.15.106:0/78185414 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348030610 con 0x7fa3500b3d60 2024-04-13T03:10:13.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa34dffb640 1 -- 172.21.15.106:0/78185414 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480309e0 con 0x7fa3500b3d60 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 -- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 msgr2=0x7fa3500d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa3500d0200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3480097a0 tx=0x7fa34802b960 comp rx=0 tx=0).stop 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 -- 172.21.15.106:0/78185414 shutdown_connections 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa3500d0200 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 --2- 172.21.15.106:0/78185414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 -- 172.21.15.106:0/78185414 >> 172.21.15.106:0/78185414 conn(0x7fa35008bf10 msgr2=0x7fa3500b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 -- 172.21.15.106:0/78185414 shutdown_connections 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.552+0000 7fa3556c8640 1 -- 172.21.15.106:0/78185414 wait complete. 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.553+0000 7fa3556c8640 1 Processor -- start 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 -- start start 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa350159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa350162800 con 0x7fa3501f89f0 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa350162680 con 0x7fa3500b3d60 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa3556c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500d4d00 con 0x7fa35015e9b0 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46304/0 (socket says 172.21.15.106:46304) 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa34f7fe640 1 -- 172.21.15.106:0/2439485951 learned_addr learned my addr 172.21.15.106:0/2439485951 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa34effd640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa350159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.554+0000 7fa34e7fc640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 -- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 msgr2=0x7fa3500ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 -- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 msgr2=0x7fa350159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa350159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 -- 172.21.15.106:0/2439485951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35015fd90 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa34f7fe640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa34802b9a0 tx=0x7fa3480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348034040 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348030410 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501f7f10 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348006e10 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.555+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501f84d0 con 0x7fa3501f89f0 2024-04-13T03:10:13.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.556+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa350068a10 con 0x7fa3501f89f0 2024-04-13T03:10:13.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.556+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa348037070 con 0x7fa3501f89f0 2024-04-13T03:10:13.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.557+0000 7fa3377fe640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.557+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa3480aded0 con 0x7fa3501f89f0 2024-04-13T03:10:13.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.557+0000 7fa34effd640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.557+0000 7fa34effd640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3400072c0 tx=0x7fa34000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.561+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3480305e0 con 0x7fa3501f89f0 2024-04-13T03:10:13.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.736+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa350162240 con 0x7fa3501f89f0 2024-04-13T03:10:13.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.737+0000 7fa3377fe640 1 -- 172.21.15.106:0/2439485951 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa34807b9d0 con 0x7fa3501f89f0 2024-04-13T03:10:13.738 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 msgr2=0x7fa33c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3400072c0 tx=0x7fa34000b040 comp rx=0 tx=0).stop 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 msgr2=0x7fa3500ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa34802b9a0 tx=0x7fa3480046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 shutdown_connections 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa35015e9b0 0x7fa3500ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501f89f0 0x7fa3500ca9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 --2- 172.21.15.106:0/2439485951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500b3d60 0x7fa350159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 >> 172.21.15.106:0/2439485951 conn(0x7fa35008bf10 msgr2=0x7fa3500d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 shutdown_connections 2024-04-13T03:10:13.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.739+0000 7fa3556c8640 1 -- 172.21.15.106:0/2439485951 wait complete. 2024-04-13T03:10:13.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-13T03:10:13.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.850+0000 7f70c53aa640 1 Processor -- start 2024-04-13T03:10:13.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.850+0000 7f70c53aa640 1 -- start start 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70c53aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c006e9b0 0x7f70c00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70c53aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c0070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70c53aa640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c00b0f10 con 0x7f70c015b620 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70c53aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f70c00cf3b0 con 0x7f70c006e9b0 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70c53aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f70c015fc60 con 0x7f70c00b0800 2024-04-13T03:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70be7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c006e9b0 0x7f70c00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70be7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70c006e9b0 0x7f70c00706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46306/0 (socket says 172.21.15.106:46306) 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70be7fc640 1 -- 172.21.15.106:0/2627383484 learned_addr learned my addr 172.21.15.106:0/2627383484 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.851+0000 7f70bf7fe640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c0070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70be7fc640 1 -- 172.21.15.106:0/2627383484 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 msgr2=0x7f70c0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70be7fc640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c0070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70be7fc640 1 -- 172.21.15.106:0/2627383484 >> v1:172.21.15.114:6789/0 conn(0x7f70c015b620 legacy=0x7f70c00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70be7fc640 1 -- 172.21.15.106:0/2627383484 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c0160190 con 0x7f70c006e9b0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70bf7fe640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c0070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70be7fc640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c006e9b0 0x7f70c00706c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70c015fde0 tx=0x7f70b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9c7d406d57aafa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70bdffb640 1 -- 172.21.15.106:0/2627383484 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4035020 con 0x7f70c006e9b0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70bdffb640 1 -- 172.21.15.106:0/2627383484 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4031830 con 0x7f70c006e9b0 2024-04-13T03:10:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.852+0000 7f70bdffb640 1 -- 172.21.15.106:0/2627383484 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4031c00 con 0x7f70c006e9b0 2024-04-13T03:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 -- 172.21.15.106:0/2627383484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c006e9b0 msgr2=0x7f70c00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c006e9b0 0x7f70c00706c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70c015fde0 tx=0x7f70b402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 -- 172.21.15.106:0/2627383484 shutdown_connections 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c0070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 --2- 172.21.15.106:0/2627383484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c006e9b0 0x7f70c00706c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 -- 172.21.15.106:0/2627383484 >> 172.21.15.106:0/2627383484 conn(0x7f70c006f180 msgr2=0x7f70c006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 -- 172.21.15.106:0/2627383484 shutdown_connections 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.853+0000 7f70c53aa640 1 -- 172.21.15.106:0/2627383484 wait complete. 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.854+0000 7f70c53aa640 1 Processor -- start 2024-04-13T03:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.854+0000 7f70c53aa640 1 -- start start 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70c006e9b0 0x7f70c0148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c00a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00cf5e0 con 0x7f70c015b620 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00cfc30 con 0x7f70c006e9b0 2024-04-13T03:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70c53aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f70c00cf8c0 con 0x7f70c00b0800 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46314/0 (socket says 172.21.15.106:46314) 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 -- 172.21.15.106:0/695241681 learned_addr learned my addr 172.21.15.106:0/695241681 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70be7fc640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c00a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70beffd640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70c006e9b0 0x7f70c0148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 -- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 msgr2=0x7f70c00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c00a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 -- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70c006e9b0 msgr2=0x7f70c0148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.855+0000 7f70bf7fe640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70c006e9b0 0x7f70c0148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70bf7fe640 1 -- 172.21.15.106:0/695241681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c0148a50 con 0x7f70c015b620 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70bf7fe640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c7f0 tx=0x7f70b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8015070 con 0x7f70c015b620 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b8007820 con 0x7f70c015b620 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c014cb10 con 0x7f70c015b620 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007c10 con 0x7f70c015b620 2024-04-13T03:10:13.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.856+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c014d0d0 con 0x7f70c015b620 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.857+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c014d310 con 0x7f70c015b620 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.857+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70b8018070 con 0x7f70c015b620 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.858+0000 7f70a77fe640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 0x7f70ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.858+0000 7f70beffd640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 0x7f70ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.858+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f70b808c030 con 0x7f70c015b620 2024-04-13T03:10:13.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.858+0000 7f70beffd640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 0x7f70ac06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70c00b2d00 tx=0x7f70b000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:13.862+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70b8059a10 con 0x7f70c015b620 2024-04-13T03:10:14.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.105+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f70c015ccd0 con 0x7f70c015b620 2024-04-13T03:10:14.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.207+0000 7f70a77fe640 1 -- 172.21.15.106:0/695241681 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v71) ==== 145+0+0 (secure 0 0 0) 0x7f70b805dd10 con 0x7f70c015b620 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 msgr2=0x7f70ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 0x7f70ac06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70c00b2d00 tx=0x7f70b000d040 comp rx=0 tx=0).stop 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 msgr2=0x7f70c006ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c7f0 tx=0x7f70b800ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 shutdown_connections 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ac06c8f0 0x7f70ac06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70c00b0800 0x7f70c00a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c015b620 0x7f70c006ce40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 --2- 172.21.15.106:0/695241681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70c006e9b0 0x7f70c0148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 >> 172.21.15.106:0/695241681 conn(0x7f70c006f180 msgr2=0x7f70c0151680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:14.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 shutdown_connections 2024-04-13T03:10:14.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.209+0000 7f70c53aa640 1 -- 172.21.15.106:0/695241681 wait complete. 2024-04-13T03:10:14.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:10:14.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.324+0000 7f3f15436640 1 Processor -- start 2024-04-13T03:10:14.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.324+0000 7f3f15436640 1 -- start start 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f15436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3f1015ed90 0x7f3f100d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f15436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f100b3d60 0x7f3f100d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f15436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f100cfd20 con 0x7f3f1015e9b0 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f15436640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3f10162af0 con 0x7f3f100b3d60 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f15436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3f10162dd0 con 0x7f3f1015ed90 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f100b3d60 0x7f3f100d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f100b3d60 0x7f3f100d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46316/0 (socket says 172.21.15.106:46316) 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f1015e9b0 0x7f3f100d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37436/0 (socket says 172.21.15.106:37436) 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 -- 172.21.15.106:0/1965190373 learned_addr learned my addr 172.21.15.106:0/1965190373 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0e7fc640 1 --2- 172.21.15.106:0/1965190373 >> v2:172.21.15.114:3300/0 conn(0x7f3f1015ed90 0x7f3f100d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 -- 172.21.15.106:0/1965190373 >> v2:172.21.15.114:3300/0 conn(0x7f3f1015ed90 msgr2=0x7f3f100d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 --2- 172.21.15.106:0/1965190373 >> v2:172.21.15.114:3300/0 conn(0x7f3f1015ed90 0x7f3f100d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 -- 172.21.15.106:0/1965190373 >> v1:172.21.15.106:6789/0 conn(0x7f3f1015e9b0 legacy=0x7f3f100d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0dffb640 1 -- 172.21.15.106:0/1965190373 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587360323 0 0) 0x7f3f100cfd20 con 0x7f3f1015e9b0 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.325+0000 7f3f0f7fe640 1 -- 172.21.15.106:0/1965190373 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101630b0 con 0x7f3f100b3d60 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f0f7fe640 1 --2- 172.21.15.106:0/1965190373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f100b3d60 0x7f3f100d0200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f1008ebd0 tx=0x7f3f0802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ccc0a7fbc97aee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f0dffb640 1 -- 172.21.15.106:0/1965190373 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0802cbb0 con 0x7f3f100b3d60 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f0dffb640 1 -- 172.21.15.106:0/1965190373 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0802cd50 con 0x7f3f100b3d60 2024-04-13T03:10:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f0dffb640 1 -- 172.21.15.106:0/1965190373 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08034610 con 0x7f3f100b3d60 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f15436640 1 -- 172.21.15.106:0/1965190373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f100b3d60 msgr2=0x7f3f100d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f15436640 1 --2- 172.21.15.106:0/1965190373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f100b3d60 0x7f3f100d0200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f1008ebd0 tx=0x7f3f0802c010 comp rx=0 tx=0).stop 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.326+0000 7f3f15436640 1 -- 172.21.15.106:0/1965190373 shutdown_connections 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.327+0000 7f3f15436640 1 --2- 172.21.15.106:0/1965190373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f100b3d60 0x7f3f100d0200 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.327+0000 7f3f15436640 1 --2- 172.21.15.106:0/1965190373 >> v2:172.21.15.114:3300/0 conn(0x7f3f1015ed90 0x7f3f100d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.327+0000 7f3f15436640 1 -- 172.21.15.106:0/1965190373 >> 172.21.15.106:0/1965190373 conn(0x7f3f1008bf10 msgr2=0x7f3f100b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.327+0000 7f3f15436640 1 -- 172.21.15.106:0/1965190373 shutdown_connections 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.327+0000 7f3f15436640 1 -- 172.21.15.106:0/1965190373 wait complete. 2024-04-13T03:10:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.328+0000 7f3f15436640 1 Processor -- start 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.328+0000 7f3f15436640 1 -- start start 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 0x7f3f1015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f100b5000 0x7f3f100cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10162800 con 0x7f3f1015e9b0 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10162680 con 0x7f3f100b3d60 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3f100d4d00 con 0x7f3f100b5000 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46322/0 (socket says 172.21.15.106:46322) 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 -- 172.21.15.106:0/1355411577 learned_addr learned my addr 172.21.15.106:0/1355411577 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0effd640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 0x7f3f1015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0f7fe640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f100b5000 0x7f3f100cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 -- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f100b5000 msgr2=0x7f3f100cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f100b5000 0x7f3f100cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 -- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 msgr2=0x7f3f1015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 0x7f3f1015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 -- 172.21.15.106:0/1355411577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f100cca60 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0effd640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 0x7f3f1015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f0e7fc640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c7f0 tx=0x7f3f0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04015070 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f04004220 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f100ccd40 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04007620 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.329+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f100cd300 con 0x7f3f1015e9b0 2024-04-13T03:10:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.330+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f100cd500 con 0x7f3f1015e9b0 2024-04-13T03:10:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.331+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f040079d0 con 0x7f3f1015e9b0 2024-04-13T03:10:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.331+0000 7f3ef77fe640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.331+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3f0408c2a0 con 0x7f3f1015e9b0 2024-04-13T03:10:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.331+0000 7f3f0effd640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.332+0000 7f3f0effd640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 0x7f3efc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f00000f60 tx=0x7f3f00007c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.335+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f100cd500 con 0x7f3f1015e9b0 2024-04-13T03:10:14.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.510+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3f10068a10 con 0x7f3f1015e9b0 2024-04-13T03:10:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.515+0000 7f3ef77fe640 1 -- 172.21.15.106:0/1355411577 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f3f0405a2d0 con 0x7f3f1015e9b0 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 msgr2=0x7f3efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 0x7f3efc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f00000f60 tx=0x7f3f00007c30 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 msgr2=0x7f3f100b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c7f0 tx=0x7f3f0400ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 shutdown_connections 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f100b5000 0x7f3f100cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f1015e9b0 0x7f3f100b4ac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 --2- 172.21.15.106:0/1355411577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f100b3d60 0x7f3f1015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 >> 172.21.15.106:0/1355411577 conn(0x7f3f1008bf10 msgr2=0x7f3f100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 shutdown_connections 2024-04-13T03:10:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.517+0000 7f3f15436640 1 -- 172.21.15.106:0/1355411577 wait complete. 2024-04-13T03:10:14.530 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:10:14.530 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:10:14.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:10:14.629 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:10:14.629 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:14.629 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:14.645 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:14.645 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:14.701 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:14.701 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:14.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:14.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.895+0000 7f1d80805640 1 Processor -- start 2024-04-13T03:10:14.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.895+0000 7f1d80805640 1 -- start start 2024-04-13T03:10:14.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d80805640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1d7c06fe90 0x7f1d7c1626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d80805640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7c0ce6f0 con 0x7f1d7c0ce1f0 2024-04-13T03:10:14.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d80805640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7c15f420 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d80805640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1d7c15fa70 con 0x7f1d7c06fe90 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d79d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1d7c06fe90 0x7f1d7c1626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d7ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1d7c0ce1f0 0x7f1d7c162be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37438/0 (socket says 172.21.15.106:37438) 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d7ad9d640 1 -- 172.21.15.106:0/1678421168 learned_addr learned my addr 172.21.15.106:0/1678421168 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d7959a640 1 -- 172.21.15.106:0/1678421168 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647974984 0 0) 0x7f1d7c0ce6f0 con 0x7f1d7c0ce1f0 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d7959a640 1 -- 172.21.15.106:0/1678421168 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d64003520 con 0x7f1d7c0ce1f0 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d79d9b640 1 -- 172.21.15.106:0/1678421168 >> v1:172.21.15.114:6789/0 conn(0x7f1d7c15ebf0 legacy=0x7f1d7c06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d79d9b640 1 -- 172.21.15.106:0/1678421168 >> v1:172.21.15.106:6789/0 conn(0x7f1d7c0ce1f0 legacy=0x7f1d7c162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.896+0000 7f1d79d9b640 1 -- 172.21.15.106:0/1678421168 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c15f700 con 0x7f1d7c06fe90 2024-04-13T03:10:14.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.897+0000 7f1d79d9b640 1 --2- 172.21.15.106:0/1678421168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c06fe90 0x7f1d7c1626a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d700097a0 tx=0x7f1d7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacf1271164e882f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.897+0000 7f1d7959a640 1 -- 172.21.15.106:0/1678421168 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70035020 con 0x7f1d7c06fe90 2024-04-13T03:10:14.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.897+0000 7f1d7959a640 1 -- 172.21.15.106:0/1678421168 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d70031830 con 0x7f1d7c06fe90 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.897+0000 7f1d7959a640 1 -- 172.21.15.106:0/1678421168 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031be0 con 0x7f1d7c06fe90 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 -- 172.21.15.106:0/1678421168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c06fe90 msgr2=0x7f1d7c1626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 --2- 172.21.15.106:0/1678421168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c06fe90 0x7f1d7c1626a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d700097a0 tx=0x7f1d7002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 -- 172.21.15.106:0/1678421168 shutdown_connections 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 --2- 172.21.15.106:0/1678421168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c06fe90 0x7f1d7c1626a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 -- 172.21.15.106:0/1678421168 >> 172.21.15.106:0/1678421168 conn(0x7f1d7c0cd6d0 msgr2=0x7f1d7c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 -- 172.21.15.106:0/1678421168 shutdown_connections 2024-04-13T03:10:14.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.898+0000 7f1d80805640 1 -- 172.21.15.106:0/1678421168 wait complete. 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.899+0000 7f1d80805640 1 Processor -- start 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.899+0000 7f1d80805640 1 -- start start 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d7c06fe90 0x7f1d7c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d7c0ce1f0 0x7f1d7c0af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c0b0650 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c0b04d0 con 0x7f1d7c0ce1f0 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7c15efd0 con 0x7f1d7c06fe90 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46328/0 (socket says 172.21.15.106:46328) 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 -- 172.21.15.106:0/4014829719 learned_addr learned my addr 172.21.15.106:0/4014829719 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:14.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d79d9b640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d7c0ce1f0 0x7f1d7c0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7a59c640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d7c06fe90 0x7f1d7c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 -- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d7c06fe90 msgr2=0x7f1d7c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d7c06fe90 0x7f1d7c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 -- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d7c0ce1f0 msgr2=0x7f1d7c0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d7c0ce1f0 0x7f1d7c0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 -- 172.21.15.106:0/4014829719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c06bb30 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d7ad9d640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400cb10 tx=0x7f1d7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74017070 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7400ac70 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c06be10 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.900+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7400ae60 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.901+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c06c3d0 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.901+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c068a10 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.902+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d7401a070 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.902+0000 7f1d6affd640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:14.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.903+0000 7f1d7a59c640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:14.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.903+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1d7408bcd0 con 0x7f1d7c15ebf0 2024-04-13T03:10:14.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.903+0000 7f1d7a59c640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c006fd0 tx=0x7f1d6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:14.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:14.907+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d74059d90 con 0x7f1d7c15ebf0 2024-04-13T03:10:15.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.082+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d7c06dbb0 con 0x7f1d7c15ebf0 2024-04-13T03:10:15.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.083+0000 7f1d6affd640 1 -- 172.21.15.106:0/4014829719 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f1d74059690 con 0x7f1d7c15ebf0 2024-04-13T03:10:15.084 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:15.084 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 msgr2=0x7f1d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c006fd0 tx=0x7f1d6c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 msgr2=0x7f1d7c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400cb10 tx=0x7f1d7400a060 comp rx=0 tx=0).stop 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 shutdown_connections 2024-04-13T03:10:15.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d7c15ebf0 0x7f1d7c0b0ea0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d7c0ce1f0 0x7f1d7c0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 --2- 172.21.15.106:0/4014829719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d7c06fe90 0x7f1d7c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 >> 172.21.15.106:0/4014829719 conn(0x7f1d7c0cd6d0 msgr2=0x7f1d7c1624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 shutdown_connections 2024-04-13T03:10:15.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.085+0000 7f1d80805640 1 -- 172.21.15.106:0/4014829719 wait complete. 2024-04-13T03:10:15.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:15.098 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:15.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.199+0000 7f171895e640 1 Processor -- start 2024-04-13T03:10:15.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.200+0000 7f171895e640 1 -- start start 2024-04-13T03:10:15.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.200+0000 7f171895e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f17140704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.200+0000 7f171895e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17140b5cf0 con 0x7f17140709e0 2024-04-13T03:10:15.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.200+0000 7f171895e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17140b5fd0 con 0x7f17140b1c30 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.200+0000 7f171895e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f17140b62b0 con 0x7f171415ee40 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f1712d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f17140709e0 0x7f171406d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37452/0 (socket says 172.21.15.106:37452) 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f1712d9d640 1 -- 172.21.15.106:0/1785729943 learned_addr learned my addr 172.21.15.106:0/1785729943 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f1711d9b640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f17140704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565531435 0 0) 0x7f17140b5cf0 con 0x7f17140709e0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16fc003520 con 0x7f17140709e0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3017617366 0 0) 0x7f16fc003520 con 0x7f17140709e0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 msgr2=0x7f17140704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f17140704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 >> v1:172.21.15.114:6789/0 conn(0x7f17140b1c30 legacy=0x7f17140b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17140b6590 con 0x7f17140709e0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f170c0019f0 con 0x7f17140709e0 2024-04-13T03:10:15.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 >> v1:172.21.15.106:6789/0 conn(0x7f17140709e0 legacy=0x7f171406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f16fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16fc003f60 0x7f16fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16fc004d40 0x7f16fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17140b62b0 con 0x7f16fc003f60 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc00baa0 con 0x7f16fc004d40 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc00bca0 con 0x7f171415ee40 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f1711d9b640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16fc003f60 0x7f16fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f171259c640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f16fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.201+0000 7f1712d9d640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16fc004d40 0x7f16fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 -- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 msgr2=0x7f16fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f171415ee40 0x7f16fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 -- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16fc004d40 msgr2=0x7f16fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16fc004d40 0x7f16fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 -- 172.21.15.106:0/1785729943 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc003520 con 0x7f16fc003f60 2024-04-13T03:10:15.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f1711d9b640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16fc003f60 0x7f16fc004800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f16fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaaf917fce6bac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704039070 con 0x7f16fc003f60 2024-04-13T03:10:15.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170402c6b0 con 0x7f16fc003f60 2024-04-13T03:10:15.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.202+0000 7f171159a640 1 -- 172.21.15.106:0/1785729943 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704009690 con 0x7f16fc003f60 2024-04-13T03:10:15.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 -- 172.21.15.106:0/1785729943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16fc003f60 msgr2=0x7f16fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 --2- 172.21.15.106:0/1785729943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16fc003f60 0x7f16fc004800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f16fc005830 comp rx=0 tx=0).stop 2024-04-13T03:10:15.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f1712d9d640 1 -- 172.21.15.106:0/1785729943 reap_dead start 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 -- 172.21.15.106:0/1785729943 shutdown_connections 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 -- 172.21.15.106:0/1785729943 >> 172.21.15.106:0/1785729943 conn(0x7f1714071870 msgr2=0x7f1714071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 -- 172.21.15.106:0/1785729943 shutdown_connections 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.203+0000 7f171895e640 1 -- 172.21.15.106:0/1785729943 wait complete. 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.204+0000 7f171895e640 1 Processor -- start 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.204+0000 7f171895e640 1 -- start start 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 0x7f1714147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 0x7f1714157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1714070830 con 0x7f1714155890 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f17140706b0 con 0x7f1714157580 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f171895e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f171406dd70 con 0x7f16fc003f60 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f1711d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f1711d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46354/0 (socket says 172.21.15.106:46354) 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.205+0000 7f1711d9b640 1 -- 172.21.15.106:0/907575294 learned_addr learned my addr 172.21.15.106:0/907575294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1712d9d640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 0x7f1714157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f171259c640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 0x7f1714147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 -- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 msgr2=0x7f1714147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 0x7f1714147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 -- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 msgr2=0x7f1714157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 0x7f1714157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 -- 172.21.15.106:0/907575294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17140cd420 con 0x7f1714155890 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f171259c640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 0x7f1714147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1712d9d640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 0x7f1714157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1711d9b640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f1704004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704039050 con 0x7f1714155890 2024-04-13T03:10:15.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704045070 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17140cd700 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040044e0 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.206+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17140cdcc0 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.207+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1714068a10 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.208+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1704003850 con 0x7f1714155890 2024-04-13T03:10:15.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.208+0000 7f1702ffd640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 0x7f16f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.208+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f17040ae1a0 con 0x7f1714155890 2024-04-13T03:10:15.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.208+0000 7f171259c640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 0x7f16f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.209+0000 7f171259c640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 0x7f16f406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17080035f0 tx=0x7f1708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.213+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f170407c1d0 con 0x7f1714155890 2024-04-13T03:10:15.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.385+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17140711f0 con 0x7f1714155890 2024-04-13T03:10:15.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.386+0000 7f1702ffd640 1 -- 172.21.15.106:0/907575294 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f170407bad0 con 0x7f1714155890 2024-04-13T03:10:15.387 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 msgr2=0x7f16f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 0x7f16f406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17080035f0 tx=0x7f1708009040 comp rx=0 tx=0).stop 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 msgr2=0x7f1714155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f1704004730 comp rx=0 tx=0).stop 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 shutdown_connections 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16f406c920 0x7f16f406ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1714157580 0x7f1714157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1714155890 0x7f1714155d10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 --2- 172.21.15.106:0/907575294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16fc003f60 0x7f1714147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 >> 172.21.15.106:0/907575294 conn(0x7f1714071870 msgr2=0x7f171415d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 shutdown_connections 2024-04-13T03:10:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.388+0000 7f171895e640 1 -- 172.21.15.106:0/907575294 wait complete. 2024-04-13T03:10:15.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:15.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.501+0000 7f96ca9e9640 1 Processor -- start 2024-04-13T03:10:15.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.501+0000 7f96ca9e9640 1 -- start start 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96ca9e9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96c415ea80 0x7f96c40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96ca9e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96ca9e9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40cfe70 con 0x7f96c40d1630 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96ca9e9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f96c415ee60 con 0x7f96c415ea80 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96ca9e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96c4162af0 con 0x7f96c4162550 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96c415ea80 0x7f96c40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96c415ea80 0x7f96c40d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46370/0 (socket says 172.21.15.106:46370) 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 -- 172.21.15.106:0/2019901303 learned_addr learned my addr 172.21.15.106:0/2019901303 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c37fe640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 -- 172.21.15.106:0/2019901303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 msgr2=0x7f96c40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.502+0000 7f96c3fff640 1 -- 172.21.15.106:0/2019901303 >> v1:172.21.15.114:6789/0 conn(0x7f96c40d1630 legacy=0x7f96c40d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.503+0000 7f96c3fff640 1 -- 172.21.15.106:0/2019901303 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c4162dd0 con 0x7f96c415ea80 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.503+0000 7f96c3fff640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40d22b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009870 tx=0x7f96b402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0049aa99c7d296b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.503+0000 7f96c2ffd640 1 -- 172.21.15.106:0/2019901303 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4034070 con 0x7f96c415ea80 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.503+0000 7f96c2ffd640 1 -- 172.21.15.106:0/2019901303 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4030840 con 0x7f96c415ea80 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.503+0000 7f96c2ffd640 1 -- 172.21.15.106:0/2019901303 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4030c10 con 0x7f96c415ea80 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2019901303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 msgr2=0x7f96c40d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40d22b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009870 tx=0x7f96b402b980 comp rx=0 tx=0).stop 2024-04-13T03:10:15.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2019901303 shutdown_connections 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2019901303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40d22b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2019901303 >> 172.21.15.106:0/2019901303 conn(0x7f96c409da50 msgr2=0x7f96c40b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2019901303 shutdown_connections 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.504+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2019901303 wait complete. 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.505+0000 7f96ca9e9640 1 Processor -- start 2024-04-13T03:10:15.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.505+0000 7f96ca9e9640 1 -- start start 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c40d1630 0x7f96c414cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c41602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40b3e70 con 0x7f96c415ea80 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4163390 con 0x7f96c40d1630 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96ca9e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40d1aa0 con 0x7f96c4162550 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46386/0 (socket says 172.21.15.106:46386) 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 -- 172.21.15.106:0/2742353647 learned_addr learned my addr 172.21.15.106:0/2742353647 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c3fff640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c40d1630 0x7f96c414cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c8f86640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c41602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 -- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 msgr2=0x7f96c41602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c41602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 -- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c40d1630 msgr2=0x7f96c414cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c40d1630 0x7f96c414cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.506+0000 7f96c37fe640 1 -- 172.21.15.106:0/2742353647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c40cc870 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96c37fe640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b800d5f0 tx=0x7f96b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80125f0 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b8019070 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c40ccb50 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80129d0 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.507+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c40cd110 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.508+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c4068a10 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.508+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96b8012b70 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.509+0000 7f96c0ff9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 0x7f96b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.509+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f96b808c1b0 con 0x7f96c415ea80 2024-04-13T03:10:15.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.509+0000 7f96c3fff640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 0x7f96b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.509+0000 7f96c3fff640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 0x7f96b006ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96b4036000 tx=0x7f96b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.513+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96b8059b10 con 0x7f96c415ea80 2024-04-13T03:10:15.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.685+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96c40b39c0 con 0x7f96c415ea80 2024-04-13T03:10:15.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.686+0000 7f96c0ff9640 1 -- 172.21.15.106:0/2742353647 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f96b805de10 con 0x7f96c415ea80 2024-04-13T03:10:15.687 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 msgr2=0x7f96b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 0x7f96b006ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96b4036000 tx=0x7f96b4008030 comp rx=0 tx=0).stop 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 msgr2=0x7f96c40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b800d5f0 tx=0x7f96b800dab0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 shutdown_connections 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96b006c920 0x7f96b006ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c4162550 0x7f96c41602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c415ea80 0x7f96c40cd910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 --2- 172.21.15.106:0/2742353647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c40d1630 0x7f96c414cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 >> 172.21.15.106:0/2742353647 conn(0x7f96c409da50 msgr2=0x7f96c40b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 shutdown_connections 2024-04-13T03:10:15.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.688+0000 7f96ca9e9640 1 -- 172.21.15.106:0/2742353647 wait complete. 2024-04-13T03:10:15.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:15.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.801+0000 7f28d3b1e640 1 Processor -- start 2024-04-13T03:10:15.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 -- start start 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f28cc0b3d60 0x7f28cc0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc0cfd20 con 0x7f28cc15ed90 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc162af0 con 0x7f28cc0b3d60 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.802+0000 7f28d3b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc162dd0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:52124/0 (socket says 172.21.15.106:52124) 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 -- 172.21.15.106:0/3746927645 learned_addr learned my addr 172.21.15.106:0/3746927645 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d20bb640 1 --2- 172.21.15.106:0/3746927645 >> v2:172.21.15.114:3300/0 conn(0x7f28cc0b3d60 0x7f28cc0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 -- 172.21.15.106:0/3746927645 >> v2:172.21.15.114:3300/0 conn(0x7f28cc0b3d60 msgr2=0x7f28cc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 --2- 172.21.15.106:0/3746927645 >> v2:172.21.15.114:3300/0 conn(0x7f28cc0b3d60 0x7f28cc0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 -- 172.21.15.106:0/3746927645 >> v1:172.21.15.114:6789/0 conn(0x7f28cc15ed90 legacy=0x7f28cc0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.803+0000 7f28d18ba640 1 -- 172.21.15.106:0/3746927645 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc1630b0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d18ba640 1 --2- 172.21.15.106:0/3746927645 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009870 tx=0x7f28c802ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e257839235af02b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d08b8640 1 -- 172.21.15.106:0/3746927645 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c802f840 con 0x7f28cc15e9b0 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d08b8640 1 -- 172.21.15.106:0/3746927645 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c802f9e0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d08b8640 1 -- 172.21.15.106:0/3746927645 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c802fdb0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d3b1e640 1 -- 172.21.15.106:0/3746927645 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 msgr2=0x7f28cc0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/3746927645 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009870 tx=0x7f28c802ba20 comp rx=0 tx=0).stop 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d3b1e640 1 -- 172.21.15.106:0/3746927645 shutdown_connections 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.804+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/3746927645 >> v2:172.21.15.114:3300/0 conn(0x7f28cc0b3d60 0x7f28cc0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.805+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/3746927645 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc15e9b0 0x7f28cc0d2300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.805+0000 7f28d3b1e640 1 -- 172.21.15.106:0/3746927645 >> 172.21.15.106:0/3746927645 conn(0x7f28cc08bf10 msgr2=0x7f28cc0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.805+0000 7f28d3b1e640 1 -- 172.21.15.106:0/3746927645 shutdown_connections 2024-04-13T03:10:15.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.805+0000 7f28d3b1e640 1 -- 172.21.15.106:0/3746927645 wait complete. 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 Processor -- start 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 -- start start 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc0b3d60 0x7f28cc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28cc0b5b60 0x7f28cc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc162800 con 0x7f28cc15e9b0 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc162680 con 0x7f28cc0b5b60 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.806+0000 7f28d3b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc0d4d00 con 0x7f28cc0b3d60 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46398/0 (socket says 172.21.15.106:46398) 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 -- 172.21.15.106:0/2523048728 learned_addr learned my addr 172.21.15.106:0/2523048728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:15.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d18ba640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc0b3d60 0x7f28cc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d20bb640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28cc0b5b60 0x7f28cc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 -- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc0b3d60 msgr2=0x7f28cc1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc0b3d60 0x7f28cc1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 -- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28cc0b5b60 msgr2=0x7f28cc15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28cc0b5b60 0x7f28cc15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 -- 172.21.15.106:0/2523048728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc06bac0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d10b9640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c390 tx=0x7f28bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc017070 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc0052d0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc0b2b80 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc003830 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.807+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc0c93a0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.808+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc068a10 con 0x7f28cc15e9b0 2024-04-13T03:10:15.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.809+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28bc01a070 con 0x7f28cc15e9b0 2024-04-13T03:10:15.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.809+0000 7f28c27fc640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:15.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.809+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f28bc08b1c0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.809+0000 7f28d18ba640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:15.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.809+0000 7f28d18ba640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 0x7f28b006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28c8036000 tx=0x7f28c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:15.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.813+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28cc06bac0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.988+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28cc0d53a0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.988+0000 7f28c27fc640 1 -- 172.21.15.106:0/2523048728 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f28bc058bb0 con 0x7f28cc15e9b0 2024-04-13T03:10:15.989 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:15.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 msgr2=0x7f28b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 0x7f28b006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28c8036000 tx=0x7f28c8008030 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 msgr2=0x7f28cc15cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c390 tx=0x7f28bc001e70 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 shutdown_connections 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28cc0b5b60 0x7f28cc15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28cc15e9b0 0x7f28cc15cb70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 --2- 172.21.15.106:0/2523048728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28cc0b3d60 0x7f28cc1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 >> 172.21.15.106:0/2523048728 conn(0x7f28cc08bf10 msgr2=0x7f28cc1618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 shutdown_connections 2024-04-13T03:10:15.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:15.990+0000 7f28d3b1e640 1 -- 172.21.15.106:0/2523048728 wait complete. 2024-04-13T03:10:16.002 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:16.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.104+0000 7f01d7898640 1 Processor -- start 2024-04-13T03:10:16.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.104+0000 7f01d7898640 1 -- start start 2024-04-13T03:10:16.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d7898640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d7898640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 0x7f01d006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d7898640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d00b5cf0 con 0x7f01d00b54a0 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d7898640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00b5fd0 con 0x7f01d0070bb0 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d7898640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00b62b0 con 0x7f01d00b1c30 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5e35640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 0x7f01d006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d4e33640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f01d00b54a0 0x7f01d00704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42808/0 (socket says 172.21.15.106:42808) 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5e35640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 0x7f01d006d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:33652/0 (socket says 172.21.15.106:33652) 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:52142/0 (socket says 172.21.15.106:52142) 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d4e33640 1 -- 172.21.15.106:0/3907626593 learned_addr learned my addr 172.21.15.106:0/3907626593 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 -- 172.21.15.106:0/3907626593 >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 msgr2=0x7f01d006d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 --2- 172.21.15.106:0/3907626593 >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 0x7f01d006d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 -- 172.21.15.106:0/3907626593 >> v1:172.21.15.114:6789/0 conn(0x7f01d00b54a0 legacy=0x7f01d00704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.105+0000 7f01d5634640 1 -- 172.21.15.106:0/3907626593 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00b6590 con 0x7f01d00b1c30 2024-04-13T03:10:16.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.106+0000 7f01d5634640 1 --2- 172.21.15.106:0/3907626593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0097d0 tx=0x7f01cc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c681e46bd9dfe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.106+0000 7f01c7fff640 1 -- 172.21.15.106:0/3907626593 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc035020 con 0x7f01d00b1c30 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.106+0000 7f01c7fff640 1 -- 172.21.15.106:0/3907626593 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc031800 con 0x7f01d00b1c30 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.106+0000 7f01c7fff640 1 -- 172.21.15.106:0/3907626593 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc031bd0 con 0x7f01d00b1c30 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 -- 172.21.15.106:0/3907626593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 msgr2=0x7f01d015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 --2- 172.21.15.106:0/3907626593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0097d0 tx=0x7f01cc02b8a0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 -- 172.21.15.106:0/3907626593 shutdown_connections 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 --2- 172.21.15.106:0/3907626593 >> v2:172.21.15.114:3300/0 conn(0x7f01d0070bb0 0x7f01d006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 --2- 172.21.15.106:0/3907626593 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d015ee40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 -- 172.21.15.106:0/3907626593 >> 172.21.15.106:0/3907626593 conn(0x7f01d0071870 msgr2=0x7f01d0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 -- 172.21.15.106:0/3907626593 shutdown_connections 2024-04-13T03:10:16.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.107+0000 7f01d7898640 1 -- 172.21.15.106:0/3907626593 wait complete. 2024-04-13T03:10:16.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.108+0000 7f01d7898640 1 Processor -- start 2024-04-13T03:10:16.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.108+0000 7f01d7898640 1 -- start start 2024-04-13T03:10:16.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00b54a0 0x7f01d0160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01d006dd20 con 0x7f01d0070bb0 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01d006dba0 con 0x7f01d00b54a0 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d7898640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01d006dea0 con 0x7f01d00b1c30 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46404/0 (socket says 172.21.15.106:46404) 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 -- 172.21.15.106:0/3242671165 learned_addr learned my addr 172.21.15.106:0/3242671165 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d4e33640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5e35640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00b54a0 0x7f01d0160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 -- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 msgr2=0x7f01d00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 -- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00b54a0 msgr2=0x7f01d0160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00b54a0 0x7f01d0160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.109+0000 7f01d5634640 1 -- 172.21.15.106:0/3242671165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00ce1b0 con 0x7f01d0070bb0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01d5634640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02beb0 tx=0x7f01cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc035040 con 0x7f01d0070bb0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc03ad00 con 0x7f01d0070bb0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d00cdbf0 con 0x7f01d0070bb0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0466a0 con 0x7f01d0070bb0 2024-04-13T03:10:16.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.110+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00cfdb0 con 0x7f01d0070bb0 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.111+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d0068a10 con 0x7f01d0070bb0 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.111+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01cc038020 con 0x7f01d0070bb0 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.112+0000 7f01c5ffb640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 0x7f01b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.112+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f01cc0add40 con 0x7f01d0070bb0 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.112+0000 7f01d4e33640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 0x7f01b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.112+0000 7f01d4e33640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 0x7f01b406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01d015ad60 tx=0x7f01c0007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.116+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01cc07b850 con 0x7f01d0070bb0 2024-04-13T03:10:16.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.290+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f01d015cde0 con 0x7f01d0070bb0 2024-04-13T03:10:16.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.291+0000 7f01c5ffb640 1 -- 172.21.15.106:0/3242671165 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f01cc07fb50 con 0x7f01d0070bb0 2024-04-13T03:10:16.292 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 msgr2=0x7f01b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 0x7f01b406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01d015ad60 tx=0x7f01c0007e80 comp rx=0 tx=0).stop 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 msgr2=0x7f01d0147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02beb0 tx=0x7f01cc0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 shutdown_connections 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f01b406c920 0x7f01b406ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01d00b1c30 0x7f01d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01d00b54a0 0x7f01d0160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 --2- 172.21.15.106:0/3242671165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d0070bb0 0x7f01d0147030 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 >> 172.21.15.106:0/3242671165 conn(0x7f01d0071870 msgr2=0x7f01d015d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:16.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 shutdown_connections 2024-04-13T03:10:16.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.293+0000 7f01d7898640 1 -- 172.21.15.106:0/3242671165 wait complete. 2024-04-13T03:10:16.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:10:16.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 Processor -- start 2024-04-13T03:10:16.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 -- start start 2024-04-13T03:10:16.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86e015e9b0 0x7f86e00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e00b3d60 0x7f86e00d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e00cfd20 con 0x7f86e015ed90 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f86e0162af0 con 0x7f86e015e9b0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.406+0000 7f86e7036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f86e0162dd0 con 0x7f86e00b3d60 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86e015e9b0 0x7f86e00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86e015e9b0 0x7f86e00d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46406/0 (socket says 172.21.15.106:46406) 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 -- 172.21.15.106:0/3929474894 learned_addr learned my addr 172.21.15.106:0/3929474894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e55d3640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e00b3d60 0x7f86e00d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 -- 172.21.15.106:0/3929474894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e00b3d60 msgr2=0x7f86e00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e00b3d60 0x7f86e00d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 -- 172.21.15.106:0/3929474894 >> v1:172.21.15.114:6789/0 conn(0x7f86e015ed90 legacy=0x7f86e00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.407+0000 7f86e4dd2640 1 -- 172.21.15.106:0/3929474894 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e01630b0 con 0x7f86e015e9b0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e4dd2640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e015e9b0 0x7f86e00d2300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009e50 tx=0x7f86d802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fc1dcabc22ecbe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86df7fe640 1 -- 172.21.15.106:0/3929474894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d802cb90 con 0x7f86e015e9b0 2024-04-13T03:10:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86df7fe640 1 -- 172.21.15.106:0/3929474894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d802cd30 con 0x7f86e015e9b0 2024-04-13T03:10:16.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86df7fe640 1 -- 172.21.15.106:0/3929474894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8034610 con 0x7f86e015e9b0 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 -- 172.21.15.106:0/3929474894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e015e9b0 msgr2=0x7f86e00d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e015e9b0 0x7f86e00d2300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009e50 tx=0x7f86d802bff0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 -- 172.21.15.106:0/3929474894 shutdown_connections 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e00b3d60 0x7f86e00d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 --2- 172.21.15.106:0/3929474894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e015e9b0 0x7f86e00d2300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 -- 172.21.15.106:0/3929474894 >> 172.21.15.106:0/3929474894 conn(0x7f86e008bf10 msgr2=0x7f86e00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 -- 172.21.15.106:0/3929474894 shutdown_connections 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.408+0000 7f86e7036640 1 -- 172.21.15.106:0/3929474894 wait complete. 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.409+0000 7f86e7036640 1 Processor -- start 2024-04-13T03:10:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 -- start start 2024-04-13T03:10:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e015e9b0 0x7f86e00b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86e00b6750 0x7f86e01515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0162800 con 0x7f86e00b3d60 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0162680 con 0x7f86e00b6750 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e7036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f86e00d4d00 con 0x7f86e015e9b0 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e4dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e4dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46408/0 (socket says 172.21.15.106:46408) 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.410+0000 7f86e4dd2640 1 -- 172.21.15.106:0/2156254420 learned_addr learned my addr 172.21.15.106:0/2156254420 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e55d3640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86e00b6750 0x7f86e01515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86dffff640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e015e9b0 0x7f86e00b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 -- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e015e9b0 msgr2=0x7f86e00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e015e9b0 0x7f86e00b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 -- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86e00b6750 msgr2=0x7f86e01515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86e00b6750 0x7f86e01515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 -- 172.21.15.106:0/2156254420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e006b8e0 con 0x7f86e00b3d60 2024-04-13T03:10:16.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e4dd2640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f80 tx=0x7f86d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8043020 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d803e450 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e00b6d60 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8009510 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.411+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e015fc80 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.412+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0068a10 con 0x7f86e00b3d60 2024-04-13T03:10:16.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.413+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d8046020 con 0x7f86e00b3d60 2024-04-13T03:10:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.413+0000 7f86dd7fa640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.413+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f86d80ae540 con 0x7f86e00b3d60 2024-04-13T03:10:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.413+0000 7f86dffff640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.413+0000 7f86dffff640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86e015c430 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.417+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86d804d030 con 0x7f86e00b3d60 2024-04-13T03:10:16.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.604+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86e00d53a0 con 0x7f86e00b3d60 2024-04-13T03:10:16.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.605+0000 7f86dd7fa640 1 -- 172.21.15.106:0/2156254420 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f86d807c580 con 0x7f86e00b3d60 2024-04-13T03:10:16.606 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 msgr2=0x7f86c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86e015c430 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 msgr2=0x7f86e00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f80 tx=0x7f86d8004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 shutdown_connections 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86e015e9b0 0x7f86e00b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86e00b6750 0x7f86e01515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 --2- 172.21.15.106:0/2156254420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86e00b3d60 0x7f86e00b43b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 >> 172.21.15.106:0/2156254420 conn(0x7f86e008bf10 msgr2=0x7f86e015c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 shutdown_connections 2024-04-13T03:10:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.607+0000 7f86e7036640 1 -- 172.21.15.106:0/2156254420 wait complete. 2024-04-13T03:10:16.619 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:10:16.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:16.619 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-13T03:10:16.647 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:10:16.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:16.647 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:10:16.702 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:16.703 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:16.721 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:16.721 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:16.780 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:16.780 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:16.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:16.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.937+0000 7f53e97f7640 1 Processor -- start 2024-04-13T03:10:16.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.937+0000 7f53e97f7640 1 -- start start 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e97f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e97f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f53e40b3d60 0x7f53e40d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e97f7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40cfd20 con 0x7f53e415ed90 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e97f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f53e4162af0 con 0x7f53e40b3d60 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e97f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f53e4162dd0 con 0x7f53e415e9b0 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e2ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e37fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f53e40b3d60 0x7f53e40d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53dbfff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f53e415ed90 0x7f53e40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42824/0 (socket says 172.21.15.106:42824) 2024-04-13T03:10:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53e2ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:52176/0 (socket says 172.21.15.106:52176) 2024-04-13T03:10:16.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.938+0000 7f53dbfff640 1 -- 172.21.15.106:0/1692608210 learned_addr learned my addr 172.21.15.106:0/1692608210 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e37fe640 1 -- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 msgr2=0x7f53e40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e37fe640 1 -- 172.21.15.106:0/1692608210 >> v1:172.21.15.114:6789/0 conn(0x7f53e415ed90 legacy=0x7f53e40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e37fe640 1 -- 172.21.15.106:0/1692608210 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41630b0 con 0x7f53e40b3d60 2024-04-13T03:10:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.939+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e40b3d60 0x7f53e40d0200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a140f56cda85dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e27fc640 1 -- 172.21.15.106:0/1692608210 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc039070 con 0x7f53e40b3d60 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e27fc640 1 -- 172.21.15.106:0/1692608210 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc02cce0 con 0x7f53e40b3d60 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e27fc640 1 -- 172.21.15.106:0/1692608210 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc034660 con 0x7f53e40b3d60 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 -- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e40b3d60 msgr2=0x7f53e40d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e40b3d60 0x7f53e40d0200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02bf30 comp rx=0 tx=0).stop 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 -- 172.21.15.106:0/1692608210 shutdown_connections 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e40b3d60 0x7f53e40d0200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1692608210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e415e9b0 0x7f53e40d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 -- 172.21.15.106:0/1692608210 >> 172.21.15.106:0/1692608210 conn(0x7f53e408bf10 msgr2=0x7f53e40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 -- 172.21.15.106:0/1692608210 shutdown_connections 2024-04-13T03:10:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.940+0000 7f53e97f7640 1 -- 172.21.15.106:0/1692608210 wait complete. 2024-04-13T03:10:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.941+0000 7f53e97f7640 1 Processor -- start 2024-04-13T03:10:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.941+0000 7f53e97f7640 1 -- start start 2024-04-13T03:10:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e40b3d60 0x7f53e41595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e415e9b0 0x7f53e406c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4162800 con 0x7f53e406d6a0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4162680 con 0x7f53e415e9b0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e97f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40d4d00 con 0x7f53e40b3d60 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46418/0 (socket says 172.21.15.106:46418) 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 -- 172.21.15.106:0/1240754226 learned_addr learned my addr 172.21.15.106:0/1240754226 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53dbfff640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e415e9b0 0x7f53e406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e40b3d60 0x7f53e41595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 -- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e40b3d60 msgr2=0x7f53e41595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e40b3d60 0x7f53e41595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 -- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e415e9b0 msgr2=0x7f53e406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e415e9b0 0x7f53e406c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.942+0000 7f53e37fe640 1 -- 172.21.15.106:0/1240754226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e406dc50 con 0x7f53e406d6a0 2024-04-13T03:10:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc039050 con 0x7f53e406d6a0 2024-04-13T03:10:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc045070 con 0x7f53e406d6a0 2024-04-13T03:10:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e406df30 con 0x7f53e406d6a0 2024-04-13T03:10:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009450 con 0x7f53e406d6a0 2024-04-13T03:10:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.943+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e406e4f0 con 0x7f53e406d6a0 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.944+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e4068a10 con 0x7f53e406d6a0 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.944+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53dc03e850 con 0x7f53e406d6a0 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.945+0000 7f53db7fe640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 0x7f53c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.945+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f53dc0aebf0 con 0x7f53e406d6a0 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.945+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 0x7f53c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:16.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.945+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 0x7f53c406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009560 tx=0x7f53d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:16.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:16.949+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53dc032e20 con 0x7f53e406d6a0 2024-04-13T03:10:17.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.127+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53e40d53a0 con 0x7f53e406d6a0 2024-04-13T03:10:17.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.128+0000 7f53db7fe640 1 -- 172.21.15.106:0/1240754226 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f53dc07cc20 con 0x7f53e406d6a0 2024-04-13T03:10:17.129 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:17.129 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 msgr2=0x7f53c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 0x7f53c406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009560 tx=0x7f53d4008040 comp rx=0 tx=0).stop 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 msgr2=0x7f53e40ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 shutdown_connections 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53c406c8f0 0x7f53c406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e406d6a0 0x7f53e40ca3f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53e415e9b0 0x7f53e406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 --2- 172.21.15.106:0/1240754226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53e40b3d60 0x7f53e41595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 >> 172.21.15.106:0/1240754226 conn(0x7f53e408bf10 msgr2=0x7f53e40b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 shutdown_connections 2024-04-13T03:10:17.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.130+0000 7f53e97f7640 1 -- 172.21.15.106:0/1240754226 wait complete. 2024-04-13T03:10:17.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:17.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:17.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.246+0000 7fdec3888640 1 Processor -- start 2024-04-13T03:10:17.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.246+0000 7fdec3888640 1 -- start start 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec3888640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdebc15fb50 0x7fdebc06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec3888640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdebc06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec3888640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdebc0cef80 con 0x7fdebc0d07c0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec3888640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc0cf260 con 0x7fdebc15fb50 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec3888640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc0cf540 con 0x7fdebc06f6c0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec0e23640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdebc0d07c0 0x7fdebc162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37464/0 (socket says 172.21.15.106:37464) 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec0e23640 1 -- 172.21.15.106:0/480450522 learned_addr learned my addr 172.21.15.106:0/480450522 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec1e25640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdebc06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359627096 0 0) 0x7fdebc0cef80 con 0x7fdebc0d07c0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdec1624640 1 --2- 172.21.15.106:0/480450522 >> v2:172.21.15.114:3300/0 conn(0x7fdebc15fb50 0x7fdebc06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdea8003520 con 0x7fdebc0d07c0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1084000352 0 0) 0x7fdea8003520 con 0x7fdebc0d07c0 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 msgr2=0x7fdebc06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdebc06be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 >> v2:172.21.15.114:3300/0 conn(0x7fdebc15fb50 msgr2=0x7fdebc06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 --2- 172.21.15.106:0/480450522 >> v2:172.21.15.114:3300/0 conn(0x7fdebc15fb50 0x7fdebc06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc0cf820 con 0x7fdebc0d07c0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.247+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fdeac001670 con 0x7fdebc0d07c0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 >> v1:172.21.15.106:6789/0 conn(0x7fdebc0d07c0 legacy=0x7fdebc162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdea80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea8003f30 0x7fdea8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdea8004d40 0x7fdea80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0cf260 con 0x7fdea8003f30 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0cf540 con 0x7fdea8004d40 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdea800ba10 con 0x7fdebc06f6c0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea8003f30 0x7fdea8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec1624640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdea80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec1e25640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdea8004d40 0x7fdea80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 -- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 msgr2=0x7fdea80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdebc06f6c0 0x7fdea80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 -- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdea8004d40 msgr2=0x7fdea80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdea8004d40 0x7fdea80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.248+0000 7fdec0e23640 1 -- 172.21.15.106:0/480450522 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea8003520 con 0x7fdea8003f30 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec0e23640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea8003f30 0x7fdea8004800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007fd0 tx=0x7fdea8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bf8ad9e56f7b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac007d50 con 0x7fdea8003f30 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac030070 con 0x7fdea8003f30 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdeb3fff640 1 -- 172.21.15.106:0/480450522 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac005d10 con 0x7fdea8003f30 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 -- 172.21.15.106:0/480450522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea8003f30 msgr2=0x7fdea8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 --2- 172.21.15.106:0/480450522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea8003f30 0x7fdea8004800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007fd0 tx=0x7fdea8005830 comp rx=0 tx=0).stop 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec1e25640 1 -- 172.21.15.106:0/480450522 reap_dead start 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 -- 172.21.15.106:0/480450522 shutdown_connections 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 -- 172.21.15.106:0/480450522 >> 172.21.15.106:0/480450522 conn(0x7fdebc0cd6a0 msgr2=0x7fdebc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 -- 172.21.15.106:0/480450522 shutdown_connections 2024-04-13T03:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.249+0000 7fdec3888640 1 -- 172.21.15.106:0/480450522 wait complete. 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 Processor -- start 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 -- start start 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdea8003f30 0x7fdebc157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdebc1561d0 0x7fdebc0b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0ced20 con 0x7fdebc1f4a70 2024-04-13T03:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0ceba0 con 0x7fdebc1561d0 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec3888640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc15ef10 con 0x7fdea8003f30 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec0e23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec0e23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46426/0 (socket says 172.21.15.106:46426) 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec0e23640 1 -- 172.21.15.106:0/4168158321 learned_addr learned my addr 172.21.15.106:0/4168158321 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec1624640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdea8003f30 0x7fdebc157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.251+0000 7fdec1e25640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdebc1561d0 0x7fdebc0b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 -- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdea8003f30 msgr2=0x7fdebc157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdea8003f30 0x7fdebc157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 -- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdebc1561d0 msgr2=0x7fdebc0b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdebc1561d0 0x7fdebc0b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 -- 172.21.15.106:0/4168158321 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1f7d80 con 0x7fdebc1f4a70 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec0e23640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c300 tx=0x7fdeac000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac001cb0 con 0x7fdebc1f4a70 2024-04-13T03:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac004bf0 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1569e0 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac007070 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.252+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc0b3e30 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.253+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc0b3940 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.254+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeac048020 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.254+0000 7fdeb1ffb640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 0x7fdea006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.254+0000 7fdec1624640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 0x7fdea006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.254+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdeac0ad020 con 0x7fdebc1f4a70 2024-04-13T03:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.254+0000 7fdec1624640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 0x7fdea006ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4014510 tx=0x7fdeb400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.258+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdeac07ab50 con 0x7fdebc1f4a70 2024-04-13T03:10:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.434+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdebc068a10 con 0x7fdebc1f4a70 2024-04-13T03:10:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.435+0000 7fdeb1ffb640 1 -- 172.21.15.106:0/4168158321 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fdeac07ee50 con 0x7fdebc1f4a70 2024-04-13T03:10:17.436 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 msgr2=0x7fdea006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 0x7fdea006ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4014510 tx=0x7fdeb400b040 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 msgr2=0x7fdebc1f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c300 tx=0x7fdeac000fe0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 shutdown_connections 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdea006c920 0x7fdea006ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdebc1561d0 0x7fdebc0b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdebc1f4a70 0x7fdebc1f7840 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 --2- 172.21.15.106:0/4168158321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdea8003f30 0x7fdebc157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 >> 172.21.15.106:0/4168158321 conn(0x7fdebc0cd6a0 msgr2=0x7fdebc162400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 shutdown_connections 2024-04-13T03:10:17.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.437+0000 7fdec3888640 1 -- 172.21.15.106:0/4168158321 wait complete. 2024-04-13T03:10:17.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:17.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.550+0000 7f57e6ede640 1 Processor -- start 2024-04-13T03:10:17.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.550+0000 7f57e6ede640 1 -- start start 2024-04-13T03:10:17.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e6ede640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f57e00704a0 0x7f57e00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e6ede640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e015ddd0 con 0x7f57e00cfea0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e6ede640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e015d4d0 con 0x7f57e00d1a60 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e6ede640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f57e015e100 con 0x7f57e00704a0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e4c7a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f57e00cfea0 0x7f57e015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37472/0 (socket says 172.21.15.106:37472) 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.551+0000 7f57e4c7a640 1 -- 172.21.15.106:0/2450866951 learned_addr learned my addr 172.21.15.106:0/2450866951 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330938154 0 0) 0x7f57e015ddd0 con 0x7f57e00cfea0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57d0003520 con 0x7f57e00cfea0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57dffff640 1 --2- 172.21.15.106:0/2450866951 >> v2:172.21.15.114:3300/0 conn(0x7f57e00704a0 0x7f57e00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1800576730 0 0) 0x7f57d0003520 con 0x7f57e00cfea0 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 >> v2:172.21.15.114:3300/0 conn(0x7f57e00704a0 msgr2=0x7f57e00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 --2- 172.21.15.106:0/2450866951 >> v2:172.21.15.114:3300/0 conn(0x7f57e00704a0 0x7f57e00d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 >> v1:172.21.15.114:6789/0 conn(0x7f57e00d1a60 legacy=0x7f57e015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e015d7b0 con 0x7f57e00cfea0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f57d4001670 con 0x7f57e00cfea0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 >> v1:172.21.15.106:6789/0 conn(0x7f57e00cfea0 legacy=0x7f57e015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57d00039b0 0x7f57d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d0004800 0x7f57d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 0x7f57d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57e015e100 con 0x7f57d00051c0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57d000c060 con 0x7f57d00039b0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57d000c260 con 0x7f57d0004800 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 0x7f57d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e4c7a640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57d00039b0 0x7f57d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57dffff640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d0004800 0x7f57d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 -- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d0004800 msgr2=0x7f57d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d0004800 0x7f57d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 -- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57d00039b0 msgr2=0x7f57d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57d00039b0 0x7f57d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.552+0000 7f57e547b640 1 -- 172.21.15.106:0/2450866951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d0003520 con 0x7f57d00051c0 2024-04-13T03:10:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.553+0000 7f57e547b640 1 -- 172.21.15.106:0/2450866951 reap_dead start 2024-04-13T03:10:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.553+0000 7f57e547b640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 0x7f57d000b9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57d8001e50 tx=0x7f57d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac180c9c0e0728 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.553+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8001a70 con 0x7f57d00051c0 2024-04-13T03:10:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.553+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8003b90 con 0x7f57d00051c0 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.553+0000 7f57df7fe640 1 -- 172.21.15.106:0/2450866951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80050d0 con 0x7f57d00051c0 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 -- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 msgr2=0x7f57d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 0x7f57d000b9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57d8001e50 tx=0x7f57d0005830 comp rx=0 tx=0).stop 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 -- 172.21.15.106:0/2450866951 shutdown_connections 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 --2- 172.21.15.106:0/2450866951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57d00051c0 0x7f57d000b9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 -- 172.21.15.106:0/2450866951 >> 172.21.15.106:0/2450866951 conn(0x7f57e0071870 msgr2=0x7f57e0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 -- 172.21.15.106:0/2450866951 shutdown_connections 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.554+0000 7f57e6ede640 1 -- 172.21.15.106:0/2450866951 wait complete. 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.555+0000 7f57e6ede640 1 Processor -- start 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.555+0000 7f57e6ede640 1 -- start start 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d00051c0 0x7f57e0146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e0156a70 0x7f57e0149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00d1520 con 0x7f57e00aa580 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00d1ef0 con 0x7f57e0156a70 2024-04-13T03:10:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e6ede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57e00d16a0 con 0x7f57d00051c0 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46454/0 (socket says 172.21.15.106:46454) 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 -- 172.21.15.106:0/1409400122 learned_addr learned my addr 172.21.15.106:0/1409400122 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e4c7a640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d00051c0 0x7f57e0146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57e547b640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e0156a70 0x7f57e0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 -- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d00051c0 msgr2=0x7f57e0146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d00051c0 0x7f57e0146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 -- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e0156a70 msgr2=0x7f57e0149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e0156a70 0x7f57e0149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.556+0000 7f57dffff640 1 -- 172.21.15.106:0/1409400122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e0149ff0 con 0x7f57e00aa580 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57dffff640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0118e0 tx=0x7f57cc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00fa30 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc00fbd0 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e00a9aa0 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc014440 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.557+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e00aaa00 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.558+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e00ae480 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.558+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57cc012040 con 0x7f57e00aa580 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.559+0000 7f57dd7fa640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.559+0000 7f57e4c7a640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.559+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f57cc08c4b0 con 0x7f57e00aa580 2024-04-13T03:10:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.559+0000 7f57e4c7a640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 0x7f57c406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57d40015d0 tx=0x7f57d40078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.563+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57cc019030 con 0x7f57e00aa580 2024-04-13T03:10:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.738+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57e0068a10 con 0x7f57e00aa580 2024-04-13T03:10:17.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.738+0000 7f57dd7fa640 1 -- 172.21.15.106:0/1409400122 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f57cc059e90 con 0x7f57e00aa580 2024-04-13T03:10:17.739 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 msgr2=0x7f57c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 0x7f57c406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57d40015d0 tx=0x7f57d40078f0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 msgr2=0x7f57e0156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0118e0 tx=0x7f57cc011ce0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 shutdown_connections 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e0156a70 0x7f57e0149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e00aa580 0x7f57e0156530 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 --2- 172.21.15.106:0/1409400122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57d00051c0 0x7f57e0146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 >> 172.21.15.106:0/1409400122 conn(0x7f57e0071870 msgr2=0x7f57e00b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 shutdown_connections 2024-04-13T03:10:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.741+0000 7f57e6ede640 1 -- 172.21.15.106:0/1409400122 wait complete. 2024-04-13T03:10:17.753 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:17.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.855+0000 7f2024e2d640 1 Processor -- start 2024-04-13T03:10:17.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.855+0000 7f2024e2d640 1 -- start start 2024-04-13T03:10:17.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f2024e2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f202013f510 0x7f202015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f2024e2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f202013f8f0 0x7f20200704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f2024e2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2020160890 con 0x7f2020160400 2024-04-13T03:10:17.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f2024e2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2020160b70 con 0x7f202013f8f0 2024-04-13T03:10:17.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f2024e2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2020160e50 con 0x7f202013f510 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f202013f8f0 0x7f20200704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f202013f8f0 0x7f20200704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46464/0 (socket says 172.21.15.106:46464) 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 -- 172.21.15.106:0/656452313 learned_addr learned my addr 172.21.15.106:0/656452313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201e59c640 1 --2- 172.21.15.106:0/656452313 >> v2:172.21.15.114:3300/0 conn(0x7f202013f510 0x7f202015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 -- 172.21.15.106:0/656452313 >> v2:172.21.15.114:3300/0 conn(0x7f202013f510 msgr2=0x7f202015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 --2- 172.21.15.106:0/656452313 >> v2:172.21.15.114:3300/0 conn(0x7f202013f510 0x7f202015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 -- 172.21.15.106:0/656452313 >> v1:172.21.15.114:6789/0 conn(0x7f2020160400 legacy=0x7f202006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:17.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.856+0000 7f201ed9d640 1 -- 172.21.15.106:0/656452313 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020161130 con 0x7f202013f8f0 2024-04-13T03:10:17.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f201ed9d640 1 --2- 172.21.15.106:0/656452313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f20200704f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20200d2190 tx=0x7f201802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d1a4b695e35849 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f201dd9b640 1 -- 172.21.15.106:0/656452313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201802ca80 con 0x7f202013f8f0 2024-04-13T03:10:17.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f201dd9b640 1 -- 172.21.15.106:0/656452313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201802cc20 con 0x7f202013f8f0 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f201dd9b640 1 -- 172.21.15.106:0/656452313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180343f0 con 0x7f202013f8f0 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 -- 172.21.15.106:0/656452313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 msgr2=0x7f20200704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 --2- 172.21.15.106:0/656452313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f20200704f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20200d2190 tx=0x7f201802bee0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 -- 172.21.15.106:0/656452313 shutdown_connections 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 --2- 172.21.15.106:0/656452313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f20200704f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 --2- 172.21.15.106:0/656452313 >> v2:172.21.15.114:3300/0 conn(0x7f202013f510 0x7f202015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.857+0000 7f2024e2d640 1 -- 172.21.15.106:0/656452313 >> 172.21.15.106:0/656452313 conn(0x7f20200d13f0 msgr2=0x7f20200d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.858+0000 7f2024e2d640 1 -- 172.21.15.106:0/656452313 shutdown_connections 2024-04-13T03:10:17.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.858+0000 7f2024e2d640 1 -- 172.21.15.106:0/656452313 wait complete. 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.859+0000 7f2024e2d640 1 Processor -- start 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.859+0000 7f2024e2d640 1 -- start start 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 0x7f202013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 0x7f202015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2020070bb0 con 0x7f202013f8f0 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2020070a30 con 0x7f202013fcd0 2024-04-13T03:10:17.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2020070d30 con 0x7f202013f510 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46476/0 (socket says 172.21.15.106:46476) 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 -- 172.21.15.106:0/1389788306 learned_addr learned my addr 172.21.15.106:0/1389788306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f201ed9d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 0x7f202015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f201e59c640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 0x7f202013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 -- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 msgr2=0x7f202013b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 0x7f202013b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 -- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 msgr2=0x7f202015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 0x7f202015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 -- 172.21.15.106:0/1389788306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020140160 con 0x7f202013f8f0 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f201ed9d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 0x7f202015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f201e59c640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 0x7f202013b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2015d9b640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f200800c430 tx=0x7f200800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008011790 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2008017070 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202013b780 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008011bb0 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.860+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202013bd40 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.861+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202006d470 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.862+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f200801c460 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.862+0000 7f20177fe640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.862+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f200808bd60 con 0x7f202013f8f0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.862+0000 7f201e59c640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:17.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.863+0000 7f201e59c640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 0x7f200006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2010006fd0 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:17.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:17.867+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f202006d470 con 0x7f202013f8f0 2024-04-13T03:10:18.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.041+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2020069120 con 0x7f202013f8f0 2024-04-13T03:10:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.042+0000 7f20177fe640 1 -- 172.21.15.106:0/1389788306 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2008059e20 con 0x7f202013f8f0 2024-04-13T03:10:18.043 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 msgr2=0x7f200006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 0x7f200006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2010006fd0 tx=0x7f2010008040 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 msgr2=0x7f2020140600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f200800c430 tx=0x7f200800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 shutdown_connections 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202013fcd0 0x7f202015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f202013f8f0 0x7f2020140600 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.043+0000 7f2024e2d640 1 --2- 172.21.15.106:0/1389788306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f202013f510 0x7f202013b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.044+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 >> 172.21.15.106:0/1389788306 conn(0x7f20200d13f0 msgr2=0x7f20200b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:18.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.044+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 shutdown_connections 2024-04-13T03:10:18.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.044+0000 7f2024e2d640 1 -- 172.21.15.106:0/1389788306 wait complete. 2024-04-13T03:10:18.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:18.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.158+0000 7fa947a36640 1 Processor -- start 2024-04-13T03:10:18.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.158+0000 7fa947a36640 1 -- start start 2024-04-13T03:10:18.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa947a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa9400b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa947a36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94006e280 con 0x7fa94006ed70 2024-04-13T03:10:18.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa947a36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94006e560 con 0x7fa9400d09f0 2024-04-13T03:10:18.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa947a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400cccf0 con 0x7fa94015e930 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa944fd1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa94006ed70 0x7fa9400d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37480/0 (socket says 172.21.15.106:37480) 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa944fd1640 1 -- 172.21.15.106:0/1917469556 learned_addr learned my addr 172.21.15.106:0/1917469556 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa9457d2640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa9400b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743281192 0 0) 0x7fa94006e280 con 0x7fa94006ed70 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa92c003520 con 0x7fa94006ed70 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 308567335 0 0) 0x7fa92c003520 con 0x7fa94006ed70 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 msgr2=0x7fa9400b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa9400b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 >> v1:172.21.15.114:6789/0 conn(0x7fa9400d09f0 legacy=0x7fa9400cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400ccfd0 con 0x7fa94006ed70 2024-04-13T03:10:18.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa930001670 con 0x7fa94006ed70 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 >> v1:172.21.15.106:6789/0 conn(0x7fa94006ed70 legacy=0x7fa9400d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa92c0039e0 0x7fa92c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c004390 0x7fa92c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa92c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400cccf0 con 0x7fa92c0039e0 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c00b9c0 con 0x7fa92c004390 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.159+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c00bbc0 con 0x7fa94015e930 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa92c0039e0 0x7fa92c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa944fd1640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c004390 0x7fa92c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa945fd3640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa92c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 -- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 msgr2=0x7fa92c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa94015e930 0x7fa92c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 -- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c004390 msgr2=0x7fa92c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:18.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c004390 0x7fa92c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 -- 172.21.15.106:0/1917469556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c003520 con 0x7fa92c0039e0 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa944fd1640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c004390 0x7fa92c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.160+0000 7fa9457d2640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa92c0039e0 0x7fa92c003dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e93ab00474b6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c039070 con 0x7fa92c0039e0 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c02c6b0 con 0x7fa92c0039e0 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa937fff640 1 -- 172.21.15.106:0/1917469556 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c009690 con 0x7fa92c0039e0 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 -- 172.21.15.106:0/1917469556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa92c0039e0 msgr2=0x7fa92c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 --2- 172.21.15.106:0/1917469556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa92c0039e0 0x7fa92c003dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa92c005830 comp rx=0 tx=0).stop 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa945fd3640 1 -- 172.21.15.106:0/1917469556 reap_dead start 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 -- 172.21.15.106:0/1917469556 shutdown_connections 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 -- 172.21.15.106:0/1917469556 >> 172.21.15.106:0/1917469556 conn(0x7fa9400cef80 msgr2=0x7fa9400cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 -- 172.21.15.106:0/1917469556 shutdown_connections 2024-04-13T03:10:18.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.161+0000 7fa947a36640 1 -- 172.21.15.106:0/1917469556 wait complete. 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.162+0000 7fa947a36640 1 Processor -- start 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 -- start start 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c0039e0 0x7fa940151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 0x7fa9400c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400ce830 con 0x7fa9400c7b50 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa94006f1d0 con 0x7fa92c0039e0 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa947a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400ce9b0 con 0x7fa9400c5fa0 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa944fd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa944fd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46500/0 (socket says 172.21.15.106:46500) 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.163+0000 7fa944fd1640 1 -- 172.21.15.106:0/669196912 learned_addr learned my addr 172.21.15.106:0/669196912 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa945fd3640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 0x7fa9400c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa9457d2640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c0039e0 0x7fa940151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 -- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 msgr2=0x7fa9400c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 0x7fa9400c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 -- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c0039e0 msgr2=0x7fa940151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c0039e0 0x7fa940151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 -- 172.21.15.106:0/669196912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400c9670 con 0x7fa9400c7b50 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa945fd3640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 0x7fa9400c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:18.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa944fd1640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e7b0 tx=0x7fa93000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93000c870 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930018070 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9400c6d20 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93000cc10 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.164+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9400c98e0 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.165+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa940068a10 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.165+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa93001d460 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.166+0000 7fa935ffb640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 0x7fa92406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.166+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa93008bd10 con 0x7fa9400c7b50 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.166+0000 7fa9457d2640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 0x7fa92406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.166+0000 7fa9457d2640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 0x7fa92406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa93c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.170+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9300596f0 con 0x7fa9400c7b50 2024-04-13T03:10:18.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.346+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa9400cfce0 con 0x7fa9400c7b50 2024-04-13T03:10:18.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.347+0000 7fa935ffb640 1 -- 172.21.15.106:0/669196912 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa93005d9f0 con 0x7fa9400c7b50 2024-04-13T03:10:18.348 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:18.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 msgr2=0x7fa92406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 0x7fa92406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa93c030000 comp rx=0 tx=0).stop 2024-04-13T03:10:18.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 msgr2=0x7fa9400c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e7b0 tx=0x7fa93000ec70 comp rx=0 tx=0).stop 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 shutdown_connections 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa92406c920 0x7fa92406ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9400c5fa0 0x7fa9400c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9400c7b50 0x7fa9400c7fd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 --2- 172.21.15.106:0/669196912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa92c0039e0 0x7fa940151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 >> 172.21.15.106:0/669196912 conn(0x7fa9400cef80 msgr2=0x7fa940070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 shutdown_connections 2024-04-13T03:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.349+0000 7fa947a36640 1 -- 172.21.15.106:0/669196912 wait complete. 2024-04-13T03:10:18.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:10:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.461+0000 7f268e5e9640 1 Processor -- start 2024-04-13T03:10:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.461+0000 7f268e5e9640 1 -- start start 2024-04-13T03:10:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f26880cfea0 0x7f268815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880704a0 0x7f26880d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26880d1a60 0x7f2688160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f268815e100 con 0x7f26880d1a60 2024-04-13T03:10:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f268815ddd0 con 0x7f26880cfea0 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268e5e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f268815d4d0 con 0x7f26880704a0 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26880d1a60 0x7f2688160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26880d1a60 0x7f2688160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46512/0 (socket says 172.21.15.106:46512) 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 -- 172.21.15.106:0/1465250125 learned_addr learned my addr 172.21.15.106:0/1465250125 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f26877fe640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880704a0 0x7f26880d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f2687fff640 1 --2- 172.21.15.106:0/1465250125 >> v2:172.21.15.114:3300/0 conn(0x7f26880cfea0 0x7f268815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 -- 172.21.15.106:0/1465250125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880704a0 msgr2=0x7f26880d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880704a0 0x7f26880d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 -- 172.21.15.106:0/1465250125 >> v2:172.21.15.114:3300/0 conn(0x7f26880cfea0 msgr2=0x7f268815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 --2- 172.21.15.106:0/1465250125 >> v2:172.21.15.114:3300/0 conn(0x7f26880cfea0 0x7f268815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.462+0000 7f268cb86640 1 -- 172.21.15.106:0/1465250125 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268815d7b0 con 0x7f26880d1a60 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268cb86640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f2688160200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f268804fed0 tx=0x7f268002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd357bebb00e917 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f2686ffd640 1 -- 172.21.15.106:0/1465250125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268002ca70 con 0x7f26880d1a60 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f2686ffd640 1 -- 172.21.15.106:0/1465250125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268002cc10 con 0x7f26880d1a60 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f2686ffd640 1 -- 172.21.15.106:0/1465250125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26800343f0 con 0x7f26880d1a60 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 -- 172.21.15.106:0/1465250125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 msgr2=0x7f2688160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f2688160200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f268804fed0 tx=0x7f268002bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 -- 172.21.15.106:0/1465250125 shutdown_connections 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f2688160200 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 --2- 172.21.15.106:0/1465250125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880704a0 0x7f26880d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 --2- 172.21.15.106:0/1465250125 >> v2:172.21.15.114:3300/0 conn(0x7f26880cfea0 0x7f268815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.463+0000 7f268e5e9640 1 -- 172.21.15.106:0/1465250125 >> 172.21.15.106:0/1465250125 conn(0x7f2688071870 msgr2=0x7f2688071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:18.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.464+0000 7f268e5e9640 1 -- 172.21.15.106:0/1465250125 shutdown_connections 2024-04-13T03:10:18.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.464+0000 7f268e5e9640 1 -- 172.21.15.106:0/1465250125 wait complete. 2024-04-13T03:10:18.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.465+0000 7f268e5e9640 1 Processor -- start 2024-04-13T03:10:18.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.465+0000 7f268e5e9640 1 -- start start 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26880704a0 0x7f2688156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880cfea0 0x7f26881570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f268815d150 con 0x7f26880d1a60 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26880d1ef0 con 0x7f26880704a0 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2688160740 con 0x7f26880cfea0 2024-04-13T03:10:18.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46518/0 (socket says 172.21.15.106:46518) 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 -- 172.21.15.106:0/2466494832 learned_addr learned my addr 172.21.15.106:0/2466494832 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f2687fff640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26880704a0 0x7f2688156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f26877fe640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880cfea0 0x7f26881570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 -- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880cfea0 msgr2=0x7f26881570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880cfea0 0x7f26881570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 -- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26880704a0 msgr2=0x7f2688156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26880704a0 0x7f2688156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 -- 172.21.15.106:0/2466494832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2688157880 con 0x7f26880d1a60 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268cb86640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2680009f40 tx=0x7f2680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680044020 con 0x7f26880d1a60 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2680040670 con 0x7f26880d1a60 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2688157b60 con 0x7f26880d1a60 2024-04-13T03:10:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680009510 con 0x7f26880d1a60 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.466+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2688158120 con 0x7f26880d1a60 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.468+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2688068a10 con 0x7f26880d1a60 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.468+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2680047020 con 0x7f26880d1a60 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.468+0000 7f2684ff9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 0x7f267406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.468+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f26800ae940 con 0x7f26880d1a60 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.468+0000 7f2687fff640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 0x7f267406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:18.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.469+0000 7f2687fff640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 0x7f267406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f267c009a50 tx=0x7f267c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:18.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.473+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f268004e030 con 0x7f26880d1a60 2024-04-13T03:10:18.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.661+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2688072360 con 0x7f26880d1a60 2024-04-13T03:10:18.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.662+0000 7f2684ff9640 1 -- 172.21.15.106:0/2466494832 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f268007ca90 con 0x7f26880d1a60 2024-04-13T03:10:18.663 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 msgr2=0x7f267406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 0x7f267406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f267c009a50 tx=0x7f267c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 msgr2=0x7f26881513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2680009f40 tx=0x7f2680004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 shutdown_connections 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f267406c8f0 0x7f267406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26880cfea0 0x7f26881570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26880d1a60 0x7f26881513d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 --2- 172.21.15.106:0/2466494832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26880704a0 0x7f2688156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:18.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.663+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 >> 172.21.15.106:0/2466494832 conn(0x7f2688071870 msgr2=0x7f268815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:18.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.664+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 shutdown_connections 2024-04-13T03:10:18.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:18.664+0000 7f268e5e9640 1 -- 172.21.15.106:0/2466494832 wait complete. 2024-04-13T03:10:18.675 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-13T03:10:18.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:18.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-13T03:10:18.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:10:18.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:18.709 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-13T03:10:18.786 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:18.787 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:18.802 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:18.802 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:18.857 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:18.857 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:18.912 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:19.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.050+0000 7f7993fb4640 1 Processor -- start 2024-04-13T03:10:19.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.050+0000 7f7993fb4640 1 -- start start 2024-04-13T03:10:19.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7993fb4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f798c070090 0x7f798c160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7993fb4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f798c0cf5a0 0x7f798c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7993fb4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f798c070470 con 0x7f798c0b0390 2024-04-13T03:10:19.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7993fb4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f798c0ceb50 con 0x7f798c0cf5a0 2024-04-13T03:10:19.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7993fb4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f798c0cf110 con 0x7f798c070090 2024-04-13T03:10:19.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f798c0cf5a0 0x7f798c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f798c0cf5a0 0x7f798c0cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46528/0 (socket says 172.21.15.106:46528) 2024-04-13T03:10:19.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 -- 172.21.15.106:0/2487900597 learned_addr learned my addr 172.21.15.106:0/2487900597 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 -- 172.21.15.106:0/2487900597 >> v2:172.21.15.114:3300/0 conn(0x7f798c070090 msgr2=0x7f798c160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 --2- 172.21.15.106:0/2487900597 >> v2:172.21.15.114:3300/0 conn(0x7f798c070090 0x7f798c160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 -- 172.21.15.106:0/2487900597 >> v1:172.21.15.114:6789/0 conn(0x7f798c0b0390 legacy=0x7f798c160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.051+0000 7f7992551640 1 -- 172.21.15.106:0/2487900597 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c15da00 con 0x7f798c0cf5a0 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.052+0000 7f7992551640 1 --2- 172.21.15.106:0/2487900597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0cf5a0 0x7f798c0cfa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f798c06e020 tx=0x7f798402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef46332e16816bc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.052+0000 7f7990d4e640 1 -- 172.21.15.106:0/2487900597 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798402c9f0 con 0x7f798c0cf5a0 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.052+0000 7f7990d4e640 1 -- 172.21.15.106:0/2487900597 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798402cb90 con 0x7f798c0cf5a0 2024-04-13T03:10:19.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.052+0000 7f7990d4e640 1 -- 172.21.15.106:0/2487900597 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79840343f0 con 0x7f798c0cf5a0 2024-04-13T03:10:19.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 -- 172.21.15.106:0/2487900597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0cf5a0 msgr2=0x7f798c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 --2- 172.21.15.106:0/2487900597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0cf5a0 0x7f798c0cfa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f798c06e020 tx=0x7f798402be50 comp rx=0 tx=0).stop 2024-04-13T03:10:19.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 -- 172.21.15.106:0/2487900597 shutdown_connections 2024-04-13T03:10:19.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 --2- 172.21.15.106:0/2487900597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0cf5a0 0x7f798c0cfa30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 --2- 172.21.15.106:0/2487900597 >> v2:172.21.15.114:3300/0 conn(0x7f798c070090 0x7f798c160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 -- 172.21.15.106:0/2487900597 >> 172.21.15.106:0/2487900597 conn(0x7f798c0cd6d0 msgr2=0x7f798c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 -- 172.21.15.106:0/2487900597 shutdown_connections 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.053+0000 7f7993fb4640 1 -- 172.21.15.106:0/2487900597 wait complete. 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.054+0000 7f7993fb4640 1 Processor -- start 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.054+0000 7f7993fb4640 1 -- start start 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f798c070090 0x7f798c156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f798c0cf5a0 0x7f798c1e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f798c06e560 con 0x7f798c0b0390 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f798c0ceda0 con 0x7f798c070090 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f798c0cd220 con 0x7f798c0cf5a0 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46542/0 (socket says 172.21.15.106:46542) 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 -- 172.21.15.106:0/1399662380 learned_addr learned my addr 172.21.15.106:0/1399662380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 -- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f798c0cf5a0 msgr2=0x7f798c1e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f798c0cf5a0 0x7f798c1e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 -- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f798c070090 msgr2=0x7f798c156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f798c070090 0x7f798c156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 -- 172.21.15.106:0/1399662380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c1556f0 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f799154f640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c410 tx=0x7f797c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c017070 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f797c0035f0 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c1559d0 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c003a20 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.055+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c155f90 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.056+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c075180 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.057+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f797c01a070 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.057+0000 7f79827fc640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.057+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f797c08b340 con 0x7f798c0b0390 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.057+0000 7f7991d50640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.058+0000 7f7991d50640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7988009670 tx=0x7f7988006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.062+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f798c075180 con 0x7f798c0b0390 2024-04-13T03:10:19.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.240+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f798c068a10 con 0x7f798c0b0390 2024-04-13T03:10:19.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.241+0000 7f79827fc640 1 -- 172.21.15.106:0/1399662380 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f797c059400 con 0x7f798c0b0390 2024-04-13T03:10:19.243 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:19.243 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 msgr2=0x7f797006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7988009670 tx=0x7f7988006bf0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 msgr2=0x7f798c1f2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c410 tx=0x7f797c00c8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 shutdown_connections 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f798c0cf5a0 0x7f798c1e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f798c0b0390 0x7f798c1f2190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 --2- 172.21.15.106:0/1399662380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f798c070090 0x7f798c156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 >> 172.21.15.106:0/1399662380 conn(0x7f798c0cd6d0 msgr2=0x7f798c0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.243+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 shutdown_connections 2024-04-13T03:10:19.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.244+0000 7f7993fb4640 1 -- 172.21.15.106:0/1399662380 wait complete. 2024-04-13T03:10:19.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:19.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:19.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.358+0000 7fdcce21c640 1 Processor -- start 2024-04-13T03:10:19.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.358+0000 7fdcce21c640 1 -- start start 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80b1c30 0x7fdcc815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdcc80b54a0 0x7fdcc80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc80709e0 0x7fdcc8162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc80b5cf0 con 0x7fdcc80709e0 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc80b5fd0 con 0x7fdcc80b54a0 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcce21c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc80b62b0 con 0x7fdcc80b1c30 2024-04-13T03:10:19.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc80709e0 0x7fdcc8162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc80709e0 0x7fdcc8162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46556/0 (socket says 172.21.15.106:46556) 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 -- 172.21.15.106:0/2078335252 learned_addr learned my addr 172.21.15.106:0/2078335252 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc77fe640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80b1c30 0x7fdcc815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 -- 172.21.15.106:0/2078335252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80b1c30 msgr2=0x7fdcc815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80b1c30 0x7fdcc815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 -- 172.21.15.106:0/2078335252 >> v2:172.21.15.114:3300/0 conn(0x7fdcc80b54a0 msgr2=0x7fdcc80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 --2- 172.21.15.106:0/2078335252 >> v2:172.21.15.114:3300/0 conn(0x7fdcc80b54a0 0x7fdcc80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.359+0000 7fdcc7fff640 1 -- 172.21.15.106:0/2078335252 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc80b6590 con 0x7fdcc80709e0 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.360+0000 7fdcc7fff640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80709e0 0x7fdcc8162430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009e50 tx=0x7fdcc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cc86fd5635d783 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.360+0000 7fdcc67fc640 1 -- 172.21.15.106:0/2078335252 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc002ca70 con 0x7fdcc80709e0 2024-04-13T03:10:19.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.360+0000 7fdcc67fc640 1 -- 172.21.15.106:0/2078335252 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc002cc10 con 0x7fdcc80709e0 2024-04-13T03:10:19.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.360+0000 7fdcc67fc640 1 -- 172.21.15.106:0/2078335252 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00343f0 con 0x7fdcc80709e0 2024-04-13T03:10:19.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 -- 172.21.15.106:0/2078335252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80709e0 msgr2=0x7fdcc8162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80709e0 0x7fdcc8162430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009e50 tx=0x7fdcc002bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 -- 172.21.15.106:0/2078335252 shutdown_connections 2024-04-13T03:10:19.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80709e0 0x7fdcc8162430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 --2- 172.21.15.106:0/2078335252 >> v2:172.21.15.114:3300/0 conn(0x7fdcc80b54a0 0x7fdcc80704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 --2- 172.21.15.106:0/2078335252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80b1c30 0x7fdcc815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 -- 172.21.15.106:0/2078335252 >> 172.21.15.106:0/2078335252 conn(0x7fdcc8071870 msgr2=0x7fdcc8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 -- 172.21.15.106:0/2078335252 shutdown_connections 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.361+0000 7fdcce21c640 1 -- 172.21.15.106:0/2078335252 wait complete. 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.362+0000 7fdcce21c640 1 Processor -- start 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.362+0000 7fdcce21c640 1 -- start start 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80709e0 0x7fdcc8156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 0x7fdcc815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8162af0 con 0x7fdcc80b1c30 2024-04-13T03:10:19.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8162970 con 0x7fdcc80b54a0 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc8162c70 con 0x7fdcc80709e0 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46566/0 (socket says 172.21.15.106:46566) 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 -- 172.21.15.106:0/393255041 learned_addr learned my addr 172.21.15.106:0/393255041 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc77fe640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80709e0 0x7fdcc8156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc7fff640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 0x7fdcc815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 -- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80709e0 msgr2=0x7fdcc8156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80709e0 0x7fdcc8156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 -- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 msgr2=0x7fdcc815b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 0x7fdcc815b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 -- 172.21.15.106:0/393255041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc80cbf70 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc7fff640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 0x7fdcc815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcc6ffd640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b410 tx=0x7fdcbc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc015020 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc011850 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc011c40 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.363+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81e99b0 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.364+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81e9f70 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.365+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc8068a10 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.365+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcbc018020 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.365+0000 7fdcaffff640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 0x7fdcb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.365+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdcbc08bde0 con 0x7fdcc80b1c30 2024-04-13T03:10:19.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.365+0000 7fdcc77fe640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 0x7fdcb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.366+0000 7fdcc77fe640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 0x7fdcb406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80d2470 tx=0x7fdcb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.369+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcbc01f020 con 0x7fdcc80b1c30 2024-04-13T03:10:19.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.543+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcc81ea310 con 0x7fdcc80b1c30 2024-04-13T03:10:19.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.544+0000 7fdcaffff640 1 -- 172.21.15.106:0/393255041 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fdcbc0172c0 con 0x7fdcc80b1c30 2024-04-13T03:10:19.545 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.545+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 msgr2=0x7fdcb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.545+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 0x7fdcb406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80d2470 tx=0x7fdcb8008040 comp rx=0 tx=0).stop 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.545+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 msgr2=0x7fdcc815afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.545+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b410 tx=0x7fdcbc00b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 shutdown_connections 2024-04-13T03:10:19.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdcb406c8f0 0x7fdcb406edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcc80b54a0 0x7fdcc815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc80b1c30 0x7fdcc815afa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 --2- 172.21.15.106:0/393255041 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcc80709e0 0x7fdcc8156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 >> 172.21.15.106:0/393255041 conn(0x7fdcc8071870 msgr2=0x7fdcc80b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 shutdown_connections 2024-04-13T03:10:19.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.546+0000 7fdcce21c640 1 -- 172.21.15.106:0/393255041 wait complete. 2024-04-13T03:10:19.558 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:19.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 Processor -- start 2024-04-13T03:10:19.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 -- start start 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 0x7fc3f815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc3f80d3220 0x7fc3f80d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f815f380 con 0x7fc3f80ba170 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f80d2d90 con 0x7fc3f80d3220 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.660+0000 7fc3fef15640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f815f600 con 0x7fc3f80cf480 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc3f80d3220 0x7fc3f80d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc3f80d3220 0x7fc3f80d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46568/0 (socket says 172.21.15.106:46568) 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2720737892 learned_addr learned my addr 172.21.15.106:0/2720737892 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fccb1640 1 --2- 172.21.15.106:0/2720737892 >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 0x7fc3f815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2720737892 >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 msgr2=0x7fc3f815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 --2- 172.21.15.106:0/2720737892 >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 0x7fc3f815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2720737892 >> v1:172.21.15.114:6789/0 conn(0x7fc3f80ba170 legacy=0x7fc3f80cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2720737892 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f815f8e0 con 0x7fc3f80d3220 2024-04-13T03:10:19.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.661+0000 7fc3fccb1640 1 --2- 172.21.15.106:0/2720737892 >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 0x7fc3f815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:19.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fd4b2640 1 --2- 172.21.15.106:0/2720737892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80d36d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806e580 tx=0x7fc3f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8b5bf9e45d5b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3f77fe640 1 -- 172.21.15.106:0/2720737892 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f002ca80 con 0x7fc3f80d3220 2024-04-13T03:10:19.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3f77fe640 1 -- 172.21.15.106:0/2720737892 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f002cc20 con 0x7fc3f80d3220 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3f77fe640 1 -- 172.21.15.106:0/2720737892 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00343f0 con 0x7fc3f80d3220 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 -- 172.21.15.106:0/2720737892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 msgr2=0x7fc3f80d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2720737892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80d36d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806e580 tx=0x7fc3f002bee0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 -- 172.21.15.106:0/2720737892 shutdown_connections 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2720737892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80d36d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2720737892 >> v2:172.21.15.114:3300/0 conn(0x7fc3f80cf480 0x7fc3f815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 -- 172.21.15.106:0/2720737892 >> 172.21.15.106:0/2720737892 conn(0x7fc3f80d47d0 msgr2=0x7fc3f80d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 -- 172.21.15.106:0/2720737892 shutdown_connections 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.662+0000 7fc3fef15640 1 -- 172.21.15.106:0/2720737892 wait complete. 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.663+0000 7fc3fef15640 1 Processor -- start 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.663+0000 7fc3fef15640 1 -- start start 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3f80ba170 0x7fc3f8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc3f80cf480 0x7fc3f815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81592f0 con 0x7fc3f80d3220 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8159170 con 0x7fc3f80ba170 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fef15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f80d3c10 con 0x7fc3f80cf480 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fd4b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fd4b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46578/0 (socket says 172.21.15.106:46578) 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.664+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2584836444 learned_addr learned my addr 172.21.15.106:0/2584836444 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3f7fff640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc3f80cf480 0x7fc3f815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc3f80cf480 msgr2=0x7fc3f815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fccb1640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3f80ba170 0x7fc3f8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc3f80cf480 0x7fc3f815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3f80ba170 msgr2=0x7fc3f8152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3f80ba170 0x7fc3f8152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 -- 172.21.15.106:0/2584836444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f80cba40 con 0x7fc3f80d3220 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fd4b2640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009f40 tx=0x7fc3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0043020 con 0x7fc3f80d3220 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f003e450 con 0x7fc3f80d3220 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f003e820 con 0x7fc3f80d3220 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f80cbce0 con 0x7fc3f80d3220 2024-04-13T03:10:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.665+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f80cc2a0 con 0x7fc3f80d3220 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.666+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f8068a10 con 0x7fc3f80d3220 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.667+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3f0046020 con 0x7fc3f80d3220 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.667+0000 7fc3f57fa640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 0x7fc3dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.667+0000 7fc3fccb1640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 0x7fc3dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.667+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc3f00ae810 con 0x7fc3f80d3220 2024-04-13T03:10:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.667+0000 7fc3fccb1640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 0x7fc3dc06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec008850 tx=0x7fc3ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.671+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3f004d030 con 0x7fc3f80d3220 2024-04-13T03:10:19.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.845+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3f80d4cb0 con 0x7fc3f80d3220 2024-04-13T03:10:19.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.846+0000 7fc3f57fa640 1 -- 172.21.15.106:0/2584836444 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc3f815f380 con 0x7fc3f80d3220 2024-04-13T03:10:19.847 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:19.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 msgr2=0x7fc3dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 0x7fc3dc06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec008850 tx=0x7fc3ec008040 comp rx=0 tx=0).stop 2024-04-13T03:10:19.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 msgr2=0x7fc3f80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009f40 tx=0x7fc3f0004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:19.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 shutdown_connections 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc3dc06c8f0 0x7fc3dc06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc3f80cf480 0x7fc3f815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f80d3220 0x7fc3f80b8990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 --2- 172.21.15.106:0/2584836444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3f80ba170 0x7fc3f8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 >> 172.21.15.106:0/2584836444 conn(0x7fc3f80d47d0 msgr2=0x7fc3f8161850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 shutdown_connections 2024-04-13T03:10:19.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.848+0000 7fc3fef15640 1 -- 172.21.15.106:0/2584836444 wait complete. 2024-04-13T03:10:19.860 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:19.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.961+0000 7f341c636640 1 Processor -- start 2024-04-13T03:10:19.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.961+0000 7f341c636640 1 -- start start 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341c636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341c636640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341c636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34140b3740 con 0x7f34140b47e0 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341c636640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f34140b3a20 con 0x7f34140b4c20 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341c636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34140b3d00 con 0x7f3414145d20 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341abd3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341abd3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:33888/0 (socket says 172.21.15.106:33888) 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f3419bd1640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f34140b47e0 0x7f341415cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42896/0 (socket says 172.21.15.106:42896) 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341abd3640 1 -- 172.21.15.106:0/1342016378 learned_addr learned my addr 172.21.15.106:0/1342016378 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:52314/0 (socket says 172.21.15.106:52314) 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 -- 172.21.15.106:0/1342016378 >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 msgr2=0x7f34140cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 --2- 172.21.15.106:0/1342016378 >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 -- 172.21.15.106:0/1342016378 >> v1:172.21.15.114:6789/0 conn(0x7f34140b47e0 legacy=0x7f341415cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341a3d2640 1 -- 172.21.15.106:0/1342016378 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34140b3fe0 con 0x7f3414145d20 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.962+0000 7f341abd3640 1 --2- 172.21.15.106:0/1342016378 >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:19.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.963+0000 7f341a3d2640 1 --2- 172.21.15.106:0/1342016378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34100097a0 tx=0x7f341002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c94adfe3815042 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.963+0000 7f34193d0640 1 -- 172.21.15.106:0/1342016378 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410035020 con 0x7f3414145d20 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.963+0000 7f34193d0640 1 -- 172.21.15.106:0/1342016378 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3410031830 con 0x7f3414145d20 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.963+0000 7f34193d0640 1 -- 172.21.15.106:0/1342016378 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410031c00 con 0x7f3414145d20 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 -- 172.21.15.106:0/1342016378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 msgr2=0x7f341406cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 --2- 172.21.15.106:0/1342016378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34100097a0 tx=0x7f341002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 -- 172.21.15.106:0/1342016378 shutdown_connections 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 --2- 172.21.15.106:0/1342016378 >> v2:172.21.15.114:3300/0 conn(0x7f34140b4c20 0x7f34140cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 --2- 172.21.15.106:0/1342016378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3414145d20 0x7f341406cb10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 -- 172.21.15.106:0/1342016378 >> 172.21.15.106:0/1342016378 conn(0x7f34140b2600 msgr2=0x7f34140d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 -- 172.21.15.106:0/1342016378 shutdown_connections 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.964+0000 7f341c636640 1 -- 172.21.15.106:0/1342016378 wait complete. 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.965+0000 7f341c636640 1 Processor -- start 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.965+0000 7f341c636640 1 -- start start 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34140b4c20 0x7f34140aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3414145d20 0x7f34141f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3414146280 con 0x7f34140b47e0 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3414146100 con 0x7f3414145d20 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341c636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f341406d050 con 0x7f34140b4c20 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46588/0 (socket says 172.21.15.106:46588) 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 -- 172.21.15.106:0/859990421 learned_addr learned my addr 172.21.15.106:0/859990421 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f3419bd1640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34140b4c20 0x7f34140aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341abd3640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3414145d20 0x7f34141f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 -- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34140b4c20 msgr2=0x7f34140aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34140b4c20 0x7f34140aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 -- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3414145d20 msgr2=0x7f34141f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3414145d20 0x7f34141f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.966+0000 7f341a3d2640 1 -- 172.21.15.106:0/859990421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34140a9c30 con 0x7f34140b47e0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f341a3d2640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f341002b910 tx=0x7f34100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410040070 con 0x7f34140b47e0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341003a930 con 0x7f34140b47e0 2024-04-13T03:10:19.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34140a9f10 con 0x7f34140b47e0 2024-04-13T03:10:19.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410035040 con 0x7f34140b47e0 2024-04-13T03:10:19.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.967+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34140aa7b0 con 0x7f34140b47e0 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.968+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3414068a10 con 0x7f34140b47e0 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.968+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f341004b060 con 0x7f34140b47e0 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.969+0000 7f340affd640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 0x7f33f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.969+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f34100ad670 con 0x7f34140b47e0 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.969+0000 7f3419bd1640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 0x7f33f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:19.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.969+0000 7f3419bd1640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 0x7f33f806ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f34140b10f0 tx=0x7f3404007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:19.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:19.973+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3410031800 con 0x7f34140b47e0 2024-04-13T03:10:20.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.148+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34140b0d60 con 0x7f34140b47e0 2024-04-13T03:10:20.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.149+0000 7f340affd640 1 -- 172.21.15.106:0/859990421 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f341007afd0 con 0x7f34140b47e0 2024-04-13T03:10:20.150 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 msgr2=0x7f33f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 0x7f33f806ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f34140b10f0 tx=0x7f3404007e80 comp rx=0 tx=0).stop 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 msgr2=0x7f34140a93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f341002b910 tx=0x7f34100046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 shutdown_connections 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33f806c920 0x7f33f806ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34140b4c20 0x7f34140aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3414145d20 0x7f34141f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 --2- 172.21.15.106:0/859990421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34140b47e0 0x7f34140a93e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 >> 172.21.15.106:0/859990421 conn(0x7f34140b2600 msgr2=0x7f34140d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:20.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 shutdown_connections 2024-04-13T03:10:20.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.151+0000 7f341c636640 1 -- 172.21.15.106:0/859990421 wait complete. 2024-04-13T03:10:20.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:20.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.266+0000 7fd09e75e640 1 Processor -- start 2024-04-13T03:10:20.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.266+0000 7fd09e75e640 1 -- start start 2024-04-13T03:10:20.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd09e75e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd098070bb0 0x7fd0981697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd09e75e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0980b5cf0 con 0x7fd0980b1c30 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd09e75e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0980b5fd0 con 0x7fd09815ee40 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd09e75e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd0980b62b0 con 0x7fd098070bb0 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd097fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd0980b1c30 0x7fd0980b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37486/0 (socket says 172.21.15.106:37486) 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd097fff640 1 -- 172.21.15.106:0/3235960734 learned_addr learned my addr 172.21.15.106:0/3235960734 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100300566 0 0) 0x7fd0980b5cf0 con 0x7fd0980b1c30 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd080003520 con 0x7fd0980b1c30 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd09ccfb640 1 --2- 172.21.15.106:0/3235960734 >> v2:172.21.15.114:3300/0 conn(0x7fd098070bb0 0x7fd0981697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1356890303 0 0) 0x7fd080003520 con 0x7fd0980b1c30 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 >> v2:172.21.15.114:3300/0 conn(0x7fd098070bb0 msgr2=0x7fd0981697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 --2- 172.21.15.106:0/3235960734 >> v2:172.21.15.114:3300/0 conn(0x7fd098070bb0 0x7fd0981697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 >> v1:172.21.15.114:6789/0 conn(0x7fd09815ee40 legacy=0x7fd0980704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0980b6590 con 0x7fd0980b1c30 2024-04-13T03:10:20.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fd088001670 con 0x7fd0980b1c30 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 >> v1:172.21.15.106:6789/0 conn(0x7fd0980b1c30 legacy=0x7fd0980b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 0x7fd080003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd080004800 0x7fd080004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800051c0 0x7fd08000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980b62b0 con 0x7fd0800039b0 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd08000c0f0 con 0x7fd080004800 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.267+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd08000c2f0 con 0x7fd0800051c0 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 0x7fd080003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd0977fe640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd080004800 0x7fd080004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd09ccfb640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800051c0 0x7fd08000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 -- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800051c0 msgr2=0x7fd08000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800051c0 0x7fd08000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 -- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd080004800 msgr2=0x7fd080004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:20.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd080004800 0x7fd080004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 -- 172.21.15.106:0/3235960734 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080003520 con 0x7fd0800039b0 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd09ccfb640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800051c0 0x7fd08000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd09ccfb640 1 -- 172.21.15.106:0/3235960734 reap_dead start 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.268+0000 7fd097fff640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 0x7fd080003db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c210 tx=0x7fd080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0090b7ee381bf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088007d10 con 0x7fd0800039b0 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd088030070 con 0x7fd0800039b0 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd096ffd640 1 -- 172.21.15.106:0/3235960734 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088005d70 con 0x7fd0800039b0 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 -- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 msgr2=0x7fd080003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 0x7fd080003db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c210 tx=0x7fd080005830 comp rx=0 tx=0).stop 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 -- 172.21.15.106:0/3235960734 shutdown_connections 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3235960734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0800039b0 0x7fd080003db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 -- 172.21.15.106:0/3235960734 >> 172.21.15.106:0/3235960734 conn(0x7fd098071870 msgr2=0x7fd098071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:20.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 -- 172.21.15.106:0/3235960734 shutdown_connections 2024-04-13T03:10:20.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.269+0000 7fd09e75e640 1 -- 172.21.15.106:0/3235960734 wait complete. 2024-04-13T03:10:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.270+0000 7fd09e75e640 1 Processor -- start 2024-04-13T03:10:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 -- start start 2024-04-13T03:10:20.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 0x7fd0980d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd098150fd0 0x7fd098151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd098070830 con 0x7fd098157ba0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980706b0 con 0x7fd098150fd0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd09e75e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd0980709b0 con 0x7fd0800039b0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46614/0 (socket says 172.21.15.106:46614) 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd0977fe640 1 -- 172.21.15.106:0/3774696972 learned_addr learned my addr 172.21.15.106:0/3774696972 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.271+0000 7fd097fff640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 0x7fd0980d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd09ccfb640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd098150fd0 0x7fd098151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 -- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 msgr2=0x7fd0980d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 0x7fd0980d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 -- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd098150fd0 msgr2=0x7fd098151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd098150fd0 0x7fd098151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 -- 172.21.15.106:0/3774696972 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0981f7b90 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd097fff640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 0x7fd0980d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd0977fe640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00f840 tx=0x7fd08c00fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c018020 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08c00d730 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00db20 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0981f7e70 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.272+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981f8430 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.273+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09815e9c0 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.273+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd08c01b020 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.274+0000 7fd094ff9640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 0x7fd08406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.274+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd08c08bf80 con 0x7fd098157ba0 2024-04-13T03:10:20.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.274+0000 7fd097fff640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 0x7fd08406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.274+0000 7fd097fff640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 0x7fd08406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd088034000 tx=0x7fd088007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.278+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd09815e9c0 con 0x7fd098157ba0 2024-04-13T03:10:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.453+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd098068a10 con 0x7fd098157ba0 2024-04-13T03:10:20.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.453+0000 7fd094ff9640 1 -- 172.21.15.106:0/3774696972 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fd08c022030 con 0x7fd098157ba0 2024-04-13T03:10:20.454 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:20.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 msgr2=0x7fd08406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 0x7fd08406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd088034000 tx=0x7fd088007000 comp rx=0 tx=0).stop 2024-04-13T03:10:20.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 msgr2=0x7fd098158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00f840 tx=0x7fd08c00fd00 comp rx=0 tx=0).stop 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 shutdown_connections 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd08406c920 0x7fd08406ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd098150fd0 0x7fd098151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd098157ba0 0x7fd098158020 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 --2- 172.21.15.106:0/3774696972 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0800039b0 0x7fd0980d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 >> 172.21.15.106:0/3774696972 conn(0x7fd098071870 msgr2=0x7fd09815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 shutdown_connections 2024-04-13T03:10:20.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.455+0000 7fd09e75e640 1 -- 172.21.15.106:0/3774696972 wait complete. 2024-04-13T03:10:20.468 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:10:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.569+0000 7f4fb6436640 1 Processor -- start 2024-04-13T03:10:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.569+0000 7f4fb6436640 1 -- start start 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4fb6436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fb0145d20 0x7f4fb006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4fb6436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b47e0 0x7f4fb015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4fb6436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb00b3740 con 0x7f4fb00b4c20 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4fb6436640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00b3a20 con 0x7f4fb0145d20 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4fb6436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb00b3d00 con 0x7f4fb00b47e0 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fb0145d20 0x7f4fb006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fb0145d20 0x7f4fb006cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46630/0 (socket says 172.21.15.106:46630) 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 -- 172.21.15.106:0/3328621907 learned_addr learned my addr 172.21.15.106:0/3328621907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faf7fe640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b47e0 0x7f4fb015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 -- 172.21.15.106:0/3328621907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b47e0 msgr2=0x7f4fb015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b47e0 0x7f4fb015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 -- 172.21.15.106:0/3328621907 >> v1:172.21.15.114:6789/0 conn(0x7f4fb00b4c20 legacy=0x7f4fb00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.570+0000 7f4faffff640 1 -- 172.21.15.106:0/3328621907 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb00b3fe0 con 0x7f4fb0145d20 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.571+0000 7f4faffff640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb006cb10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009870 tx=0x7f4fa002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743a3da7e6648a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.571+0000 7f4faeffd640 1 -- 172.21.15.106:0/3328621907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0034070 con 0x7f4fb0145d20 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.571+0000 7f4faeffd640 1 -- 172.21.15.106:0/3328621907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa0030840 con 0x7f4fb0145d20 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.571+0000 7f4faeffd640 1 -- 172.21.15.106:0/3328621907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0030c10 con 0x7f4fb0145d20 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 -- 172.21.15.106:0/3328621907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 msgr2=0x7f4fb006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb006cb10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009870 tx=0x7f4fa002b980 comp rx=0 tx=0).stop 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 -- 172.21.15.106:0/3328621907 shutdown_connections 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b47e0 0x7f4fb015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 --2- 172.21.15.106:0/3328621907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb006cb10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 -- 172.21.15.106:0/3328621907 >> 172.21.15.106:0/3328621907 conn(0x7f4fb00b2600 msgr2=0x7f4fb00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 -- 172.21.15.106:0/3328621907 shutdown_connections 2024-04-13T03:10:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.572+0000 7f4fb6436640 1 -- 172.21.15.106:0/3328621907 wait complete. 2024-04-13T03:10:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.573+0000 7f4fb6436640 1 Processor -- start 2024-04-13T03:10:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.573+0000 7f4fb6436640 1 -- start start 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 0x7f4fb013bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b4c20 0x7f4fb01e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0146280 con 0x7f4fb0145d20 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0146100 con 0x7f4fb00b47e0 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb6436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb006d050 con 0x7f4fb00b4c20 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46632/0 (socket says 172.21.15.106:46632) 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 -- 172.21.15.106:0/413438674 learned_addr learned my addr 172.21.15.106:0/413438674 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4faf7fe640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b4c20 0x7f4fb01e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4faffff640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 0x7f4fb013bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 -- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b4c20 msgr2=0x7f4fb01e6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b4c20 0x7f4fb01e6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 -- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 msgr2=0x7f4fb013bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 0x7f4fb013bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 -- 172.21.15.106:0/413438674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb015fec0 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4faffff640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 0x7f4fb013bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4fb49d3640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800c710 tx=0x7f4fa800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8016020 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.574+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa8006d50 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.575+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8003bb0 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.575+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb015d8e0 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.575+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb015dea0 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.575+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0068a10 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.576+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fa800a850 con 0x7f4fb0145d20 2024-04-13T03:10:20.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.576+0000 7f4facff9640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:20.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.576+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4fa808bac0 con 0x7f4fb0145d20 2024-04-13T03:10:20.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.576+0000 7f4faffff640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:20.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.577+0000 7f4faffff640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0036000 tx=0x7f4fa003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:20.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.581+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fa8020020 con 0x7f4fb0145d20 2024-04-13T03:10:20.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.769+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4fb0144b10 con 0x7f4fb0145d20 2024-04-13T03:10:20.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.769+0000 7f4facff9640 1 -- 172.21.15.106:0/413438674 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f4fa8059b80 con 0x7f4fb0145d20 2024-04-13T03:10:20.770 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:10:20.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 msgr2=0x7f4f9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0036000 tx=0x7f4fa003b040 comp rx=0 tx=0).stop 2024-04-13T03:10:20.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 msgr2=0x7f4fb01e73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:20.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800c710 tx=0x7f4fa800cbd0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 shutdown_connections 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f9c06c8f0 0x7f4f9c06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fb00b4c20 0x7f4fb01e6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fb0145d20 0x7f4fb01e73e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 --2- 172.21.15.106:0/413438674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb00b47e0 0x7f4fb013bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 >> 172.21.15.106:0/413438674 conn(0x7f4fb00b2600 msgr2=0x7f4fb01439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 shutdown_connections 2024-04-13T03:10:20.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:20.771+0000 7f4fb6436640 1 -- 172.21.15.106:0/413438674 wait complete. 2024-04-13T03:10:20.783 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-13T03:10:20.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:20.783 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-13T03:10:20.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:10:20.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:20.811 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-13T03:10:20.866 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:20.866 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:20.885 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:20.885 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:20.943 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:10:20.943 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:21.001 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:21.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.102+0000 7fd095baa640 1 Processor -- start 2024-04-13T03:10:21.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.102+0000 7fd095baa640 1 -- start start 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd095baa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd0900caf30 0x7fd0901630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd095baa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0900cd700 con 0x7fd090144f10 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd095baa640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0900cd9e0 con 0x7fd0900d0bc0 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd095baa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd0900cd4d0 con 0x7fd0900caf30 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd090144f10 0x7fd0900ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37500/0 (socket says 172.21.15.106:37500) 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08f7fe640 1 -- 172.21.15.106:0/98724782 learned_addr learned my addr 172.21.15.106:0/98724782 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386311335 0 0) 0x7fd0900cd700 con 0x7fd090144f10 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd078003520 con 0x7fd090144f10 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08effd640 1 --2- 172.21.15.106:0/98724782 >> v2:172.21.15.114:3300/0 conn(0x7fd0900caf30 0x7fd0901630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4160083849 0 0) 0x7fd078003520 con 0x7fd090144f10 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 >> v2:172.21.15.114:3300/0 conn(0x7fd0900caf30 msgr2=0x7fd0901630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 --2- 172.21.15.106:0/98724782 >> v2:172.21.15.114:3300/0 conn(0x7fd0900caf30 0x7fd0901630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 >> v1:172.21.15.114:6789/0 conn(0x7fd0900d0bc0 legacy=0x7fd0900cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.103+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0900cdcc0 con 0x7fd090144f10 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fd080001670 con 0x7fd090144f10 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 >> v1:172.21.15.106:6789/0 conn(0x7fd090144f10 legacy=0x7fd0900ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd078004800 0x7fd078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0780051c0 0x7fd07800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd07800c040 con 0x7fd0780039b0 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd07800c1f0 con 0x7fd078004800 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd07800c3f0 con 0x7fd0780051c0 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd078003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08effd640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd078004800 0x7fd078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08ffff640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0780051c0 0x7fd07800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 -- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0780051c0 msgr2=0x7fd07800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0780051c0 0x7fd07800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 -- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd078004800 msgr2=0x7fd078004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd078004800 0x7fd078004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08f7fe640 1 -- 172.21.15.106:0/98724782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078003520 con 0x7fd0780039b0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.104+0000 7fd08ffff640 1 -- 172.21.15.106:0/98724782 reap_dead start 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd078003dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c240 tx=0x7fd078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd94a4c67d5f50a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080007cf0 con 0x7fd0780039b0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd080030070 con 0x7fd0780039b0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd08e7fc640 1 -- 172.21.15.106:0/98724782 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080005d30 con 0x7fd0780039b0 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 -- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 msgr2=0x7fd078003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd078003dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c240 tx=0x7fd078005830 comp rx=0 tx=0).stop 2024-04-13T03:10:21.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 -- 172.21.15.106:0/98724782 shutdown_connections 2024-04-13T03:10:21.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 --2- 172.21.15.106:0/98724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd078003dd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 -- 172.21.15.106:0/98724782 >> 172.21.15.106:0/98724782 conn(0x7fd09006f180 msgr2=0x7fd09006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 -- 172.21.15.106:0/98724782 shutdown_connections 2024-04-13T03:10:21.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.105+0000 7fd095baa640 1 -- 172.21.15.106:0/98724782 wait complete. 2024-04-13T03:10:21.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 Processor -- start 2024-04-13T03:10:21.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 -- start start 2024-04-13T03:10:21.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 0x7fd09013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 0x7fd0900b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd09006c150 con 0x7fd0780039b0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0900d0fd0 con 0x7fd09013c8a0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.107+0000 7fd095baa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd09006c2d0 con 0x7fd09013b330 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46650/0 (socket says 172.21.15.106:46650) 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 -- 172.21.15.106:0/1956994378 learned_addr learned my addr 172.21.15.106:0/1956994378 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08effd640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 0x7fd09013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08ffff640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 0x7fd0900b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 -- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 msgr2=0x7fd0900b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 0x7fd0900b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 -- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 msgr2=0x7fd09013adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 0x7fd09013adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 -- 172.21.15.106:0/1956994378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09015e940 con 0x7fd0780039b0 2024-04-13T03:10:21.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08ffff640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 0x7fd0900b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08effd640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 0x7fd09013adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd08f7fe640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c360 tx=0x7fd0900cd420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080001c30 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd080004bf0 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09013b800 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080007070 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.108+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09013bdc0 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.109+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0900ccab0 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.110+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd080048020 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.110+0000 7fd077fff640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 0x7fd07c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.110+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd0800ad190 con 0x7fd0780039b0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.110+0000 7fd08effd640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 0x7fd07c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.111+0000 7fd08effd640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 0x7fd07c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0900b0570 tx=0x7fd084008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.114+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0900ccab0 con 0x7fd0780039b0 2024-04-13T03:10:21.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.291+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0900cb720 con 0x7fd0780039b0 2024-04-13T03:10:21.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.292+0000 7fd077fff640 1 -- 172.21.15.106:0/1956994378 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fd09015e940 con 0x7fd0780039b0 2024-04-13T03:10:21.294 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:21.294 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 msgr2=0x7fd07c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 0x7fd07c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0900b0570 tx=0x7fd084008040 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 msgr2=0x7fd09015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c360 tx=0x7fd0900cd420 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 shutdown_connections 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd07c06c8f0 0x7fd07c06edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd09013b330 0x7fd0900b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.294+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd09013c8a0 0x7fd09013adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.295+0000 7fd095baa640 1 --2- 172.21.15.106:0/1956994378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0780039b0 0x7fd09015e400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.295+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 >> 172.21.15.106:0/1956994378 conn(0x7fd09006f180 msgr2=0x7fd0900cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.295+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 shutdown_connections 2024-04-13T03:10:21.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.295+0000 7fd095baa640 1 -- 172.21.15.106:0/1956994378 wait complete. 2024-04-13T03:10:21.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:21.307 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:21.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 Processor -- start 2024-04-13T03:10:21.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 -- start start 2024-04-13T03:10:21.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f668015d400 0x7f668013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f668015d7e0 0x7f66800cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66801429b0 con 0x7f66800cf720 2024-04-13T03:10:21.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f668006c0d0 con 0x7f668015d7e0 2024-04-13T03:10:21.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.410+0000 7f6686155640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f668006ba80 con 0x7f668015d400 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f668015d7e0 0x7f66800cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f668015d7e0 0x7f66800cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46660/0 (socket says 172.21.15.106:46660) 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 -- 172.21.15.106:0/44746584 learned_addr learned my addr 172.21.15.106:0/44746584 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f667f7fe640 1 --2- 172.21.15.106:0/44746584 >> v2:172.21.15.114:3300/0 conn(0x7f668015d400 0x7f668013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 -- 172.21.15.106:0/44746584 >> v2:172.21.15.114:3300/0 conn(0x7f668015d400 msgr2=0x7f668013fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 --2- 172.21.15.106:0/44746584 >> v2:172.21.15.114:3300/0 conn(0x7f668015d400 0x7f668013fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 -- 172.21.15.106:0/44746584 >> v1:172.21.15.114:6789/0 conn(0x7f66800cf720 legacy=0x7f6680141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.411+0000 7f6676ffd640 1 -- 172.21.15.106:0/44746584 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668006bd60 con 0x7f668015d7e0 2024-04-13T03:10:21.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6676ffd640 1 --2- 172.21.15.106:0/44746584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d7e0 0x7f66800cf1e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f66800b3090 tx=0x7f666802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ece0dbab8d6511 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f667effd640 1 -- 172.21.15.106:0/44746584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668035020 con 0x7f668015d7e0 2024-04-13T03:10:21.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f667effd640 1 -- 172.21.15.106:0/44746584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6668031870 con 0x7f668015d7e0 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f667effd640 1 -- 172.21.15.106:0/44746584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668031c40 con 0x7f668015d7e0 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 -- 172.21.15.106:0/44746584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d7e0 msgr2=0x7f66800cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 --2- 172.21.15.106:0/44746584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d7e0 0x7f66800cf1e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f66800b3090 tx=0x7f666802b910 comp rx=0 tx=0).stop 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 -- 172.21.15.106:0/44746584 shutdown_connections 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 --2- 172.21.15.106:0/44746584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d7e0 0x7f66800cf1e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 --2- 172.21.15.106:0/44746584 >> v2:172.21.15.114:3300/0 conn(0x7f668015d400 0x7f668013fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 -- 172.21.15.106:0/44746584 >> 172.21.15.106:0/44746584 conn(0x7f66800b2600 msgr2=0x7f66800d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 -- 172.21.15.106:0/44746584 shutdown_connections 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.412+0000 7f6686155640 1 -- 172.21.15.106:0/44746584 wait complete. 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.413+0000 7f6686155640 1 Processor -- start 2024-04-13T03:10:21.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 -- start start 2024-04-13T03:10:21.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66800cf720 0x7f6680152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668015d7e0 0x7f668015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6680142550 con 0x7f668015d400 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66801423d0 con 0x7f668015d7e0 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.414+0000 7f6686155640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66801426d0 con 0x7f66800cf720 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46672/0 (socket says 172.21.15.106:46672) 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 -- 172.21.15.106:0/2180420659 learned_addr learned my addr 172.21.15.106:0/2180420659 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f667f7fe640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66800cf720 0x7f6680152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f667ffff640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668015d7e0 0x7f668015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 -- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66800cf720 msgr2=0x7f6680152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66800cf720 0x7f6680152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 -- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668015d7e0 msgr2=0x7f668015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668015d7e0 0x7f668015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 -- 172.21.15.106:0/2180420659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66801e74b0 con 0x7f668015d400 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6676ffd640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f66680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668035040 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6668030d00 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801e6930 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66680456a0 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.415+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801e6ef0 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.416+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680068a10 con 0x7f668015d400 2024-04-13T03:10:21.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.417+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6668038020 con 0x7f668015d400 2024-04-13T03:10:21.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.417+0000 7f667cff9640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.417+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f66680ae220 con 0x7f668015d400 2024-04-13T03:10:21.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.417+0000 7f667f7fe640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.418+0000 7f667f7fe640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 0x7f666406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f668013f450 tx=0x7f6670008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.421+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f666802f320 con 0x7f668015d400 2024-04-13T03:10:21.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.597+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66800b35f0 con 0x7f668015d400 2024-04-13T03:10:21.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.598+0000 7f667cff9640 1 -- 172.21.15.106:0/2180420659 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f666807c2e0 con 0x7f668015d400 2024-04-13T03:10:21.599 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:21.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.599+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 msgr2=0x7f666406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.599+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 0x7f666406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f668013f450 tx=0x7f6670008040 comp rx=0 tx=0).stop 2024-04-13T03:10:21.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.599+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 msgr2=0x7f66801e5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.599+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f66680046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 shutdown_connections 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f668015d7e0 0x7f668015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f668015d400 0x7f66801e5fc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 --2- 172.21.15.106:0/2180420659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66800cf720 0x7f6680152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 >> 172.21.15.106:0/2180420659 conn(0x7f66800b2600 msgr2=0x7f66800b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 shutdown_connections 2024-04-13T03:10:21.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.600+0000 7f6686155640 1 -- 172.21.15.106:0/2180420659 wait complete. 2024-04-13T03:10:21.612 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.714+0000 7f34f1baa640 1 Processor -- start 2024-04-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.714+0000 7f34f1baa640 1 -- start start 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 0x7f34ec0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ec06ed70 0x7f34ec0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec0d0820 0x7f34ec0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f34ec0d0c90 con 0x7f34ec06ed70 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f34ec06f200 con 0x7f34ec15e930 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34f1baa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34ec06e280 con 0x7f34ec0d0820 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ec06ed70 0x7f34ec0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34ec06ed70 0x7f34ec0d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46682/0 (socket says 172.21.15.106:46682) 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 -- 172.21.15.106:0/3267327585 learned_addr learned my addr 172.21.15.106:0/3267327585 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34ebfff640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec0d0820 0x7f34ec0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eb7fe640 1 --2- 172.21.15.106:0/3267327585 >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 0x7f34ec0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 -- 172.21.15.106:0/3267327585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec0d0820 msgr2=0x7f34ec0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec0d0820 0x7f34ec0cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 -- 172.21.15.106:0/3267327585 >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 msgr2=0x7f34ec0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 --2- 172.21.15.106:0/3267327585 >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 0x7f34ec0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eaffd640 1 -- 172.21.15.106:0/3267327585 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec06e560 con 0x7f34ec06ed70 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.715+0000 7f34eb7fe640 1 --2- 172.21.15.106:0/3267327585 >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 0x7f34ec0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34eaffd640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0d02e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0b50f0 tx=0x7f34e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20202131ba46910f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34ea7fc640 1 -- 172.21.15.106:0/3267327585 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0035020 con 0x7f34ec06ed70 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34ea7fc640 1 -- 172.21.15.106:0/3267327585 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e0031870 con 0x7f34ec06ed70 2024-04-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34ea7fc640 1 -- 172.21.15.106:0/3267327585 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0031c40 con 0x7f34ec06ed70 2024-04-13T03:10:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34f1baa640 1 -- 172.21.15.106:0/3267327585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 msgr2=0x7f34ec0d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.716+0000 7f34f1baa640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0d02e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0b50f0 tx=0x7f34e002b910 comp rx=0 tx=0).stop 2024-04-13T03:10:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 -- 172.21.15.106:0/3267327585 shutdown_connections 2024-04-13T03:10:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec0d0820 0x7f34ec0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 --2- 172.21.15.106:0/3267327585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0d02e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 --2- 172.21.15.106:0/3267327585 >> v2:172.21.15.114:3300/0 conn(0x7f34ec15e930 0x7f34ec0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 -- 172.21.15.106:0/3267327585 >> 172.21.15.106:0/3267327585 conn(0x7f34ec0cef80 msgr2=0x7f34ec0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 -- 172.21.15.106:0/3267327585 shutdown_connections 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.717+0000 7f34f1baa640 1 -- 172.21.15.106:0/3267327585 wait complete. 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.718+0000 7f34f1baa640 1 Processor -- start 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.718+0000 7f34f1baa640 1 -- start start 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34ec0d0820 0x7f34ec14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec15e930 0x7f34ec1e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec0cd2b0 con 0x7f34ec06ed70 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec0ccfd0 con 0x7f34ec0d0820 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34ec0ce830 con 0x7f34ec15e930 2024-04-13T03:10:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46686/0 (socket says 172.21.15.106:46686) 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 -- 172.21.15.106:0/2725127276 learned_addr learned my addr 172.21.15.106:0/2725127276 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 -- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec15e930 msgr2=0x7f34ec1e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34ebfff640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec15e930 0x7f34ec1e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eaffd640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34ec0d0820 0x7f34ec14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec15e930 0x7f34ec1e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 -- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34ec0d0820 msgr2=0x7f34ec14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34ec0d0820 0x7f34ec14a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 -- 172.21.15.106:0/2725127276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec14aa10 con 0x7f34ec06ed70 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34eb7fe640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c7f0 tx=0x7f34dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc015070 con 0x7f34ec06ed70 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34dc007820 con 0x7f34ec06ed70 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec14ad90 con 0x7f34ec06ed70 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc007bd0 con 0x7f34ec06ed70 2024-04-13T03:10:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.719+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec14b350 con 0x7f34ec06ed70 2024-04-13T03:10:21.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.720+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ec068a10 con 0x7f34ec06ed70 2024-04-13T03:10:21.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.721+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34dc018070 con 0x7f34ec06ed70 2024-04-13T03:10:21.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.721+0000 7f34d3fff640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 0x7f34d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:21.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.721+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f34dc08c020 con 0x7f34ec06ed70 2024-04-13T03:10:21.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.721+0000 7f34eaffd640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 0x7f34d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:21.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.721+0000 7f34eaffd640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 0x7f34d806ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34e0036000 tx=0x7f34e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:21.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.725+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34ec14aa10 con 0x7f34ec06ed70 2024-04-13T03:10:21.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.901+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34ec0b5550 con 0x7f34ec06ed70 2024-04-13T03:10:21.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.902+0000 7f34d3fff640 1 -- 172.21.15.106:0/2725127276 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f34dc05a050 con 0x7f34ec06ed70 2024-04-13T03:10:21.903 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 msgr2=0x7f34d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 0x7f34d806ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34e0036000 tx=0x7f34e0008030 comp rx=0 tx=0).stop 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 msgr2=0x7f34ec0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c7f0 tx=0x7f34dc00ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 shutdown_connections 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34d806c920 0x7f34d806ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34ec15e930 0x7f34ec1e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34ec0d0820 0x7f34ec14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 --2- 172.21.15.106:0/2725127276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ec06ed70 0x7f34ec0c9f30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.903+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 >> 172.21.15.106:0/2725127276 conn(0x7f34ec0cef80 msgr2=0x7f34ec06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.904+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 shutdown_connections 2024-04-13T03:10:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:21.904+0000 7f34f1baa640 1 -- 172.21.15.106:0/2725127276 wait complete. 2024-04-13T03:10:21.916 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:22.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.019+0000 7fad10b1e640 1 Processor -- start 2024-04-13T03:10:22.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.019+0000 7fad10b1e640 1 -- start start 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.019+0000 7fad10b1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fad0c0b1c30 0x7fad0c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad10b1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad0c0b54a0 0x7fad0c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad10b1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad0c0b5cf0 con 0x7fad0c0709e0 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad10b1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c0b5fd0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad10b1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c0b62b0 con 0x7fad0c0b1c30 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad0c0b54a0 0x7fad0c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad0c0b54a0 0x7fad0c0704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46688/0 (socket says 172.21.15.106:46688) 2024-04-13T03:10:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad0ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad0c0709e0 0x7fad0c162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37502/0 (socket says 172.21.15.106:37502) 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 -- 172.21.15.106:0/3165624695 learned_addr learned my addr 172.21.15.106:0/3165624695 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 -- 172.21.15.106:0/3165624695 >> v2:172.21.15.114:3300/0 conn(0x7fad0c0b1c30 msgr2=0x7fad0c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 --2- 172.21.15.106:0/3165624695 >> v2:172.21.15.114:3300/0 conn(0x7fad0c0b1c30 0x7fad0c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 -- 172.21.15.106:0/3165624695 >> v1:172.21.15.106:6789/0 conn(0x7fad0c0709e0 legacy=0x7fad0c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad03fff640 1 -- 172.21.15.106:0/3165624695 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c0b6590 con 0x7fad0c0b54a0 2024-04-13T03:10:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.020+0000 7fad09d9b640 1 -- 172.21.15.106:0/3165624695 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038244572 0 0) 0x7fad0c0b5cf0 con 0x7fad0c0709e0 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad03fff640 1 --2- 172.21.15.106:0/3165624695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c0704a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0687a0 tx=0x7facf402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5374cd0ce4b248f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad09d9b640 1 -- 172.21.15.106:0/3165624695 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4034070 con 0x7fad0c0b54a0 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad09d9b640 1 -- 172.21.15.106:0/3165624695 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf4030860 con 0x7fad0c0b54a0 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad09d9b640 1 -- 172.21.15.106:0/3165624695 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4030c30 con 0x7fad0c0b54a0 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 -- 172.21.15.106:0/3165624695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 msgr2=0x7fad0c0704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 --2- 172.21.15.106:0/3165624695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c0704a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0687a0 tx=0x7facf402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 -- 172.21.15.106:0/3165624695 shutdown_connections 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 --2- 172.21.15.106:0/3165624695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c0704a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 --2- 172.21.15.106:0/3165624695 >> v2:172.21.15.114:3300/0 conn(0x7fad0c0b1c30 0x7fad0c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 -- 172.21.15.106:0/3165624695 >> 172.21.15.106:0/3165624695 conn(0x7fad0c071870 msgr2=0x7fad0c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 -- 172.21.15.106:0/3165624695 shutdown_connections 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.021+0000 7fad10b1e640 1 -- 172.21.15.106:0/3165624695 wait complete. 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 Processor -- start 2024-04-13T03:10:22.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 -- start start 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 0x7fad0c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 0x7fad0c146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c06e7f0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c06e670 con 0x7fad0c0b1c30 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.023+0000 7fad10b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c06e970 con 0x7fad0c0709e0 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46692/0 (socket says 172.21.15.106:46692) 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 -- 172.21.15.106:0/2271985244 learned_addr learned my addr 172.21.15.106:0/2271985244 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad03fff640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 0x7fad0c146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0a59c640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 0x7fad0c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 -- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 msgr2=0x7fad0c151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 0x7fad0c151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 -- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 msgr2=0x7fad0c146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 0x7fad0c146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 -- 172.21.15.106:0/2271985244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c1545f0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad03fff640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 0x7fad0c146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0a59c640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 0x7fad0c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad0ad9d640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c7a0 tx=0x7fad0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04018070 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0400a640 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c1548d0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400aab0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.024+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c1542c0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.025+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c068a10 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.026+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad0401b070 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.026+0000 7fad02ffd640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.026+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fad0408b4b0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.026+0000 7fad0a59c640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.026+0000 7fad0a59c640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 0x7facec06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facfc006fd0 tx=0x7facfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.030+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad04059570 con 0x7fad0c0b54a0 2024-04-13T03:10:22.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.206+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad0c0711f0 con 0x7fad0c0b54a0 2024-04-13T03:10:22.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.207+0000 7fad02ffd640 1 -- 172.21.15.106:0/2271985244 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fad04058e70 con 0x7fad0c0b54a0 2024-04-13T03:10:22.208 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 msgr2=0x7facec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 0x7facec06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facfc006fd0 tx=0x7facfc008040 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 msgr2=0x7fad0c153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c7a0 tx=0x7fad0400cc60 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 shutdown_connections 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad0c0b54a0 0x7fad0c153d80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad0c0b1c30 0x7fad0c146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 --2- 172.21.15.106:0/2271985244 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fad0c0709e0 0x7fad0c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 >> 172.21.15.106:0/2271985244 conn(0x7fad0c071870 msgr2=0x7fad0c06fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 shutdown_connections 2024-04-13T03:10:22.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.209+0000 7fad10b1e640 1 -- 172.21.15.106:0/2271985244 wait complete. 2024-04-13T03:10:22.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:22.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.323+0000 7fd05bd1e640 1 Processor -- start 2024-04-13T03:10:22.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.323+0000 7fd05bd1e640 1 -- start start 2024-04-13T03:10:22.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05bd1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0540b1c30 0x7fd05415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05bd1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd0540704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05bd1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0540b5cf0 con 0x7fd0540709e0 2024-04-13T03:10:22.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05bd1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd0540b5fd0 con 0x7fd0540b1c30 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05bd1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd0540b62b0 con 0x7fd0540b54a0 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0540b1c30 0x7fd05415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05a2bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd0540709e0 0x7fd054162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37516/0 (socket says 172.21.15.106:37516) 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd0540b1c30 0x7fd05415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46698/0 (socket says 172.21.15.106:46698) 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd05a2bb640 1 -- 172.21.15.106:0/421432289 learned_addr learned my addr 172.21.15.106:0/421432289 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 -- 172.21.15.106:0/421432289 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 msgr2=0x7fd0540704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd0592b9640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd0540704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd0540704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 -- 172.21.15.106:0/421432289 >> v1:172.21.15.106:6789/0 conn(0x7fd0540709e0 legacy=0x7fd054162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd058ab8640 1 -- 172.21.15.106:0/421432289 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416538252 0 0) 0x7fd0540b5cf0 con 0x7fd0540709e0 2024-04-13T03:10:22.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.324+0000 7fd059aba640 1 -- 172.21.15.106:0/421432289 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540b6590 con 0x7fd0540b1c30 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.325+0000 7fd059aba640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd05415ee40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd05002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42f96e7affdc134 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.325+0000 7fd058ab8640 1 -- 172.21.15.106:0/421432289 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050035020 con 0x7fd0540b1c30 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.325+0000 7fd058ab8640 1 -- 172.21.15.106:0/421432289 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050031800 con 0x7fd0540b1c30 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.325+0000 7fd058ab8640 1 -- 172.21.15.106:0/421432289 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050031bd0 con 0x7fd0540b1c30 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 -- 172.21.15.106:0/421432289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 msgr2=0x7fd05415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd05415ee40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd05002b8a0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 -- 172.21.15.106:0/421432289 shutdown_connections 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd0540704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/421432289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd05415ee40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 -- 172.21.15.106:0/421432289 >> 172.21.15.106:0/421432289 conn(0x7fd054071870 msgr2=0x7fd054071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 -- 172.21.15.106:0/421432289 shutdown_connections 2024-04-13T03:10:22.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.326+0000 7fd05bd1e640 1 -- 172.21.15.106:0/421432289 wait complete. 2024-04-13T03:10:22.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.327+0000 7fd05bd1e640 1 Processor -- start 2024-04-13T03:10:22.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.327+0000 7fd05bd1e640 1 -- start start 2024-04-13T03:10:22.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0540709e0 0x7fd0540d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd054140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd05406e7f0 con 0x7fd0540b1c30 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd05406e670 con 0x7fd0540709e0 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd05406e970 con 0x7fd0540b54a0 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46708/0 (socket says 172.21.15.106:46708) 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 -- 172.21.15.106:0/3678183071 learned_addr learned my addr 172.21.15.106:0/3678183071 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 -- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 msgr2=0x7fd054140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd054140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 -- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0540709e0 msgr2=0x7fd0540d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0540709e0 0x7fd0540d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 -- 172.21.15.106:0/3678183071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd054151cd0 con 0x7fd0540b1c30 2024-04-13T03:10:22.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd0592b9640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c7f0 tx=0x7fd04400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044015070 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044004220 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd054151fb0 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044003610 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.328+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0540cfff0 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.329+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054068a10 con 0x7fd0540b1c30 2024-04-13T03:10:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.330+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0440039c0 con 0x7fd0540b1c30 2024-04-13T03:10:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.330+0000 7fd04a7fc640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 0x7fd03806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.330+0000 7fd059aba640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 0x7fd03806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.330+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd04408c1a0 con 0x7fd0540b1c30 2024-04-13T03:10:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.331+0000 7fd059aba640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 0x7fd03806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd050036000 tx=0x7fd050030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.335+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd04405a260 con 0x7fd0540b1c30 2024-04-13T03:10:22.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.511+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0540711f0 con 0x7fd0540b1c30 2024-04-13T03:10:22.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.511+0000 7fd04a7fc640 1 -- 172.21.15.106:0/3678183071 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fd044059b60 con 0x7fd0540b1c30 2024-04-13T03:10:22.513 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 msgr2=0x7fd03806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 0x7fd03806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd050036000 tx=0x7fd050030000 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 msgr2=0x7fd0540cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c7f0 tx=0x7fd04400ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 shutdown_connections 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd03806c8f0 0x7fd03806edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd0540b54a0 0x7fd054140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0540b1c30 0x7fd0540cdb70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 --2- 172.21.15.106:0/3678183071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0540709e0 0x7fd0540d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 >> 172.21.15.106:0/3678183071 conn(0x7fd054071870 msgr2=0x7fd05415c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 shutdown_connections 2024-04-13T03:10:22.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.513+0000 7fd05bd1e640 1 -- 172.21.15.106:0/3678183071 wait complete. 2024-04-13T03:10:22.525 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:10:22.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.625+0000 7f096978c640 1 Processor -- start 2024-04-13T03:10:22.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.625+0000 7f096978c640 1 -- start start 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f096978c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09640b3c00 0x7f09640b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f096978c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096415eee0 con 0x7f096415eb00 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f096978c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09640ba580 con 0x7f09640cfc60 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f096978c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09640d2b30 con 0x7f09640b3c00 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f09637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09640b3c00 0x7f09640b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f0962ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f096415eb00 0x7f09640cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37532/0 (socket says 172.21.15.106:37532) 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f0962ffd640 1 -- 172.21.15.106:0/2842480639 learned_addr learned my addr 172.21.15.106:0/2842480639 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f09637fe640 1 -- 172.21.15.106:0/2842480639 >> v1:172.21.15.114:6789/0 conn(0x7f09640cfc60 legacy=0x7f09640d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f09637fe640 1 -- 172.21.15.106:0/2842480639 >> v1:172.21.15.106:6789/0 conn(0x7f096415eb00 legacy=0x7f09640cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f09637fe640 1 -- 172.21.15.106:0/2842480639 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640d2d80 con 0x7f09640b3c00 2024-04-13T03:10:22.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.626+0000 7f0961ffb640 1 -- 172.21.15.106:0/2842480639 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297733464 0 0) 0x7f096415eee0 con 0x7f096415eb00 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.627+0000 7f09637fe640 1 --2- 172.21.15.106:0/2842480639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09640b3c00 0x7f09640b7920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d90 tx=0x7f095c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d2641bd00da537 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.627+0000 7f0961ffb640 1 -- 172.21.15.106:0/2842480639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c039070 con 0x7f09640b3c00 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.627+0000 7f0961ffb640 1 -- 172.21.15.106:0/2842480639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c02cc60 con 0x7f09640b3c00 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.627+0000 7f0961ffb640 1 -- 172.21.15.106:0/2842480639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c034660 con 0x7f09640b3c00 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 -- 172.21.15.106:0/2842480639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09640b3c00 msgr2=0x7f09640b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 --2- 172.21.15.106:0/2842480639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09640b3c00 0x7f09640b7920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d90 tx=0x7f095c02beb0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 -- 172.21.15.106:0/2842480639 shutdown_connections 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 --2- 172.21.15.106:0/2842480639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09640b3c00 0x7f09640b7920 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 -- 172.21.15.106:0/2842480639 >> 172.21.15.106:0/2842480639 conn(0x7f09640d4780 msgr2=0x7f09640d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 -- 172.21.15.106:0/2842480639 shutdown_connections 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.628+0000 7f096978c640 1 -- 172.21.15.106:0/2842480639 wait complete. 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.629+0000 7f096978c640 1 Processor -- start 2024-04-13T03:10:22.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.629+0000 7f096978c640 1 -- start start 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 0x7f0964158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 0x7f0964155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09640ba110 con 0x7f096415eb00 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09640d3340 con 0x7f09640b3c00 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f096978c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f09640ba290 con 0x7f09640cfc60 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46732/0 (socket says 172.21.15.106:46732) 2024-04-13T03:10:22.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 -- 172.21.15.106:0/1092394253 learned_addr learned my addr 172.21.15.106:0/1092394253 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f0962ffd640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 0x7f0964158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09627fc640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 0x7f0964155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 -- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 msgr2=0x7f0964155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 0x7f0964155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 -- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 msgr2=0x7f0964158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 0x7f0964158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09637fe640 1 -- 172.21.15.106:0/1092394253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0964155840 con 0x7f096415eb00 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f09627fc640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 0x7f0964155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.630+0000 7f0962ffd640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 0x7f0964158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f09637fe640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d90 tx=0x7f095c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c039050 con 0x7f096415eb00 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c03a040 con 0x7f096415eb00 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096415a220 con 0x7f096415eb00 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c03e610 con 0x7f096415eb00 2024-04-13T03:10:22.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.631+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096415a7e0 con 0x7f096415eb00 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.632+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0964069120 con 0x7f096415eb00 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.632+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f095c047020 con 0x7f096415eb00 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.632+0000 7f094b7fe640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.633+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f095c0ae290 con 0x7f096415eb00 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.633+0000 7f0962ffd640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:22.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.633+0000 7f0962ffd640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0954000f60 tx=0x7f0954007c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:22.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.637+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f095c04e020 con 0x7f096415eb00 2024-04-13T03:10:22.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.823+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f096415ded0 con 0x7f096415eb00 2024-04-13T03:10:22.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.824+0000 7f094b7fe640 1 -- 172.21.15.106:0/1092394253 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f095c07c410 con 0x7f096415eb00 2024-04-13T03:10:22.825 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 msgr2=0x7f095006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0954000f60 tx=0x7f0954007c50 comp rx=0 tx=0).stop 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 msgr2=0x7f0964146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d90 tx=0x7f095c004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 shutdown_connections 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f09640cfc60 0x7f0964155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f096415eb00 0x7f0964146c20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 --2- 172.21.15.106:0/1092394253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09640b3c00 0x7f0964158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:22.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 >> 172.21.15.106:0/1092394253 conn(0x7f09640d4780 msgr2=0x7f096415f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:22.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 shutdown_connections 2024-04-13T03:10:22.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:22.826+0000 7f096978c640 1 -- 172.21.15.106:0/1092394253 wait complete. 2024-04-13T03:10:22.838 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-13T03:10:22.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:22.838 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-13T03:10:22.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:10:22.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:22.871 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-13T03:10:22.928 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:10:22.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:23.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.028+0000 7f6cf0d1e640 1 Processor -- start 2024-04-13T03:10:23.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.028+0000 7f6cf0d1e640 1 -- start start 2024-04-13T03:10:23.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cf0d1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6cec15e980 0x7f6cec162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cf0d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cec0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cf0d1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cec15fe40 con 0x7f6cec0cd380 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cf0d1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6cec0cd7f0 con 0x7f6cec15e980 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cf0d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec1633f0 con 0x7f6cec15ed60 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cead9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6cec0cd380 0x7f6cec0d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53672/0 (socket says 172.21.15.106:53672) 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cead9d640 1 -- 172.21.15.106:0/4125583037 learned_addr learned my addr 172.21.15.106:0/4125583037 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710342386 0 0) 0x7f6cec15fe40 con 0x7f6cec0cd380 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cd8003520 con 0x7f6cec0cd380 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cea59c640 1 --2- 172.21.15.106:0/4125583037 >> v2:172.21.15.114:3300/0 conn(0x7f6cec15e980 0x7f6cec162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cec0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1518651132 0 0) 0x7f6cd8003520 con 0x7f6cec0cd380 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 msgr2=0x7f6cec0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cec0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 >> v2:172.21.15.114:3300/0 conn(0x7f6cec15e980 msgr2=0x7f6cec162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 --2- 172.21.15.106:0/4125583037 >> v2:172.21.15.114:3300/0 conn(0x7f6cec15e980 0x7f6cec162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec15f2c0 con 0x7f6cec0cd380 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6ce40019f0 con 0x7f6cec0cd380 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 >> v1:172.21.15.106:6789/0 conn(0x7f6cec0cd380 legacy=0x7f6cec0d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cd80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd8004d40 0x7f6cd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec0cd7f0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec1633f0 con 0x7f6cd8004d40 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd800bb50 con 0x7f6cec15ed60 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.029+0000 7f6cea59c640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cd80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6cead9d640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd8004d40 0x7f6cd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 -- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 msgr2=0x7f6cd80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec15ed60 0x7f6cd80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 -- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd8004d40 msgr2=0x7f6cd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd8004d40 0x7f6cd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 -- 172.21.15.106:0/4125583037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd8003520 con 0x7f6cd8003fe0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.030+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cd8004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400ba00 tx=0x7f6cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c00160d4376f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4036020 con 0x7f6cd8003fe0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd4032620 con 0x7f6cd8003fe0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6ce9d9b640 1 -- 172.21.15.106:0/4125583037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400b600 con 0x7f6cd8003fe0 2024-04-13T03:10:23.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/4125583037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 msgr2=0x7f6cd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/4125583037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cd8004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400ba00 tx=0x7f6cd8005830 comp rx=0 tx=0).stop 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cead9d640 1 -- 172.21.15.106:0/4125583037 reap_dead start 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/4125583037 shutdown_connections 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/4125583037 >> 172.21.15.106:0/4125583037 conn(0x7f6cec0d2550 msgr2=0x7f6cec0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/4125583037 shutdown_connections 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.031+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/4125583037 wait complete. 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.032+0000 7f6cf0d1e640 1 Processor -- start 2024-04-13T03:10:23.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.032+0000 7f6cf0d1e640 1 -- start start 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec14f790 0x7f6cec0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cec0cbca0 0x7f6cec160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec15fb60 con 0x7f6cd8003fe0 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec15f880 con 0x7f6cec0cbca0 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cf0d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec0d1e20 con 0x7f6cec14f790 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59342/0 (socket says 172.21.15.106:59342) 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 -- 172.21.15.106:0/722143297 learned_addr learned my addr 172.21.15.106:0/722143297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 -- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec14f790 msgr2=0x7f6cec0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec14f790 0x7f6cec0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec14f790 0x7f6cec0cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cea59c640 1 -- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cec0cbca0 msgr2=0x7f6cec160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.033+0000 7f6cead9d640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cec0cbca0 0x7f6cec160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6cea59c640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cec0cbca0 0x7f6cec160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6cea59c640 1 -- 172.21.15.106:0/722143297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec0cc240 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6cea59c640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b7f0 tx=0x7f6cdc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc018070 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc0107d0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec0cc520 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc010ba0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.034+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec1f51a0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.035+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec0cd970 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.035+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cdc01b070 con 0x7f6cd8003fe0 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.036+0000 7f6ce2ffd640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 0x7f6ccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.036+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6cdc08c370 con 0x7f6cd8003fe0 2024-04-13T03:10:23.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.036+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 0x7f6ccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.036+0000 7f6ce3fff640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 0x7f6ccc06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6cec15f1f0 tx=0x7f6cd4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.040+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cec0cd970 con 0x7f6cd8003fe0 2024-04-13T03:10:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.252+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cec0b84d0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.253+0000 7f6ce2ffd640 1 -- 172.21.15.106:0/722143297 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f6cdc05a3a0 con 0x7f6cd8003fe0 2024-04-13T03:10:23.254 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:23.254 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:10:23.254 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 msgr2=0x7f6ccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 0x7f6ccc06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6cec15f1f0 tx=0x7f6cd4030000 comp rx=0 tx=0).stop 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 msgr2=0x7f6cec1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b7f0 tx=0x7f6cdc00bcb0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 shutdown_connections 2024-04-13T03:10:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6ccc06c8f0 0x7f6ccc06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cec14f790 0x7f6cec0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cec0cbca0 0x7f6cec160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 --2- 172.21.15.106:0/722143297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd8003fe0 0x7f6cec1f56c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 >> 172.21.15.106:0/722143297 conn(0x7f6cec0d2550 msgr2=0x7f6cec15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 shutdown_connections 2024-04-13T03:10:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.255+0000 7f6cf0d1e640 1 -- 172.21.15.106:0/722143297 wait complete. 2024-04-13T03:10:23.267 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:23.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.365+0000 7fae00c76640 1 Processor -- start 2024-04-13T03:10:23.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.365+0000 7fae00c76640 1 -- start start 2024-04-13T03:10:23.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fae00c76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc06cb10 0x7fadfc072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fae00c76640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadfc0998d0 con 0x7fadfc0cdf50 2024-04-13T03:10:23.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fae00c76640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadfc073020 con 0x7fadfc06e150 2024-04-13T03:10:23.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fae00c76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fadfc073300 con 0x7fadfc06cb10 2024-04-13T03:10:23.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fadfa59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fadfc0cdf50 0x7fadfc0ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53682/0 (socket says 172.21.15.106:53682) 2024-04-13T03:10:23.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.366+0000 7fadfa59c640 1 -- 172.21.15.106:0/98010861 learned_addr learned my addr 172.21.15.106:0/98010861 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975421132 0 0) 0x7fadfc0998d0 con 0x7fadfc0cdf50 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc06cb10 0x7fadfc072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade4003520 con 0x7fadfc0cdf50 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2752922166 0 0) 0x7fade4003520 con 0x7fadfc0cdf50 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc06cb10 msgr2=0x7fadfc072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc06cb10 0x7fadfc072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 >> v1:172.21.15.114:6789/0 conn(0x7fadfc06e150 legacy=0x7fadfc06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc0735e0 con 0x7fadfc0cdf50 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fadec0019f0 con 0x7fadfc0cdf50 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 >> v1:172.21.15.106:6789/0 conn(0x7fadfc0cdf50 legacy=0x7fadfc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade40039e0 0x7fade4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fade4004800 0x7fade4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fade40051c0 0x7fade400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc073300 con 0x7fade4004800 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fade400c060 con 0x7fade40039e0 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fade400e470 con 0x7fade40051c0 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc06cb10 0x7fadfc072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadfa59c640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade40039e0 0x7fade4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fade4004800 0x7fade4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadfad9d640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fade40051c0 0x7fade400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.367+0000 7fadf9d9b640 1 -- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fade40051c0 msgr2=0x7fade400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fade40051c0 0x7fade400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf9d9b640 1 -- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade40039e0 msgr2=0x7fade4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade40039e0 0x7fade4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf9d9b640 1 -- 172.21.15.106:0/98010861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4003520 con 0x7fade4004800 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fade4004800 0x7fade4004c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006fd0 tx=0x7fade4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c25fdd3f3e13695 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf000f040 con 0x7fade4004800 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0032600 con 0x7fade4004800 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.368+0000 7fadf959a640 1 -- 172.21.15.106:0/98010861 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00329d0 con 0x7fade4004800 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 -- 172.21.15.106:0/98010861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fade4004800 msgr2=0x7fade4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 --2- 172.21.15.106:0/98010861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fade4004800 0x7fade4004c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006fd0 tx=0x7fade4005830 comp rx=0 tx=0).stop 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fadfad9d640 1 -- 172.21.15.106:0/98010861 reap_dead start 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 -- 172.21.15.106:0/98010861 shutdown_connections 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 -- 172.21.15.106:0/98010861 >> 172.21.15.106:0/98010861 conn(0x7fadfc0b1f10 msgr2=0x7fadfc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 -- 172.21.15.106:0/98010861 shutdown_connections 2024-04-13T03:10:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.369+0000 7fae00c76640 1 -- 172.21.15.106:0/98010861 wait complete. 2024-04-13T03:10:23.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.370+0000 7fae00c76640 1 Processor -- start 2024-04-13T03:10:23.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.370+0000 7fae00c76640 1 -- start start 2024-04-13T03:10:23.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4004800 0x7fadfc0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 0x7fadfc09a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc0728a0 con 0x7fadfc06ea80 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc072720 con 0x7fade4004800 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fae00c76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fadfc072a20 con 0x7fadfc09a300 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59370/0 (socket says 172.21.15.106:59370) 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 -- 172.21.15.106:0/867733111 learned_addr learned my addr 172.21.15.106:0/867733111 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadfa59c640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4004800 0x7fadfc0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadfad9d640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 0x7fadfc09a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 -- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 msgr2=0x7fadfc09a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 0x7fadfc09a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 -- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4004800 msgr2=0x7fadfc0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4004800 0x7fadfc0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadf9d9b640 1 -- 172.21.15.106:0/867733111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc09b030 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.371+0000 7fadfad9d640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 0x7fadfc09a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fadf9d9b640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fadf002bd90 tx=0x7fadf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00319e0 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0031b80 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc14ad50 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf003c410 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.372+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc14b310 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.373+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadfc068a10 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.373+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadf003c5b0 con 0x7fadfc06ea80 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.374+0000 7fadeaffd640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 0x7faddc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.374+0000 7fadfa59c640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 0x7faddc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.374+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fadf00ae180 con 0x7fadfc06ea80 2024-04-13T03:10:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.374+0000 7fadfa59c640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 0x7faddc06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadec0013d0 tx=0x7fadec006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.378+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadf0040030 con 0x7fadfc06ea80 2024-04-13T03:10:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.555+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadfc0998d0 con 0x7fadfc06ea80 2024-04-13T03:10:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.556+0000 7fadeaffd640 1 -- 172.21.15.106:0/867733111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fadf007c240 con 0x7fadfc06ea80 2024-04-13T03:10:23.557 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:23.557 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 msgr2=0x7faddc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 0x7faddc06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadec0013d0 tx=0x7fadec006040 comp rx=0 tx=0).stop 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 msgr2=0x7fadfc06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fadf002bd90 tx=0x7fadf0004730 comp rx=0 tx=0).stop 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 shutdown_connections 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faddc06c920 0x7faddc06ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fadfc09a300 0x7fadfc09a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadfc06ea80 0x7fadfc06eee0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 --2- 172.21.15.106:0/867733111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4004800 0x7fadfc0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 >> 172.21.15.106:0/867733111 conn(0x7fadfc0b1f10 msgr2=0x7fadfc06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 shutdown_connections 2024-04-13T03:10:23.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.558+0000 7fae00c76640 1 -- 172.21.15.106:0/867733111 wait complete. 2024-04-13T03:10:23.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:09:22.361852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:09:23.267069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T03:09:26.991920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T03:10:23.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:23.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.673+0000 7f450038c640 1 Processor -- start 2024-04-13T03:10:23.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.673+0000 7f450038c640 1 -- start start 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f450038c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 0x7f44f80ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f450038c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80706d0 con 0x7f44f815c820 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f450038c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80709b0 con 0x7f44f80cd0a0 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f450038c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f44f80cd510 con 0x7f44f80b7f00 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f44fe128640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f44f815c820 0x7f44f806fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53696/0 (socket says 172.21.15.106:53696) 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.674+0000 7f44fe128640 1 -- 172.21.15.106:0/2164170882 learned_addr learned my addr 172.21.15.106:0/2164170882 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd927640 1 --2- 172.21.15.106:0/2164170882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 0x7f44f80ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140113582 0 0) 0x7f44f80706d0 con 0x7f44f815c820 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e4003520 con 0x7f44f815c820 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341413570 0 0) 0x7f44f80709b0 con 0x7f44f80cd0a0 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44f80706d0 con 0x7f44f80cd0a0 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104529934 0 0) 0x7f44e4003520 con 0x7f44f815c820 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd927640 1 -- 172.21.15.106:0/2164170882 >> v1:172.21.15.114:6789/0 conn(0x7f44f80cd0a0 legacy=0x7f44f80d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd927640 1 -- 172.21.15.106:0/2164170882 >> v1:172.21.15.106:6789/0 conn(0x7f44f815c820 legacy=0x7f44f806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.675+0000 7f44fd927640 1 -- 172.21.15.106:0/2164170882 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8070c90 con 0x7f44f80b7f00 2024-04-13T03:10:23.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f44fd927640 1 --2- 172.21.15.106:0/2164170882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 0x7f44f80ccb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097d0 tx=0x7f44e802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7732eac5846cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8037070 con 0x7f44f80b7f00 2024-04-13T03:10:23.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e8030510 con 0x7f44f80b7f00 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f44fd126640 1 -- 172.21.15.106:0/2164170882 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40019f0 con 0x7f44f80b7f00 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f450038c640 1 -- 172.21.15.106:0/2164170882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 msgr2=0x7f44f80ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f450038c640 1 --2- 172.21.15.106:0/2164170882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 0x7f44f80ccb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097d0 tx=0x7f44e802b800 comp rx=0 tx=0).stop 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f450038c640 1 -- 172.21.15.106:0/2164170882 shutdown_connections 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f450038c640 1 --2- 172.21.15.106:0/2164170882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80b7f00 0x7f44f80ccb60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.676+0000 7f450038c640 1 -- 172.21.15.106:0/2164170882 >> 172.21.15.106:0/2164170882 conn(0x7f44f80d2510 msgr2=0x7f44f80d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.677+0000 7f450038c640 1 -- 172.21.15.106:0/2164170882 shutdown_connections 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.677+0000 7f450038c640 1 -- 172.21.15.106:0/2164170882 wait complete. 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 Processor -- start 2024-04-13T03:10:23.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 -- start start 2024-04-13T03:10:23.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 0x7f44f8159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80cd0a0 0x7f44f8157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80d0ff0 con 0x7f44f815c820 2024-04-13T03:10:23.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80d03c0 con 0x7f44f80b7f00 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f450038c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f44f80d1170 con 0x7f44f80cd0a0 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f44fe929640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f44fe929640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59386/0 (socket says 172.21.15.106:59386) 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.678+0000 7f44fe929640 1 -- 172.21.15.106:0/525359122 learned_addr learned my addr 172.21.15.106:0/525359122 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fd927640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80cd0a0 0x7f44f8157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe128640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 0x7f44f8159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 -- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80cd0a0 msgr2=0x7f44f8157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80cd0a0 0x7f44f8157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 -- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 msgr2=0x7f44f8159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 0x7f44f8159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 -- 172.21.15.106:0/525359122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8157b80 con 0x7f44f815c820 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe128640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 0x7f44f8159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:23.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44fe929640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c430 tx=0x7f44f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0017070 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f00052d0 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f815a1c0 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00037f0 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.679+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f80c9710 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.680+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f8069120 con 0x7f44f815c820 2024-04-13T03:10:23.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.681+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44f001a070 con 0x7f44f815c820 2024-04-13T03:10:23.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.681+0000 7f44eeffd640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 0x7f44dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.681+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f44f008b250 con 0x7f44f815c820 2024-04-13T03:10:23.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.681+0000 7f44fe128640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 0x7f44dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.681+0000 7f44fe128640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 0x7f44dc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001b60 tx=0x7f44f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.685+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44f0058c40 con 0x7f44f815c820 2024-04-13T03:10:23.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.861+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44f815fff0 con 0x7f44f815c820 2024-04-13T03:10:23.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.862+0000 7f44eeffd640 1 -- 172.21.15.106:0/525359122 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f44f005cf40 con 0x7f44f815c820 2024-04-13T03:10:23.863 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 msgr2=0x7f44dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 0x7f44dc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001b60 tx=0x7f44f4009040 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 msgr2=0x7f44f80c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c430 tx=0x7f44f000c8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 shutdown_connections 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44dc06c8f0 0x7f44dc06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44f80cd0a0 0x7f44f8157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44f815c820 0x7f44f80c88b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 --2- 172.21.15.106:0/525359122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f80b7f00 0x7f44f8159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.863+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 >> 172.21.15.106:0/525359122 conn(0x7f44f80d2510 msgr2=0x7f44f815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.864+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 shutdown_connections 2024-04-13T03:10:23.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.864+0000 7f450038c640 1 -- 172.21.15.106:0/525359122 wait complete. 2024-04-13T03:10:23.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:23.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.978+0000 7f92f0ead640 1 Processor -- start 2024-04-13T03:10:23.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.978+0000 7f92f0ead640 1 -- start start 2024-04-13T03:10:23.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92f0ead640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f92ec15e8c0 0x7f92ec0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92f0ead640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92f0ead640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ec0b5060 con 0x7f92ec073350 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92f0ead640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f92ec0b5340 con 0x7f92ec15e8c0 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92f0ead640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec06c500 con 0x7f92ec15eca0 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f92ec15e8c0 0x7f92ec0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f92ec15e8c0 0x7f92ec0ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59392/0 (socket says 172.21.15.106:59392) 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 -- 172.21.15.106:0/1918214830 learned_addr learned my addr 172.21.15.106:0/1918214830 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92e3fff640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 -- 172.21.15.106:0/1918214830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 msgr2=0x7f92ec072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 -- 172.21.15.106:0/1918214830 >> v1:172.21.15.114:6789/0 conn(0x7f92ec073350 legacy=0x7f92ec0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92ea59c640 1 -- 172.21.15.106:0/1918214830 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec06c7e0 con 0x7f92ec15e8c0 2024-04-13T03:10:23.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.979+0000 7f92e3fff640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.980+0000 7f92ea59c640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec15e8c0 0x7f92ec0ce250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4e0ea67435a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.980+0000 7f92e9d9b640 1 -- 172.21.15.106:0/1918214830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc034070 con 0x7f92ec15e8c0 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.980+0000 7f92e9d9b640 1 -- 172.21.15.106:0/1918214830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92dc030840 con 0x7f92ec15e8c0 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.980+0000 7f92e9d9b640 1 -- 172.21.15.106:0/1918214830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc030c10 con 0x7f92ec15e8c0 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 -- 172.21.15.106:0/1918214830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec15e8c0 msgr2=0x7f92ec0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec15e8c0 0x7f92ec0ce250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02b980 comp rx=0 tx=0).stop 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 -- 172.21.15.106:0/1918214830 shutdown_connections 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 --2- 172.21.15.106:0/1918214830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec15e8c0 0x7f92ec0ce250 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 -- 172.21.15.106:0/1918214830 >> 172.21.15.106:0/1918214830 conn(0x7f92ec06f0d0 msgr2=0x7f92ec06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 -- 172.21.15.106:0/1918214830 shutdown_connections 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.981+0000 7f92f0ead640 1 -- 172.21.15.106:0/1918214830 wait complete. 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.982+0000 7f92f0ead640 1 Processor -- start 2024-04-13T03:10:23.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.982+0000 7f92f0ead640 1 -- start start 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 0x7f92ec1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec06cd00 con 0x7f92ec073350 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0b4d50 con 0x7f92ec15e8c0 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec06ce80 con 0x7f92ec15eca0 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59398/0 (socket says 172.21.15.106:59398) 2024-04-13T03:10:23.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 -- 172.21.15.106:0/799301218 learned_addr learned my addr 172.21.15.106:0/799301218 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ead9d640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92e3fff640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 0x7f92ec1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 -- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 msgr2=0x7f92ec157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 -- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 msgr2=0x7f92ec1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 0x7f92ec1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 -- 172.21.15.106:0/799301218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec161a40 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92e3fff640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 0x7f92ec1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92ea59c640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc048070 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92dc030640 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec153e00 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.983+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc034040 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.984+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec160740 con 0x7f92ec073350 2024-04-13T03:10:23.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.984+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec068a10 con 0x7f92ec073350 2024-04-13T03:10:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.985+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92dc04b070 con 0x7f92ec073350 2024-04-13T03:10:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.985+0000 7f92e2ffd640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 0x7f92cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.985+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f92dc0ae070 con 0x7f92ec073350 2024-04-13T03:10:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.985+0000 7f92e3fff640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 0x7f92cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.986+0000 7f92e3fff640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 0x7f92cc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:23.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:23.989+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92dc07c0a0 con 0x7f92ec073350 2024-04-13T03:10:24.047 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:10:24.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:10:24.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.168+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f92ec0cdbf0 con 0x7f92ec073350 2024-04-13T03:10:24.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.169+0000 7f92e2ffd640 1 -- 172.21.15.106:0/799301218 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f92dc07b9a0 con 0x7f92ec073350 2024-04-13T03:10:24.170 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.170+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 msgr2=0x7f92cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.170+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 0x7f92cc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4008040 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 msgr2=0x7f92ec161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc004730 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 shutdown_connections 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f92cc06c8f0 0x7f92cc06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f92ec15eca0 0x7f92ec157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92ec15e8c0 0x7f92ec1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 --2- 172.21.15.106:0/799301218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92ec073350 0x7f92ec161500 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 >> 172.21.15.106:0/799301218 conn(0x7f92ec06f0d0 msgr2=0x7f92ec0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 shutdown_connections 2024-04-13T03:10:24.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.171+0000 7f92f0ead640 1 -- 172.21.15.106:0/799301218 wait complete. 2024-04-13T03:10:24.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.285+0000 7fccea9f7640 1 Processor -- start 2024-04-13T03:10:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.285+0000 7fccea9f7640 1 -- start start 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fccea9f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcce4156b40 0x7fcce406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fccea9f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce406ebf0 0x7fcce415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fccea9f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce415d690 con 0x7fcce415d110 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fccea9f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcce406b8e0 con 0x7fcce4156b40 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fccea9f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcce406bea0 con 0x7fcce406ebf0 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcce4156b40 0x7fcce406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce8f94640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcce415d110 0x7fcce406c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53708/0 (socket says 172.21.15.106:53708) 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcce4156b40 0x7fcce406e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59406/0 (socket says 172.21.15.106:59406) 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce8f94640 1 -- 172.21.15.106:0/2370257632 learned_addr learned my addr 172.21.15.106:0/2370257632 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 -- 172.21.15.106:0/2370257632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce406ebf0 msgr2=0x7fcce415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 --2- 172.21.15.106:0/2370257632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce406ebf0 0x7fcce415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 -- 172.21.15.106:0/2370257632 >> v1:172.21.15.106:6789/0 conn(0x7fcce415d110 legacy=0x7fcce406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce3fff640 1 -- 172.21.15.106:0/2370257632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce415d970 con 0x7fcce4156b40 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.286+0000 7fcce2ffd640 1 -- 172.21.15.106:0/2370257632 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604413747 0 0) 0x7fcce415d690 con 0x7fcce415d110 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fcce3fff640 1 --2- 172.21.15.106:0/2370257632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce4156b40 0x7fcce406e6b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fccd40097a0 tx=0x7fccd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f67e6823efed1af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fcce2ffd640 1 -- 172.21.15.106:0/2370257632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035020 con 0x7fcce4156b40 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fcce2ffd640 1 -- 172.21.15.106:0/2370257632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd4031830 con 0x7fcce4156b40 2024-04-13T03:10:24.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fcce2ffd640 1 -- 172.21.15.106:0/2370257632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4031c00 con 0x7fcce4156b40 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fccea9f7640 1 -- 172.21.15.106:0/2370257632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce4156b40 msgr2=0x7fcce406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.287+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2370257632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce4156b40 0x7fcce406e6b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fccd40097a0 tx=0x7fccd402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 -- 172.21.15.106:0/2370257632 shutdown_connections 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2370257632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce406ebf0 0x7fcce415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2370257632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce4156b40 0x7fcce406e6b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 -- 172.21.15.106:0/2370257632 >> 172.21.15.106:0/2370257632 conn(0x7fcce40cd6a0 msgr2=0x7fcce40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 -- 172.21.15.106:0/2370257632 shutdown_connections 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.288+0000 7fccea9f7640 1 -- 172.21.15.106:0/2370257632 wait complete. 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.289+0000 7fccea9f7640 1 Processor -- start 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.289+0000 7fccea9f7640 1 -- start start 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce4156b40 0x7fcce4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcce415d110 0x7fcce4142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcce406d880 con 0x7fcce406ebf0 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4156f20 con 0x7fcce415d110 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcce406da00 con 0x7fcce4156b40 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59408/0 (socket says 172.21.15.106:59408) 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 -- 172.21.15.106:0/2531807129 learned_addr learned my addr 172.21.15.106:0/2531807129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 -- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce4156b40 msgr2=0x7fcce4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce4156b40 0x7fcce4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 -- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcce415d110 msgr2=0x7fcce4142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcce415d110 0x7fcce4142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 -- 172.21.15.106:0/2531807129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4140640 con 0x7fcce406ebf0 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce3fff640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098c0 tx=0x7fccd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd403ba80 con 0x7fcce406ebf0 2024-04-13T03:10:24.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd403bc20 con 0x7fcce406ebf0 2024-04-13T03:10:24.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce4158280 con 0x7fcce406ebf0 2024-04-13T03:10:24.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd40453f0 con 0x7fcce406ebf0 2024-04-13T03:10:24.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.290+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce4158840 con 0x7fcce406ebf0 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.291+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce4068a10 con 0x7fcce406ebf0 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.292+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccd404c060 con 0x7fcce406ebf0 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.292+0000 7fcce0ff9640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.292+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fccd40add20 con 0x7fcce406ebf0 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.292+0000 7fcce37fe640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.292+0000 7fcce37fe640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 0x7fccd006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcce41527a0 tx=0x7fccd8006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.296+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fccd4040070 con 0x7fcce406ebf0 2024-04-13T03:10:24.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.471+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcce415a740 con 0x7fcce406ebf0 2024-04-13T03:10:24.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.471+0000 7fcce0ff9640 1 -- 172.21.15.106:0/2531807129 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fccd402f2e0 con 0x7fcce406ebf0 2024-04-13T03:10:24.473 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 msgr2=0x7fccd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 0x7fccd006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcce41527a0 tx=0x7fccd8006c90 comp rx=0 tx=0).stop 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 msgr2=0x7fcce4140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098c0 tx=0x7fccd40046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 shutdown_connections 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcce4156b40 0x7fcce4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcce415d110 0x7fcce4142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.473+0000 7fccea9f7640 1 --2- 172.21.15.106:0/2531807129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcce406ebf0 0x7fcce4140100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.474+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 >> 172.21.15.106:0/2531807129 conn(0x7fcce40cd6a0 msgr2=0x7fcce40adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.474+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 shutdown_connections 2024-04-13T03:10:24.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.474+0000 7fccea9f7640 1 -- 172.21.15.106:0/2531807129 wait complete. 2024-04-13T03:10:24.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T03:10:24.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.588+0000 7f94ae420640 1 Processor -- start 2024-04-13T03:10:24.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.588+0000 7f94ae420640 1 -- start start 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ae420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94a80c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ae420640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80d21d0 con 0x7f94a80cd260 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ae420640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80b1f40 con 0x7f94a80d1c30 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ae420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80cd8b0 con 0x7f94a815c5f0 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ac9bd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f94a80cd260 0x7f94a80b2d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53710/0 (socket says 172.21.15.106:53710) 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94ac9bd640 1 -- 172.21.15.106:0/541433916 learned_addr learned my addr 172.21.15.106:0/541433916 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94a7fff640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94a80c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924331174 0 0) 0x7f94a80d21d0 con 0x7f94a80cd260 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.589+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9490003520 con 0x7f94a80cd260 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3912616825 0 0) 0x7f9490003520 con 0x7f94a80cd260 2024-04-13T03:10:24.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 msgr2=0x7f94a80c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94a80c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 >> v1:172.21.15.114:6789/0 conn(0x7f94a80d1c30 legacy=0x7f94a80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a80b2220 con 0x7f94a80cd260 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f94a00019f0 con 0x7f94a80cd260 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 >> v1:172.21.15.106:6789/0 conn(0x7f94a80cd260 legacy=0x7f94a80b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94900039e0 0x7f9490003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9490004390 0x7f9490004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94900050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f949000b9c0 con 0x7f94900039e0 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f949000bb40 con 0x7f9490004390 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f949000bd40 con 0x7f94a815c5f0 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94900039e0 0x7f9490003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a77fe640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9490004390 0x7f9490004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94ac9bd640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94900050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 -- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 msgr2=0x7f94900050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a815c5f0 0x7f94900050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 -- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9490004390 msgr2=0x7f9490004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9490004390 0x7f9490004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.590+0000 7f94a7fff640 1 -- 172.21.15.106:0/541433916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9490003520 con 0x7f94900039e0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.591+0000 7f94a7fff640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94900039e0 0x7f9490003dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ac0 tx=0x7f9490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a804f2969993dc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.591+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498039070 con 0x7f94900039e0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.591+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949802c6b0 con 0x7f94900039e0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.591+0000 7f94a6ffd640 1 -- 172.21.15.106:0/541433916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498009690 con 0x7f94900039e0 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 -- 172.21.15.106:0/541433916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94900039e0 msgr2=0x7f9490003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 --2- 172.21.15.106:0/541433916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94900039e0 0x7f9490003dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ac0 tx=0x7f9490005830 comp rx=0 tx=0).stop 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ac9bd640 1 -- 172.21.15.106:0/541433916 reap_dead start 2024-04-13T03:10:24.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 -- 172.21.15.106:0/541433916 shutdown_connections 2024-04-13T03:10:24.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 -- 172.21.15.106:0/541433916 >> 172.21.15.106:0/541433916 conn(0x7f94a80cef80 msgr2=0x7f94a80cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 -- 172.21.15.106:0/541433916 shutdown_connections 2024-04-13T03:10:24.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.592+0000 7f94ae420640 1 -- 172.21.15.106:0/541433916 wait complete. 2024-04-13T03:10:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.593+0000 7f94ae420640 1 Processor -- start 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.593+0000 7f94ae420640 1 -- start start 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94900039e0 0x7f94a815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 0x7f94a81522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815c9d0 con 0x7f94a81f5c00 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80b2870 con 0x7f94900039e0 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f94a815cb50 con 0x7f94a81f3290 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59434/0 (socket says 172.21.15.106:59434) 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 -- 172.21.15.106:0/3701465887 learned_addr learned my addr 172.21.15.106:0/3701465887 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a7fff640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94900039e0 0x7f94a815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a77fe640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 0x7f94a81522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 -- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 msgr2=0x7f94a81522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 0x7f94a81522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 -- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94900039e0 msgr2=0x7f94a815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94900039e0 0x7f94a815e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 -- 172.21.15.106:0/3701465887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81f6550 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a77fe640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 0x7f94a81522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ac9bd640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b4ff0 tx=0x7f94a0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0017020 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a000ab60 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00133f0 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.594+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81f6830 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.595+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81e5e10 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.596+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a81e6010 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.596+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94a001a020 con 0x7f94a81f5c00 2024-04-13T03:10:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.596+0000 7f94a4ff9640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 0x7f949406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.596+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f94a008c5f0 con 0x7f94a81f5c00 2024-04-13T03:10:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.596+0000 7f94a7fff640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 0x7f949406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.597+0000 7f94a7fff640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 0x7f949406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ac0 tx=0x7f9498030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.601+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94a81e6010 con 0x7f94a81f5c00 2024-04-13T03:10:24.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.774+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f94a80c6ff0 con 0x7f94a81f5c00 2024-04-13T03:10:24.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.774+0000 7f94a4ff9640 1 -- 172.21.15.106:0/3701465887 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f94a0021020 con 0x7f94a81f5c00 2024-04-13T03:10:24.776 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.776+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 msgr2=0x7f949406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.776+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 0x7f949406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ac0 tx=0x7f9498030000 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.776+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 msgr2=0x7f94a815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.776+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b4ff0 tx=0x7f94a0007a90 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 shutdown_connections 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f949406c920 0x7f949406ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94a81f3290 0x7f94a81522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94a81f5c00 0x7f94a815adc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 --2- 172.21.15.106:0/3701465887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94900039e0 0x7f94a815e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 >> 172.21.15.106:0/3701465887 conn(0x7f94a80cef80 msgr2=0x7f94a80d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 shutdown_connections 2024-04-13T03:10:24.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.777+0000 7f94ae420640 1 -- 172.21.15.106:0/3701465887 wait complete. 2024-04-13T03:10:24.789 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:24.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.888+0000 7f756e41c640 1 Processor -- start 2024-04-13T03:10:24.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.888+0000 7f756e41c640 1 -- start start 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756e41c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f75680ce580 0x7f75680d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756e41c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75680b5b10 0x7f756815e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756e41c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75680d0b40 con 0x7f75680d06c0 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756e41c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f75680d1be0 con 0x7f75680b5b10 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756e41c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f75680d0e20 con 0x7f75680ce580 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75680b5b10 0x7f756815e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75680b5b10 0x7f756815e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59440/0 (socket says 172.21.15.106:59440) 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f75677fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f75680d06c0 0x7f75680b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53714/0 (socket says 172.21.15.106:53714) 2024-04-13T03:10:24.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 -- 172.21.15.106:0/3589675144 learned_addr learned my addr 172.21.15.106:0/3589675144 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f7567fff640 1 --2- 172.21.15.106:0/3589675144 >> v2:172.21.15.114:3300/0 conn(0x7f75680ce580 0x7f75680d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 -- 172.21.15.106:0/3589675144 >> v2:172.21.15.114:3300/0 conn(0x7f75680ce580 msgr2=0x7f75680d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 --2- 172.21.15.106:0/3589675144 >> v2:172.21.15.114:3300/0 conn(0x7f75680ce580 0x7f75680d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 -- 172.21.15.106:0/3589675144 >> v1:172.21.15.106:6789/0 conn(0x7f75680d06c0 legacy=0x7f75680b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f756c9b9640 1 -- 172.21.15.106:0/3589675144 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75680d1100 con 0x7f75680b5b10 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.889+0000 7f7566ffd640 1 -- 172.21.15.106:0/3589675144 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716816938 0 0) 0x7f75680d0b40 con 0x7f75680d06c0 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.890+0000 7f756c9b9640 1 --2- 172.21.15.106:0/3589675144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f756815e220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f75680d0080 tx=0x7f756002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de443d80ab363ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.890+0000 7f7566ffd640 1 -- 172.21.15.106:0/3589675144 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756002cb20 con 0x7f75680b5b10 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.890+0000 7f7566ffd640 1 -- 172.21.15.106:0/3589675144 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756002ccc0 con 0x7f75680b5b10 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.890+0000 7f7566ffd640 1 -- 172.21.15.106:0/3589675144 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560034610 con 0x7f75680b5b10 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.890+0000 7f756e41c640 1 -- 172.21.15.106:0/3589675144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 msgr2=0x7f756815e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 --2- 172.21.15.106:0/3589675144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f756815e220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f75680d0080 tx=0x7f756002bf80 comp rx=0 tx=0).stop 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 -- 172.21.15.106:0/3589675144 shutdown_connections 2024-04-13T03:10:24.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 --2- 172.21.15.106:0/3589675144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f756815e220 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 --2- 172.21.15.106:0/3589675144 >> v2:172.21.15.114:3300/0 conn(0x7f75680ce580 0x7f75680d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 -- 172.21.15.106:0/3589675144 >> 172.21.15.106:0/3589675144 conn(0x7f75680d25b0 msgr2=0x7f75680d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:24.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 -- 172.21.15.106:0/3589675144 shutdown_connections 2024-04-13T03:10:24.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.891+0000 7f756e41c640 1 -- 172.21.15.106:0/3589675144 wait complete. 2024-04-13T03:10:24.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.892+0000 7f756e41c640 1 Processor -- start 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.892+0000 7f756e41c640 1 -- start start 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75680ce580 0x7f756806ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75680d06c0 0x7f75680ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75680ce960 con 0x7f75680b5b10 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f75680d16c0 con 0x7f75680d06c0 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f75680b7890 con 0x7f75680ce580 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59442/0 (socket says 172.21.15.106:59442) 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 -- 172.21.15.106:0/1005849226 learned_addr learned my addr 172.21.15.106:0/1005849226 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f75677fe640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75680ce580 0x7f756806ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756c9b9640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75680d06c0 0x7f75680ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 -- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75680ce580 msgr2=0x7f756806ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75680ce580 0x7f756806ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 -- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75680d06c0 msgr2=0x7f75680ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75680d06c0 0x7f75680ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 -- 172.21.15.106:0/1005849226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756806f2c0 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7567fff640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f755800d800 tx=0x7f755800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755800b6c0 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7558019070 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755800ba80 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756806f600 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.893+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75681f7e20 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.894+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7568074750 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.895+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f755800bc20 con 0x7f75680b5b10 2024-04-13T03:10:24.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.895+0000 7f7564ff9640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 0x7f755406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:24.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.895+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f755808c440 con 0x7f75680b5b10 2024-04-13T03:10:24.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.895+0000 7f75677fe640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 0x7f755406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:24.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.896+0000 7f75677fe640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 0x7f755406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f756806e640 tx=0x7f755c007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:24.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:24.899+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f756806f600 con 0x7f75680b5b10 2024-04-13T03:10:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.112+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75680c9e90 con 0x7f75680b5b10 2024-04-13T03:10:25.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.113+0000 7f7564ff9640 1 -- 172.21.15.106:0/1005849226 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f755801c360 con 0x7f75680b5b10 2024-04-13T03:10:25.114 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:25.114 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:10:25.115 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:10:25.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 msgr2=0x7f755406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 0x7f755406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f756806e640 tx=0x7f755c007c00 comp rx=0 tx=0).stop 2024-04-13T03:10:25.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 msgr2=0x7f7568070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f755800d800 tx=0x7f755800dcc0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 shutdown_connections 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f755406c8f0 0x7f755406edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75680ce580 0x7f756806ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75680d06c0 0x7f75680ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 --2- 172.21.15.106:0/1005849226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75680b5b10 0x7f7568070e80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 >> 172.21.15.106:0/1005849226 conn(0x7f75680d25b0 msgr2=0x7f75680cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 shutdown_connections 2024-04-13T03:10:25.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.115+0000 7f756e41c640 1 -- 172.21.15.106:0/1005849226 wait complete. 2024-04-13T03:10:25.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:25.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.227+0000 7f5194915640 1 Processor -- start 2024-04-13T03:10:25.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.227+0000 7f5194915640 1 -- start start 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f5194915640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f51901622d0 0x7f51900d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f5194915640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f51900b8080 0x7f51901688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f5194915640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51900d3530 con 0x7f51900d1c00 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f5194915640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f51901626b0 con 0x7f51901622d0 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f5194915640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f51900d0b60 con 0x7f51900b8080 2024-04-13T03:10:25.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f51901622d0 0x7f51900d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f51901622d0 0x7f51900d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59456/0 (socket says 172.21.15.106:59456) 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518dd9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f51900d1c00 0x7f51900b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53730/0 (socket says 172.21.15.106:53730) 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 -- 172.21.15.106:0/2178827986 learned_addr learned my addr 172.21.15.106:0/2178827986 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 -- 172.21.15.106:0/2178827986 >> v2:172.21.15.114:3300/0 conn(0x7f51900b8080 msgr2=0x7f51901688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518ed9d640 1 --2- 172.21.15.106:0/2178827986 >> v2:172.21.15.114:3300/0 conn(0x7f51900b8080 0x7f51901688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 --2- 172.21.15.106:0/2178827986 >> v2:172.21.15.114:3300/0 conn(0x7f51900b8080 0x7f51901688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518d59a640 1 -- 172.21.15.106:0/2178827986 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599996923 0 0) 0x7f51900d3530 con 0x7f51900d1c00 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 -- 172.21.15.106:0/2178827986 >> v1:172.21.15.106:6789/0 conn(0x7f51900d1c00 legacy=0x7f51900b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.228+0000 7f518e59c640 1 -- 172.21.15.106:0/2178827986 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51900d0e40 con 0x7f51901622d0 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f518e59c640 1 --2- 172.21.15.106:0/2178827986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51901622d0 0x7f51900d04d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f51900ceef0 tx=0x7f518002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ff25b32e0d2c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f518d59a640 1 -- 172.21.15.106:0/2178827986 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180034070 con 0x7f51901622d0 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f518d59a640 1 -- 172.21.15.106:0/2178827986 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180030860 con 0x7f51901622d0 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f518d59a640 1 -- 172.21.15.106:0/2178827986 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180030c30 con 0x7f51901622d0 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f5194915640 1 -- 172.21.15.106:0/2178827986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51901622d0 msgr2=0x7f51900d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f5194915640 1 --2- 172.21.15.106:0/2178827986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51901622d0 0x7f51900d04d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f51900ceef0 tx=0x7f518002b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.229+0000 7f5194915640 1 -- 172.21.15.106:0/2178827986 shutdown_connections 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.230+0000 7f5194915640 1 --2- 172.21.15.106:0/2178827986 >> v2:172.21.15.114:3300/0 conn(0x7f51900b8080 0x7f51901688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.230+0000 7f5194915640 1 --2- 172.21.15.106:0/2178827986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51901622d0 0x7f51900d04d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.230+0000 7f5194915640 1 -- 172.21.15.106:0/2178827986 >> 172.21.15.106:0/2178827986 conn(0x7f51900d25d0 msgr2=0x7f51900d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.230+0000 7f5194915640 1 -- 172.21.15.106:0/2178827986 shutdown_connections 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.230+0000 7f5194915640 1 -- 172.21.15.106:0/2178827986 wait complete. 2024-04-13T03:10:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.231+0000 7f5194915640 1 Processor -- start 2024-04-13T03:10:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.231+0000 7f5194915640 1 -- start start 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51900b8080 0x7f519014a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 0x7f5190155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51900d16e0 con 0x7f51900d1c00 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51900d1400 con 0x7f51901622d0 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f51900cc6f0 con 0x7f51900b8080 2024-04-13T03:10:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59460/0 (socket says 172.21.15.106:59460) 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 -- 172.21.15.106:0/171472481 learned_addr learned my addr 172.21.15.106:0/171472481 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518ed9d640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 0x7f5190155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518e59c640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51900b8080 0x7f519014a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 -- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51900b8080 msgr2=0x7f519014a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51900b8080 0x7f519014a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 -- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 msgr2=0x7f5190155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 0x7f5190155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 -- 172.21.15.106:0/171472481 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519014ae00 con 0x7f51900d1c00 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518ed9d640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 0x7f5190155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f518dd9b640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f517800c500 tx=0x7f517800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178018070 con 0x7f51900d1c00 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178007580 con 0x7f51900d1c00 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178004a20 con 0x7f51900d1c00 2024-04-13T03:10:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5190158350 con 0x7f51900d1c00 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.232+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51900c7bd0 con 0x7f51900d1c00 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.233+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5190068a10 con 0x7f51900d1c00 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.234+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f517801b070 con 0x7f51900d1c00 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.234+0000 7f5186ffd640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 0x7f517006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.234+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f517808b190 con 0x7f51900d1c00 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.234+0000 7f518e59c640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 0x7f517006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.235+0000 7f518e59c640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 0x7f517006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5180036000 tx=0x7f518003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.239+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5178059250 con 0x7f51900d1c00 2024-04-13T03:10:25.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.416+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f519015b0d0 con 0x7f51900d1c00 2024-04-13T03:10:25.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.417+0000 7f5186ffd640 1 -- 172.21.15.106:0/171472481 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f5178058b50 con 0x7f51900d1c00 2024-04-13T03:10:25.419 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:25.419 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:25.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 msgr2=0x7f517006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 0x7f517006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5180036000 tx=0x7f518003b040 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 msgr2=0x7f5190154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f517800c500 tx=0x7f517800c9c0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 shutdown_connections 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f517006c8f0 0x7f517006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51901622d0 0x7f5190155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51900d1c00 0x7f5190154fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 --2- 172.21.15.106:0/171472481 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f51900b8080 0x7f519014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 >> 172.21.15.106:0/171472481 conn(0x7f51900d25d0 msgr2=0x7f51900ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.419+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 shutdown_connections 2024-04-13T03:10:25.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.420+0000 7f5194915640 1 -- 172.21.15.106:0/171472481 wait complete. 2024-04-13T03:10:25.432 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:10:25.432 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:25.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.530+0000 7f06bf311640 1 Processor -- start 2024-04-13T03:10:25.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.530+0000 7f06bf311640 1 -- start start 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bf311640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06b80ce1f0 0x7f06b8162db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bf311640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b80ce6f0 con 0x7f06b815ebf0 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bf311640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b815f420 con 0x7f06b806fe90 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bf311640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f06b815fa70 con 0x7f06b80ce1f0 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bd0ad640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f06b815ebf0 0x7f06b806f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53740/0 (socket says 172.21.15.106:53740) 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.531+0000 7f06bd0ad640 1 -- 172.21.15.106:0/1121822661 learned_addr learned my addr 172.21.15.106:0/1121822661 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911506875 0 0) 0x7f06b80ce6f0 con 0x7f06b815ebf0 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1121822661 >> v2:172.21.15.114:3300/0 conn(0x7f06b80ce1f0 0x7f06b8162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a4003520 con 0x7f06b815ebf0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188712680 0 0) 0x7f06a4003520 con 0x7f06b815ebf0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 >> v2:172.21.15.114:3300/0 conn(0x7f06b80ce1f0 msgr2=0x7f06b8162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 --2- 172.21.15.106:0/1121822661 >> v2:172.21.15.114:3300/0 conn(0x7f06b80ce1f0 0x7f06b8162db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 >> v1:172.21.15.114:6789/0 conn(0x7f06b806fe90 legacy=0x7f06b81626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b815f700 con 0x7f06b815ebf0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f06b40019f0 con 0x7f06b815ebf0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 >> v1:172.21.15.106:6789/0 conn(0x7f06b815ebf0 legacy=0x7f06b806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 0x7f06a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06b815fa70 con 0x7f06a40039b0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06a400c060 con 0x7f06a4004800 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06a400c260 con 0x7f06a40051c0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 0x7f06a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bc8ac640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 -- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40051c0 msgr2=0x7f06a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 -- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a4004800 msgr2=0x7f06a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd0ad640 1 -- 172.21.15.106:0/1121822661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4003520 con 0x7f06a40039b0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bc8ac640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.532+0000 7f06bd8ae640 1 -- 172.21.15.106:0/1121822661 reap_dead start 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.533+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 0x7f06a4003dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06b400a840 tx=0x7f06a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966cf95c3f1cd015 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.533+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400a460 con 0x7f06a40039b0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.533+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4034650 con 0x7f06a40039b0 2024-04-13T03:10:25.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.533+0000 7f06affff640 1 -- 172.21.15.106:0/1121822661 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4034a80 con 0x7f06a40039b0 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 -- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 msgr2=0x7f06a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 0x7f06a4003dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06b400a840 tx=0x7f06a4005830 comp rx=0 tx=0).stop 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 -- 172.21.15.106:0/1121822661 shutdown_connections 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 --2- 172.21.15.106:0/1121822661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06a40039b0 0x7f06a4003dd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 -- 172.21.15.106:0/1121822661 >> 172.21.15.106:0/1121822661 conn(0x7f06b80cd6d0 msgr2=0x7f06b80cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 -- 172.21.15.106:0/1121822661 shutdown_connections 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.534+0000 7f06bf311640 1 -- 172.21.15.106:0/1121822661 wait complete. 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.535+0000 7f06bf311640 1 Processor -- start 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.535+0000 7f06bf311640 1 -- start start 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a40039b0 0x7f06b8150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06b806ba80 0x7f06b81f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06b80b0650 con 0x7f06b8160110 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06b80b04d0 con 0x7f06a40039b0 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06b815efd0 con 0x7f06b806ba80 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59490/0 (socket says 172.21.15.106:59490) 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 -- 172.21.15.106:0/1655140829 learned_addr learned my addr 172.21.15.106:0/1655140829 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bc8ac640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06b806ba80 0x7f06b81f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a40039b0 0x7f06b8150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 -- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06b806ba80 msgr2=0x7f06b81f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06b806ba80 0x7f06b81f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 -- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a40039b0 msgr2=0x7f06b8150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a40039b0 0x7f06b8150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 -- 172.21.15.106:0/1655140829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b81f5760 con 0x7f06b8160110 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bd8ae640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ce620 tx=0x7f06b0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0019020 con 0x7f06b8160110 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b0003610 con 0x7f06b8160110 2024-04-13T03:10:25.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b806bff0 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.536+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0003a00 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.537+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b806c5b0 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.538+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b8068a10 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.538+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06b001c020 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.538+0000 7f06adffb640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 0x7f069c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.539+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f06b008bfc0 con 0x7f06b8160110 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.539+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 0x7f069c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.539+0000 7f06bd0ad640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 0x7f069c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034000 tx=0x7f06b4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.543+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06b0023030 con 0x7f06b8160110 2024-04-13T03:10:25.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.755+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b80cdd00 con 0x7f06b8160110 2024-04-13T03:10:25.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.755+0000 7f06adffb640 1 -- 172.21.15.106:0/1655140829 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f06b81f5760 con 0x7f06b8160110 2024-04-13T03:10:25.756 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:25.757 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:10:25.757 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 msgr2=0x7f069c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 0x7f069c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034000 tx=0x7f06b4009030 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 msgr2=0x7f06b815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ce620 tx=0x7f06b0011920 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 shutdown_connections 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f069c06c8f0 0x7f069c06edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06b806ba80 0x7f06b81f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06b8160110 0x7f06b815d800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 --2- 172.21.15.106:0/1655140829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06a40039b0 0x7f06b8150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 >> 172.21.15.106:0/1655140829 conn(0x7f06b80cd6d0 msgr2=0x7f06b8159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 shutdown_connections 2024-04-13T03:10:25.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.757+0000 7f06bf311640 1 -- 172.21.15.106:0/1655140829 wait complete. 2024-04-13T03:10:25.769 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:25.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.868+0000 7f6e7b80f640 1 Processor -- start 2024-04-13T03:10:25.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 -- start start 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e7415ec20 0x7f6e740b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e740d14d0 0x7f6e740b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e740b6120 con 0x7f6e740ceb70 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6e740cde50 con 0x7f6e7415ec20 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e7b80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e740cd240 con 0x7f6e740d14d0 2024-04-13T03:10:25.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e795ab640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e7415ec20 0x7f6e740b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e795ab640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e7415ec20 0x7f6e740b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59492/0 (socket says 172.21.15.106:59492) 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.869+0000 7f6e795ab640 1 -- 172.21.15.106:0/4112540866 learned_addr learned my addr 172.21.15.106:0/4112540866 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e79dac640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e740d14d0 0x7f6e740b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e795ab640 1 -- 172.21.15.106:0/4112540866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e740d14d0 msgr2=0x7f6e740b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e795ab640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e740d14d0 0x7f6e740b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e795ab640 1 -- 172.21.15.106:0/4112540866 >> v1:172.21.15.106:6789/0 conn(0x7f6e740ceb70 legacy=0x7f6e740b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e795ab640 1 -- 172.21.15.106:0/4112540866 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e740cd520 con 0x7f6e7415ec20 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.870+0000 7f6e795ab640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e7415ec20 0x7f6e740b53c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009e50 tx=0x7f6e6c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef97f1f6047d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e6bfff640 1 -- 172.21.15.106:0/4112540866 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c02cb00 con 0x7f6e7415ec20 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e6bfff640 1 -- 172.21.15.106:0/4112540866 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6c02cca0 con 0x7f6e7415ec20 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e6bfff640 1 -- 172.21.15.106:0/4112540866 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c034610 con 0x7f6e7415ec20 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 -- 172.21.15.106:0/4112540866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e7415ec20 msgr2=0x7f6e740b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e7415ec20 0x7f6e740b53c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009e50 tx=0x7f6e6c02bf60 comp rx=0 tx=0).stop 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 -- 172.21.15.106:0/4112540866 shutdown_connections 2024-04-13T03:10:25.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e740d14d0 0x7f6e740b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/4112540866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e7415ec20 0x7f6e740b53c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 -- 172.21.15.106:0/4112540866 >> 172.21.15.106:0/4112540866 conn(0x7f6e740d2530 msgr2=0x7f6e740d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:25.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 -- 172.21.15.106:0/4112540866 shutdown_connections 2024-04-13T03:10:25.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.871+0000 7f6e7b80f640 1 -- 172.21.15.106:0/4112540866 wait complete. 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.872+0000 7f6e7b80f640 1 Processor -- start 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.872+0000 7f6e7b80f640 1 -- start start 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e740d14d0 0x7f6e741619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e7415ec20 0x7f6e740c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e740cced0 con 0x7f6e740ca010 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e740cdae0 con 0x7f6e740d14d0 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e7b80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e741626b0 con 0x7f6e7415ec20 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e79dac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e79dac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59506/0 (socket says 172.21.15.106:59506) 2024-04-13T03:10:25.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e79dac640 1 -- 172.21.15.106:0/3567683537 learned_addr learned my addr 172.21.15.106:0/3567683537 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e795ab640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e740d14d0 0x7f6e741619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.873+0000 7f6e78daa640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e7415ec20 0x7f6e740c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 -- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e7415ec20 msgr2=0x7f6e740c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e7415ec20 0x7f6e740c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 -- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e740d14d0 msgr2=0x7f6e741619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e740d14d0 0x7f6e741619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 -- 172.21.15.106:0/3567683537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e74161f30 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e79dac640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c7f0 tx=0x7f6e7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70015070 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70004220 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007620 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e740cbb20 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.874+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e740cc0e0 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.875+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e740cc2e0 con 0x7f6e740ca010 2024-04-13T03:10:25.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.875+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e700079d0 con 0x7f6e740ca010 2024-04-13T03:10:25.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.876+0000 7f6e69ffb640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 0x7f6e5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:25.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.876+0000 7f6e795ab640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 0x7f6e5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:25.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.876+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6e7008be70 con 0x7f6e740ca010 2024-04-13T03:10:25.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.876+0000 7f6e795ab640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 0x7f6e5806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c035000 tx=0x7f6e6c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:25.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:25.880+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e740cc2e0 con 0x7f6e740ca010 2024-04-13T03:10:26.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.091+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e74068a10 con 0x7f6e740ca010 2024-04-13T03:10:26.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.091+0000 7f6e69ffb640 1 -- 172.21.15.106:0/3567683537 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f6e700598e0 con 0x7f6e740ca010 2024-04-13T03:10:26.093 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:26.093 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/2386858765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2386858765},{"type":"v1","addr":"172.21.15.114:6833","nonce":2386858765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3513524984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3513524984},{"type":"v1","addr":"172.21.15.106:6836","nonce":3513524984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4155755603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4155755603},{"type":"v1","addr":"172.21.15.106:6837","nonce":4155755603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:09:24.350589+0000","modified":"2024-04-13T03:09:46.396950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4025109601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4025109601},{"type":"v1","addr":"172.21.15.114:6835","nonce":4025109601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-13T03:10:26.093 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 msgr2=0x7f6e5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 0x7f6e5806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c035000 tx=0x7f6e6c030000 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 msgr2=0x7f6e741f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c7f0 tx=0x7f6e7000ccb0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 shutdown_connections 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e5806c8f0 0x7f6e5806edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e7415ec20 0x7f6e740c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e740ca010 0x7f6e741f4ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 --2- 172.21.15.106:0/3567683537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e740d14d0 0x7f6e741619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.093+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 >> 172.21.15.106:0/3567683537 conn(0x7f6e740d2530 msgr2=0x7f6e740b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.094+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 shutdown_connections 2024-04-13T03:10:26.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.094+0000 7f6e7b80f640 1 -- 172.21.15.106:0/3567683537 wait complete. 2024-04-13T03:10:26.105 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.204+0000 7f9c50b8c640 1 Processor -- start 2024-04-13T03:10:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.204+0000 7f9c50b8c640 1 -- start start 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c50b8c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9c4c0d4e00 0x7f9c4c169610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c50b8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4c0d24d0 con 0x7f9c4c15ead0 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c50b8c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4c15eeb0 con 0x7f9c4c0d1e50 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c50b8c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c0d1060 con 0x7f9c4c0d4e00 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c4a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9c4c15ead0 0x7f9c4c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53766/0 (socket says 172.21.15.106:53766) 2024-04-13T03:10:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.205+0000 7f9c4a59c640 1 -- 172.21.15.106:0/2362167819 learned_addr learned my addr 172.21.15.106:0/2362167819 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c4ad9d640 1 --2- 172.21.15.106:0/2362167819 >> v2:172.21.15.114:3300/0 conn(0x7f9c4c0d4e00 0x7f9c4c169610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084732377 0 0) 0x7f9c4c0d24d0 con 0x7f9c4c15ead0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c38003520 con 0x7f9c4c15ead0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3878190867 0 0) 0x7f9c38003520 con 0x7f9c4c15ead0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 >> v2:172.21.15.114:3300/0 conn(0x7f9c4c0d4e00 msgr2=0x7f9c4c169610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 --2- 172.21.15.106:0/2362167819 >> v2:172.21.15.114:3300/0 conn(0x7f9c4c0d4e00 0x7f9c4c169610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 >> v1:172.21.15.114:6789/0 conn(0x7f9c4c0d1e50 legacy=0x7f9c4c0d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c0d2770 con 0x7f9c4c15ead0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9c3c001670 con 0x7f9c4c15ead0 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 >> v1:172.21.15.106:6789/0 conn(0x7f9c4c15ead0 legacy=0x7f9c4c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c380039b0 0x7f9c38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 0x7f9c38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c380051c0 0x7f9c3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0d1060 con 0x7f9c38004800 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800c060 con 0x7f9c380051c0 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3800c260 con 0x7f9c380039b0 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 0x7f9c38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c4a59c640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c380039b0 0x7f9c38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c4ad9d640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c380051c0 0x7f9c3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 -- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c380039b0 msgr2=0x7f9c38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c380039b0 0x7f9c38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 -- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c380051c0 msgr2=0x7f9c3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c380051c0 0x7f9c3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c43fff640 1 -- 172.21.15.106:0/2362167819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c38003520 con 0x7f9c38004800 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.206+0000 7f9c4ad9d640 1 -- 172.21.15.106:0/2362167819 reap_dead start 2024-04-13T03:10:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.207+0000 7f9c43fff640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 0x7f9c38004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9c34001720 tx=0x7f9c38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6afa538107d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.207+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34001370 con 0x7f9c38004800 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.207+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c340052d0 con 0x7f9c38004800 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.207+0000 7f9c49d9b640 1 -- 172.21.15.106:0/2362167819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34003830 con 0x7f9c38004800 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 -- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 msgr2=0x7f9c38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 0x7f9c38004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9c34001720 tx=0x7f9c38005830 comp rx=0 tx=0).stop 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 -- 172.21.15.106:0/2362167819 shutdown_connections 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/2362167819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c38004800 0x7f9c38004c80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 -- 172.21.15.106:0/2362167819 >> 172.21.15.106:0/2362167819 conn(0x7f9c4c09da70 msgr2=0x7f9c4c0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 -- 172.21.15.106:0/2362167819 shutdown_connections 2024-04-13T03:10:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.208+0000 7f9c50b8c640 1 -- 172.21.15.106:0/2362167819 wait complete. 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.209+0000 7f9c50b8c640 1 Processor -- start 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.209+0000 7f9c50b8c640 1 -- start start 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c38004800 0x7f9c4c160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c4c0b41a0 0x7f9c4c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0b12f0 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0d2d30 con 0x7f9c38004800 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c50b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4c0b1470 con 0x7f9c4c0b41a0 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59530/0 (socket says 172.21.15.106:59530) 2024-04-13T03:10:26.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 -- 172.21.15.106:0/1476218840 learned_addr learned my addr 172.21.15.106:0/1476218840 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c4ad9d640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c4c0b41a0 0x7f9c4c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c4a59c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c38004800 0x7f9c4c160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 -- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c4c0b41a0 msgr2=0x7f9c4c159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c4c0b41a0 0x7f9c4c159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 -- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c38004800 msgr2=0x7f9c4c160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c38004800 0x7f9c4c160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 -- 172.21.15.106:0/1476218840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c1f8880 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.210+0000 7f9c43fff640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c34000c00 tx=0x7f9c34001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.211+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34041020 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.211+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c34005480 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.211+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c15ad50 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.211+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34007a30 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.211+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c15b310 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.212+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c068a10 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.212+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c34044020 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.212+0000 7f9c42ffd640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.213+0000 7f9c4a59c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.213+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9c340adab0 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.213+0000 7f9c4a59c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c06a8d0 tx=0x7f9c3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.217+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c3404b030 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.395+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c4c068060 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.396+0000 7f9c42ffd640 1 -- 172.21.15.106:0/1476218840 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f9c4c1f8880 con 0x7f9c4c15a8c0 2024-04-13T03:10:26.397 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:26.397 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:06.177339+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:09:22.361852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T03:09:23.267069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T03:09:26.991920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 msgr2=0x7f9c2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c06a8d0 tx=0x7f9c3c009040 comp rx=0 tx=0).stop 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 msgr2=0x7f9c4c1f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c34000c00 tx=0x7f9c34001120 comp rx=0 tx=0).stop 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 shutdown_connections 2024-04-13T03:10:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c4c0b41a0 0x7f9c4c159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c4c15a8c0 0x7f9c4c1f7cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 --2- 172.21.15.106:0/1476218840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c38004800 0x7f9c4c160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 >> 172.21.15.106:0/1476218840 conn(0x7f9c4c09da70 msgr2=0x7f9c4c0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 shutdown_connections 2024-04-13T03:10:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.398+0000 7f9c50b8c640 1 -- 172.21.15.106:0/1476218840 wait complete. 2024-04-13T03:10:26.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:10:26.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.511+0000 7fb3a3e36640 1 Processor -- start 2024-04-13T03:10:26.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.511+0000 7fb3a3e36640 1 -- start start 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a3e36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb39c0d09f0 0x7fb39c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a3e36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb39c06e280 con 0x7fb39c15e930 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a3e36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb39c06e560 con 0x7fb39c06ed70 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a3e36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb39c0cccf0 con 0x7fb39c0d09f0 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a1bd2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb39c15e930 0x7fb39c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53772/0 (socket says 172.21.15.106:53772) 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a1bd2640 1 -- 172.21.15.106:0/178163534 learned_addr learned my addr 172.21.15.106:0/178163534 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/178163534 >> v2:172.21.15.114:3300/0 conn(0x7fb39c0d09f0 0x7fb39c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287165237 0 0) 0x7fb39c06e280 con 0x7fb39c15e930 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb388003520 con 0x7fb39c15e930 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117064790 0 0) 0x7fb39c06e560 con 0x7fb39c06ed70 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb39c06e280 con 0x7fb39c06ed70 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1468007145 0 0) 0x7fb388003520 con 0x7fb39c15e930 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 >> v2:172.21.15.114:3300/0 conn(0x7fb39c0d09f0 msgr2=0x7fb39c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 --2- 172.21.15.106:0/178163534 >> v2:172.21.15.114:3300/0 conn(0x7fb39c0d09f0 0x7fb39c0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 >> v1:172.21.15.114:6789/0 conn(0x7fb39c06ed70 legacy=0x7fb39c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c0ccfd0 con 0x7fb39c15e930 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb398001670 con 0x7fb39c15e930 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 >> v1:172.21.15.106:6789/0 conn(0x7fb39c15e930 legacy=0x7fb39c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388003b40 0x7fb388003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.512+0000 7fb3a0bd0640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3880051c0 0x7fb38800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0cccf0 con 0x7fb388004800 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb38800c0f0 con 0x7fb388003b40 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb38800c2f0 con 0x7fb3880051c0 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a1bd2640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388003b40 0x7fb388003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a23d3640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3880051c0 0x7fb38800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 -- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3880051c0 msgr2=0x7fb38800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3880051c0 0x7fb38800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 -- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388003b40 msgr2=0x7fb388003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388003b40 0x7fb388003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a13d1640 1 -- 172.21.15.106:0/178163534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb388003520 con 0x7fb388004800 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.513+0000 7fb3a23d3640 1 -- 172.21.15.106:0/178163534 reap_dead start 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 0x7fb388004c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009ac0 tx=0x7fb388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46a2d00dbbc0013 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c039070 con 0x7fb388004800 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c02c6b0 con 0x7fb388004800 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a0bd0640 1 -- 172.21.15.106:0/178163534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c009690 con 0x7fb388004800 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 -- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 msgr2=0x7fb388004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 0x7fb388004c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009ac0 tx=0x7fb388005830 comp rx=0 tx=0).stop 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 -- 172.21.15.106:0/178163534 shutdown_connections 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/178163534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 -- 172.21.15.106:0/178163534 >> 172.21.15.106:0/178163534 conn(0x7fb39c0cef80 msgr2=0x7fb39c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 -- 172.21.15.106:0/178163534 shutdown_connections 2024-04-13T03:10:26.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.514+0000 7fb3a3e36640 1 -- 172.21.15.106:0/178163534 wait complete. 2024-04-13T03:10:26.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 Processor -- start 2024-04-13T03:10:26.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 -- start start 2024-04-13T03:10:26.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 0x7fb39c1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 0x7fb39c15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0ce830 con 0x7fb39c0c9780 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c06f1a0 con 0x7fb388004800 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a3e36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb39c0ce9b0 con 0x7fb39c0c71b0 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a13d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a13d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59542/0 (socket says 172.21.15.106:59542) 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.516+0000 7fb3a13d1640 1 -- 172.21.15.106:0/1050327197 learned_addr learned my addr 172.21.15.106:0/1050327197 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a1bd2640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 0x7fb39c1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a23d3640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 0x7fb39c15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 -- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 msgr2=0x7fb39c15ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 0x7fb39c15ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 -- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 msgr2=0x7fb39c1f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 0x7fb39c1f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 -- 172.21.15.106:0/1050327197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c1f4df0 con 0x7fb39c0c9780 2024-04-13T03:10:26.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a23d3640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 0x7fb39c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a1bd2640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 0x7fb39c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a13d1640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c042020 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c03a040 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c039070 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c1f50d0 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.517+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1f5690 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.518+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c068a10 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.518+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb38c045020 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.519+0000 7fb3927fc640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.519+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb38c0ad320 con 0x7fb39c0c9780 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.519+0000 7fb3a1bd2640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.519+0000 7fb3a1bd2640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 0x7fb38006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b5260 tx=0x7fb398009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.523+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb38c04c070 con 0x7fb39c0c9780 2024-04-13T03:10:26.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.734+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb39c06d630 con 0x7fb39c0c9780 2024-04-13T03:10:26.735 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:10:26.734+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:10:26.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.738+0000 7fb3927fc640 1 -- 172.21.15.106:0/1050327197 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fb38c07b350 con 0x7fb39c0c9780 2024-04-13T03:10:26.739 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:10:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 msgr2=0x7fb38006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 0x7fb38006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b5260 tx=0x7fb398009040 comp rx=0 tx=0).stop 2024-04-13T03:10:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 msgr2=0x7fb39c14d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c004730 comp rx=0 tx=0).stop 2024-04-13T03:10:26.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 shutdown_connections 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb39c0c71b0 0x7fb39c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb39c0c9780 0x7fb39c14d8e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 --2- 172.21.15.106:0/1050327197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb388004800 0x7fb39c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 >> 172.21.15.106:0/1050327197 conn(0x7fb39c0cef80 msgr2=0x7fb39c0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 shutdown_connections 2024-04-13T03:10:26.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.740+0000 7fb3a3e36640 1 -- 172.21.15.106:0/1050327197 wait complete. 2024-04-13T03:10:26.743 INFO:tasks.ceph.mds.d.smithi114.stderr: -6649> 2024-04-13T03:09:17.947+0000 7f977e7fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-04-13T03:10:26.745 INFO:tasks.ceph.mds.d.smithi114.stderr: -5440> 2024-04-13T03:09:28.950+0000 7f97787f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:10:26.753 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:10:26.776 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:10:26.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.854+0000 7f793c436640 1 Processor -- start 2024-04-13T03:10:26.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.854+0000 7f793c436640 1 -- start start 2024-04-13T03:10:26.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793c436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 0x7f79340b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793c436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7934072e10 0x7f79340732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793c436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7934159600 con 0x7f793415bbc0 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793c436640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f793415c6e0 con 0x7f7934072e10 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793c436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f79340b4690 con 0x7f7934159220 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f79399d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7934072e10 0x7f79340732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793a9d3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f793415bbc0 0x7f793415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53778/0 (socket says 172.21.15.106:53778) 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.855+0000 7f793a9d3640 1 -- 172.21.15.106:0/1371279554 learned_addr learned my addr 172.21.15.106:0/1371279554 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f793a1d2640 1 --2- 172.21.15.106:0/1371279554 >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 0x7f79340b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f79399d1640 1 -- 172.21.15.106:0/1371279554 >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 msgr2=0x7f79340b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f79399d1640 1 --2- 172.21.15.106:0/1371279554 >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 0x7f79340b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f79399d1640 1 -- 172.21.15.106:0/1371279554 >> v1:172.21.15.106:6789/0 conn(0x7f793415bbc0 legacy=0x7f793415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f79399d1640 1 -- 172.21.15.106:0/1371279554 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793415c960 con 0x7f7934072e10 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f79391d0640 1 -- 172.21.15.106:0/1371279554 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281703858 0 0) 0x7f792c001110 con 0x7f793415bbc0 2024-04-13T03:10:26.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.856+0000 7f793a1d2640 1 --2- 172.21.15.106:0/1371279554 >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 0x7f79340b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:26.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f79399d1640 1 --2- 172.21.15.106:0/1371279554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934072e10 0x7f79340732a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79340b7630 tx=0x7f792402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6618e826c2007f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f79391d0640 1 -- 172.21.15.106:0/1371279554 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924035020 con 0x7f7934072e10 2024-04-13T03:10:26.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f79391d0640 1 -- 172.21.15.106:0/1371279554 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7924031830 con 0x7f7934072e10 2024-04-13T03:10:26.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f79391d0640 1 -- 172.21.15.106:0/1371279554 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924031c00 con 0x7f7934072e10 2024-04-13T03:10:26.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 -- 172.21.15.106:0/1371279554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934072e10 msgr2=0x7f79340732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 --2- 172.21.15.106:0/1371279554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934072e10 0x7f79340732a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79340b7630 tx=0x7f792402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 -- 172.21.15.106:0/1371279554 shutdown_connections 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 --2- 172.21.15.106:0/1371279554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934072e10 0x7f79340732a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 --2- 172.21.15.106:0/1371279554 >> v2:172.21.15.114:3300/0 conn(0x7f7934159220 0x7f79340b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 -- 172.21.15.106:0/1371279554 >> 172.21.15.106:0/1371279554 conn(0x7f793406f180 msgr2=0x7f793406f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 -- 172.21.15.106:0/1371279554 shutdown_connections 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.857+0000 7f793c436640 1 -- 172.21.15.106:0/1371279554 wait complete. 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.858+0000 7f793c436640 1 Processor -- start 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.858+0000 7f793c436640 1 -- start start 2024-04-13T03:10:26.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7934072e10 0x7f79341e5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793415bbc0 0x7f79340a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79340cf400 con 0x7f7934159220 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79340b4c30 con 0x7f793415bbc0 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793c436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f79340cf580 con 0x7f7934072e10 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f79399d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f79399d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59562/0 (socket says 172.21.15.106:59562) 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f79399d1640 1 -- 172.21.15.106:0/4049191299 learned_addr learned my addr 172.21.15.106:0/4049191299 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793a1d2640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7934072e10 0x7f79341e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f793a9d3640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793415bbc0 0x7f79340a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.859+0000 7f79399d1640 1 -- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7934072e10 msgr2=0x7f79341e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f79399d1640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7934072e10 0x7f79341e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f79399d1640 1 -- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793415bbc0 msgr2=0x7f79340a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f79399d1640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793415bbc0 0x7f79340a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:26.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f79399d1640 1 -- 172.21.15.106:0/4049191299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79341f2590 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f79399d1640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7924009890 tx=0x7f79240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924041070 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792403aa30 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79341e6700 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924035040 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.860+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79341f2b30 con 0x7f7934159220 2024-04-13T03:10:26.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.861+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79341f23a0 con 0x7f7934159220 2024-04-13T03:10:26.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.861+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7924038020 con 0x7f7934159220 2024-04-13T03:10:26.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.862+0000 7f792affd640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:26.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.862+0000 7f793a1d2640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:26.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.862+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f79240ad950 con 0x7f7934159220 2024-04-13T03:10:26.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.862+0000 7f793a1d2640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7930003b80 tx=0x7f7930009890 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:26.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:26.866+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f792407b2d0 con 0x7f7934159220 2024-04-13T03:10:27.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.079+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7934069120 con 0x7f7934159220 2024-04-13T03:10:27.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.741+0000 7f792affd640 1 -- 172.21.15.106:0/4049191299 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f792407f5d0 con 0x7f7934159220 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 msgr2=0x7f791806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7930003b80 tx=0x7f7930009890 comp rx=0 tx=0).stop 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 msgr2=0x7f79341e5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7924009890 tx=0x7f79240046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 shutdown_connections 2024-04-13T03:10:27.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793415bbc0 0x7f79340a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7934159220 0x7f79341e5ed0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.743+0000 7f793c436640 1 --2- 172.21.15.106:0/4049191299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7934072e10 0x7f79341e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.744+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 >> 172.21.15.106:0/4049191299 conn(0x7f793406f180 msgr2=0x7f79340cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.744+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 shutdown_connections 2024-04-13T03:10:27.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.744+0000 7f793c436640 1 -- 172.21.15.106:0/4049191299 wait complete. 2024-04-13T03:10:27.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:10:27.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.861+0000 7fb64ae2d640 1 Processor -- start 2024-04-13T03:10:27.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.861+0000 7fb64ae2d640 1 -- start start 2024-04-13T03:10:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb64ae2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb644145470 0x7fb644145af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb64ae2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6440d11e0 con 0x7fb644070e40 2024-04-13T03:10:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb64ae2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6440d05d0 con 0x7fb644146030 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb64ae2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb6440d03a0 con 0x7fb644145470 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6493ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb644070e40 0x7fb64406c450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53788/0 (socket says 172.21.15.106:53788) 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6493ca640 1 -- 172.21.15.106:0/2583257333 learned_addr learned my addr 172.21.15.106:0/2583257333 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb648bc9640 1 --2- 172.21.15.106:0/2583257333 >> v2:172.21.15.114:3300/0 conn(0x7fb644145470 0x7fb644145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143586354 0 0) 0x7fb6440d11e0 con 0x7fb644070e40 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb62c003520 con 0x7fb644070e40 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 774206417 0 0) 0x7fb62c003520 con 0x7fb644070e40 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 >> v2:172.21.15.114:3300/0 conn(0x7fb644145470 msgr2=0x7fb644145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 --2- 172.21.15.106:0/2583257333 >> v2:172.21.15.114:3300/0 conn(0x7fb644145470 0x7fb644145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 >> v1:172.21.15.114:6789/0 conn(0x7fb644146030 legacy=0x7fb644070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6440d08b0 con 0x7fb644070e40 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.862+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb634001670 con 0x7fb644070e40 2024-04-13T03:10:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 >> v1:172.21.15.106:6789/0 conn(0x7fb644070e40 legacy=0x7fb64406c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb62c0039b0 0x7fb62c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0051c0 0x7fb62c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c00c030 con 0x7fb62c004800 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c00c1e0 con 0x7fb62c0051c0 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb62c00c3e0 con 0x7fb62c0039b0 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb648bc9640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb62c0039b0 0x7fb62c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6493ca640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0051c0 0x7fb62c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 -- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb62c0039b0 msgr2=0x7fb62c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb62c0039b0 0x7fb62c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 -- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0051c0 msgr2=0x7fb62c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0051c0 0x7fb62c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb6493ca640 1 -- 172.21.15.106:0/2583257333 reap_dead start 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.863+0000 7fb643fff640 1 -- 172.21.15.106:0/2583257333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c003520 con 0x7fb62c004800 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb643fff640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb638001750 tx=0x7fb62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4afa92f5e86670 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638001370 con 0x7fb62c004800 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6380052d0 con 0x7fb62c004800 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb6437fe640 1 -- 172.21.15.106:0/2583257333 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638003830 con 0x7fb62c004800 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb64ae2d640 1 -- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 msgr2=0x7fb62c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.864+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb638001750 tx=0x7fb62c005830 comp rx=0 tx=0).stop 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.865+0000 7fb64ae2d640 1 -- 172.21.15.106:0/2583257333 shutdown_connections 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.865+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/2583257333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.865+0000 7fb64ae2d640 1 -- 172.21.15.106:0/2583257333 >> 172.21.15.106:0/2583257333 conn(0x7fb64406f290 msgr2=0x7fb64406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.865+0000 7fb64ae2d640 1 -- 172.21.15.106:0/2583257333 shutdown_connections 2024-04-13T03:10:27.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.865+0000 7fb64ae2d640 1 -- 172.21.15.106:0/2583257333 wait complete. 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.866+0000 7fb64ae2d640 1 Processor -- start 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.866+0000 7fb64ae2d640 1 -- start start 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 0x7fb64415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64415bbb0 0x7fb6440aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb644145850 con 0x7fb62c004800 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440d0e70 con 0x7fb64415bbb0 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440cee30 con 0x7fb6441f30c0 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59592/0 (socket says 172.21.15.106:59592) 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 -- 172.21.15.106:0/4038375878 learned_addr learned my addr 172.21.15.106:0/4038375878 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb643fff640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 0x7fb64415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 -- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 msgr2=0x7fb64415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb6493ca640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64415bbb0 0x7fb6440aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 0x7fb64415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 -- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64415bbb0 msgr2=0x7fb6440aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64415bbb0 0x7fb6440aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 -- 172.21.15.106:0/4038375878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb644160030 con 0x7fb62c004800 2024-04-13T03:10:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb643fff640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 0x7fb64415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb648bc9640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb63c011e30 tx=0x7fb63c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c015630 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c0157d0 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb644160310 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c015ba0 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.867+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441608d0 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.868+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb644068a10 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.869+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63c01d400 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.869+0000 7fb6417fa640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 0x7fb63006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.869+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fb63c08d360 con 0x7fb62c004800 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.869+0000 7fb643fff640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 0x7fb63006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:27.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.870+0000 7fb643fff640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 0x7fb63006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb638033000 tx=0x7fb638009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:27.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:27.874+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb63c019030 con 0x7fb62c004800 2024-04-13T03:10:28.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.050+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb644070c00 con 0x7fb62c004800 2024-04-13T03:10:28.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.742+0000 7fb6417fa640 1 -- 172.21.15.106:0/4038375878 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fb63c010040 con 0x7fb62c004800 2024-04-13T03:10:28.744 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.746+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 msgr2=0x7fb63006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.746+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 0x7fb63006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb638033000 tx=0x7fb638009030 comp rx=0 tx=0).stop 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.746+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 msgr2=0x7fb64415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.746+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb63c011e30 tx=0x7fb63c00f580 comp rx=0 tx=0).stop 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 shutdown_connections 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb63006c8f0 0x7fb63006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6441f30c0 0x7fb64415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64415bbb0 0x7fb6440aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 --2- 172.21.15.106:0/4038375878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb62c004800 0x7fb64415f990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 >> 172.21.15.106:0/4038375878 conn(0x7fb64406f290 msgr2=0x7fb64406e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:28.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 shutdown_connections 2024-04-13T03:10:28.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.747+0000 7fb64ae2d640 1 -- 172.21.15.106:0/4038375878 wait complete. 2024-04-13T03:10:28.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:10:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.866+0000 7f994a15e640 1 Processor -- start 2024-04-13T03:10:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 -- start start 2024-04-13T03:10:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f994415c7c0 0x7f9944160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99440ccc00 0x7f99440cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 0x7f99440d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9944161220 con 0x7f99440ccc00 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f99440cd580 con 0x7f99440d0690 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f994a15e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f99440d1f20 con 0x7f994415c7c0 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.867+0000 7f9942ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99440ccc00 0x7f99440cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99440ccc00 0x7f99440cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59594/0 (socket says 172.21.15.106:59594) 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 -- 172.21.15.106:0/2719600068 learned_addr learned my addr 172.21.15.106:0/2719600068 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f99437fe640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f994415c7c0 0x7f9944160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9943fff640 1 --2- 172.21.15.106:0/2719600068 >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 0x7f99440d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 -- 172.21.15.106:0/2719600068 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f994415c7c0 msgr2=0x7f9944160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f994415c7c0 0x7f9944160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 -- 172.21.15.106:0/2719600068 >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 msgr2=0x7f99440d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2719600068 >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 0x7f99440d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9942ffd640 1 -- 172.21.15.106:0/2719600068 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440d2200 con 0x7f99440ccc00 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.868+0000 7f9943fff640 1 --2- 172.21.15.106:0/2719600068 >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 0x7f99440d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f99440cd040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99440714f0 tx=0x7f992c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6da65267d1bffe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f99427fc640 1 -- 172.21.15.106:0/2719600068 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c02e070 con 0x7f99440ccc00 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f99427fc640 1 -- 172.21.15.106:0/2719600068 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c037070 con 0x7f99440ccc00 2024-04-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f99427fc640 1 -- 172.21.15.106:0/2719600068 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c031a00 con 0x7f99440ccc00 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 -- 172.21.15.106:0/2719600068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 msgr2=0x7f99440cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f99440cd040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99440714f0 tx=0x7f992c02b880 comp rx=0 tx=0).stop 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 -- 172.21.15.106:0/2719600068 shutdown_connections 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 --2- 172.21.15.106:0/2719600068 >> v2:172.21.15.114:3300/0 conn(0x7f99440d0690 0x7f99440d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f99440cd040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 --2- 172.21.15.106:0/2719600068 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f994415c7c0 0x7f9944160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 -- 172.21.15.106:0/2719600068 >> 172.21.15.106:0/2719600068 conn(0x7f99440d25b0 msgr2=0x7f99440d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 -- 172.21.15.106:0/2719600068 shutdown_connections 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.869+0000 7f994a15e640 1 -- 172.21.15.106:0/2719600068 wait complete. 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.870+0000 7f994a15e640 1 Processor -- start 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 -- start start 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99440d0690 0x7f9944151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 0x7f994406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f994406fa50 con 0x7f99440ccc00 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f994406f770 con 0x7f994415c7c0 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f994a15e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f994415cba0 con 0x7f99440d0690 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f99437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f99437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59602/0 (socket says 172.21.15.106:59602) 2024-04-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.871+0000 7f99437fe640 1 -- 172.21.15.106:0/2863748956 learned_addr learned my addr 172.21.15.106:0/2863748956 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99440d0690 0x7f9944151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f9943fff640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 0x7f994406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 -- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99440d0690 msgr2=0x7f9944151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99440d0690 0x7f9944151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 -- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 msgr2=0x7f994406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 0x7f994406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 -- 172.21.15.106:0/2863748956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440c6ab0 con 0x7f99440ccc00 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f9943fff640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 0x7f994406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f99437fe640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f993400d830 tx=0x7f993400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993400b6c0 con 0x7f99440ccc00 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934019070 con 0x7f99440ccc00 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993400ba80 con 0x7f99440ccc00 2024-04-13T03:10:28.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994406e4e0 con 0x7f99440ccc00 2024-04-13T03:10:28.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.872+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994406eaa0 con 0x7f99440ccc00 2024-04-13T03:10:28.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.873+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944068a10 con 0x7f99440ccc00 2024-04-13T03:10:28.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.873+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f993401e460 con 0x7f99440ccc00 2024-04-13T03:10:28.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.874+0000 7f993bfff640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 0x7f992406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:28.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.874+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f993408c430 con 0x7f99440ccc00 2024-04-13T03:10:28.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.874+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 0x7f992406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:28.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.874+0000 7f9942ffd640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 0x7f992406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f992c036000 tx=0x7f992c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:28.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:28.878+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f993405a5c0 con 0x7f99440ccc00 2024-04-13T03:10:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.060+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99440d3310 con 0x7f99440ccc00 2024-04-13T03:10:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.748+0000 7f993bfff640 1 -- 172.21.15.106:0/2863748956 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f9934059ec0 con 0x7f99440ccc00 2024-04-13T03:10:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 msgr2=0x7f992406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 0x7f992406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f992c036000 tx=0x7f992c03d040 comp rx=0 tx=0).stop 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 msgr2=0x7f994415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f993400d830 tx=0x7f993400dcf0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 shutdown_connections 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f992406c8f0 0x7f992406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99440d0690 0x7f9944151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.754+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f994415c7c0 0x7f994406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.755+0000 7f994a15e640 1 --2- 172.21.15.106:0/2863748956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99440ccc00 0x7f994415f890 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.755+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 >> 172.21.15.106:0/2863748956 conn(0x7f99440d25b0 msgr2=0x7f99440b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:29.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.755+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 shutdown_connections 2024-04-13T03:10:29.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.755+0000 7f994a15e640 1 -- 172.21.15.106:0/2863748956 wait complete. 2024-04-13T03:10:29.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-13T03:10:29.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.876+0000 7f02cd9f7640 1 Processor -- start 2024-04-13T03:10:29.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.876+0000 7f02cd9f7640 1 -- start start 2024-04-13T03:10:29.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02cd9f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f02c80d2370 0x7f02c80b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02cd9f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f02c80b3920 0x7f02c815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02cd9f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c815c870 con 0x7f02c80d3490 2024-04-13T03:10:29.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02cd9f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f02c80d5420 con 0x7f02c80b3920 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02cd9f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f02c815ee70 con 0x7f02c80d2370 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f02c80b3920 0x7f02c815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f02c80d3490 0x7f02c80d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53794/0 (socket says 172.21.15.106:53794) 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c6ffd640 1 -- 172.21.15.106:0/997395389 learned_addr learned my addr 172.21.15.106:0/997395389 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c77fe640 1 -- 172.21.15.106:0/997395389 >> v2:172.21.15.114:3300/0 conn(0x7f02c80d2370 msgr2=0x7f02c80b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02bffff640 1 --2- 172.21.15.106:0/997395389 >> v2:172.21.15.114:3300/0 conn(0x7f02c80d2370 0x7f02c80b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c77fe640 1 --2- 172.21.15.106:0/997395389 >> v2:172.21.15.114:3300/0 conn(0x7f02c80d2370 0x7f02c80b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c77fe640 1 -- 172.21.15.106:0/997395389 >> v1:172.21.15.106:6789/0 conn(0x7f02c80d3490 legacy=0x7f02c80d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c67fc640 1 -- 172.21.15.106:0/997395389 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161592642 0 0) 0x7f02c815c870 con 0x7f02c80d3490 2024-04-13T03:10:29.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.877+0000 7f02c77fe640 1 -- 172.21.15.106:0/997395389 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c80d2800 con 0x7f02c80b3920 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02c77fe640 1 --2- 172.21.15.106:0/997395389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80b3920 0x7f02c815f360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02c806edc0 tx=0x7f02c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e7ecbdeb30679e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02c67fc640 1 -- 172.21.15.106:0/997395389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0039070 con 0x7f02c80b3920 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02c67fc640 1 -- 172.21.15.106:0/997395389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c002cbf0 con 0x7f02c80b3920 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02c67fc640 1 -- 172.21.15.106:0/997395389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0034440 con 0x7f02c80b3920 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02cd9f7640 1 -- 172.21.15.106:0/997395389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80b3920 msgr2=0x7f02c815f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.878+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/997395389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80b3920 0x7f02c815f360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02c806edc0 tx=0x7f02c002be40 comp rx=0 tx=0).stop 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 -- 172.21.15.106:0/997395389 shutdown_connections 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/997395389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80b3920 0x7f02c815f360 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/997395389 >> v2:172.21.15.114:3300/0 conn(0x7f02c80d2370 0x7f02c80b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 -- 172.21.15.106:0/997395389 >> 172.21.15.106:0/997395389 conn(0x7f02c80b9760 msgr2=0x7f02c80b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:29.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 -- 172.21.15.106:0/997395389 shutdown_connections 2024-04-13T03:10:29.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.879+0000 7f02cd9f7640 1 -- 172.21.15.106:0/997395389 wait complete. 2024-04-13T03:10:29.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 Processor -- start 2024-04-13T03:10:29.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 -- start start 2024-04-13T03:10:29.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f02c80b3920 0x7f02c8156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 0x7f02c80d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d30a0 con 0x7f02c80cb870 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d2dc0 con 0x7f02c80d3490 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02cd9f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80b1930 con 0x7f02c80b3920 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.880+0000 7f02c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59616/0 (socket says 172.21.15.106:59616) 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 -- 172.21.15.106:0/3709893049 learned_addr learned my addr 172.21.15.106:0/3709893049 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c6ffd640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f02c80b3920 0x7f02c8156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02bffff640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 0x7f02c80d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 -- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f02c80b3920 msgr2=0x7f02c8156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f02c80b3920 0x7f02c8156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 -- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 msgr2=0x7f02c80d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 0x7f02c80d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 -- 172.21.15.106:0/3709893049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c80b4fd0 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02bffff640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 0x7f02c80d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c77fe640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009e80 tx=0x7f02c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0039050 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c0045070 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c806c930 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c003e830 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.881+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c806cef0 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.882+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8068a10 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.882+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02c003e9d0 con 0x7f02c80cb870 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.883+0000 7f02c4ff9640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 0x7f02a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:29.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.883+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f02c00ae890 con 0x7f02c80cb870 2024-04-13T03:10:29.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.883+0000 7f02c6ffd640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 0x7f02a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:29.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.883+0000 7f02c6ffd640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 0x7f02a806ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:29.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:29.887+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02c80b4fd0 con 0x7f02c80cb870 2024-04-13T03:10:30.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.069+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f02c8161de0 con 0x7f02c80cb870 2024-04-13T03:10:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.752+0000 7f02c4ff9640 1 -- 172.21.15.106:0/3709893049 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f02c007ccc0 con 0x7f02c80cb870 2024-04-13T03:10:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data_ec' removed 2024-04-13T03:10:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.756+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 msgr2=0x7f02a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.756+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 0x7f02a806ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8007b50 comp rx=0 tx=0).stop 2024-04-13T03:10:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.756+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 msgr2=0x7f02c80d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.756+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009e80 tx=0x7f02c0004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 shutdown_connections 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f02a806c920 0x7f02a806ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02c80cb870 0x7f02c80d19d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02c80d3490 0x7f02c80d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 --2- 172.21.15.106:0/3709893049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f02c80b3920 0x7f02c8156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.757+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 >> 172.21.15.106:0/3709893049 conn(0x7f02c80b9760 msgr2=0x7f02c815da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.758+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 shutdown_connections 2024-04-13T03:10:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.758+0000 7f02cd9f7640 1 -- 172.21.15.106:0/3709893049 wait complete. 2024-04-13T03:10:30.774 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:10:30.775 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:10:30.775 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:10:30.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:10:30.775 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:10:30.775 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:10:30.775 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:10:30.776 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:10:30.776 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:10:30.776 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:10:30.776 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:10:30.776 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:10:30.776 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:10:30.776 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:10:30.777 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:10:30.777 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:10:30.785 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:10:30.786 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:10:30.786 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:10:30.787 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:10:30.788 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:10:30.788 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:10:30.788 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:10:30.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:10:30.829 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:10:30.829 DEBUG:teuthology.parallel:result is None 2024-04-13T03:10:30.830 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:10:30.830 DEBUG:teuthology.parallel:result is None 2024-04-13T03:10:30.831 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:10:30.831 DEBUG:teuthology.parallel:result is None 2024-04-13T03:10:30.832 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:10:30.832 DEBUG:teuthology.parallel:result is None 2024-04-13T03:10:30.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:10:30.908 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:10:30.908 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:10:30.914 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:10:30.914 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:10:30.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 Processor -- start 2024-04-13T03:10:30.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 -- start start 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13a815e9d0 0x7f13a8162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 0x7f13a80d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 0x7f13a815f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f13a80d5060 con 0x7f13a815e9d0 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f13a8162520 con 0x7f13a80d46c0 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.939+0000 7f13ad555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f13a8162ae0 con 0x7f13a815edb0 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13a815e9d0 0x7f13a8162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13a815e9d0 0x7f13a8162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59652/0 (socket says 172.21.15.106:59652) 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 -- 172.21.15.106:0/2018586890 learned_addr learned my addr 172.21.15.106:0/2018586890 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f139e7fc640 1 --2- 172.21.15.106:0/2018586890 >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 0x7f13a80d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a77fe640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 0x7f13a815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 -- 172.21.15.106:0/2018586890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 msgr2=0x7f13a815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 0x7f13a815f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 -- 172.21.15.106:0/2018586890 >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 msgr2=0x7f13a80d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/2018586890 >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 0x7f13a80d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 -- 172.21.15.106:0/2018586890 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a815f780 con 0x7f13a815e9d0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f139e7fc640 1 --2- 172.21.15.106:0/2018586890 >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 0x7f13a80d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a77fe640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 0x7f13a815f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.940+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a815e9d0 0x7f13a8162f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f13a80b7260 tx=0x7f139802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98c63a59995dd91 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13a67fc640 1 -- 172.21.15.106:0/2018586890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139802e070 con 0x7f13a815e9d0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13a67fc640 1 -- 172.21.15.106:0/2018586890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1398037070 con 0x7f13a815e9d0 2024-04-13T03:10:30.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13a67fc640 1 -- 172.21.15.106:0/2018586890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398031a00 con 0x7f13a815e9d0 2024-04-13T03:10:30.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 -- 172.21.15.106:0/2018586890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a815e9d0 msgr2=0x7f13a8162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:30.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a815e9d0 0x7f13a8162f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f13a80b7260 tx=0x7f139802b880 comp rx=0 tx=0).stop 2024-04-13T03:10:30.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 -- 172.21.15.106:0/2018586890 shutdown_connections 2024-04-13T03:10:30.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815edb0 0x7f13a815f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 --2- 172.21.15.106:0/2018586890 >> v2:172.21.15.114:3300/0 conn(0x7f13a80d46c0 0x7f13a80d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 --2- 172.21.15.106:0/2018586890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a815e9d0 0x7f13a8162f70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 -- 172.21.15.106:0/2018586890 >> 172.21.15.106:0/2018586890 conn(0x7f13a808bf10 msgr2=0x7f13a80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 -- 172.21.15.106:0/2018586890 shutdown_connections 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.941+0000 7f13ad555640 1 -- 172.21.15.106:0/2018586890 wait complete. 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.942+0000 7f13ad555640 1 Processor -- start 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 -- start start 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 0x7f13a806dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 0x7f13a8161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13a815fd10 con 0x7f13a80d46c0 2024-04-13T03:10:30.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f13a81634b0 con 0x7f13a815edb0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13ad555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f13a8162770 con 0x7f13a815e9d0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13a6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13a6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59664/0 (socket says 172.21.15.106:59664) 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.943+0000 7f13a6ffd640 1 -- 172.21.15.106:0/4001810370 learned_addr learned my addr 172.21.15.106:0/4001810370 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f139e7fc640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 0x7f13a806dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a77fe640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 0x7f13a8161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 -- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 msgr2=0x7f13a806dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 0x7f13a806dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 -- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 msgr2=0x7f13a8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 0x7f13a8161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 -- 172.21.15.106:0/4001810370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a81f8270 con 0x7f13a80d46c0 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f139e7fc640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 0x7f13a806dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a77fe640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 0x7f13a8161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:30.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13a6ffd640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f139802b8c0 tx=0x7f13980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398032880 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1398032a40 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a81f8550 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398032e30 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.944+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a81f8b10 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.945+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a8069120 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.946+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f139804c060 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.946+0000 7f139ffff640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 0x7f138c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.946+0000 7f139e7fc640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 0x7f138c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.946+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f13980ad580 con 0x7f13a80d46c0 2024-04-13T03:10:30.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.946+0000 7f139e7fc640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 0x7f138c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13a806dac0 tx=0x7f1390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:30.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:30.950+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1398040070 con 0x7f13a80d46c0 2024-04-13T03:10:31.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.144+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f13a80d1690 con 0x7f13a80d46c0 2024-04-13T03:10:31.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.761+0000 7f139ffff640 1 -- 172.21.15.106:0/4001810370 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f139807bc40 con 0x7f13a80d46c0 2024-04-13T03:10:31.762 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 msgr2=0x7f138c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 0x7f138c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13a806dac0 tx=0x7f1390008040 comp rx=0 tx=0).stop 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 msgr2=0x7f13a814da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f139802b8c0 tx=0x7f13980046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 shutdown_connections 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f138c06c8f0 0x7f138c06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13a815e9d0 0x7f13a806dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13a815edb0 0x7f13a8161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 --2- 172.21.15.106:0/4001810370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13a80d46c0 0x7f13a814da30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 >> 172.21.15.106:0/4001810370 conn(0x7f13a808bf10 msgr2=0x7f13a80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:31.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 shutdown_connections 2024-04-13T03:10:31.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.764+0000 7f13ad555640 1 -- 172.21.15.106:0/4001810370 wait complete. 2024-04-13T03:10:31.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:10:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.881+0000 7f6a0442d640 1 Processor -- start 2024-04-13T03:10:31.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.881+0000 7f6a0442d640 1 -- start start 2024-04-13T03:10:31.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a0442d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f69fc0cd350 0x7f69fc0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:31.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a0442d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc0b5e50 con 0x7f69fc15ce00 2024-04-13T03:10:31.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a0442d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc0b5030 con 0x7f69fc0b1c70 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a0442d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0b8310 con 0x7f69fc0cd350 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a021c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f69fc0cd350 0x7f69fc0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a029ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f69fc15ce00 0x7f69fc0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53810/0 (socket says 172.21.15.106:53810) 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.882+0000 7f6a029ca640 1 -- 172.21.15.106:0/946109392 learned_addr learned my addr 172.21.15.106:0/946109392 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.883+0000 7f6a021c9640 1 -- 172.21.15.106:0/946109392 >> v1:172.21.15.114:6789/0 conn(0x7f69fc0b1c70 legacy=0x7f69fc0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.883+0000 7f6a011c7640 1 -- 172.21.15.106:0/946109392 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156960471 0 0) 0x7f69f4001490 con 0x7f69fc15ce00 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.883+0000 7f6a021c9640 1 -- 172.21.15.106:0/946109392 >> v1:172.21.15.106:6789/0 conn(0x7f69fc15ce00 legacy=0x7f69fc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.883+0000 7f6a021c9640 1 -- 172.21.15.106:0/946109392 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0b4d50 con 0x7f69fc0cd350 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.884+0000 7f6a021c9640 1 --2- 172.21.15.106:0/946109392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0cd350 0x7f69fc0cd730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0097a0 tx=0x7f69ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a420de58c34a44a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.884+0000 7f6a011c7640 1 -- 172.21.15.106:0/946109392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0315a0 con 0x7f69fc0cd350 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.884+0000 7f6a011c7640 1 -- 172.21.15.106:0/946109392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ec037070 con 0x7f69fc0cd350 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.884+0000 7f6a011c7640 1 -- 172.21.15.106:0/946109392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec031960 con 0x7f69fc0cd350 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 -- 172.21.15.106:0/946109392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0cd350 msgr2=0x7f69fc0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 --2- 172.21.15.106:0/946109392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0cd350 0x7f69fc0cd730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0097a0 tx=0x7f69ec02b840 comp rx=0 tx=0).stop 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 -- 172.21.15.106:0/946109392 shutdown_connections 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 --2- 172.21.15.106:0/946109392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0cd350 0x7f69fc0cd730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 -- 172.21.15.106:0/946109392 >> 172.21.15.106:0/946109392 conn(0x7f69fc0b4100 msgr2=0x7f69fc0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 -- 172.21.15.106:0/946109392 shutdown_connections 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.885+0000 7f6a0442d640 1 -- 172.21.15.106:0/946109392 wait complete. 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.886+0000 7f6a0442d640 1 Processor -- start 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.886+0000 7f6a0442d640 1 -- start start 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 0x7f69fc06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 0x7f69fc0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0cdc70 con 0x7f69fc0b1c70 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0d1000 con 0x7f69fc0cd350 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0b7f30 con 0x7f69fc15ce00 2024-04-13T03:10:31.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59680/0 (socket says 172.21.15.106:59680) 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 -- 172.21.15.106:0/420649929 learned_addr learned my addr 172.21.15.106:0/420649929 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a019c8640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 0x7f69fc06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a029ca640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 0x7f69fc0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 -- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 msgr2=0x7f69fc0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 0x7f69fc0c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 -- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 msgr2=0x7f69fc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 0x7f69fc06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 -- 172.21.15.106:0/420649929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc06dab0 con 0x7f69fc0b1c70 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a019c8640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 0x7f69fc06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a029ca640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 0x7f69fc0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a021c9640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f69ec034040 tx=0x7f69ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec032840 con 0x7f69fc0b1c70 2024-04-13T03:10:31.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ec0329e0 con 0x7f69fc0b1c70 2024-04-13T03:10:31.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc06dd90 con 0x7f69fc0b1c70 2024-04-13T03:10:31.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.887+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec032dd0 con 0x7f69fc0b1c70 2024-04-13T03:10:31.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.888+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc06e350 con 0x7f69fc0b1c70 2024-04-13T03:10:31.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.889+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc075d80 con 0x7f69fc0b1c70 2024-04-13T03:10:31.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.889+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69ec03b3f0 con 0x7f69fc0b1c70 2024-04-13T03:10:31.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.889+0000 7f69f2ffd640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:31.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.889+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f69ec02f030 con 0x7f69fc0b1c70 2024-04-13T03:10:31.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.890+0000 7f6a019c8640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:31.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.890+0000 7f6a019c8640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 0x7f69e006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0b4bc0 tx=0x7f69f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:31.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:31.894+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69fc075d80 con 0x7f69fc0b1c70 2024-04-13T03:10:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.143+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f69fc0b4860 con 0x7f69fc0b1c70 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.144+0000 7f69f2ffd640 1 -- 172.21.15.106:0/420649929 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f69ec040070 con 0x7f69fc0b1c70 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.145 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.146 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:10:32.148 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:10:32.150 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.151 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.152 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.154 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:10:32.155 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.156 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mds", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlm1jnFCRAAAWEiQJp5AzUjy4RkSVpEZw==", 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mds" 2024-04-13T03:10:32.157 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mgr", 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlmtUHFCRAAFuOpjg989vT5lNp7P29Fsw==", 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-osd", 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlmdEjFCRAAnujHD1WOyqXWC0sfbqkLPA==", 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-osd" 2024-04-13T03:10:32.158 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd", 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlmzk7FCRAAFlafIqMEeVbEPOdM0Ys3Bg==", 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlmllXFCRAA1Z6L8mdA/3ReYd8Wazqd8A==", 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-13T03:10:32.159 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rgw", 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBM9xlmXlzFCRAAf19mc0UL8eX036Bmef1zow==", 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:10:32.160 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:10:32.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:10:32.162 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:10:32.162 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.162 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:32.162 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:10:32.162 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:10:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 msgr2=0x7f69e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 0x7f69e006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0b4bc0 tx=0x7f69f8009040 comp rx=0 tx=0).stop 2024-04-13T03:10:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 msgr2=0x7f69fc0cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f69ec034040 tx=0x7f69ec0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 shutdown_connections 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f69fc15ce00 0x7f69fc0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69fc0cd350 0x7f69fc06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 --2- 172.21.15.106:0/420649929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69fc0b1c70 0x7f69fc0cc1d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 >> 172.21.15.106:0/420649929 conn(0x7f69fc0b4100 msgr2=0x7f69fc070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 shutdown_connections 2024-04-13T03:10:32.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.146+0000 7f6a0442d640 1 -- 172.21.15.106:0/420649929 wait complete. 2024-04-13T03:10:32.164 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-13T03:10:32.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.270+0000 7f781322d640 1 Processor -- start 2024-04-13T03:10:32.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.270+0000 7f781322d640 1 -- start start 2024-04-13T03:10:32.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f781322d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f780c0af800 0x7f780c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f781322d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f780c06f690 0x7f780c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f781322d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780c15c200 con 0x7f780c06f200 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f781322d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f780c0b5090 con 0x7f780c06f690 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f781322d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f780c06fb00 con 0x7f780c0af800 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f78117ca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f780c06f690 0x7f780c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f780bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f780c06f200 0x7f780c15e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53826/0 (socket says 172.21.15.106:53826) 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.271+0000 7f780bfff640 1 -- 172.21.15.106:0/2716922610 learned_addr learned my addr 172.21.15.106:0/2716922610 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f7810fc9640 1 --2- 172.21.15.106:0/2716922610 >> v2:172.21.15.114:3300/0 conn(0x7f780c0af800 0x7f780c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f78117ca640 1 -- 172.21.15.106:0/2716922610 >> v2:172.21.15.114:3300/0 conn(0x7f780c0af800 msgr2=0x7f780c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f78117ca640 1 --2- 172.21.15.106:0/2716922610 >> v2:172.21.15.114:3300/0 conn(0x7f780c0af800 0x7f780c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f78117ca640 1 -- 172.21.15.106:0/2716922610 >> v1:172.21.15.106:6789/0 conn(0x7f780c06f200 legacy=0x7f780c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f78117ca640 1 -- 172.21.15.106:0/2716922610 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c0b3ff0 con 0x7f780c06f690 2024-04-13T03:10:32.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.272+0000 7f780b7fe640 1 -- 172.21.15.106:0/2716922610 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785667210 0 0) 0x7f77f8001110 con 0x7f780c06f200 2024-04-13T03:10:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.273+0000 7f78117ca640 1 --2- 172.21.15.106:0/2716922610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f690 0x7f780c15ef40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f780c0b2e50 tx=0x7f780402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7512d4a07a41583a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.273+0000 7f780b7fe640 1 -- 172.21.15.106:0/2716922610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804039070 con 0x7f780c06f690 2024-04-13T03:10:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.273+0000 7f780b7fe640 1 -- 172.21.15.106:0/2716922610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780402cc80 con 0x7f780c06f690 2024-04-13T03:10:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.273+0000 7f780b7fe640 1 -- 172.21.15.106:0/2716922610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804034660 con 0x7f780c06f690 2024-04-13T03:10:32.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 -- 172.21.15.106:0/2716922610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f690 msgr2=0x7f780c15ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 --2- 172.21.15.106:0/2716922610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f690 0x7f780c15ef40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f780c0b2e50 tx=0x7f780402bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 -- 172.21.15.106:0/2716922610 shutdown_connections 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 --2- 172.21.15.106:0/2716922610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f690 0x7f780c15ef40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 --2- 172.21.15.106:0/2716922610 >> v2:172.21.15.114:3300/0 conn(0x7f780c0af800 0x7f780c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 -- 172.21.15.106:0/2716922610 >> 172.21.15.106:0/2716922610 conn(0x7f780c0b2600 msgr2=0x7f780c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 -- 172.21.15.106:0/2716922610 shutdown_connections 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.274+0000 7f781322d640 1 -- 172.21.15.106:0/2716922610 wait complete. 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.275+0000 7f781322d640 1 Processor -- start 2024-04-13T03:10:32.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.275+0000 7f781322d640 1 -- start start 2024-04-13T03:10:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 0x7f780c13ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 0x7f780c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f780c0b4890 con 0x7f780c06f200 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f780c0b45b0 con 0x7f780c0af800 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f781322d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f780c0b4b70 con 0x7f780c06f690 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59688/0 (socket says 172.21.15.106:59688) 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 -- 172.21.15.106:0/1758323864 learned_addr learned my addr 172.21.15.106:0/1758323864 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f78117ca640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 0x7f780c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f780bfff640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 0x7f780c13ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 -- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 msgr2=0x7f780c13ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 0x7f780c13ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 -- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 msgr2=0x7f780c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 0x7f780c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 -- 172.21.15.106:0/1758323864 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c143da0 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f78117ca640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 0x7f780c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f780bfff640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 0x7f780c13ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.276+0000 7f7810fc9640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f780000d800 tx=0x7f780000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.277+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780000b6c0 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.277+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7800019070 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.277+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c144080 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.277+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780000ba80 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.277+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c14a900 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.278+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c068a10 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.278+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f780000bc20 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.279+0000 7f78097fa640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 0x7f77f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.279+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f780008bd40 con 0x7f780c06f200 2024-04-13T03:10:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.279+0000 7f780bfff640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 0x7f77f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.279+0000 7f780bfff640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 0x7f77f006ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f780c14f2d0 tx=0x7f77f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.283+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f780001c360 con 0x7f780c06f200 2024-04-13T03:10:32.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.537+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f780c06cc70 con 0x7f780c06f200 2024-04-13T03:10:32.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.539+0000 7f78097fa640 1 -- 172.21.15.106:0/1758323864 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f780005a3c0 con 0x7f780c06f200 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 msgr2=0x7f77f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 0x7f77f006ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f780c14f2d0 tx=0x7f77f8009040 comp rx=0 tx=0).stop 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 msgr2=0x7f780c143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f780000d800 tx=0x7f780000dcc0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 shutdown_connections 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f77f006c920 0x7f77f006ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f780c06f690 0x7f780c13ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f780c0af800 0x7f780c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 --2- 172.21.15.106:0/1758323864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c06f200 0x7f780c143520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 >> 172.21.15.106:0/1758323864 conn(0x7f780c0b2600 msgr2=0x7f780c0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 shutdown_connections 2024-04-13T03:10:32.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.541+0000 7f781322d640 1 -- 172.21.15.106:0/1758323864 wait complete. 2024-04-13T03:10:32.556 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-13T03:10:32.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.657+0000 7f7330a98640 1 Processor -- start 2024-04-13T03:10:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.657+0000 7f7330a98640 1 -- start start 2024-04-13T03:10:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f732c15d380 0x7f732c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0b7f40 0x7f732c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f732c0ccd40 0x7f732c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f732c0d0f70 con 0x7f732c15d380 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f732c0d0640 con 0x7f732c0ccd40 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f7330a98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f732c0d0920 con 0x7f732c0b7f40 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f732a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f732c15d380 0x7f732c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f732a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f732c15d380 0x7f732c15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56394/0 (socket says 172.21.15.106:56394) 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.658+0000 7f732a59c640 1 -- 172.21.15.106:0/4044549632 learned_addr learned my addr 172.21.15.106:0/4044549632 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f7323fff640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0b7f40 0x7f732c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732ad9d640 1 --2- 172.21.15.106:0/4044549632 >> v2:172.21.15.114:3300/0 conn(0x7f732c0ccd40 0x7f732c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 -- 172.21.15.106:0/4044549632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0b7f40 msgr2=0x7f732c06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0b7f40 0x7f732c06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 -- 172.21.15.106:0/4044549632 >> v2:172.21.15.114:3300/0 conn(0x7f732c0ccd40 msgr2=0x7f732c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 --2- 172.21.15.106:0/4044549632 >> v2:172.21.15.114:3300/0 conn(0x7f732c0ccd40 0x7f732c0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 -- 172.21.15.106:0/4044549632 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c0d0c00 con 0x7f732c15d380 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.659+0000 7f732a59c640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c15d760 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f732c0b8600 tx=0x7f731c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad53c811f9c8ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7329d9b640 1 -- 172.21.15.106:0/4044549632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c02e070 con 0x7f732c15d380 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7329d9b640 1 -- 172.21.15.106:0/4044549632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c037070 con 0x7f732c15d380 2024-04-13T03:10:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7329d9b640 1 -- 172.21.15.106:0/4044549632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c031a00 con 0x7f732c15d380 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 -- 172.21.15.106:0/4044549632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 msgr2=0x7f732c15d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c15d760 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f732c0b8600 tx=0x7f731c02b880 comp rx=0 tx=0).stop 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 -- 172.21.15.106:0/4044549632 shutdown_connections 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 --2- 172.21.15.106:0/4044549632 >> v2:172.21.15.114:3300/0 conn(0x7f732c0ccd40 0x7f732c0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0b7f40 0x7f732c06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 --2- 172.21.15.106:0/4044549632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c15d760 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 -- 172.21.15.106:0/4044549632 >> 172.21.15.106:0/4044549632 conn(0x7f732c0d2530 msgr2=0x7f732c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 -- 172.21.15.106:0/4044549632 shutdown_connections 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.660+0000 7f7330a98640 1 -- 172.21.15.106:0/4044549632 wait complete. 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.661+0000 7f7330a98640 1 Processor -- start 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 -- start start 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 0x7f732c146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 0x7f732c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f732c0cd920 con 0x7f732c15d380 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f732c0d1400 con 0x7f732c0b7f40 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7330a98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f732c0c8b00 con 0x7f732c0ccd40 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f732ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f732ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56408/0 (socket says 172.21.15.106:56408) 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f732ad9d640 1 -- 172.21.15.106:0/3862768578 learned_addr learned my addr 172.21.15.106:0/3862768578 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.662+0000 7f7323fff640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 0x7f732c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732a59c640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 0x7f732c146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 -- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 msgr2=0x7f732c1f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 0x7f732c1f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 -- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 msgr2=0x7f732c146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 0x7f732c146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 -- 172.21.15.106:0/3862768578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c146360 con 0x7f732c15d380 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7323fff640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 0x7f732c1f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732a59c640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 0x7f732c146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f732ad9d640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f732400efc0 tx=0x7f732400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324018020 con 0x7f732c15d380 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7324014480 con 0x7f732c15d380 2024-04-13T03:10:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324014870 con 0x7f732c15d380 2024-04-13T03:10:32.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c146640 con 0x7f732c15d380 2024-04-13T03:10:32.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.663+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c0c8de0 con 0x7f732c15d380 2024-04-13T03:10:32.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.664+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c069120 con 0x7f732c15d380 2024-04-13T03:10:32.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.665+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7324005d10 con 0x7f732c15d380 2024-04-13T03:10:32.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.665+0000 7f7322ffd640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:32.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.665+0000 7f732a59c640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:32.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.665+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f732408c3c0 con 0x7f732c15d380 2024-04-13T03:10:32.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.666+0000 7f732a59c640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 0x7f730c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f731c036000 tx=0x7f731c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:32.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.669+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7324022070 con 0x7f732c15d380 2024-04-13T03:10:32.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.922+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f732c0d20a0 con 0x7f732c15d380 2024-04-13T03:10:32.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.927+0000 7f7322ffd640 1 -- 172.21.15.106:0/3862768578 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f7324010040 con 0x7f732c15d380 2024-04-13T03:10:32.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 msgr2=0x7f730c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 0x7f730c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f731c036000 tx=0x7f731c03d040 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 msgr2=0x7f732c0c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f732400efc0 tx=0x7f732400c490 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 shutdown_connections 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f732c0ccd40 0x7f732c1f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c15d380 0x7f732c0c8060 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 --2- 172.21.15.106:0/3862768578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f732c0b7f40 0x7f732c146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 >> 172.21.15.106:0/3862768578 conn(0x7f732c0d2530 msgr2=0x7f732c1527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 shutdown_connections 2024-04-13T03:10:32.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:32.929+0000 7f7330a98640 1 -- 172.21.15.106:0/3862768578 wait complete. 2024-04-13T03:10:32.941 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-13T03:10:33.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.051+0000 7fe703836640 1 Processor -- start 2024-04-13T03:10:33.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 -- start start 2024-04-13T03:10:33.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe6fc15e930 0x7fe6fc0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6fc06f200 con 0x7fe6fc06ed70 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc06e280 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.052+0000 7fe703836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc06e560 con 0x7fe6fc15e930 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56420/0 (socket says 172.21.15.106:56420) 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 -- 172.21.15.106:0/2303742288 learned_addr learned my addr 172.21.15.106:0/2303742288 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe7015d2640 1 --2- 172.21.15.106:0/2303742288 >> v2:172.21.15.114:3300/0 conn(0x7fe6fc15e930 0x7fe6fc0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 -- 172.21.15.106:0/2303742288 >> v2:172.21.15.114:3300/0 conn(0x7fe6fc15e930 msgr2=0x7fe6fc0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 --2- 172.21.15.106:0/2303742288 >> v2:172.21.15.114:3300/0 conn(0x7fe6fc15e930 0x7fe6fc0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 -- 172.21.15.106:0/2303742288 >> v1:172.21.15.114:6789/0 conn(0x7fe6fc06ed70 legacy=0x7fe6fc0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.053+0000 7fe701dd3640 1 -- 172.21.15.106:0/2303742288 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc0cccf0 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe701dd3640 1 --2- 172.21.15.106:0/2303742288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b50f0 tx=0x7fe6f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bf121d4383d33b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe6f3fff640 1 -- 172.21.15.106:0/2303742288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4035020 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe6f3fff640 1 -- 172.21.15.106:0/2303742288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4031870 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe6f3fff640 1 -- 172.21.15.106:0/2303742288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4031c40 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 -- 172.21.15.106:0/2303742288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc0d09f0 msgr2=0x7fe6fc0cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 --2- 172.21.15.106:0/2303742288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b50f0 tx=0x7fe6f402b910 comp rx=0 tx=0).stop 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 -- 172.21.15.106:0/2303742288 shutdown_connections 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 --2- 172.21.15.106:0/2303742288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc0d09f0 0x7fe6fc0cd7d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 --2- 172.21.15.106:0/2303742288 >> v2:172.21.15.114:3300/0 conn(0x7fe6fc15e930 0x7fe6fc0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.054+0000 7fe703836640 1 -- 172.21.15.106:0/2303742288 >> 172.21.15.106:0/2303742288 conn(0x7fe6fc0cef80 msgr2=0x7fe6fc0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.055+0000 7fe703836640 1 -- 172.21.15.106:0/2303742288 shutdown_connections 2024-04-13T03:10:33.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.055+0000 7fe703836640 1 -- 172.21.15.106:0/2303742288 wait complete. 2024-04-13T03:10:33.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 Processor -- start 2024-04-13T03:10:33.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 -- start start 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6fc0d09f0 0x7fe6fc1f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 0x7fe6fc1f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0ce830 con 0x7fe6fc06ed70 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0cd2b0 con 0x7fe6fc15e930 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe703836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc0ce9b0 con 0x7fe6fc0d09f0 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe7015d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe7015d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56422/0 (socket says 172.21.15.106:56422) 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.056+0000 7fe7015d2640 1 -- 172.21.15.106:0/2196376267 learned_addr learned my addr 172.21.15.106:0/2196376267 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe701dd3640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 0x7fe6fc1f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe700dd1640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6fc0d09f0 0x7fe6fc1f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 -- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6fc0d09f0 msgr2=0x7fe6fc1f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6fc0d09f0 0x7fe6fc1f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 -- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 msgr2=0x7fe6fc1f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 0x7fe6fc1f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 -- 172.21.15.106:0/2196376267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc0c8e80 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe701dd3640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 0x7fe6fc1f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe7015d2640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007e40 tx=0x7fe6f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8014620 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f8019070 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80149e0 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc0c80c0 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.057+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc0c8960 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.058+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc0c84b0 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.059+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6f8014b80 con 0x7fe6fc06ed70 2024-04-13T03:10:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.059+0000 7fe6f1ffb640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 0x7fe6e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.059+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fe6f808c240 con 0x7fe6fc06ed70 2024-04-13T03:10:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.059+0000 7fe700dd1640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 0x7fe6e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.059+0000 7fe700dd1640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 0x7fe6e006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc06f860 tx=0x7fe6ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.063+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6f805a830 con 0x7fe6fc06ed70 2024-04-13T03:10:33.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.304+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fe6fc068a10 con 0x7fe6fc06ed70 2024-04-13T03:10:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.308+0000 7fe6f1ffb640 1 -- 172.21.15.106:0/2196376267 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fe6f805a130 con 0x7fe6fc06ed70 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 msgr2=0x7fe6e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 0x7fe6e006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc06f860 tx=0x7fe6ec009040 comp rx=0 tx=0).stop 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 msgr2=0x7fe6fc147000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007e40 tx=0x7fe6f800e4b0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 shutdown_connections 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe6e006c8f0 0x7fe6e006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6fc0d09f0 0x7fe6fc1f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6fc15e930 0x7fe6fc1f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 --2- 172.21.15.106:0/2196376267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6fc06ed70 0x7fe6fc147000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 >> 172.21.15.106:0/2196376267 conn(0x7fe6fc0cef80 msgr2=0x7fe6fc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 shutdown_connections 2024-04-13T03:10:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.310+0000 7fe703836640 1 -- 172.21.15.106:0/2196376267 wait complete. 2024-04-13T03:10:33.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-13T03:10:33.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.422+0000 7f2d79298640 1 Processor -- start 2024-04-13T03:10:33.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.422+0000 7f2d79298640 1 -- start start 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d79298640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d740cf480 0x7f2d7415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d79298640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7415f380 con 0x7f2d740d3220 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d79298640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7415f600 con 0x7f2d740ba170 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d79298640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2d740d2d90 con 0x7f2d740cf480 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d740cf480 0x7f2d7415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d737fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2d740d3220 0x7f2d740d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54908/0 (socket says 172.21.15.106:54908) 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d740cf480 0x7f2d7415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56434/0 (socket says 172.21.15.106:56434) 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d737fe640 1 -- 172.21.15.106:0/3800762850 learned_addr learned my addr 172.21.15.106:0/3800762850 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3800762850 >> v1:172.21.15.114:6789/0 conn(0x7f2d740ba170 legacy=0x7f2d740cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d71ffb640 1 -- 172.21.15.106:0/3800762850 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060846247 0 0) 0x7f2d6c001110 con 0x7f2d740d3220 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3800762850 >> v1:172.21.15.106:6789/0 conn(0x7f2d740d3220 legacy=0x7f2d740d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.423+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3800762850 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7415f8e0 con 0x7f2d740cf480 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.424+0000 7f2d72ffd640 1 --2- 172.21.15.106:0/3800762850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740cf480 0x7f2d7415ee40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009d90 tx=0x7f2d6402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e291a0110304b274 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.424+0000 7f2d71ffb640 1 -- 172.21.15.106:0/3800762850 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64039070 con 0x7f2d740cf480 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.424+0000 7f2d71ffb640 1 -- 172.21.15.106:0/3800762850 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6402cc60 con 0x7f2d740cf480 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.424+0000 7f2d71ffb640 1 -- 172.21.15.106:0/3800762850 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64034660 con 0x7f2d740cf480 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 -- 172.21.15.106:0/3800762850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740cf480 msgr2=0x7f2d7415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 --2- 172.21.15.106:0/3800762850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740cf480 0x7f2d7415ee40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009d90 tx=0x7f2d6402beb0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 -- 172.21.15.106:0/3800762850 shutdown_connections 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 --2- 172.21.15.106:0/3800762850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740cf480 0x7f2d7415ee40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 -- 172.21.15.106:0/3800762850 >> 172.21.15.106:0/3800762850 conn(0x7f2d740d47d0 msgr2=0x7f2d740d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:33.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 -- 172.21.15.106:0/3800762850 shutdown_connections 2024-04-13T03:10:33.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.425+0000 7f2d79298640 1 -- 172.21.15.106:0/3800762850 wait complete. 2024-04-13T03:10:33.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.426+0000 7f2d79298640 1 Processor -- start 2024-04-13T03:10:33.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.426+0000 7f2d79298640 1 -- start start 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d740cf480 0x7f2d740cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d740d3220 0x7f2d740c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d741592f0 con 0x7f2d740ba170 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2d74159170 con 0x7f2d740d3220 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d79298640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2d740d3de0 con 0x7f2d740cf480 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56448/0 (socket says 172.21.15.106:56448) 2024-04-13T03:10:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3423833643 learned_addr learned my addr 172.21.15.106:0/3423833643 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d737fe640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d740d3220 0x7f2d740c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d727fc640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d740cf480 0x7f2d740cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d740cf480 msgr2=0x7f2d740cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d740cf480 0x7f2d740cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d740d3220 msgr2=0x7f2d740c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d740d3220 0x7f2d740c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.427+0000 7f2d72ffd640 1 -- 172.21.15.106:0/3423833643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d740c9b10 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d72ffd640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2d64037040 tx=0x7f2d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64044020 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6403f670 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d740c8530 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64039040 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.428+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d740c8af0 con 0x7f2d740ba170 2024-04-13T03:10:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.429+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d74068a10 con 0x7f2d740ba170 2024-04-13T03:10:33.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.429+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d64047020 con 0x7f2d740ba170 2024-04-13T03:10:33.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.429+0000 7f2d5b7fe640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 0x7f2d6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.430+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f2d640ae610 con 0x7f2d740ba170 2024-04-13T03:10:33.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.430+0000 7f2d727fc640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 0x7f2d6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.430+0000 7f2d727fc640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 0x7f2d6006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ce5f0 tx=0x7f2d68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.434+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d6404e030 con 0x7f2d740ba170 2024-04-13T03:10:33.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.674+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f2d74160990 con 0x7f2d740ba170 2024-04-13T03:10:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.678+0000 7f2d5b7fe640 1 -- 172.21.15.106:0/3423833643 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f2d6407cc00 con 0x7f2d740ba170 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.680+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 msgr2=0x7f2d6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.680+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 0x7f2d6006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ce5f0 tx=0x7f2d68009040 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.680+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 msgr2=0x7f2d740c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.680+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2d64037040 tx=0x7f2d64004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 shutdown_connections 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6006c8f0 0x7f2d6006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d740cf480 0x7f2d740cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d740d3220 0x7f2d740c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 --2- 172.21.15.106:0/3423833643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d740ba170 0x7f2d740c7e70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 >> 172.21.15.106:0/3423833643 conn(0x7f2d740d47d0 msgr2=0x7f2d740d4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:33.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 shutdown_connections 2024-04-13T03:10:33.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.681+0000 7f2d79298640 1 -- 172.21.15.106:0/3423833643 wait complete. 2024-04-13T03:10:33.692 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-13T03:10:33.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.794+0000 7f2b52fcc640 1 Processor -- start 2024-04-13T03:10:33.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.794+0000 7f2b52fcc640 1 -- start start 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b52fcc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2b4c0b53c0 0x7f2b4c0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b52fcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b4c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b52fcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4c1635a0 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b52fcc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c0b5d60 con 0x7f2b4c0b53c0 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b52fcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c0cd2c0 con 0x7f2b4c0d0430 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b50d68640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2b4c15fbb0 0x7f2b4c0cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54914/0 (socket says 172.21.15.106:54914) 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.795+0000 7f2b50d68640 1 -- 172.21.15.106:0/4018828610 learned_addr learned my addr 172.21.15.106:0/4018828610 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b51569640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b4c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/4018828610 >> v2:172.21.15.114:3300/0 conn(0x7f2b4c0b53c0 0x7f2b4c0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405610358 0 0) 0x7f2b4c1635a0 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b34003520 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2584642760 0 0) 0x7f2b34003520 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 msgr2=0x7f2b4c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b4c0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 >> v2:172.21.15.114:3300/0 conn(0x7f2b4c0b53c0 msgr2=0x7f2b4c0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 --2- 172.21.15.106:0/4018828610 >> v2:172.21.15.114:3300/0 conn(0x7f2b4c0b53c0 0x7f2b4c0b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c0cccb0 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2b3c001670 con 0x7f2b4c15fbb0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 >> v1:172.21.15.106:6789/0 conn(0x7f2b4c15fbb0 legacy=0x7f2b4c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b34004800 0x7f2b34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0cd2c0 con 0x7f2b340039b0 2024-04-13T03:10:33.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400b9c0 con 0x7f2b34004800 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3400bb40 con 0x7f2b4c0d0430 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b34004800 0x7f2b34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b51569640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 -- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 msgr2=0x7f2b340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 -- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b34004800 msgr2=0x7f2b34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b34004800 0x7f2b34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b50d68640 1 -- 172.21.15.106:0/4018828610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b34003520 con 0x7f2b340039b0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.796+0000 7f2b51569640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c0d0430 0x7f2b340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.797+0000 7f2b50d68640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b34003dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c210 tx=0x7f2b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f686ebb94da3df22 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.797+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c007d50 con 0x7f2b340039b0 2024-04-13T03:10:33.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.797+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c030070 con 0x7f2b340039b0 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.797+0000 7f2b4b7fe640 1 -- 172.21.15.106:0/4018828610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c02cd40 con 0x7f2b340039b0 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 -- 172.21.15.106:0/4018828610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 msgr2=0x7f2b34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/4018828610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b34003dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c210 tx=0x7f2b34005830 comp rx=0 tx=0).stop 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b51569640 1 -- 172.21.15.106:0/4018828610 reap_dead start 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 -- 172.21.15.106:0/4018828610 shutdown_connections 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 -- 172.21.15.106:0/4018828610 >> 172.21.15.106:0/4018828610 conn(0x7f2b4c0d2530 msgr2=0x7f2b4c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 -- 172.21.15.106:0/4018828610 shutdown_connections 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.798+0000 7f2b52fcc640 1 -- 172.21.15.106:0/4018828610 wait complete. 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.799+0000 7f2b52fcc640 1 Processor -- start 2024-04-13T03:10:33.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.799+0000 7f2b52fcc640 1 -- start start 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 0x7f2b4c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 0x7f2b4c1f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0cde60 con 0x7f2b340039b0 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0ccf50 con 0x7f2b4c06dc90 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b52fcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c15ff90 con 0x7f2b4c06e110 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56454/0 (socket says 172.21.15.106:56454) 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 -- 172.21.15.106:0/1322812086 learned_addr learned my addr 172.21.15.106:0/1322812086 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:33.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 0x7f2b4c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b51569640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 0x7f2b4c1f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 -- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 msgr2=0x7f2b4c1f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 0x7f2b4c1f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 -- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 msgr2=0x7f2b4c1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 0x7f2b4c1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 -- 172.21.15.106:0/1322812086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c1f4f90 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 0x7f2b4c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b51569640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 0x7f2b4c1f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.800+0000 7f2b50d68640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c330 tx=0x7f2b3c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.801+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c045070 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.801+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0049e0 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.801+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c06e670 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.801+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c004db0 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.801+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c06ec30 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.802+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c069120 con 0x7f2b340039b0 2024-04-13T03:10:33.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.802+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b3c007070 con 0x7f2b340039b0 2024-04-13T03:10:33.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.802+0000 7f2b497fa640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 0x7f2b3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:33.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.802+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f2b3c0aca00 con 0x7f2b340039b0 2024-04-13T03:10:33.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.802+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 0x7f2b3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:33.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.803+0000 7f2b4bfff640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 0x7f2b3806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0b3fd0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:33.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:33.807+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b3c07b0d0 con 0x7f2b340039b0 2024-04-13T03:10:34.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.047+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f2b4c0d3190 con 0x7f2b340039b0 2024-04-13T03:10:34.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.051+0000 7f2b497fa640 1 -- 172.21.15.106:0/1322812086 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f2b3c07a9d0 con 0x7f2b340039b0 2024-04-13T03:10:34.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 msgr2=0x7f2b3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 0x7f2b3806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0b3fd0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2024-04-13T03:10:34.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 msgr2=0x7f2b4c153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c330 tx=0x7f2b3c001a40 comp rx=0 tx=0).stop 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 shutdown_connections 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b3806c920 0x7f2b3806ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b4c06e110 0x7f2b4c1f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b4c06dc90 0x7f2b4c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 --2- 172.21.15.106:0/1322812086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b340039b0 0x7f2b4c153360 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 >> 172.21.15.106:0/1322812086 conn(0x7f2b4c0d2530 msgr2=0x7f2b4c1625d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 shutdown_connections 2024-04-13T03:10:34.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.053+0000 7f2b52fcc640 1 -- 172.21.15.106:0/1322812086 wait complete. 2024-04-13T03:10:34.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-13T03:10:34.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.164+0000 7f7e87820640 1 Processor -- start 2024-04-13T03:10:34.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 -- start start 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 0x7f7e800d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7e800b3d60 0x7f7e800d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e800cfd20 con 0x7f7e8015ed90 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7e80162af0 con 0x7f7e800b3d60 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.165+0000 7f7e87820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7e80162dd0 con 0x7f7e8015e9b0 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e855bc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 0x7f7e800d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e85dbd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7e800b3d60 0x7f7e800d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e84dbb640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7e8015ed90 0x7f7e800d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:54332/0 (socket says 172.21.15.106:54332) 2024-04-13T03:10:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e84dbb640 1 -- 172.21.15.106:0/1705211975 learned_addr learned my addr 172.21.15.106:0/1705211975 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e85dbd640 1 -- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 msgr2=0x7f7e800d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e85dbd640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 0x7f7e800d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e85dbd640 1 -- 172.21.15.106:0/1705211975 >> v1:172.21.15.114:6789/0 conn(0x7f7e8015ed90 legacy=0x7f7e800d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e77fff640 1 -- 172.21.15.106:0/1705211975 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089441976 0 0) 0x7f7e800cfd20 con 0x7f7e8015ed90 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e85dbd640 1 -- 172.21.15.106:0/1705211975 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801630b0 con 0x7f7e800b3d60 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.166+0000 7f7e855bc640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 0x7f7e800d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.167+0000 7f7e85dbd640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e800b3d60 0x7f7e800d0200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0097a0 tx=0x7f7e7c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e114c02aae5f2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.167+0000 7f7e77fff640 1 -- 172.21.15.106:0/1705211975 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c034070 con 0x7f7e800b3d60 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.167+0000 7f7e77fff640 1 -- 172.21.15.106:0/1705211975 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c030610 con 0x7f7e800b3d60 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.167+0000 7f7e77fff640 1 -- 172.21.15.106:0/1705211975 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0309e0 con 0x7f7e800b3d60 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 -- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e800b3d60 msgr2=0x7f7e800d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e800b3d60 0x7f7e800d0200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0097a0 tx=0x7f7e7c02b960 comp rx=0 tx=0).stop 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 -- 172.21.15.106:0/1705211975 shutdown_connections 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e800b3d60 0x7f7e800d0200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 --2- 172.21.15.106:0/1705211975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e8015e9b0 0x7f7e800d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 -- 172.21.15.106:0/1705211975 >> 172.21.15.106:0/1705211975 conn(0x7f7e8008bf10 msgr2=0x7f7e800b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 -- 172.21.15.106:0/1705211975 shutdown_connections 2024-04-13T03:10:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.168+0000 7f7e87820640 1 -- 172.21.15.106:0/1705211975 wait complete. 2024-04-13T03:10:34.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.169+0000 7f7e87820640 1 Processor -- start 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.169+0000 7f7e87820640 1 -- start start 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e800b3d60 0x7f7e800b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 0x7f7e8006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80162800 con 0x7f7e80151c10 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80162680 con 0x7f7e8015e9b0 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e87820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7e800d4d00 con 0x7f7e800b3d60 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56456/0 (socket says 172.21.15.106:56456) 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 -- 172.21.15.106:0/434284404 learned_addr learned my addr 172.21.15.106:0/434284404 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e855bc640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e800b3d60 0x7f7e800b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e84dbb640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 0x7f7e8006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 -- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e800b3d60 msgr2=0x7f7e800b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e800b3d60 0x7f7e800b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 -- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 msgr2=0x7f7e8006dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 0x7f7e8006dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 -- 172.21.15.106:0/434284404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8006e390 con 0x7f7e80151c10 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e84dbb640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 0x7f7e8006dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.170+0000 7f7e85dbd640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c02b9a0 tx=0x7f7e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.171+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c034040 con 0x7f7e80151c10 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.171+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c030410 con 0x7f7e80151c10 2024-04-13T03:10:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.171+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c006e10 con 0x7f7e80151c10 2024-04-13T03:10:34.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.171+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8006e670 con 0x7f7e80151c10 2024-04-13T03:10:34.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.171+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e800b4670 con 0x7f7e80151c10 2024-04-13T03:10:34.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.172+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e800b4870 con 0x7f7e80151c10 2024-04-13T03:10:34.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.172+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e7c04c060 con 0x7f7e80151c10 2024-04-13T03:10:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.173+0000 7f7e75ffb640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 0x7f7e6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.173+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f7e7c0ad310 con 0x7f7e80151c10 2024-04-13T03:10:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.173+0000 7f7e855bc640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 0x7f7e6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.173+0000 7f7e855bc640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 0x7f7e6406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e78007440 tx=0x7f7e7800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.177+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e800b4870 con 0x7f7e80151c10 2024-04-13T03:10:34.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.419+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f7e80068a10 con 0x7f7e80151c10 2024-04-13T03:10:34.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.423+0000 7f7e75ffb640 1 -- 172.21.15.106:0/434284404 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f7e7c0abe20 con 0x7f7e80151c10 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.424+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 msgr2=0x7f7e6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.424+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 0x7f7e6406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e78007440 tx=0x7f7e7800b040 comp rx=0 tx=0).stop 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.424+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 msgr2=0x7f7e801613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.424+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c02b9a0 tx=0x7f7e7c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 shutdown_connections 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7e6406c8f0 0x7f7e6406edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e80151c10 0x7f7e801613c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e8015e9b0 0x7f7e8006dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 --2- 172.21.15.106:0/434284404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e800b3d60 0x7f7e800b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 >> 172.21.15.106:0/434284404 conn(0x7f7e8008bf10 msgr2=0x7f7e800b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 shutdown_connections 2024-04-13T03:10:34.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.425+0000 7f7e87820640 1 -- 172.21.15.106:0/434284404 wait complete. 2024-04-13T03:10:34.436 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.535+0000 7ff48e91e640 1 Processor -- start 2024-04-13T03:10:34.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.535+0000 7ff48e91e640 1 -- start start 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.535+0000 7ff48e91e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff488166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff48e91e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff488072de0 con 0x7ff48806cb80 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff48e91e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff488073090 con 0x7ff4880cdf50 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff48e91e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff488073370 con 0x7ff48806f180 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff4877fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff48806cb80 0x7ff4880726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54916/0 (socket says 172.21.15.106:54916) 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff4877fe640 1 -- 172.21.15.106:0/2878028514 learned_addr learned my addr 172.21.15.106:0/2878028514 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff488166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350887019 0 0) 0x7ff488072de0 con 0x7ff48806cb80 2024-04-13T03:10:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff470003520 con 0x7ff48806cb80 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4145319198 0 0) 0x7ff470003520 con 0x7ff48806cb80 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 msgr2=0x7ff488166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff488166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 >> v1:172.21.15.114:6789/0 conn(0x7ff4880cdf50 legacy=0x7ff4880ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff488073650 con 0x7ff48806cb80 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7ff47c001670 con 0x7ff48806cb80 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 >> v1:172.21.15.106:6789/0 conn(0x7ff48806cb80 legacy=0x7ff4880726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff488073370 con 0x7ff470004d40 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff47000ba70 con 0x7ff4700039e0 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff47000bc70 con 0x7ff48806f180 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff487fff640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.536+0000 7ff4877fe640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 -- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 msgr2=0x7ff470004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff48806f180 0x7ff470004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 -- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4700039e0 msgr2=0x7ff470003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 -- 172.21.15.106:0/2878028514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff470003520 con 0x7ff470004d40 2024-04-13T03:10:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.537+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff480009f90 tx=0x7ff470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51c69c46459eddc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480039070 con 0x7ff470004d40 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48002cbd0 con 0x7ff470004d40 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff486ffd640 1 -- 172.21.15.106:0/2878028514 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480009c10 con 0x7ff470004d40 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 -- 172.21.15.106:0/2878028514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470004d40 msgr2=0x7ff4700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2878028514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff480009f90 tx=0x7ff470005830 comp rx=0 tx=0).stop 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48cebb640 1 -- 172.21.15.106:0/2878028514 reap_dead start 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 -- 172.21.15.106:0/2878028514 shutdown_connections 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 -- 172.21.15.106:0/2878028514 >> 172.21.15.106:0/2878028514 conn(0x7ff4880b1f10 msgr2=0x7ff4880b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 -- 172.21.15.106:0/2878028514 shutdown_connections 2024-04-13T03:10:34.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.538+0000 7ff48e91e640 1 -- 172.21.15.106:0/2878028514 wait complete. 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.539+0000 7ff48e91e640 1 Processor -- start 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.539+0000 7ff48e91e640 1 -- start start 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff488150f50 0x7ff48815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48815d940 0x7ff4880c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff488099930 con 0x7ff488151330 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4880997b0 con 0x7ff48815d940 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48e91e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff4880b53a0 con 0x7ff488150f50 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48cebb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48cebb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56466/0 (socket says 172.21.15.106:56466) 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48cebb640 1 -- 172.21.15.106:0/2272080294 learned_addr learned my addr 172.21.15.106:0/2272080294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:34.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff487fff640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff488150f50 0x7ff48815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff4877fe640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48815d940 0x7ff4880c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48cebb640 1 -- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff488150f50 msgr2=0x7ff48815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.540+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff488150f50 0x7ff48815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48cebb640 1 -- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48815d940 msgr2=0x7ff4880c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48815d940 0x7ff4880c07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48cebb640 1 -- 172.21.15.106:0/2272080294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48815dec0 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48cebb640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff480009f90 tx=0x7ff480009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480039030 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48003a040 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480009c10 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4880c14d0 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.541+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4880c1a90 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.542+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff488068a10 con 0x7ff488151330 2024-04-13T03:10:34.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.542+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff480045030 con 0x7ff488151330 2024-04-13T03:10:34.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.543+0000 7ff484ff9640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.543+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7ff4800ad600 con 0x7ff488151330 2024-04-13T03:10:34.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.543+0000 7ff487fff640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.543+0000 7ff487fff640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff478005590 tx=0x7ff478009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.547+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff48004c060 con 0x7ff488151330 2024-04-13T03:10:34.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.723+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff488072140 con 0x7ff488151330 2024-04-13T03:10:34.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.724+0000 7ff484ff9640 1 -- 172.21.15.106:0/2272080294 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13200 (secure 0 0 0) 0x7ff48007bc80 con 0x7ff488151330 2024-04-13T03:10:34.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:34.725 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":35,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:31.768924+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:34.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 msgr2=0x7ff47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff478005590 tx=0x7ff478009040 comp rx=0 tx=0).stop 2024-04-13T03:10:34.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 msgr2=0x7ff4880c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff480009f90 tx=0x7ff480009750 comp rx=0 tx=0).stop 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 shutdown_connections 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff488151330 0x7ff4880c0d20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48815d940 0x7ff4880c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 --2- 172.21.15.106:0/2272080294 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff488150f50 0x7ff48815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 >> 172.21.15.106:0/2272080294 conn(0x7ff4880b1f10 msgr2=0x7ff48806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 shutdown_connections 2024-04-13T03:10:34.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.726+0000 7ff48e91e640 1 -- 172.21.15.106:0/2272080294 wait complete. 2024-04-13T03:10:34.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:10:34.739 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.841+0000 7f817466d640 1 Processor -- start 2024-04-13T03:10:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.841+0000 7f817466d640 1 -- start start 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.841+0000 7f817466d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f816c15b4e0 0x7f816c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f817466d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f816c0b5390 0x7f816c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f817466d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f816c0b3690 con 0x7f816c0b4730 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f817466d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f816c0b3970 con 0x7f816c15b4e0 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f817466d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f816c0b3c50 con 0x7f816c0b5390 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f816c15b4e0 0x7f816c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f816c15b4e0 0x7f816c15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56480/0 (socket says 172.21.15.106:56480) 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 -- 172.21.15.106:0/3146724287 learned_addr learned my addr 172.21.15.106:0/3146724287 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172c0a640 1 --2- 172.21.15.106:0/3146724287 >> v2:172.21.15.114:3300/0 conn(0x7f816c0b5390 0x7f816c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 -- 172.21.15.106:0/3146724287 >> v2:172.21.15.114:3300/0 conn(0x7f816c0b5390 msgr2=0x7f816c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 --2- 172.21.15.106:0/3146724287 >> v2:172.21.15.114:3300/0 conn(0x7f816c0b5390 0x7f816c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 -- 172.21.15.106:0/3146724287 >> v1:172.21.15.114:6789/0 conn(0x7f816c0b4730 legacy=0x7f816c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.842+0000 7f8172409640 1 -- 172.21.15.106:0/3146724287 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0b3f30 con 0x7f816c15b4e0 2024-04-13T03:10:34.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f8172409640 1 --2- 172.21.15.106:0/3146724287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c15fb60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f816c0cd9e0 tx=0x7f816802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b0514d66988bb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f8171407640 1 -- 172.21.15.106:0/3146724287 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168035020 con 0x7f816c15b4e0 2024-04-13T03:10:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f8171407640 1 -- 172.21.15.106:0/3146724287 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168031870 con 0x7f816c15b4e0 2024-04-13T03:10:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f8171407640 1 -- 172.21.15.106:0/3146724287 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168031c40 con 0x7f816c15b4e0 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 -- 172.21.15.106:0/3146724287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 msgr2=0x7f816c15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 --2- 172.21.15.106:0/3146724287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c15fb60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f816c0cd9e0 tx=0x7f816802b910 comp rx=0 tx=0).stop 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 -- 172.21.15.106:0/3146724287 shutdown_connections 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 --2- 172.21.15.106:0/3146724287 >> v2:172.21.15.114:3300/0 conn(0x7f816c0b5390 0x7f816c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 --2- 172.21.15.106:0/3146724287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c15fb60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.843+0000 7f817466d640 1 -- 172.21.15.106:0/3146724287 >> 172.21.15.106:0/3146724287 conn(0x7f816c0b2600 msgr2=0x7f816c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.844+0000 7f817466d640 1 -- 172.21.15.106:0/3146724287 shutdown_connections 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.844+0000 7f817466d640 1 -- 172.21.15.106:0/3146724287 wait complete. 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 Processor -- start 2024-04-13T03:10:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 -- start start 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f816c0b4730 0x7f816c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 0x7f816c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f816c06d0c0 con 0x7f816c15b4e0 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0b32d0 con 0x7f816c0b5390 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.845+0000 7f817466d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f816c0cf570 con 0x7f816c0b4730 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56492/0 (socket says 172.21.15.106:56492) 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 -- 172.21.15.106:0/3158422835 learned_addr learned my addr 172.21.15.106:0/3158422835 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172409640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f816c0b4730 0x7f816c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8171c08640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 0x7f816c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 -- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f816c0b4730 msgr2=0x7f816c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f816c0b4730 0x7f816c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 -- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 msgr2=0x7f816c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 0x7f816c0cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 -- 172.21.15.106:0/3158422835 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0cf850 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8171c08640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 0x7f816c0cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8172c0a640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f816400cd40 tx=0x7f816400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164003b90 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8164003d30 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c0cfb30 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164004240 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.846+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c0af270 con 0x7f816c15b4e0 2024-04-13T03:10:34.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.847+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c068a10 con 0x7f816c15b4e0 2024-04-13T03:10:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.848+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81640043e0 con 0x7f816c15b4e0 2024-04-13T03:10:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.848+0000 7f8162ffd640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 0x7f815006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.848+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f816408c2b0 con 0x7f816c15b4e0 2024-04-13T03:10:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.848+0000 7f8172409640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 0x7f815006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:34.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.849+0000 7f8172409640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 0x7f815006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8168036000 tx=0x7f816803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:34.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:34.852+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f816405a930 con 0x7f816c15b4e0 2024-04-13T03:10:35.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.025+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f816c0cfd00 con 0x7f816c15b4e0 2024-04-13T03:10:35.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.026+0000 7f8162ffd640 1 -- 172.21.15.106:0/3158422835 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f816405a230 con 0x7f816c15b4e0 2024-04-13T03:10:35.027 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.027+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 msgr2=0x7f815006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 0x7f815006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8168036000 tx=0x7f816803c040 comp rx=0 tx=0).stop 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 msgr2=0x7f816c0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f816400cd40 tx=0x7f816400a660 comp rx=0 tx=0).stop 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 shutdown_connections 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f815006c8f0 0x7f815006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f816c15b4e0 0x7f816c0cbe50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f816c0b5390 0x7f816c0cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 --2- 172.21.15.106:0/3158422835 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f816c0b4730 0x7f816c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 >> 172.21.15.106:0/3158422835 conn(0x7f816c0b2600 msgr2=0x7f816c06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:35.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 shutdown_connections 2024-04-13T03:10:35.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.028+0000 7f817466d640 1 -- 172.21.15.106:0/3158422835 wait complete. 2024-04-13T03:10:35.039 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:10:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.138+0000 7fdb1655e640 1 Processor -- start 2024-04-13T03:10:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.138+0000 7fdb1655e640 1 -- start start 2024-04-13T03:10:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb1655e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb10160c10 0x7fdb1006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb1655e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 0x7fdb1006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb1655e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1015d690 con 0x7fdb10156b50 2024-04-13T03:10:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb1655e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdb10161080 con 0x7fdb10160c10 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb1655e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1015d970 con 0x7fdb1006e2b0 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb10160c10 0x7fdb1006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdb10156b50 0x7fdb1015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54932/0 (socket says 172.21.15.106:54932) 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0ffff640 1 -- 172.21.15.106:0/2055447187 learned_addr learned my addr 172.21.15.106:0/2055447187 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/2055447187 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 msgr2=0x7fdb1006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb14afb640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 0x7fdb1006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0effd640 1 -- 172.21.15.106:0/2055447187 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022564533 0 0) 0x7fdb1015d690 con 0x7fdb10156b50 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0f7fe640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 0x7fdb1006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/2055447187 >> v1:172.21.15.106:6789/0 conn(0x7fdb10156b50 legacy=0x7fdb1015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/2055447187 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb10160410 con 0x7fdb10160c10 2024-04-13T03:10:35.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.139+0000 7fdb14afb640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 0x7fdb1006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb0f7fe640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10160c10 0x7fdb1006dd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb040097a0 tx=0x7fdb0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb027a8633a981 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb0effd640 1 -- 172.21.15.106:0/2055447187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04035020 con 0x7fdb10160c10 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb0effd640 1 -- 172.21.15.106:0/2055447187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb04031830 con 0x7fdb10160c10 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb0effd640 1 -- 172.21.15.106:0/2055447187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04031c00 con 0x7fdb10160c10 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb1655e640 1 -- 172.21.15.106:0/2055447187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10160c10 msgr2=0x7fdb1006dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.140+0000 7fdb1655e640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10160c10 0x7fdb1006dd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb040097a0 tx=0x7fdb0402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 -- 172.21.15.106:0/2055447187 shutdown_connections 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb1006e2b0 0x7fdb1006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 --2- 172.21.15.106:0/2055447187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10160c10 0x7fdb1006dd70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 -- 172.21.15.106:0/2055447187 >> 172.21.15.106:0/2055447187 conn(0x7fdb100b1f10 msgr2=0x7fdb100b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 -- 172.21.15.106:0/2055447187 shutdown_connections 2024-04-13T03:10:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.141+0000 7fdb1655e640 1 -- 172.21.15.106:0/2055447187 wait complete. 2024-04-13T03:10:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.142+0000 7fdb1655e640 1 Processor -- start 2024-04-13T03:10:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.142+0000 7fdb1655e640 1 -- start start 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 0x7fdb10159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 0x7fdb1006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10156f30 con 0x7fdb10156b50 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1015d320 con 0x7fdb1006e2b0 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb101f3d30 con 0x7fdb10160c10 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56512/0 (socket says 172.21.15.106:56512) 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/1023918211 learned_addr learned my addr 172.21.15.106:0/1023918211 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 msgr2=0x7fdb1006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0ffff640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 0x7fdb10159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb14afb640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 0x7fdb1006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 0x7fdb1006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 msgr2=0x7fdb10159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 0x7fdb10159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 -- 172.21.15.106:0/1023918211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1015fed0 con 0x7fdb10156b50 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb14afb640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 0x7fdb1006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0ffff640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 0x7fdb10159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0f7fe640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdb0402bee0 tx=0x7fdb040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04041070 con 0x7fdb10156b50 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0403a970 con 0x7fdb10156b50 2024-04-13T03:10:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb101601b0 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04035040 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.143+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb100c77d0 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.144+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb10068a10 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.145+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb04038020 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.145+0000 7fdb0cff9640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.145+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fdb040ad470 con 0x7fdb10156b50 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.145+0000 7fdb0ffff640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.146+0000 7fdb0ffff640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 0x7fdafc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000f60 tx=0x7fdb00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.150+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb0407ba60 con 0x7fdb10156b50 2024-04-13T03:10:35.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.327+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdb1006ba90 con 0x7fdb10156b50 2024-04-13T03:10:35.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.327+0000 7fdb0cff9640 1 -- 172.21.15.106:0/1023918211 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7fdb0407b360 con 0x7fdb10156b50 2024-04-13T03:10:35.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:10:35.329 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:10:35.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 msgr2=0x7fdafc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 0x7fdafc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000f60 tx=0x7fdb00009040 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 msgr2=0x7fdb1015f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdb0402bee0 tx=0x7fdb040046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 shutdown_connections 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdafc06c920 0x7fdafc06ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb10160c10 0x7fdb1006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb10156b50 0x7fdb1015f680 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 --2- 172.21.15.106:0/1023918211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb1006e2b0 0x7fdb10159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 >> 172.21.15.106:0/1023918211 conn(0x7fdb100b1f10 msgr2=0x7fdb100b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 shutdown_connections 2024-04-13T03:10:35.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.329+0000 7fdb1655e640 1 -- 172.21.15.106:0/1023918211 wait complete. 2024-04-13T03:10:35.341 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:10:35.341 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:10:35.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.443+0000 7f0d652c4640 1 Processor -- start 2024-04-13T03:10:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 -- start start 2024-04-13T03:10:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 0x7f0d6006f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0d6006f9a0 0x7f0d600b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d6015b0f0 con 0x7f0d600b4830 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600b4d40 con 0x7f0d6006f9a0 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d652c4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6006e980 con 0x7f0d6015ad10 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d5e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0d6006f9a0 0x7f0d600b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d5e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0d6006f9a0 0x7f0d600b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56516/0 (socket says 172.21.15.106:56516) 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d5f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0d600b4830 0x7f0d6006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54938/0 (socket says 172.21.15.106:54938) 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.444+0000 7f0d5e7fc640 1 -- 172.21.15.106:0/2372383705 learned_addr learned my addr 172.21.15.106:0/2372383705 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5effd640 1 --2- 172.21.15.106:0/2372383705 >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 0x7f0d6006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5e7fc640 1 -- 172.21.15.106:0/2372383705 >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 msgr2=0x7f0d6006f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5e7fc640 1 --2- 172.21.15.106:0/2372383705 >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 0x7f0d6006f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5e7fc640 1 -- 172.21.15.106:0/2372383705 >> v1:172.21.15.106:6789/0 conn(0x7f0d600b4830 legacy=0x7f0d6006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5dffb640 1 -- 172.21.15.106:0/2372383705 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093141439 0 0) 0x7f0d6015b0f0 con 0x7f0d600b4830 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5e7fc640 1 -- 172.21.15.106:0/2372383705 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6006ec60 con 0x7f0d6006f9a0 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5effd640 1 --2- 172.21.15.106:0/2372383705 >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 0x7f0d6006f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.445+0000 7f0d5e7fc640 1 --2- 172.21.15.106:0/2372383705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d600b42f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0d6006d780 tx=0x7f0d4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27292e4e649a39a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d5dffb640 1 -- 172.21.15.106:0/2372383705 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48034070 con 0x7f0d6006f9a0 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d5dffb640 1 -- 172.21.15.106:0/2372383705 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48030860 con 0x7f0d6006f9a0 2024-04-13T03:10:35.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d5dffb640 1 -- 172.21.15.106:0/2372383705 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48030c30 con 0x7f0d6006f9a0 2024-04-13T03:10:35.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 -- 172.21.15.106:0/2372383705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 msgr2=0x7f0d600b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2372383705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d600b42f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0d6006d780 tx=0x7f0d4802b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 -- 172.21.15.106:0/2372383705 shutdown_connections 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2372383705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d600b42f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2372383705 >> v2:172.21.15.114:3300/0 conn(0x7f0d6015ad10 0x7f0d6006f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 -- 172.21.15.106:0/2372383705 >> 172.21.15.106:0/2372383705 conn(0x7f0d600d0370 msgr2=0x7f0d600d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 -- 172.21.15.106:0/2372383705 shutdown_connections 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.446+0000 7f0d652c4640 1 -- 172.21.15.106:0/2372383705 wait complete. 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.447+0000 7f0d652c4640 1 Processor -- start 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 -- start start 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d600b4830 0x7f0d60139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d6015ad10 0x7f0d601527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6006c880 con 0x7f0d6006f9a0 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600b3dd0 con 0x7f0d600b4830 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d652c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6006ca00 con 0x7f0d6015ad10 2024-04-13T03:10:35.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56528/0 (socket says 172.21.15.106:56528) 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.448+0000 7f0d5effd640 1 -- 172.21.15.106:0/2642947727 learned_addr learned my addr 172.21.15.106:0/2642947727 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5e7fc640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d600b4830 0x7f0d60139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5f7fe640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d6015ad10 0x7f0d601527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 -- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d6015ad10 msgr2=0x7f0d601527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d6015ad10 0x7f0d601527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 -- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d600b4830 msgr2=0x7f0d60139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d600b4830 0x7f0d60139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 -- 172.21.15.106:0/2642947727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d600cdb90 con 0x7f0d6006f9a0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d5effd640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007f20 tx=0x7f0d5000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5001a070 con 0x7f0d6006f9a0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50003b90 con 0x7f0d6006f9a0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d600cde70 con 0x7f0d6006f9a0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50004030 con 0x7f0d6006f9a0 2024-04-13T03:10:35.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.449+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d600ce800 con 0x7f0d6006f9a0 2024-04-13T03:10:35.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.450+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6014f530 con 0x7f0d6006f9a0 2024-04-13T03:10:35.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.450+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d5001d070 con 0x7f0d6006f9a0 2024-04-13T03:10:35.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.451+0000 7f0d577fe640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:35.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.451+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f0d5008bab0 con 0x7f0d6006f9a0 2024-04-13T03:10:35.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.451+0000 7f0d5e7fc640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:35.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.451+0000 7f0d5e7fc640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 0x7f0d4006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d48036000 tx=0x7f0d4803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:35.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.455+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d50010070 con 0x7f0d6006f9a0 2024-04-13T03:10:35.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:35.631+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0d60068a10 con 0x7f0d6006f9a0 2024-04-13T03:10:36.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.056+0000 7f0d577fe640 1 -- 172.21.15.106:0/2642947727 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f0d50059ae0 con 0x7f0d6006f9a0 2024-04-13T03:10:36.057 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:10:36.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.058+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 msgr2=0x7f0d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.058+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 0x7f0d4006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d48036000 tx=0x7f0d4803b040 comp rx=0 tx=0).stop 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.058+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 msgr2=0x7f0d6014eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.058+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007f20 tx=0x7f0d5000d560 comp rx=0 tx=0).stop 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 shutdown_connections 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d6015ad10 0x7f0d601527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d600b4830 0x7f0d60139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 --2- 172.21.15.106:0/2642947727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d6006f9a0 0x7f0d6014eb30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 >> 172.21.15.106:0/2642947727 conn(0x7f0d600d0370 msgr2=0x7f0d60153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 shutdown_connections 2024-04-13T03:10:36.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.059+0000 7f0d652c4640 1 -- 172.21.15.106:0/2642947727 wait complete. 2024-04-13T03:10:36.074 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:10:36.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.180+0000 7fb09a7c8640 1 Processor -- start 2024-04-13T03:10:36.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.180+0000 7fb09a7c8640 1 -- start start 2024-04-13T03:10:36.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb09a7c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb0940b1c30 0x7fb09415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb09a7c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 0x7fb0940704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb09a7c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0940b5cf0 con 0x7fb0940709e0 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb09a7c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb0940b5fd0 con 0x7fb0940b1c30 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb09a7c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb0940b62b0 con 0x7fb0940b54a0 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb0937fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 0x7fb0940704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb0940b1c30 0x7fb09415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb0940b1c30 0x7fb09415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:59984/0 (socket says 172.21.15.106:59984) 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb098d65640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb0940709e0 0x7fb094162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:54360/0 (socket says 172.21.15.106:54360) 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 -- 172.21.15.106:0/1546500210 learned_addr learned my addr 172.21.15.106:0/1546500210 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 -- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 msgr2=0x7fb0940704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 0x7fb0940704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 -- 172.21.15.106:0/1546500210 >> v1:172.21.15.114:6789/0 conn(0x7fb0940709e0 legacy=0x7fb094162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb093fff640 1 -- 172.21.15.106:0/1546500210 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0940b6590 con 0x7fb0940b1c30 2024-04-13T03:10:36.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.181+0000 7fb0937fe640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 0x7fb0940704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.182+0000 7fb093fff640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940b1c30 0x7fb09415ee40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0840098a0 tx=0x7fb08402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44cd3fc7eaf8d356 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.182+0000 7fb092ffd640 1 -- 172.21.15.106:0/1546500210 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084034070 con 0x7fb0940b1c30 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.182+0000 7fb092ffd640 1 -- 172.21.15.106:0/1546500210 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb084030840 con 0x7fb0940b1c30 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.182+0000 7fb092ffd640 1 -- 172.21.15.106:0/1546500210 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084030c10 con 0x7fb0940b1c30 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 -- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940b1c30 msgr2=0x7fb09415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940b1c30 0x7fb09415ee40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0840098a0 tx=0x7fb08402b980 comp rx=0 tx=0).stop 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 -- 172.21.15.106:0/1546500210 shutdown_connections 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b54a0 0x7fb0940704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/1546500210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940b1c30 0x7fb09415ee40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 -- 172.21.15.106:0/1546500210 >> 172.21.15.106:0/1546500210 conn(0x7fb094071870 msgr2=0x7fb094071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:36.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 -- 172.21.15.106:0/1546500210 shutdown_connections 2024-04-13T03:10:36.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.183+0000 7fb09a7c8640 1 -- 172.21.15.106:0/1546500210 wait complete. 2024-04-13T03:10:36.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.184+0000 7fb09a7c8640 1 Processor -- start 2024-04-13T03:10:36.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.184+0000 7fb09a7c8640 1 -- start start 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 0x7fb094160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 0x7fb094151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb09406e7f0 con 0x7fb0940b54a0 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb09406e670 con 0x7fb0940709e0 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb09a7c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb09406e970 con 0x7fb0940b1c30 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56536/0 (socket says 172.21.15.106:56536) 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 -- 172.21.15.106:0/300815222 learned_addr learned my addr 172.21.15.106:0/300815222 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb093fff640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 0x7fb094160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb0937fe640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 0x7fb094151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 -- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 msgr2=0x7fb094151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 0x7fb094151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 -- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 msgr2=0x7fb094160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 0x7fb094160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb098d65640 1 -- 172.21.15.106:0/300815222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb094162920 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb0937fe640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 0x7fb094151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.185+0000 7fb093fff640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 0x7fb094160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb098d65640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c780 tx=0x7fb08c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c017070 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08c00a640 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb094161880 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00aa30 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.186+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb094162120 con 0x7fb0940b54a0 2024-04-13T03:10:36.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.187+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb094068a10 con 0x7fb0940b54a0 2024-04-13T03:10:36.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.187+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb08c01a070 con 0x7fb0940b54a0 2024-04-13T03:10:36.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.188+0000 7fb090ff9640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 0x7fb08006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:36.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.188+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7fb08c08b0a0 con 0x7fb0940b54a0 2024-04-13T03:10:36.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.188+0000 7fb093fff640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 0x7fb08006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:36.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.188+0000 7fb093fff640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 0x7fb08006ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb084036000 tx=0x7fb08403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:36.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.193+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb094162920 con 0x7fb0940b54a0 2024-04-13T03:10:36.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:36.367+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb0940711f0 con 0x7fb0940b54a0 2024-04-13T03:10:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.064+0000 7fb090ff9640 1 -- 172.21.15.106:0/300815222 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7fb08c058e50 con 0x7fb0940b54a0 2024-04-13T03:10:37.066 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:10:37.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 msgr2=0x7fb08006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:37.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 0x7fb08006ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb084036000 tx=0x7fb08403b040 comp rx=0 tx=0).stop 2024-04-13T03:10:37.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 msgr2=0x7fb094140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c780 tx=0x7fb08c00cc40 comp rx=0 tx=0).stop 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 shutdown_connections 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb08006c920 0x7fb08006ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb0940b1c30 0x7fb094151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0940b54a0 0x7fb094140ff0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 --2- 172.21.15.106:0/300815222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0940709e0 0x7fb094160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 >> 172.21.15.106:0/300815222 conn(0x7fb094071870 msgr2=0x7fb0940d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 shutdown_connections 2024-04-13T03:10:37.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.067+0000 7fb09a7c8640 1 -- 172.21.15.106:0/300815222 wait complete. 2024-04-13T03:10:37.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:10:37.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.191+0000 7f83a6660640 1 Processor -- start 2024-04-13T03:10:37.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.191+0000 7f83a6660640 1 -- start start 2024-04-13T03:10:37.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 0x7f83a015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 0x7f83a00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83a00709e0 0x7f83a0162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f83a00b5cf0 con 0x7f83a00709e0 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f83a00b5fd0 con 0x7f83a00b1c30 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a6660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00b62b0 con 0x7f83a00b54a0 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a4bfd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83a00709e0 0x7f83a0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a4bfd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83a00709e0 0x7f83a0162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56542/0 (socket says 172.21.15.106:56542) 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a4bfd640 1 -- 172.21.15.106:0/1850488527 learned_addr learned my addr 172.21.15.106:0/1850488527 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f839f7fe640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 0x7f83a00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f839ffff640 1 --2- 172.21.15.106:0/1850488527 >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 0x7f83a015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a4bfd640 1 -- 172.21.15.106:0/1850488527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 msgr2=0x7f83a00704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:37.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.192+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 0x7f83a00704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f83a4bfd640 1 -- 172.21.15.106:0/1850488527 >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 msgr2=0x7f83a015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/1850488527 >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 0x7f83a015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f83a4bfd640 1 -- 172.21.15.106:0/1850488527 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00b6590 con 0x7f83a00709e0 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f839f7fe640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 0x7f83a00704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f839ffff640 1 --2- 172.21.15.106:0/1850488527 >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 0x7f83a015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00709e0 0x7f83a0162430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8398009e50 tx=0x7f839802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78abcff84d180b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f839effd640 1 -- 172.21.15.106:0/1850488527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839802ca70 con 0x7f83a00709e0 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f839effd640 1 -- 172.21.15.106:0/1850488527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839802cc10 con 0x7f83a00709e0 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.193+0000 7f839effd640 1 -- 172.21.15.106:0/1850488527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980343f0 con 0x7f83a00709e0 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 -- 172.21.15.106:0/1850488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00709e0 msgr2=0x7f83a0162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00709e0 0x7f83a0162430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8398009e50 tx=0x7f839802bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 -- 172.21.15.106:0/1850488527 shutdown_connections 2024-04-13T03:10:37.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00709e0 0x7f83a0162430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 --2- 172.21.15.106:0/1850488527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b54a0 0x7f83a00704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 --2- 172.21.15.106:0/1850488527 >> v2:172.21.15.114:3300/0 conn(0x7f83a00b1c30 0x7f83a015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 -- 172.21.15.106:0/1850488527 >> 172.21.15.106:0/1850488527 conn(0x7f83a0071870 msgr2=0x7f83a0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 -- 172.21.15.106:0/1850488527 shutdown_connections 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.194+0000 7f83a6660640 1 -- 172.21.15.106:0/1850488527 wait complete. 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.195+0000 7f83a6660640 1 Processor -- start 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.195+0000 7f83a6660640 1 -- start start 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83a00709e0 0x7f83a014e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 0x7f83a00b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0162af0 con 0x7f83a00b54a0 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0162970 con 0x7f83a00709e0 2024-04-13T03:10:37.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a6660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83a0162c70 con 0x7f83a00b1c30 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56544/0 (socket says 172.21.15.106:56544) 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 -- 172.21.15.106:0/2375104510 learned_addr learned my addr 172.21.15.106:0/2375104510 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f839f7fe640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 0x7f83a00b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f839ffff640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83a00709e0 0x7f83a014e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 -- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 msgr2=0x7f83a00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 0x7f83a00b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 -- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83a00709e0 msgr2=0x7f83a014e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83a00709e0 0x7f83a014e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.196+0000 7f83a4bfd640 1 -- 172.21.15.106:0/2375104510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a014ee10 con 0x7f83a00b54a0 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f839f7fe640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 0x7f83a00b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f83a4bfd640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8398009f70 tx=0x7f8398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:37.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398043020 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839803e450 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a00aafc0 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398009510 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.197+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a00ce4a0 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.198+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a0068a10 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.199+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8398046020 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.199+0000 7f839cff9640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 0x7f838c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.199+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f83980ae3f0 con 0x7f83a00b54a0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.199+0000 7f839ffff640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 0x7f838c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:37.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.199+0000 7f839ffff640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 0x7f838c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8390006fd0 tx=0x7f8390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:37.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.204+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f839804d030 con 0x7f83a00b54a0 2024-04-13T03:10:37.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:37.419+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f83a015d8e0 con 0x7f83a00b54a0 2024-04-13T03:10:38.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.066+0000 7f839cff9640 1 -- 172.21.15.106:0/2375104510 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f83a014ee10 con 0x7f83a00b54a0 2024-04-13T03:10:38.067 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 5 and data pool 6 2024-04-13T03:10:38.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 msgr2=0x7f838c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 0x7f838c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8390006fd0 tx=0x7f8390008040 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 msgr2=0x7f83a015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8398009f70 tx=0x7f8398004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 shutdown_connections 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f838c06c8f0 0x7f838c06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83a00b1c30 0x7f83a00b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a00b54a0 0x7f83a015de00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 --2- 172.21.15.106:0/2375104510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83a00709e0 0x7f83a014e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 >> 172.21.15.106:0/2375104510 conn(0x7f83a0071870 msgr2=0x7f83a0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 shutdown_connections 2024-04-13T03:10:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.068+0000 7f83a6660640 1 -- 172.21.15.106:0/2375104510 wait complete. 2024-04-13T03:10:38.088 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.191+0000 7f7853836640 1 Processor -- start 2024-04-13T03:10:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.191+0000 7f7853836640 1 -- start start 2024-04-13T03:10:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7853836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f784c15e9b0 0x7f784c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7853836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15ed90 0x7f784c0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7853836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f784c0cfd20 con 0x7f784c0b3d60 2024-04-13T03:10:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7853836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f784c162af0 con 0x7f784c15e9b0 2024-04-13T03:10:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7853836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f784c162dd0 con 0x7f784c15ed90 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f78515d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f784c15e9b0 0x7f784c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f78515d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f784c15e9b0 0x7f784c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56560/0 (socket says 172.21.15.106:56560) 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f78515d2640 1 -- 172.21.15.106:0/2890040800 learned_addr learned my addr 172.21.15.106:0/2890040800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.192+0000 7f7850dd1640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15ed90 0x7f784c0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f78515d2640 1 -- 172.21.15.106:0/2890040800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15ed90 msgr2=0x7f784c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f78515d2640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15ed90 0x7f784c0d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f78515d2640 1 -- 172.21.15.106:0/2890040800 >> v1:172.21.15.114:6789/0 conn(0x7f784c0b3d60 legacy=0x7f784c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f78515d2640 1 -- 172.21.15.106:0/2890040800 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1630b0 con 0x7f784c15e9b0 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f78515d2640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c15e9b0 0x7f784c0d2300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7848009e50 tx=0x7f784802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c7c41a9d1c094d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.193+0000 7f7843fff640 1 -- 172.21.15.106:0/2890040800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784802cb90 con 0x7f784c15e9b0 2024-04-13T03:10:38.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7843fff640 1 -- 172.21.15.106:0/2890040800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784802cd30 con 0x7f784c15e9b0 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7843fff640 1 -- 172.21.15.106:0/2890040800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848034610 con 0x7f784c15e9b0 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 -- 172.21.15.106:0/2890040800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c15e9b0 msgr2=0x7f784c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c15e9b0 0x7f784c0d2300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7848009e50 tx=0x7f784802bff0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 -- 172.21.15.106:0/2890040800 shutdown_connections 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15ed90 0x7f784c0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 --2- 172.21.15.106:0/2890040800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c15e9b0 0x7f784c0d2300 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 -- 172.21.15.106:0/2890040800 >> 172.21.15.106:0/2890040800 conn(0x7f784c08bf10 msgr2=0x7f784c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 -- 172.21.15.106:0/2890040800 shutdown_connections 2024-04-13T03:10:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.194+0000 7f7853836640 1 -- 172.21.15.106:0/2890040800 wait complete. 2024-04-13T03:10:38.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.195+0000 7f7853836640 1 Processor -- start 2024-04-13T03:10:38.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.195+0000 7f7853836640 1 -- start start 2024-04-13T03:10:38.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15e9b0 0x7f784c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f784c14da00 0x7f784c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f784c162800 con 0x7f784c0b3d60 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f784c162680 con 0x7f784c14da00 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7853836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f784c06eb20 con 0x7f784c15e9b0 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56572/0 (socket says 172.21.15.106:56572) 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 -- 172.21.15.106:0/2710805279 learned_addr learned my addr 172.21.15.106:0/2710805279 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7850dd1640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15e9b0 0x7f784c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f7851dd3640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f784c14da00 0x7f784c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 -- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15e9b0 msgr2=0x7f784c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15e9b0 0x7f784c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 -- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f784c14da00 msgr2=0x7f784c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f784c14da00 0x7f784c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.196+0000 7f78515d2640 1 -- 172.21.15.106:0/2710805279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c160940 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f78515d2640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7848000f80 tx=0x7f7848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848043020 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784803e450 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848009510 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c160c20 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.197+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1611e0 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.198+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c068a10 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.198+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7848046020 con 0x7f784c0b3d60 2024-04-13T03:10:38.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.199+0000 7f7841ffb640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 0x7f783006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.199+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f78480ae300 con 0x7f784c0b3d60 2024-04-13T03:10:38.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.199+0000 7f7850dd1640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 0x7f783006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.199+0000 7f7850dd1640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 0x7f783006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f784c06d610 tx=0x7f783c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.203+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f784804d030 con 0x7f784c0b3d60 2024-04-13T03:10:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.378+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f784c0d53a0 con 0x7f784c0b3d60 2024-04-13T03:10:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.379+0000 7f7841ffb640 1 -- 172.21.15.106:0/2710805279 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7f784807bf50 con 0x7f784c0b3d60 2024-04-13T03:10:38.381 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:38.381 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":38,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:37.420950+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.381+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 msgr2=0x7f783006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 0x7f783006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f784c06d610 tx=0x7f783c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 msgr2=0x7f784c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7848000f80 tx=0x7f7848004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 shutdown_connections 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f783006c8f0 0x7f783006edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f784c15e9b0 0x7f784c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f784c14da00 0x7f784c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 --2- 172.21.15.106:0/2710805279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784c0b3d60 0x7f784c1601d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 >> 172.21.15.106:0/2710805279 conn(0x7f784c08bf10 msgr2=0x7f784c15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 shutdown_connections 2024-04-13T03:10:38.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.382+0000 7f7853836640 1 -- 172.21.15.106:0/2710805279 wait complete. 2024-04-13T03:10:38.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.493+0000 7fe4b4d5e640 1 Processor -- start 2024-04-13T03:10:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.493+0000 7fe4b4d5e640 1 -- start start 2024-04-13T03:10:38.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4b4d5e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4b00d1c00 0x7fe4b00d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4b4d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 0x7fe4b01688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4b4d5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b00d3530 con 0x7fe4b01622d0 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4b4d5e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b01626b0 con 0x7fe4b00d1c00 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4b4d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00d0b60 con 0x7fe4b00b8080 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4b00d1c00 0x7fe4b00d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4b00d1c00 0x7fe4b00d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56574/0 (socket says 172.21.15.106:56574) 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 -- 172.21.15.106:0/95319715 learned_addr learned my addr 172.21.15.106:0/95319715 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4ae59c640 1 --1- 172.21.15.106:0/95319715 >> v1:172.21.15.106:6789/0 conn(0x7fe4b01622d0 0x7fe4b00b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54948/0 (socket says 172.21.15.106:54948) 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4aed9d640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 0x7fe4b01688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 -- 172.21.15.106:0/95319715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 msgr2=0x7fe4b01688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 0x7fe4b01688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 -- 172.21.15.106:0/95319715 >> v1:172.21.15.106:6789/0 conn(0x7fe4b01622d0 legacy=0x7fe4b00b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4add9b640 1 -- 172.21.15.106:0/95319715 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219331516 0 0) 0x7fe4b00d3530 con 0x7fe4b01622d0 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4a7fff640 1 -- 172.21.15.106:0/95319715 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b00d0e40 con 0x7fe4b00d1c00 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.494+0000 7fe4aed9d640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 0x7fe4b01688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b00d1c00 0x7fe4b00d04d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe498009870 tx=0x7fe49802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16b00a50a44dea server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4add9b640 1 -- 172.21.15.106:0/95319715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498034070 con 0x7fe4b00d1c00 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4add9b640 1 -- 172.21.15.106:0/95319715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe498030840 con 0x7fe4b00d1c00 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4add9b640 1 -- 172.21.15.106:0/95319715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498030c10 con 0x7fe4b00d1c00 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/95319715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b00d1c00 msgr2=0x7fe4b00d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.495+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b00d1c00 0x7fe4b00d04d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe498009870 tx=0x7fe49802b980 comp rx=0 tx=0).stop 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/95319715 shutdown_connections 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00b8080 0x7fe4b01688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/95319715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b00d1c00 0x7fe4b00d04d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/95319715 >> 172.21.15.106:0/95319715 conn(0x7fe4b00d25d0 msgr2=0x7fe4b00d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/95319715 shutdown_connections 2024-04-13T03:10:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.496+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/95319715 wait complete. 2024-04-13T03:10:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.497+0000 7fe4b4d5e640 1 Processor -- start 2024-04-13T03:10:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.497+0000 7fe4b4d5e640 1 -- start start 2024-04-13T03:10:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4b00b8080 0x7fe4b014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 0x7fe4b00c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00d16e0 con 0x7fe4b01622d0 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00d1400 con 0x7fe4b00b8080 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00cc6f0 con 0x7fe4b00d1c00 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56590/0 (socket says 172.21.15.106:56590) 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 -- 172.21.15.106:0/3023786955 learned_addr learned my addr 172.21.15.106:0/3023786955 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 0x7fe4b00c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 -- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 msgr2=0x7fe4b00c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 0x7fe4b00c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 -- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4b00b8080 msgr2=0x7fe4b014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4ae59c640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4b00b8080 0x7fe4b014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4b00b8080 0x7fe4b014cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 -- 172.21.15.106:0/3023786955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b00c70f0 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 0x7fe4b00c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4aed9d640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800cf20 tx=0x7fe4a800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a801a070 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a800d840 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800dc30 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b00c73d0 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.498+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b014c470 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.499+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b0068a10 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.500+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4a801d070 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.500+0000 7fe4a6ffd640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 0x7fe49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.500+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe4a808c510 con 0x7fe4b01622d0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.500+0000 7fe4ae59c640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 0x7fe49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.500+0000 7fe4ae59c640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 0x7fe49006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000f60 tx=0x7fe4a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.504+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4a805a840 con 0x7fe4b01622d0 2024-04-13T03:10:38.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.678+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4b015b6b0 con 0x7fe4b01622d0 2024-04-13T03:10:38.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.679+0000 7fe4a6ffd640 1 -- 172.21.15.106:0/3023786955 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7fe4a805a140 con 0x7fe4b01622d0 2024-04-13T03:10:38.680 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:38.681 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":38,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:37.420950+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 msgr2=0x7fe49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 0x7fe49006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000f60 tx=0x7fe4a0009040 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 msgr2=0x7fe4b00c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800cf20 tx=0x7fe4a800a7d0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 shutdown_connections 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe49006c8f0 0x7fe49006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4b00d1c00 0x7fe4b00c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4b01622d0 0x7fe4b00c8a00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 --2- 172.21.15.106:0/3023786955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4b00b8080 0x7fe4b014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 >> 172.21.15.106:0/3023786955 conn(0x7fe4b00d25d0 msgr2=0x7fe4b015fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 shutdown_connections 2024-04-13T03:10:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.681+0000 7fe4b4d5e640 1 -- 172.21.15.106:0/3023786955 wait complete. 2024-04-13T03:10:38.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:10:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.795+0000 7fb723c88640 1 Processor -- start 2024-04-13T03:10:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.795+0000 7fb723c88640 1 -- start start 2024-04-13T03:10:38.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb723c88640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb71c15f080 0x7fb71c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb723c88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c15fe40 con 0x7fb71c0704f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb723c88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c1633d0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb723c88640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb71c162f40 con 0x7fb71c15f080 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721a24640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb71c0704f0 0x7fb71c0708d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54952/0 (socket says 172.21.15.106:54952) 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721a24640 1 -- 172.21.15.106:0/1775814979 learned_addr learned my addr 172.21.15.106:0/1775814979 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721223640 1 --2- 172.21.15.106:0/1775814979 >> v2:172.21.15.114:3300/0 conn(0x7fb71c15f080 0x7fb71c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308785714 0 0) 0x7fb71c15fe40 con 0x7fb71c0704f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb708003520 con 0x7fb71c0704f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262968966 0 0) 0x7fb71c1633d0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb71c15fe40 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3818844010 0 0) 0x7fb708003520 con 0x7fb71c0704f0 2024-04-13T03:10:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721223640 1 -- 172.21.15.106:0/1775814979 >> v1:172.21.15.114:6789/0 conn(0x7fb71c0cf2f0 legacy=0x7fb71c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721223640 1 -- 172.21.15.106:0/1775814979 >> v1:172.21.15.106:6789/0 conn(0x7fb71c0704f0 legacy=0x7fb71c0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.796+0000 7fb721223640 1 -- 172.21.15.106:0/1775814979 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c162610 con 0x7fb71c15f080 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.797+0000 7fb721223640 1 --2- 172.21.15.106:0/1775814979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c15f4e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0097a0 tx=0x7fb70c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c77019f703b44376 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.797+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c038070 con 0x7fb71c15f080 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.797+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c031500 con 0x7fb71c15f080 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.797+0000 7fb720a22640 1 -- 172.21.15.106:0/1775814979 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140019f0 con 0x7fb71c15f080 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 -- 172.21.15.106:0/1775814979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 msgr2=0x7fb71c15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 --2- 172.21.15.106:0/1775814979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c15f4e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0097a0 tx=0x7fb70c02b820 comp rx=0 tx=0).stop 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 -- 172.21.15.106:0/1775814979 shutdown_connections 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 --2- 172.21.15.106:0/1775814979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c15f4e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 -- 172.21.15.106:0/1775814979 >> 172.21.15.106:0/1775814979 conn(0x7fb71c0d13f0 msgr2=0x7fb71c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 -- 172.21.15.106:0/1775814979 shutdown_connections 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.798+0000 7fb723c88640 1 -- 172.21.15.106:0/1775814979 wait complete. 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.799+0000 7fb723c88640 1 Processor -- start 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.799+0000 7fb723c88640 1 -- start start 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb71c0704f0 0x7fb71c1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c13be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c06ead0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c162bd0 con 0x7fb71c15f080 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c15fa20 con 0x7fb71c0704f0 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56600/0 (socket says 172.21.15.106:56600) 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 -- 172.21.15.106:0/2312461738 learned_addr learned my addr 172.21.15.106:0/2312461738 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:38.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721a24640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb71c0704f0 0x7fb71c1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb722225640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c13be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 -- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb71c0704f0 msgr2=0x7fb71c1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb71c0704f0 0x7fb71c1535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 -- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 msgr2=0x7fb71c13be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c13be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 -- 172.21.15.106:0/2312461738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c153d10 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb721223640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000c00 tx=0x7fb70c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c042020 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c0336f0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c038040 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.800+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c153ff0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.801+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1545b0 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.802+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c069120 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.802+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb70c045020 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.802+0000 7fb7127fc640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 0x7fb70006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.803+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb70c0ad560 con 0x7fb71c0cf2f0 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.803+0000 7fb721a24640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 0x7fb70006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:38.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.803+0000 7fb721a24640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 0x7fb70006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb714001350 tx=0x7fb714009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:38.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:38.807+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb70c04c030 con 0x7fb71c0cf2f0 2024-04-13T03:10:39.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.018+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb71c071330 con 0x7fb71c0cf2f0 2024-04-13T03:10:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.084+0000 7fb7127fc640 1 -- 172.21.15.106:0/2312461738 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fb70c030070 con 0x7fb71c0cf2f0 2024-04-13T03:10:39.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 msgr2=0x7fb70006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 0x7fb70006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb714001350 tx=0x7fb714009040 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 msgr2=0x7fb71c1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000c00 tx=0x7fb70c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 shutdown_connections 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb70006c8f0 0x7fb70006edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb71c15f080 0x7fb71c13be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb71c0cf2f0 0x7fb71c1502b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 --2- 172.21.15.106:0/2312461738 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb71c0704f0 0x7fb71c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 >> 172.21.15.106:0/2312461738 conn(0x7fb71c0d13f0 msgr2=0x7fb71c072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 shutdown_connections 2024-04-13T03:10:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.087+0000 7fb723c88640 1 -- 172.21.15.106:0/2312461738 wait complete. 2024-04-13T03:10:39.099 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:10:39.099 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:10:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.201+0000 7fe7f7eb7640 1 Processor -- start 2024-04-13T03:10:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.201+0000 7fe7f7eb7640 1 -- start start 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f7eb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f00af800 0x7fe7f015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f7eb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7f006f200 0x7fe7f006f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f7eb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f015c200 con 0x7fe7f015e830 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f7eb7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f006fba0 con 0x7fe7f006f200 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f7eb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00b5090 con 0x7fe7f00af800 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7f006f200 0x7fe7f006f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f6454640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe7f015e830 0x7fe7f015ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54972/0 (socket says 172.21.15.106:54972) 2024-04-13T03:10:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f6454640 1 -- 172.21.15.106:0/1871220142 learned_addr learned my addr 172.21.15.106:0/1871220142 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7f006f200 0x7fe7f006f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56612/0 (socket says 172.21.15.106:56612) 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5c53640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f00af800 0x7fe7f015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 -- 172.21.15.106:0/1871220142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f00af800 msgr2=0x7fe7f015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f00af800 0x7fe7f015bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 -- 172.21.15.106:0/1871220142 >> v1:172.21.15.106:6789/0 conn(0x7fe7f015e830 legacy=0x7fe7f015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f4c51640 1 -- 172.21.15.106:0/1871220142 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263254272 0 0) 0x7fe7e8001490 con 0x7fe7f015e830 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.202+0000 7fe7f5452640 1 -- 172.21.15.106:0/1871220142 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f00b3ff0 con 0x7fe7f006f200 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.203+0000 7fe7f5452640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f006f200 0x7fe7f006f660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097a0 tx=0x7fe7e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f01ba081b6ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.203+0000 7fe7f4c51640 1 -- 172.21.15.106:0/1871220142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0035020 con 0x7fe7f006f200 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.203+0000 7fe7f4c51640 1 -- 172.21.15.106:0/1871220142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e0031830 con 0x7fe7f006f200 2024-04-13T03:10:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.203+0000 7fe7f4c51640 1 -- 172.21.15.106:0/1871220142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0031c00 con 0x7fe7f006f200 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/1871220142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f006f200 msgr2=0x7fe7f006f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f006f200 0x7fe7f006f660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097a0 tx=0x7fe7e002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/1871220142 shutdown_connections 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f006f200 0x7fe7f006f660 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/1871220142 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f00af800 0x7fe7f015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/1871220142 >> 172.21.15.106:0/1871220142 conn(0x7fe7f00b2600 msgr2=0x7fe7f00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/1871220142 shutdown_connections 2024-04-13T03:10:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.204+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/1871220142 wait complete. 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.205+0000 7fe7f7eb7640 1 Processor -- start 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.205+0000 7fe7f7eb7640 1 -- start start 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7f006f200 0x7fe7f014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f015e830 0x7fe7f00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00b4890 con 0x7fe7f00af800 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00b45b0 con 0x7fe7f006f200 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00b4b70 con 0x7fe7f015e830 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56622/0 (socket says 172.21.15.106:56622) 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 -- 172.21.15.106:0/3827965159 learned_addr learned my addr 172.21.15.106:0/3827965159 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 -- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f015e830 msgr2=0x7fe7f00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f015e830 0x7fe7f00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 -- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7f006f200 msgr2=0x7fe7f014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7f006f200 0x7fe7f014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 -- 172.21.15.106:0/3827965159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f014c690 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f5452640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0034040 tx=0x7fe7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0040070 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e00399a0 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f00b0150 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0035040 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.206+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f00b0710 con 0x7fe7f00af800 2024-04-13T03:10:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.207+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0068a10 con 0x7fe7f00af800 2024-04-13T03:10:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.208+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7e0038020 con 0x7fe7f00af800 2024-04-13T03:10:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.208+0000 7fe7e67fc640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 0x7fe7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.208+0000 7fe7f5c53640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 0x7fe7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.208+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe7e00ad670 con 0x7fe7f00af800 2024-04-13T03:10:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.209+0000 7fe7f5c53640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 0x7fe7d406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0098f0 tx=0x7fe7ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.213+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7e007b920 con 0x7fe7f00af800 2024-04-13T03:10:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:39.425+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe7f006cfb0 con 0x7fe7f00af800 2024-04-13T03:10:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.089+0000 7fe7e67fc640 1 -- 172.21.15.106:0/3827965159 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fe7e007b220 con 0x7fe7f00af800 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 msgr2=0x7fe7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 0x7fe7d406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0098f0 tx=0x7fe7ec008040 comp rx=0 tx=0).stop 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 msgr2=0x7fe7f00cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0034040 tx=0x7fe7e00046c0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 shutdown_connections 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7d406c8f0 0x7fe7d406edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7f015e830 0x7fe7f00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7f00af800 0x7fe7f00cd9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 --2- 172.21.15.106:0/3827965159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7f006f200 0x7fe7f014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 >> 172.21.15.106:0/3827965159 conn(0x7fe7f00b2600 msgr2=0x7fe7f014df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 shutdown_connections 2024-04-13T03:10:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.092+0000 7fe7f7eb7640 1 -- 172.21.15.106:0/3827965159 wait complete. 2024-04-13T03:10:40.105 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:10:40.105 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.207+0000 7f6d1a515640 1 Processor -- start 2024-04-13T03:10:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.207+0000 7f6d1a515640 1 -- start start 2024-04-13T03:10:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d1a515640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d1406cb10 0x7f6d14072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d1a515640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d1406e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d1a515640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d140998d0 con 0x7f6d140cdf50 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d1a515640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6d14073020 con 0x7f6d1406cb10 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d1a515640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6d14073300 con 0x7f6d1406e150 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d13fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6d140cdf50 0x7f6d140ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54974/0 (socket says 172.21.15.106:54974) 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d13fff640 1 -- 172.21.15.106:0/1671652980 learned_addr learned my addr 172.21.15.106:0/1671652980 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/1671652980 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d1406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d137fe640 1 --2- 172.21.15.106:0/1671652980 >> v2:172.21.15.114:3300/0 conn(0x7f6d1406cb10 0x7f6d14072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844147840 0 0) 0x7f6d140998d0 con 0x7f6d140cdf50 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cfc005730 con 0x7f6d140cdf50 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d18ab2640 1 -- 172.21.15.106:0/1671652980 >> v2:172.21.15.114:3300/0 conn(0x7f6d1406cb10 msgr2=0x7f6d14072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/1671652980 >> v2:172.21.15.114:3300/0 conn(0x7f6d1406cb10 0x7f6d14072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.208+0000 7f6d18ab2640 1 -- 172.21.15.106:0/1671652980 >> v1:172.21.15.106:6789/0 conn(0x7f6d140cdf50 legacy=0x7f6d140ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.209+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3087216449 0 0) 0x7f6cfc005730 con 0x7f6d140cdf50 2024-04-13T03:10:40.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.209+0000 7f6d18ab2640 1 -- 172.21.15.106:0/1671652980 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140735e0 con 0x7f6d1406e150 2024-04-13T03:10:40.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.209+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/1671652980 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d1406e600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009d90 tx=0x7f6d0c02bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7fdb1cde17ed91 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.209+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c039070 con 0x7f6d1406e150 2024-04-13T03:10:40.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c02c8a0 con 0x7f6d1406e150 2024-04-13T03:10:40.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d12ffd640 1 -- 172.21.15.106:0/1671652980 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04001670 con 0x7f6d1406e150 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 -- 172.21.15.106:0/1671652980 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 msgr2=0x7f6d1406e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 --2- 172.21.15.106:0/1671652980 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d1406e600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009d90 tx=0x7f6d0c02bd70 comp rx=0 tx=0).stop 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 -- 172.21.15.106:0/1671652980 shutdown_connections 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 --2- 172.21.15.106:0/1671652980 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d1406e600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 --2- 172.21.15.106:0/1671652980 >> v2:172.21.15.114:3300/0 conn(0x7f6d1406cb10 0x7f6d14072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 -- 172.21.15.106:0/1671652980 >> 172.21.15.106:0/1671652980 conn(0x7f6d140b1f10 msgr2=0x7f6d140b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 -- 172.21.15.106:0/1671652980 shutdown_connections 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.210+0000 7f6d1a515640 1 -- 172.21.15.106:0/1671652980 wait complete. 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.211+0000 7f6d1a515640 1 Processor -- start 2024-04-13T03:10:40.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.211+0000 7f6d1a515640 1 -- start start 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1406cb10 0x7f6d140c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d141f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14072de0 con 0x7f6d140cdf50 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14072c60 con 0x7f6d1406cb10 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d1a515640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1406eb40 con 0x7f6d1406e150 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56624/0 (socket says 172.21.15.106:56624) 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 -- 172.21.15.106:0/466235028 learned_addr learned my addr 172.21.15.106:0/466235028 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:40.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d13fff640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1406cb10 0x7f6d140c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d137fe640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d141f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 -- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 msgr2=0x7f6d141f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d141f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 -- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1406cb10 msgr2=0x7f6d140c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1406cb10 0x7f6d140c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.212+0000 7f6d18ab2640 1 -- 172.21.15.106:0/466235028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141f0f70 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d18ab2640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009eb0 tx=0x7f6d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c044020 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c034c40 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d140c7580 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c039040 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.213+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d140c7b00 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.214+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d14068a10 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.214+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d0c047020 con 0x7f6d140cdf50 2024-04-13T03:10:40.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.215+0000 7f6d10ff9640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 0x7f6d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.215+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6d0c0ae220 con 0x7f6d140cdf50 2024-04-13T03:10:40.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.215+0000 7f6d13fff640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 0x7f6d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.215+0000 7f6d13fff640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 0x7f6d0006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007940 tx=0x7f6d04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.219+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d0c04e030 con 0x7f6d140cdf50 2024-04-13T03:10:40.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.429+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d140ce7c0 con 0x7f6d140cdf50 2024-04-13T03:10:40.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.430+0000 7f6d10ff9640 1 -- 172.21.15.106:0/466235028 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f6d141f0f70 con 0x7f6d140cdf50 2024-04-13T03:10:40.431 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:40.431 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:10:40.431 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 msgr2=0x7f6d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 0x7f6d0006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007940 tx=0x7f6d04009040 comp rx=0 tx=0).stop 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 msgr2=0x7f6d140c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009eb0 tx=0x7f6d0c004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 shutdown_connections 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6d0006c8f0 0x7f6d0006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d1406e150 0x7f6d141f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d140cdf50 0x7f6d140c8f90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 --2- 172.21.15.106:0/466235028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1406cb10 0x7f6d140c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 >> 172.21.15.106:0/466235028 conn(0x7f6d140b1f10 msgr2=0x7f6d140afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 shutdown_connections 2024-04-13T03:10:40.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.432+0000 7f6d1a515640 1 -- 172.21.15.106:0/466235028 wait complete. 2024-04-13T03:10:40.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 Processor -- start 2024-04-13T03:10:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 -- start start 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41a815e9b0 0x7f41a80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a815ed90 0x7f41a80d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a80cfd20 con 0x7f41a80b3d60 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f41a8162af0 con 0x7f41a815e9b0 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.543+0000 7f41ad636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f41a8162dd0 con 0x7f41a815ed90 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41a815e9b0 0x7f41a80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41a815e9b0 0x7f41a80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56640/0 (socket says 172.21.15.106:56640) 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 -- 172.21.15.106:0/97529503 learned_addr learned my addr 172.21.15.106:0/97529503 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a67fc640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a815ed90 0x7f41a80d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 -- 172.21.15.106:0/97529503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a815ed90 msgr2=0x7f41a80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a815ed90 0x7f41a80d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 -- 172.21.15.106:0/97529503 >> v1:172.21.15.114:6789/0 conn(0x7f41a80b3d60 legacy=0x7f41a80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.544+0000 7f41a6ffd640 1 -- 172.21.15.106:0/97529503 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a81630b0 con 0x7f41a815e9b0 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41a6ffd640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80d2300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4198009870 tx=0x7f419802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ba1e6dcd5f8240 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41a5ffb640 1 -- 172.21.15.106:0/97529503 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419802f840 con 0x7f41a815e9b0 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41a5ffb640 1 -- 172.21.15.106:0/97529503 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419802f9e0 con 0x7f41a815e9b0 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41a5ffb640 1 -- 172.21.15.106:0/97529503 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419802fdb0 con 0x7f41a815e9b0 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 -- 172.21.15.106:0/97529503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 msgr2=0x7f41a80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80d2300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4198009870 tx=0x7f419802ba20 comp rx=0 tx=0).stop 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 -- 172.21.15.106:0/97529503 shutdown_connections 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a815ed90 0x7f41a80d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 --2- 172.21.15.106:0/97529503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80d2300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 -- 172.21.15.106:0/97529503 >> 172.21.15.106:0/97529503 conn(0x7f41a808bf10 msgr2=0x7f41a80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 -- 172.21.15.106:0/97529503 shutdown_connections 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.545+0000 7f41ad636640 1 -- 172.21.15.106:0/97529503 wait complete. 2024-04-13T03:10:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 Processor -- start 2024-04-13T03:10:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 -- start start 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a80b3d60 0x7f41a80b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a80b6e70 0x7f41a815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8162800 con 0x7f41a815e9b0 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8162680 con 0x7f41a80b3d60 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.547+0000 7f41ad636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f41a806eb20 con 0x7f41a80b6e70 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56654/0 (socket says 172.21.15.106:56654) 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 -- 172.21.15.106:0/3895443854 learned_addr learned my addr 172.21.15.106:0/3895443854 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a77fe640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a80b6e70 0x7f41a815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a6ffd640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a80b3d60 0x7f41a80b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 -- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a80b6e70 msgr2=0x7f41a815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a80b6e70 0x7f41a815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 -- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a80b3d60 msgr2=0x7f41a80b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a80b3d60 0x7f41a80b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 -- 172.21.15.106:0/3895443854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a80ca5d0 con 0x7f41a815e9b0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41a67fc640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d800 tx=0x7f419c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c005990 con 0x7f41a815e9b0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c005b30 con 0x7f41a815e9b0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a80ca8b0 con 0x7f41a815e9b0 2024-04-13T03:10:40.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0096a0 con 0x7f41a815e9b0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.548+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a80b4910 con 0x7f41a815e9b0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.549+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a8068a10 con 0x7f41a815e9b0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.550+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f419c009840 con 0x7f41a815e9b0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.550+0000 7f418f7fe640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 0x7f419406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.550+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f419c08bdb0 con 0x7f41a815e9b0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.550+0000 7f41a6ffd640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 0x7f419406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.551+0000 7f41a6ffd640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 0x7f419406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4198036000 tx=0x7f4198008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.554+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f419c059a90 con 0x7f41a815e9b0 2024-04-13T03:10:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.733+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41a80d5000 con 0x7f41a815e9b0 2024-04-13T03:10:40.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.734+0000 7f418f7fe640 1 -- 172.21.15.106:0/3895443854 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7f419c05dd90 con 0x7f41a815e9b0 2024-04-13T03:10:40.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:40.735 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:39.078056+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 msgr2=0x7f419406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 0x7f419406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4198036000 tx=0x7f4198008030 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 msgr2=0x7f41a80c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f419c00d800 tx=0x7f419c00dcc0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 shutdown_connections 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f419406c8f0 0x7f419406edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a80b6e70 0x7f41a815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a815e9b0 0x7f41a80c9b10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 --2- 172.21.15.106:0/3895443854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a80b3d60 0x7f41a80b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 >> 172.21.15.106:0/3895443854 conn(0x7f41a808bf10 msgr2=0x7f41a80b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 shutdown_connections 2024-04-13T03:10:40.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.736+0000 7f41ad636640 1 -- 172.21.15.106:0/3895443854 wait complete. 2024-04-13T03:10:40.748 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:10:40.748 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:10:40.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.847+0000 7f58bd01c640 1 Processor -- start 2024-04-13T03:10:40.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.847+0000 7f58bd01c640 1 -- start start 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58bd01c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f58b815ec20 0x7f58b80b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58bd01c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b80b6120 con 0x7f58b80d14d0 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58bd01c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b80cd240 con 0x7f58b80ceb70 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58bd01c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80cde50 con 0x7f58b815ec20 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58b759e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f58b80d14d0 0x7f58b80b7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54986/0 (socket says 172.21.15.106:54986) 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58b759e640 1 -- 172.21.15.106:0/3278611966 learned_addr learned my addr 172.21.15.106:0/3278611966 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:40.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.848+0000 7f58b6d9d640 1 --2- 172.21.15.106:0/3278611966 >> v2:172.21.15.114:3300/0 conn(0x7f58b815ec20 0x7f58b80b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416421657 0 0) 0x7f58b80b6120 con 0x7f58b80d14d0 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58a0003520 con 0x7f58b80d14d0 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1780892941 0 0) 0x7f58b80cd240 con 0x7f58b80ceb70 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58b80b6120 con 0x7f58b80ceb70 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b6d9d640 1 -- 172.21.15.106:0/3278611966 >> v1:172.21.15.114:6789/0 conn(0x7f58b80ceb70 legacy=0x7f58b80b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b6d9d640 1 -- 172.21.15.106:0/3278611966 >> v1:172.21.15.106:6789/0 conn(0x7f58b80d14d0 legacy=0x7f58b80b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.849+0000 7f58b6d9d640 1 -- 172.21.15.106:0/3278611966 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80cd520 con 0x7f58b815ec20 2024-04-13T03:10:40.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58b6d9d640 1 --2- 172.21.15.106:0/3278611966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b815ec20 0x7f58b80b53c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009d90 tx=0x7f58a802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c45789db8cdc85d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8039070 con 0x7f58b815ec20 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a802cd00 con 0x7f58b815ec20 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58b5d9b640 1 -- 172.21.15.106:0/3278611966 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8034640 con 0x7f58b815ec20 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 -- 172.21.15.106:0/3278611966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b815ec20 msgr2=0x7f58b80b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 --2- 172.21.15.106:0/3278611966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b815ec20 0x7f58b80b53c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009d90 tx=0x7f58a802bf50 comp rx=0 tx=0).stop 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 -- 172.21.15.106:0/3278611966 shutdown_connections 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 --2- 172.21.15.106:0/3278611966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b815ec20 0x7f58b80b53c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 -- 172.21.15.106:0/3278611966 >> 172.21.15.106:0/3278611966 conn(0x7f58b80d2530 msgr2=0x7f58b80d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 -- 172.21.15.106:0/3278611966 shutdown_connections 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.850+0000 7f58bd01c640 1 -- 172.21.15.106:0/3278611966 wait complete. 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 Processor -- start 2024-04-13T03:10:40.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 -- start start 2024-04-13T03:10:40.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58b80d14d0 0x7f58b81f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b81f82a0 0x7f58b81f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80cced0 con 0x7f58b815ec20 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80cdae0 con 0x7f58b81f82a0 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.852+0000 7f58bd01c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58b80b83b0 con 0x7f58b80d14d0 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56660/0 (socket says 172.21.15.106:56660) 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 -- 172.21.15.106:0/1931661630 learned_addr learned my addr 172.21.15.106:0/1931661630 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b759e640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b81f82a0 0x7f58b81f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b6d9d640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58b80d14d0 0x7f58b81f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 -- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58b80d14d0 msgr2=0x7f58b81f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58b80d14d0 0x7f58b81f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 -- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b81f82a0 msgr2=0x7f58b81f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b81f82a0 0x7f58b81f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 -- 172.21.15.106:0/1931661630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b814fee0 con 0x7f58b815ec20 2024-04-13T03:10:40.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58b659c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c530 tx=0x7f58ac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0052d0 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac005490 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b81f4d20 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac012870 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.853+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b81f9020 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.854+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b8068a10 con 0x7f58b815ec20 2024-04-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.855+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58ac012a10 con 0x7f58b815ec20 2024-04-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.855+0000 7f589f7fe640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 0x7f58a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.855+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f58ac08b800 con 0x7f58b815ec20 2024-04-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.855+0000 7f58b6d9d640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 0x7f58a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.856+0000 7f58b6d9d640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 0x7f58a406ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a8036000 tx=0x7f58a8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:40.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:40.859+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58b814fee0 con 0x7f58b815ec20 2024-04-13T03:10:41.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.031+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58b80c68f0 con 0x7f58a406c920 2024-04-13T03:10:41.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.037+0000 7f589f7fe640 1 -- 172.21.15.106:0/1931661630 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f58b80c68f0 con 0x7f58a406c920 2024-04-13T03:10:41.041 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:41.042 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:10:41.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 msgr2=0x7f58a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 0x7f58a406ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a8036000 tx=0x7f58a8030000 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 msgr2=0x7f58b814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c530 tx=0x7f58ac00c9f0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 shutdown_connections 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58a406c920 0x7f58a406ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58b81f82a0 0x7f58b81f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58b815ec20 0x7f58b814f9a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 --2- 172.21.15.106:0/1931661630 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58b80d14d0 0x7f58b81f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 >> 172.21.15.106:0/1931661630 conn(0x7f58b80d2530 msgr2=0x7f58b80b7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 shutdown_connections 2024-04-13T03:10:41.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.040+0000 7f58bd01c640 1 -- 172.21.15.106:0/1931661630 wait complete. 2024-04-13T03:10:41.051 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-13T03:10:40.377601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4428,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4534272,"data_stored":2613849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319790"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072135+0000","last_change":"2024-04-13T03:10:37.072135+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072135+0000","last_undegraded":"2024-04-13T03:10:37.072135+0000","last_fullsized":"2024-04-13T03:10:37.072135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075618+0000","last_change":"2024-04-13T03:10:37.075618+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075618+0000","last_undegraded":"2024-04-13T03:10:37.075618+0000","last_fullsized":"2024-04-13T03:10:37.075618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073998+0000","last_change":"2024-04-13T03:10:37.073998+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073998+0000","last_undegraded":"2024-04-13T03:10:37.073998+0000","last_fullsized":"2024-04-13T03:10:37.073998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071804+0000","last_change":"2024-04-13T03:10:37.071804+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071804+0000","last_undegraded":"2024-04-13T03:10:37.071804+0000","last_fullsized":"2024-04-13T03:10:37.071804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.070206+0000","last_change":"2024-04-13T03:10:37.070206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.070206+0000","last_undegraded":"2024-04-13T03:10:37.070206+0000","last_fullsized":"2024-04-13T03:10:37.070206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079460+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079460+0000","last_undegraded":"2024-04-13T03:10:37.079460+0000","last_fullsized":"2024-04-13T03:10:37.079460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072953+0000","last_change":"2024-04-13T03:10:37.072953+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072953+0000","last_undegraded":"2024-04-13T03:10:37.072953+0000","last_fullsized":"2024-04-13T03:10:37.072953+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076313+0000","last_change":"2024-04-13T03:10:37.076313+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076313+0000","last_undegraded":"2024-04-13T03:10:37.076313+0000","last_fullsized":"2024-04-13T03:10:37.076313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075430+0000","last_change":"2024-04-13T03:10:37.075430+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075430+0000","last_undegraded":"2024-04-13T03:10:37.075430+0000","last_fullsized":"2024-04-13T03:10:37.075430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074866+0000","last_change":"2024-04-13T03:10:37.074866+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074866+0000","last_undegraded":"2024-04-13T03:10:37.074866+0000","last_fullsized":"2024-04-13T03:10:37.074866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073210+0000","last_change":"2024-04-13T03:10:37.073210+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073210+0000","last_undegraded":"2024-04-13T03:10:37.073210+0000","last_fullsized":"2024-04-13T03:10:37.073210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079554+0000","last_change":"2024-04-13T03:10:37.079554+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079554+0000","last_undegraded":"2024-04-13T03:10:37.079554+0000","last_fullsized":"2024-04-13T03:10:37.079554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077683+0000","last_change":"2024-04-13T03:10:37.077683+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077683+0000","last_undegraded":"2024-04-13T03:10:37.077683+0000","last_fullsized":"2024-04-13T03:10:37.077683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076708+0000","last_change":"2024-04-13T03:10:37.076708+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076708+0000","last_undegraded":"2024-04-13T03:10:37.076708+0000","last_fullsized":"2024-04-13T03:10:37.076708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079088+0000","last_change":"2024-04-13T03:10:37.079088+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079088+0000","last_undegraded":"2024-04-13T03:10:37.079088+0000","last_fullsized":"2024-04-13T03:10:37.079088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.081502+0000","last_change":"2024-04-13T03:10:37.081502+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.081502+0000","last_undegraded":"2024-04-13T03:10:37.081502+0000","last_fullsized":"2024-04-13T03:10:37.081502+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076788+0000","last_change":"2024-04-13T03:10:37.076788+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076788+0000","last_undegraded":"2024-04-13T03:10:37.076788+0000","last_fullsized":"2024-04-13T03:10:37.076788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072395+0000","last_change":"2024-04-13T03:10:37.072395+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072395+0000","last_undegraded":"2024-04-13T03:10:37.072395+0000","last_fullsized":"2024-04-13T03:10:37.072395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076537+0000","last_change":"2024-04-13T03:10:37.076537+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076537+0000","last_undegraded":"2024-04-13T03:10:37.076537+0000","last_fullsized":"2024-04-13T03:10:37.076537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075931+0000","last_change":"2024-04-13T03:10:37.075931+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075931+0000","last_undegraded":"2024-04-13T03:10:37.075931+0000","last_fullsized":"2024-04-13T03:10:37.075931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073585+0000","last_change":"2024-04-13T03:10:37.073585+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073585+0000","last_undegraded":"2024-04-13T03:10:37.073585+0000","last_fullsized":"2024-04-13T03:10:37.073585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078497+0000","last_change":"2024-04-13T03:10:37.078497+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078497+0000","last_undegraded":"2024-04-13T03:10:37.078497+0000","last_fullsized":"2024-04-13T03:10:37.078497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074338+0000","last_change":"2024-04-13T03:10:37.074338+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074338+0000","last_undegraded":"2024-04-13T03:10:37.074338+0000","last_fullsized":"2024-04-13T03:10:37.074338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077015+0000","last_change":"2024-04-13T03:10:37.077015+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077015+0000","last_undegraded":"2024-04-13T03:10:37.077015+0000","last_fullsized":"2024-04-13T03:10:37.077015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069793+0000","last_change":"2024-04-13T03:10:37.069793+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069793+0000","last_undegraded":"2024-04-13T03:10:37.069793+0000","last_fullsized":"2024-04-13T03:10:37.069793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076536+0000","last_change":"2024-04-13T03:10:37.076536+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076536+0000","last_undegraded":"2024-04-13T03:10:37.076536+0000","last_fullsized":"2024-04-13T03:10:37.076536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074266+0000","last_change":"2024-04-13T03:10:37.074266+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074266+0000","last_undegraded":"2024-04-13T03:10:37.074266+0000","last_fullsized":"2024-04-13T03:10:37.074266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075155+0000","last_change":"2024-04-13T03:10:37.075155+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075155+0000","last_undegraded":"2024-04-13T03:10:37.075155+0000","last_fullsized":"2024-04-13T03:10:37.075155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076048+0000","last_change":"2024-04-13T03:10:37.076048+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076048+0000","last_undegraded":"2024-04-13T03:10:37.076048+0000","last_fullsized":"2024-04-13T03:10:37.076048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078031+0000","last_change":"2024-04-13T03:10:37.078031+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078031+0000","last_undegraded":"2024-04-13T03:10:37.078031+0000","last_fullsized":"2024-04-13T03:10:37.078031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069807+0000","last_change":"2024-04-13T03:10:37.069807+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069807+0000","last_undegraded":"2024-04-13T03:10:37.069807+0000","last_fullsized":"2024-04-13T03:10:37.069807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079002+0000","last_change":"2024-04-13T03:10:37.079002+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079002+0000","last_undegraded":"2024-04-13T03:10:37.079002+0000","last_fullsized":"2024-04-13T03:10:37.079002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092039+0000","last_change":"2024-04-13T03:10:37.085786+0000","last_active":"2024-04-13T03:10:37.092039+0000","last_peered":"2024-04-13T03:10:37.092039+0000","last_clean":"2024-04-13T03:10:37.092039+0000","last_became_active":"2024-04-13T03:10:37.085500+0000","last_became_peered":"2024-04-13T03:10:37.085500+0000","last_unstale":"2024-04-13T03:10:37.092039+0000","last_undegraded":"2024-04-13T03:10:37.092039+0000","last_fullsized":"2024-04-13T03:10:37.092039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:13.188140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069756+0000","last_change":"2024-04-13T03:10:37.069756+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069756+0000","last_undegraded":"2024-04-13T03:10:37.069756+0000","last_fullsized":"2024-04-13T03:10:37.069756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086872+0000","last_change":"2024-04-13T03:10:37.083216+0000","last_active":"2024-04-13T03:10:37.086872+0000","last_peered":"2024-04-13T03:10:37.086872+0000","last_clean":"2024-04-13T03:10:37.086872+0000","last_became_active":"2024-04-13T03:10:37.082975+0000","last_became_peered":"2024-04-13T03:10:37.082975+0000","last_unstale":"2024-04-13T03:10:37.086872+0000","last_undegraded":"2024-04-13T03:10:37.086872+0000","last_fullsized":"2024-04-13T03:10:37.086872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072632+0000","last_change":"2024-04-13T03:10:37.072632+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072632+0000","last_undegraded":"2024-04-13T03:10:37.072632+0000","last_fullsized":"2024-04-13T03:10:37.072632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089581+0000","last_change":"2024-04-13T03:10:37.084531+0000","last_active":"2024-04-13T03:10:37.089581+0000","last_peered":"2024-04-13T03:10:37.089581+0000","last_clean":"2024-04-13T03:10:37.089581+0000","last_became_active":"2024-04-13T03:10:37.084244+0000","last_became_peered":"2024-04-13T03:10:37.084244+0000","last_unstale":"2024-04-13T03:10:37.089581+0000","last_undegraded":"2024-04-13T03:10:37.089581+0000","last_fullsized":"2024-04-13T03:10:37.089581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072539+0000","last_change":"2024-04-13T03:10:37.072539+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072539+0000","last_undegraded":"2024-04-13T03:10:37.072539+0000","last_fullsized":"2024-04-13T03:10:37.072539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086974+0000","last_change":"2024-04-13T03:10:37.083577+0000","last_active":"2024-04-13T03:10:37.086974+0000","last_peered":"2024-04-13T03:10:37.086974+0000","last_clean":"2024-04-13T03:10:37.086974+0000","last_became_active":"2024-04-13T03:10:37.083299+0000","last_became_peered":"2024-04-13T03:10:37.083299+0000","last_unstale":"2024-04-13T03:10:37.086974+0000","last_undegraded":"2024-04-13T03:10:37.086974+0000","last_fullsized":"2024-04-13T03:10:37.086974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072796+0000","last_change":"2024-04-13T03:10:37.072796+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072796+0000","last_undegraded":"2024-04-13T03:10:37.072796+0000","last_fullsized":"2024-04-13T03:10:37.072796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089546+0000","last_change":"2024-04-13T03:10:37.086817+0000","last_active":"2024-04-13T03:10:37.089546+0000","last_peered":"2024-04-13T03:10:37.089546+0000","last_clean":"2024-04-13T03:10:37.089546+0000","last_became_active":"2024-04-13T03:10:37.086544+0000","last_became_peered":"2024-04-13T03:10:37.086544+0000","last_unstale":"2024-04-13T03:10:37.089546+0000","last_undegraded":"2024-04-13T03:10:37.089546+0000","last_fullsized":"2024-04-13T03:10:37.089546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074859+0000","last_change":"2024-04-13T03:10:37.074859+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074859+0000","last_undegraded":"2024-04-13T03:10:37.074859+0000","last_fullsized":"2024-04-13T03:10:37.074859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094811+0000","last_change":"2024-04-13T03:10:37.089557+0000","last_active":"2024-04-13T03:10:37.094811+0000","last_peered":"2024-04-13T03:10:37.094811+0000","last_clean":"2024-04-13T03:10:37.094811+0000","last_became_active":"2024-04-13T03:10:37.089249+0000","last_became_peered":"2024-04-13T03:10:37.089249+0000","last_unstale":"2024-04-13T03:10:37.094811+0000","last_undegraded":"2024-04-13T03:10:37.094811+0000","last_fullsized":"2024-04-13T03:10:37.094811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:19.190392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073086+0000","last_change":"2024-04-13T03:10:37.073086+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073086+0000","last_undegraded":"2024-04-13T03:10:37.073086+0000","last_fullsized":"2024-04-13T03:10:37.073086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092667+0000","last_change":"2024-04-13T03:10:37.088764+0000","last_active":"2024-04-13T03:10:37.092667+0000","last_peered":"2024-04-13T03:10:37.092667+0000","last_clean":"2024-04-13T03:10:37.092667+0000","last_became_active":"2024-04-13T03:10:37.088507+0000","last_became_peered":"2024-04-13T03:10:37.088507+0000","last_unstale":"2024-04-13T03:10:37.092667+0000","last_undegraded":"2024-04-13T03:10:37.092667+0000","last_fullsized":"2024-04-13T03:10:37.092667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074137+0000","last_change":"2024-04-13T03:10:37.074137+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074137+0000","last_undegraded":"2024-04-13T03:10:37.074137+0000","last_fullsized":"2024-04-13T03:10:37.074137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087592+0000","last_change":"2024-04-13T03:10:37.083447+0000","last_active":"2024-04-13T03:10:37.087592+0000","last_peered":"2024-04-13T03:10:37.087592+0000","last_clean":"2024-04-13T03:10:37.087592+0000","last_became_active":"2024-04-13T03:10:37.083159+0000","last_became_peered":"2024-04-13T03:10:37.083159+0000","last_unstale":"2024-04-13T03:10:37.087592+0000","last_undegraded":"2024-04-13T03:10:37.087592+0000","last_fullsized":"2024-04-13T03:10:37.087592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071855+0000","last_change":"2024-04-13T03:10:37.071855+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071855+0000","last_undegraded":"2024-04-13T03:10:37.071855+0000","last_fullsized":"2024-04-13T03:10:37.071855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083991+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083991+0000","last_peered":"2024-04-13T03:10:37.083991+0000","last_clean":"2024-04-13T03:10:37.083991+0000","last_became_active":"2024-04-13T03:10:37.081899+0000","last_became_peered":"2024-04-13T03:10:37.081899+0000","last_unstale":"2024-04-13T03:10:37.083991+0000","last_undegraded":"2024-04-13T03:10:37.083991+0000","last_fullsized":"2024-04-13T03:10:37.083991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:04:52.620360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.084130+0000","last_change":"2024-04-13T03:10:37.084130+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.084130+0000","last_undegraded":"2024-04-13T03:10:37.084130+0000","last_fullsized":"2024-04-13T03:10:37.084130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087058+0000","last_change":"2024-04-13T03:10:37.085297+0000","last_active":"2024-04-13T03:10:37.087058+0000","last_peered":"2024-04-13T03:10:37.087058+0000","last_clean":"2024-04-13T03:10:37.087058+0000","last_became_active":"2024-04-13T03:10:37.085009+0000","last_became_peered":"2024-04-13T03:10:37.085009+0000","last_unstale":"2024-04-13T03:10:37.087058+0000","last_undegraded":"2024-04-13T03:10:37.087058+0000","last_fullsized":"2024-04-13T03:10:37.087058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092276+0000","last_change":"2024-04-13T03:10:37.088374+0000","last_active":"2024-04-13T03:10:37.092276+0000","last_peered":"2024-04-13T03:10:37.092276+0000","last_clean":"2024-04-13T03:10:37.092276+0000","last_became_active":"2024-04-13T03:10:37.088068+0000","last_became_peered":"2024-04-13T03:10:37.088068+0000","last_unstale":"2024-04-13T03:10:37.092276+0000","last_undegraded":"2024-04-13T03:10:37.092276+0000","last_fullsized":"2024-04-13T03:10:37.092276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073472+0000","last_change":"2024-04-13T03:10:37.073472+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073472+0000","last_undegraded":"2024-04-13T03:10:37.073472+0000","last_fullsized":"2024-04-13T03:10:37.073472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083655+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083655+0000","last_peered":"2024-04-13T03:10:37.083655+0000","last_clean":"2024-04-13T03:10:37.083655+0000","last_became_active":"2024-04-13T03:10:37.081896+0000","last_became_peered":"2024-04-13T03:10:37.081896+0000","last_unstale":"2024-04-13T03:10:37.083655+0000","last_undegraded":"2024-04-13T03:10:37.083655+0000","last_fullsized":"2024-04-13T03:10:37.083655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:38.859158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072382+0000","last_change":"2024-04-13T03:10:37.072382+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072382+0000","last_undegraded":"2024-04-13T03:10:37.072382+0000","last_fullsized":"2024-04-13T03:10:37.072382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.082603+0000","last_change":"2024-04-13T03:10:37.081457+0000","last_active":"2024-04-13T03:10:37.082603+0000","last_peered":"2024-04-13T03:10:37.082603+0000","last_clean":"2024-04-13T03:10:37.082603+0000","last_became_active":"2024-04-13T03:10:37.081232+0000","last_became_peered":"2024-04-13T03:10:37.081232+0000","last_unstale":"2024-04-13T03:10:37.082603+0000","last_undegraded":"2024-04-13T03:10:37.082603+0000","last_fullsized":"2024-04-13T03:10:37.082603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:13.251395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074188+0000","last_change":"2024-04-13T03:10:37.074188+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074188+0000","last_undegraded":"2024-04-13T03:10:37.074188+0000","last_fullsized":"2024-04-13T03:10:37.074188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.083620+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.083346+0000","last_became_peered":"2024-04-13T03:10:37.083346+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079369+0000","last_change":"2024-04-13T03:10:37.079369+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079369+0000","last_undegraded":"2024-04-13T03:10:37.079369+0000","last_fullsized":"2024-04-13T03:10:37.079369+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092066+0000","last_change":"2024-04-13T03:10:37.086440+0000","last_active":"2024-04-13T03:10:37.092066+0000","last_peered":"2024-04-13T03:10:37.092066+0000","last_clean":"2024-04-13T03:10:37.092066+0000","last_became_active":"2024-04-13T03:10:37.086158+0000","last_became_peered":"2024-04-13T03:10:37.086158+0000","last_unstale":"2024-04-13T03:10:37.092066+0000","last_undegraded":"2024-04-13T03:10:37.092066+0000","last_fullsized":"2024-04-13T03:10:37.092066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:07.798696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073362+0000","last_change":"2024-04-13T03:10:37.073362+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073362+0000","last_undegraded":"2024-04-13T03:10:37.073362+0000","last_fullsized":"2024-04-13T03:10:37.073362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094829+0000","last_change":"2024-04-13T03:10:37.089769+0000","last_active":"2024-04-13T03:10:37.094829+0000","last_peered":"2024-04-13T03:10:37.094829+0000","last_clean":"2024-04-13T03:10:37.094829+0000","last_became_active":"2024-04-13T03:10:37.089284+0000","last_became_peered":"2024-04-13T03:10:37.089284+0000","last_unstale":"2024-04-13T03:10:37.094829+0000","last_undegraded":"2024-04-13T03:10:37.094829+0000","last_fullsized":"2024-04-13T03:10:37.094829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:21.425166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075351+0000","last_change":"2024-04-13T03:10:37.075351+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075351+0000","last_undegraded":"2024-04-13T03:10:37.075351+0000","last_fullsized":"2024-04-13T03:10:37.075351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092234+0000","last_change":"2024-04-13T03:10:37.087631+0000","last_active":"2024-04-13T03:10:37.092234+0000","last_peered":"2024-04-13T03:10:37.092234+0000","last_clean":"2024-04-13T03:10:37.092234+0000","last_became_active":"2024-04-13T03:10:37.087385+0000","last_became_peered":"2024-04-13T03:10:37.087385+0000","last_unstale":"2024-04-13T03:10:37.092234+0000","last_undegraded":"2024-04-13T03:10:37.092234+0000","last_fullsized":"2024-04-13T03:10:37.092234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074341+0000","last_change":"2024-04-13T03:10:37.074341+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074341+0000","last_undegraded":"2024-04-13T03:10:37.074341+0000","last_fullsized":"2024-04-13T03:10:37.074341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087689+0000","last_change":"2024-04-13T03:10:37.085447+0000","last_active":"2024-04-13T03:10:37.087689+0000","last_peered":"2024-04-13T03:10:37.087689+0000","last_clean":"2024-04-13T03:10:37.087689+0000","last_became_active":"2024-04-13T03:10:37.085157+0000","last_became_peered":"2024-04-13T03:10:37.085157+0000","last_unstale":"2024-04-13T03:10:37.087689+0000","last_undegraded":"2024-04-13T03:10:37.087689+0000","last_fullsized":"2024-04-13T03:10:37.087689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073200+0000","last_change":"2024-04-13T03:10:37.073200+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073200+0000","last_undegraded":"2024-04-13T03:10:37.073200+0000","last_fullsized":"2024-04-13T03:10:37.073200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092500+0000","last_change":"2024-04-13T03:10:37.087391+0000","last_active":"2024-04-13T03:10:37.092500+0000","last_peered":"2024-04-13T03:10:37.092500+0000","last_clean":"2024-04-13T03:10:37.092500+0000","last_became_active":"2024-04-13T03:10:37.087131+0000","last_became_peered":"2024-04-13T03:10:37.087131+0000","last_unstale":"2024-04-13T03:10:37.092500+0000","last_undegraded":"2024-04-13T03:10:37.092500+0000","last_fullsized":"2024-04-13T03:10:37.092500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:41.946253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071737+0000","last_change":"2024-04-13T03:10:37.071737+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071737+0000","last_undegraded":"2024-04-13T03:10:37.071737+0000","last_fullsized":"2024-04-13T03:10:37.071737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089556+0000","last_change":"2024-04-13T03:10:37.084719+0000","last_active":"2024-04-13T03:10:37.089556+0000","last_peered":"2024-04-13T03:10:37.089556+0000","last_clean":"2024-04-13T03:10:37.089556+0000","last_became_active":"2024-04-13T03:10:37.084472+0000","last_became_peered":"2024-04-13T03:10:37.084472+0000","last_unstale":"2024-04-13T03:10:37.089556+0000","last_undegraded":"2024-04-13T03:10:37.089556+0000","last_fullsized":"2024-04-13T03:10:37.089556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078007+0000","last_change":"2024-04-13T03:10:37.078007+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078007+0000","last_undegraded":"2024-04-13T03:10:37.078007+0000","last_fullsized":"2024-04-13T03:10:37.078007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080873+0000","last_change":"2024-04-13T03:10:37.078763+0000","last_active":"2024-04-13T03:10:37.080873+0000","last_peered":"2024-04-13T03:10:37.080873+0000","last_clean":"2024-04-13T03:10:37.080873+0000","last_became_active":"2024-04-13T03:10:37.078484+0000","last_became_peered":"2024-04-13T03:10:37.078484+0000","last_unstale":"2024-04-13T03:10:37.080873+0000","last_undegraded":"2024-04-13T03:10:37.080873+0000","last_fullsized":"2024-04-13T03:10:37.080873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:54:33.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074170+0000","last_change":"2024-04-13T03:10:37.074170+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074170+0000","last_undegraded":"2024-04-13T03:10:37.074170+0000","last_fullsized":"2024-04-13T03:10:37.074170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.085242+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.084999+0000","last_became_peered":"2024-04-13T03:10:37.084999+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:31:56.747006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077206+0000","last_change":"2024-04-13T03:10:37.077206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077206+0000","last_undegraded":"2024-04-13T03:10:37.077206+0000","last_fullsized":"2024-04-13T03:10:37.077206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092322+0000","last_change":"2024-04-13T03:10:37.088188+0000","last_active":"2024-04-13T03:10:37.092322+0000","last_peered":"2024-04-13T03:10:37.092322+0000","last_clean":"2024-04-13T03:10:37.092322+0000","last_became_active":"2024-04-13T03:10:37.087921+0000","last_became_peered":"2024-04-13T03:10:37.087921+0000","last_unstale":"2024-04-13T03:10:37.092322+0000","last_undegraded":"2024-04-13T03:10:37.092322+0000","last_fullsized":"2024-04-13T03:10:37.092322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:01.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073727+0000","last_change":"2024-04-13T03:10:37.073727+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073727+0000","last_undegraded":"2024-04-13T03:10:37.073727+0000","last_fullsized":"2024-04-13T03:10:37.073727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086411+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.086411+0000","last_peered":"2024-04-13T03:10:37.086411+0000","last_clean":"2024-04-13T03:10:37.086411+0000","last_became_active":"2024-04-13T03:10:37.079188+0000","last_became_peered":"2024-04-13T03:10:37.079188+0000","last_unstale":"2024-04-13T03:10:37.086411+0000","last_undegraded":"2024-04-13T03:10:37.086411+0000","last_fullsized":"2024-04-13T03:10:37.086411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:41.463691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069800+0000","last_change":"2024-04-13T03:10:37.069800+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069800+0000","last_undegraded":"2024-04-13T03:10:37.069800+0000","last_fullsized":"2024-04-13T03:10:37.069800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092256+0000","last_change":"2024-04-13T03:10:37.088272+0000","last_active":"2024-04-13T03:10:37.092256+0000","last_peered":"2024-04-13T03:10:37.092256+0000","last_clean":"2024-04-13T03:10:37.092256+0000","last_became_active":"2024-04-13T03:10:37.088023+0000","last_became_peered":"2024-04-13T03:10:37.088023+0000","last_unstale":"2024-04-13T03:10:37.092256+0000","last_undegraded":"2024-04-13T03:10:37.092256+0000","last_fullsized":"2024-04-13T03:10:37.092256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.083615+0000","last_change":"2024-04-13T03:10:37.083615+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.083615+0000","last_undegraded":"2024-04-13T03:10:37.083615+0000","last_fullsized":"2024-04-13T03:10:37.083615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089636+0000","last_change":"2024-04-13T03:10:37.080915+0000","last_active":"2024-04-13T03:10:37.089636+0000","last_peered":"2024-04-13T03:10:37.089636+0000","last_clean":"2024-04-13T03:10:37.089636+0000","last_became_active":"2024-04-13T03:10:37.080658+0000","last_became_peered":"2024-04-13T03:10:37.080658+0000","last_unstale":"2024-04-13T03:10:37.089636+0000","last_undegraded":"2024-04-13T03:10:37.089636+0000","last_fullsized":"2024-04-13T03:10:37.089636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:02.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075238+0000","last_change":"2024-04-13T03:10:37.075238+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075238+0000","last_undegraded":"2024-04-13T03:10:37.075238+0000","last_fullsized":"2024-04-13T03:10:37.075238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086721+0000","last_change":"2024-04-13T03:10:37.081490+0000","last_active":"2024-04-13T03:10:37.086721+0000","last_peered":"2024-04-13T03:10:37.086721+0000","last_clean":"2024-04-13T03:10:37.086721+0000","last_became_active":"2024-04-13T03:10:37.081226+0000","last_became_peered":"2024-04-13T03:10:37.081226+0000","last_unstale":"2024-04-13T03:10:37.086721+0000","last_undegraded":"2024-04-13T03:10:37.086721+0000","last_fullsized":"2024-04-13T03:10:37.086721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071935+0000","last_change":"2024-04-13T03:10:37.071935+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071935+0000","last_undegraded":"2024-04-13T03:10:37.071935+0000","last_fullsized":"2024-04-13T03:10:37.071935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086903+0000","last_change":"2024-04-13T03:10:37.083750+0000","last_active":"2024-04-13T03:10:37.086903+0000","last_peered":"2024-04-13T03:10:37.086903+0000","last_clean":"2024-04-13T03:10:37.086903+0000","last_became_active":"2024-04-13T03:10:37.083512+0000","last_became_peered":"2024-04-13T03:10:37.083512+0000","last_unstale":"2024-04-13T03:10:37.086903+0000","last_undegraded":"2024-04-13T03:10:37.086903+0000","last_fullsized":"2024-04-13T03:10:37.086903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069680+0000","last_change":"2024-04-13T03:10:37.069680+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069680+0000","last_undegraded":"2024-04-13T03:10:37.069680+0000","last_fullsized":"2024-04-13T03:10:37.069680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087143+0000","last_change":"2024-04-13T03:10:37.082886+0000","last_active":"2024-04-13T03:10:37.087143+0000","last_peered":"2024-04-13T03:10:37.087143+0000","last_clean":"2024-04-13T03:10:37.087143+0000","last_became_active":"2024-04-13T03:10:37.082622+0000","last_became_peered":"2024-04-13T03:10:37.082622+0000","last_unstale":"2024-04-13T03:10:37.087143+0000","last_undegraded":"2024-04-13T03:10:37.087143+0000","last_fullsized":"2024-04-13T03:10:37.087143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:41.556737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077247+0000","last_change":"2024-04-13T03:10:37.077247+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077247+0000","last_undegraded":"2024-04-13T03:10:37.077247+0000","last_fullsized":"2024-04-13T03:10:37.077247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087667+0000","last_change":"2024-04-13T03:10:37.084769+0000","last_active":"2024-04-13T03:10:37.087667+0000","last_peered":"2024-04-13T03:10:37.087667+0000","last_clean":"2024-04-13T03:10:37.087667+0000","last_became_active":"2024-04-13T03:10:37.084496+0000","last_became_peered":"2024-04-13T03:10:37.084496+0000","last_unstale":"2024-04-13T03:10:37.087667+0000","last_undegraded":"2024-04-13T03:10:37.087667+0000","last_fullsized":"2024-04-13T03:10:37.087667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079692+0000","last_change":"2024-04-13T03:10:37.079692+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079692+0000","last_undegraded":"2024-04-13T03:10:37.079692+0000","last_fullsized":"2024-04-13T03:10:37.079692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083507+0000","last_change":"2024-04-13T03:10:37.082340+0000","last_active":"2024-04-13T03:10:37.083507+0000","last_peered":"2024-04-13T03:10:37.083507+0000","last_clean":"2024-04-13T03:10:37.083507+0000","last_became_active":"2024-04-13T03:10:37.082069+0000","last_became_peered":"2024-04-13T03:10:37.082069+0000","last_unstale":"2024-04-13T03:10:37.083507+0000","last_undegraded":"2024-04-13T03:10:37.083507+0000","last_fullsized":"2024-04-13T03:10:37.083507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:42.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076397+0000","last_change":"2024-04-13T03:10:37.076397+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076397+0000","last_undegraded":"2024-04-13T03:10:37.076397+0000","last_fullsized":"2024-04-13T03:10:37.076397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094834+0000","last_change":"2024-04-13T03:10:37.090291+0000","last_active":"2024-04-13T03:10:37.094834+0000","last_peered":"2024-04-13T03:10:37.094834+0000","last_clean":"2024-04-13T03:10:37.094834+0000","last_became_active":"2024-04-13T03:10:37.090004+0000","last_became_peered":"2024-04-13T03:10:37.090004+0000","last_unstale":"2024-04-13T03:10:37.094834+0000","last_undegraded":"2024-04-13T03:10:37.094834+0000","last_fullsized":"2024-04-13T03:10:37.094834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080293+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:10:37.080293+0000","last_peered":"2024-04-13T03:10:37.080293+0000","last_clean":"2024-04-13T03:10:37.080293+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:10:37.080293+0000","last_undegraded":"2024-04-13T03:10:37.080293+0000","last_fullsized":"2024-04-13T03:10:37.080293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:10:41.056 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:10:41.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.191+0000 7f949e9f7640 1 Processor -- start 2024-04-13T03:10:41.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.191+0000 7f949e9f7640 1 -- start start 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949e9f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f94980cdf50 0x7f94980ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949e9f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f949806e150 0x7f949806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949e9f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94980ce870 con 0x7f949806cb10 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949e9f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f94980998d0 con 0x7f949806e150 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949e9f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9498073020 con 0x7f94980cdf50 2024-04-13T03:10:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f949806e150 0x7f949806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f94977fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f949806cb10 0x7f9498072720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54992/0 (socket says 172.21.15.106:54992) 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f949806e150 0x7f949806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56674/0 (socket says 172.21.15.106:56674) 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f94977fe640 1 -- 172.21.15.106:0/408036709 learned_addr learned my addr 172.21.15.106:0/408036709 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f9497fff640 1 --2- 172.21.15.106:0/408036709 >> v2:172.21.15.114:3300/0 conn(0x7f94980cdf50 0x7f94980ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 -- 172.21.15.106:0/408036709 >> v2:172.21.15.114:3300/0 conn(0x7f94980cdf50 msgr2=0x7f94980ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 --2- 172.21.15.106:0/408036709 >> v2:172.21.15.114:3300/0 conn(0x7f94980cdf50 0x7f94980ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 -- 172.21.15.106:0/408036709 >> v1:172.21.15.106:6789/0 conn(0x7f949806cb10 legacy=0x7f9498072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f949cf94640 1 -- 172.21.15.106:0/408036709 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9498073300 con 0x7f949806e150 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.192+0000 7f9496ffd640 1 -- 172.21.15.106:0/408036709 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459793938 0 0) 0x7f94980ce870 con 0x7f949806cb10 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.193+0000 7f949cf94640 1 --2- 172.21.15.106:0/408036709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f949806e150 0x7f949806e600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f949806d680 tx=0x7f949002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9572d888fa55c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.193+0000 7f9496ffd640 1 -- 172.21.15.106:0/408036709 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490039070 con 0x7f949806e150 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.193+0000 7f9496ffd640 1 -- 172.21.15.106:0/408036709 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949002cc80 con 0x7f949806e150 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.193+0000 7f9496ffd640 1 -- 172.21.15.106:0/408036709 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490034660 con 0x7f949806e150 2024-04-13T03:10:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 -- 172.21.15.106:0/408036709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f949806e150 msgr2=0x7f949806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 --2- 172.21.15.106:0/408036709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f949806e150 0x7f949806e600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f949806d680 tx=0x7f949002bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 -- 172.21.15.106:0/408036709 shutdown_connections 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 --2- 172.21.15.106:0/408036709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f949806e150 0x7f949806e600 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 --2- 172.21.15.106:0/408036709 >> v2:172.21.15.114:3300/0 conn(0x7f94980cdf50 0x7f94980ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 -- 172.21.15.106:0/408036709 >> 172.21.15.106:0/408036709 conn(0x7f94980b1f10 msgr2=0x7f94980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 -- 172.21.15.106:0/408036709 shutdown_connections 2024-04-13T03:10:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.194+0000 7f949e9f7640 1 -- 172.21.15.106:0/408036709 wait complete. 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.195+0000 7f949e9f7640 1 Processor -- start 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.195+0000 7f949e9f7640 1 -- start start 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f949806cb10 0x7f94980c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f949806e150 0x7f9498152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f949806eb40 con 0x7f94980cdf50 2024-04-13T03:10:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94980738c0 con 0x7f949806cb10 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949e9f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f949806ecc0 con 0x7f949806e150 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56688/0 (socket says 172.21.15.106:56688) 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 -- 172.21.15.106:0/939983833 learned_addr learned my addr 172.21.15.106:0/939983833 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f9497fff640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f949806cb10 0x7f94980c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f94977fe640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f949806e150 0x7f9498152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 -- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f949806e150 msgr2=0x7f9498152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f949806e150 0x7f9498152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 -- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f949806cb10 msgr2=0x7f94980c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f949806cb10 0x7f94980c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.196+0000 7f949cf94640 1 -- 172.21.15.106:0/939983833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94981f3770 con 0x7f94980cdf50 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f949cf94640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9490009e80 tx=0x7f9490004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490044030 con 0x7f94980cdf50 2024-04-13T03:10:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949002ca80 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94981f3a50 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490039040 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.197+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981f4010 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.198+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9498068a10 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.198+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9490047070 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.199+0000 7f9494ff9640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 0x7f948406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.199+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f94900ae460 con 0x7f94980cdf50 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.199+0000 7f9497fff640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 0x7f948406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.199+0000 7f9497fff640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 0x7f948406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94880079a0 tx=0x7f9488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.203+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f949004e070 con 0x7f94980cdf50 2024-04-13T03:10:41.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.374+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f949806c580 con 0x7f948406c8f0 2024-04-13T03:10:41.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.379+0000 7f9494ff9640 1 -- 172.21.15.106:0/939983833 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f949806c580 con 0x7f948406c8f0 2024-04-13T03:10:41.383 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:41.384 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 msgr2=0x7f948406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 0x7f948406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94880079a0 tx=0x7f9488008040 comp rx=0 tx=0).stop 2024-04-13T03:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 msgr2=0x7f94980ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9490009e80 tx=0x7f9490004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 shutdown_connections 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f948406c8f0 0x7f948406edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f949806e150 0x7f9498152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94980cdf50 0x7f94980ad180 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 --2- 172.21.15.106:0/939983833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f949806cb10 0x7f94980c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.382+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 >> 172.21.15.106:0/939983833 conn(0x7f94980b1f10 msgr2=0x7f94980b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.383+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 shutdown_connections 2024-04-13T03:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.383+0000 7f949e9f7640 1 -- 172.21.15.106:0/939983833 wait complete. 2024-04-13T03:10:41.394 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-13T03:10:40.377601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4428,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4534272,"data_stored":2613849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319790"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072135+0000","last_change":"2024-04-13T03:10:37.072135+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072135+0000","last_undegraded":"2024-04-13T03:10:37.072135+0000","last_fullsized":"2024-04-13T03:10:37.072135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075618+0000","last_change":"2024-04-13T03:10:37.075618+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075618+0000","last_undegraded":"2024-04-13T03:10:37.075618+0000","last_fullsized":"2024-04-13T03:10:37.075618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073998+0000","last_change":"2024-04-13T03:10:37.073998+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073998+0000","last_undegraded":"2024-04-13T03:10:37.073998+0000","last_fullsized":"2024-04-13T03:10:37.073998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071804+0000","last_change":"2024-04-13T03:10:37.071804+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071804+0000","last_undegraded":"2024-04-13T03:10:37.071804+0000","last_fullsized":"2024-04-13T03:10:37.071804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.070206+0000","last_change":"2024-04-13T03:10:37.070206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.070206+0000","last_undegraded":"2024-04-13T03:10:37.070206+0000","last_fullsized":"2024-04-13T03:10:37.070206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079460+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079460+0000","last_undegraded":"2024-04-13T03:10:37.079460+0000","last_fullsized":"2024-04-13T03:10:37.079460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072953+0000","last_change":"2024-04-13T03:10:37.072953+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072953+0000","last_undegraded":"2024-04-13T03:10:37.072953+0000","last_fullsized":"2024-04-13T03:10:37.072953+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076313+0000","last_change":"2024-04-13T03:10:37.076313+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076313+0000","last_undegraded":"2024-04-13T03:10:37.076313+0000","last_fullsized":"2024-04-13T03:10:37.076313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075430+0000","last_change":"2024-04-13T03:10:37.075430+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075430+0000","last_undegraded":"2024-04-13T03:10:37.075430+0000","last_fullsized":"2024-04-13T03:10:37.075430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074866+0000","last_change":"2024-04-13T03:10:37.074866+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074866+0000","last_undegraded":"2024-04-13T03:10:37.074866+0000","last_fullsized":"2024-04-13T03:10:37.074866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073210+0000","last_change":"2024-04-13T03:10:37.073210+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073210+0000","last_undegraded":"2024-04-13T03:10:37.073210+0000","last_fullsized":"2024-04-13T03:10:37.073210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079554+0000","last_change":"2024-04-13T03:10:37.079554+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079554+0000","last_undegraded":"2024-04-13T03:10:37.079554+0000","last_fullsized":"2024-04-13T03:10:37.079554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077683+0000","last_change":"2024-04-13T03:10:37.077683+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077683+0000","last_undegraded":"2024-04-13T03:10:37.077683+0000","last_fullsized":"2024-04-13T03:10:37.077683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076708+0000","last_change":"2024-04-13T03:10:37.076708+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076708+0000","last_undegraded":"2024-04-13T03:10:37.076708+0000","last_fullsized":"2024-04-13T03:10:37.076708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079088+0000","last_change":"2024-04-13T03:10:37.079088+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079088+0000","last_undegraded":"2024-04-13T03:10:37.079088+0000","last_fullsized":"2024-04-13T03:10:37.079088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.081502+0000","last_change":"2024-04-13T03:10:37.081502+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.081502+0000","last_undegraded":"2024-04-13T03:10:37.081502+0000","last_fullsized":"2024-04-13T03:10:37.081502+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076788+0000","last_change":"2024-04-13T03:10:37.076788+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076788+0000","last_undegraded":"2024-04-13T03:10:37.076788+0000","last_fullsized":"2024-04-13T03:10:37.076788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072395+0000","last_change":"2024-04-13T03:10:37.072395+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072395+0000","last_undegraded":"2024-04-13T03:10:37.072395+0000","last_fullsized":"2024-04-13T03:10:37.072395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076537+0000","last_change":"2024-04-13T03:10:37.076537+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076537+0000","last_undegraded":"2024-04-13T03:10:37.076537+0000","last_fullsized":"2024-04-13T03:10:37.076537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075931+0000","last_change":"2024-04-13T03:10:37.075931+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075931+0000","last_undegraded":"2024-04-13T03:10:37.075931+0000","last_fullsized":"2024-04-13T03:10:37.075931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073585+0000","last_change":"2024-04-13T03:10:37.073585+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073585+0000","last_undegraded":"2024-04-13T03:10:37.073585+0000","last_fullsized":"2024-04-13T03:10:37.073585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078497+0000","last_change":"2024-04-13T03:10:37.078497+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078497+0000","last_undegraded":"2024-04-13T03:10:37.078497+0000","last_fullsized":"2024-04-13T03:10:37.078497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074338+0000","last_change":"2024-04-13T03:10:37.074338+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074338+0000","last_undegraded":"2024-04-13T03:10:37.074338+0000","last_fullsized":"2024-04-13T03:10:37.074338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077015+0000","last_change":"2024-04-13T03:10:37.077015+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077015+0000","last_undegraded":"2024-04-13T03:10:37.077015+0000","last_fullsized":"2024-04-13T03:10:37.077015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069793+0000","last_change":"2024-04-13T03:10:37.069793+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069793+0000","last_undegraded":"2024-04-13T03:10:37.069793+0000","last_fullsized":"2024-04-13T03:10:37.069793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076536+0000","last_change":"2024-04-13T03:10:37.076536+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076536+0000","last_undegraded":"2024-04-13T03:10:37.076536+0000","last_fullsized":"2024-04-13T03:10:37.076536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074266+0000","last_change":"2024-04-13T03:10:37.074266+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074266+0000","last_undegraded":"2024-04-13T03:10:37.074266+0000","last_fullsized":"2024-04-13T03:10:37.074266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075155+0000","last_change":"2024-04-13T03:10:37.075155+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075155+0000","last_undegraded":"2024-04-13T03:10:37.075155+0000","last_fullsized":"2024-04-13T03:10:37.075155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076048+0000","last_change":"2024-04-13T03:10:37.076048+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076048+0000","last_undegraded":"2024-04-13T03:10:37.076048+0000","last_fullsized":"2024-04-13T03:10:37.076048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078031+0000","last_change":"2024-04-13T03:10:37.078031+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078031+0000","last_undegraded":"2024-04-13T03:10:37.078031+0000","last_fullsized":"2024-04-13T03:10:37.078031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069807+0000","last_change":"2024-04-13T03:10:37.069807+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069807+0000","last_undegraded":"2024-04-13T03:10:37.069807+0000","last_fullsized":"2024-04-13T03:10:37.069807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079002+0000","last_change":"2024-04-13T03:10:37.079002+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079002+0000","last_undegraded":"2024-04-13T03:10:37.079002+0000","last_fullsized":"2024-04-13T03:10:37.079002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092039+0000","last_change":"2024-04-13T03:10:37.085786+0000","last_active":"2024-04-13T03:10:37.092039+0000","last_peered":"2024-04-13T03:10:37.092039+0000","last_clean":"2024-04-13T03:10:37.092039+0000","last_became_active":"2024-04-13T03:10:37.085500+0000","last_became_peered":"2024-04-13T03:10:37.085500+0000","last_unstale":"2024-04-13T03:10:37.092039+0000","last_undegraded":"2024-04-13T03:10:37.092039+0000","last_fullsized":"2024-04-13T03:10:37.092039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:13.188140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069756+0000","last_change":"2024-04-13T03:10:37.069756+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069756+0000","last_undegraded":"2024-04-13T03:10:37.069756+0000","last_fullsized":"2024-04-13T03:10:37.069756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086872+0000","last_change":"2024-04-13T03:10:37.083216+0000","last_active":"2024-04-13T03:10:37.086872+0000","last_peered":"2024-04-13T03:10:37.086872+0000","last_clean":"2024-04-13T03:10:37.086872+0000","last_became_active":"2024-04-13T03:10:37.082975+0000","last_became_peered":"2024-04-13T03:10:37.082975+0000","last_unstale":"2024-04-13T03:10:37.086872+0000","last_undegraded":"2024-04-13T03:10:37.086872+0000","last_fullsized":"2024-04-13T03:10:37.086872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072632+0000","last_change":"2024-04-13T03:10:37.072632+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072632+0000","last_undegraded":"2024-04-13T03:10:37.072632+0000","last_fullsized":"2024-04-13T03:10:37.072632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089581+0000","last_change":"2024-04-13T03:10:37.084531+0000","last_active":"2024-04-13T03:10:37.089581+0000","last_peered":"2024-04-13T03:10:37.089581+0000","last_clean":"2024-04-13T03:10:37.089581+0000","last_became_active":"2024-04-13T03:10:37.084244+0000","last_became_peered":"2024-04-13T03:10:37.084244+0000","last_unstale":"2024-04-13T03:10:37.089581+0000","last_undegraded":"2024-04-13T03:10:37.089581+0000","last_fullsized":"2024-04-13T03:10:37.089581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072539+0000","last_change":"2024-04-13T03:10:37.072539+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072539+0000","last_undegraded":"2024-04-13T03:10:37.072539+0000","last_fullsized":"2024-04-13T03:10:37.072539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086974+0000","last_change":"2024-04-13T03:10:37.083577+0000","last_active":"2024-04-13T03:10:37.086974+0000","last_peered":"2024-04-13T03:10:37.086974+0000","last_clean":"2024-04-13T03:10:37.086974+0000","last_became_active":"2024-04-13T03:10:37.083299+0000","last_became_peered":"2024-04-13T03:10:37.083299+0000","last_unstale":"2024-04-13T03:10:37.086974+0000","last_undegraded":"2024-04-13T03:10:37.086974+0000","last_fullsized":"2024-04-13T03:10:37.086974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072796+0000","last_change":"2024-04-13T03:10:37.072796+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072796+0000","last_undegraded":"2024-04-13T03:10:37.072796+0000","last_fullsized":"2024-04-13T03:10:37.072796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089546+0000","last_change":"2024-04-13T03:10:37.086817+0000","last_active":"2024-04-13T03:10:37.089546+0000","last_peered":"2024-04-13T03:10:37.089546+0000","last_clean":"2024-04-13T03:10:37.089546+0000","last_became_active":"2024-04-13T03:10:37.086544+0000","last_became_peered":"2024-04-13T03:10:37.086544+0000","last_unstale":"2024-04-13T03:10:37.089546+0000","last_undegraded":"2024-04-13T03:10:37.089546+0000","last_fullsized":"2024-04-13T03:10:37.089546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074859+0000","last_change":"2024-04-13T03:10:37.074859+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074859+0000","last_undegraded":"2024-04-13T03:10:37.074859+0000","last_fullsized":"2024-04-13T03:10:37.074859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094811+0000","last_change":"2024-04-13T03:10:37.089557+0000","last_active":"2024-04-13T03:10:37.094811+0000","last_peered":"2024-04-13T03:10:37.094811+0000","last_clean":"2024-04-13T03:10:37.094811+0000","last_became_active":"2024-04-13T03:10:37.089249+0000","last_became_peered":"2024-04-13T03:10:37.089249+0000","last_unstale":"2024-04-13T03:10:37.094811+0000","last_undegraded":"2024-04-13T03:10:37.094811+0000","last_fullsized":"2024-04-13T03:10:37.094811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:19.190392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073086+0000","last_change":"2024-04-13T03:10:37.073086+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073086+0000","last_undegraded":"2024-04-13T03:10:37.073086+0000","last_fullsized":"2024-04-13T03:10:37.073086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092667+0000","last_change":"2024-04-13T03:10:37.088764+0000","last_active":"2024-04-13T03:10:37.092667+0000","last_peered":"2024-04-13T03:10:37.092667+0000","last_clean":"2024-04-13T03:10:37.092667+0000","last_became_active":"2024-04-13T03:10:37.088507+0000","last_became_peered":"2024-04-13T03:10:37.088507+0000","last_unstale":"2024-04-13T03:10:37.092667+0000","last_undegraded":"2024-04-13T03:10:37.092667+0000","last_fullsized":"2024-04-13T03:10:37.092667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074137+0000","last_change":"2024-04-13T03:10:37.074137+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074137+0000","last_undegraded":"2024-04-13T03:10:37.074137+0000","last_fullsized":"2024-04-13T03:10:37.074137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087592+0000","last_change":"2024-04-13T03:10:37.083447+0000","last_active":"2024-04-13T03:10:37.087592+0000","last_peered":"2024-04-13T03:10:37.087592+0000","last_clean":"2024-04-13T03:10:37.087592+0000","last_became_active":"2024-04-13T03:10:37.083159+0000","last_became_peered":"2024-04-13T03:10:37.083159+0000","last_unstale":"2024-04-13T03:10:37.087592+0000","last_undegraded":"2024-04-13T03:10:37.087592+0000","last_fullsized":"2024-04-13T03:10:37.087592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071855+0000","last_change":"2024-04-13T03:10:37.071855+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071855+0000","last_undegraded":"2024-04-13T03:10:37.071855+0000","last_fullsized":"2024-04-13T03:10:37.071855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083991+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083991+0000","last_peered":"2024-04-13T03:10:37.083991+0000","last_clean":"2024-04-13T03:10:37.083991+0000","last_became_active":"2024-04-13T03:10:37.081899+0000","last_became_peered":"2024-04-13T03:10:37.081899+0000","last_unstale":"2024-04-13T03:10:37.083991+0000","last_undegraded":"2024-04-13T03:10:37.083991+0000","last_fullsized":"2024-04-13T03:10:37.083991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:04:52.620360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.084130+0000","last_change":"2024-04-13T03:10:37.084130+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.084130+0000","last_undegraded":"2024-04-13T03:10:37.084130+0000","last_fullsized":"2024-04-13T03:10:37.084130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087058+0000","last_change":"2024-04-13T03:10:37.085297+0000","last_active":"2024-04-13T03:10:37.087058+0000","last_peered":"2024-04-13T03:10:37.087058+0000","last_clean":"2024-04-13T03:10:37.087058+0000","last_became_active":"2024-04-13T03:10:37.085009+0000","last_became_peered":"2024-04-13T03:10:37.085009+0000","last_unstale":"2024-04-13T03:10:37.087058+0000","last_undegraded":"2024-04-13T03:10:37.087058+0000","last_fullsized":"2024-04-13T03:10:37.087058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092276+0000","last_change":"2024-04-13T03:10:37.088374+0000","last_active":"2024-04-13T03:10:37.092276+0000","last_peered":"2024-04-13T03:10:37.092276+0000","last_clean":"2024-04-13T03:10:37.092276+0000","last_became_active":"2024-04-13T03:10:37.088068+0000","last_became_peered":"2024-04-13T03:10:37.088068+0000","last_unstale":"2024-04-13T03:10:37.092276+0000","last_undegraded":"2024-04-13T03:10:37.092276+0000","last_fullsized":"2024-04-13T03:10:37.092276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073472+0000","last_change":"2024-04-13T03:10:37.073472+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073472+0000","last_undegraded":"2024-04-13T03:10:37.073472+0000","last_fullsized":"2024-04-13T03:10:37.073472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083655+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083655+0000","last_peered":"2024-04-13T03:10:37.083655+0000","last_clean":"2024-04-13T03:10:37.083655+0000","last_became_active":"2024-04-13T03:10:37.081896+0000","last_became_peered":"2024-04-13T03:10:37.081896+0000","last_unstale":"2024-04-13T03:10:37.083655+0000","last_undegraded":"2024-04-13T03:10:37.083655+0000","last_fullsized":"2024-04-13T03:10:37.083655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:38.859158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072382+0000","last_change":"2024-04-13T03:10:37.072382+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072382+0000","last_undegraded":"2024-04-13T03:10:37.072382+0000","last_fullsized":"2024-04-13T03:10:37.072382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.082603+0000","last_change":"2024-04-13T03:10:37.081457+0000","last_active":"2024-04-13T03:10:37.082603+0000","last_peered":"2024-04-13T03:10:37.082603+0000","last_clean":"2024-04-13T03:10:37.082603+0000","last_became_active":"2024-04-13T03:10:37.081232+0000","last_became_peered":"2024-04-13T03:10:37.081232+0000","last_unstale":"2024-04-13T03:10:37.082603+0000","last_undegraded":"2024-04-13T03:10:37.082603+0000","last_fullsized":"2024-04-13T03:10:37.082603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:13.251395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074188+0000","last_change":"2024-04-13T03:10:37.074188+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074188+0000","last_undegraded":"2024-04-13T03:10:37.074188+0000","last_fullsized":"2024-04-13T03:10:37.074188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.083620+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.083346+0000","last_became_peered":"2024-04-13T03:10:37.083346+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079369+0000","last_change":"2024-04-13T03:10:37.079369+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079369+0000","last_undegraded":"2024-04-13T03:10:37.079369+0000","last_fullsized":"2024-04-13T03:10:37.079369+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092066+0000","last_change":"2024-04-13T03:10:37.086440+0000","last_active":"2024-04-13T03:10:37.092066+0000","last_peered":"2024-04-13T03:10:37.092066+0000","last_clean":"2024-04-13T03:10:37.092066+0000","last_became_active":"2024-04-13T03:10:37.086158+0000","last_became_peered":"2024-04-13T03:10:37.086158+0000","last_unstale":"2024-04-13T03:10:37.092066+0000","last_undegraded":"2024-04-13T03:10:37.092066+0000","last_fullsized":"2024-04-13T03:10:37.092066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:07.798696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073362+0000","last_change":"2024-04-13T03:10:37.073362+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073362+0000","last_undegraded":"2024-04-13T03:10:37.073362+0000","last_fullsized":"2024-04-13T03:10:37.073362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094829+0000","last_change":"2024-04-13T03:10:37.089769+0000","last_active":"2024-04-13T03:10:37.094829+0000","last_peered":"2024-04-13T03:10:37.094829+0000","last_clean":"2024-04-13T03:10:37.094829+0000","last_became_active":"2024-04-13T03:10:37.089284+0000","last_became_peered":"2024-04-13T03:10:37.089284+0000","last_unstale":"2024-04-13T03:10:37.094829+0000","last_undegraded":"2024-04-13T03:10:37.094829+0000","last_fullsized":"2024-04-13T03:10:37.094829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:21.425166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075351+0000","last_change":"2024-04-13T03:10:37.075351+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075351+0000","last_undegraded":"2024-04-13T03:10:37.075351+0000","last_fullsized":"2024-04-13T03:10:37.075351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092234+0000","last_change":"2024-04-13T03:10:37.087631+0000","last_active":"2024-04-13T03:10:37.092234+0000","last_peered":"2024-04-13T03:10:37.092234+0000","last_clean":"2024-04-13T03:10:37.092234+0000","last_became_active":"2024-04-13T03:10:37.087385+0000","last_became_peered":"2024-04-13T03:10:37.087385+0000","last_unstale":"2024-04-13T03:10:37.092234+0000","last_undegraded":"2024-04-13T03:10:37.092234+0000","last_fullsized":"2024-04-13T03:10:37.092234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074341+0000","last_change":"2024-04-13T03:10:37.074341+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074341+0000","last_undegraded":"2024-04-13T03:10:37.074341+0000","last_fullsized":"2024-04-13T03:10:37.074341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087689+0000","last_change":"2024-04-13T03:10:37.085447+0000","last_active":"2024-04-13T03:10:37.087689+0000","last_peered":"2024-04-13T03:10:37.087689+0000","last_clean":"2024-04-13T03:10:37.087689+0000","last_became_active":"2024-04-13T03:10:37.085157+0000","last_became_peered":"2024-04-13T03:10:37.085157+0000","last_unstale":"2024-04-13T03:10:37.087689+0000","last_undegraded":"2024-04-13T03:10:37.087689+0000","last_fullsized":"2024-04-13T03:10:37.087689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073200+0000","last_change":"2024-04-13T03:10:37.073200+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073200+0000","last_undegraded":"2024-04-13T03:10:37.073200+0000","last_fullsized":"2024-04-13T03:10:37.073200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092500+0000","last_change":"2024-04-13T03:10:37.087391+0000","last_active":"2024-04-13T03:10:37.092500+0000","last_peered":"2024-04-13T03:10:37.092500+0000","last_clean":"2024-04-13T03:10:37.092500+0000","last_became_active":"2024-04-13T03:10:37.087131+0000","last_became_peered":"2024-04-13T03:10:37.087131+0000","last_unstale":"2024-04-13T03:10:37.092500+0000","last_undegraded":"2024-04-13T03:10:37.092500+0000","last_fullsized":"2024-04-13T03:10:37.092500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:41.946253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071737+0000","last_change":"2024-04-13T03:10:37.071737+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071737+0000","last_undegraded":"2024-04-13T03:10:37.071737+0000","last_fullsized":"2024-04-13T03:10:37.071737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089556+0000","last_change":"2024-04-13T03:10:37.084719+0000","last_active":"2024-04-13T03:10:37.089556+0000","last_peered":"2024-04-13T03:10:37.089556+0000","last_clean":"2024-04-13T03:10:37.089556+0000","last_became_active":"2024-04-13T03:10:37.084472+0000","last_became_peered":"2024-04-13T03:10:37.084472+0000","last_unstale":"2024-04-13T03:10:37.089556+0000","last_undegraded":"2024-04-13T03:10:37.089556+0000","last_fullsized":"2024-04-13T03:10:37.089556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078007+0000","last_change":"2024-04-13T03:10:37.078007+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078007+0000","last_undegraded":"2024-04-13T03:10:37.078007+0000","last_fullsized":"2024-04-13T03:10:37.078007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080873+0000","last_change":"2024-04-13T03:10:37.078763+0000","last_active":"2024-04-13T03:10:37.080873+0000","last_peered":"2024-04-13T03:10:37.080873+0000","last_clean":"2024-04-13T03:10:37.080873+0000","last_became_active":"2024-04-13T03:10:37.078484+0000","last_became_peered":"2024-04-13T03:10:37.078484+0000","last_unstale":"2024-04-13T03:10:37.080873+0000","last_undegraded":"2024-04-13T03:10:37.080873+0000","last_fullsized":"2024-04-13T03:10:37.080873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:54:33.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074170+0000","last_change":"2024-04-13T03:10:37.074170+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074170+0000","last_undegraded":"2024-04-13T03:10:37.074170+0000","last_fullsized":"2024-04-13T03:10:37.074170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.085242+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.084999+0000","last_became_peered":"2024-04-13T03:10:37.084999+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:31:56.747006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077206+0000","last_change":"2024-04-13T03:10:37.077206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077206+0000","last_undegraded":"2024-04-13T03:10:37.077206+0000","last_fullsized":"2024-04-13T03:10:37.077206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092322+0000","last_change":"2024-04-13T03:10:37.088188+0000","last_active":"2024-04-13T03:10:37.092322+0000","last_peered":"2024-04-13T03:10:37.092322+0000","last_clean":"2024-04-13T03:10:37.092322+0000","last_became_active":"2024-04-13T03:10:37.087921+0000","last_became_peered":"2024-04-13T03:10:37.087921+0000","last_unstale":"2024-04-13T03:10:37.092322+0000","last_undegraded":"2024-04-13T03:10:37.092322+0000","last_fullsized":"2024-04-13T03:10:37.092322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:01.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073727+0000","last_change":"2024-04-13T03:10:37.073727+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073727+0000","last_undegraded":"2024-04-13T03:10:37.073727+0000","last_fullsized":"2024-04-13T03:10:37.073727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086411+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.086411+0000","last_peered":"2024-04-13T03:10:37.086411+0000","last_clean":"2024-04-13T03:10:37.086411+0000","last_became_active":"2024-04-13T03:10:37.079188+0000","last_became_peered":"2024-04-13T03:10:37.079188+0000","last_unstale":"2024-04-13T03:10:37.086411+0000","last_undegraded":"2024-04-13T03:10:37.086411+0000","last_fullsized":"2024-04-13T03:10:37.086411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:41.463691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069800+0000","last_change":"2024-04-13T03:10:37.069800+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069800+0000","last_undegraded":"2024-04-13T03:10:37.069800+0000","last_fullsized":"2024-04-13T03:10:37.069800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092256+0000","last_change":"2024-04-13T03:10:37.088272+0000","last_active":"2024-04-13T03:10:37.092256+0000","last_peered":"2024-04-13T03:10:37.092256+0000","last_clean":"2024-04-13T03:10:37.092256+0000","last_became_active":"2024-04-13T03:10:37.088023+0000","last_became_peered":"2024-04-13T03:10:37.088023+0000","last_unstale":"2024-04-13T03:10:37.092256+0000","last_undegraded":"2024-04-13T03:10:37.092256+0000","last_fullsized":"2024-04-13T03:10:37.092256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.083615+0000","last_change":"2024-04-13T03:10:37.083615+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.083615+0000","last_undegraded":"2024-04-13T03:10:37.083615+0000","last_fullsized":"2024-04-13T03:10:37.083615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089636+0000","last_change":"2024-04-13T03:10:37.080915+0000","last_active":"2024-04-13T03:10:37.089636+0000","last_peered":"2024-04-13T03:10:37.089636+0000","last_clean":"2024-04-13T03:10:37.089636+0000","last_became_active":"2024-04-13T03:10:37.080658+0000","last_became_peered":"2024-04-13T03:10:37.080658+0000","last_unstale":"2024-04-13T03:10:37.089636+0000","last_undegraded":"2024-04-13T03:10:37.089636+0000","last_fullsized":"2024-04-13T03:10:37.089636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:02.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075238+0000","last_change":"2024-04-13T03:10:37.075238+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075238+0000","last_undegraded":"2024-04-13T03:10:37.075238+0000","last_fullsized":"2024-04-13T03:10:37.075238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086721+0000","last_change":"2024-04-13T03:10:37.081490+0000","last_active":"2024-04-13T03:10:37.086721+0000","last_peered":"2024-04-13T03:10:37.086721+0000","last_clean":"2024-04-13T03:10:37.086721+0000","last_became_active":"2024-04-13T03:10:37.081226+0000","last_became_peered":"2024-04-13T03:10:37.081226+0000","last_unstale":"2024-04-13T03:10:37.086721+0000","last_undegraded":"2024-04-13T03:10:37.086721+0000","last_fullsized":"2024-04-13T03:10:37.086721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071935+0000","last_change":"2024-04-13T03:10:37.071935+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071935+0000","last_undegraded":"2024-04-13T03:10:37.071935+0000","last_fullsized":"2024-04-13T03:10:37.071935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086903+0000","last_change":"2024-04-13T03:10:37.083750+0000","last_active":"2024-04-13T03:10:37.086903+0000","last_peered":"2024-04-13T03:10:37.086903+0000","last_clean":"2024-04-13T03:10:37.086903+0000","last_became_active":"2024-04-13T03:10:37.083512+0000","last_became_peered":"2024-04-13T03:10:37.083512+0000","last_unstale":"2024-04-13T03:10:37.086903+0000","last_undegraded":"2024-04-13T03:10:37.086903+0000","last_fullsized":"2024-04-13T03:10:37.086903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069680+0000","last_change":"2024-04-13T03:10:37.069680+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069680+0000","last_undegraded":"2024-04-13T03:10:37.069680+0000","last_fullsized":"2024-04-13T03:10:37.069680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087143+0000","last_change":"2024-04-13T03:10:37.082886+0000","last_active":"2024-04-13T03:10:37.087143+0000","last_peered":"2024-04-13T03:10:37.087143+0000","last_clean":"2024-04-13T03:10:37.087143+0000","last_became_active":"2024-04-13T03:10:37.082622+0000","last_became_peered":"2024-04-13T03:10:37.082622+0000","last_unstale":"2024-04-13T03:10:37.087143+0000","last_undegraded":"2024-04-13T03:10:37.087143+0000","last_fullsized":"2024-04-13T03:10:37.087143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:41.556737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077247+0000","last_change":"2024-04-13T03:10:37.077247+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077247+0000","last_undegraded":"2024-04-13T03:10:37.077247+0000","last_fullsized":"2024-04-13T03:10:37.077247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087667+0000","last_change":"2024-04-13T03:10:37.084769+0000","last_active":"2024-04-13T03:10:37.087667+0000","last_peered":"2024-04-13T03:10:37.087667+0000","last_clean":"2024-04-13T03:10:37.087667+0000","last_became_active":"2024-04-13T03:10:37.084496+0000","last_became_peered":"2024-04-13T03:10:37.084496+0000","last_unstale":"2024-04-13T03:10:37.087667+0000","last_undegraded":"2024-04-13T03:10:37.087667+0000","last_fullsized":"2024-04-13T03:10:37.087667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079692+0000","last_change":"2024-04-13T03:10:37.079692+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079692+0000","last_undegraded":"2024-04-13T03:10:37.079692+0000","last_fullsized":"2024-04-13T03:10:37.079692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083507+0000","last_change":"2024-04-13T03:10:37.082340+0000","last_active":"2024-04-13T03:10:37.083507+0000","last_peered":"2024-04-13T03:10:37.083507+0000","last_clean":"2024-04-13T03:10:37.083507+0000","last_became_active":"2024-04-13T03:10:37.082069+0000","last_became_peered":"2024-04-13T03:10:37.082069+0000","last_unstale":"2024-04-13T03:10:37.083507+0000","last_undegraded":"2024-04-13T03:10:37.083507+0000","last_fullsized":"2024-04-13T03:10:37.083507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:42.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076397+0000","last_change":"2024-04-13T03:10:37.076397+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076397+0000","last_undegraded":"2024-04-13T03:10:37.076397+0000","last_fullsized":"2024-04-13T03:10:37.076397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094834+0000","last_change":"2024-04-13T03:10:37.090291+0000","last_active":"2024-04-13T03:10:37.094834+0000","last_peered":"2024-04-13T03:10:37.094834+0000","last_clean":"2024-04-13T03:10:37.094834+0000","last_became_active":"2024-04-13T03:10:37.090004+0000","last_became_peered":"2024-04-13T03:10:37.090004+0000","last_unstale":"2024-04-13T03:10:37.094834+0000","last_undegraded":"2024-04-13T03:10:37.094834+0000","last_fullsized":"2024-04-13T03:10:37.094834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080293+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:10:37.080293+0000","last_peered":"2024-04-13T03:10:37.080293+0000","last_clean":"2024-04-13T03:10:37.080293+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:10:37.080293+0000","last_undegraded":"2024-04-13T03:10:37.080293+0000","last_fullsized":"2024-04-13T03:10:37.080293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:10:41.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:10:41.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.534+0000 7f4b3018c640 1 Processor -- start 2024-04-13T03:10:41.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 -- start start 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b280cdf50 0x7f4b280ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b2806cb80 0x7f4b280726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b28072c10 0x7f4b2806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4b280ce870 con 0x7f4b2806cb80 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4b28073090 con 0x7f4b28072c10 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.535+0000 7f4b3018c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b28073370 con 0x7f4b280cdf50 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b2806cb80 0x7f4b280726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b2806cb80 0x7f4b280726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56704/0 (socket says 172.21.15.106:56704) 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 -- 172.21.15.106:0/2978206533 learned_addr learned my addr 172.21.15.106:0/2978206533 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:41.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2df28640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b280cdf50 0x7f4b280ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2e729640 1 --2- 172.21.15.106:0/2978206533 >> v2:172.21.15.114:3300/0 conn(0x7f4b28072c10 0x7f4b2806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 -- 172.21.15.106:0/2978206533 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b280cdf50 msgr2=0x7f4b280ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b280cdf50 0x7f4b280ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 -- 172.21.15.106:0/2978206533 >> v2:172.21.15.114:3300/0 conn(0x7f4b28072c10 msgr2=0x7f4b2806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 --2- 172.21.15.106:0/2978206533 >> v2:172.21.15.114:3300/0 conn(0x7f4b28072c10 0x7f4b2806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.536+0000 7f4b2d727640 1 -- 172.21.15.106:0/2978206533 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28073650 con 0x7f4b2806cb80 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b2d727640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2806cb80 0x7f4b280726d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b280cecc0 tx=0x7f4b2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec84cd8a53c129d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b2cf26640 1 -- 172.21.15.106:0/2978206533 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24035020 con 0x7f4b2806cb80 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b2cf26640 1 -- 172.21.15.106:0/2978206533 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b24031870 con 0x7f4b2806cb80 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b2cf26640 1 -- 172.21.15.106:0/2978206533 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24031c40 con 0x7f4b2806cb80 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 -- 172.21.15.106:0/2978206533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2806cb80 msgr2=0x7f4b280726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2806cb80 0x7f4b280726d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b280cecc0 tx=0x7f4b2402b910 comp rx=0 tx=0).stop 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 -- 172.21.15.106:0/2978206533 shutdown_connections 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 --2- 172.21.15.106:0/2978206533 >> v2:172.21.15.114:3300/0 conn(0x7f4b28072c10 0x7f4b2806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2806cb80 0x7f4b280726d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 --2- 172.21.15.106:0/2978206533 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b280cdf50 0x7f4b280ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 -- 172.21.15.106:0/2978206533 >> 172.21.15.106:0/2978206533 conn(0x7f4b280b1f10 msgr2=0x7f4b280b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 -- 172.21.15.106:0/2978206533 shutdown_connections 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.537+0000 7f4b3018c640 1 -- 172.21.15.106:0/2978206533 wait complete. 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.538+0000 7f4b3018c640 1 Processor -- start 2024-04-13T03:10:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.538+0000 7f4b3018c640 1 -- start start 2024-04-13T03:10:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 0x7f4b280c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b28072c10 0x7f4b2815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28099930 con 0x7f4b280cdf50 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280997b0 con 0x7f4b28072c10 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b3018c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2806be20 con 0x7f4b2806cb80 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b2e729640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b2e729640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56712/0 (socket says 172.21.15.106:56712) 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b2e729640 1 -- 172.21.15.106:0/3428968811 learned_addr learned my addr 172.21.15.106:0/3428968811 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b2df28640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 0x7f4b280c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.539+0000 7f4b2d727640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b28072c10 0x7f4b2815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 -- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 msgr2=0x7f4b280c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 0x7f4b280c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 -- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b28072c10 msgr2=0x7f4b2815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b28072c10 0x7f4b2815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 -- 172.21.15.106:0/3428968811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28158e20 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2df28640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 0x7f4b280c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b2e729640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007e90 tx=0x7f4b2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20014420 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b20019070 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b28159100 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20014820 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.540+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281596c0 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.541+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b28068a10 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.541+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b200149c0 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.542+0000 7f4b1e7fc640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 0x7f4b0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.542+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4b2008c550 con 0x7f4b280cdf50 2024-04-13T03:10:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.542+0000 7f4b2df28640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 0x7f4b0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.542+0000 7f4b2df28640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 0x7f4b0c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b18006fd0 tx=0x7f4b18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.546+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b2005a7f0 con 0x7f4b280cdf50 2024-04-13T03:10:41.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.719+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b2806eb70 con 0x7f4b0c06c8f0 2024-04-13T03:10:41.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.725+0000 7f4b1e7fc640 1 -- 172.21.15.106:0/3428968811 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f4b2806eb70 con 0x7f4b0c06c8f0 2024-04-13T03:10:41.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:41.730 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:10:41.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.727+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 msgr2=0x7f4b0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.727+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 0x7f4b0c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b18006fd0 tx=0x7f4b18008040 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.727+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 msgr2=0x7f4b28154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.727+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007e90 tx=0x7f4b2000e4b0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 shutdown_connections 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b0c06c8f0 0x7f4b0c06edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b280cdf50 0x7f4b28154320 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b28072c10 0x7f4b2815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 --2- 172.21.15.106:0/3428968811 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b2806cb80 0x7f4b280c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 >> 172.21.15.106:0/3428968811 conn(0x7f4b280b1f10 msgr2=0x7f4b280b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 shutdown_connections 2024-04-13T03:10:41.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.728+0000 7f4b3018c640 1 -- 172.21.15.106:0/3428968811 wait complete. 2024-04-13T03:10:41.739 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-13T03:10:40.377601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4428,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4534272,"data_stored":2613849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319790"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072135+0000","last_change":"2024-04-13T03:10:37.072135+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072135+0000","last_undegraded":"2024-04-13T03:10:37.072135+0000","last_fullsized":"2024-04-13T03:10:37.072135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075618+0000","last_change":"2024-04-13T03:10:37.075618+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075618+0000","last_undegraded":"2024-04-13T03:10:37.075618+0000","last_fullsized":"2024-04-13T03:10:37.075618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073998+0000","last_change":"2024-04-13T03:10:37.073998+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073998+0000","last_undegraded":"2024-04-13T03:10:37.073998+0000","last_fullsized":"2024-04-13T03:10:37.073998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071804+0000","last_change":"2024-04-13T03:10:37.071804+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071804+0000","last_undegraded":"2024-04-13T03:10:37.071804+0000","last_fullsized":"2024-04-13T03:10:37.071804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.070206+0000","last_change":"2024-04-13T03:10:37.070206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.070206+0000","last_undegraded":"2024-04-13T03:10:37.070206+0000","last_fullsized":"2024-04-13T03:10:37.070206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079460+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079460+0000","last_undegraded":"2024-04-13T03:10:37.079460+0000","last_fullsized":"2024-04-13T03:10:37.079460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072953+0000","last_change":"2024-04-13T03:10:37.072953+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072953+0000","last_undegraded":"2024-04-13T03:10:37.072953+0000","last_fullsized":"2024-04-13T03:10:37.072953+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076313+0000","last_change":"2024-04-13T03:10:37.076313+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076313+0000","last_undegraded":"2024-04-13T03:10:37.076313+0000","last_fullsized":"2024-04-13T03:10:37.076313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075430+0000","last_change":"2024-04-13T03:10:37.075430+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075430+0000","last_undegraded":"2024-04-13T03:10:37.075430+0000","last_fullsized":"2024-04-13T03:10:37.075430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074866+0000","last_change":"2024-04-13T03:10:37.074866+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074866+0000","last_undegraded":"2024-04-13T03:10:37.074866+0000","last_fullsized":"2024-04-13T03:10:37.074866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073210+0000","last_change":"2024-04-13T03:10:37.073210+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073210+0000","last_undegraded":"2024-04-13T03:10:37.073210+0000","last_fullsized":"2024-04-13T03:10:37.073210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079554+0000","last_change":"2024-04-13T03:10:37.079554+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079554+0000","last_undegraded":"2024-04-13T03:10:37.079554+0000","last_fullsized":"2024-04-13T03:10:37.079554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077683+0000","last_change":"2024-04-13T03:10:37.077683+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077683+0000","last_undegraded":"2024-04-13T03:10:37.077683+0000","last_fullsized":"2024-04-13T03:10:37.077683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076708+0000","last_change":"2024-04-13T03:10:37.076708+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076708+0000","last_undegraded":"2024-04-13T03:10:37.076708+0000","last_fullsized":"2024-04-13T03:10:37.076708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079088+0000","last_change":"2024-04-13T03:10:37.079088+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079088+0000","last_undegraded":"2024-04-13T03:10:37.079088+0000","last_fullsized":"2024-04-13T03:10:37.079088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.081502+0000","last_change":"2024-04-13T03:10:37.081502+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.081502+0000","last_undegraded":"2024-04-13T03:10:37.081502+0000","last_fullsized":"2024-04-13T03:10:37.081502+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076788+0000","last_change":"2024-04-13T03:10:37.076788+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076788+0000","last_undegraded":"2024-04-13T03:10:37.076788+0000","last_fullsized":"2024-04-13T03:10:37.076788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072395+0000","last_change":"2024-04-13T03:10:37.072395+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072395+0000","last_undegraded":"2024-04-13T03:10:37.072395+0000","last_fullsized":"2024-04-13T03:10:37.072395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076537+0000","last_change":"2024-04-13T03:10:37.076537+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076537+0000","last_undegraded":"2024-04-13T03:10:37.076537+0000","last_fullsized":"2024-04-13T03:10:37.076537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075931+0000","last_change":"2024-04-13T03:10:37.075931+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075931+0000","last_undegraded":"2024-04-13T03:10:37.075931+0000","last_fullsized":"2024-04-13T03:10:37.075931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073585+0000","last_change":"2024-04-13T03:10:37.073585+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073585+0000","last_undegraded":"2024-04-13T03:10:37.073585+0000","last_fullsized":"2024-04-13T03:10:37.073585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078497+0000","last_change":"2024-04-13T03:10:37.078497+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078497+0000","last_undegraded":"2024-04-13T03:10:37.078497+0000","last_fullsized":"2024-04-13T03:10:37.078497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074338+0000","last_change":"2024-04-13T03:10:37.074338+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074338+0000","last_undegraded":"2024-04-13T03:10:37.074338+0000","last_fullsized":"2024-04-13T03:10:37.074338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077015+0000","last_change":"2024-04-13T03:10:37.077015+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077015+0000","last_undegraded":"2024-04-13T03:10:37.077015+0000","last_fullsized":"2024-04-13T03:10:37.077015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069793+0000","last_change":"2024-04-13T03:10:37.069793+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069793+0000","last_undegraded":"2024-04-13T03:10:37.069793+0000","last_fullsized":"2024-04-13T03:10:37.069793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076536+0000","last_change":"2024-04-13T03:10:37.076536+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076536+0000","last_undegraded":"2024-04-13T03:10:37.076536+0000","last_fullsized":"2024-04-13T03:10:37.076536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074266+0000","last_change":"2024-04-13T03:10:37.074266+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074266+0000","last_undegraded":"2024-04-13T03:10:37.074266+0000","last_fullsized":"2024-04-13T03:10:37.074266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075155+0000","last_change":"2024-04-13T03:10:37.075155+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075155+0000","last_undegraded":"2024-04-13T03:10:37.075155+0000","last_fullsized":"2024-04-13T03:10:37.075155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076048+0000","last_change":"2024-04-13T03:10:37.076048+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076048+0000","last_undegraded":"2024-04-13T03:10:37.076048+0000","last_fullsized":"2024-04-13T03:10:37.076048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078031+0000","last_change":"2024-04-13T03:10:37.078031+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078031+0000","last_undegraded":"2024-04-13T03:10:37.078031+0000","last_fullsized":"2024-04-13T03:10:37.078031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069807+0000","last_change":"2024-04-13T03:10:37.069807+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069807+0000","last_undegraded":"2024-04-13T03:10:37.069807+0000","last_fullsized":"2024-04-13T03:10:37.069807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079002+0000","last_change":"2024-04-13T03:10:37.079002+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079002+0000","last_undegraded":"2024-04-13T03:10:37.079002+0000","last_fullsized":"2024-04-13T03:10:37.079002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092039+0000","last_change":"2024-04-13T03:10:37.085786+0000","last_active":"2024-04-13T03:10:37.092039+0000","last_peered":"2024-04-13T03:10:37.092039+0000","last_clean":"2024-04-13T03:10:37.092039+0000","last_became_active":"2024-04-13T03:10:37.085500+0000","last_became_peered":"2024-04-13T03:10:37.085500+0000","last_unstale":"2024-04-13T03:10:37.092039+0000","last_undegraded":"2024-04-13T03:10:37.092039+0000","last_fullsized":"2024-04-13T03:10:37.092039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:13.188140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069756+0000","last_change":"2024-04-13T03:10:37.069756+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069756+0000","last_undegraded":"2024-04-13T03:10:37.069756+0000","last_fullsized":"2024-04-13T03:10:37.069756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086872+0000","last_change":"2024-04-13T03:10:37.083216+0000","last_active":"2024-04-13T03:10:37.086872+0000","last_peered":"2024-04-13T03:10:37.086872+0000","last_clean":"2024-04-13T03:10:37.086872+0000","last_became_active":"2024-04-13T03:10:37.082975+0000","last_became_peered":"2024-04-13T03:10:37.082975+0000","last_unstale":"2024-04-13T03:10:37.086872+0000","last_undegraded":"2024-04-13T03:10:37.086872+0000","last_fullsized":"2024-04-13T03:10:37.086872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072632+0000","last_change":"2024-04-13T03:10:37.072632+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072632+0000","last_undegraded":"2024-04-13T03:10:37.072632+0000","last_fullsized":"2024-04-13T03:10:37.072632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089581+0000","last_change":"2024-04-13T03:10:37.084531+0000","last_active":"2024-04-13T03:10:37.089581+0000","last_peered":"2024-04-13T03:10:37.089581+0000","last_clean":"2024-04-13T03:10:37.089581+0000","last_became_active":"2024-04-13T03:10:37.084244+0000","last_became_peered":"2024-04-13T03:10:37.084244+0000","last_unstale":"2024-04-13T03:10:37.089581+0000","last_undegraded":"2024-04-13T03:10:37.089581+0000","last_fullsized":"2024-04-13T03:10:37.089581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072539+0000","last_change":"2024-04-13T03:10:37.072539+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072539+0000","last_undegraded":"2024-04-13T03:10:37.072539+0000","last_fullsized":"2024-04-13T03:10:37.072539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086974+0000","last_change":"2024-04-13T03:10:37.083577+0000","last_active":"2024-04-13T03:10:37.086974+0000","last_peered":"2024-04-13T03:10:37.086974+0000","last_clean":"2024-04-13T03:10:37.086974+0000","last_became_active":"2024-04-13T03:10:37.083299+0000","last_became_peered":"2024-04-13T03:10:37.083299+0000","last_unstale":"2024-04-13T03:10:37.086974+0000","last_undegraded":"2024-04-13T03:10:37.086974+0000","last_fullsized":"2024-04-13T03:10:37.086974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072796+0000","last_change":"2024-04-13T03:10:37.072796+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072796+0000","last_undegraded":"2024-04-13T03:10:37.072796+0000","last_fullsized":"2024-04-13T03:10:37.072796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089546+0000","last_change":"2024-04-13T03:10:37.086817+0000","last_active":"2024-04-13T03:10:37.089546+0000","last_peered":"2024-04-13T03:10:37.089546+0000","last_clean":"2024-04-13T03:10:37.089546+0000","last_became_active":"2024-04-13T03:10:37.086544+0000","last_became_peered":"2024-04-13T03:10:37.086544+0000","last_unstale":"2024-04-13T03:10:37.089546+0000","last_undegraded":"2024-04-13T03:10:37.089546+0000","last_fullsized":"2024-04-13T03:10:37.089546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074859+0000","last_change":"2024-04-13T03:10:37.074859+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074859+0000","last_undegraded":"2024-04-13T03:10:37.074859+0000","last_fullsized":"2024-04-13T03:10:37.074859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094811+0000","last_change":"2024-04-13T03:10:37.089557+0000","last_active":"2024-04-13T03:10:37.094811+0000","last_peered":"2024-04-13T03:10:37.094811+0000","last_clean":"2024-04-13T03:10:37.094811+0000","last_became_active":"2024-04-13T03:10:37.089249+0000","last_became_peered":"2024-04-13T03:10:37.089249+0000","last_unstale":"2024-04-13T03:10:37.094811+0000","last_undegraded":"2024-04-13T03:10:37.094811+0000","last_fullsized":"2024-04-13T03:10:37.094811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:19.190392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073086+0000","last_change":"2024-04-13T03:10:37.073086+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073086+0000","last_undegraded":"2024-04-13T03:10:37.073086+0000","last_fullsized":"2024-04-13T03:10:37.073086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092667+0000","last_change":"2024-04-13T03:10:37.088764+0000","last_active":"2024-04-13T03:10:37.092667+0000","last_peered":"2024-04-13T03:10:37.092667+0000","last_clean":"2024-04-13T03:10:37.092667+0000","last_became_active":"2024-04-13T03:10:37.088507+0000","last_became_peered":"2024-04-13T03:10:37.088507+0000","last_unstale":"2024-04-13T03:10:37.092667+0000","last_undegraded":"2024-04-13T03:10:37.092667+0000","last_fullsized":"2024-04-13T03:10:37.092667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074137+0000","last_change":"2024-04-13T03:10:37.074137+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074137+0000","last_undegraded":"2024-04-13T03:10:37.074137+0000","last_fullsized":"2024-04-13T03:10:37.074137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087592+0000","last_change":"2024-04-13T03:10:37.083447+0000","last_active":"2024-04-13T03:10:37.087592+0000","last_peered":"2024-04-13T03:10:37.087592+0000","last_clean":"2024-04-13T03:10:37.087592+0000","last_became_active":"2024-04-13T03:10:37.083159+0000","last_became_peered":"2024-04-13T03:10:37.083159+0000","last_unstale":"2024-04-13T03:10:37.087592+0000","last_undegraded":"2024-04-13T03:10:37.087592+0000","last_fullsized":"2024-04-13T03:10:37.087592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071855+0000","last_change":"2024-04-13T03:10:37.071855+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071855+0000","last_undegraded":"2024-04-13T03:10:37.071855+0000","last_fullsized":"2024-04-13T03:10:37.071855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083991+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083991+0000","last_peered":"2024-04-13T03:10:37.083991+0000","last_clean":"2024-04-13T03:10:37.083991+0000","last_became_active":"2024-04-13T03:10:37.081899+0000","last_became_peered":"2024-04-13T03:10:37.081899+0000","last_unstale":"2024-04-13T03:10:37.083991+0000","last_undegraded":"2024-04-13T03:10:37.083991+0000","last_fullsized":"2024-04-13T03:10:37.083991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:04:52.620360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.084130+0000","last_change":"2024-04-13T03:10:37.084130+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.084130+0000","last_undegraded":"2024-04-13T03:10:37.084130+0000","last_fullsized":"2024-04-13T03:10:37.084130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087058+0000","last_change":"2024-04-13T03:10:37.085297+0000","last_active":"2024-04-13T03:10:37.087058+0000","last_peered":"2024-04-13T03:10:37.087058+0000","last_clean":"2024-04-13T03:10:37.087058+0000","last_became_active":"2024-04-13T03:10:37.085009+0000","last_became_peered":"2024-04-13T03:10:37.085009+0000","last_unstale":"2024-04-13T03:10:37.087058+0000","last_undegraded":"2024-04-13T03:10:37.087058+0000","last_fullsized":"2024-04-13T03:10:37.087058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092276+0000","last_change":"2024-04-13T03:10:37.088374+0000","last_active":"2024-04-13T03:10:37.092276+0000","last_peered":"2024-04-13T03:10:37.092276+0000","last_clean":"2024-04-13T03:10:37.092276+0000","last_became_active":"2024-04-13T03:10:37.088068+0000","last_became_peered":"2024-04-13T03:10:37.088068+0000","last_unstale":"2024-04-13T03:10:37.092276+0000","last_undegraded":"2024-04-13T03:10:37.092276+0000","last_fullsized":"2024-04-13T03:10:37.092276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073472+0000","last_change":"2024-04-13T03:10:37.073472+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073472+0000","last_undegraded":"2024-04-13T03:10:37.073472+0000","last_fullsized":"2024-04-13T03:10:37.073472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083655+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083655+0000","last_peered":"2024-04-13T03:10:37.083655+0000","last_clean":"2024-04-13T03:10:37.083655+0000","last_became_active":"2024-04-13T03:10:37.081896+0000","last_became_peered":"2024-04-13T03:10:37.081896+0000","last_unstale":"2024-04-13T03:10:37.083655+0000","last_undegraded":"2024-04-13T03:10:37.083655+0000","last_fullsized":"2024-04-13T03:10:37.083655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:38.859158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072382+0000","last_change":"2024-04-13T03:10:37.072382+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072382+0000","last_undegraded":"2024-04-13T03:10:37.072382+0000","last_fullsized":"2024-04-13T03:10:37.072382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.082603+0000","last_change":"2024-04-13T03:10:37.081457+0000","last_active":"2024-04-13T03:10:37.082603+0000","last_peered":"2024-04-13T03:10:37.082603+0000","last_clean":"2024-04-13T03:10:37.082603+0000","last_became_active":"2024-04-13T03:10:37.081232+0000","last_became_peered":"2024-04-13T03:10:37.081232+0000","last_unstale":"2024-04-13T03:10:37.082603+0000","last_undegraded":"2024-04-13T03:10:37.082603+0000","last_fullsized":"2024-04-13T03:10:37.082603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:13.251395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074188+0000","last_change":"2024-04-13T03:10:37.074188+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074188+0000","last_undegraded":"2024-04-13T03:10:37.074188+0000","last_fullsized":"2024-04-13T03:10:37.074188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.083620+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.083346+0000","last_became_peered":"2024-04-13T03:10:37.083346+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079369+0000","last_change":"2024-04-13T03:10:37.079369+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079369+0000","last_undegraded":"2024-04-13T03:10:37.079369+0000","last_fullsized":"2024-04-13T03:10:37.079369+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092066+0000","last_change":"2024-04-13T03:10:37.086440+0000","last_active":"2024-04-13T03:10:37.092066+0000","last_peered":"2024-04-13T03:10:37.092066+0000","last_clean":"2024-04-13T03:10:37.092066+0000","last_became_active":"2024-04-13T03:10:37.086158+0000","last_became_peered":"2024-04-13T03:10:37.086158+0000","last_unstale":"2024-04-13T03:10:37.092066+0000","last_undegraded":"2024-04-13T03:10:37.092066+0000","last_fullsized":"2024-04-13T03:10:37.092066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:07.798696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073362+0000","last_change":"2024-04-13T03:10:37.073362+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073362+0000","last_undegraded":"2024-04-13T03:10:37.073362+0000","last_fullsized":"2024-04-13T03:10:37.073362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094829+0000","last_change":"2024-04-13T03:10:37.089769+0000","last_active":"2024-04-13T03:10:37.094829+0000","last_peered":"2024-04-13T03:10:37.094829+0000","last_clean":"2024-04-13T03:10:37.094829+0000","last_became_active":"2024-04-13T03:10:37.089284+0000","last_became_peered":"2024-04-13T03:10:37.089284+0000","last_unstale":"2024-04-13T03:10:37.094829+0000","last_undegraded":"2024-04-13T03:10:37.094829+0000","last_fullsized":"2024-04-13T03:10:37.094829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:21.425166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075351+0000","last_change":"2024-04-13T03:10:37.075351+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075351+0000","last_undegraded":"2024-04-13T03:10:37.075351+0000","last_fullsized":"2024-04-13T03:10:37.075351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092234+0000","last_change":"2024-04-13T03:10:37.087631+0000","last_active":"2024-04-13T03:10:37.092234+0000","last_peered":"2024-04-13T03:10:37.092234+0000","last_clean":"2024-04-13T03:10:37.092234+0000","last_became_active":"2024-04-13T03:10:37.087385+0000","last_became_peered":"2024-04-13T03:10:37.087385+0000","last_unstale":"2024-04-13T03:10:37.092234+0000","last_undegraded":"2024-04-13T03:10:37.092234+0000","last_fullsized":"2024-04-13T03:10:37.092234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074341+0000","last_change":"2024-04-13T03:10:37.074341+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074341+0000","last_undegraded":"2024-04-13T03:10:37.074341+0000","last_fullsized":"2024-04-13T03:10:37.074341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087689+0000","last_change":"2024-04-13T03:10:37.085447+0000","last_active":"2024-04-13T03:10:37.087689+0000","last_peered":"2024-04-13T03:10:37.087689+0000","last_clean":"2024-04-13T03:10:37.087689+0000","last_became_active":"2024-04-13T03:10:37.085157+0000","last_became_peered":"2024-04-13T03:10:37.085157+0000","last_unstale":"2024-04-13T03:10:37.087689+0000","last_undegraded":"2024-04-13T03:10:37.087689+0000","last_fullsized":"2024-04-13T03:10:37.087689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073200+0000","last_change":"2024-04-13T03:10:37.073200+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073200+0000","last_undegraded":"2024-04-13T03:10:37.073200+0000","last_fullsized":"2024-04-13T03:10:37.073200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092500+0000","last_change":"2024-04-13T03:10:37.087391+0000","last_active":"2024-04-13T03:10:37.092500+0000","last_peered":"2024-04-13T03:10:37.092500+0000","last_clean":"2024-04-13T03:10:37.092500+0000","last_became_active":"2024-04-13T03:10:37.087131+0000","last_became_peered":"2024-04-13T03:10:37.087131+0000","last_unstale":"2024-04-13T03:10:37.092500+0000","last_undegraded":"2024-04-13T03:10:37.092500+0000","last_fullsized":"2024-04-13T03:10:37.092500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:41.946253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071737+0000","last_change":"2024-04-13T03:10:37.071737+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071737+0000","last_undegraded":"2024-04-13T03:10:37.071737+0000","last_fullsized":"2024-04-13T03:10:37.071737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089556+0000","last_change":"2024-04-13T03:10:37.084719+0000","last_active":"2024-04-13T03:10:37.089556+0000","last_peered":"2024-04-13T03:10:37.089556+0000","last_clean":"2024-04-13T03:10:37.089556+0000","last_became_active":"2024-04-13T03:10:37.084472+0000","last_became_peered":"2024-04-13T03:10:37.084472+0000","last_unstale":"2024-04-13T03:10:37.089556+0000","last_undegraded":"2024-04-13T03:10:37.089556+0000","last_fullsized":"2024-04-13T03:10:37.089556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078007+0000","last_change":"2024-04-13T03:10:37.078007+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078007+0000","last_undegraded":"2024-04-13T03:10:37.078007+0000","last_fullsized":"2024-04-13T03:10:37.078007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080873+0000","last_change":"2024-04-13T03:10:37.078763+0000","last_active":"2024-04-13T03:10:37.080873+0000","last_peered":"2024-04-13T03:10:37.080873+0000","last_clean":"2024-04-13T03:10:37.080873+0000","last_became_active":"2024-04-13T03:10:37.078484+0000","last_became_peered":"2024-04-13T03:10:37.078484+0000","last_unstale":"2024-04-13T03:10:37.080873+0000","last_undegraded":"2024-04-13T03:10:37.080873+0000","last_fullsized":"2024-04-13T03:10:37.080873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:54:33.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074170+0000","last_change":"2024-04-13T03:10:37.074170+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074170+0000","last_undegraded":"2024-04-13T03:10:37.074170+0000","last_fullsized":"2024-04-13T03:10:37.074170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.085242+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.084999+0000","last_became_peered":"2024-04-13T03:10:37.084999+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:31:56.747006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077206+0000","last_change":"2024-04-13T03:10:37.077206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077206+0000","last_undegraded":"2024-04-13T03:10:37.077206+0000","last_fullsized":"2024-04-13T03:10:37.077206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092322+0000","last_change":"2024-04-13T03:10:37.088188+0000","last_active":"2024-04-13T03:10:37.092322+0000","last_peered":"2024-04-13T03:10:37.092322+0000","last_clean":"2024-04-13T03:10:37.092322+0000","last_became_active":"2024-04-13T03:10:37.087921+0000","last_became_peered":"2024-04-13T03:10:37.087921+0000","last_unstale":"2024-04-13T03:10:37.092322+0000","last_undegraded":"2024-04-13T03:10:37.092322+0000","last_fullsized":"2024-04-13T03:10:37.092322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:01.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073727+0000","last_change":"2024-04-13T03:10:37.073727+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073727+0000","last_undegraded":"2024-04-13T03:10:37.073727+0000","last_fullsized":"2024-04-13T03:10:37.073727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086411+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.086411+0000","last_peered":"2024-04-13T03:10:37.086411+0000","last_clean":"2024-04-13T03:10:37.086411+0000","last_became_active":"2024-04-13T03:10:37.079188+0000","last_became_peered":"2024-04-13T03:10:37.079188+0000","last_unstale":"2024-04-13T03:10:37.086411+0000","last_undegraded":"2024-04-13T03:10:37.086411+0000","last_fullsized":"2024-04-13T03:10:37.086411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:41.463691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069800+0000","last_change":"2024-04-13T03:10:37.069800+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069800+0000","last_undegraded":"2024-04-13T03:10:37.069800+0000","last_fullsized":"2024-04-13T03:10:37.069800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092256+0000","last_change":"2024-04-13T03:10:37.088272+0000","last_active":"2024-04-13T03:10:37.092256+0000","last_peered":"2024-04-13T03:10:37.092256+0000","last_clean":"2024-04-13T03:10:37.092256+0000","last_became_active":"2024-04-13T03:10:37.088023+0000","last_became_peered":"2024-04-13T03:10:37.088023+0000","last_unstale":"2024-04-13T03:10:37.092256+0000","last_undegraded":"2024-04-13T03:10:37.092256+0000","last_fullsized":"2024-04-13T03:10:37.092256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.083615+0000","last_change":"2024-04-13T03:10:37.083615+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.083615+0000","last_undegraded":"2024-04-13T03:10:37.083615+0000","last_fullsized":"2024-04-13T03:10:37.083615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089636+0000","last_change":"2024-04-13T03:10:37.080915+0000","last_active":"2024-04-13T03:10:37.089636+0000","last_peered":"2024-04-13T03:10:37.089636+0000","last_clean":"2024-04-13T03:10:37.089636+0000","last_became_active":"2024-04-13T03:10:37.080658+0000","last_became_peered":"2024-04-13T03:10:37.080658+0000","last_unstale":"2024-04-13T03:10:37.089636+0000","last_undegraded":"2024-04-13T03:10:37.089636+0000","last_fullsized":"2024-04-13T03:10:37.089636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:02.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075238+0000","last_change":"2024-04-13T03:10:37.075238+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075238+0000","last_undegraded":"2024-04-13T03:10:37.075238+0000","last_fullsized":"2024-04-13T03:10:37.075238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086721+0000","last_change":"2024-04-13T03:10:37.081490+0000","last_active":"2024-04-13T03:10:37.086721+0000","last_peered":"2024-04-13T03:10:37.086721+0000","last_clean":"2024-04-13T03:10:37.086721+0000","last_became_active":"2024-04-13T03:10:37.081226+0000","last_became_peered":"2024-04-13T03:10:37.081226+0000","last_unstale":"2024-04-13T03:10:37.086721+0000","last_undegraded":"2024-04-13T03:10:37.086721+0000","last_fullsized":"2024-04-13T03:10:37.086721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071935+0000","last_change":"2024-04-13T03:10:37.071935+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071935+0000","last_undegraded":"2024-04-13T03:10:37.071935+0000","last_fullsized":"2024-04-13T03:10:37.071935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086903+0000","last_change":"2024-04-13T03:10:37.083750+0000","last_active":"2024-04-13T03:10:37.086903+0000","last_peered":"2024-04-13T03:10:37.086903+0000","last_clean":"2024-04-13T03:10:37.086903+0000","last_became_active":"2024-04-13T03:10:37.083512+0000","last_became_peered":"2024-04-13T03:10:37.083512+0000","last_unstale":"2024-04-13T03:10:37.086903+0000","last_undegraded":"2024-04-13T03:10:37.086903+0000","last_fullsized":"2024-04-13T03:10:37.086903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069680+0000","last_change":"2024-04-13T03:10:37.069680+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069680+0000","last_undegraded":"2024-04-13T03:10:37.069680+0000","last_fullsized":"2024-04-13T03:10:37.069680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087143+0000","last_change":"2024-04-13T03:10:37.082886+0000","last_active":"2024-04-13T03:10:37.087143+0000","last_peered":"2024-04-13T03:10:37.087143+0000","last_clean":"2024-04-13T03:10:37.087143+0000","last_became_active":"2024-04-13T03:10:37.082622+0000","last_became_peered":"2024-04-13T03:10:37.082622+0000","last_unstale":"2024-04-13T03:10:37.087143+0000","last_undegraded":"2024-04-13T03:10:37.087143+0000","last_fullsized":"2024-04-13T03:10:37.087143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:41.556737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077247+0000","last_change":"2024-04-13T03:10:37.077247+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077247+0000","last_undegraded":"2024-04-13T03:10:37.077247+0000","last_fullsized":"2024-04-13T03:10:37.077247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087667+0000","last_change":"2024-04-13T03:10:37.084769+0000","last_active":"2024-04-13T03:10:37.087667+0000","last_peered":"2024-04-13T03:10:37.087667+0000","last_clean":"2024-04-13T03:10:37.087667+0000","last_became_active":"2024-04-13T03:10:37.084496+0000","last_became_peered":"2024-04-13T03:10:37.084496+0000","last_unstale":"2024-04-13T03:10:37.087667+0000","last_undegraded":"2024-04-13T03:10:37.087667+0000","last_fullsized":"2024-04-13T03:10:37.087667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079692+0000","last_change":"2024-04-13T03:10:37.079692+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079692+0000","last_undegraded":"2024-04-13T03:10:37.079692+0000","last_fullsized":"2024-04-13T03:10:37.079692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083507+0000","last_change":"2024-04-13T03:10:37.082340+0000","last_active":"2024-04-13T03:10:37.083507+0000","last_peered":"2024-04-13T03:10:37.083507+0000","last_clean":"2024-04-13T03:10:37.083507+0000","last_became_active":"2024-04-13T03:10:37.082069+0000","last_became_peered":"2024-04-13T03:10:37.082069+0000","last_unstale":"2024-04-13T03:10:37.083507+0000","last_undegraded":"2024-04-13T03:10:37.083507+0000","last_fullsized":"2024-04-13T03:10:37.083507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:42.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076397+0000","last_change":"2024-04-13T03:10:37.076397+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076397+0000","last_undegraded":"2024-04-13T03:10:37.076397+0000","last_fullsized":"2024-04-13T03:10:37.076397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094834+0000","last_change":"2024-04-13T03:10:37.090291+0000","last_active":"2024-04-13T03:10:37.094834+0000","last_peered":"2024-04-13T03:10:37.094834+0000","last_clean":"2024-04-13T03:10:37.094834+0000","last_became_active":"2024-04-13T03:10:37.090004+0000","last_became_peered":"2024-04-13T03:10:37.090004+0000","last_unstale":"2024-04-13T03:10:37.094834+0000","last_undegraded":"2024-04-13T03:10:37.094834+0000","last_fullsized":"2024-04-13T03:10:37.094834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080293+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:10:37.080293+0000","last_peered":"2024-04-13T03:10:37.080293+0000","last_clean":"2024-04-13T03:10:37.080293+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:10:37.080293+0000","last_undegraded":"2024-04-13T03:10:37.080293+0000","last_fullsized":"2024-04-13T03:10:37.080293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:10:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-13T03:10:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072135+0000', 'last_change': '2024-04-13T03:10:37.072135+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072135+0000', 'last_undegraded': '2024-04-13T03:10:37.072135+0000', 'last_fullsized': '2024-04-13T03:10:37.072135+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-13T03:10:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075618+0000', 'last_change': '2024-04-13T03:10:37.075618+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075618+0000', 'last_undegraded': '2024-04-13T03:10:37.075618+0000', 'last_fullsized': '2024-04-13T03:10:37.075618+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073998+0000', 'last_change': '2024-04-13T03:10:37.073998+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073998+0000', 'last_undegraded': '2024-04-13T03:10:37.073998+0000', 'last_fullsized': '2024-04-13T03:10:37.073998+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072457+0000', 'last_change': '2024-04-13T03:10:37.072457+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072457+0000', 'last_undegraded': '2024-04-13T03:10:37.072457+0000', 'last_fullsized': '2024-04-13T03:10:37.072457+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.071804+0000', 'last_change': '2024-04-13T03:10:37.071804+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.071804+0000', 'last_undegraded': '2024-04-13T03:10:37.071804+0000', 'last_fullsized': '2024-04-13T03:10:37.071804+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.070206+0000', 'last_change': '2024-04-13T03:10:37.070206+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.070206+0000', 'last_undegraded': '2024-04-13T03:10:37.070206+0000', 'last_fullsized': '2024-04-13T03:10:37.070206+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-13T03:10:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079460+0000', 'last_change': '2024-04-13T03:10:37.079460+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079460+0000', 'last_undegraded': '2024-04-13T03:10:37.079460+0000', 'last_fullsized': '2024-04-13T03:10:37.079460+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072953+0000', 'last_change': '2024-04-13T03:10:37.072953+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072953+0000', 'last_undegraded': '2024-04-13T03:10:37.072953+0000', 'last_fullsized': '2024-04-13T03:10:37.072953+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076313+0000', 'last_change': '2024-04-13T03:10:37.076313+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076313+0000', 'last_undegraded': '2024-04-13T03:10:37.076313+0000', 'last_fullsized': '2024-04-13T03:10:37.076313+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075430+0000', 'last_change': '2024-04-13T03:10:37.075430+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075430+0000', 'last_undegraded': '2024-04-13T03:10:37.075430+0000', 'last_fullsized': '2024-04-13T03:10:37.075430+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074866+0000', 'last_change': '2024-04-13T03:10:37.074866+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074866+0000', 'last_undegraded': '2024-04-13T03:10:37.074866+0000', 'last_fullsized': '2024-04-13T03:10:37.074866+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073210+0000', 'last_change': '2024-04-13T03:10:37.073210+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073210+0000', 'last_undegraded': '2024-04-13T03:10:37.073210+0000', 'last_fullsized': '2024-04-13T03:10:37.073210+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079554+0000', 'last_change': '2024-04-13T03:10:37.079554+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079554+0000', 'last_undegraded': '2024-04-13T03:10:37.079554+0000', 'last_fullsized': '2024-04-13T03:10:37.079554+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.077683+0000', 'last_change': '2024-04-13T03:10:37.077683+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.077683+0000', 'last_undegraded': '2024-04-13T03:10:37.077683+0000', 'last_fullsized': '2024-04-13T03:10:37.077683+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076708+0000', 'last_change': '2024-04-13T03:10:37.076708+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076708+0000', 'last_undegraded': '2024-04-13T03:10:37.076708+0000', 'last_fullsized': '2024-04-13T03:10:37.076708+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-13T03:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079088+0000', 'last_change': '2024-04-13T03:10:37.079088+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079088+0000', 'last_undegraded': '2024-04-13T03:10:37.079088+0000', 'last_fullsized': '2024-04-13T03:10:37.079088+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.081502+0000', 'last_change': '2024-04-13T03:10:37.081502+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.081502+0000', 'last_undegraded': '2024-04-13T03:10:37.081502+0000', 'last_fullsized': '2024-04-13T03:10:37.081502+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076788+0000', 'last_change': '2024-04-13T03:10:37.076788+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076788+0000', 'last_undegraded': '2024-04-13T03:10:37.076788+0000', 'last_fullsized': '2024-04-13T03:10:37.076788+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072395+0000', 'last_change': '2024-04-13T03:10:37.072395+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072395+0000', 'last_undegraded': '2024-04-13T03:10:37.072395+0000', 'last_fullsized': '2024-04-13T03:10:37.072395+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076537+0000', 'last_change': '2024-04-13T03:10:37.076537+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076537+0000', 'last_undegraded': '2024-04-13T03:10:37.076537+0000', 'last_fullsized': '2024-04-13T03:10:37.076537+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-13T03:10:41.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075931+0000', 'last_change': '2024-04-13T03:10:37.075931+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075931+0000', 'last_undegraded': '2024-04-13T03:10:37.075931+0000', 'last_fullsized': '2024-04-13T03:10:37.075931+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073585+0000', 'last_change': '2024-04-13T03:10:37.073585+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073585+0000', 'last_undegraded': '2024-04-13T03:10:37.073585+0000', 'last_fullsized': '2024-04-13T03:10:37.073585+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.078497+0000', 'last_change': '2024-04-13T03:10:37.078497+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.078497+0000', 'last_undegraded': '2024-04-13T03:10:37.078497+0000', 'last_fullsized': '2024-04-13T03:10:37.078497+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074338+0000', 'last_change': '2024-04-13T03:10:37.074338+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074338+0000', 'last_undegraded': '2024-04-13T03:10:37.074338+0000', 'last_fullsized': '2024-04-13T03:10:37.074338+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.077015+0000', 'last_change': '2024-04-13T03:10:37.077015+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.077015+0000', 'last_undegraded': '2024-04-13T03:10:37.077015+0000', 'last_fullsized': '2024-04-13T03:10:37.077015+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-13T03:10:41.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.069793+0000', 'last_change': '2024-04-13T03:10:37.069793+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.069793+0000', 'last_undegraded': '2024-04-13T03:10:37.069793+0000', 'last_fullsized': '2024-04-13T03:10:37.069793+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076536+0000', 'last_change': '2024-04-13T03:10:37.076536+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076536+0000', 'last_undegraded': '2024-04-13T03:10:37.076536+0000', 'last_fullsized': '2024-04-13T03:10:37.076536+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074266+0000', 'last_change': '2024-04-13T03:10:37.074266+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074266+0000', 'last_undegraded': '2024-04-13T03:10:37.074266+0000', 'last_fullsized': '2024-04-13T03:10:37.074266+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075155+0000', 'last_change': '2024-04-13T03:10:37.075155+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075155+0000', 'last_undegraded': '2024-04-13T03:10:37.075155+0000', 'last_fullsized': '2024-04-13T03:10:37.075155+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076048+0000', 'last_change': '2024-04-13T03:10:37.076048+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076048+0000', 'last_undegraded': '2024-04-13T03:10:37.076048+0000', 'last_fullsized': '2024-04-13T03:10:37.076048+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-13T03:10:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.078031+0000', 'last_change': '2024-04-13T03:10:37.078031+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.078031+0000', 'last_undegraded': '2024-04-13T03:10:37.078031+0000', 'last_fullsized': '2024-04-13T03:10:37.078031+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.069807+0000', 'last_change': '2024-04-13T03:10:37.069807+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.069807+0000', 'last_undegraded': '2024-04-13T03:10:37.069807+0000', 'last_fullsized': '2024-04-13T03:10:37.069807+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079002+0000', 'last_change': '2024-04-13T03:10:37.079002+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079002+0000', 'last_undegraded': '2024-04-13T03:10:37.079002+0000', 'last_fullsized': '2024-04-13T03:10:37.079002+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.069756+0000', 'last_change': '2024-04-13T03:10:37.069756+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.069756+0000', 'last_undegraded': '2024-04-13T03:10:37.069756+0000', 'last_fullsized': '2024-04-13T03:10:37.069756+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072632+0000', 'last_change': '2024-04-13T03:10:37.072632+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072632+0000', 'last_undegraded': '2024-04-13T03:10:37.072632+0000', 'last_fullsized': '2024-04-13T03:10:37.072632+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072539+0000', 'last_change': '2024-04-13T03:10:37.072539+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072539+0000', 'last_undegraded': '2024-04-13T03:10:37.072539+0000', 'last_fullsized': '2024-04-13T03:10:37.072539+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072796+0000', 'last_change': '2024-04-13T03:10:37.072796+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072796+0000', 'last_undegraded': '2024-04-13T03:10:37.072796+0000', 'last_fullsized': '2024-04-13T03:10:37.072796+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074859+0000', 'last_change': '2024-04-13T03:10:37.074859+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074859+0000', 'last_undegraded': '2024-04-13T03:10:37.074859+0000', 'last_fullsized': '2024-04-13T03:10:37.074859+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073086+0000', 'last_change': '2024-04-13T03:10:37.073086+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073086+0000', 'last_undegraded': '2024-04-13T03:10:37.073086+0000', 'last_fullsized': '2024-04-13T03:10:37.073086+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074137+0000', 'last_change': '2024-04-13T03:10:37.074137+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074137+0000', 'last_undegraded': '2024-04-13T03:10:37.074137+0000', 'last_fullsized': '2024-04-13T03:10:37.074137+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.071855+0000', 'last_change': '2024-04-13T03:10:37.071855+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.071855+0000', 'last_undegraded': '2024-04-13T03:10:37.071855+0000', 'last_fullsized': '2024-04-13T03:10:37.071855+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.084130+0000', 'last_change': '2024-04-13T03:10:37.084130+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.084130+0000', 'last_undegraded': '2024-04-13T03:10:37.084130+0000', 'last_fullsized': '2024-04-13T03:10:37.084130+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072457+0000', 'last_change': '2024-04-13T03:10:37.072457+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072457+0000', 'last_undegraded': '2024-04-13T03:10:37.072457+0000', 'last_fullsized': '2024-04-13T03:10:37.072457+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073472+0000', 'last_change': '2024-04-13T03:10:37.073472+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073472+0000', 'last_undegraded': '2024-04-13T03:10:37.073472+0000', 'last_fullsized': '2024-04-13T03:10:37.073472+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-13T03:10:41.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.072382+0000', 'last_change': '2024-04-13T03:10:37.072382+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.072382+0000', 'last_undegraded': '2024-04-13T03:10:37.072382+0000', 'last_fullsized': '2024-04-13T03:10:37.072382+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074188+0000', 'last_change': '2024-04-13T03:10:37.074188+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074188+0000', 'last_undegraded': '2024-04-13T03:10:37.074188+0000', 'last_fullsized': '2024-04-13T03:10:37.074188+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079369+0000', 'last_change': '2024-04-13T03:10:37.079369+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079369+0000', 'last_undegraded': '2024-04-13T03:10:37.079369+0000', 'last_fullsized': '2024-04-13T03:10:37.079369+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073362+0000', 'last_change': '2024-04-13T03:10:37.073362+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073362+0000', 'last_undegraded': '2024-04-13T03:10:37.073362+0000', 'last_fullsized': '2024-04-13T03:10:37.073362+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075351+0000', 'last_change': '2024-04-13T03:10:37.075351+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075351+0000', 'last_undegraded': '2024-04-13T03:10:37.075351+0000', 'last_fullsized': '2024-04-13T03:10:37.075351+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-13T03:10:41.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074341+0000', 'last_change': '2024-04-13T03:10:37.074341+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074341+0000', 'last_undegraded': '2024-04-13T03:10:37.074341+0000', 'last_fullsized': '2024-04-13T03:10:37.074341+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073200+0000', 'last_change': '2024-04-13T03:10:37.073200+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073200+0000', 'last_undegraded': '2024-04-13T03:10:37.073200+0000', 'last_fullsized': '2024-04-13T03:10:37.073200+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.071737+0000', 'last_change': '2024-04-13T03:10:37.071737+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.071737+0000', 'last_undegraded': '2024-04-13T03:10:37.071737+0000', 'last_fullsized': '2024-04-13T03:10:37.071737+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.078007+0000', 'last_change': '2024-04-13T03:10:37.078007+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.078007+0000', 'last_undegraded': '2024-04-13T03:10:37.078007+0000', 'last_fullsized': '2024-04-13T03:10:37.078007+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.074170+0000', 'last_change': '2024-04-13T03:10:37.074170+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.074170+0000', 'last_undegraded': '2024-04-13T03:10:37.074170+0000', 'last_fullsized': '2024-04-13T03:10:37.074170+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-13T03:10:41.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.077206+0000', 'last_change': '2024-04-13T03:10:37.077206+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.077206+0000', 'last_undegraded': '2024-04-13T03:10:37.077206+0000', 'last_fullsized': '2024-04-13T03:10:37.077206+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.073727+0000', 'last_change': '2024-04-13T03:10:37.073727+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.073727+0000', 'last_undegraded': '2024-04-13T03:10:37.073727+0000', 'last_fullsized': '2024-04-13T03:10:37.073727+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.069800+0000', 'last_change': '2024-04-13T03:10:37.069800+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.069800+0000', 'last_undegraded': '2024-04-13T03:10:37.069800+0000', 'last_fullsized': '2024-04-13T03:10:37.069800+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.083615+0000', 'last_change': '2024-04-13T03:10:37.083615+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.083615+0000', 'last_undegraded': '2024-04-13T03:10:37.083615+0000', 'last_fullsized': '2024-04-13T03:10:37.083615+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-13T03:10:41.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.075238+0000', 'last_change': '2024-04-13T03:10:37.075238+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.075238+0000', 'last_undegraded': '2024-04-13T03:10:37.075238+0000', 'last_fullsized': '2024-04-13T03:10:37.075238+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.071935+0000', 'last_change': '2024-04-13T03:10:37.071935+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.071935+0000', 'last_undegraded': '2024-04-13T03:10:37.071935+0000', 'last_fullsized': '2024-04-13T03:10:37.071935+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.069680+0000', 'last_change': '2024-04-13T03:10:37.069680+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.069680+0000', 'last_undegraded': '2024-04-13T03:10:37.069680+0000', 'last_fullsized': '2024-04-13T03:10:37.069680+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.077247+0000', 'last_change': '2024-04-13T03:10:37.077247+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.077247+0000', 'last_undegraded': '2024-04-13T03:10:37.077247+0000', 'last_fullsized': '2024-04-13T03:10:37.077247+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.079692+0000', 'last_change': '2024-04-13T03:10:37.079692+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.079692+0000', 'last_undegraded': '2024-04-13T03:10:37.079692+0000', 'last_fullsized': '2024-04-13T03:10:37.079692+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-13T03:10:41.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:10:37.076397+0000', 'last_change': '2024-04-13T03:10:37.076397+0000', 'last_active': '2024-04-13T03:10:37.057207+0000', 'last_peered': '2024-04-13T03:10:37.057207+0000', 'last_clean': '2024-04-13T03:10:37.057207+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:10:37.076397+0000', 'last_undegraded': '2024-04-13T03:10:37.076397+0000', 'last_fullsized': '2024-04-13T03:10:37.076397+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'last_clean_scrub_stamp': '2024-04-13T03:10:37.057207+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:10:41.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-13T03:10:41.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.878+0000 7f601cb15640 1 Processor -- start 2024-04-13T03:10:41.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.878+0000 7f601cb15640 1 -- start start 2024-04-13T03:10:41.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601cb15640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f601815ca70 0x7f601806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601cb15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601cb15640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601815ce50 con 0x7f60180cd390 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601cb15640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60180cd7c0 con 0x7f601815ca70 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601cb15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6018070480 con 0x7f60180b7ef0 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f601815ca70 0x7f601806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f6016d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f60180cd390 0x7f6018160580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55000/0 (socket says 172.21.15.106:55000) 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f601815ca70 0x7f601806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56728/0 (socket says 172.21.15.106:56728) 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f6016d9d640 1 -- 172.21.15.106:0/3036966175 learned_addr learned my addr 172.21.15.106:0/3036966175 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f6015d9b640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 -- 172.21.15.106:0/3036966175 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 msgr2=0x7f60180cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 -- 172.21.15.106:0/3036966175 >> v1:172.21.15.106:6789/0 conn(0x7f60180cd390 legacy=0x7f6018160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.879+0000 7f601659c640 1 -- 172.21.15.106:0/3036966175 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6018070760 con 0x7f601815ca70 2024-04-13T03:10:41.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.880+0000 7f601659c640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f601815ca70 0x7f601806fea0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2207001c6b15024b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.880+0000 7f601559a640 1 -- 172.21.15.106:0/3036966175 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008039070 con 0x7f601815ca70 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.880+0000 7f601559a640 1 -- 172.21.15.106:0/3036966175 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600802cce0 con 0x7f601815ca70 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.880+0000 7f601559a640 1 -- 172.21.15.106:0/3036966175 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008034660 con 0x7f601815ca70 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 -- 172.21.15.106:0/3036966175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f601815ca70 msgr2=0x7f601806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f601815ca70 0x7f601806fea0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600802bf30 comp rx=0 tx=0).stop 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 -- 172.21.15.106:0/3036966175 shutdown_connections 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 --2- 172.21.15.106:0/3036966175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f601815ca70 0x7f601806fea0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 -- 172.21.15.106:0/3036966175 >> 172.21.15.106:0/3036966175 conn(0x7f60180d2530 msgr2=0x7f60180d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 -- 172.21.15.106:0/3036966175 shutdown_connections 2024-04-13T03:10:41.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.881+0000 7f601cb15640 1 -- 172.21.15.106:0/3036966175 wait complete. 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.882+0000 7f601cb15640 1 Processor -- start 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.882+0000 7f601cb15640 1 -- start start 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f601815ca70 0x7f6018151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60180b8360 con 0x7f60180cd390 2024-04-13T03:10:41.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6018070d20 con 0x7f601815ca70 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601cb15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60180b84e0 con 0x7f60180b7ef0 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56740/0 (socket says 172.21.15.106:56740) 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 -- 172.21.15.106:0/143038545 learned_addr learned my addr 172.21.15.106:0/143038545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f601659c640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6016d9d640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f601815ca70 0x7f6018151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 -- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 msgr2=0x7f60180c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 -- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f601815ca70 msgr2=0x7f6018151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f601815ca70 0x7f6018151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 -- 172.21.15.106:0/143038545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180c6fa0 con 0x7f60180cd390 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.883+0000 7f6015d9b640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d620 tx=0x7f600c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.884+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0125f0 con 0x7f60180cd390 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.884+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c019070 con 0x7f60180cd390 2024-04-13T03:10:41.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.884+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0129d0 con 0x7f60180cd390 2024-04-13T03:10:41.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.884+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60180c7280 con 0x7f60180cd390 2024-04-13T03:10:41.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.884+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60180c7b20 con 0x7f60180cd390 2024-04-13T03:10:41.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.885+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6018068a10 con 0x7f60180cd390 2024-04-13T03:10:41.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.885+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f600c012b70 con 0x7f60180cd390 2024-04-13T03:10:41.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.886+0000 7f6006ffd640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 0x7f5ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:41.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.886+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f600c08c140 con 0x7f60180cd390 2024-04-13T03:10:41.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.886+0000 7f601659c640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 0x7f5ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:41.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.886+0000 7f601659c640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 0x7f5ff806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6008036000 tx=0x7f6008030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:41.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:41.890+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f600c05a360 con 0x7f60180cd390 2024-04-13T03:10:42.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.134+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6018071320 con 0x7f60180cd390 2024-04-13T03:10:42.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.135+0000 7f6006ffd640 1 -- 172.21.15.106:0/143038545 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1195 (secure 0 0 0) 0x7f600c059c60 con 0x7f60180cd390 2024-04-13T03:10:42.136 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:42.136 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"d7ff0368-2781-4238-b477-50485b08a223","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":101,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1712977751,"num_in_osds":8,"osd_in_since":1712977742,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225882112,"bytes_avail":772868231168,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":5059}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-13T03:10:20.374384+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 msgr2=0x7f5ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 0x7f5ff806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6008036000 tx=0x7f6008030000 comp rx=0 tx=0).stop 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 msgr2=0x7f60180b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d620 tx=0x7f600c00dae0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 shutdown_connections 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5ff806c8f0 0x7f5ff806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f601815ca70 0x7f6018151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60180cd390 0x7f60180b36e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 --2- 172.21.15.106:0/143038545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60180b7ef0 0x7f60180c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 >> 172.21.15.106:0/143038545 conn(0x7f60180d2530 msgr2=0x7f601815ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 shutdown_connections 2024-04-13T03:10:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.137+0000 7f601cb15640 1 -- 172.21.15.106:0/143038545 wait complete. 2024-04-13T03:10:42.148 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-13T03:10:42.148 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:10:42.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.248+0000 7f21be1bb640 1 Processor -- start 2024-04-13T03:10:42.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.248+0000 7f21be1bb640 1 -- start start 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21b8070090 0x7f21b8160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 0x7f21b8160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 0x7f21b806ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f21b80ceb50 con 0x7f21b8070090 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f21b80cf110 con 0x7f21b8160ff0 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21be1bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8070470 con 0x7f21b80b0390 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21b8070090 0x7f21b8160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21b8070090 0x7f21b8160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56748/0 (socket says 172.21.15.106:56748) 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 -- 172.21.15.106:0/1255144768 learned_addr learned my addr 172.21.15.106:0/1255144768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 0x7f21b8160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b7fff640 1 --2- 172.21.15.106:0/1255144768 >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 0x7f21b806ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 -- 172.21.15.106:0/1255144768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 msgr2=0x7f21b8160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 0x7f21b8160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 -- 172.21.15.106:0/1255144768 >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 msgr2=0x7f21b806ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1255144768 >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 0x7f21b806ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b77fe640 1 -- 172.21.15.106:0/1255144768 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b815da00 con 0x7f21b8070090 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 0x7f21b8160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.249+0000 7f21b7fff640 1 --2- 172.21.15.106:0/1255144768 >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 0x7f21b806ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:42.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.250+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b8070090 0x7f21b8160570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21b8068050 tx=0x7f21a802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef83a3ed18f37153 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.250+0000 7f21b6ffd640 1 -- 172.21.15.106:0/1255144768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a802e070 con 0x7f21b8070090 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.250+0000 7f21b6ffd640 1 -- 172.21.15.106:0/1255144768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8037070 con 0x7f21b8070090 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.250+0000 7f21b6ffd640 1 -- 172.21.15.106:0/1255144768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8031a00 con 0x7f21b8070090 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 -- 172.21.15.106:0/1255144768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b8070090 msgr2=0x7f21b8160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b8070090 0x7f21b8160570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21b8068050 tx=0x7f21a802b880 comp rx=0 tx=0).stop 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 -- 172.21.15.106:0/1255144768 shutdown_connections 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1255144768 >> v2:172.21.15.114:3300/0 conn(0x7f21b8160ff0 0x7f21b806ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b80b0390 0x7f21b8160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1255144768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b8070090 0x7f21b8160570 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 -- 172.21.15.106:0/1255144768 >> 172.21.15.106:0/1255144768 conn(0x7f21b80cd6d0 msgr2=0x7f21b80cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 -- 172.21.15.106:0/1255144768 shutdown_connections 2024-04-13T03:10:42.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.251+0000 7f21be1bb640 1 -- 172.21.15.106:0/1255144768 wait complete. 2024-04-13T03:10:42.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.252+0000 7f21be1bb640 1 Processor -- start 2024-04-13T03:10:42.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.252+0000 7f21be1bb640 1 -- start start 2024-04-13T03:10:42.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 0x7f21b815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 0x7f21b815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21b806e560 con 0x7f21b80b0390 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f21b80ceda0 con 0x7f21b8070090 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f21b80cd220 con 0x7f21b8160ff0 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56752/0 (socket says 172.21.15.106:56752) 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 -- 172.21.15.106:0/1104647741 learned_addr learned my addr 172.21.15.106:0/1104647741 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b7fff640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 0x7f21b815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 0x7f21b815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 -- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 msgr2=0x7f21b815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 0x7f21b815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 -- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 msgr2=0x7f21b815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 0x7f21b815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 -- 172.21.15.106:0/1104647741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b814c070 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b7fff640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 0x7f21b815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 0x7f21b815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e810 tx=0x7f21a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a000f040 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a0004030 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b81e64c0 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004400 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.253+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b81e6a40 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.254+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b8068a10 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.255+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21a000c6d0 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.255+0000 7f21b4ff9640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 0x7f219c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.255+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f21a008b8f0 con 0x7f21b80b0390 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.255+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 0x7f219c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:42.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.256+0000 7f21b77fe640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 0x7f219c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21a8036000 tx=0x7f21a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:42.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.260+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21a0017020 con 0x7f21b80b0390 2024-04-13T03:10:42.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.433+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21b80d0840 con 0x7f219c06c8f0 2024-04-13T03:10:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.438+0000 7f21b4ff9640 1 -- 172.21.15.106:0/1104647741 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246634 (secure 0 0 0) 0x7f21b80d0840 con 0x7f219c06c8f0 2024-04-13T03:10:42.441 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:42.443 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 msgr2=0x7f219c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 0x7f219c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21a8036000 tx=0x7f21a8008030 comp rx=0 tx=0).stop 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 msgr2=0x7f21b815e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f21a000e810 tx=0x7f21a000ecd0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 shutdown_connections 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f219c06c8f0 0x7f219c06edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f21b8160ff0 0x7f21b815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b80b0390 0x7f21b815e440 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 --2- 172.21.15.106:0/1104647741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21b8070090 0x7f21b815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:42.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 >> 172.21.15.106:0/1104647741 conn(0x7f21b80cd6d0 msgr2=0x7f21b80b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:42.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 shutdown_connections 2024-04-13T03:10:42.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:42.441+0000 7f21be1bb640 1 -- 172.21.15.106:0/1104647741 wait complete. 2024-04-13T03:10:42.453 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-13T03:10:42.377915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4428,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4534272,"data_stored":2613849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000907"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072135+0000","last_change":"2024-04-13T03:10:37.072135+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072135+0000","last_undegraded":"2024-04-13T03:10:37.072135+0000","last_fullsized":"2024-04-13T03:10:37.072135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075618+0000","last_change":"2024-04-13T03:10:37.075618+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075618+0000","last_undegraded":"2024-04-13T03:10:37.075618+0000","last_fullsized":"2024-04-13T03:10:37.075618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073998+0000","last_change":"2024-04-13T03:10:37.073998+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073998+0000","last_undegraded":"2024-04-13T03:10:37.073998+0000","last_fullsized":"2024-04-13T03:10:37.073998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071804+0000","last_change":"2024-04-13T03:10:37.071804+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071804+0000","last_undegraded":"2024-04-13T03:10:37.071804+0000","last_fullsized":"2024-04-13T03:10:37.071804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.070206+0000","last_change":"2024-04-13T03:10:37.070206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.070206+0000","last_undegraded":"2024-04-13T03:10:37.070206+0000","last_fullsized":"2024-04-13T03:10:37.070206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079460+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079460+0000","last_undegraded":"2024-04-13T03:10:37.079460+0000","last_fullsized":"2024-04-13T03:10:37.079460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072953+0000","last_change":"2024-04-13T03:10:37.072953+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072953+0000","last_undegraded":"2024-04-13T03:10:37.072953+0000","last_fullsized":"2024-04-13T03:10:37.072953+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076313+0000","last_change":"2024-04-13T03:10:37.076313+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076313+0000","last_undegraded":"2024-04-13T03:10:37.076313+0000","last_fullsized":"2024-04-13T03:10:37.076313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075430+0000","last_change":"2024-04-13T03:10:37.075430+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075430+0000","last_undegraded":"2024-04-13T03:10:37.075430+0000","last_fullsized":"2024-04-13T03:10:37.075430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074866+0000","last_change":"2024-04-13T03:10:37.074866+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074866+0000","last_undegraded":"2024-04-13T03:10:37.074866+0000","last_fullsized":"2024-04-13T03:10:37.074866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073210+0000","last_change":"2024-04-13T03:10:37.073210+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073210+0000","last_undegraded":"2024-04-13T03:10:37.073210+0000","last_fullsized":"2024-04-13T03:10:37.073210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079554+0000","last_change":"2024-04-13T03:10:37.079554+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079554+0000","last_undegraded":"2024-04-13T03:10:37.079554+0000","last_fullsized":"2024-04-13T03:10:37.079554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077683+0000","last_change":"2024-04-13T03:10:37.077683+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077683+0000","last_undegraded":"2024-04-13T03:10:37.077683+0000","last_fullsized":"2024-04-13T03:10:37.077683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076708+0000","last_change":"2024-04-13T03:10:37.076708+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076708+0000","last_undegraded":"2024-04-13T03:10:37.076708+0000","last_fullsized":"2024-04-13T03:10:37.076708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079088+0000","last_change":"2024-04-13T03:10:37.079088+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079088+0000","last_undegraded":"2024-04-13T03:10:37.079088+0000","last_fullsized":"2024-04-13T03:10:37.079088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.081502+0000","last_change":"2024-04-13T03:10:37.081502+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.081502+0000","last_undegraded":"2024-04-13T03:10:37.081502+0000","last_fullsized":"2024-04-13T03:10:37.081502+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076788+0000","last_change":"2024-04-13T03:10:37.076788+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076788+0000","last_undegraded":"2024-04-13T03:10:37.076788+0000","last_fullsized":"2024-04-13T03:10:37.076788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072395+0000","last_change":"2024-04-13T03:10:37.072395+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072395+0000","last_undegraded":"2024-04-13T03:10:37.072395+0000","last_fullsized":"2024-04-13T03:10:37.072395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076537+0000","last_change":"2024-04-13T03:10:37.076537+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076537+0000","last_undegraded":"2024-04-13T03:10:37.076537+0000","last_fullsized":"2024-04-13T03:10:37.076537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075931+0000","last_change":"2024-04-13T03:10:37.075931+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075931+0000","last_undegraded":"2024-04-13T03:10:37.075931+0000","last_fullsized":"2024-04-13T03:10:37.075931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073585+0000","last_change":"2024-04-13T03:10:37.073585+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073585+0000","last_undegraded":"2024-04-13T03:10:37.073585+0000","last_fullsized":"2024-04-13T03:10:37.073585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078497+0000","last_change":"2024-04-13T03:10:37.078497+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078497+0000","last_undegraded":"2024-04-13T03:10:37.078497+0000","last_fullsized":"2024-04-13T03:10:37.078497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074338+0000","last_change":"2024-04-13T03:10:37.074338+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074338+0000","last_undegraded":"2024-04-13T03:10:37.074338+0000","last_fullsized":"2024-04-13T03:10:37.074338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077015+0000","last_change":"2024-04-13T03:10:37.077015+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077015+0000","last_undegraded":"2024-04-13T03:10:37.077015+0000","last_fullsized":"2024-04-13T03:10:37.077015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069793+0000","last_change":"2024-04-13T03:10:37.069793+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069793+0000","last_undegraded":"2024-04-13T03:10:37.069793+0000","last_fullsized":"2024-04-13T03:10:37.069793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076536+0000","last_change":"2024-04-13T03:10:37.076536+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076536+0000","last_undegraded":"2024-04-13T03:10:37.076536+0000","last_fullsized":"2024-04-13T03:10:37.076536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074266+0000","last_change":"2024-04-13T03:10:37.074266+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074266+0000","last_undegraded":"2024-04-13T03:10:37.074266+0000","last_fullsized":"2024-04-13T03:10:37.074266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075155+0000","last_change":"2024-04-13T03:10:37.075155+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075155+0000","last_undegraded":"2024-04-13T03:10:37.075155+0000","last_fullsized":"2024-04-13T03:10:37.075155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076048+0000","last_change":"2024-04-13T03:10:37.076048+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076048+0000","last_undegraded":"2024-04-13T03:10:37.076048+0000","last_fullsized":"2024-04-13T03:10:37.076048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078031+0000","last_change":"2024-04-13T03:10:37.078031+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078031+0000","last_undegraded":"2024-04-13T03:10:37.078031+0000","last_fullsized":"2024-04-13T03:10:37.078031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069807+0000","last_change":"2024-04-13T03:10:37.069807+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069807+0000","last_undegraded":"2024-04-13T03:10:37.069807+0000","last_fullsized":"2024-04-13T03:10:37.069807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079002+0000","last_change":"2024-04-13T03:10:37.079002+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079002+0000","last_undegraded":"2024-04-13T03:10:37.079002+0000","last_fullsized":"2024-04-13T03:10:37.079002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092039+0000","last_change":"2024-04-13T03:10:37.085786+0000","last_active":"2024-04-13T03:10:37.092039+0000","last_peered":"2024-04-13T03:10:37.092039+0000","last_clean":"2024-04-13T03:10:37.092039+0000","last_became_active":"2024-04-13T03:10:37.085500+0000","last_became_peered":"2024-04-13T03:10:37.085500+0000","last_unstale":"2024-04-13T03:10:37.092039+0000","last_undegraded":"2024-04-13T03:10:37.092039+0000","last_fullsized":"2024-04-13T03:10:37.092039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:13.188140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069756+0000","last_change":"2024-04-13T03:10:37.069756+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069756+0000","last_undegraded":"2024-04-13T03:10:37.069756+0000","last_fullsized":"2024-04-13T03:10:37.069756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086872+0000","last_change":"2024-04-13T03:10:37.083216+0000","last_active":"2024-04-13T03:10:37.086872+0000","last_peered":"2024-04-13T03:10:37.086872+0000","last_clean":"2024-04-13T03:10:37.086872+0000","last_became_active":"2024-04-13T03:10:37.082975+0000","last_became_peered":"2024-04-13T03:10:37.082975+0000","last_unstale":"2024-04-13T03:10:37.086872+0000","last_undegraded":"2024-04-13T03:10:37.086872+0000","last_fullsized":"2024-04-13T03:10:37.086872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072632+0000","last_change":"2024-04-13T03:10:37.072632+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072632+0000","last_undegraded":"2024-04-13T03:10:37.072632+0000","last_fullsized":"2024-04-13T03:10:37.072632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089581+0000","last_change":"2024-04-13T03:10:37.084531+0000","last_active":"2024-04-13T03:10:37.089581+0000","last_peered":"2024-04-13T03:10:37.089581+0000","last_clean":"2024-04-13T03:10:37.089581+0000","last_became_active":"2024-04-13T03:10:37.084244+0000","last_became_peered":"2024-04-13T03:10:37.084244+0000","last_unstale":"2024-04-13T03:10:37.089581+0000","last_undegraded":"2024-04-13T03:10:37.089581+0000","last_fullsized":"2024-04-13T03:10:37.089581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072539+0000","last_change":"2024-04-13T03:10:37.072539+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072539+0000","last_undegraded":"2024-04-13T03:10:37.072539+0000","last_fullsized":"2024-04-13T03:10:37.072539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086974+0000","last_change":"2024-04-13T03:10:37.083577+0000","last_active":"2024-04-13T03:10:37.086974+0000","last_peered":"2024-04-13T03:10:37.086974+0000","last_clean":"2024-04-13T03:10:37.086974+0000","last_became_active":"2024-04-13T03:10:37.083299+0000","last_became_peered":"2024-04-13T03:10:37.083299+0000","last_unstale":"2024-04-13T03:10:37.086974+0000","last_undegraded":"2024-04-13T03:10:37.086974+0000","last_fullsized":"2024-04-13T03:10:37.086974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072796+0000","last_change":"2024-04-13T03:10:37.072796+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072796+0000","last_undegraded":"2024-04-13T03:10:37.072796+0000","last_fullsized":"2024-04-13T03:10:37.072796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089546+0000","last_change":"2024-04-13T03:10:37.086817+0000","last_active":"2024-04-13T03:10:37.089546+0000","last_peered":"2024-04-13T03:10:37.089546+0000","last_clean":"2024-04-13T03:10:37.089546+0000","last_became_active":"2024-04-13T03:10:37.086544+0000","last_became_peered":"2024-04-13T03:10:37.086544+0000","last_unstale":"2024-04-13T03:10:37.089546+0000","last_undegraded":"2024-04-13T03:10:37.089546+0000","last_fullsized":"2024-04-13T03:10:37.089546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074859+0000","last_change":"2024-04-13T03:10:37.074859+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074859+0000","last_undegraded":"2024-04-13T03:10:37.074859+0000","last_fullsized":"2024-04-13T03:10:37.074859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094811+0000","last_change":"2024-04-13T03:10:37.089557+0000","last_active":"2024-04-13T03:10:37.094811+0000","last_peered":"2024-04-13T03:10:37.094811+0000","last_clean":"2024-04-13T03:10:37.094811+0000","last_became_active":"2024-04-13T03:10:37.089249+0000","last_became_peered":"2024-04-13T03:10:37.089249+0000","last_unstale":"2024-04-13T03:10:37.094811+0000","last_undegraded":"2024-04-13T03:10:37.094811+0000","last_fullsized":"2024-04-13T03:10:37.094811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:19.190392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073086+0000","last_change":"2024-04-13T03:10:37.073086+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073086+0000","last_undegraded":"2024-04-13T03:10:37.073086+0000","last_fullsized":"2024-04-13T03:10:37.073086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092667+0000","last_change":"2024-04-13T03:10:37.088764+0000","last_active":"2024-04-13T03:10:37.092667+0000","last_peered":"2024-04-13T03:10:37.092667+0000","last_clean":"2024-04-13T03:10:37.092667+0000","last_became_active":"2024-04-13T03:10:37.088507+0000","last_became_peered":"2024-04-13T03:10:37.088507+0000","last_unstale":"2024-04-13T03:10:37.092667+0000","last_undegraded":"2024-04-13T03:10:37.092667+0000","last_fullsized":"2024-04-13T03:10:37.092667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074137+0000","last_change":"2024-04-13T03:10:37.074137+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074137+0000","last_undegraded":"2024-04-13T03:10:37.074137+0000","last_fullsized":"2024-04-13T03:10:37.074137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087592+0000","last_change":"2024-04-13T03:10:37.083447+0000","last_active":"2024-04-13T03:10:37.087592+0000","last_peered":"2024-04-13T03:10:37.087592+0000","last_clean":"2024-04-13T03:10:37.087592+0000","last_became_active":"2024-04-13T03:10:37.083159+0000","last_became_peered":"2024-04-13T03:10:37.083159+0000","last_unstale":"2024-04-13T03:10:37.087592+0000","last_undegraded":"2024-04-13T03:10:37.087592+0000","last_fullsized":"2024-04-13T03:10:37.087592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071855+0000","last_change":"2024-04-13T03:10:37.071855+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071855+0000","last_undegraded":"2024-04-13T03:10:37.071855+0000","last_fullsized":"2024-04-13T03:10:37.071855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083991+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083991+0000","last_peered":"2024-04-13T03:10:37.083991+0000","last_clean":"2024-04-13T03:10:37.083991+0000","last_became_active":"2024-04-13T03:10:37.081899+0000","last_became_peered":"2024-04-13T03:10:37.081899+0000","last_unstale":"2024-04-13T03:10:37.083991+0000","last_undegraded":"2024-04-13T03:10:37.083991+0000","last_fullsized":"2024-04-13T03:10:37.083991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:04:52.620360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.084130+0000","last_change":"2024-04-13T03:10:37.084130+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.084130+0000","last_undegraded":"2024-04-13T03:10:37.084130+0000","last_fullsized":"2024-04-13T03:10:37.084130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087058+0000","last_change":"2024-04-13T03:10:37.085297+0000","last_active":"2024-04-13T03:10:37.087058+0000","last_peered":"2024-04-13T03:10:37.087058+0000","last_clean":"2024-04-13T03:10:37.087058+0000","last_became_active":"2024-04-13T03:10:37.085009+0000","last_became_peered":"2024-04-13T03:10:37.085009+0000","last_unstale":"2024-04-13T03:10:37.087058+0000","last_undegraded":"2024-04-13T03:10:37.087058+0000","last_fullsized":"2024-04-13T03:10:37.087058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072457+0000","last_change":"2024-04-13T03:10:37.072457+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072457+0000","last_undegraded":"2024-04-13T03:10:37.072457+0000","last_fullsized":"2024-04-13T03:10:37.072457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092276+0000","last_change":"2024-04-13T03:10:37.088374+0000","last_active":"2024-04-13T03:10:37.092276+0000","last_peered":"2024-04-13T03:10:37.092276+0000","last_clean":"2024-04-13T03:10:37.092276+0000","last_became_active":"2024-04-13T03:10:37.088068+0000","last_became_peered":"2024-04-13T03:10:37.088068+0000","last_unstale":"2024-04-13T03:10:37.092276+0000","last_undegraded":"2024-04-13T03:10:37.092276+0000","last_fullsized":"2024-04-13T03:10:37.092276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073472+0000","last_change":"2024-04-13T03:10:37.073472+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073472+0000","last_undegraded":"2024-04-13T03:10:37.073472+0000","last_fullsized":"2024-04-13T03:10:37.073472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083655+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:37.083655+0000","last_peered":"2024-04-13T03:10:37.083655+0000","last_clean":"2024-04-13T03:10:37.083655+0000","last_became_active":"2024-04-13T03:10:37.081896+0000","last_became_peered":"2024-04-13T03:10:37.081896+0000","last_unstale":"2024-04-13T03:10:37.083655+0000","last_undegraded":"2024-04-13T03:10:37.083655+0000","last_fullsized":"2024-04-13T03:10:37.083655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:38.859158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.072382+0000","last_change":"2024-04-13T03:10:37.072382+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.072382+0000","last_undegraded":"2024-04-13T03:10:37.072382+0000","last_fullsized":"2024-04-13T03:10:37.072382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.082603+0000","last_change":"2024-04-13T03:10:37.081457+0000","last_active":"2024-04-13T03:10:37.082603+0000","last_peered":"2024-04-13T03:10:37.082603+0000","last_clean":"2024-04-13T03:10:37.082603+0000","last_became_active":"2024-04-13T03:10:37.081232+0000","last_became_peered":"2024-04-13T03:10:37.081232+0000","last_unstale":"2024-04-13T03:10:37.082603+0000","last_undegraded":"2024-04-13T03:10:37.082603+0000","last_fullsized":"2024-04-13T03:10:37.082603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:37:13.251395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074188+0000","last_change":"2024-04-13T03:10:37.074188+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074188+0000","last_undegraded":"2024-04-13T03:10:37.074188+0000","last_fullsized":"2024-04-13T03:10:37.074188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.083620+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.083346+0000","last_became_peered":"2024-04-13T03:10:37.083346+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079369+0000","last_change":"2024-04-13T03:10:37.079369+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079369+0000","last_undegraded":"2024-04-13T03:10:37.079369+0000","last_fullsized":"2024-04-13T03:10:37.079369+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092066+0000","last_change":"2024-04-13T03:10:37.086440+0000","last_active":"2024-04-13T03:10:37.092066+0000","last_peered":"2024-04-13T03:10:37.092066+0000","last_clean":"2024-04-13T03:10:37.092066+0000","last_became_active":"2024-04-13T03:10:37.086158+0000","last_became_peered":"2024-04-13T03:10:37.086158+0000","last_unstale":"2024-04-13T03:10:37.092066+0000","last_undegraded":"2024-04-13T03:10:37.092066+0000","last_fullsized":"2024-04-13T03:10:37.092066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:07.798696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073362+0000","last_change":"2024-04-13T03:10:37.073362+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073362+0000","last_undegraded":"2024-04-13T03:10:37.073362+0000","last_fullsized":"2024-04-13T03:10:37.073362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094829+0000","last_change":"2024-04-13T03:10:37.089769+0000","last_active":"2024-04-13T03:10:37.094829+0000","last_peered":"2024-04-13T03:10:37.094829+0000","last_clean":"2024-04-13T03:10:37.094829+0000","last_became_active":"2024-04-13T03:10:37.089284+0000","last_became_peered":"2024-04-13T03:10:37.089284+0000","last_unstale":"2024-04-13T03:10:37.094829+0000","last_undegraded":"2024-04-13T03:10:37.094829+0000","last_fullsized":"2024-04-13T03:10:37.094829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:21.425166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075351+0000","last_change":"2024-04-13T03:10:37.075351+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075351+0000","last_undegraded":"2024-04-13T03:10:37.075351+0000","last_fullsized":"2024-04-13T03:10:37.075351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092234+0000","last_change":"2024-04-13T03:10:37.087631+0000","last_active":"2024-04-13T03:10:37.092234+0000","last_peered":"2024-04-13T03:10:37.092234+0000","last_clean":"2024-04-13T03:10:37.092234+0000","last_became_active":"2024-04-13T03:10:37.087385+0000","last_became_peered":"2024-04-13T03:10:37.087385+0000","last_unstale":"2024-04-13T03:10:37.092234+0000","last_undegraded":"2024-04-13T03:10:37.092234+0000","last_fullsized":"2024-04-13T03:10:37.092234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074341+0000","last_change":"2024-04-13T03:10:37.074341+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074341+0000","last_undegraded":"2024-04-13T03:10:37.074341+0000","last_fullsized":"2024-04-13T03:10:37.074341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087689+0000","last_change":"2024-04-13T03:10:37.085447+0000","last_active":"2024-04-13T03:10:37.087689+0000","last_peered":"2024-04-13T03:10:37.087689+0000","last_clean":"2024-04-13T03:10:37.087689+0000","last_became_active":"2024-04-13T03:10:37.085157+0000","last_became_peered":"2024-04-13T03:10:37.085157+0000","last_unstale":"2024-04-13T03:10:37.087689+0000","last_undegraded":"2024-04-13T03:10:37.087689+0000","last_fullsized":"2024-04-13T03:10:37.087689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073200+0000","last_change":"2024-04-13T03:10:37.073200+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073200+0000","last_undegraded":"2024-04-13T03:10:37.073200+0000","last_fullsized":"2024-04-13T03:10:37.073200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092500+0000","last_change":"2024-04-13T03:10:37.087391+0000","last_active":"2024-04-13T03:10:37.092500+0000","last_peered":"2024-04-13T03:10:37.092500+0000","last_clean":"2024-04-13T03:10:37.092500+0000","last_became_active":"2024-04-13T03:10:37.087131+0000","last_became_peered":"2024-04-13T03:10:37.087131+0000","last_unstale":"2024-04-13T03:10:37.092500+0000","last_undegraded":"2024-04-13T03:10:37.092500+0000","last_fullsized":"2024-04-13T03:10:37.092500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:41.946253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071737+0000","last_change":"2024-04-13T03:10:37.071737+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071737+0000","last_undegraded":"2024-04-13T03:10:37.071737+0000","last_fullsized":"2024-04-13T03:10:37.071737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089556+0000","last_change":"2024-04-13T03:10:37.084719+0000","last_active":"2024-04-13T03:10:37.089556+0000","last_peered":"2024-04-13T03:10:37.089556+0000","last_clean":"2024-04-13T03:10:37.089556+0000","last_became_active":"2024-04-13T03:10:37.084472+0000","last_became_peered":"2024-04-13T03:10:37.084472+0000","last_unstale":"2024-04-13T03:10:37.089556+0000","last_undegraded":"2024-04-13T03:10:37.089556+0000","last_fullsized":"2024-04-13T03:10:37.089556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.078007+0000","last_change":"2024-04-13T03:10:37.078007+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.078007+0000","last_undegraded":"2024-04-13T03:10:37.078007+0000","last_fullsized":"2024-04-13T03:10:37.078007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080873+0000","last_change":"2024-04-13T03:10:37.078763+0000","last_active":"2024-04-13T03:10:37.080873+0000","last_peered":"2024-04-13T03:10:37.080873+0000","last_clean":"2024-04-13T03:10:37.080873+0000","last_became_active":"2024-04-13T03:10:37.078484+0000","last_became_peered":"2024-04-13T03:10:37.078484+0000","last_unstale":"2024-04-13T03:10:37.080873+0000","last_undegraded":"2024-04-13T03:10:37.080873+0000","last_fullsized":"2024-04-13T03:10:37.080873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:54:33.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.074170+0000","last_change":"2024-04-13T03:10:37.074170+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.074170+0000","last_undegraded":"2024-04-13T03:10:37.074170+0000","last_fullsized":"2024-04-13T03:10:37.074170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089604+0000","last_change":"2024-04-13T03:10:37.085242+0000","last_active":"2024-04-13T03:10:37.089604+0000","last_peered":"2024-04-13T03:10:37.089604+0000","last_clean":"2024-04-13T03:10:37.089604+0000","last_became_active":"2024-04-13T03:10:37.084999+0000","last_became_peered":"2024-04-13T03:10:37.084999+0000","last_unstale":"2024-04-13T03:10:37.089604+0000","last_undegraded":"2024-04-13T03:10:37.089604+0000","last_fullsized":"2024-04-13T03:10:37.089604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:31:56.747006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077206+0000","last_change":"2024-04-13T03:10:37.077206+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077206+0000","last_undegraded":"2024-04-13T03:10:37.077206+0000","last_fullsized":"2024-04-13T03:10:37.077206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092322+0000","last_change":"2024-04-13T03:10:37.088188+0000","last_active":"2024-04-13T03:10:37.092322+0000","last_peered":"2024-04-13T03:10:37.092322+0000","last_clean":"2024-04-13T03:10:37.092322+0000","last_became_active":"2024-04-13T03:10:37.087921+0000","last_became_peered":"2024-04-13T03:10:37.087921+0000","last_unstale":"2024-04-13T03:10:37.092322+0000","last_undegraded":"2024-04-13T03:10:37.092322+0000","last_fullsized":"2024-04-13T03:10:37.092322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:01.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.073727+0000","last_change":"2024-04-13T03:10:37.073727+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.073727+0000","last_undegraded":"2024-04-13T03:10:37.073727+0000","last_fullsized":"2024-04-13T03:10:37.073727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086411+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:37.086411+0000","last_peered":"2024-04-13T03:10:37.086411+0000","last_clean":"2024-04-13T03:10:37.086411+0000","last_became_active":"2024-04-13T03:10:37.079188+0000","last_became_peered":"2024-04-13T03:10:37.079188+0000","last_unstale":"2024-04-13T03:10:37.086411+0000","last_undegraded":"2024-04-13T03:10:37.086411+0000","last_fullsized":"2024-04-13T03:10:37.086411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:41.463691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069800+0000","last_change":"2024-04-13T03:10:37.069800+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069800+0000","last_undegraded":"2024-04-13T03:10:37.069800+0000","last_fullsized":"2024-04-13T03:10:37.069800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.092256+0000","last_change":"2024-04-13T03:10:37.088272+0000","last_active":"2024-04-13T03:10:37.092256+0000","last_peered":"2024-04-13T03:10:37.092256+0000","last_clean":"2024-04-13T03:10:37.092256+0000","last_became_active":"2024-04-13T03:10:37.088023+0000","last_became_peered":"2024-04-13T03:10:37.088023+0000","last_unstale":"2024-04-13T03:10:37.092256+0000","last_undegraded":"2024-04-13T03:10:37.092256+0000","last_fullsized":"2024-04-13T03:10:37.092256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.083615+0000","last_change":"2024-04-13T03:10:37.083615+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.083615+0000","last_undegraded":"2024-04-13T03:10:37.083615+0000","last_fullsized":"2024-04-13T03:10:37.083615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.089636+0000","last_change":"2024-04-13T03:10:37.080915+0000","last_active":"2024-04-13T03:10:37.089636+0000","last_peered":"2024-04-13T03:10:37.089636+0000","last_clean":"2024-04-13T03:10:37.089636+0000","last_became_active":"2024-04-13T03:10:37.080658+0000","last_became_peered":"2024-04-13T03:10:37.080658+0000","last_unstale":"2024-04-13T03:10:37.089636+0000","last_undegraded":"2024-04-13T03:10:37.089636+0000","last_fullsized":"2024-04-13T03:10:37.089636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:02.927009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.075238+0000","last_change":"2024-04-13T03:10:37.075238+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.075238+0000","last_undegraded":"2024-04-13T03:10:37.075238+0000","last_fullsized":"2024-04-13T03:10:37.075238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086721+0000","last_change":"2024-04-13T03:10:37.081490+0000","last_active":"2024-04-13T03:10:37.086721+0000","last_peered":"2024-04-13T03:10:37.086721+0000","last_clean":"2024-04-13T03:10:37.086721+0000","last_became_active":"2024-04-13T03:10:37.081226+0000","last_became_peered":"2024-04-13T03:10:37.081226+0000","last_unstale":"2024-04-13T03:10:37.086721+0000","last_undegraded":"2024-04-13T03:10:37.086721+0000","last_fullsized":"2024-04-13T03:10:37.086721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.071935+0000","last_change":"2024-04-13T03:10:37.071935+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.071935+0000","last_undegraded":"2024-04-13T03:10:37.071935+0000","last_fullsized":"2024-04-13T03:10:37.071935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.086903+0000","last_change":"2024-04-13T03:10:37.083750+0000","last_active":"2024-04-13T03:10:37.086903+0000","last_peered":"2024-04-13T03:10:37.086903+0000","last_clean":"2024-04-13T03:10:37.086903+0000","last_became_active":"2024-04-13T03:10:37.083512+0000","last_became_peered":"2024-04-13T03:10:37.083512+0000","last_unstale":"2024-04-13T03:10:37.086903+0000","last_undegraded":"2024-04-13T03:10:37.086903+0000","last_fullsized":"2024-04-13T03:10:37.086903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.069680+0000","last_change":"2024-04-13T03:10:37.069680+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.069680+0000","last_undegraded":"2024-04-13T03:10:37.069680+0000","last_fullsized":"2024-04-13T03:10:37.069680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087143+0000","last_change":"2024-04-13T03:10:37.082886+0000","last_active":"2024-04-13T03:10:37.087143+0000","last_peered":"2024-04-13T03:10:37.087143+0000","last_clean":"2024-04-13T03:10:37.087143+0000","last_became_active":"2024-04-13T03:10:37.082622+0000","last_became_peered":"2024-04-13T03:10:37.082622+0000","last_unstale":"2024-04-13T03:10:37.087143+0000","last_undegraded":"2024-04-13T03:10:37.087143+0000","last_fullsized":"2024-04-13T03:10:37.087143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:41.556737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.077247+0000","last_change":"2024-04-13T03:10:37.077247+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.077247+0000","last_undegraded":"2024-04-13T03:10:37.077247+0000","last_fullsized":"2024-04-13T03:10:37.077247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.087667+0000","last_change":"2024-04-13T03:10:37.084769+0000","last_active":"2024-04-13T03:10:37.087667+0000","last_peered":"2024-04-13T03:10:37.087667+0000","last_clean":"2024-04-13T03:10:37.087667+0000","last_became_active":"2024-04-13T03:10:37.084496+0000","last_became_peered":"2024-04-13T03:10:37.084496+0000","last_unstale":"2024-04-13T03:10:37.087667+0000","last_undegraded":"2024-04-13T03:10:37.087667+0000","last_fullsized":"2024-04-13T03:10:37.087667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.079692+0000","last_change":"2024-04-13T03:10:37.079692+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.079692+0000","last_undegraded":"2024-04-13T03:10:37.079692+0000","last_fullsized":"2024-04-13T03:10:37.079692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.083507+0000","last_change":"2024-04-13T03:10:37.082340+0000","last_active":"2024-04-13T03:10:37.083507+0000","last_peered":"2024-04-13T03:10:37.083507+0000","last_clean":"2024-04-13T03:10:37.083507+0000","last_became_active":"2024-04-13T03:10:37.082069+0000","last_became_peered":"2024-04-13T03:10:37.082069+0000","last_unstale":"2024-04-13T03:10:37.083507+0000","last_undegraded":"2024-04-13T03:10:37.083507+0000","last_fullsized":"2024-04-13T03:10:37.083507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:42.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-13T03:10:37.076397+0000","last_change":"2024-04-13T03:10:37.076397+0000","last_active":"2024-04-13T03:10:37.057207+0000","last_peered":"2024-04-13T03:10:37.057207+0000","last_clean":"2024-04-13T03:10:37.057207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:10:37.076397+0000","last_undegraded":"2024-04-13T03:10:37.076397+0000","last_fullsized":"2024-04-13T03:10:37.076397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.094834+0000","last_change":"2024-04-13T03:10:37.090291+0000","last_active":"2024-04-13T03:10:37.094834+0000","last_peered":"2024-04-13T03:10:37.094834+0000","last_clean":"2024-04-13T03:10:37.094834+0000","last_became_active":"2024-04-13T03:10:37.090004+0000","last_became_peered":"2024-04-13T03:10:37.090004+0000","last_unstale":"2024-04-13T03:10:37.094834+0000","last_undegraded":"2024-04-13T03:10:37.094834+0000","last_fullsized":"2024-04-13T03:10:37.094834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:07:36.490163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-13T03:10:37.080293+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:10:37.080293+0000","last_peered":"2024-04-13T03:10:37.080293+0000","last_clean":"2024-04-13T03:10:37.080293+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:10:37.080293+0000","last_undegraded":"2024-04-13T03:10:37.080293+0000","last_fullsized":"2024-04-13T03:10:37.080293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:10:42.770 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:10:42.769+0000 7f7d439f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:10:45.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:10:45.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.567+0000 7f7af1c60640 1 Processor -- start 2024-04-13T03:10:45.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.567+0000 7f7af1c60640 1 -- start start 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7aec15e9c0 0x7f7aec0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 0x7f7aec0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7aec0b6700 0x7f7aec06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7aec0d27f0 con 0x7f7aec0b6700 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7aec15eda0 con 0x7f7aec15e9c0 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7af1c60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec162c20 con 0x7f7aec1624b0 2024-04-13T03:10:45.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7aec0b6700 0x7f7aec06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7aec0b6700 0x7f7aec06b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43960/0 (socket says 172.21.15.106:43960) 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 -- 172.21.15.106:0/3903104371 learned_addr learned my addr 172.21.15.106:0/3903104371 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aeaffd640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 0x7f7aec0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aeb7fe640 1 --2- 172.21.15.106:0/3903104371 >> v2:172.21.15.114:3300/0 conn(0x7f7aec15e9c0 0x7f7aec0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 -- 172.21.15.106:0/3903104371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 msgr2=0x7f7aec0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 0x7f7aec0b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 -- 172.21.15.106:0/3903104371 >> v2:172.21.15.114:3300/0 conn(0x7f7aec15e9c0 msgr2=0x7f7aec0ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 --2- 172.21.15.106:0/3903104371 >> v2:172.21.15.114:3300/0 conn(0x7f7aec15e9c0 0x7f7aec0ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aebfff640 1 -- 172.21.15.106:0/3903104371 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec162f00 con 0x7f7aec0b6700 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.568+0000 7f7aeaffd640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 0x7f7aec0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.569+0000 7f7aebfff640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec0b6700 0x7f7aec06b8e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7aec06cda0 tx=0x7f7ae402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb389f1417e8d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.569+0000 7f7aea7fc640 1 -- 172.21.15.106:0/3903104371 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae402ca70 con 0x7f7aec0b6700 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.569+0000 7f7aea7fc640 1 -- 172.21.15.106:0/3903104371 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae402cc10 con 0x7f7aec0b6700 2024-04-13T03:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.569+0000 7f7aea7fc640 1 -- 172.21.15.106:0/3903104371 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40343f0 con 0x7f7aec0b6700 2024-04-13T03:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 -- 172.21.15.106:0/3903104371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec0b6700 msgr2=0x7f7aec06b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec0b6700 0x7f7aec06b8e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7aec06cda0 tx=0x7f7ae402bed0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 -- 172.21.15.106:0/3903104371 shutdown_connections 2024-04-13T03:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec0b6700 0x7f7aec06b8e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 --2- 172.21.15.106:0/3903104371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec1624b0 0x7f7aec0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 --2- 172.21.15.106:0/3903104371 >> v2:172.21.15.114:3300/0 conn(0x7f7aec15e9c0 0x7f7aec0ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 -- 172.21.15.106:0/3903104371 >> 172.21.15.106:0/3903104371 conn(0x7f7aec0d11c0 msgr2=0x7f7aec0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 -- 172.21.15.106:0/3903104371 shutdown_connections 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.570+0000 7f7af1c60640 1 -- 172.21.15.106:0/3903104371 wait complete. 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.571+0000 7f7af1c60640 1 Processor -- start 2024-04-13T03:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.571+0000 7f7af1c60640 1 -- start start 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec0b6700 0x7f7aec152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec1624b0 0x7f7aec161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec162910 con 0x7f7aec15e9c0 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec0d2a40 con 0x7f7aec1624b0 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7af1c60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec06be20 con 0x7f7aec0b6700 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43964/0 (socket says 172.21.15.106:43964) 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7aeaffd640 1 -- 172.21.15.106:0/1547503248 learned_addr learned my addr 172.21.15.106:0/1547503248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7aeb7fe640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec0b6700 0x7f7aec152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.572+0000 7f7aebfff640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec1624b0 0x7f7aec161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 -- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec0b6700 msgr2=0x7f7aec152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec0b6700 0x7f7aec152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 -- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec1624b0 msgr2=0x7f7aec161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec1624b0 0x7f7aec161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 -- 172.21.15.106:0/1547503248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec0c8650 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7aeaffd640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000d800 tx=0x7f7ae000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0004030 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae00041d0 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec0c8930 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000b6c0 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.573+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec0c8ef0 con 0x7f7aec15e9c0 2024-04-13T03:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.574+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec068a10 con 0x7f7aec15e9c0 2024-04-13T03:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.574+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae000b860 con 0x7f7aec15e9c0 2024-04-13T03:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.575+0000 7f7ad3fff640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.575+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7ae008bd80 con 0x7f7aec15e9c0 2024-04-13T03:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.575+0000 7f7aeb7fe640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.575+0000 7f7aeb7fe640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 0x7f7ad806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.579+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ae00599d0 con 0x7f7aec15e9c0 2024-04-13T03:10:45.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.751+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7aec0d3c40 con 0x7f7ad806c8f0 2024-04-13T03:10:45.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.757+0000 7f7ad3fff640 1 -- 172.21.15.106:0/1547503248 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f7aec0d3c40 con 0x7f7ad806c8f0 2024-04-13T03:10:45.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:45.763 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:10:45.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 msgr2=0x7f7ad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 0x7f7ad806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc008040 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 msgr2=0x7f7aec160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000d800 tx=0x7f7ae000dcc0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 shutdown_connections 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec1624b0 0x7f7aec161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aec15e9c0 0x7f7aec160a00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 --2- 172.21.15.106:0/1547503248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7aec0b6700 0x7f7aec152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.759+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 >> 172.21.15.106:0/1547503248 conn(0x7f7aec0d11c0 msgr2=0x7f7aec0d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.760+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 shutdown_connections 2024-04-13T03:10:45.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.760+0000 7f7af1c60640 1 -- 172.21.15.106:0/1547503248 wait complete. 2024-04-13T03:10:45.772 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-13T03:10:44.378224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220920,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753800,"statfs":{"total":773094113280,"available":772867891200,"internally_reserved":0,"allocated":4857856,"data_stored":2762876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.312252"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088608+0000","last_change":"2024-04-13T03:10:38.089724+0000","last_active":"2024-04-13T03:10:39.088608+0000","last_peered":"2024-04-13T03:10:39.088608+0000","last_clean":"2024-04-13T03:10:39.088608+0000","last_became_active":"2024-04-13T03:10:38.089448+0000","last_became_peered":"2024-04-13T03:10:38.089448+0000","last_unstale":"2024-04-13T03:10:39.088608+0000","last_undegraded":"2024-04-13T03:10:39.088608+0000","last_fullsized":"2024-04-13T03:10:39.088608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:19:31.109811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087997+0000","last_change":"2024-04-13T03:10:38.081921+0000","last_active":"2024-04-13T03:10:39.087997+0000","last_peered":"2024-04-13T03:10:39.087997+0000","last_clean":"2024-04-13T03:10:39.087997+0000","last_became_active":"2024-04-13T03:10:38.081646+0000","last_became_peered":"2024-04-13T03:10:38.081646+0000","last_unstale":"2024-04-13T03:10:39.087997+0000","last_undegraded":"2024-04-13T03:10:39.087997+0000","last_fullsized":"2024-04-13T03:10:39.087997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:53:46.681800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087640+0000","last_change":"2024-04-13T03:10:38.102839+0000","last_active":"2024-04-13T03:10:39.087640+0000","last_peered":"2024-04-13T03:10:39.087640+0000","last_clean":"2024-04-13T03:10:39.087640+0000","last_became_active":"2024-04-13T03:10:38.102555+0000","last_became_peered":"2024-04-13T03:10:38.102555+0000","last_unstale":"2024-04-13T03:10:39.087640+0000","last_undegraded":"2024-04-13T03:10:39.087640+0000","last_fullsized":"2024-04-13T03:10:39.087640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:05.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923684+0000","last_change":"2024-04-13T03:10:38.089564+0000","last_active":"2024-04-13T03:10:39.923684+0000","last_peered":"2024-04-13T03:10:39.923684+0000","last_clean":"2024-04-13T03:10:39.923684+0000","last_became_active":"2024-04-13T03:10:38.089331+0000","last_became_peered":"2024-04-13T03:10:38.089331+0000","last_unstale":"2024-04-13T03:10:39.923684+0000","last_undegraded":"2024-04-13T03:10:39.923684+0000","last_fullsized":"2024-04-13T03:10:39.923684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:02.266679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923432+0000","last_change":"2024-04-13T03:10:38.085779+0000","last_active":"2024-04-13T03:10:39.923432+0000","last_peered":"2024-04-13T03:10:39.923432+0000","last_clean":"2024-04-13T03:10:39.923432+0000","last_became_active":"2024-04-13T03:10:38.085512+0000","last_became_peered":"2024-04-13T03:10:38.085512+0000","last_unstale":"2024-04-13T03:10:39.923432+0000","last_undegraded":"2024-04-13T03:10:39.923432+0000","last_fullsized":"2024-04-13T03:10:39.923432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:08.800677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923526+0000","last_change":"2024-04-13T03:10:38.091757+0000","last_active":"2024-04-13T03:10:39.923526+0000","last_peered":"2024-04-13T03:10:39.923526+0000","last_clean":"2024-04-13T03:10:39.923526+0000","last_became_active":"2024-04-13T03:10:38.091486+0000","last_became_peered":"2024-04-13T03:10:38.091486+0000","last_unstale":"2024-04-13T03:10:39.923526+0000","last_undegraded":"2024-04-13T03:10:39.923526+0000","last_fullsized":"2024-04-13T03:10:39.923526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:41.891801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088589+0000","last_change":"2024-04-13T03:10:38.087315+0000","last_active":"2024-04-13T03:10:39.088589+0000","last_peered":"2024-04-13T03:10:39.088589+0000","last_clean":"2024-04-13T03:10:39.088589+0000","last_became_active":"2024-04-13T03:10:38.087036+0000","last_became_peered":"2024-04-13T03:10:38.087036+0000","last_unstale":"2024-04-13T03:10:39.088589+0000","last_undegraded":"2024-04-13T03:10:39.088589+0000","last_fullsized":"2024-04-13T03:10:39.088589+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:12:21.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087636+0000","last_change":"2024-04-13T03:10:38.097941+0000","last_active":"2024-04-13T03:10:39.087636+0000","last_peered":"2024-04-13T03:10:39.087636+0000","last_clean":"2024-04-13T03:10:39.087636+0000","last_became_active":"2024-04-13T03:10:38.097670+0000","last_became_peered":"2024-04-13T03:10:38.097670+0000","last_unstale":"2024-04-13T03:10:39.087636+0000","last_undegraded":"2024-04-13T03:10:39.087636+0000","last_fullsized":"2024-04-13T03:10:39.087636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:05.257864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088138+0000","last_change":"2024-04-13T03:10:38.095013+0000","last_active":"2024-04-13T03:10:39.088138+0000","last_peered":"2024-04-13T03:10:39.088138+0000","last_clean":"2024-04-13T03:10:39.088138+0000","last_became_active":"2024-04-13T03:10:38.094745+0000","last_became_peered":"2024-04-13T03:10:38.094745+0000","last_unstale":"2024-04-13T03:10:39.088138+0000","last_undegraded":"2024-04-13T03:10:39.088138+0000","last_fullsized":"2024-04-13T03:10:39.088138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:47.557663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088383+0000","last_change":"2024-04-13T03:10:38.100685+0000","last_active":"2024-04-13T03:10:39.088383+0000","last_peered":"2024-04-13T03:10:39.088383+0000","last_clean":"2024-04-13T03:10:39.088383+0000","last_became_active":"2024-04-13T03:10:38.099705+0000","last_became_peered":"2024-04-13T03:10:38.099705+0000","last_unstale":"2024-04-13T03:10:39.088383+0000","last_undegraded":"2024-04-13T03:10:39.088383+0000","last_fullsized":"2024-04-13T03:10:39.088383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:30:43.275335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.089067+0000","last_change":"2024-04-13T03:10:38.096269+0000","last_active":"2024-04-13T03:10:39.089067+0000","last_peered":"2024-04-13T03:10:39.089067+0000","last_clean":"2024-04-13T03:10:39.089067+0000","last_became_active":"2024-04-13T03:10:38.095819+0000","last_became_peered":"2024-04-13T03:10:38.095819+0000","last_unstale":"2024-04-13T03:10:39.089067+0000","last_undegraded":"2024-04-13T03:10:39.089067+0000","last_fullsized":"2024-04-13T03:10:39.089067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:43:00.620908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088206+0000","last_change":"2024-04-13T03:10:38.087019+0000","last_active":"2024-04-13T03:10:39.088206+0000","last_peered":"2024-04-13T03:10:39.088206+0000","last_clean":"2024-04-13T03:10:39.088206+0000","last_became_active":"2024-04-13T03:10:38.086727+0000","last_became_peered":"2024-04-13T03:10:38.086727+0000","last_unstale":"2024-04-13T03:10:39.088206+0000","last_undegraded":"2024-04-13T03:10:39.088206+0000","last_fullsized":"2024-04-13T03:10:39.088206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:06:37.773824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088233+0000","last_change":"2024-04-13T03:10:38.099390+0000","last_active":"2024-04-13T03:10:39.088233+0000","last_peered":"2024-04-13T03:10:39.088233+0000","last_clean":"2024-04-13T03:10:39.088233+0000","last_became_active":"2024-04-13T03:10:38.099105+0000","last_became_peered":"2024-04-13T03:10:38.099105+0000","last_unstale":"2024-04-13T03:10:39.088233+0000","last_undegraded":"2024-04-13T03:10:39.088233+0000","last_fullsized":"2024-04-13T03:10:39.088233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:50.379248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923884+0000","last_change":"2024-04-13T03:10:38.097654+0000","last_active":"2024-04-13T03:10:39.923884+0000","last_peered":"2024-04-13T03:10:39.923884+0000","last_clean":"2024-04-13T03:10:39.923884+0000","last_became_active":"2024-04-13T03:10:38.097318+0000","last_became_peered":"2024-04-13T03:10:38.097318+0000","last_unstale":"2024-04-13T03:10:39.923884+0000","last_undegraded":"2024-04-13T03:10:39.923884+0000","last_fullsized":"2024-04-13T03:10:39.923884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:05.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088149+0000","last_change":"2024-04-13T03:10:38.101376+0000","last_active":"2024-04-13T03:10:39.088149+0000","last_peered":"2024-04-13T03:10:39.088149+0000","last_clean":"2024-04-13T03:10:39.088149+0000","last_became_active":"2024-04-13T03:10:38.100219+0000","last_became_peered":"2024-04-13T03:10:38.100219+0000","last_unstale":"2024-04-13T03:10:39.088149+0000","last_undegraded":"2024-04-13T03:10:39.088149+0000","last_fullsized":"2024-04-13T03:10:39.088149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:33:22.122718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923834+0000","last_change":"2024-04-13T03:10:38.103196+0000","last_active":"2024-04-13T03:10:39.923834+0000","last_peered":"2024-04-13T03:10:39.923834+0000","last_clean":"2024-04-13T03:10:39.923834+0000","last_became_active":"2024-04-13T03:10:38.102559+0000","last_became_peered":"2024-04-13T03:10:38.102559+0000","last_unstale":"2024-04-13T03:10:39.923834+0000","last_undegraded":"2024-04-13T03:10:39.923834+0000","last_fullsized":"2024-04-13T03:10:39.923834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:32.685309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.925337+0000","last_change":"2024-04-13T03:10:38.103174+0000","last_active":"2024-04-13T03:10:39.925337+0000","last_peered":"2024-04-13T03:10:39.925337+0000","last_clean":"2024-04-13T03:10:39.925337+0000","last_became_active":"2024-04-13T03:10:38.102909+0000","last_became_peered":"2024-04-13T03:10:38.102909+0000","last_unstale":"2024-04-13T03:10:39.925337+0000","last_undegraded":"2024-04-13T03:10:39.925337+0000","last_fullsized":"2024-04-13T03:10:39.925337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:30:43.275335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924844+0000","last_change":"2024-04-13T03:10:38.090227+0000","last_active":"2024-04-13T03:10:39.924844+0000","last_peered":"2024-04-13T03:10:39.924844+0000","last_clean":"2024-04-13T03:10:39.924844+0000","last_became_active":"2024-04-13T03:10:38.089916+0000","last_became_peered":"2024-04-13T03:10:38.089916+0000","last_unstale":"2024-04-13T03:10:39.924844+0000","last_undegraded":"2024-04-13T03:10:39.924844+0000","last_fullsized":"2024-04-13T03:10:39.924844+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:15:53.561596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923718+0000","last_change":"2024-04-13T03:10:38.087023+0000","last_active":"2024-04-13T03:10:39.923718+0000","last_peered":"2024-04-13T03:10:39.923718+0000","last_clean":"2024-04-13T03:10:39.923718+0000","last_became_active":"2024-04-13T03:10:38.086758+0000","last_became_peered":"2024-04-13T03:10:38.086758+0000","last_unstale":"2024-04-13T03:10:39.923718+0000","last_undegraded":"2024-04-13T03:10:39.923718+0000","last_fullsized":"2024-04-13T03:10:39.923718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:20.192373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924686+0000","last_change":"2024-04-13T03:10:38.090384+0000","last_active":"2024-04-13T03:10:39.924686+0000","last_peered":"2024-04-13T03:10:39.924686+0000","last_clean":"2024-04-13T03:10:39.924686+0000","last_became_active":"2024-04-13T03:10:38.090068+0000","last_became_peered":"2024-04-13T03:10:38.090068+0000","last_unstale":"2024-04-13T03:10:39.924686+0000","last_undegraded":"2024-04-13T03:10:39.924686+0000","last_fullsized":"2024-04-13T03:10:39.924686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:30.694280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088099+0000","last_change":"2024-04-13T03:10:38.100485+0000","last_active":"2024-04-13T03:10:39.088099+0000","last_peered":"2024-04-13T03:10:39.088099+0000","last_clean":"2024-04-13T03:10:39.088099+0000","last_became_active":"2024-04-13T03:10:38.100182+0000","last_became_peered":"2024-04-13T03:10:38.100182+0000","last_unstale":"2024-04-13T03:10:39.088099+0000","last_undegraded":"2024-04-13T03:10:39.088099+0000","last_fullsized":"2024-04-13T03:10:39.088099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:06:37.773824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924535+0000","last_change":"2024-04-13T03:10:38.094322+0000","last_active":"2024-04-13T03:10:39.924535+0000","last_peered":"2024-04-13T03:10:39.924535+0000","last_clean":"2024-04-13T03:10:39.924535+0000","last_became_active":"2024-04-13T03:10:38.094080+0000","last_became_peered":"2024-04-13T03:10:38.094080+0000","last_unstale":"2024-04-13T03:10:39.924535+0000","last_undegraded":"2024-04-13T03:10:39.924535+0000","last_fullsized":"2024-04-13T03:10:39.924535+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:06:37.773824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923821+0000","last_change":"2024-04-13T03:10:38.086366+0000","last_active":"2024-04-13T03:10:39.923821+0000","last_peered":"2024-04-13T03:10:39.923821+0000","last_clean":"2024-04-13T03:10:39.923821+0000","last_became_active":"2024-04-13T03:10:38.086116+0000","last_became_peered":"2024-04-13T03:10:38.086116+0000","last_unstale":"2024-04-13T03:10:39.923821+0000","last_undegraded":"2024-04-13T03:10:39.923821+0000","last_fullsized":"2024-04-13T03:10:39.923821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:05.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923161+0000","last_change":"2024-04-13T03:10:38.090409+0000","last_active":"2024-04-13T03:10:39.923161+0000","last_peered":"2024-04-13T03:10:39.923161+0000","last_clean":"2024-04-13T03:10:39.923161+0000","last_became_active":"2024-04-13T03:10:38.089969+0000","last_became_peered":"2024-04-13T03:10:38.089969+0000","last_unstale":"2024-04-13T03:10:39.923161+0000","last_undegraded":"2024-04-13T03:10:39.923161+0000","last_fullsized":"2024-04-13T03:10:39.923161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:14.190121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924954+0000","last_change":"2024-04-13T03:10:38.082904+0000","last_active":"2024-04-13T03:10:39.924954+0000","last_peered":"2024-04-13T03:10:39.924954+0000","last_clean":"2024-04-13T03:10:39.924954+0000","last_became_active":"2024-04-13T03:10:38.082617+0000","last_became_peered":"2024-04-13T03:10:38.082617+0000","last_unstale":"2024-04-13T03:10:39.924954+0000","last_undegraded":"2024-04-13T03:10:39.924954+0000","last_fullsized":"2024-04-13T03:10:39.924954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:57.566042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923276+0000","last_change":"2024-04-13T03:10:38.097607+0000","last_active":"2024-04-13T03:10:39.923276+0000","last_peered":"2024-04-13T03:10:39.923276+0000","last_clean":"2024-04-13T03:10:39.923276+0000","last_became_active":"2024-04-13T03:10:38.097333+0000","last_became_peered":"2024-04-13T03:10:38.097333+0000","last_unstale":"2024-04-13T03:10:39.923276+0000","last_undegraded":"2024-04-13T03:10:39.923276+0000","last_fullsized":"2024-04-13T03:10:39.923276+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:32.685309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924849+0000","last_change":"2024-04-13T03:10:38.090676+0000","last_active":"2024-04-13T03:10:39.924849+0000","last_peered":"2024-04-13T03:10:39.924849+0000","last_clean":"2024-04-13T03:10:39.924849+0000","last_became_active":"2024-04-13T03:10:38.089480+0000","last_became_peered":"2024-04-13T03:10:38.089480+0000","last_unstale":"2024-04-13T03:10:39.924849+0000","last_undegraded":"2024-04-13T03:10:39.924849+0000","last_fullsized":"2024-04-13T03:10:39.924849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:49:33.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088643+0000","last_change":"2024-04-13T03:10:38.088072+0000","last_active":"2024-04-13T03:10:39.088643+0000","last_peered":"2024-04-13T03:10:39.088643+0000","last_clean":"2024-04-13T03:10:39.088643+0000","last_became_active":"2024-04-13T03:10:38.087223+0000","last_became_peered":"2024-04-13T03:10:38.087223+0000","last_unstale":"2024-04-13T03:10:39.088643+0000","last_undegraded":"2024-04-13T03:10:39.088643+0000","last_fullsized":"2024-04-13T03:10:39.088643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:14:15.250968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924046+0000","last_change":"2024-04-13T03:10:38.084318+0000","last_active":"2024-04-13T03:10:39.924046+0000","last_peered":"2024-04-13T03:10:39.924046+0000","last_clean":"2024-04-13T03:10:39.924046+0000","last_became_active":"2024-04-13T03:10:38.084051+0000","last_became_peered":"2024-04-13T03:10:38.084051+0000","last_unstale":"2024-04-13T03:10:39.924046+0000","last_undegraded":"2024-04-13T03:10:39.924046+0000","last_fullsized":"2024-04-13T03:10:39.924046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:31:56.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924431+0000","last_change":"2024-04-13T03:10:38.085116+0000","last_active":"2024-04-13T03:10:39.924431+0000","last_peered":"2024-04-13T03:10:39.924431+0000","last_clean":"2024-04-13T03:10:39.924431+0000","last_became_active":"2024-04-13T03:10:38.084842+0000","last_became_peered":"2024-04-13T03:10:38.084842+0000","last_unstale":"2024-04-13T03:10:39.924431+0000","last_undegraded":"2024-04-13T03:10:39.924431+0000","last_fullsized":"2024-04-13T03:10:39.924431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:42:28.561668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924470+0000","last_change":"2024-04-13T03:10:38.085235+0000","last_active":"2024-04-13T03:10:39.924470+0000","last_peered":"2024-04-13T03:10:39.924470+0000","last_clean":"2024-04-13T03:10:39.924470+0000","last_became_active":"2024-04-13T03:10:38.084970+0000","last_became_peered":"2024-04-13T03:10:38.084970+0000","last_unstale":"2024-04-13T03:10:39.924470+0000","last_undegraded":"2024-04-13T03:10:39.924470+0000","last_fullsized":"2024-04-13T03:10:39.924470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:16.042403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923116+0000","last_change":"2024-04-13T03:10:38.082888+0000","last_active":"2024-04-13T03:10:39.923116+0000","last_peered":"2024-04-13T03:10:39.923116+0000","last_clean":"2024-04-13T03:10:39.923116+0000","last_became_active":"2024-04-13T03:10:38.082569+0000","last_became_peered":"2024-04-13T03:10:38.082569+0000","last_unstale":"2024-04-13T03:10:39.923116+0000","last_undegraded":"2024-04-13T03:10:39.923116+0000","last_fullsized":"2024-04-13T03:10:39.923116+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:43.765643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.925183+0000","last_change":"2024-04-13T03:10:38.103054+0000","last_active":"2024-04-13T03:10:39.925183+0000","last_peered":"2024-04-13T03:10:39.925183+0000","last_clean":"2024-04-13T03:10:39.925183+0000","last_became_active":"2024-04-13T03:10:38.102761+0000","last_became_peered":"2024-04-13T03:10:38.102761+0000","last_unstale":"2024-04-13T03:10:39.925183+0000","last_undegraded":"2024-04-13T03:10:39.925183+0000","last_fullsized":"2024-04-13T03:10:39.925183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:06:37.773824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924937+0000","last_change":"2024-04-13T03:10:37.085786+0000","last_active":"2024-04-13T03:10:39.924937+0000","last_peered":"2024-04-13T03:10:39.924937+0000","last_clean":"2024-04-13T03:10:39.924937+0000","last_became_active":"2024-04-13T03:10:37.085500+0000","last_became_peered":"2024-04-13T03:10:37.085500+0000","last_unstale":"2024-04-13T03:10:39.924937+0000","last_undegraded":"2024-04-13T03:10:39.924937+0000","last_fullsized":"2024-04-13T03:10:39.924937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:19.190392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923565+0000","last_change":"2024-04-13T03:10:38.102738+0000","last_active":"2024-04-13T03:10:39.923565+0000","last_peered":"2024-04-13T03:10:39.923565+0000","last_clean":"2024-04-13T03:10:39.923565+0000","last_became_active":"2024-04-13T03:10:38.102467+0000","last_became_peered":"2024-04-13T03:10:38.102467+0000","last_unstale":"2024-04-13T03:10:39.923565+0000","last_undegraded":"2024-04-13T03:10:39.923565+0000","last_fullsized":"2024-04-13T03:10:39.923565+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:16.042403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087869+0000","last_change":"2024-04-13T03:10:37.083216+0000","last_active":"2024-04-13T03:10:39.087869+0000","last_peered":"2024-04-13T03:10:39.087869+0000","last_clean":"2024-04-13T03:10:39.087869+0000","last_became_active":"2024-04-13T03:10:37.082975+0000","last_became_peered":"2024-04-13T03:10:37.082975+0000","last_unstale":"2024-04-13T03:10:39.087869+0000","last_undegraded":"2024-04-13T03:10:39.087869+0000","last_fullsized":"2024-04-13T03:10:39.087869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924280+0000","last_change":"2024-04-13T03:10:38.099658+0000","last_active":"2024-04-13T03:10:39.924280+0000","last_peered":"2024-04-13T03:10:39.924280+0000","last_clean":"2024-04-13T03:10:39.924280+0000","last_became_active":"2024-04-13T03:10:38.099374+0000","last_became_peered":"2024-04-13T03:10:38.099374+0000","last_unstale":"2024-04-13T03:10:39.924280+0000","last_undegraded":"2024-04-13T03:10:39.924280+0000","last_fullsized":"2024-04-13T03:10:39.924280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:42:08.312907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923668+0000","last_change":"2024-04-13T03:10:37.084531+0000","last_active":"2024-04-13T03:10:39.923668+0000","last_peered":"2024-04-13T03:10:39.923668+0000","last_clean":"2024-04-13T03:10:39.923668+0000","last_became_active":"2024-04-13T03:10:37.084244+0000","last_became_peered":"2024-04-13T03:10:37.084244+0000","last_unstale":"2024-04-13T03:10:39.923668+0000","last_undegraded":"2024-04-13T03:10:39.923668+0000","last_fullsized":"2024-04-13T03:10:39.923668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:42:27.559686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088509+0000","last_change":"2024-04-13T03:10:38.096330+0000","last_active":"2024-04-13T03:10:39.088509+0000","last_peered":"2024-04-13T03:10:39.088509+0000","last_clean":"2024-04-13T03:10:39.088509+0000","last_became_active":"2024-04-13T03:10:38.096061+0000","last_became_peered":"2024-04-13T03:10:38.096061+0000","last_unstale":"2024-04-13T03:10:39.088509+0000","last_undegraded":"2024-04-13T03:10:39.088509+0000","last_fullsized":"2024-04-13T03:10:39.088509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:28:46.775774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924416+0000","last_change":"2024-04-13T03:10:37.083577+0000","last_active":"2024-04-13T03:10:39.924416+0000","last_peered":"2024-04-13T03:10:39.924416+0000","last_clean":"2024-04-13T03:10:39.924416+0000","last_became_active":"2024-04-13T03:10:37.083299+0000","last_became_peered":"2024-04-13T03:10:37.083299+0000","last_unstale":"2024-04-13T03:10:39.924416+0000","last_undegraded":"2024-04-13T03:10:39.924416+0000","last_fullsized":"2024-04-13T03:10:39.924416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:42:27.559686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087600+0000","last_change":"2024-04-13T03:10:38.085707+0000","last_active":"2024-04-13T03:10:39.087600+0000","last_peered":"2024-04-13T03:10:39.087600+0000","last_clean":"2024-04-13T03:10:39.087600+0000","last_became_active":"2024-04-13T03:10:38.083235+0000","last_became_peered":"2024-04-13T03:10:38.083235+0000","last_unstale":"2024-04-13T03:10:39.087600+0000","last_undegraded":"2024-04-13T03:10:39.087600+0000","last_fullsized":"2024-04-13T03:10:39.087600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:13:43.765643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088211+0000","last_change":"2024-04-13T03:10:37.086817+0000","last_active":"2024-04-13T03:10:39.088211+0000","last_peered":"2024-04-13T03:10:39.088211+0000","last_clean":"2024-04-13T03:10:39.088211+0000","last_became_active":"2024-04-13T03:10:37.086544+0000","last_became_peered":"2024-04-13T03:10:37.086544+0000","last_unstale":"2024-04-13T03:10:39.088211+0000","last_undegraded":"2024-04-13T03:10:39.088211+0000","last_fullsized":"2024-04-13T03:10:39.088211+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:29.692298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088736+0000","last_change":"2024-04-13T03:10:38.100417+0000","last_active":"2024-04-13T03:10:39.088736+0000","last_peered":"2024-04-13T03:10:39.088736+0000","last_clean":"2024-04-13T03:10:39.088736+0000","last_became_active":"2024-04-13T03:10:38.099905+0000","last_became_peered":"2024-04-13T03:10:38.099905+0000","last_unstale":"2024-04-13T03:10:39.088736+0000","last_undegraded":"2024-04-13T03:10:39.088736+0000","last_fullsized":"2024-04-13T03:10:39.088736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:56:46.246882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924553+0000","last_change":"2024-04-13T03:10:37.089557+0000","last_active":"2024-04-13T03:10:39.924553+0000","last_peered":"2024-04-13T03:10:39.924553+0000","last_clean":"2024-04-13T03:10:39.924553+0000","last_became_active":"2024-04-13T03:10:37.089249+0000","last_became_peered":"2024-04-13T03:10:37.089249+0000","last_unstale":"2024-04-13T03:10:39.924553+0000","last_undegraded":"2024-04-13T03:10:39.924553+0000","last_fullsized":"2024-04-13T03:10:39.924553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:18.906288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924416+0000","last_change":"2024-04-13T03:10:38.098012+0000","last_active":"2024-04-13T03:10:39.924416+0000","last_peered":"2024-04-13T03:10:39.924416+0000","last_clean":"2024-04-13T03:10:39.924416+0000","last_became_active":"2024-04-13T03:10:38.097743+0000","last_became_peered":"2024-04-13T03:10:38.097743+0000","last_unstale":"2024-04-13T03:10:39.924416+0000","last_undegraded":"2024-04-13T03:10:39.924416+0000","last_fullsized":"2024-04-13T03:10:39.924416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:47.557663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923286+0000","last_change":"2024-04-13T03:10:37.088764+0000","last_active":"2024-04-13T03:10:39.923286+0000","last_peered":"2024-04-13T03:10:39.923286+0000","last_clean":"2024-04-13T03:10:39.923286+0000","last_became_active":"2024-04-13T03:10:37.088507+0000","last_became_peered":"2024-04-13T03:10:37.088507+0000","last_unstale":"2024-04-13T03:10:39.923286+0000","last_undegraded":"2024-04-13T03:10:39.923286+0000","last_fullsized":"2024-04-13T03:10:39.923286+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:31:55.316202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087948+0000","last_change":"2024-04-13T03:10:38.090584+0000","last_active":"2024-04-13T03:10:39.087948+0000","last_peered":"2024-04-13T03:10:39.087948+0000","last_clean":"2024-04-13T03:10:39.087948+0000","last_became_active":"2024-04-13T03:10:38.090329+0000","last_became_peered":"2024-04-13T03:10:38.090329+0000","last_unstale":"2024-04-13T03:10:39.087948+0000","last_undegraded":"2024-04-13T03:10:39.087948+0000","last_fullsized":"2024-04-13T03:10:39.087948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:43:00.620908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923623+0000","last_change":"2024-04-13T03:10:37.083447+0000","last_active":"2024-04-13T03:10:39.923623+0000","last_peered":"2024-04-13T03:10:39.923623+0000","last_clean":"2024-04-13T03:10:39.923623+0000","last_became_active":"2024-04-13T03:10:37.083159+0000","last_became_peered":"2024-04-13T03:10:37.083159+0000","last_unstale":"2024-04-13T03:10:39.923623+0000","last_undegraded":"2024-04-13T03:10:39.923623+0000","last_fullsized":"2024-04-13T03:10:39.923623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:31:55.316202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923806+0000","last_change":"2024-04-13T03:10:38.089426+0000","last_active":"2024-04-13T03:10:39.923806+0000","last_peered":"2024-04-13T03:10:39.923806+0000","last_clean":"2024-04-13T03:10:39.923806+0000","last_became_active":"2024-04-13T03:10:38.089024+0000","last_became_peered":"2024-04-13T03:10:38.089024+0000","last_unstale":"2024-04-13T03:10:39.923806+0000","last_undegraded":"2024-04-13T03:10:39.923806+0000","last_fullsized":"2024-04-13T03:10:39.923806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:22.427148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088590+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:39.088590+0000","last_peered":"2024-04-13T03:10:39.088590+0000","last_clean":"2024-04-13T03:10:39.088590+0000","last_became_active":"2024-04-13T03:10:37.081899+0000","last_became_peered":"2024-04-13T03:10:37.081899+0000","last_unstale":"2024-04-13T03:10:39.088590+0000","last_undegraded":"2024-04-13T03:10:39.088590+0000","last_fullsized":"2024-04-13T03:10:39.088590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:15.040422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.925229+0000","last_change":"2024-04-13T03:10:38.099563+0000","last_active":"2024-04-13T03:10:39.925229+0000","last_peered":"2024-04-13T03:10:39.925229+0000","last_clean":"2024-04-13T03:10:39.925229+0000","last_became_active":"2024-04-13T03:10:38.099275+0000","last_became_peered":"2024-04-13T03:10:38.099275+0000","last_unstale":"2024-04-13T03:10:39.925229+0000","last_undegraded":"2024-04-13T03:10:39.925229+0000","last_fullsized":"2024-04-13T03:10:39.925229+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:17:50.379248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088605+0000","last_change":"2024-04-13T03:10:37.085297+0000","last_active":"2024-04-13T03:10:39.088605+0000","last_peered":"2024-04-13T03:10:39.088605+0000","last_clean":"2024-04-13T03:10:39.088605+0000","last_became_active":"2024-04-13T03:10:37.085009+0000","last_became_peered":"2024-04-13T03:10:37.085009+0000","last_unstale":"2024-04-13T03:10:39.088605+0000","last_undegraded":"2024-04-13T03:10:39.088605+0000","last_fullsized":"2024-04-13T03:10:39.088605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:31:55.316202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087878+0000","last_change":"2024-04-13T03:10:38.089248+0000","last_active":"2024-04-13T03:10:39.087878+0000","last_peered":"2024-04-13T03:10:39.087878+0000","last_clean":"2024-04-13T03:10:39.087878+0000","last_became_active":"2024-04-13T03:10:38.088978+0000","last_became_peered":"2024-04-13T03:10:38.088978+0000","last_unstale":"2024-04-13T03:10:39.087878+0000","last_undegraded":"2024-04-13T03:10:39.087878+0000","last_fullsized":"2024-04-13T03:10:39.087878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:41.891801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.089339+0000","last_change":"2024-04-13T03:10:37.088374+0000","last_active":"2024-04-13T03:10:39.089339+0000","last_peered":"2024-04-13T03:10:39.089339+0000","last_clean":"2024-04-13T03:10:39.089339+0000","last_became_active":"2024-04-13T03:10:37.088068+0000","last_became_peered":"2024-04-13T03:10:37.088068+0000","last_unstale":"2024-04-13T03:10:39.089339+0000","last_undegraded":"2024-04-13T03:10:39.089339+0000","last_fullsized":"2024-04-13T03:10:39.089339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:42:27.559686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924083+0000","last_change":"2024-04-13T03:10:38.093721+0000","last_active":"2024-04-13T03:10:39.924083+0000","last_peered":"2024-04-13T03:10:39.924083+0000","last_clean":"2024-04-13T03:10:39.924083+0000","last_became_active":"2024-04-13T03:10:38.093429+0000","last_became_peered":"2024-04-13T03:10:38.093429+0000","last_unstale":"2024-04-13T03:10:39.924083+0000","last_undegraded":"2024-04-13T03:10:39.924083+0000","last_fullsized":"2024-04-13T03:10:39.924083+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:16.042403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088106+0000","last_change":"2024-04-13T03:10:37.082163+0000","last_active":"2024-04-13T03:10:39.088106+0000","last_peered":"2024-04-13T03:10:39.088106+0000","last_clean":"2024-04-13T03:10:39.088106+0000","last_became_active":"2024-04-13T03:10:37.081896+0000","last_became_peered":"2024-04-13T03:10:37.081896+0000","last_unstale":"2024-04-13T03:10:39.088106+0000","last_undegraded":"2024-04-13T03:10:39.088106+0000","last_fullsized":"2024-04-13T03:10:39.088106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:40.889819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087735+0000","last_change":"2024-04-13T03:10:38.094977+0000","last_active":"2024-04-13T03:10:39.087735+0000","last_peered":"2024-04-13T03:10:39.087735+0000","last_clean":"2024-04-13T03:10:39.087735+0000","last_became_active":"2024-04-13T03:10:38.094708+0000","last_became_peered":"2024-04-13T03:10:38.094708+0000","last_unstale":"2024-04-13T03:10:39.087735+0000","last_undegraded":"2024-04-13T03:10:39.087735+0000","last_fullsized":"2024-04-13T03:10:39.087735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:32.685309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088339+0000","last_change":"2024-04-13T03:10:37.081457+0000","last_active":"2024-04-13T03:10:39.088339+0000","last_peered":"2024-04-13T03:10:39.088339+0000","last_clean":"2024-04-13T03:10:39.088339+0000","last_became_active":"2024-04-13T03:10:37.081232+0000","last_became_peered":"2024-04-13T03:10:37.081232+0000","last_unstale":"2024-04-13T03:10:39.088339+0000","last_undegraded":"2024-04-13T03:10:39.088339+0000","last_fullsized":"2024-04-13T03:10:39.088339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:04.787297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088474+0000","last_change":"2024-04-13T03:10:38.086787+0000","last_active":"2024-04-13T03:10:39.088474+0000","last_peered":"2024-04-13T03:10:39.088474+0000","last_clean":"2024-04-13T03:10:39.088474+0000","last_became_active":"2024-04-13T03:10:38.086539+0000","last_became_peered":"2024-04-13T03:10:38.086539+0000","last_unstale":"2024-04-13T03:10:39.088474+0000","last_undegraded":"2024-04-13T03:10:39.088474+0000","last_fullsized":"2024-04-13T03:10:39.088474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:56:49.820124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923567+0000","last_change":"2024-04-13T03:10:37.083620+0000","last_active":"2024-04-13T03:10:39.923567+0000","last_peered":"2024-04-13T03:10:39.923567+0000","last_clean":"2024-04-13T03:10:39.923567+0000","last_became_active":"2024-04-13T03:10:37.083346+0000","last_became_peered":"2024-04-13T03:10:37.083346+0000","last_unstale":"2024-04-13T03:10:39.923567+0000","last_undegraded":"2024-04-13T03:10:39.923567+0000","last_fullsized":"2024-04-13T03:10:39.923567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:29.692298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924668+0000","last_change":"2024-04-13T03:10:38.083953+0000","last_active":"2024-04-13T03:10:39.924668+0000","last_peered":"2024-04-13T03:10:39.924668+0000","last_clean":"2024-04-13T03:10:39.924668+0000","last_became_active":"2024-04-13T03:10:38.083674+0000","last_became_peered":"2024-04-13T03:10:38.083674+0000","last_unstale":"2024-04-13T03:10:39.924668+0000","last_undegraded":"2024-04-13T03:10:39.924668+0000","last_fullsized":"2024-04-13T03:10:39.924668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:42:08.312907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924012+0000","last_change":"2024-04-13T03:10:37.086440+0000","last_active":"2024-04-13T03:10:39.924012+0000","last_peered":"2024-04-13T03:10:39.924012+0000","last_clean":"2024-04-13T03:10:39.924012+0000","last_became_active":"2024-04-13T03:10:37.086158+0000","last_became_peered":"2024-04-13T03:10:37.086158+0000","last_unstale":"2024-04-13T03:10:39.924012+0000","last_undegraded":"2024-04-13T03:10:39.924012+0000","last_fullsized":"2024-04-13T03:10:39.924012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:54:08.030879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088239+0000","last_change":"2024-04-13T03:10:38.083175+0000","last_active":"2024-04-13T03:10:39.088239+0000","last_peered":"2024-04-13T03:10:39.088239+0000","last_clean":"2024-04-13T03:10:39.088239+0000","last_became_active":"2024-04-13T03:10:38.082910+0000","last_became_peered":"2024-04-13T03:10:38.082910+0000","last_unstale":"2024-04-13T03:10:39.088239+0000","last_undegraded":"2024-04-13T03:10:39.088239+0000","last_fullsized":"2024-04-13T03:10:39.088239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:55.271313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923936+0000","last_change":"2024-04-13T03:10:37.089769+0000","last_active":"2024-04-13T03:10:39.923936+0000","last_peered":"2024-04-13T03:10:39.923936+0000","last_clean":"2024-04-13T03:10:39.923936+0000","last_became_active":"2024-04-13T03:10:37.089284+0000","last_became_peered":"2024-04-13T03:10:37.089284+0000","last_unstale":"2024-04-13T03:10:39.923936+0000","last_undegraded":"2024-04-13T03:10:39.923936+0000","last_fullsized":"2024-04-13T03:10:39.923936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:03:05.993486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088027+0000","last_change":"2024-04-13T03:10:38.090333+0000","last_active":"2024-04-13T03:10:39.088027+0000","last_peered":"2024-04-13T03:10:39.088027+0000","last_clean":"2024-04-13T03:10:39.088027+0000","last_became_active":"2024-04-13T03:10:38.090094+0000","last_became_peered":"2024-04-13T03:10:38.090094+0000","last_unstale":"2024-04-13T03:10:39.088027+0000","last_undegraded":"2024-04-13T03:10:39.088027+0000","last_fullsized":"2024-04-13T03:10:39.088027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:33:48.108906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088990+0000","last_change":"2024-04-13T03:10:37.087631+0000","last_active":"2024-04-13T03:10:39.088990+0000","last_peered":"2024-04-13T03:10:39.088990+0000","last_clean":"2024-04-13T03:10:39.088990+0000","last_became_active":"2024-04-13T03:10:37.087385+0000","last_became_peered":"2024-04-13T03:10:37.087385+0000","last_unstale":"2024-04-13T03:10:39.088990+0000","last_undegraded":"2024-04-13T03:10:39.088990+0000","last_fullsized":"2024-04-13T03:10:39.088990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:15:52.559615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.089223+0000","last_change":"2024-04-13T03:10:38.101050+0000","last_active":"2024-04-13T03:10:39.089223+0000","last_peered":"2024-04-13T03:10:39.089223+0000","last_clean":"2024-04-13T03:10:39.089223+0000","last_became_active":"2024-04-13T03:10:38.100536+0000","last_became_peered":"2024-04-13T03:10:38.100536+0000","last_unstale":"2024-04-13T03:10:39.089223+0000","last_undegraded":"2024-04-13T03:10:39.089223+0000","last_fullsized":"2024-04-13T03:10:39.089223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:53:46.681800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924580+0000","last_change":"2024-04-13T03:10:37.085447+0000","last_active":"2024-04-13T03:10:39.924580+0000","last_peered":"2024-04-13T03:10:39.924580+0000","last_clean":"2024-04-13T03:10:39.924580+0000","last_became_active":"2024-04-13T03:10:37.085157+0000","last_became_peered":"2024-04-13T03:10:37.085157+0000","last_unstale":"2024-04-13T03:10:39.924580+0000","last_undegraded":"2024-04-13T03:10:39.924580+0000","last_fullsized":"2024-04-13T03:10:39.924580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:29.692298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924047+0000","last_change":"2024-04-13T03:10:38.094959+0000","last_active":"2024-04-13T03:10:39.924047+0000","last_peered":"2024-04-13T03:10:39.924047+0000","last_clean":"2024-04-13T03:10:39.924047+0000","last_became_active":"2024-04-13T03:10:38.094697+0000","last_became_peered":"2024-04-13T03:10:38.094697+0000","last_unstale":"2024-04-13T03:10:39.924047+0000","last_undegraded":"2024-04-13T03:10:39.924047+0000","last_fullsized":"2024-04-13T03:10:39.924047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:32.685309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923690+0000","last_change":"2024-04-13T03:10:37.087391+0000","last_active":"2024-04-13T03:10:39.923690+0000","last_peered":"2024-04-13T03:10:39.923690+0000","last_clean":"2024-04-13T03:10:39.923690+0000","last_became_active":"2024-04-13T03:10:37.087131+0000","last_became_peered":"2024-04-13T03:10:37.087131+0000","last_unstale":"2024-04-13T03:10:39.923690+0000","last_undegraded":"2024-04-13T03:10:39.923690+0000","last_fullsized":"2024-04-13T03:10:39.923690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:56:34.181384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923465+0000","last_change":"2024-04-13T03:10:38.091977+0000","last_active":"2024-04-13T03:10:39.923465+0000","last_peered":"2024-04-13T03:10:39.923465+0000","last_clean":"2024-04-13T03:10:39.923465+0000","last_became_active":"2024-04-13T03:10:38.091707+0000","last_became_peered":"2024-04-13T03:10:38.091707+0000","last_unstale":"2024-04-13T03:10:39.923465+0000","last_undegraded":"2024-04-13T03:10:39.923465+0000","last_fullsized":"2024-04-13T03:10:39.923465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:54:09.032860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923356+0000","last_change":"2024-04-13T03:10:37.084719+0000","last_active":"2024-04-13T03:10:39.923356+0000","last_peered":"2024-04-13T03:10:39.923356+0000","last_clean":"2024-04-13T03:10:39.923356+0000","last_became_active":"2024-04-13T03:10:37.084472+0000","last_became_peered":"2024-04-13T03:10:37.084472+0000","last_unstale":"2024-04-13T03:10:39.923356+0000","last_undegraded":"2024-04-13T03:10:39.923356+0000","last_fullsized":"2024-04-13T03:10:39.923356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924098+0000","last_change":"2024-04-13T03:10:38.086533+0000","last_active":"2024-04-13T03:10:39.924098+0000","last_peered":"2024-04-13T03:10:39.924098+0000","last_clean":"2024-04-13T03:10:39.924098+0000","last_became_active":"2024-04-13T03:10:38.086267+0000","last_became_peered":"2024-04-13T03:10:38.086267+0000","last_unstale":"2024-04-13T03:10:39.924098+0000","last_undegraded":"2024-04-13T03:10:39.924098+0000","last_fullsized":"2024-04-13T03:10:39.924098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:41.891801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923524+0000","last_change":"2024-04-13T03:10:37.078763+0000","last_active":"2024-04-13T03:10:39.923524+0000","last_peered":"2024-04-13T03:10:39.923524+0000","last_clean":"2024-04-13T03:10:39.923524+0000","last_became_active":"2024-04-13T03:10:37.078484+0000","last_became_peered":"2024-04-13T03:10:37.078484+0000","last_unstale":"2024-04-13T03:10:39.923524+0000","last_undegraded":"2024-04-13T03:10:39.923524+0000","last_fullsized":"2024-04-13T03:10:39.923524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:34:15.118172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087973+0000","last_change":"2024-04-13T03:10:38.086356+0000","last_active":"2024-04-13T03:10:39.087973+0000","last_peered":"2024-04-13T03:10:39.087973+0000","last_clean":"2024-04-13T03:10:39.087973+0000","last_became_active":"2024-04-13T03:10:38.086117+0000","last_became_peered":"2024-04-13T03:10:38.086117+0000","last_unstale":"2024-04-13T03:10:39.087973+0000","last_undegraded":"2024-04-13T03:10:39.087973+0000","last_fullsized":"2024-04-13T03:10:39.087973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:16.042403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923548+0000","last_change":"2024-04-13T03:10:37.085242+0000","last_active":"2024-04-13T03:10:39.923548+0000","last_peered":"2024-04-13T03:10:39.923548+0000","last_clean":"2024-04-13T03:10:39.923548+0000","last_became_active":"2024-04-13T03:10:37.084999+0000","last_became_peered":"2024-04-13T03:10:37.084999+0000","last_unstale":"2024-04-13T03:10:39.923548+0000","last_undegraded":"2024-04-13T03:10:39.923548+0000","last_fullsized":"2024-04-13T03:10:39.923548+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:52:01.983365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088143+0000","last_change":"2024-04-13T03:10:38.101267+0000","last_active":"2024-04-13T03:10:39.088143+0000","last_peered":"2024-04-13T03:10:39.088143+0000","last_clean":"2024-04-13T03:10:39.088143+0000","last_became_active":"2024-04-13T03:10:38.100726+0000","last_became_peered":"2024-04-13T03:10:38.100726+0000","last_unstale":"2024-04-13T03:10:39.088143+0000","last_undegraded":"2024-04-13T03:10:39.088143+0000","last_fullsized":"2024-04-13T03:10:39.088143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:43.904476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924443+0000","last_change":"2024-04-13T03:10:37.088188+0000","last_active":"2024-04-13T03:10:39.924443+0000","last_peered":"2024-04-13T03:10:39.924443+0000","last_clean":"2024-04-13T03:10:39.924443+0000","last_became_active":"2024-04-13T03:10:37.087921+0000","last_became_peered":"2024-04-13T03:10:37.087921+0000","last_unstale":"2024-04-13T03:10:39.924443+0000","last_undegraded":"2024-04-13T03:10:39.924443+0000","last_fullsized":"2024-04-13T03:10:39.924443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:44.644760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088098+0000","last_change":"2024-04-13T03:10:38.089972+0000","last_active":"2024-04-13T03:10:39.088098+0000","last_peered":"2024-04-13T03:10:39.088098+0000","last_clean":"2024-04-13T03:10:39.088098+0000","last_became_active":"2024-04-13T03:10:38.089725+0000","last_became_peered":"2024-04-13T03:10:38.089725+0000","last_unstale":"2024-04-13T03:10:39.088098+0000","last_undegraded":"2024-04-13T03:10:39.088098+0000","last_fullsized":"2024-04-13T03:10:39.088098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:35:30.559149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923259+0000","last_change":"2024-04-13T03:10:37.079460+0000","last_active":"2024-04-13T03:10:39.923259+0000","last_peered":"2024-04-13T03:10:39.923259+0000","last_clean":"2024-04-13T03:10:39.923259+0000","last_became_active":"2024-04-13T03:10:37.079188+0000","last_became_peered":"2024-04-13T03:10:37.079188+0000","last_unstale":"2024-04-13T03:10:39.923259+0000","last_undegraded":"2024-04-13T03:10:39.923259+0000","last_fullsized":"2024-04-13T03:10:39.923259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:52:46.308350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923768+0000","last_change":"2024-04-13T03:10:38.097298+0000","last_active":"2024-04-13T03:10:39.923768+0000","last_peered":"2024-04-13T03:10:39.923768+0000","last_clean":"2024-04-13T03:10:39.923768+0000","last_became_active":"2024-04-13T03:10:38.097011+0000","last_became_peered":"2024-04-13T03:10:38.097011+0000","last_unstale":"2024-04-13T03:10:39.923768+0000","last_undegraded":"2024-04-13T03:10:39.923768+0000","last_fullsized":"2024-04-13T03:10:39.923768+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:42:08.312907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.089224+0000","last_change":"2024-04-13T03:10:37.088272+0000","last_active":"2024-04-13T03:10:39.089224+0000","last_peered":"2024-04-13T03:10:39.089224+0000","last_clean":"2024-04-13T03:10:39.089224+0000","last_became_active":"2024-04-13T03:10:37.088023+0000","last_became_peered":"2024-04-13T03:10:37.088023+0000","last_unstale":"2024-04-13T03:10:39.089224+0000","last_undegraded":"2024-04-13T03:10:39.089224+0000","last_fullsized":"2024-04-13T03:10:39.089224+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:49:32.968152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.925196+0000","last_change":"2024-04-13T03:10:38.099535+0000","last_active":"2024-04-13T03:10:39.925196+0000","last_peered":"2024-04-13T03:10:39.925196+0000","last_clean":"2024-04-13T03:10:39.925196+0000","last_became_active":"2024-04-13T03:10:38.099263+0000","last_became_peered":"2024-04-13T03:10:38.099263+0000","last_unstale":"2024-04-13T03:10:39.925196+0000","last_undegraded":"2024-04-13T03:10:39.925196+0000","last_fullsized":"2024-04-13T03:10:39.925196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:43.904476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923710+0000","last_change":"2024-04-13T03:10:37.080915+0000","last_active":"2024-04-13T03:10:39.923710+0000","last_peered":"2024-04-13T03:10:39.923710+0000","last_clean":"2024-04-13T03:10:39.923710+0000","last_became_active":"2024-04-13T03:10:37.080658+0000","last_became_peered":"2024-04-13T03:10:37.080658+0000","last_unstale":"2024-04-13T03:10:39.923710+0000","last_undegraded":"2024-04-13T03:10:39.923710+0000","last_fullsized":"2024-04-13T03:10:39.923710+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:47:05.567513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088278+0000","last_change":"2024-04-13T03:10:38.088376+0000","last_active":"2024-04-13T03:10:39.088278+0000","last_peered":"2024-04-13T03:10:39.088278+0000","last_clean":"2024-04-13T03:10:39.088278+0000","last_became_active":"2024-04-13T03:10:38.088103+0000","last_became_peered":"2024-04-13T03:10:38.088103+0000","last_unstale":"2024-04-13T03:10:39.088278+0000","last_undegraded":"2024-04-13T03:10:39.088278+0000","last_fullsized":"2024-04-13T03:10:39.088278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:09:37.472759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087974+0000","last_change":"2024-04-13T03:10:37.081490+0000","last_active":"2024-04-13T03:10:39.087974+0000","last_peered":"2024-04-13T03:10:39.087974+0000","last_clean":"2024-04-13T03:10:39.087974+0000","last_became_active":"2024-04-13T03:10:37.081226+0000","last_became_peered":"2024-04-13T03:10:37.081226+0000","last_unstale":"2024-04-13T03:10:39.087974+0000","last_undegraded":"2024-04-13T03:10:39.087974+0000","last_fullsized":"2024-04-13T03:10:39.087974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:31:55.316202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088111+0000","last_change":"2024-04-13T03:10:38.084617+0000","last_active":"2024-04-13T03:10:39.088111+0000","last_peered":"2024-04-13T03:10:39.088111+0000","last_clean":"2024-04-13T03:10:39.088111+0000","last_became_active":"2024-04-13T03:10:38.084374+0000","last_became_peered":"2024-04-13T03:10:38.084374+0000","last_unstale":"2024-04-13T03:10:39.088111+0000","last_undegraded":"2024-04-13T03:10:39.088111+0000","last_fullsized":"2024-04-13T03:10:39.088111+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:42:08.312907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087775+0000","last_change":"2024-04-13T03:10:37.083750+0000","last_active":"2024-04-13T03:10:39.087775+0000","last_peered":"2024-04-13T03:10:39.087775+0000","last_clean":"2024-04-13T03:10:39.087775+0000","last_became_active":"2024-04-13T03:10:37.083512+0000","last_became_peered":"2024-04-13T03:10:37.083512+0000","last_unstale":"2024-04-13T03:10:39.087775+0000","last_undegraded":"2024-04-13T03:10:39.087775+0000","last_fullsized":"2024-04-13T03:10:39.087775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923178+0000","last_change":"2024-04-13T03:10:38.083312+0000","last_active":"2024-04-13T03:10:39.923178+0000","last_peered":"2024-04-13T03:10:39.923178+0000","last_clean":"2024-04-13T03:10:39.923178+0000","last_became_active":"2024-04-13T03:10:38.083059+0000","last_became_peered":"2024-04-13T03:10:38.083059+0000","last_unstale":"2024-04-13T03:10:39.923178+0000","last_undegraded":"2024-04-13T03:10:39.923178+0000","last_fullsized":"2024-04-13T03:10:39.923178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:05.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924472+0000","last_change":"2024-04-13T03:10:37.082886+0000","last_active":"2024-04-13T03:10:39.924472+0000","last_peered":"2024-04-13T03:10:39.924472+0000","last_clean":"2024-04-13T03:10:39.924472+0000","last_became_active":"2024-04-13T03:10:37.082622+0000","last_became_peered":"2024-04-13T03:10:37.082622+0000","last_unstale":"2024-04-13T03:10:39.924472+0000","last_undegraded":"2024-04-13T03:10:39.924472+0000","last_fullsized":"2024-04-13T03:10:39.924472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:55:01.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924061+0000","last_change":"2024-04-13T03:10:38.094276+0000","last_active":"2024-04-13T03:10:39.924061+0000","last_peered":"2024-04-13T03:10:39.924061+0000","last_clean":"2024-04-13T03:10:39.924061+0000","last_became_active":"2024-04-13T03:10:38.094001+0000","last_became_peered":"2024-04-13T03:10:38.094001+0000","last_unstale":"2024-04-13T03:10:39.924061+0000","last_undegraded":"2024-04-13T03:10:39.924061+0000","last_fullsized":"2024-04-13T03:10:39.924061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:59:41.891801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.923622+0000","last_change":"2024-04-13T03:10:37.084769+0000","last_active":"2024-04-13T03:10:39.923622+0000","last_peered":"2024-04-13T03:10:39.923622+0000","last_clean":"2024-04-13T03:10:39.923622+0000","last_became_active":"2024-04-13T03:10:37.084496+0000","last_became_peered":"2024-04-13T03:10:37.084496+0000","last_unstale":"2024-04-13T03:10:39.923622+0000","last_undegraded":"2024-04-13T03:10:39.923622+0000","last_fullsized":"2024-04-13T03:10:39.923622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924831+0000","last_change":"2024-04-13T03:10:38.096256+0000","last_active":"2024-04-13T03:10:39.924831+0000","last_peered":"2024-04-13T03:10:39.924831+0000","last_clean":"2024-04-13T03:10:39.924831+0000","last_became_active":"2024-04-13T03:10:38.095792+0000","last_became_peered":"2024-04-13T03:10:38.095792+0000","last_unstale":"2024-04-13T03:10:39.924831+0000","last_undegraded":"2024-04-13T03:10:39.924831+0000","last_fullsized":"2024-04-13T03:10:39.924831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:47.557663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.087719+0000","last_change":"2024-04-13T03:10:37.082340+0000","last_active":"2024-04-13T03:10:39.087719+0000","last_peered":"2024-04-13T03:10:39.087719+0000","last_clean":"2024-04-13T03:10:39.087719+0000","last_became_active":"2024-04-13T03:10:37.082069+0000","last_became_peered":"2024-04-13T03:10:37.082069+0000","last_unstale":"2024-04-13T03:10:39.087719+0000","last_undegraded":"2024-04-13T03:10:39.087719+0000","last_fullsized":"2024-04-13T03:10:39.087719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:31.683327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.088818+0000","last_change":"2024-04-13T03:10:38.097089+0000","last_active":"2024-04-13T03:10:39.088818+0000","last_peered":"2024-04-13T03:10:39.088818+0000","last_clean":"2024-04-13T03:10:39.088818+0000","last_became_active":"2024-04-13T03:10:38.096545+0000","last_became_peered":"2024-04-13T03:10:38.096545+0000","last_unstale":"2024-04-13T03:10:39.088818+0000","last_undegraded":"2024-04-13T03:10:39.088818+0000","last_fullsized":"2024-04-13T03:10:39.088818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:37.057207+0000","last_clean_scrub_stamp":"2024-04-13T03:10:37.057207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:05.257864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.925093+0000","last_change":"2024-04-13T03:10:37.090291+0000","last_active":"2024-04-13T03:10:39.925093+0000","last_peered":"2024-04-13T03:10:39.925093+0000","last_clean":"2024-04-13T03:10:39.925093+0000","last_became_active":"2024-04-13T03:10:37.090004+0000","last_became_peered":"2024-04-13T03:10:37.090004+0000","last_unstale":"2024-04-13T03:10:39.925093+0000","last_undegraded":"2024-04-13T03:10:39.925093+0000","last_fullsized":"2024-04-13T03:10:39.925093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:10:36.055226+0000","last_clean_scrub_stamp":"2024-04-13T03:10:36.055226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:46:25.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-13T03:10:39.924191+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:10:39.924191+0000","last_peered":"2024-04-13T03:10:39.924191+0000","last_clean":"2024-04-13T03:10:39.924191+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:10:39.924191+0000","last_undegraded":"2024-04-13T03:10:39.924191+0000","last_fullsized":"2024-04-13T03:10:39.924191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":197747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":197769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":197235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":197255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":787567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:10:45.776 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:10:45.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:10:45.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.918+0000 7f5b2378c640 1 Processor -- start 2024-04-13T03:10:45.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.918+0000 7f5b2378c640 1 -- start start 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b2378c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b1c070bb0 0x7f5b1c1697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b2378c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1c0b5cf0 con 0x7f5b1c15ee40 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b2378c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1c0b5fd0 con 0x7f5b1c0b1c30 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b2378c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5b1c0b62b0 con 0x7f5b1c070bb0 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b21d29640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b1c070bb0 0x7f5b1c1697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b20d27640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5b1c15ee40 0x7f5b1c0704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43072/0 (socket says 172.21.15.106:43072) 2024-04-13T03:10:45.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b21d29640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b1c070bb0 0x7f5b1c1697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43968/0 (socket says 172.21.15.106:43968) 2024-04-13T03:10:45.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b20d27640 1 -- 172.21.15.106:0/3013005531 learned_addr learned my addr 172.21.15.106:0/3013005531 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:45.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b21d29640 1 -- 172.21.15.106:0/3013005531 >> v1:172.21.15.114:6789/0 conn(0x7f5b1c0b1c30 legacy=0x7f5b1c0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:45.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b21d29640 1 -- 172.21.15.106:0/3013005531 >> v1:172.21.15.106:6789/0 conn(0x7f5b1c15ee40 legacy=0x7f5b1c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.919+0000 7f5b21d29640 1 -- 172.21.15.106:0/3013005531 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c0b6590 con 0x7f5b1c070bb0 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.920+0000 7f5b21d29640 1 --2- 172.21.15.106:0/3013005531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c070bb0 0x7f5b1c1697c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b1402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f660acf8b32e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.920+0000 7f5b13fff640 1 -- 172.21.15.106:0/3013005531 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14039070 con 0x7f5b1c070bb0 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.920+0000 7f5b13fff640 1 -- 172.21.15.106:0/3013005531 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1402cc60 con 0x7f5b1c070bb0 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.920+0000 7f5b13fff640 1 -- 172.21.15.106:0/3013005531 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14034660 con 0x7f5b1c070bb0 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 -- 172.21.15.106:0/3013005531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c070bb0 msgr2=0x7f5b1c1697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3013005531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c070bb0 0x7f5b1c1697c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b1402beb0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 -- 172.21.15.106:0/3013005531 shutdown_connections 2024-04-13T03:10:45.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3013005531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c070bb0 0x7f5b1c1697c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 -- 172.21.15.106:0/3013005531 >> 172.21.15.106:0/3013005531 conn(0x7f5b1c071870 msgr2=0x7f5b1c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:45.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 -- 172.21.15.106:0/3013005531 shutdown_connections 2024-04-13T03:10:45.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.921+0000 7f5b2378c640 1 -- 172.21.15.106:0/3013005531 wait complete. 2024-04-13T03:10:45.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.922+0000 7f5b2378c640 1 Processor -- start 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.922+0000 7f5b2378c640 1 -- start start 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.922+0000 7f5b2378c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b1c070bb0 0x7f5b1c169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b1c0b1c30 0x7f5b1c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c169d00 con 0x7f5b1c15ee40 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c06e8c0 con 0x7f5b1c0b1c30 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1c169e80 con 0x7f5b1c070bb0 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43976/0 (socket says 172.21.15.106:43976) 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 -- 172.21.15.106:0/3779262434 learned_addr learned my addr 172.21.15.106:0/3779262434 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21528640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b1c070bb0 0x7f5b1c169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b20d27640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b1c0b1c30 0x7f5b1c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 -- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b1c070bb0 msgr2=0x7f5b1c169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b1c070bb0 0x7f5b1c169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 -- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b1c0b1c30 msgr2=0x7f5b1c154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b1c0b1c30 0x7f5b1c154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 -- 172.21.15.106:0/3779262434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c0cd0d0 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b21d29640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14044030 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1402ca60 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c0cbcb0 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.923+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14039040 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.924+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c0cc8d0 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.924+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c068a10 con 0x7f5b1c15ee40 2024-04-13T03:10:45.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.925+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b14047070 con 0x7f5b1c15ee40 2024-04-13T03:10:45.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.925+0000 7f5b11ffb640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 0x7f5b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:45.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.925+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5b140ae550 con 0x7f5b1c15ee40 2024-04-13T03:10:45.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.925+0000 7f5b21528640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 0x7f5b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:45.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.926+0000 7f5b21528640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 0x7f5b0006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b18007fd0 tx=0x7f5b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:45.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:45.930+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b1404e070 con 0x7f5b1c15ee40 2024-04-13T03:10:46.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.178+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5b1c15f7b0 con 0x7f5b1c15ee40 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.180+0000 7f5b11ffb640 1 -- 172.21.15.106:0/3779262434 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f5b1c0cd0d0 con 0x7f5b1c15ee40 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:10:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:10:46.183 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:10:46.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 msgr2=0x7f5b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 0x7f5b0006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b18007fd0 tx=0x7f5b18009040 comp rx=0 tx=0).stop 2024-04-13T03:10:46.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 msgr2=0x7f5b1c162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b14004a40 comp rx=0 tx=0).stop 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 shutdown_connections 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b0006c8f0 0x7f5b0006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1c15ee40 0x7f5b1c162140 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b1c0b1c30 0x7f5b1c154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 --2- 172.21.15.106:0/3779262434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b1c070bb0 0x7f5b1c169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 >> 172.21.15.106:0/3779262434 conn(0x7f5b1c071870 msgr2=0x7f5b1c0b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 shutdown_connections 2024-04-13T03:10:46.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.182+0000 7f5b2378c640 1 -- 172.21.15.106:0/3779262434 wait complete. 2024-04-13T03:10:46.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:46.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.292+0000 7fc2f7908640 1 Processor -- start 2024-04-13T03:10:46.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 -- start start 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc2f015c7d0 0x7fc2f0160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc2f00cd2b0 0x7fc2f00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f00d2310 con 0x7fc2f00c60a0 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f00b1f30 con 0x7fc2f015c7d0 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.293+0000 7fc2f7908640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f00b2210 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc2f015c7d0 0x7fc2f0160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc2f015c7d0 0x7fc2f0160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43990/0 (socket says 172.21.15.106:43990) 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 -- 172.21.15.106:0/2842493184 learned_addr learned my addr 172.21.15.106:0/2842493184 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:46.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f4ea3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc2f00c60a0 0x7fc2f00ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43080/0 (socket says 172.21.15.106:43080) 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f5ea5640 1 --2- 172.21.15.106:0/2842493184 >> v2:172.21.15.114:3300/0 conn(0x7fc2f00cd2b0 0x7fc2f00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 -- 172.21.15.106:0/2842493184 >> v2:172.21.15.114:3300/0 conn(0x7fc2f00cd2b0 msgr2=0x7fc2f00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 --2- 172.21.15.106:0/2842493184 >> v2:172.21.15.114:3300/0 conn(0x7fc2f00cd2b0 0x7fc2f00d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 -- 172.21.15.106:0/2842493184 >> v1:172.21.15.106:6789/0 conn(0x7fc2f00c60a0 legacy=0x7fc2f00ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2e7fff640 1 -- 172.21.15.106:0/2842493184 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59333125 0 0) 0x7fc2f00d2310 con 0x7fc2f00c60a0 2024-04-13T03:10:46.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.294+0000 7fc2f56a4640 1 -- 172.21.15.106:0/2842493184 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f00b24f0 con 0x7fc2f015c7d0 2024-04-13T03:10:46.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f56a4640 1 --2- 172.21.15.106:0/2842493184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f015c7d0 0x7fc2f0160dd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0160300 tx=0x7fc2e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a26d09e8c3721 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:46.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2e7fff640 1 -- 172.21.15.106:0/2842493184 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0034070 con 0x7fc2f015c7d0 2024-04-13T03:10:46.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2e7fff640 1 -- 172.21.15.106:0/2842493184 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e0030860 con 0x7fc2f015c7d0 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2e7fff640 1 -- 172.21.15.106:0/2842493184 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0030c30 con 0x7fc2f015c7d0 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 -- 172.21.15.106:0/2842493184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f015c7d0 msgr2=0x7fc2f0160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 --2- 172.21.15.106:0/2842493184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f015c7d0 0x7fc2f0160dd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0160300 tx=0x7fc2e002b9a0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 -- 172.21.15.106:0/2842493184 shutdown_connections 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 --2- 172.21.15.106:0/2842493184 >> v2:172.21.15.114:3300/0 conn(0x7fc2f00cd2b0 0x7fc2f00d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 --2- 172.21.15.106:0/2842493184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f015c7d0 0x7fc2f0160dd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 -- 172.21.15.106:0/2842493184 >> 172.21.15.106:0/2842493184 conn(0x7fc2f00d2550 msgr2=0x7fc2f00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 -- 172.21.15.106:0/2842493184 shutdown_connections 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.295+0000 7fc2f7908640 1 -- 172.21.15.106:0/2842493184 wait complete. 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.296+0000 7fc2f7908640 1 Processor -- start 2024-04-13T03:10:46.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 -- start start 2024-04-13T03:10:46.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2f00c60a0 0x7fc2f00b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 0x7fc2f006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f015cbb0 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00b2cf0 con 0x7fc2f015c7d0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f7908640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f0156370 con 0x7fc2f00c60a0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f4ea3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f4ea3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44006/0 (socket says 172.21.15.106:44006) 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f4ea3640 1 -- 172.21.15.106:0/1861202402 learned_addr learned my addr 172.21.15.106:0/1861202402 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.297+0000 7fc2f56a4640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2f00c60a0 0x7fc2f00b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f5ea5640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 0x7fc2f006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 -- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2f00c60a0 msgr2=0x7fc2f00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2f00c60a0 0x7fc2f00b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 -- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 msgr2=0x7fc2f006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 0x7fc2f006e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 -- 172.21.15.106:0/1861202402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f00cc3d0 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f5ea5640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 0x7fc2f006e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f4ea3640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c4a0 tx=0x7fc2ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec018070 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0074e0 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f00b5730 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec004a20 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.298+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f00b5cf0 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.299+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0069120 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.300+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2ec01b070 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.300+0000 7fc2e5ffb640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 0x7fc2d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:46.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.300+0000 7fc2f56a4640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 0x7fc2d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:46.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.300+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc2ec08aeb0 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.301+0000 7fc2f56a4640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 0x7fc2d406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0036000 tx=0x7fc2e003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:46.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.305+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2ec058b90 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.521+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2f0071260 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.521+0000 7fc2e5ffb640 1 -- 172.21.15.106:0/1861202402 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fc2ec05ce90 con 0x7fc2f00cd2b0 2024-04-13T03:10:46.523 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:46.524 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:10:46.524 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 msgr2=0x7fc2d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 0x7fc2d406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0036000 tx=0x7fc2e003c040 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 msgr2=0x7fc2f015d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c4a0 tx=0x7fc2ec00c960 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 shutdown_connections 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc2d406c8f0 0x7fc2d406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2f015c7d0 0x7fc2f006e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2f00cd2b0 0x7fc2f015d5c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 --2- 172.21.15.106:0/1861202402 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2f00c60a0 0x7fc2f00b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.523+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 >> 172.21.15.106:0/1861202402 conn(0x7fc2f00d2550 msgr2=0x7fc2f006fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:46.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.524+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 shutdown_connections 2024-04-13T03:10:46.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:46.524+0000 7fc2f7908640 1 -- 172.21.15.106:0/1861202402 wait complete. 2024-04-13T03:10:46.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:10:37.420915+0000', 'modified': '2024-04-13T03:10:39.426685+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5059}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5059': {'gid': 5059, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6835/1027593505', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6833', 'nonce': 1027593505}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 1027593505}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5059, 'qdb_cluster': [5059]} 2024-04-13T03:10:46.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:10:46.939 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:10:47.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.039+0000 7f1b97955640 1 Processor -- start 2024-04-13T03:10:47.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.039+0000 7f1b97955640 1 -- start start 2024-04-13T03:10:47.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.040+0000 7f1b97955640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.040+0000 7f1b97955640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.040+0000 7f1b97955640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b900706b0 con 0x7f1b90145f60 2024-04-13T03:10:47.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.040+0000 7f1b97955640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1b90071750 con 0x7f1b9006f800 2024-04-13T03:10:47.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.040+0000 7f1b97955640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900701a0 con 0x7f1b900cdc10 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b94ef0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b94ef0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:44742/0 (socket says 172.21.15.106:44742) 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:59236/0 (socket says 172.21.15.106:59236) 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b94ef0640 1 -- 172.21.15.106:0/4235445323 learned_addr learned my addr 172.21.15.106:0/4235445323 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b956f1640 1 --1- 172.21.15.106:0/4235445323 >> v1:172.21.15.114:6789/0 conn(0x7f1b90145f60 0x7f1b900d0a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:38316/0 (socket says 172.21.15.106:38316) 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 -- 172.21.15.106:0/4235445323 >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 msgr2=0x7f1b9006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/4235445323 >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 -- 172.21.15.106:0/4235445323 >> v1:172.21.15.114:6789/0 conn(0x7f1b90145f60 legacy=0x7f1b900d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b95ef2640 1 -- 172.21.15.106:0/4235445323 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b90070990 con 0x7f1b900cdc10 2024-04-13T03:10:47.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.041+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/4235445323 >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:47.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/4235445323 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097a0 tx=0x7f1b8802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914fe721646154c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b8ffff640 1 -- 172.21.15.106:0/4235445323 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b880315a0 con 0x7f1b900cdc10 2024-04-13T03:10:47.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b8ffff640 1 -- 172.21.15.106:0/4235445323 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88037070 con 0x7f1b900cdc10 2024-04-13T03:10:47.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b8ffff640 1 -- 172.21.15.106:0/4235445323 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88031960 con 0x7f1b900cdc10 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 -- 172.21.15.106:0/4235445323 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 msgr2=0x7f1b900ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 --2- 172.21.15.106:0/4235445323 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097a0 tx=0x7f1b8802b840 comp rx=0 tx=0).stop 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 -- 172.21.15.106:0/4235445323 shutdown_connections 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 --2- 172.21.15.106:0/4235445323 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b900cdc10 0x7f1b900ce0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 --2- 172.21.15.106:0/4235445323 >> v2:172.21.15.114:3300/0 conn(0x7f1b9006f800 0x7f1b9006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.042+0000 7f1b97955640 1 -- 172.21.15.106:0/4235445323 >> 172.21.15.106:0/4235445323 conn(0x7f1b900d1b60 msgr2=0x7f1b9006f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.043+0000 7f1b97955640 1 -- 172.21.15.106:0/4235445323 shutdown_connections 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.043+0000 7f1b97955640 1 -- 172.21.15.106:0/4235445323 wait complete. 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 Processor -- start 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 -- start start 2024-04-13T03:10:47.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9006f800 0x7f1b9012beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 0x7f1b901f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90070f50 con 0x7f1b900cdc10 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b900712c0 con 0x7f1b9006f800 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.044+0000 7f1b97955640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900d1190 con 0x7f1b90145f60 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44016/0 (socket says 172.21.15.106:44016) 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 -- 172.21.15.106:0/4037310071 learned_addr learned my addr 172.21.15.106:0/4037310071 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 0x7f1b901f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b956f1640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9006f800 0x7f1b9012beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 -- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 msgr2=0x7f1b901f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 0x7f1b901f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 -- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9006f800 msgr2=0x7f1b9012beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9006f800 0x7f1b9012beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 -- 172.21.15.106:0/4037310071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9012c890 con 0x7f1b900cdc10 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 0x7f1b901f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9b0 tx=0x7f1b8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b8dffb640 1 -- 172.21.15.106:0/4037310071 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84017070 con 0x7f1b900cdc10 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b8dffb640 1 -- 172.21.15.106:0/4037310071 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8400c9a0 con 0x7f1b900cdc10 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9012cb70 con 0x7f1b900cdc10 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b8dffb640 1 -- 172.21.15.106:0/4037310071 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400cd50 con 0x7f1b900cdc10 2024-04-13T03:10:47.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.045+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9013edd0 con 0x7f1b900cdc10 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.047+0000 7f1b8dffb640 1 -- 172.21.15.106:0/4037310071 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8401a070 con 0x7f1b900cdc10 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.047+0000 7f1b8dffb640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 0x7f1b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.047+0000 7f1b8dffb640 1 -- 172.21.15.106:0/4037310071 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1b8408c060 con 0x7f1b900cdc10 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.047+0000 7f1b956f1640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 0x7f1b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.048+0000 7f1b956f1640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 0x7f1b7c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b800012e0 tx=0x7f1b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.048+0000 7f1b9a818740 1 Processor -- start 2024-04-13T03:10:47.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.048+0000 7f1b9a818740 1 -- start start 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 0x561a2fc74b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561a2fbaf990 con 0x561a2fc452c0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561a2fb56620 con 0x561a2fc71de0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561a2fbab950 con 0x561a2fc721c0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44032/0 (socket says 172.21.15.106:44032) 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 -- 172.21.15.106:0/3969588299 learned_addr learned my addr 172.21.15.106:0/3969588299 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 0x561a2fc74b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b956f1640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 -- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 msgr2=0x561a2fc74b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 0x561a2fc74b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 -- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 msgr2=0x561a2fc44d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc44d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b95ef2640 1 -- 172.21.15.106:0/3969588299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2f8c22d0 con 0x561a2fc452c0 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.049+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 0x561a2fc74b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098c0 tx=0x7f1b88030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f159950c9bd057 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b76ffd640 1 -- 172.21.15.106:0/3969588299 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88031820 con 0x561a2fc452c0 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b76ffd640 1 -- 172.21.15.106:0/3969588299 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88037040 con 0x561a2fc452c0 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b76ffd640 1 -- 172.21.15.106:0/3969588299 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88031bc0 con 0x561a2fc452c0 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 -- 172.21.15.106:0/3969588299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 msgr2=0x561a2fc750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098c0 tx=0x7f1b88030000 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 -- 172.21.15.106:0/3969588299 shutdown_connections 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc721c0 0x561a2fc74b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc452c0 0x561a2fc750b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3969588299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc44d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 -- 172.21.15.106:0/3969588299 >> 172.21.15.106:0/3969588299 conn(0x561a2fc46ff0 msgr2=0x561a2fc8fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 -- 172.21.15.106:0/3969588299 shutdown_connections 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.050+0000 7f1b9a818740 1 -- 172.21.15.106:0/3969588299 wait complete. 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.051+0000 7f1b9a818740 1 -- start start 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.051+0000 7f1b9a818740 1 Processor -- start 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc6af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc69ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561a2fc49590 con 0x561a2fc721c0 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561a2fb699e0 con 0x561a2fc71de0 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561a2fc48980 con 0x561a2fc6d2f0 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b9a818740 10 client.0 _ll_register_callbacks cb 0x561a2fba5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44046/0 (socket says 172.21.15.106:44046) 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3345443971 learned_addr learned my addr 172.21.15.106:0/3345443971 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc69ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b956f1640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc6af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 msgr2=0x561a2fc69ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc69ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 msgr2=0x561a2fc6af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc6af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3345443971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2f98c6b0 con 0x561a2fc721c0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc69ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1b8408a2a0 tx=0x7f1b8405a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d396d6a35993b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.052+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84021040 con 0x561a2fc721c0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1b84058f30 con 0x561a2fc721c0 2024-04-13T03:10:47.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b88037040 con 0x561a2fc721c0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1b84059130 con 0x561a2fc721c0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 10 client.5583 fetch_fsmap learned FSMap version 16 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x561a2fc72d30 con 0x561a2fc721c0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88037040 con 0x561a2fc721c0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f1b84060090 con 0x561a2fc721c0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 10 client.5583 fetch_fsmap finished waiting for FSMap version 16 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 10 client.5583 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 20 client.5583 populate_metadata read hostname 'smithi106' 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc74000 0x561a2fca2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 4 client.5583 mds_command: new command op to 5059 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 --> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2fbbd3b0 con 0x561a2fc74000 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.053+0000 7f1b956f1640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc74000 0x561a2fca2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b6ffff640 10 client.5583 ms_handle_connect on v2:172.21.15.114:6833/1027593505 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b956f1640 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc74000 0x561a2fca2ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b8d247ef0a07fc server_cookie=86c6e1f33b1eff08 in_seq=0 out_seq=0 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b6ffff640 1 -- 172.21.15.106:0/3345443971 <== mds.0 v2:172.21.15.114:6833/1027593505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a2fbbd3b0 con 0x561a2fc74000 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b6ffff640 10 client.5583 handle_command_reply: tid=0 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 client.5583 shutdown 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 msgr2=0x561a2fc5b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1b8408a2a0 tx=0x7f1b8405a320 comp rx=0 tx=0).stop 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 shutdown_connections 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc74000 0x561a2fca2ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc69ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b6ffff640 0 client.5583 ms_handle_reset on v2:172.21.15.114:6833/1027593505 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc721c0 0x561a2fc5b5d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3345443971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc71de0 0x561a2fc6af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.054+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 >> 172.21.15.106:0/3345443971 conn(0x561a2fc452c0 msgr2=0x561a2fc91fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.055+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 shutdown_connections 2024-04-13T03:10:47.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.055+0000 7f1b9a818740 1 -- 172.21.15.106:0/3345443971 wait complete. 2024-04-13T03:10:47.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.055+0000 7f1b9a818740 20 client.5583 trim_cache size 0 max 16384 2024-04-13T03:10:47.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.072+0000 7f1b9a818740 1 Processor -- start 2024-04-13T03:10:47.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.072+0000 7f1b9a818740 1 -- start start 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc92210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901fe3d0 con 0x561a2fc71de0 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90141c40 con 0x561a2fc721c0 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b90140000 con 0x561a2fc6d2f0 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44048/0 (socket says 172.21.15.106:44048) 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3913985162 learned_addr learned my addr 172.21.15.106:0/3913985162 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b956f1640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc92210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 msgr2=0x561a2fc92210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc92210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 msgr2=0x561a2fc43c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc43c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.073+0000 7f1b94ef0640 1 -- 172.21.15.106:0/3913985162 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2fbbaee0 con 0x561a2fc71de0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e980 tx=0x7f1b84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6589dc8ce5b75682 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b767fc640 1 -- 172.21.15.106:0/3913985162 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8407e070 con 0x561a2fc71de0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b767fc640 1 -- 172.21.15.106:0/3913985162 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b84060090 con 0x561a2fc71de0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b767fc640 1 -- 172.21.15.106:0/3913985162 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8408a2d0 con 0x561a2fc71de0 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 -- 172.21.15.106:0/3913985162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 msgr2=0x561a2fc93470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e980 tx=0x7f1b84014000 comp rx=0 tx=0).stop 2024-04-13T03:10:47.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 -- 172.21.15.106:0/3913985162 shutdown_connections 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc71de0 0x561a2fc93470 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 --2- 172.21.15.106:0/3913985162 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc6d2f0 0x561a2fc92210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 -- 172.21.15.106:0/3913985162 >> 172.21.15.106:0/3913985162 conn(0x561a2fc452c0 msgr2=0x561a2fc6c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 -- 172.21.15.106:0/3913985162 shutdown_connections 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.074+0000 7f1b9a818740 1 -- 172.21.15.106:0/3913985162 wait complete. 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.075+0000 7f1b9a818740 1 -- start start 2024-04-13T03:10:47.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.075+0000 7f1b9a818740 1 Processor -- start 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc71de0 0x561a2fc441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561a2fc59240 con 0x561a2fc6d2f0 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561a2fc73010 con 0x561a2fc721c0 2024-04-13T03:10:47.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561a2fc78a50 con 0x561a2fc71de0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b9a818740 10 client.0 _ll_register_callbacks cb 0x7f1b84009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44056/0 (socket says 172.21.15.106:44056) 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 -- 172.21.15.106:0/665514130 learned_addr learned my addr 172.21.15.106:0/665514130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc71de0 0x561a2fc441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b95ef2640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 -- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc71de0 msgr2=0x561a2fc441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc71de0 0x561a2fc441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 -- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 msgr2=0x561a2fc59e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc59e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 -- 172.21.15.106:0/665514130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2fc68570 con 0x561a2fc6d2f0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b956f1640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x561a2fc73b00 tx=0x7f1b80000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7623f8ee8f924b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:10:47.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b800070d0 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.076+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1b80007270 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b84060090 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1b80034050 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 10 client.5589 fetch_fsmap learned FSMap version 16 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x561a2fc68780 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b84060090 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f1b80037020 con 0x561a2fc6d2f0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 10 client.5589 fetch_fsmap finished waiting for FSMap version 16 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 10 client.5589 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 20 client.5589 populate_metadata read hostname 'smithi106' 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc49fd0 0x561a2fc4a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 4 client.5589 mds_command: new command op to 5059 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 --> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] -- command(tid 0: {"prefix": "status"}) -- 0x561a2fc6c450 con 0x561a2fc49fd0 2024-04-13T03:10:47.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.077+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc49fd0 0x561a2fc4a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b94ef0640 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc49fd0 0x561a2fc4a3b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1badfa2807db79 server_cookie=7b7337bf24d6e2bb in_seq=0 out_seq=0 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b6ffff640 10 client.5589 ms_handle_connect on v2:172.21.15.114:6833/1027593505 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b6ffff640 1 -- 172.21.15.106:0/665514130 <== mds.0 v2:172.21.15.114:6833/1027593505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561a2fc6c450 con 0x561a2fc49fd0 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b6ffff640 10 client.5589 handle_command_reply: tid=0 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 client.5589 shutdown 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 msgr2=0x561a2fc43c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x561a2fc73b00 tx=0x7f1b80000f40 comp rx=0 tx=0).stop 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 shutdown_connections 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:6833/1027593505,v1:172.21.15.114:6835/1027593505] conn(0x561a2fc49fd0 0x561a2fc4a3b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561a2fc71de0 0x561a2fc441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a2fc721c0 0x561a2fc59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b6ffff640 0 client.5589 ms_handle_reset on v2:172.21.15.114:6833/1027593505 2024-04-13T03:10:47.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 --2- 172.21.15.106:0/665514130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561a2fc6d2f0 0x561a2fc43c90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 >> 172.21.15.106:0/665514130 conn(0x561a2fc452c0 msgr2=0x561a2fc93470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 shutdown_connections 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 1 -- 172.21.15.106:0/665514130 wait complete. 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.078+0000 7f1b9a818740 20 client.5589 trim_cache size 0 max 16384 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5059, 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:10:47.087 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 9.0117720349999999, 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 16, 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 39, 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 39, 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 16.164575978999999 2024-04-13T03:10:47.088 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 msgr2=0x7f1b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 0x7f1b7c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b800012e0 tx=0x7f1b80009040 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 msgr2=0x7f1b901f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9b0 tx=0x7f1b8400ee70 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 shutdown_connections 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b7c06c8f0 0x7f1b7c06edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b90145f60 0x7f1b901f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b900cdc10 0x7f1b901f6040 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 --2- 172.21.15.106:0/4037310071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9006f800 0x7f1b9012beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 >> 172.21.15.106:0/4037310071 conn(0x7f1b900d1b60 msgr2=0x7f1b90160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 shutdown_connections 2024-04-13T03:10:47.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.079+0000 7f1b97955640 1 -- 172.21.15.106:0/4037310071 wait complete. 2024-04-13T03:10:47.091 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:10:47.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.190+0000 7f7de43e9640 1 Processor -- start 2024-04-13T03:10:47.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.190+0000 7f7de43e9640 1 -- start start 2024-04-13T03:10:47.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de43e9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7ddc162550 0x7f7ddc0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de43e9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de43e9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ddc0cfe70 con 0x7f7ddc15ea80 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de43e9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7ddc0d1f80 con 0x7f7ddc0d1630 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de43e9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7ddc15ee60 con 0x7f7ddc162550 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de2986640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de2986640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44062/0 (socket says 172.21.15.106:44062) 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.191+0000 7f7de2986640 1 -- 172.21.15.106:0/863942750 learned_addr learned my addr 172.21.15.106:0/863942750 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de1984640 1 --2- 172.21.15.106:0/863942750 >> v2:172.21.15.114:3300/0 conn(0x7f7ddc162550 0x7f7ddc0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de2986640 1 -- 172.21.15.106:0/863942750 >> v2:172.21.15.114:3300/0 conn(0x7f7ddc162550 msgr2=0x7f7ddc0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de2986640 1 --2- 172.21.15.106:0/863942750 >> v2:172.21.15.114:3300/0 conn(0x7f7ddc162550 0x7f7ddc0d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de2986640 1 -- 172.21.15.106:0/863942750 >> v1:172.21.15.114:6789/0 conn(0x7f7ddc15ea80 legacy=0x7f7ddc0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:47.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de2986640 1 -- 172.21.15.106:0/863942750 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc162af0 con 0x7f7ddc0d1630 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de2986640 1 --2- 172.21.15.106:0/863942750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc073a30 tx=0x7f7dd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df4d0ba0f943634 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de1183640 1 -- 172.21.15.106:0/863942750 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4035020 con 0x7f7ddc0d1630 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de1183640 1 -- 172.21.15.106:0/863942750 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd4031870 con 0x7f7ddc0d1630 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.192+0000 7f7de1183640 1 -- 172.21.15.106:0/863942750 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4031c40 con 0x7f7ddc0d1630 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 -- 172.21.15.106:0/863942750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 msgr2=0x7f7ddc0d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 --2- 172.21.15.106:0/863942750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc073a30 tx=0x7f7dd402b910 comp rx=0 tx=0).stop 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 -- 172.21.15.106:0/863942750 shutdown_connections 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 --2- 172.21.15.106:0/863942750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc0d1a40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 --2- 172.21.15.106:0/863942750 >> v2:172.21.15.114:3300/0 conn(0x7f7ddc162550 0x7f7ddc0d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 -- 172.21.15.106:0/863942750 >> 172.21.15.106:0/863942750 conn(0x7f7ddc09da50 msgr2=0x7f7ddc0b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 -- 172.21.15.106:0/863942750 shutdown_connections 2024-04-13T03:10:47.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.193+0000 7f7de43e9640 1 -- 172.21.15.106:0/863942750 wait complete. 2024-04-13T03:10:47.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.194+0000 7f7de43e9640 1 Processor -- start 2024-04-13T03:10:47.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.194+0000 7f7de43e9640 1 -- start start 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ddc15ea80 0x7f7ddc15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ddc162550 0x7f7ddc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc163390 con 0x7f7ddc0d1630 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc1630b0 con 0x7f7ddc162550 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de43e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ddc0d0320 con 0x7f7ddc15ea80 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44068/0 (socket says 172.21.15.106:44068) 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 -- 172.21.15.106:0/288273677 learned_addr learned my addr 172.21.15.106:0/288273677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de1984640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ddc15ea80 0x7f7ddc15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2986640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ddc162550 0x7f7ddc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 -- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ddc15ea80 msgr2=0x7f7ddc15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ddc15ea80 0x7f7ddc15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 -- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ddc162550 msgr2=0x7f7ddc06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ddc162550 0x7f7ddc06d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 -- 172.21.15.106:0/288273677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc06e820 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.195+0000 7f7de2185640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c7a0 tx=0x7f7dd800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.196+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8017070 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.196+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd8003da0 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.196+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc0c8d50 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.196+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80052b0 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.196+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc0c92d0 con 0x7f7ddc0d1630 2024-04-13T03:10:47.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.197+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc068a10 con 0x7f7ddc0d1630 2024-04-13T03:10:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.197+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dd801a070 con 0x7f7ddc0d1630 2024-04-13T03:10:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.198+0000 7f7dd2ffd640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 0x7f7dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.198+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7dd808b2b0 con 0x7f7ddc0d1630 2024-04-13T03:10:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.198+0000 7f7de1984640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 0x7f7dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.198+0000 7f7de1984640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 0x7f7dc006edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc15df70 tx=0x7f7dcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.202+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7dd8058f90 con 0x7f7ddc0d1630 2024-04-13T03:10:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.414+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ddc0d5350 con 0x7f7ddc0d1630 2024-04-13T03:10:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.415+0000 7f7dd2ffd640 1 -- 172.21.15.106:0/288273677 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f7dd805d290 con 0x7f7ddc0d1630 2024-04-13T03:10:47.417 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:10:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 msgr2=0x7f7dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 0x7f7dc006edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc15df70 tx=0x7f7dcc008040 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 msgr2=0x7f7ddc15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c7a0 tx=0x7f7dd800cc60 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 shutdown_connections 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7dc006c8f0 0x7f7dc006edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ddc15ea80 0x7f7ddc15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ddc162550 0x7f7ddc06d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 --2- 172.21.15.106:0/288273677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ddc0d1630 0x7f7ddc15d250 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.416+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 >> 172.21.15.106:0/288273677 conn(0x7f7ddc09da50 msgr2=0x7f7ddc0b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.417+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 shutdown_connections 2024-04-13T03:10:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47.417+0000 7f7de43e9640 1 -- 172.21.15.106:0/288273677 wait complete. 2024-04-13T03:10:47.429 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:10:47.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:10:47.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:10:47.429 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:10:47.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:10:47.430 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:10:47.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:10:47.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:10:47.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:10:47.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.430 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.445 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:10:47.446 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:10:47.447 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:10:47.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.448 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:10:47.512 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:10:47.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.513 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:10:47.570 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:10:47.570 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:10:47.570 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:10:47.570 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:10:47.570 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:10:47.571 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:10:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:10:47.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:47.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.704 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:10:47.719 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:10:47.719 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:10:47.719 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:10:47.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:10:47.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:47.785 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:10:47.839 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:10:47.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:10:47.840 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:10:47.893 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:10:47.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:47.977 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:47.977 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:10:47.989 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:10:47.990 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:10:47.990 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:10:47.990 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:10:47.991 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:10:47.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:47.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:10:48.018 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:10:48.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:10:48.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:48.121 INFO:teuthology.orchestra.run.smithi106.stdout:client5206 2024-04-13T03:10:48.121 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:48.121 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:48.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.137 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-13T03:10:48.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.198 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-13T03:10:48.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.254 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-13T03:10:48.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.311 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-13T03:10:48.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.369 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-13T03:10:48.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:48.426 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-13T03:10:48.486 INFO:teuthology.orchestra.run.smithi106.stdout:./ 2024-04-13T03:10:48.486 INFO:teuthology.orchestra.run.smithi106.stdout:./rfile 2024-04-13T03:10:48.486 INFO:teuthology.orchestra.run.smithi106.stdout:./parentdir 2024-04-13T03:10:48.487 INFO:teuthology.orchestra.run.smithi106.stdout:./parentdir/pfile 2024-04-13T03:10:48.487 INFO:teuthology.orchestra.run.smithi106.stdout:./parentdir/childdir 2024-04-13T03:10:48.487 INFO:teuthology.orchestra.run.smithi106.stdout:./parentdir/childdir/cfile 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-13T03:10:48.487 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:48.591 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:10:48.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:48.600 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:48.600 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:48.601 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:48.601 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:48.601 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-13T03:10:48.601 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:48.705 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2024-04-13T03:10:48.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:48.714 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:48.714 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:48.715 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:48.715 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:48.715 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-13T03:10:48.715 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:48.820 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:10:48.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-13T03:10:48.829 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:48.933 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628279 2024-04-13T03:10:48.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-13T03:10:48.942 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:49.047 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:10:49.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-13T03:10:49.059 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:10:49.164 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628280 2024-04-13T03:10:49.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:10:49.172 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:49.173 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:49.228 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:10:49.228 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:10:49.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:10:49.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.423+0000 7f9c0097b640 1 Processor -- start 2024-04-13T03:10:49.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.423+0000 7f9c0097b640 1 -- start start 2024-04-13T03:10:49.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9c0097b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9c0097b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9c0097b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc0b6120 con 0x7f9bfc0ceb70 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9c0097b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9bfc0cde50 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9c0097b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc0cd240 con 0x7f9bfc15ec20 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9bfad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9bfad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44074/0 (socket says 172.21.15.106:44074) 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.424+0000 7f9bfad9d640 1 -- 172.21.15.106:0/3263255895 learned_addr learned my addr 172.21.15.106:0/3263255895 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfa59c640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfad9d640 1 -- 172.21.15.106:0/3263255895 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 msgr2=0x7f9bfc0b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfad9d640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc0b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfad9d640 1 -- 172.21.15.106:0/3263255895 >> v1:172.21.15.114:6789/0 conn(0x7f9bfc0ceb70 legacy=0x7f9bfc0b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfad9d640 1 -- 172.21.15.106:0/3263255895 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc0cd520 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.425+0000 7f9bfad9d640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b808228f5c0c35f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9bf959a640 1 -- 172.21.15.106:0/3263255895 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4034070 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9bf959a640 1 -- 172.21.15.106:0/3263255895 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4030840 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9bf959a640 1 -- 172.21.15.106:0/3263255895 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4030c10 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 -- 172.21.15.106:0/3263255895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 msgr2=0x7f9bfc0b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf402b980 comp rx=0 tx=0).stop 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 -- 172.21.15.106:0/3263255895 shutdown_connections 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc0b5900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 --2- 172.21.15.106:0/3263255895 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 -- 172.21.15.106:0/3263255895 >> 172.21.15.106:0/3263255895 conn(0x7f9bfc0d2530 msgr2=0x7f9bfc0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 -- 172.21.15.106:0/3263255895 shutdown_connections 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.426+0000 7f9c0097b640 1 -- 172.21.15.106:0/3263255895 wait complete. 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 Processor -- start 2024-04-13T03:10:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 -- start start 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 0x7f9bfc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0cced0 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0cdae0 con 0x7f9bfc161240 2024-04-13T03:10:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9c0097b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc1626b0 con 0x7f9bfc15ec20 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44084/0 (socket says 172.21.15.106:44084) 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.428+0000 7f9bfa59c640 1 -- 172.21.15.106:0/2483816509 learned_addr learned my addr 172.21.15.106:0/2483816509 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 -- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 msgr2=0x7f9bfc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfad9d640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 0x7f9bfc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bf9d9b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 -- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 msgr2=0x7f9bfc156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 0x7f9bfc156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 -- 172.21.15.106:0/2483816509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc0c9850 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfad9d640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 0x7f9bfc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bf9d9b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9bfa59c640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007e40 tx=0x7f9bec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec014620 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec019070 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec014a00 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc0c9b30 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.429+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc156d80 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.430+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bfc0c95e0 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.431+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bec014ba0 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.431+0000 7f9beaffd640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 0x7f9bdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.431+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9bec08c580 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.431+0000 7f9bf9d9b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 0x7f9bdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.431+0000 7f9bf9d9b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 0x7f9bdc06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0c5f60 tx=0x7f9bf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.435+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bec05a820 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.610+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bfc068a10 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.612+0000 7f9beaffd640 1 -- 172.21.15.106:0/2483816509 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7f9bec05a120 con 0x7f9bfc0d14d0 2024-04-13T03:10:49.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:49.613 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:39.078056+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 msgr2=0x7f9bdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 0x7f9bdc06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0c5f60 tx=0x7f9bf0009040 comp rx=0 tx=0).stop 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 msgr2=0x7f9bfc160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007e40 tx=0x7f9bec00e4b0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 shutdown_connections 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bdc06c8f0 0x7f9bdc06edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bfc15ec20 0x7f9bfc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bfc161240 0x7f9bfc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.613+0000 7f9c0097b640 1 --2- 172.21.15.106:0/2483816509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bfc0d14d0 0x7f9bfc160d00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.614+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 >> 172.21.15.106:0/2483816509 conn(0x7f9bfc0d2530 msgr2=0x7f9bfc0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.614+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 shutdown_connections 2024-04-13T03:10:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.614+0000 7f9c0097b640 1 -- 172.21.15.106:0/2483816509 wait complete. 2024-04-13T03:10:49.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:10:35.635640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:10:36.371611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:10:49.626 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:10:49.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.727+0000 7f1c2bde9640 1 Processor -- start 2024-04-13T03:10:49.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.727+0000 7f1c2bde9640 1 -- start start 2024-04-13T03:10:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.728+0000 7f1c2bde9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.728+0000 7f1c2bde9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c240cf2f0 0x7f1c240cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.728+0000 7f1c2bde9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2415fe40 con 0x7f1c240704f0 2024-04-13T03:10:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.728+0000 7f1c2bde9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1c24162f40 con 0x7f1c240cf2f0 2024-04-13T03:10:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.728+0000 7f1c2bde9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c241633d0 con 0x7f1c2415f080 2024-04-13T03:10:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c240cf2f0 0x7f1c240cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c29384640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c29384640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:59318/0 (socket says 172.21.15.106:59318) 2024-04-13T03:10:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c240cf2f0 0x7f1c240cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:44828/0 (socket says 172.21.15.106:44828) 2024-04-13T03:10:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c29384640 1 -- 172.21.15.106:0/1266160427 learned_addr learned my addr 172.21.15.106:0/1266160427 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c29b85640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1c240704f0 0x7f1c240708d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:38346/0 (socket says 172.21.15.106:38346) 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 -- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 msgr2=0x7f1c2415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 -- 172.21.15.106:0/1266160427 >> v1:172.21.15.114:6789/0 conn(0x7f1c240704f0 legacy=0x7f1c240708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c2a386640 1 -- 172.21.15.106:0/1266160427 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c24162610 con 0x7f1c240cf2f0 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c28b83640 1 -- 172.21.15.106:0/1266160427 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785452782 0 0) 0x7f1c2415fe40 con 0x7f1c240704f0 2024-04-13T03:10:49.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.729+0000 7f1c29384640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c2a386640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240cf2f0 0x7f1c240cf780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097a0 tx=0x7f1c1c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83908d670062241 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c28b83640 1 -- 172.21.15.106:0/1266160427 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c035020 con 0x7f1c240cf2f0 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c28b83640 1 -- 172.21.15.106:0/1266160427 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c031830 con 0x7f1c240cf2f0 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c28b83640 1 -- 172.21.15.106:0/1266160427 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c031c00 con 0x7f1c240cf2f0 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240cf2f0 msgr2=0x7f1c240cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.730+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240cf2f0 0x7f1c240cf780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097a0 tx=0x7f1c1c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1266160427 shutdown_connections 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240cf2f0 0x7f1c240cf780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1266160427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1266160427 >> 172.21.15.106:0/1266160427 conn(0x7f1c240d13f0 msgr2=0x7f1c240d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1266160427 shutdown_connections 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.731+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1266160427 wait complete. 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.732+0000 7f1c2bde9640 1 Processor -- start 2024-04-13T03:10:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.732+0000 7f1c2bde9640 1 -- start start 2024-04-13T03:10:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240704f0 0x7f1c24156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2406ead0 con 0x7f1c240cf2f0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24162bd0 con 0x7f1c240704f0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2415fa20 con 0x7f1c2415f080 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44098/0 (socket says 172.21.15.106:44098) 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 -- 172.21.15.106:0/1751189894 learned_addr learned my addr 172.21.15.106:0/1751189894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2a386640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29b85640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240704f0 0x7f1c24156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 -- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 msgr2=0x7f1c2414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 -- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240704f0 msgr2=0x7f1c24156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240704f0 0x7f1c24156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 -- 172.21.15.106:0/1751189894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c240aee20 con 0x7f1c240cf2f0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c29384640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9d0 tx=0x7f1c1400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14017070 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1400ca60 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c240af100 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400ce10 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.733+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c240af6c0 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.734+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c24069120 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.735+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c1401a070 con 0x7f1c240cf2f0 2024-04-13T03:10:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.735+0000 7f1c1a7fc640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 0x7f1c0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.735+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1c1408bee0 con 0x7f1c240cf2f0 2024-04-13T03:10:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.735+0000 7f1c29b85640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 0x7f1c0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.735+0000 7f1c29b85640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 0x7f1c0806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c200012e0 tx=0x7f1c20001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:49.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.739+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c14059bc0 con 0x7f1c240cf2f0 2024-04-13T03:10:49.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.914+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c2415ed60 con 0x7f1c240cf2f0 2024-04-13T03:10:49.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.914+0000 7f1c1a7fc640 1 -- 172.21.15.106:0/1751189894 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f1c1405dec0 con 0x7f1c240cf2f0 2024-04-13T03:10:49.916 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 msgr2=0x7f1c0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 0x7f1c0806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c200012e0 tx=0x7f1c20001630 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 msgr2=0x7f1c240afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9d0 tx=0x7f1c1400ee90 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 shutdown_connections 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c0806c8f0 0x7f1c0806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c2415f080 0x7f1c2414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c240cf2f0 0x7f1c240afec0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 --2- 172.21.15.106:0/1751189894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c240704f0 0x7f1c24156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 >> 172.21.15.106:0/1751189894 conn(0x7f1c240d13f0 msgr2=0x7f1c241617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 shutdown_connections 2024-04-13T03:10:49.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:49.916+0000 7f1c2bde9640 1 -- 172.21.15.106:0/1751189894 wait complete. 2024-04-13T03:10:49.928 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:10:50.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.031+0000 7f0331911640 1 Processor -- start 2024-04-13T03:10:50.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.031+0000 7f0331911640 1 -- start start 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0331911640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f032c15c680 0x7f032c15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0331911640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f032c0d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0331911640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c0d2d90 con 0x7f032c0d2830 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0331911640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0d3070 con 0x7f032c15c680 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0331911640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f032c15f0d0 con 0x7f032c0d23b0 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f032c0d2830 0x7f032c1624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43086/0 (socket says 172.21.15.106:43086) 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032b7fe640 1 -- 172.21.15.106:0/3965783723 learned_addr learned my addr 172.21.15.106:0/3965783723 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f0323fff640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f032c0d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061678398 0 0) 0x7f032c0d2d90 con 0x7f032c0d2830 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032affd640 1 --2- 172.21.15.106:0/3965783723 >> v2:172.21.15.114:3300/0 conn(0x7f032c15c680 0x7f032c15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0318003520 con 0x7f032c0d2830 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2580572004 0 0) 0x7f0318003520 con 0x7f032c0d2830 2024-04-13T03:10:50.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 msgr2=0x7f032c0d4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f032c0d4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 >> v2:172.21.15.114:3300/0 conn(0x7f032c15c680 msgr2=0x7f032c15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 --2- 172.21.15.106:0/3965783723 >> v2:172.21.15.114:3300/0 conn(0x7f032c15c680 0x7f032c15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c15f3b0 con 0x7f032c0d2830 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.032+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f03240019f0 con 0x7f032c0d2830 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 >> v1:172.21.15.106:6789/0 conn(0x7f032c0d2830 legacy=0x7f032c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f03180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0318003f30 0x7f0318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0318004d40 0x7f03180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0d3070 con 0x7f0318004d40 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f032c15f0d0 con 0x7f0318003f30 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f031800b9c0 con 0x7f032c0d23b0 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0318004d40 0x7f03180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032affd640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f03180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f0323fff640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0318003f30 0x7f0318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 -- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 msgr2=0x7f03180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c0d23b0 0x7f03180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 -- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0318003f30 msgr2=0x7f0318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0318003f30 0x7f0318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.033+0000 7f032b7fe640 1 -- 172.21.15.106:0/3965783723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0318003520 con 0x7f0318004d40 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f032b7fe640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0318004d40 0x7f03180051c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f03240013d0 tx=0x7f0318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5197d0fc478ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032400a4e0 con 0x7f0318004d40 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0324034650 con 0x7f0318004d40 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f032a7fc640 1 -- 172.21.15.106:0/3965783723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324034a40 con 0x7f0318004d40 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 -- 172.21.15.106:0/3965783723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0318004d40 msgr2=0x7f03180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 --2- 172.21.15.106:0/3965783723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0318004d40 0x7f03180051c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f03240013d0 tx=0x7f0318005830 comp rx=0 tx=0).stop 2024-04-13T03:10:50.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f032b7fe640 1 -- 172.21.15.106:0/3965783723 reap_dead start 2024-04-13T03:10:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 -- 172.21.15.106:0/3965783723 shutdown_connections 2024-04-13T03:10:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 -- 172.21.15.106:0/3965783723 >> 172.21.15.106:0/3965783723 conn(0x7f032c0b97a0 msgr2=0x7f032c0b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 -- 172.21.15.106:0/3965783723 shutdown_connections 2024-04-13T03:10:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.034+0000 7f0331911640 1 -- 172.21.15.106:0/3965783723 wait complete. 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 Processor -- start 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 -- start start 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c14b570 0x7f032c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 0x7f032c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0b3b60 con 0x7f032c0c98a0 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0b39e0 con 0x7f032c0b4780 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f0331911640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f032c162bc0 con 0x7f032c14b570 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44114/0 (socket says 172.21.15.106:44114) 2024-04-13T03:10:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.036+0000 7f032b7fe640 1 -- 172.21.15.106:0/1467908688 learned_addr learned my addr 172.21.15.106:0/1467908688 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032affd640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c14b570 0x7f032c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0323fff640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 0x7f032c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 -- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c14b570 msgr2=0x7f032c0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c14b570 0x7f032c0b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 -- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 msgr2=0x7f032c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 0x7f032c0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 -- 172.21.15.106:0/1467908688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0c9d20 con 0x7f032c0c98a0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0323fff640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 0x7f032c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f032b7fe640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f03240013d0 tx=0x7f0324001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324039040 con 0x7f032c0c98a0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0324043070 con 0x7f032c0c98a0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c0c9fb0 con 0x7f032c0c98a0 2024-04-13T03:10:50.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324034a20 con 0x7f032c0c98a0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.037+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c0ca570 con 0x7f032c0c98a0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.038+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c0d2d90 con 0x7f032c0c98a0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.038+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0324034bc0 con 0x7f032c0c98a0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.039+0000 7f0328ff9640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 0x7f030c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.039+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f03240ada90 con 0x7f032c0c98a0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.039+0000 7f032affd640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 0x7f030c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.039+0000 7f032affd640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 0x7f030c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f031c009b80 tx=0x7f031c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.043+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f032c0d2d90 con 0x7f032c0c98a0 2024-04-13T03:10:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.220+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f032c068a10 con 0x7f032c0c98a0 2024-04-13T03:10:50.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.220+0000 7f0328ff9640 1 -- 172.21.15.106:0/1467908688 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7f03240019b0 con 0x7f032c0c98a0 2024-04-13T03:10:50.222 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 msgr2=0x7f030c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 0x7f030c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f031c009b80 tx=0x7f031c008040 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 msgr2=0x7f032c0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f03240013d0 tx=0x7f0324001360 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 shutdown_connections 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f030c06c8f0 0x7f030c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f032c0c98a0 0x7f032c0b6180 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f032c0b4780 0x7f032c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 --2- 172.21.15.106:0/1467908688 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f032c14b570 0x7f032c0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 >> 172.21.15.106:0/1467908688 conn(0x7f032c0b97a0 msgr2=0x7f032c0d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.222+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 shutdown_connections 2024-04-13T03:10:50.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.223+0000 7f0331911640 1 -- 172.21.15.106:0/1467908688 wait complete. 2024-04-13T03:10:50.234 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:10:50.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.336+0000 7f4a8b5e0640 1 Processor -- start 2024-04-13T03:10:50.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.337+0000 7f4a8b5e0640 1 -- start start 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.337+0000 7f4a8b5e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a84070bb0 0x7f4a841697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.337+0000 7f4a8b5e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a840b5cf0 con 0x7f4a840b1c30 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.337+0000 7f4a8b5e0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a840b5fd0 con 0x7f4a8415ee40 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.337+0000 7f4a8b5e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4a840b62b0 con 0x7f4a84070bb0 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a89b7d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a84070bb0 0x7f4a841697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a8937c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4a840b1c30 0x7f4a840b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43096/0 (socket says 172.21.15.106:43096) 2024-04-13T03:10:50.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a8937c640 1 -- 172.21.15.106:0/3581706897 learned_addr learned my addr 172.21.15.106:0/3581706897 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a89b7d640 1 -- 172.21.15.106:0/3581706897 >> v1:172.21.15.114:6789/0 conn(0x7f4a8415ee40 legacy=0x7f4a840704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a89b7d640 1 -- 172.21.15.106:0/3581706897 >> v1:172.21.15.106:6789/0 conn(0x7f4a840b1c30 legacy=0x7f4a840b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a7bfff640 1 -- 172.21.15.106:0/3581706897 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871397590 0 0) 0x7f4a840b5cf0 con 0x7f4a840b1c30 2024-04-13T03:10:50.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.338+0000 7f4a89b7d640 1 -- 172.21.15.106:0/3581706897 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840b6590 con 0x7f4a84070bb0 2024-04-13T03:10:50.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a89b7d640 1 --2- 172.21.15.106:0/3581706897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a841697c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009d90 tx=0x7f4a7c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba62ce087000b455 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a7bfff640 1 -- 172.21.15.106:0/3581706897 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c039070 con 0x7f4a84070bb0 2024-04-13T03:10:50.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a7bfff640 1 -- 172.21.15.106:0/3581706897 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c02cd00 con 0x7f4a84070bb0 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a7bfff640 1 -- 172.21.15.106:0/3581706897 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c034660 con 0x7f4a84070bb0 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3581706897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 msgr2=0x7f4a841697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3581706897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a841697c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009d90 tx=0x7f4a7c02bf50 comp rx=0 tx=0).stop 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3581706897 shutdown_connections 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3581706897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a841697c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3581706897 >> 172.21.15.106:0/3581706897 conn(0x7f4a84071870 msgr2=0x7f4a84071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3581706897 shutdown_connections 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.339+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3581706897 wait complete. 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 Processor -- start 2024-04-13T03:10:50.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 -- start start 2024-04-13T03:10:50.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a840b1c30 0x7f4a8415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8415ee40 0x7f4a840d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84169d00 con 0x7f4a84070bb0 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8406e880 con 0x7f4a8415ee40 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8b5e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4a84169e80 con 0x7f4a840b1c30 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8937c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8937c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44136/0 (socket says 172.21.15.106:44136) 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.341+0000 7f4a8937c640 1 -- 172.21.15.106:0/3493140793 learned_addr learned my addr 172.21.15.106:0/3493140793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 -- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a840b1c30 msgr2=0x7f4a8415b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a89b7d640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8415ee40 0x7f4a840d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a88b7b640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a840b1c30 0x7f4a8415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a840b1c30 0x7f4a8415b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 -- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8415ee40 msgr2=0x7f4a840d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8415ee40 0x7f4a840d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 -- 172.21.15.106:0/3493140793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840d1b20 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8937c640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c4a0 tx=0x7f4a7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74018070 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a740074e0 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a84160110 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74004a20 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.342+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8415b860 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.343+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a840ce2d0 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.343+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a7401b070 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.344+0000 7f4a79ffb640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.344+0000 7f4a88b7b640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.344+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4a7408b040 con 0x7f4a84070bb0 2024-04-13T03:10:50.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.344+0000 7f4a88b7b640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 0x7f4a6806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b3d40 tx=0x7f4a80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.348+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a740592e0 con 0x7f4a84070bb0 2024-04-13T03:10:50.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.524+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a84068a10 con 0x7f4a84070bb0 2024-04-13T03:10:50.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.524+0000 7f4a79ffb640 1 -- 172.21.15.106:0/3493140793 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f4a74058be0 con 0x7f4a84070bb0 2024-04-13T03:10:50.525 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:10:50.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 msgr2=0x7f4a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 0x7f4a6806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b3d40 tx=0x7f4a80009040 comp rx=0 tx=0).stop 2024-04-13T03:10:50.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 msgr2=0x7f4a840cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c4a0 tx=0x7f4a7400c960 comp rx=0 tx=0).stop 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 shutdown_connections 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a840b1c30 0x7f4a8415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8415ee40 0x7f4a840d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 --2- 172.21.15.106:0/3493140793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a84070bb0 0x7f4a840cb820 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 >> 172.21.15.106:0/3493140793 conn(0x7f4a84071870 msgr2=0x7f4a84071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 shutdown_connections 2024-04-13T03:10:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.526+0000 7f4a8b5e0640 1 -- 172.21.15.106:0/3493140793 wait complete. 2024-04-13T03:10:50.538 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:10:50.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.639+0000 7f629b060640 1 Processor -- start 2024-04-13T03:10:50.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.639+0000 7f629b060640 1 -- start start 2024-04-13T03:10:50.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f629b060640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f629415ee40 0x7f62940704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f629b060640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940709e0 0x7f629406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f629b060640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62940b5cf0 con 0x7f62940b1c30 2024-04-13T03:10:50.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f629b060640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f62940b5fd0 con 0x7f629415ee40 2024-04-13T03:10:50.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f629b060640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f62940b62b0 con 0x7f62940709e0 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f629415ee40 0x7f62940704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f629415ee40 0x7f62940704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44148/0 (socket says 172.21.15.106:44148) 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 -- 172.21.15.106:0/2542100129 learned_addr learned my addr 172.21.15.106:0/2542100129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f62995fd640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940709e0 0x7f629406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 -- 172.21.15.106:0/2542100129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940709e0 msgr2=0x7f629406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940709e0 0x7f629406d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 -- 172.21.15.106:0/2542100129 >> v1:172.21.15.106:6789/0 conn(0x7f62940b1c30 legacy=0x7f62940b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f6293fff640 1 -- 172.21.15.106:0/2542100129 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62940b6590 con 0x7f629415ee40 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.640+0000 7f62937fe640 1 -- 172.21.15.106:0/2542100129 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2739939622 0 0) 0x7f62940b5cf0 con 0x7f62940b1c30 2024-04-13T03:10:50.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f6293fff640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f62940704a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6280009870 tx=0x7f628002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eec07feb85334ad server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f62937fe640 1 -- 172.21.15.106:0/2542100129 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280034070 con 0x7f629415ee40 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f62937fe640 1 -- 172.21.15.106:0/2542100129 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6280030840 con 0x7f629415ee40 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f62937fe640 1 -- 172.21.15.106:0/2542100129 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280030c10 con 0x7f629415ee40 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f629b060640 1 -- 172.21.15.106:0/2542100129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 msgr2=0x7f62940704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.641+0000 7f629b060640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f62940704a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6280009870 tx=0x7f628002b980 comp rx=0 tx=0).stop 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 -- 172.21.15.106:0/2542100129 shutdown_connections 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940709e0 0x7f629406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 --2- 172.21.15.106:0/2542100129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f62940704a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 -- 172.21.15.106:0/2542100129 >> 172.21.15.106:0/2542100129 conn(0x7f6294071870 msgr2=0x7f6294071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 -- 172.21.15.106:0/2542100129 shutdown_connections 2024-04-13T03:10:50.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.642+0000 7f629b060640 1 -- 172.21.15.106:0/2542100129 wait complete. 2024-04-13T03:10:50.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.643+0000 7f629b060640 1 Processor -- start 2024-04-13T03:10:50.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.643+0000 7f629b060640 1 -- start start 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.643+0000 7f629b060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 0x7f62940b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.643+0000 7f629b060640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 0x7f6294159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f629406df30 con 0x7f629415ee40 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f629406ddb0 con 0x7f62940709e0 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f629406e0b0 con 0x7f62940b1c30 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44152/0 (socket says 172.21.15.106:44152) 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 -- 172.21.15.106:0/3679545699 learned_addr learned my addr 172.21.15.106:0/3679545699 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f6293fff640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 0x7f6294159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f6298dfc640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 0x7f62940b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 -- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 msgr2=0x7f6294159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 0x7f6294159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 -- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 msgr2=0x7f62940b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 0x7f62940b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 -- 172.21.15.106:0/3679545699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62940cc910 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f6293fff640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 0x7f6294159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f6298dfc640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 0x7f62940b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62995fd640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f628c007e40 tx=0x7f628c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c014620 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c019070 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62940ccbf0 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c014a20 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.644+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62941f5340 con 0x7f629415ee40 2024-04-13T03:10:50.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.645+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6294068a10 con 0x7f629415ee40 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.646+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f628c014bc0 con 0x7f629415ee40 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.646+0000 7f62917fa640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 0x7f627806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.646+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f628c08c540 con 0x7f629415ee40 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.646+0000 7f6298dfc640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 0x7f627806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.646+0000 7f6298dfc640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 0x7f627806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6288000f60 tx=0x7f6288009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:50.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.650+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f628c05a870 con 0x7f629415ee40 2024-04-13T03:10:50.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.839+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62940711f0 con 0x7f629415ee40 2024-04-13T03:10:50.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.839+0000 7f62917fa640 1 -- 172.21.15.106:0/3679545699 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) ==== 80+0+0 (secure 0 0 0) 0x7f628c05a170 con 0x7f629415ee40 2024-04-13T03:10:50.840 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 msgr2=0x7f627806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 0x7f627806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6288000f60 tx=0x7f6288009040 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 msgr2=0x7f629415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f628c007e40 tx=0x7f628c00e4b0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 shutdown_connections 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f627806c8f0 0x7f627806edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f62940b1c30 0x7f6294159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f629415ee40 0x7f629415a1e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 --2- 172.21.15.106:0/3679545699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62940709e0 0x7f62940b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.841+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 >> 172.21.15.106:0/3679545699 conn(0x7f6294071870 msgr2=0x7f62940d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.842+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 shutdown_connections 2024-04-13T03:10:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:50.842+0000 7f629b060640 1 -- 172.21.15.106:0/3679545699 wait complete. 2024-04-13T03:10:50.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:10:50.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:50.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:10:50.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:10:50.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:10:50.903 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:10:50.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.063+0000 7f049fc76640 1 Processor -- start 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 -- start start 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 0x7f04980d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 0x7f04980d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04980cfd20 con 0x7f04980b3d60 2024-04-13T03:10:51.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0498162af0 con 0x7f049815e9b0 2024-04-13T03:10:51.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.064+0000 7f049fc76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0498162dd0 con 0x7f049815ed90 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049da12640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 0x7f04980d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049d211640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 0x7f04980d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049e213640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f04980b3d60 0x7f04980d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:38356/0 (socket says 172.21.15.106:38356) 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049e213640 1 -- 172.21.15.106:0/3963751090 learned_addr learned my addr 172.21.15.106:0/3963751090 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049ca10640 1 -- 172.21.15.106:0/3963751090 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242292205 0 0) 0x7f04980cfd20 con 0x7f04980b3d60 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049ca10640 1 -- 172.21.15.106:0/3963751090 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0480003520 con 0x7f04980b3d60 2024-04-13T03:10:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049d211640 1 -- 172.21.15.106:0/3963751090 >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 msgr2=0x7f04980d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049d211640 1 --2- 172.21.15.106:0/3963751090 >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 0x7f04980d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049d211640 1 -- 172.21.15.106:0/3963751090 >> v1:172.21.15.114:6789/0 conn(0x7f04980b3d60 legacy=0x7f04980d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049d211640 1 -- 172.21.15.106:0/3963751090 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04981630b0 con 0x7f049815ed90 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.065+0000 7f049da12640 1 --2- 172.21.15.106:0/3963751090 >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 0x7f04980d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.066+0000 7f049d211640 1 --2- 172.21.15.106:0/3963751090 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 0x7f04980d45f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f048c009de0 tx=0x7f048c02bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15074e02d1086d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.066+0000 7f049ca10640 1 -- 172.21.15.106:0/3963751090 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c039070 con 0x7f049815ed90 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.066+0000 7f049ca10640 1 -- 172.21.15.106:0/3963751090 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c02cd30 con 0x7f049815ed90 2024-04-13T03:10:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.066+0000 7f049ca10640 1 -- 172.21.15.106:0/3963751090 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c034660 con 0x7f049815ed90 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 -- 172.21.15.106:0/3963751090 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 msgr2=0x7f04980d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 --2- 172.21.15.106:0/3963751090 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 0x7f04980d45f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f048c009de0 tx=0x7f048c02bf80 comp rx=0 tx=0).stop 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 -- 172.21.15.106:0/3963751090 shutdown_connections 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 --2- 172.21.15.106:0/3963751090 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f049815ed90 0x7f04980d45f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 --2- 172.21.15.106:0/3963751090 >> v2:172.21.15.114:3300/0 conn(0x7f049815e9b0 0x7f04980d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 -- 172.21.15.106:0/3963751090 >> 172.21.15.106:0/3963751090 conn(0x7f049808bf10 msgr2=0x7f04980b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 -- 172.21.15.106:0/3963751090 shutdown_connections 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.067+0000 7f049fc76640 1 -- 172.21.15.106:0/3963751090 wait complete. 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.068+0000 7f049fc76640 1 Processor -- start 2024-04-13T03:10:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.068+0000 7f049fc76640 1 -- start start 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f04980b3d60 0x7f049815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f049815e9b0 0x7f04980b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0498162800 con 0x7f04980b5ee0 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0498162680 con 0x7f049815e9b0 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f049806eb20 con 0x7f04980b3d60 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44168/0 (socket says 172.21.15.106:44168) 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 -- 172.21.15.106:0/2943510730 learned_addr learned my addr 172.21.15.106:0/2943510730 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049da12640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f04980b3d60 0x7f049815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049d211640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f049815e9b0 0x7f04980b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 -- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f04980b3d60 msgr2=0x7f049815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f04980b3d60 0x7f049815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 -- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f049815e9b0 msgr2=0x7f04980b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f049815e9b0 0x7f04980b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 -- 172.21.15.106:0/2943510730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04980b6450 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049e213640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f048800c430 tx=0x7f048800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488017070 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488005ce0 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04980b6730 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04880072f0 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.069+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04980b42e0 con 0x7f04980b5ee0 2024-04-13T03:10:51.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.070+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498068a10 con 0x7f04980b5ee0 2024-04-13T03:10:51.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.071+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f048801a070 con 0x7f04980b5ee0 2024-04-13T03:10:51.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.071+0000 7f04967fc640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 0x7f048406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.071+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f048808b070 con 0x7f04980b5ee0 2024-04-13T03:10:51.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.071+0000 7f049da12640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 0x7f048406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.072+0000 7f049da12640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 0x7f048406ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0490009560 tx=0x7f0490008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.075+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0488059310 con 0x7f04980b5ee0 2024-04-13T03:10:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.288+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04980d1a30 con 0x7f04980b5ee0 2024-04-13T03:10:51.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.288+0000 7f04967fc640 1 -- 172.21.15.106:0/2943510730 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f0488058c10 con 0x7f04980b5ee0 2024-04-13T03:10:51.289 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:51.289 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:10:51.290 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 msgr2=0x7f048406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 0x7f048406ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0490009560 tx=0x7f0490008040 comp rx=0 tx=0).stop 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 msgr2=0x7f049815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f048800c430 tx=0x7f048800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 shutdown_connections 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f048406c920 0x7f048406ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04980b5ee0 0x7f049815c600 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f049815e9b0 0x7f04980b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 --2- 172.21.15.106:0/2943510730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f04980b3d60 0x7f049815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 >> 172.21.15.106:0/2943510730 conn(0x7f049808bf10 msgr2=0x7f04980b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:51.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 shutdown_connections 2024-04-13T03:10:51.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.290+0000 7f049fc76640 1 -- 172.21.15.106:0/2943510730 wait complete. 2024-04-13T03:10:51.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-13T03:10:51.443 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:10:51.443 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:10:51.444 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:10:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.543+0000 7f20b821c640 1 Processor -- start 2024-04-13T03:10:51.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.543+0000 7f20b821c640 1 -- start start 2024-04-13T03:10:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b821c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 0x7f20b00d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b821c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00d0190 con 0x7f20b00cfd30 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b821c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b0162820 con 0x7f20b00d0a80 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b821c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20b01625d0 con 0x7f20b00d4e00 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b57b7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f20b00cfd30 0x7f20b00d46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43120/0 (socket says 172.21.15.106:43120) 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b57b7640 1 -- 172.21.15.106:0/1952738975 learned_addr learned my addr 172.21.15.106:0/1952738975 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.544+0000 7f20b67b9640 1 --2- 172.21.15.106:0/1952738975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 0x7f20b00d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928430676 0 0) 0x7f20b00d0190 con 0x7f20b00cfd30 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f209c003520 con 0x7f20b00cfd30 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515820689 0 0) 0x7f20b0162820 con 0x7f20b00d0a80 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20b00d0190 con 0x7f20b00d0a80 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b67b9640 1 -- 172.21.15.106:0/1952738975 >> v1:172.21.15.114:6789/0 conn(0x7f20b00d0a80 legacy=0x7f20b00d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b67b9640 1 -- 172.21.15.106:0/1952738975 >> v1:172.21.15.106:6789/0 conn(0x7f20b00cfd30 legacy=0x7f20b00d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2650141185 0 0) 0x7f209c003520 con 0x7f20b00cfd30 2024-04-13T03:10:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.545+0000 7f20b67b9640 1 -- 172.21.15.106:0/1952738975 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0162b00 con 0x7f20b00d4e00 2024-04-13T03:10:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b67b9640 1 --2- 172.21.15.106:0/1952738975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 0x7f20b00d2530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20b0162750 tx=0x7f20a802b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b763665e6020bc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8039070 con 0x7f20b00d4e00 2024-04-13T03:10:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a8030070 con 0x7f20b00d4e00 2024-04-13T03:10:51.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b4fb6640 1 -- 172.21.15.106:0/1952738975 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0001670 con 0x7f20b00d4e00 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 -- 172.21.15.106:0/1952738975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 msgr2=0x7f20b00d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 --2- 172.21.15.106:0/1952738975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 0x7f20b00d2530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20b0162750 tx=0x7f20a802b7f0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 -- 172.21.15.106:0/1952738975 shutdown_connections 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 --2- 172.21.15.106:0/1952738975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00d4e00 0x7f20b00d2530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 -- 172.21.15.106:0/1952738975 >> 172.21.15.106:0/1952738975 conn(0x7f20b009da70 msgr2=0x7f20b00b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.546+0000 7f20b821c640 1 -- 172.21.15.106:0/1952738975 shutdown_connections 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.547+0000 7f20b821c640 1 -- 172.21.15.106:0/1952738975 wait complete. 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 Processor -- start 2024-04-13T03:10:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 -- start start 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00cfd30 0x7f20b0150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b00d4e00 0x7f20b006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00d41b0 con 0x7f20b00d0a80 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0163430 con 0x7f20b00d4e00 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b821c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20b01630c0 con 0x7f20b00cfd30 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b57b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b57b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44176/0 (socket says 172.21.15.106:44176) 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.548+0000 7f20b57b7640 1 -- 172.21.15.106:0/2219626436 learned_addr learned my addr 172.21.15.106:0/2219626436 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b5fb8640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00cfd30 0x7f20b0150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b67b9640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b00d4e00 0x7f20b006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 -- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00cfd30 msgr2=0x7f20b0150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00cfd30 0x7f20b0150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 -- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b00d4e00 msgr2=0x7f20b006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b00d4e00 0x7f20b006b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 -- 172.21.15.106:0/2219626436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b00cb9b0 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b57b7640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c780 tx=0x7f20a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a0005ad0 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b00ca720 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0007070 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.549+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b00cbc70 con 0x7f20b00d0a80 2024-04-13T03:10:51.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.550+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b0069120 con 0x7f20b00d0a80 2024-04-13T03:10:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.551+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20a001a070 con 0x7f20b00d0a80 2024-04-13T03:10:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.551+0000 7f20a67fc640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 0x7f209406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:10:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.551+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f20a008b1a0 con 0x7f20b00d0a80 2024-04-13T03:10:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.551+0000 7f20b5fb8640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 0x7f209406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:10:51.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.551+0000 7f20b5fb8640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 0x7f209406ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007fd0 tx=0x7f20ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:10:51.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.555+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20a00594d0 con 0x7f20b00d0a80 2024-04-13T03:10:51.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.768+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20b00d53d0 con 0x7f20b00d0a80 2024-04-13T03:10:51.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.769+0000 7f20a67fc640 1 -- 172.21.15.106:0/2219626436 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f20a0058dd0 con 0x7f20b00d0a80 2024-04-13T03:10:51.770 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:10:51.770 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:10:51.770 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 msgr2=0x7f209406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 0x7f209406ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007fd0 tx=0x7f20ac009040 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 msgr2=0x7f20b00c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c780 tx=0x7f20a000cc40 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 shutdown_connections 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209406c920 0x7f209406ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b00d4e00 0x7f20b006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00d0a80 0x7f20b00c9cd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 --2- 172.21.15.106:0/2219626436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b00cfd30 0x7f20b0150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:10:51.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.770+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 >> 172.21.15.106:0/2219626436 conn(0x7f20b009da70 msgr2=0x7f20b00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:10:51.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.771+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 shutdown_connections 2024-04-13T03:10:51.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:10:51.771+0000 7f20b821c640 1 -- 172.21.15.106:0/2219626436 wait complete. 2024-04-13T03:10:51.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2024-04-13T03:10:51.907 INFO:teuthology.orchestra.run.smithi114.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-13T03:10:51.908 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-13T03:10:54.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:10:54.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:11:01.908 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-13T03:11:01.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 Processor -- start 2024-04-13T03:11:01.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 -- start start 2024-04-13T03:11:01.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 0x557479c83380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479d5c730 0x557479d5cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557479c86910 con 0x557479d5c350 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557479c86790 con 0x557479d5c730 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.956+0000 7feb1683df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557479c86a90 con 0x557479c83c30 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54616/0 (socket says 172.21.15.106:54616) 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 -- 172.21.15.106:0/2057856070 learned_addr learned my addr 172.21.15.106:0/2057856070 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb1384b640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 0x557479c83380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb1404c640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479d5c730 0x557479d5cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 -- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 msgr2=0x557479c83380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 0x557479c83380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 -- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479d5c730 msgr2=0x557479d5cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479d5c730 0x557479d5cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 -- 172.21.15.106:0/2057856070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557479d566f0 con 0x557479d5c350 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb1384b640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 0x557479c83380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb0bfff640 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feafc00a7d0 tx=0x7feafc00a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea56b34c1b0f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb12849640 1 -- 172.21.15.106:0/2057856070 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc018070 con 0x557479d5c350 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb12849640 1 -- 172.21.15.106:0/2057856070 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc007600 con 0x557479d5c350 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.957+0000 7feb12849640 1 -- 172.21.15.106:0/2057856070 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0079d0 con 0x557479d5c350 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 -- 172.21.15.106:0/2057856070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 msgr2=0x557479d5b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feafc00a7d0 tx=0x7feafc00a8c0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 -- 172.21.15.106:0/2057856070 shutdown_connections 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479d5c730 0x557479d5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5b3f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 --2- 172.21.15.106:0/2057856070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479c83c30 0x557479c83380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 -- 172.21.15.106:0/2057856070 >> 172.21.15.106:0/2057856070 conn(0x557479c84f30 msgr2=0x557479ccc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 -- 172.21.15.106:0/2057856070 shutdown_connections 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.958+0000 7feb1683df40 1 -- 172.21.15.106:0/2057856070 wait complete. 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 Processor -- start 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 -- start start 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479c83c30 0x557479d49c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479d5c730 0x557479d39f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557479d5eb60 con 0x557479d5c350 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557479d5e8c0 con 0x557479c83c30 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb1683df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557479d5ee00 con 0x557479d5c730 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.959+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54618/0 (socket says 172.21.15.106:54618) 2024-04-13T03:11:01.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 -- 172.21.15.106:0/531076205 learned_addr learned my addr 172.21.15.106:0/531076205 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb1384b640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479c83c30 0x557479d49c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb1404c640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479d5c730 0x557479d39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 -- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479d5c730 msgr2=0x557479d39f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557479d5c730 0x557479d39f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 -- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479c83c30 msgr2=0x557479d49c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557479c83c30 0x557479d49c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 -- 172.21.15.106:0/531076205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557479df5600 con 0x557479d5c350 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb0bfff640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5cdc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feafc00afd0 tx=0x7feafc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb11847640 1 -- 172.21.15.106:0/531076205 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc018050 con 0x557479d5c350 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb11847640 1 -- 172.21.15.106:0/531076205 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc024070 con 0x557479d5c350 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557479d4ad20 con 0x557479d5c350 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb11847640 1 -- 172.21.15.106:0/531076205 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc013a80 con 0x557479d5c350 2024-04-13T03:11:01.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.960+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557479d4afc0 con 0x557479d5c350 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb11847640 1 -- 172.21.15.106:0/531076205 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feafc013c20 con 0x557479d5c350 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb11847640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feaf406b2d0 0x7feaf406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb11847640 1 -- 172.21.15.106:0/531076205 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7feafc08bce0 con 0x557479d5c350 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb1384b640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feaf406b2d0 0x7feaf406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb1683df40 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557479d5d300 0x557479d48960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.962+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x557479d48ea0 con 0x557479d5d300 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1404c640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557479d5d300 0x557479d48960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1384b640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feaf406b2d0 0x7feaf406d790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb04054000 tx=0x7feb0404e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1404c640 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557479d5d300 0x557479d48960 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1404c640 1 -- 172.21.15.106:0/531076205 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7feb0c00e650 con 0x557479d5d300 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557479d5d300 msgr2=0x557479d48960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1683df40 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557479d5d300 0x557479d48960 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:01.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feaf406b2d0 msgr2=0x7feaf406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1683df40 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feaf406b2d0 0x7feaf406d790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb04054000 tx=0x7feb0404e000 comp rx=0 tx=0).stop 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.963+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 msgr2=0x557479d5cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1683df40 1 --2- 172.21.15.106:0/531076205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557479d5c350 0x557479d5cdc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feafc00afd0 tx=0x7feafc003020 comp rx=0 tx=0).stop 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1404c640 1 -- 172.21.15.106:0/531076205 reap_dead start 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 shutdown_connections 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 >> 172.21.15.106:0/531076205 conn(0x557479c84f30 msgr2=0x557479cb3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 shutdown_connections 2024-04-13T03:11:01.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:01.964+0000 7feb1683df40 1 -- 172.21.15.106:0/531076205 wait complete. 2024-04-13T03:11:01.970 INFO:teuthology.orchestra.run.smithi106.stdout:mytag 2024-04-13T03:11:01.970 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-13T03:11:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.013+0000 7f9bf023ef40 1 Processor -- start 2024-04-13T03:11:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.013+0000 7f9bf023ef40 1 -- start start 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bd7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96be1170 0x55be96bd82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55be96b54b10 con 0x55be96b0b340 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be96ab3d00 con 0x55be96be1170 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55be96b54db0 con 0x55be96b3a3e0 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54634/0 (socket says 172.21.15.106:54634) 2024-04-13T03:11:02.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 -- 172.21.15.106:0/298871996 learned_addr learned my addr 172.21.15.106:0/298871996 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9beda4c640 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96be1170 0x55be96bd82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9beca4a640 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bd7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 -- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 msgr2=0x55be96bd7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bd7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 -- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96be1170 msgr2=0x55be96bd82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96be1170 0x55be96bd82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.014+0000 7f9bed24b640 1 -- 172.21.15.106:0/298871996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be96bd88a0 con 0x55be96b0b340 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bed24b640 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9be0044720 tx=0x7f9be004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7358ecf7d92d34f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bdf7fe640 1 -- 172.21.15.106:0/298871996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0057070 con 0x55be96b0b340 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bdf7fe640 1 -- 172.21.15.106:0/298871996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0061070 con 0x55be96b0b340 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bdf7fe640 1 -- 172.21.15.106:0/298871996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0052d10 con 0x55be96b0b340 2024-04-13T03:11:02.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 -- 172.21.15.106:0/298871996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 msgr2=0x55be96bd1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9be0044720 tx=0x7f9be004c000 comp rx=0 tx=0).stop 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 -- 172.21.15.106:0/298871996 shutdown_connections 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bd7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96be1170 0x55be96bd82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/298871996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96b0b340 0x55be96bd1280 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 -- 172.21.15.106:0/298871996 >> 172.21.15.106:0/298871996 conn(0x55be96b0bf30 msgr2=0x55be96bdf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 -- 172.21.15.106:0/298871996 shutdown_connections 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.015+0000 7f9bf023ef40 1 -- 172.21.15.106:0/298871996 wait complete. 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.016+0000 7f9bf023ef40 1 Processor -- start 2024-04-13T03:11:02.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.016+0000 7f9bf023ef40 1 -- start start 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96b0b340 0x55be96bdd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bc7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 0x55be96bbc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55be96b50aa0 con 0x55be96be1170 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be96bdfe20 con 0x55be96b0b340 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55be96bda080 con 0x55be96b3a3e0 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 0x55be96bbc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 0x55be96bbc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54642/0 (socket says 172.21.15.106:54642) 2024-04-13T03:11:02.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 -- 172.21.15.106:0/3399995788 learned_addr learned my addr 172.21.15.106:0/3399995788 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bed24b640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96b0b340 0x55be96bdd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beca4a640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bc7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 -- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 msgr2=0x55be96bc7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55be96b3a3e0 0x55be96bc7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 -- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96b0b340 msgr2=0x55be96bdd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be96b0b340 0x55be96bdd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 -- 172.21.15.106:0/3399995788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be96bc84a0 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9beda4c640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 0x55be96bbc130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9be800a220 tx=0x7f9be800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bde7fc640 1 -- 172.21.15.106:0/3399995788 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8015070 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bde7fc640 1 -- 172.21.15.106:0/3399995788 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be8011890 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be96b365a0 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bde7fc640 1 -- 172.21.15.106:0/3399995788 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8011c80 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.017+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be96b36b60 con 0x55be96be1170 2024-04-13T03:11:02.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bde7fc640 1 -- 172.21.15.106:0/3399995788 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9be8018070 con 0x55be96be1170 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bde7fc640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bd006b300 0x7f9bd006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bed24b640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bd006b300 0x7f9bd006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bde7fc640 1 -- 172.21.15.106:0/3399995788 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9be808a990 con 0x55be96be1170 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55be96bd8b10 0x55be96b37640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.019+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55be96b37b80 con 0x55be96bd8b10 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9bed24b640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bd006b300 0x7f9bd006d7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9be005e040 tx=0x7f9be004f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9beca4a640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55be96bd8b10 0x55be96b37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9beca4a640 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55be96bd8b10 0x55be96b37640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9beca4a640 1 -- 172.21.15.106:0/3399995788 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f9be400f870 con 0x55be96bd8b10 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55be96bd8b10 msgr2=0x55be96b37640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.020+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55be96bd8b10 0x55be96b37640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bd006b300 msgr2=0x7f9bd006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bd006b300 0x7f9bd006d7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9be005e040 tx=0x7f9be004f000 comp rx=0 tx=0).stop 2024-04-13T03:11:02.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 msgr2=0x55be96bbc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 --2- 172.21.15.106:0/3399995788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be96be1170 0x55be96bbc130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9be800a220 tx=0x7f9be800a6e0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9beda4c640 1 -- 172.21.15.106:0/3399995788 reap_dead start 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 shutdown_connections 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 >> 172.21.15.106:0/3399995788 conn(0x55be96b0bf30 msgr2=0x55be96bdf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 shutdown_connections 2024-04-13T03:11:02.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.021+0000 7f9bf023ef40 1 -- 172.21.15.106:0/3399995788 wait complete. 2024-04-13T03:11:02.027 INFO:teuthology.orchestra.run.smithi106.stdout:mytag 2024-04-13T03:11:02.027 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-13T03:11:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.110+0000 7fed2363bf40 1 Processor -- start 2024-04-13T03:11:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.110+0000 7fed2363bf40 1 -- start start 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64b95eec0 0x55d64ba1e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b9a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d64b9a61d0 con 0x55d64ba33450 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d64b906d00 con 0x55d64ba33070 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2363bf40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d64b99e910 con 0x55d64b95eec0 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54656/0 (socket says 172.21.15.106:54656) 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 -- 172.21.15.106:0/2918610365 learned_addr learned my addr 172.21.15.106:0/2918610365 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed2064b640 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64b95eec0 0x55d64ba1e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed17fff640 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b9a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 -- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64b95eec0 msgr2=0x55d64ba1e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64b95eec0 0x55d64ba1e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 -- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 msgr2=0x55d64b9a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b9a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 -- 172.21.15.106:0/2918610365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64b9a0600 con 0x55d64ba33450 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed18001510 tx=0x7fed180019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79542846a2f1be39 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed1f649640 1 -- 172.21.15.106:0/2918610365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18003650 con 0x55d64ba33450 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed1f649640 1 -- 172.21.15.106:0/2918610365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed180037f0 con 0x55d64ba33450 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.111+0000 7fed1f649640 1 -- 172.21.15.106:0/2918610365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18003c60 con 0x55d64ba33450 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- 172.21.15.106:0/2918610365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 msgr2=0x55d64ba1bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed18001510 tx=0x7fed180019d0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- 172.21.15.106:0/2918610365 shutdown_connections 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64ba33450 0x55d64ba1bef0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b9a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2918610365 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64b95eec0 0x55d64ba1e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- 172.21.15.106:0/2918610365 >> 172.21.15.106:0/2918610365 conn(0x55d64b98a360 msgr2=0x55d64b99f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- 172.21.15.106:0/2918610365 shutdown_connections 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- 172.21.15.106:0/2918610365 wait complete. 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 Processor -- start 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.112+0000 7fed2363bf40 1 -- start start 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 0x55d64ba1b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b95d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64ba33450 0x55d64b95d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d64b99eb40 con 0x55d64b95eec0 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d64b9a08a0 con 0x55d64ba33070 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2363bf40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d64b99ede0 con 0x55d64ba33450 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2064b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 0x55d64ba1b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2064b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 0x55d64ba1b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54666/0 (socket says 172.21.15.106:54666) 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2064b640 1 -- 172.21.15.106:0/2020293698 learned_addr learned my addr 172.21.15.106:0/2020293698 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64ba33450 0x55d64b95d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed17fff640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b95d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.113+0000 7fed2064b640 1 -- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64ba33450 msgr2=0x55d64b95d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2064b640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d64ba33450 0x55d64b95d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2064b640 1 -- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 msgr2=0x55d64b95d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2064b640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d64ba33070 0x55d64b95d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2064b640 1 -- 172.21.15.106:0/2020293698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64b960310 con 0x55d64b95eec0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2064b640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 0x55d64ba1b900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed10046e40 tx=0x7fed10002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed1e647640 1 -- 172.21.15.106:0/2020293698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10068020 con 0x55d64b95eec0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed1e647640 1 -- 172.21.15.106:0/2020293698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10064440 con 0x55d64b95eec0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed1e647640 1 -- 172.21.15.106:0/2020293698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10064830 con 0x55d64b95eec0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d64b9605f0 con 0x55d64b95eec0 2024-04-13T03:11:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.114+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64ba1be40 con 0x55d64b95eec0 2024-04-13T03:11:02.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.115+0000 7fed1e647640 1 -- 172.21.15.106:0/2020293698 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed1006b020 con 0x55d64b95eec0 2024-04-13T03:11:02.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed1e647640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed0006b2d0 0x7fed0006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed1e647640 1 -- 172.21.15.106:0/2020293698 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fed100cebc0 con 0x55d64b95eec0 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed17fff640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed0006b2d0 0x7fed0006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55d64ba1c660 0x55d64ba2f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55d64ba2fe90 con 0x55d64ba1c660 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed17fff640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed0006b2d0 0x7fed0006d790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55d64ba1cb60 tx=0x7fed08005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55d64ba1c660 0x55d64ba2f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.116+0000 7fed20e4c640 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55d64ba1c660 0x55d64ba2f950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed20e4c640 1 -- 172.21.15.106:0/2020293698 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fed1800f030 con 0x55d64ba1c660 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55d64ba1c660 msgr2=0x55d64ba2f950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55d64ba1c660 0x55d64ba2f950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed0006b2d0 msgr2=0x7fed0006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed0006b2d0 0x7fed0006d790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55d64ba1cb60 tx=0x7fed08005b80 comp rx=0 tx=0).stop 2024-04-13T03:11:02.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 msgr2=0x55d64ba1b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 --2- 172.21.15.106:0/2020293698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d64b95eec0 0x55d64ba1b900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed10046e40 tx=0x7fed10002fb0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed20e4c640 1 -- 172.21.15.106:0/2020293698 reap_dead start 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 shutdown_connections 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 >> 172.21.15.106:0/2020293698 conn(0x55d64b98a360 msgr2=0x55d64b98c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 shutdown_connections 2024-04-13T03:11:02.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.117+0000 7fed2363bf40 1 -- 172.21.15.106:0/2020293698 wait complete. 2024-04-13T03:11:02.124 INFO:teuthology.orchestra.run.smithi106.stdout:mytag 2024-04-13T03:11:02.124 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-13T03:11:02.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.165+0000 7fa2c7f5ff40 1 Processor -- start 2024-04-13T03:11:02.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.165+0000 7fa2c7f5ff40 1 -- start start 2024-04-13T03:11:02.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5b8b6d0 0x55eca5baa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c33350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eca5ba6780 con 0x55eca5c3a500 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eca5b09d00 con 0x55eca5b8b6d0 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55eca5ba6a20 con 0x55eca5c39be0 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54688/0 (socket says 172.21.15.106:54688) 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 -- 172.21.15.106:0/2261523146 learned_addr learned my addr 172.21.15.106:0/2261523146 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c584b640 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5b8b6d0 0x55eca5baa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2bffff640 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c33350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 -- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 msgr2=0x55eca5c33350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c33350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 -- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5b8b6d0 msgr2=0x55eca5baa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5b8b6d0 0x55eca5baa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.166+0000 7fa2c604c640 1 -- 172.21.15.106:0/2261523146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca5b64b90 con 0x55eca5c3a500 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006f80 tx=0x7fa2c0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76143771fa391e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c4849640 1 -- 172.21.15.106:0/2261523146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003650 con 0x55eca5c3a500 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c4849640 1 -- 172.21.15.106:0/2261523146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0003830 con 0x55eca5c3a500 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c4849640 1 -- 172.21.15.106:0/2261523146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003c60 con 0x55eca5c3a500 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2261523146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 msgr2=0x55eca5b905e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006f80 tx=0x7fa2c0001ac0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2261523146 shutdown_connections 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c33350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5c3a500 0x55eca5b905e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2261523146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5b8b6d0 0x55eca5baa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2261523146 >> 172.21.15.106:0/2261523146 conn(0x55eca5b8cae0 msgr2=0x55eca5ba5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2261523146 shutdown_connections 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.167+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2261523146 wait complete. 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.168+0000 7fa2c7f5ff40 1 Processor -- start 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.168+0000 7fa2c7f5ff40 1 -- start start 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.168+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 0x55eca5c1a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c18550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5c3a500 0x55eca5c34360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eca5c325b0 con 0x55eca5b8b6d0 2024-04-13T03:11:02.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eca5c32850 con 0x55eca5c3a500 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55eca5c32af0 con 0x55eca5c39be0 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 0x55eca5c1a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 0x55eca5c1a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54702/0 (socket says 172.21.15.106:54702) 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 -- 172.21.15.106:0/2400970916 learned_addr learned my addr 172.21.15.106:0/2400970916 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5c3a500 0x55eca5c34360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2bffff640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c18550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 -- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 msgr2=0x55eca5c18550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eca5c39be0 0x55eca5c18550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 -- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5c3a500 msgr2=0x55eca5c34360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eca5c3a500 0x55eca5c34360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 -- 172.21.15.106:0/2400970916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca5c19370 con 0x55eca5b8b6d0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c584b640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 0x55eca5c1a0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80086f0 tx=0x7fa2b8008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2beffd640 1 -- 172.21.15.106:0/2400970916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8018070 con 0x55eca5b8b6d0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2beffd640 1 -- 172.21.15.106:0/2400970916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b8006970 con 0x55eca5b8b6d0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2beffd640 1 -- 172.21.15.106:0/2400970916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8006d60 con 0x55eca5b8b6d0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eca5c19650 con 0x55eca5b8b6d0 2024-04-13T03:11:02.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.169+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca5c19c10 con 0x55eca5b8b6d0 2024-04-13T03:11:02.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2beffd640 1 -- 172.21.15.106:0/2400970916 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b801b070 con 0x55eca5b8b6d0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2beffd640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2a806b300 0x7fa2a806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2beffd640 1 -- 172.21.15.106:0/2400970916 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa2b808a1c0 con 0x55eca5b8b6d0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2bffff640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2a806b300 0x7fa2a806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55eca5c18e70 0x55eca5c38690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55eca5c38bd0 con 0x55eca5c18e70 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2bffff640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2a806b300 0x7fa2a806d7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0057000 tx=0x7fa2b0051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.171+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55eca5c18e70 0x55eca5c38690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c604c640 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55eca5c18e70 0x55eca5c38690 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c604c640 1 -- 172.21.15.106:0/2400970916 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa2c000f030 con 0x55eca5c18e70 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55eca5c18e70 msgr2=0x55eca5c38690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55eca5c18e70 0x55eca5c38690 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2a806b300 msgr2=0x7fa2a806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2a806b300 0x7fa2a806d7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0057000 tx=0x7fa2b0051000 comp rx=0 tx=0).stop 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 msgr2=0x55eca5c1a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 --2- 172.21.15.106:0/2400970916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eca5b8b6d0 0x55eca5c1a0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80086f0 tx=0x7fa2b8008bb0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c604c640 1 -- 172.21.15.106:0/2400970916 reap_dead start 2024-04-13T03:11:02.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 shutdown_connections 2024-04-13T03:11:02.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 >> 172.21.15.106:0/2400970916 conn(0x55eca5b8cae0 msgr2=0x55eca5ba9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 shutdown_connections 2024-04-13T03:11:02.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.172+0000 7fa2c7f5ff40 1 -- 172.21.15.106:0/2400970916 wait complete. 2024-04-13T03:11:02.182 INFO:teuthology.orchestra.run.smithi106.stdout:mytag 2024-04-13T03:11:02.183 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-13T03:11:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 Processor -- start 2024-04-13T03:11:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 -- start start 2024-04-13T03:11:02.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124c72380 0x560124cb50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124cb6060 0x560124c6e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560124cb6440 con 0x560124c9ef10 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560124c17d00 con 0x560124c72380 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd073e37f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560124c71c40 con 0x560124cb6060 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd07064a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd07064a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54722/0 (socket says 172.21.15.106:54722) 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.264+0000 7fd07064a640 1 -- 172.21.15.106:0/3028515148 learned_addr learned my addr 172.21.15.106:0/3028515148 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 -- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124cb6060 msgr2=0x560124c6e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124cb6060 0x560124c6e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 -- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124c72380 msgr2=0x560124cb50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124c72380 0x560124cb50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 -- 172.21.15.106:0/3028515148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560124d3d600 con 0x560124c9ef10 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd07064a640 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c9e0 tx=0x7fd05800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc1e24ebcded3b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd06f648640 1 -- 172.21.15.106:0/3028515148 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058008890 con 0x560124c9ef10 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd06f648640 1 -- 172.21.15.106:0/3028515148 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd058008a30 con 0x560124c9ef10 2024-04-13T03:11:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd06f648640 1 -- 172.21.15.106:0/3028515148 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058008de0 con 0x560124c9ef10 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 -- 172.21.15.106:0/3028515148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 msgr2=0x560124c9cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c9e0 tx=0x7fd05800cad0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 -- 172.21.15.106:0/3028515148 shutdown_connections 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124cb6060 0x560124c6e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c9ef10 0x560124c9cec0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 --2- 172.21.15.106:0/3028515148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124c72380 0x560124cb50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.265+0000 7fd073e37f40 1 -- 172.21.15.106:0/3028515148 >> 172.21.15.106:0/3028515148 conn(0x560124c6ff30 msgr2=0x560124c70310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.266+0000 7fd073e37f40 1 -- 172.21.15.106:0/3028515148 shutdown_connections 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.266+0000 7fd073e37f40 1 -- 172.21.15.106:0/3028515148 wait complete. 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.266+0000 7fd073e37f40 1 Processor -- start 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.266+0000 7fd073e37f40 1 -- start start 2024-04-13T03:11:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 0x560124d2a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124c9ef10 0x560124d3b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124cb6060 0x560124d3bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560124d3dd30 con 0x560124c72380 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560124d370b0 con 0x560124cb6060 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560124d36e10 con 0x560124c9ef10 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 0x560124d2a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 0x560124d2a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54728/0 (socket says 172.21.15.106:54728) 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 -- 172.21.15.106:0/2148401046 learned_addr learned my addr 172.21.15.106:0/2148401046 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd07064a640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124c9ef10 0x560124d3b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd07164c640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124cb6060 0x560124d3bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 -- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124c9ef10 msgr2=0x560124d3b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124c9ef10 0x560124d3b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 -- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124cb6060 msgr2=0x560124d3bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560124cb6060 0x560124d3bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 -- 172.21.15.106:0/2148401046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560124d2ac80 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd07064a640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560124c9ef10 0x560124d3b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd070e4b640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 0x560124d2a740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bde0 tx=0x7fd068009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd06e646640 1 -- 172.21.15.106:0/2148401046 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068008070 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd06e646640 1 -- 172.21.15.106:0/2148401046 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06800f650 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560124cb58b0 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd06e646640 1 -- 172.21.15.106:0/2148401046 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06800fa40 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.267+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560124cb55d0 con 0x560124c72380 2024-04-13T03:11:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd06e646640 1 -- 172.21.15.106:0/2148401046 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd06800fbe0 con 0x560124c72380 2024-04-13T03:11:02.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd06e646640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd05006b2d0 0x7fd05006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd06e646640 1 -- 172.21.15.106:0/2148401046 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd06808ab40 con 0x560124c72380 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd07064a640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd05006b2d0 0x7fd05006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd073e37f40 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560124d35f40 0x560124d31970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x560124d31eb0 con 0x560124d35f40 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.269+0000 7fd07064a640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd05006b2d0 0x7fd05006d790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x560124d3c370 tx=0x7fd05800e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd07164c640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560124d35f40 0x560124d31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd07164c640 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560124d35f40 0x560124d31970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd07164c640 1 -- 172.21.15.106:0/2148401046 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fd06005e040 con 0x560124d35f40 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560124d35f40 msgr2=0x560124d31970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560124d35f40 0x560124d31970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd05006b2d0 msgr2=0x7fd05006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd05006b2d0 0x7fd05006d790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x560124d3c370 tx=0x7fd05800e000 comp rx=0 tx=0).stop 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 msgr2=0x560124d2a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 --2- 172.21.15.106:0/2148401046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560124c72380 0x560124d2a740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bde0 tx=0x7fd068009560 comp rx=0 tx=0).stop 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd07164c640 1 -- 172.21.15.106:0/2148401046 reap_dead start 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 shutdown_connections 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.270+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 >> 172.21.15.106:0/2148401046 conn(0x560124c6ff30 msgr2=0x560124c6f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.271+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 shutdown_connections 2024-04-13T03:11:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.271+0000 7fd073e37f40 1 -- 172.21.15.106:0/2148401046 wait complete. 2024-04-13T03:11:02.278 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:11:02.278 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:02.354 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:02.404 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:11:02.404 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:02.404 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:02.404 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:02.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:11:02.420 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:02.420 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:02.420 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:02.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:02.437 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:02.437 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:02.437 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:02.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:11:02.475 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:02.475 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:02.475 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:02.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:02.494 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:02.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.613+0000 7f1cfdcb7640 1 Processor -- start 2024-04-13T03:11:02.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.613+0000 7f1cfdcb7640 1 -- start start 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cfdcb7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1cf8156b50 0x7f1cf8160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cfdcb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf815d690 con 0x7f1cf806e480 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cfdcb7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf815d970 con 0x7f1cf815cc10 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cfdcb7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf8160410 con 0x7f1cf8156b50 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cf7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1cf806e480 0x7f1cf806bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51662/0 (socket says 172.21.15.106:51662) 2024-04-13T03:11:02.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.614+0000 7f1cf7fff640 1 -- 172.21.15.106:0/2655098250 learned_addr learned my addr 172.21.15.106:0/2655098250 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994457182 0 0) 0x7f1cf815d690 con 0x7f1cf806e480 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ce0003520 con 0x7f1cf806e480 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/2655098250 >> v2:172.21.15.114:3300/0 conn(0x7f1cf8156b50 0x7f1cf8160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4199947282 0 0) 0x7f1ce0003520 con 0x7f1cf806e480 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 >> v2:172.21.15.114:3300/0 conn(0x7f1cf8156b50 msgr2=0x7f1cf8160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 --2- 172.21.15.106:0/2655098250 >> v2:172.21.15.114:3300/0 conn(0x7f1cf8156b50 0x7f1cf8160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 >> v1:172.21.15.114:6789/0 conn(0x7f1cf815cc10 legacy=0x7f1cf806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf81606f0 con 0x7f1cf806e480 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f1cf00019f0 con 0x7f1cf806e480 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 >> v1:172.21.15.106:6789/0 conn(0x7f1cf806e480 legacy=0x7f1cf806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce00039b0 0x7f1ce0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 0x7f1ce0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ce00051c0 0x7f1ce000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8160410 con 0x7f1ce0004800 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce000c110 con 0x7f1ce00051c0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce000c310 con 0x7f1ce00039b0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 0x7f1ce0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce00039b0 0x7f1ce0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf7fff640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ce00051c0 0x7f1ce000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 -- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce00039b0 msgr2=0x7f1ce0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce00039b0 0x7f1ce0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 -- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ce00051c0 msgr2=0x7f1ce000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ce00051c0 0x7f1ce000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf7fff640 1 -- 172.21.15.106:0/2655098250 reap_dead start 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.615+0000 7f1cf6ffd640 1 -- 172.21.15.106:0/2655098250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce0003520 con 0x7f1ce0004800 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.616+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 0x7f1ce0004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1cec001750 tx=0x7f1ce0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446b86c6cd0d941 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.616+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec001370 con 0x7f1ce0004800 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.616+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0052d0 con 0x7f1ce0004800 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.616+0000 7f1cf67fc640 1 -- 172.21.15.106:0/2655098250 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec003830 con 0x7f1ce0004800 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 msgr2=0x7f1ce0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 0x7f1ce0004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1cec001750 tx=0x7f1ce0005830 comp rx=0 tx=0).stop 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/2655098250 shutdown_connections 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/2655098250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ce0004800 0x7f1ce0004c80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/2655098250 >> 172.21.15.106:0/2655098250 conn(0x7f1cf80b1f10 msgr2=0x7f1cf80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/2655098250 shutdown_connections 2024-04-13T03:11:02.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.617+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/2655098250 wait complete. 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.618+0000 7f1cfdcb7640 1 Processor -- start 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.618+0000 7f1cfdcb7640 1 -- start start 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 0x7f1cf8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 0x7f1cf80c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf80afc20 con 0x7f1cf80c3100 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf80afaa0 con 0x7f1cf815dc40 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cfdcb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf815d080 con 0x7f1ce0004800 2024-04-13T03:11:02.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54754/0 (socket says 172.21.15.106:54754) 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 -- 172.21.15.106:0/776275892 learned_addr learned my addr 172.21.15.106:0/776275892 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 0x7f1cf8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 0x7f1cf80c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 -- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 msgr2=0x7f1cf8155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 0x7f1cf8155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 -- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 msgr2=0x7f1cf80c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 0x7f1cf80c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 -- 172.21.15.106:0/776275892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf80cf010 con 0x7f1cf80c3100 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 0x7f1cf8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf6ffd640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 0x7f1cf80c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cf7fff640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80afe50 tx=0x7f1cf0007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000aac0 con 0x7f1cf80c3100 2024-04-13T03:11:02.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.619+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf0018070 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.620+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf80cf290 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.620+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00133f0 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.620+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf80cf850 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.621+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf80cfb80 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.621+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cf0013590 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.621+0000 7f1cdffff640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 0x7f1ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.621+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1cf008c110 con 0x7f1cf80c3100 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.621+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 0x7f1ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.622+0000 7f1cf77fe640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 0x7f1ce406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006fd0 tx=0x7f1ce8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.626+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cf80cf010 con 0x7f1cf80c3100 2024-04-13T03:11:02.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.837+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cf8068a10 con 0x7f1cf80c3100 2024-04-13T03:11:02.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.838+0000 7f1cdffff640 1 -- 172.21.15.106:0/776275892 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f1cf005a440 con 0x7f1cf80c3100 2024-04-13T03:11:02.839 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:02.839 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:11:02.839 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:11:02.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 msgr2=0x7f1ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 0x7f1ce406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006fd0 tx=0x7f1ce8008040 comp rx=0 tx=0).stop 2024-04-13T03:11:02.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 msgr2=0x7f1cf815e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80afe50 tx=0x7f1cf0007b00 comp rx=0 tx=0).stop 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 shutdown_connections 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1ce406c8f0 0x7f1ce406edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cf80c3100 0x7f1cf815e5e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cf815dc40 0x7f1cf80c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 --2- 172.21.15.106:0/776275892 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1ce0004800 0x7f1cf8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 >> 172.21.15.106:0/776275892 conn(0x7f1cf80b1f10 msgr2=0x7f1cf8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 shutdown_connections 2024-04-13T03:11:02.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.840+0000 7f1cfdcb7640 1 -- 172.21.15.106:0/776275892 wait complete. 2024-04-13T03:11:02.852 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:02.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.951+0000 7ff612080640 1 Processor -- start 2024-04-13T03:11:02.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.951+0000 7ff612080640 1 -- start start 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.952+0000 7ff612080640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff60c0cefe0 0x7ff60c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.952+0000 7ff612080640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0b3980 0x7ff60c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.952+0000 7ff612080640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c0cf900 con 0x7ff60c0d2880 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.952+0000 7ff612080640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff60c0cfda0 con 0x7ff60c0cefe0 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.952+0000 7ff612080640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c06bfe0 con 0x7ff60c0b3980 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60b7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff60c0cefe0 0x7ff60c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60bfff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff60c0d2880 0x7ff60c0d2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:41120/0 (socket says 172.21.15.106:41120) 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60affd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0b3980 0x7ff60c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60bfff640 1 -- 172.21.15.106:0/3387255676 learned_addr learned my addr 172.21.15.106:0/3387255676 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60b7fe640 1 -- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0b3980 msgr2=0x7ff60c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:02.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60b7fe640 1 --2- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0b3980 0x7ff60c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60b7fe640 1 -- 172.21.15.106:0/3387255676 >> v1:172.21.15.114:6789/0 conn(0x7ff60c0d2880 legacy=0x7ff60c0d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60a7fc640 1 -- 172.21.15.106:0/3387255676 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159638581 0 0) 0x7ff60c0cf900 con 0x7ff60c0d2880 2024-04-13T03:11:02.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.953+0000 7ff60b7fe640 1 -- 172.21.15.106:0/3387255676 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c06c2c0 con 0x7ff60c0cefe0 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff60b7fe640 1 --2- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c06b8e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd77b5a686928bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff60a7fc640 1 -- 172.21.15.106:0/3387255676 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc035020 con 0x7ff60c0cefe0 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff60a7fc640 1 -- 172.21.15.106:0/3387255676 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc031830 con 0x7ff60c0cefe0 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff60a7fc640 1 -- 172.21.15.106:0/3387255676 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc031c00 con 0x7ff60c0cefe0 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 -- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 msgr2=0x7ff60c06b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 --2- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c06b8e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 -- 172.21.15.106:0/3387255676 shutdown_connections 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 --2- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0b3980 0x7ff60c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 --2- 172.21.15.106:0/3387255676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c06b8e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 -- 172.21.15.106:0/3387255676 >> 172.21.15.106:0/3387255676 conn(0x7ff60c0d4780 msgr2=0x7ff60c0d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:02.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 -- 172.21.15.106:0/3387255676 shutdown_connections 2024-04-13T03:11:02.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.954+0000 7ff612080640 1 -- 172.21.15.106:0/3387255676 wait complete. 2024-04-13T03:11:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 Processor -- start 2024-04-13T03:11:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 -- start start 2024-04-13T03:11:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c0d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0d2880 0x7ff60c0d4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c06cb60 con 0x7ff60c0b3980 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c06c880 con 0x7ff60c0cefe0 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.956+0000 7ff612080640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c0b8820 con 0x7ff60c0d2880 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50292/0 (socket says 172.21.15.106:50292) 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 -- 172.21.15.106:0/855801846 learned_addr learned my addr 172.21.15.106:0/855801846 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60bfff640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0d2880 0x7ff60c0d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 -- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0d2880 msgr2=0x7ff60c0d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0d2880 0x7ff60c0d4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60affd640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c0d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 -- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 msgr2=0x7ff60c0d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c0d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 -- 172.21.15.106:0/855801846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c1535e0 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff60b7fe640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc035040 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc004230 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c1f8160 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc03a640 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.957+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1f8720 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.958+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c075180 con 0x7ff60c0b3980 2024-04-13T03:11:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.959+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5fc038020 con 0x7ff60c0b3980 2024-04-13T03:11:02.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.959+0000 7ff5f3fff640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 0x7ff5f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:02.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.959+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff5fc0adb80 con 0x7ff60c0b3980 2024-04-13T03:11:02.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.959+0000 7ff60affd640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 0x7ff5f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:02.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.959+0000 7ff60affd640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 0x7ff5f806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b61a0 tx=0x7ff600008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:02.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:02.963+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff60c075180 con 0x7ff60c0b3980 2024-04-13T03:11:03.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.138+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff60c068a10 con 0x7ff60c0b3980 2024-04-13T03:11:03.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.139+0000 7ff5f3fff640 1 -- 172.21.15.106:0/855801846 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7ff5fc07bda0 con 0x7ff60c0b3980 2024-04-13T03:11:03.140 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:03.140 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:39.078056+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 msgr2=0x7ff5f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 0x7ff5f806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b61a0 tx=0x7ff600008040 comp rx=0 tx=0).stop 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 msgr2=0x7ff60c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc0046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 shutdown_connections 2024-04-13T03:11:03.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5f806c8f0 0x7ff5f806edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff60c0d2880 0x7ff60c0d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff60c0cefe0 0x7ff60c0d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 --2- 172.21.15.106:0/855801846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c0b3980 0x7ff60c0ce3f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 >> 172.21.15.106:0/855801846 conn(0x7ff60c0d4780 msgr2=0x7ff60c0b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 shutdown_connections 2024-04-13T03:11:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.141+0000 7ff612080640 1 -- 172.21.15.106:0/855801846 wait complete. 2024-04-13T03:11:03.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:10:35.635640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:10:36.371611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:11:03.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:11:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.256+0000 7f37de636640 1 Processor -- start 2024-04-13T03:11:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.256+0000 7f37de636640 1 -- start start 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37de636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d815ed90 0x7f37d80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37de636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d80cfd20 con 0x7f37d815e9b0 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37de636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d8162af0 con 0x7f37d80b3d60 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37de636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f37d8162dd0 con 0x7f37d815ed90 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37d7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f37d815e9b0 0x7f37d80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39412/0 (socket says 172.21.15.106:39412) 2024-04-13T03:11:03.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.257+0000 7f37d7fff640 1 -- 172.21.15.106:0/1608075627 learned_addr learned my addr 172.21.15.106:0/1608075627 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d77fe640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d815ed90 0x7f37d80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916438120 0 0) 0x7f37d80cfd20 con 0x7f37d815e9b0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37c0003520 con 0x7f37d815e9b0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136598918 0 0) 0x7f37c0003520 con 0x7f37d815e9b0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d815ed90 msgr2=0x7f37d80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d815ed90 0x7f37d80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 >> v1:172.21.15.114:6789/0 conn(0x7f37d80b3d60 legacy=0x7f37d80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d81630b0 con 0x7f37d815e9b0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f37c8001690 con 0x7f37d815e9b0 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 >> v1:172.21.15.106:6789/0 conn(0x7f37d815e9b0 legacy=0x7f37d80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37c00051c0 0x7f37c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8162dd0 con 0x7f37c00039e0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37c000bf00 con 0x7f37c0004800 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f37c000c100 con 0x7f37c00051c0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d77fe640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37dcbd3640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37c00051c0 0x7f37c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 -- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37c00051c0 msgr2=0x7f37c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37c00051c0 0x7f37c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 -- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c0004800 msgr2=0x7f37c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.258+0000 7f37d7fff640 1 -- 172.21.15.106:0/1608075627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c0003520 con 0x7f37c00039e0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.259+0000 7f37d7fff640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37c0003e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c210 tx=0x7f37c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26297e06437c2a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.259+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8007d10 con 0x7f37c00039e0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.259+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c8030070 con 0x7f37c00039e0 2024-04-13T03:11:03.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.259+0000 7f37d6ffd640 1 -- 172.21.15.106:0/1608075627 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8005d30 con 0x7f37c00039e0 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 -- 172.21.15.106:0/1608075627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 msgr2=0x7f37c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 --2- 172.21.15.106:0/1608075627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37c0003e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c210 tx=0x7f37c0005830 comp rx=0 tx=0).stop 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37dcbd3640 1 -- 172.21.15.106:0/1608075627 reap_dead start 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 -- 172.21.15.106:0/1608075627 shutdown_connections 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 -- 172.21.15.106:0/1608075627 >> 172.21.15.106:0/1608075627 conn(0x7f37d808bf10 msgr2=0x7f37d80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 -- 172.21.15.106:0/1608075627 shutdown_connections 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.260+0000 7f37de636640 1 -- 172.21.15.106:0/1608075627 wait complete. 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.261+0000 7f37de636640 1 Processor -- start 2024-04-13T03:11:03.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.261+0000 7f37de636640 1 -- start start 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 0x7f37d81f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d81f4440 0x7f37d80cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8162800 con 0x7f37c00039e0 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8162680 con 0x7f37d81f4440 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37de636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f37d80d4d00 con 0x7f37d80ca610 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50306/0 (socket says 172.21.15.106:50306) 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 -- 172.21.15.106:0/3120008498 learned_addr learned my addr 172.21.15.106:0/3120008498 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37dcbd3640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d81f4440 0x7f37d80cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d77fe640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 0x7f37d81f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 -- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 msgr2=0x7f37d81f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 0x7f37d81f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 -- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d81f4440 msgr2=0x7f37d80cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d81f4440 0x7f37d80cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 -- 172.21.15.106:0/3120008498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d815ed90 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d77fe640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 0x7f37d81f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d7fff640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c210 tx=0x7f37d8163280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.262+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8001c50 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.263+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c8004bf0 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.263+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d81f4ce0 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.263+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8007070 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.263+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81f52a0 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.264+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8068a10 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.264+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37c8007210 con 0x7f37c00039e0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.265+0000 7f37d4ff9640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.265+0000 7f37d77fe640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.265+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f37c80ad330 con 0x7f37c00039e0 2024-04-13T03:11:03.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.265+0000 7f37d77fe640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 0x7f37c406ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37d80d3e60 tx=0x7f37cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.269+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37c807b660 con 0x7f37c00039e0 2024-04-13T03:11:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.447+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37d80d18d0 con 0x7f37c00039e0 2024-04-13T03:11:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.448+0000 7f37d4ff9640 1 -- 172.21.15.106:0/3120008498 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f37c807af60 con 0x7f37c00039e0 2024-04-13T03:11:03.449 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 msgr2=0x7f37c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 0x7f37c406ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37d80d3e60 tx=0x7f37cc008040 comp rx=0 tx=0).stop 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 msgr2=0x7f37d8152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c210 tx=0x7f37d8163280 comp rx=0 tx=0).stop 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 shutdown_connections 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37d80ca610 0x7f37d81f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d81f4440 0x7f37d80cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 --2- 172.21.15.106:0/3120008498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c00039e0 0x7f37d8152b40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 >> 172.21.15.106:0/3120008498 conn(0x7f37d808bf10 msgr2=0x7f37d80b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 shutdown_connections 2024-04-13T03:11:03.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.450+0000 7f37de636640 1 -- 172.21.15.106:0/3120008498 wait complete. 2024-04-13T03:11:03.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:11:03.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 Processor -- start 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 -- start start 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f429815e9b0 0x7f42980d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f429815ed90 0x7f42980d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42980cfd20 con 0x7f42980b3d60 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4298162af0 con 0x7f429815e9b0 2024-04-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.564+0000 7f429d4de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4298162dd0 con 0x7f429815ed90 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f429815e9b0 0x7f42980d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f429815e9b0 0x7f42980d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50312/0 (socket says 172.21.15.106:50312) 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 -- 172.21.15.106:0/2251373982 learned_addr learned my addr 172.21.15.106:0/2251373982 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f42967fc640 1 --2- 172.21.15.106:0/2251373982 >> v2:172.21.15.114:3300/0 conn(0x7f429815ed90 0x7f42980d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 -- 172.21.15.106:0/2251373982 >> v2:172.21.15.114:3300/0 conn(0x7f429815ed90 msgr2=0x7f42980d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:03.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 --2- 172.21.15.106:0/2251373982 >> v2:172.21.15.114:3300/0 conn(0x7f429815ed90 0x7f42980d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 -- 172.21.15.106:0/2251373982 >> v1:172.21.15.114:6789/0 conn(0x7f42980b3d60 legacy=0x7f42980d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.565+0000 7f4296ffd640 1 -- 172.21.15.106:0/2251373982 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42981630b0 con 0x7f429815e9b0 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f4296ffd640 1 --2- 172.21.15.106:0/2251373982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f429815e9b0 0x7f42980d2300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f429808ebd0 tx=0x7f428802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e27d74ebab3216d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f4295ffb640 1 -- 172.21.15.106:0/2251373982 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428802f850 con 0x7f429815e9b0 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f4295ffb640 1 -- 172.21.15.106:0/2251373982 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428802f9f0 con 0x7f429815e9b0 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f4295ffb640 1 -- 172.21.15.106:0/2251373982 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428802fdc0 con 0x7f429815e9b0 2024-04-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 -- 172.21.15.106:0/2251373982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f429815e9b0 msgr2=0x7f42980d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 --2- 172.21.15.106:0/2251373982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f429815e9b0 0x7f42980d2300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f429808ebd0 tx=0x7f428802ba30 comp rx=0 tx=0).stop 2024-04-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 -- 172.21.15.106:0/2251373982 shutdown_connections 2024-04-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 --2- 172.21.15.106:0/2251373982 >> v2:172.21.15.114:3300/0 conn(0x7f429815ed90 0x7f42980d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 --2- 172.21.15.106:0/2251373982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f429815e9b0 0x7f42980d2300 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 -- 172.21.15.106:0/2251373982 >> 172.21.15.106:0/2251373982 conn(0x7f429808bf10 msgr2=0x7f42980b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 -- 172.21.15.106:0/2251373982 shutdown_connections 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.566+0000 7f429d4de640 1 -- 172.21.15.106:0/2251373982 wait complete. 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 Processor -- start 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 -- start start 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42980b3d60 0x7f429815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f429815e9b0 0x7f42980b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4298162800 con 0x7f42980b59d0 2024-04-13T03:11:03.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4298162680 con 0x7f429815e9b0 2024-04-13T03:11:03.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.568+0000 7f429d4de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f429806eb20 con 0x7f42980b3d60 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50322/0 (socket says 172.21.15.106:50322) 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 -- 172.21.15.106:0/3209568332 learned_addr learned my addr 172.21.15.106:0/3209568332 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f4296ffd640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42980b3d60 0x7f429815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42967fc640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f429815e9b0 0x7f42980b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 -- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42980b3d60 msgr2=0x7f429815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42980b3d60 0x7f429815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 -- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f429815e9b0 msgr2=0x7f42980b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f429815e9b0 0x7f42980b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 -- 172.21.15.106:0/3209568332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42980cdcd0 con 0x7f42980b59d0 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f42977fe640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f429000c7a0 tx=0x7f429000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290003b70 con 0x7f42980b59d0 2024-04-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290003d30 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429000aa90 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42980cdfb0 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.569+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42980ce570 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.570+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4298068a10 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.571+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f429000ac30 con 0x7f42980b59d0 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.571+0000 7f427f7fe640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 0x7f428406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.571+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f429008bbe0 con 0x7f42980b59d0 2024-04-13T03:11:03.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.571+0000 7f4296ffd640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 0x7f428406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.571+0000 7f4296ffd640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 0x7f428406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4288036000 tx=0x7f4288008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.575+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4290059e80 con 0x7f42980b59d0 2024-04-13T03:11:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.747+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42980b2760 con 0x7f42980b59d0 2024-04-13T03:11:03.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.747+0000 7f427f7fe640 1 -- 172.21.15.106:0/3209568332 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7f4290059780 con 0x7f42980b59d0 2024-04-13T03:11:03.748 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:11:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 msgr2=0x7f428406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 0x7f428406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4288036000 tx=0x7f4288008030 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 msgr2=0x7f42980c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f429000c7a0 tx=0x7f429000cc60 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 shutdown_connections 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f428406c920 0x7f428406ede0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980b59d0 0x7f42980c9d60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f429815e9b0 0x7f42980b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 --2- 172.21.15.106:0/3209568332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42980b3d60 0x7f429815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 >> 172.21.15.106:0/3209568332 conn(0x7f429808bf10 msgr2=0x7f42980b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 shutdown_connections 2024-04-13T03:11:03.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.749+0000 7f429d4de640 1 -- 172.21.15.106:0/3209568332 wait complete. 2024-04-13T03:11:03.761 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:11:03.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.863+0000 7fbab95c8640 1 Processor -- start 2024-04-13T03:11:03.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.863+0000 7fbab95c8640 1 -- start start 2024-04-13T03:11:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab95c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbab415ea80 0x7fbab40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab95c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbab40d1630 0x7fbab40d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab95c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab40cfe70 con 0x7fbab4162550 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab95c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40d1f80 con 0x7fbab40d1630 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab95c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbab415ee60 con 0x7fbab415ea80 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbab40d1630 0x7fbab40d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab27fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbab4162550 0x7fbab40b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39414/0 (socket says 172.21.15.106:39414) 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab27fc640 1 -- 172.21.15.106:0/603808135 learned_addr learned my addr 172.21.15.106:0/603808135 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab37fe640 1 -- 172.21.15.106:0/603808135 >> v2:172.21.15.114:3300/0 conn(0x7fbab415ea80 msgr2=0x7fbab40d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab37fe640 1 --2- 172.21.15.106:0/603808135 >> v2:172.21.15.114:3300/0 conn(0x7fbab415ea80 0x7fbab40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab37fe640 1 -- 172.21.15.106:0/603808135 >> v1:172.21.15.106:6789/0 conn(0x7fbab4162550 legacy=0x7fbab40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab37fe640 1 -- 172.21.15.106:0/603808135 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab4162af0 con 0x7fbab40d1630 2024-04-13T03:11:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.864+0000 7fbab1ffb640 1 -- 172.21.15.106:0/603808135 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777498506 0 0) 0x7fbab40cfe70 con 0x7fbab4162550 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.865+0000 7fbab37fe640 1 --2- 172.21.15.106:0/603808135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab40d1630 0x7fbab40d1a40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbab40ba750 tx=0x7fbaac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dab0cd6db047db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.865+0000 7fbab1ffb640 1 -- 172.21.15.106:0/603808135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac035020 con 0x7fbab40d1630 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.865+0000 7fbab1ffb640 1 -- 172.21.15.106:0/603808135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaac031830 con 0x7fbab40d1630 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.865+0000 7fbab1ffb640 1 -- 172.21.15.106:0/603808135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac031c00 con 0x7fbab40d1630 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 -- 172.21.15.106:0/603808135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab40d1630 msgr2=0x7fbab40d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 --2- 172.21.15.106:0/603808135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab40d1630 0x7fbab40d1a40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbab40ba750 tx=0x7fbaac02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 -- 172.21.15.106:0/603808135 shutdown_connections 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 --2- 172.21.15.106:0/603808135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab40d1630 0x7fbab40d1a40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 --2- 172.21.15.106:0/603808135 >> v2:172.21.15.114:3300/0 conn(0x7fbab415ea80 0x7fbab40d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 -- 172.21.15.106:0/603808135 >> 172.21.15.106:0/603808135 conn(0x7fbab409da50 msgr2=0x7fbab40b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 -- 172.21.15.106:0/603808135 shutdown_connections 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.866+0000 7fbab95c8640 1 -- 172.21.15.106:0/603808135 wait complete. 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.867+0000 7fbab95c8640 1 Processor -- start 2024-04-13T03:11:03.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.867+0000 7fbab95c8640 1 -- start start 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 0x7fbab414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 0x7fbab40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4163390 con 0x7fbab415ea80 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41630b0 con 0x7fbab4162550 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab95c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbab40d0320 con 0x7fbab40d1630 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50336/0 (socket says 172.21.15.106:50336) 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 -- 172.21.15.106:0/3463097660 learned_addr learned my addr 172.21.15.106:0/3463097660 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab2ffd640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 0x7fbab414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 -- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 msgr2=0x7fbab414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab37fe640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 0x7fbab40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 0x7fbab414fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 -- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 msgr2=0x7fbab40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 0x7fbab40ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 -- 172.21.15.106:0/3463097660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab415d1f0 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab2ffd640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 0x7fbab414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab37fe640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 0x7fbab40ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.868+0000 7fbab27fc640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c3f0 tx=0x7fbaa800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.869+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80074e0 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.869+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa80047e0 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.869+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8004bd0 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.869+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab406be20 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.869+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab40cd0f0 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.870+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab4075180 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.870+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaa8011070 con 0x7fbab415ea80 2024-04-13T03:11:03.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.870+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:03.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.871+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fbaa808b940 con 0x7fbab415ea80 2024-04-13T03:11:03.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.871+0000 7fbab2ffd640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:03.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.871+0000 7fbab2ffd640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007eb0 tx=0x7fbaa40073c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:03.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:03.875+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbab4075180 con 0x7fbab415ea80 2024-04-13T03:11:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.051+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbab4068a10 con 0x7fbab415ea80 2024-04-13T03:11:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.051+0000 7fba9b7fe640 1 -- 172.21.15.106:0/3463097660 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7fbab415d1f0 con 0x7fbab415ea80 2024-04-13T03:11:04.052 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:11:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 msgr2=0x7fbaa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007eb0 tx=0x7fbaa40073c0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 msgr2=0x7fbab406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c3f0 tx=0x7fbaa800c8b0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 shutdown_connections 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab4162550 0x7fbab40ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbab415ea80 0x7fbab406b8e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 --2- 172.21.15.106:0/3463097660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbab40d1630 0x7fbab414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 >> 172.21.15.106:0/3463097660 conn(0x7fbab409da50 msgr2=0x7fbab415db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 shutdown_connections 2024-04-13T03:11:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.053+0000 7fbab95c8640 1 -- 172.21.15.106:0/3463097660 wait complete. 2024-04-13T03:11:04.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:04.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.164+0000 7f4ba4005640 1 Processor -- start 2024-04-13T03:11:04.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.164+0000 7f4ba4005640 1 -- start start 2024-04-13T03:11:04.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba4005640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b9c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba4005640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b9c06cb80 0x7f4b9c0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba4005640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9c0ce870 con 0x7f4b9c072c10 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba4005640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9c073090 con 0x7f4b9c06cb80 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba4005640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9c073370 con 0x7f4b9c0cdf50 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba25a2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4b9c072c10 0x7f4b9c06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39424/0 (socket says 172.21.15.106:39424) 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba25a2640 1 -- 172.21.15.106:0/3327862539 learned_addr learned my addr 172.21.15.106:0/3327862539 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b9c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111168598 0 0) 0x7f4b9c0ce870 con 0x7f4b9c072c10 2024-04-13T03:11:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba15a0640 1 --2- 172.21.15.106:0/3327862539 >> v2:172.21.15.114:3300/0 conn(0x7f4b9c06cb80 0x7f4b9c0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b88003520 con 0x7f4b9c072c10 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2717162286 0 0) 0x7f4b88003520 con 0x7f4b9c072c10 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 msgr2=0x7f4b9c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b9c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 >> v2:172.21.15.114:3300/0 conn(0x7f4b9c06cb80 msgr2=0x7f4b9c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 --2- 172.21.15.106:0/3327862539 >> v2:172.21.15.114:3300/0 conn(0x7f4b9c06cb80 0x7f4b9c0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c073650 con 0x7f4b9c072c10 2024-04-13T03:11:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4b940019f0 con 0x7f4b9c072c10 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 >> v1:172.21.15.106:6789/0 conn(0x7f4b9c072c10 legacy=0x7f4b9c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.165+0000 7f4ba0d9f640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba0d9f640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba0d9f640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c073090 con 0x7f4b880039b0 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c073370 con 0x7f4b88004d40 2024-04-13T03:11:04.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8800dbd0 con 0x7f4b9c0cdf50 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba15a0640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba25a2640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 -- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 msgr2=0x7f4b88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c0cdf50 0x7f4b88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 -- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b88004d40 msgr2=0x7f4b880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.166+0000 7f4ba1da1640 1 -- 172.21.15.106:0/3327862539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88003520 con 0x7f4b880039b0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ac0 tx=0x7f4b88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d408ed29dbfb210 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98039070 con 0x7f4b880039b0 2024-04-13T03:11:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9802c6b0 con 0x7f4b880039b0 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba0d9f640 1 -- 172.21.15.106:0/3327862539 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98009690 con 0x7f4b880039b0 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 -- 172.21.15.106:0/3327862539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b880039b0 msgr2=0x7f4b88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 --2- 172.21.15.106:0/3327862539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ac0 tx=0x7f4b88005830 comp rx=0 tx=0).stop 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba25a2640 1 -- 172.21.15.106:0/3327862539 reap_dead start 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 -- 172.21.15.106:0/3327862539 shutdown_connections 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 -- 172.21.15.106:0/3327862539 >> 172.21.15.106:0/3327862539 conn(0x7f4b9c0b1f10 msgr2=0x7f4b9c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 -- 172.21.15.106:0/3327862539 shutdown_connections 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.167+0000 7f4ba4005640 1 -- 172.21.15.106:0/3327862539 wait complete. 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.168+0000 7f4ba4005640 1 Processor -- start 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 -- start start 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b880039b0 0x7f4b9c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c1e6230 0x7f4b9c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c099930 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c0997b0 con 0x7f4b880039b0 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba4005640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9c06f180 con 0x7f4b9c1e6230 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba15a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba15a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50362/0 (socket says 172.21.15.106:50362) 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba15a0640 1 -- 172.21.15.106:0/4202697993 learned_addr learned my addr 172.21.15.106:0/4202697993 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b880039b0 0x7f4b9c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.169+0000 7f4ba25a2640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c1e6230 0x7f4b9c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 -- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c1e6230 msgr2=0x7f4b9c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c1e6230 0x7f4b9c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 -- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b880039b0 msgr2=0x7f4b9c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b880039b0 0x7f4b9c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 -- 172.21.15.106:0/4202697993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1e4d70 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba15a0640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011aa0 tx=0x7f4b8c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c00fac0 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8c00fc60 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1e5010 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c014440 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.170+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1e55d0 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.171+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9c0726d0 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.171+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b8c012040 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.172+0000 7f4b927fc640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 0x7f4b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.172+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4b8c08c140 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.172+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 0x7f4b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.172+0000 7f4ba1da1640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 0x7f4b8006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b98036000 tx=0x7f4b98030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.176+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b9c0726d0 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.388+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9c068a10 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.388+0000 7f4b927fc640 1 -- 172.21.15.106:0/4202697993 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f4b8c019030 con 0x7f4b9c1e5dd0 2024-04-13T03:11:04.390 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:04.390 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:11:04.390 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 msgr2=0x7f4b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 0x7f4b8006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b98036000 tx=0x7f4b98030000 comp rx=0 tx=0).stop 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 msgr2=0x7f4b9c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011aa0 tx=0x7f4b8c011ea0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 shutdown_connections 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b8006c8f0 0x7f4b8006edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b9c1e6230 0x7f4b9c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b9c1e5dd0 0x7f4b9c155bc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 --2- 172.21.15.106:0/4202697993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b880039b0 0x7f4b9c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 >> 172.21.15.106:0/4202697993 conn(0x7f4b9c0b1f10 msgr2=0x7f4b9c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.390+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 shutdown_connections 2024-04-13T03:11:04.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.391+0000 7f4ba4005640 1 -- 172.21.15.106:0/4202697993 wait complete. 2024-04-13T03:11:04.403 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:04.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.501+0000 7f9093515640 1 Processor -- start 2024-04-13T03:11:04.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.501+0000 7f9093515640 1 -- start start 2024-04-13T03:11:04.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9093515640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f908c0cd4c0 0x7f908c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9093515640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f908c0b81f0 0x7f908c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9093515640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f908c0b54c0 con 0x7f908c15cb00 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9093515640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f908c071320 con 0x7f908c0b81f0 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9093515640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f908c070d60 con 0x7f908c0cd4c0 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f90912b1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f908c15cb00 0x7f908c0ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39436/0 (socket says 172.21.15.106:39436) 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f90912b1640 1 -- 172.21.15.106:0/2675387349 learned_addr learned my addr 172.21.15.106:0/2675387349 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.502+0000 7f9091ab2640 1 --2- 172.21.15.106:0/2675387349 >> v2:172.21.15.106:3300/0 conn(0x7f908c0b81f0 0x7f908c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9091ab2640 1 -- 172.21.15.106:0/2675387349 >> v2:172.21.15.114:3300/0 conn(0x7f908c0cd4c0 msgr2=0x7f908c160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9091ab2640 1 --2- 172.21.15.106:0/2675387349 >> v2:172.21.15.114:3300/0 conn(0x7f908c0cd4c0 0x7f908c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9091ab2640 1 -- 172.21.15.106:0/2675387349 >> v1:172.21.15.106:6789/0 conn(0x7f908c15cb00 legacy=0x7f908c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9091ab2640 1 -- 172.21.15.106:0/2675387349 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c071040 con 0x7f908c0b81f0 2024-04-13T03:11:04.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9083fff640 1 -- 172.21.15.106:0/2675387349 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160843837 0 0) 0x7f9088001490 con 0x7f908c15cb00 2024-04-13T03:11:04.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9091ab2640 1 --2- 172.21.15.106:0/2675387349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c0d0cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f908c06c670 tx=0x7f908402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a739af80dd3a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.503+0000 7f9083fff640 1 -- 172.21.15.106:0/2675387349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084035020 con 0x7f908c0b81f0 2024-04-13T03:11:04.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9083fff640 1 -- 172.21.15.106:0/2675387349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9084031830 con 0x7f908c0b81f0 2024-04-13T03:11:04.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9083fff640 1 -- 172.21.15.106:0/2675387349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084031c00 con 0x7f908c0b81f0 2024-04-13T03:11:04.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 -- 172.21.15.106:0/2675387349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 msgr2=0x7f908c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 --2- 172.21.15.106:0/2675387349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c0d0cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f908c06c670 tx=0x7f908402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 -- 172.21.15.106:0/2675387349 shutdown_connections 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 --2- 172.21.15.106:0/2675387349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c0d0cd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 --2- 172.21.15.106:0/2675387349 >> v2:172.21.15.114:3300/0 conn(0x7f908c0cd4c0 0x7f908c160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 -- 172.21.15.106:0/2675387349 >> 172.21.15.106:0/2675387349 conn(0x7f908c0d2530 msgr2=0x7f908c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 -- 172.21.15.106:0/2675387349 shutdown_connections 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.504+0000 7f9093515640 1 -- 172.21.15.106:0/2675387349 wait complete. 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.505+0000 7f9093515640 1 Processor -- start 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.505+0000 7f9093515640 1 -- start start 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f908c0cd4c0 0x7f908c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f908c15cb00 0x7f908c0b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0d1210 con 0x7f908c0b81f0 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0d07b0 con 0x7f908c0cd4c0 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9093515640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0d1390 con 0x7f908c15cb00 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50390/0 (socket says 172.21.15.106:50390) 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 -- 172.21.15.106:0/439861286 learned_addr learned my addr 172.21.15.106:0/439861286 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9090ab0640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f908c0cd4c0 0x7f908c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f9091ab2640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f908c15cb00 0x7f908c0b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 -- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f908c15cb00 msgr2=0x7f908c0b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f908c15cb00 0x7f908c0b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 -- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f908c0cd4c0 msgr2=0x7f908c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f908c0cd4c0 0x7f908c0b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.506+0000 7f90912b1640 1 -- 172.21.15.106:0/439861286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c0cb3d0 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f90912b1640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f908800c7c0 tx=0x7f908800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088018070 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908800a640 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c0cb6b0 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800aa30 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.507+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c0cbc70 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.508+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c068a10 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.508+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f908801b070 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.508+0000 7f9081ffb640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 0x7f907006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.509+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f908808b330 con 0x7f908c0b81f0 2024-04-13T03:11:04.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.509+0000 7f9090ab0640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 0x7f907006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.509+0000 7f9090ab0640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 0x7f907006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f908c0b4060 tx=0x7f907c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.513+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f908c0b54c0 con 0x7f908c0b81f0 2024-04-13T03:11:04.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.689+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f908c15fd10 con 0x7f908c0b81f0 2024-04-13T03:11:04.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.690+0000 7f9081ffb640 1 -- 172.21.15.106:0/439861286 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7f90880595d0 con 0x7f908c0b81f0 2024-04-13T03:11:04.691 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:04.691 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:10:39.078056+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:04.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 msgr2=0x7f907006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 0x7f907006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f908c0b4060 tx=0x7f907c008040 comp rx=0 tx=0).stop 2024-04-13T03:11:04.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 msgr2=0x7f908c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f908800c7c0 tx=0x7f908800cc80 comp rx=0 tx=0).stop 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 shutdown_connections 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f907006c8f0 0x7f907006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f908c15cb00 0x7f908c0b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f908c0cd4c0 0x7f908c0b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 --2- 172.21.15.106:0/439861286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f908c0b81f0 0x7f908c157d40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 >> 172.21.15.106:0/439861286 conn(0x7f908c0d2530 msgr2=0x7f908c0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 shutdown_connections 2024-04-13T03:11:04.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.692+0000 7f9093515640 1 -- 172.21.15.106:0/439861286 wait complete. 2024-04-13T03:11:04.704 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:11:04.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:04.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.803+0000 7f86e01bb640 1 Processor -- start 2024-04-13T03:11:04.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 -- start start 2024-04-13T03:11:04.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86d815e980 0x7f86d8162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d815ed60 0x7f86d80cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d815fe40 con 0x7f86d80cd380 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f86d81633f0 con 0x7f86d815ed60 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.804+0000 7f86e01bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f86d80cd780 con 0x7f86d815e980 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d815ed60 0x7f86d80cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f86d815ed60 0x7f86d80cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50392/0 (socket says 172.21.15.106:50392) 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 -- 172.21.15.106:0/3106501785 learned_addr learned my addr 172.21.15.106:0/3106501785 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86ddf57640 1 --2- 172.21.15.106:0/3106501785 >> v2:172.21.15.114:3300/0 conn(0x7f86d815e980 0x7f86d8162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 -- 172.21.15.106:0/3106501785 >> v2:172.21.15.114:3300/0 conn(0x7f86d815e980 msgr2=0x7f86d8162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 --2- 172.21.15.106:0/3106501785 >> v2:172.21.15.114:3300/0 conn(0x7f86d815e980 0x7f86d8162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 -- 172.21.15.106:0/3106501785 >> v1:172.21.15.114:6789/0 conn(0x7f86d80cd380 legacy=0x7f86d80d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.805+0000 7f86dd756640 1 -- 172.21.15.106:0/3106501785 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d815f2c0 con 0x7f86d815ed60 2024-04-13T03:11:04.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86dd756640 1 --2- 172.21.15.106:0/3106501785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d815ed60 0x7f86d80cce40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f86d80d31f0 tx=0x7f86c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13f4ae39fa0d343 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86dcf55640 1 -- 172.21.15.106:0/3106501785 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8034070 con 0x7f86d815ed60 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86dcf55640 1 -- 172.21.15.106:0/3106501785 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c8030860 con 0x7f86d815ed60 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86dcf55640 1 -- 172.21.15.106:0/3106501785 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8030c30 con 0x7f86d815ed60 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 -- 172.21.15.106:0/3106501785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d815ed60 msgr2=0x7f86d80cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3106501785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d815ed60 0x7f86d80cce40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f86d80d31f0 tx=0x7f86c802b9a0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 -- 172.21.15.106:0/3106501785 shutdown_connections 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3106501785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d815ed60 0x7f86d80cce40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3106501785 >> v2:172.21.15.114:3300/0 conn(0x7f86d815e980 0x7f86d8162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 -- 172.21.15.106:0/3106501785 >> 172.21.15.106:0/3106501785 conn(0x7f86d80d2550 msgr2=0x7f86d80d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 -- 172.21.15.106:0/3106501785 shutdown_connections 2024-04-13T03:11:04.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.806+0000 7f86e01bb640 1 -- 172.21.15.106:0/3106501785 wait complete. 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 Processor -- start 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 -- start start 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 0x7f86d80c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 0x7f86d80c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86d815fb60 con 0x7f86d80cd380 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86d815f880 con 0x7f86d815e980 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86e01bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f86d80d1e20 con 0x7f86d815ed60 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86ddf57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86ddf57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50404/0 (socket says 172.21.15.106:50404) 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.808+0000 7f86ddf57640 1 -- 172.21.15.106:0/3957194526 learned_addr learned my addr 172.21.15.106:0/3957194526 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86dd756640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 0x7f86d80c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86de758640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 0x7f86d80c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 -- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 msgr2=0x7f86d80c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 0x7f86d80c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 -- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 msgr2=0x7f86d80c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 0x7f86d80c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 -- 172.21.15.106:0/3957194526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d80c67a0 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86de758640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 0x7f86d80c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86dd756640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 0x7f86d80c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ddf57640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86d000ce10 tx=0x7f86d000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003b90 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d0003d30 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d80c6a80 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0004240 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.809+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d80c7040 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.810+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d8069120 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.811+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d00043e0 con 0x7f86d80cd380 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.811+0000 7f86ce7fc640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:04.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.811+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f86d008c840 con 0x7f86d80cd380 2024-04-13T03:11:04.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.811+0000 7f86dd756640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:04.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.811+0000 7f86dd756640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 0x7f86bc06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86c8036000 tx=0x7f86c803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:04.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:04.815+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86d001d340 con 0x7f86d80cd380 2024-04-13T03:11:05.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.026+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86d80b84d0 con 0x7f86d80cd380 2024-04-13T03:11:05.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.027+0000 7f86ce7fc640 1 -- 172.21.15.106:0/3957194526 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f86d005aae0 con 0x7f86d80cd380 2024-04-13T03:11:05.028 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:05.028 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:11:05.029 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:11:05.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 msgr2=0x7f86bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 0x7f86bc06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86c8036000 tx=0x7f86c803b040 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 msgr2=0x7f86d80c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86d000ce10 tx=0x7f86d000a660 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 shutdown_connections 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86d815ed60 0x7f86d80c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e980 0x7f86d80c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 --2- 172.21.15.106:0/3957194526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86d80cd380 0x7f86d80c7840 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 >> 172.21.15.106:0/3957194526 conn(0x7f86d80d2550 msgr2=0x7f86d8160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 shutdown_connections 2024-04-13T03:11:05.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.029+0000 7f86e01bb640 1 -- 172.21.15.106:0/3957194526 wait complete. 2024-04-13T03:11:05.041 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:05.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.140+0000 7fb9ab80f640 1 Processor -- start 2024-04-13T03:11:05.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.140+0000 7fb9ab80f640 1 -- start start 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 0x7fb9a406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40b5bf0 0x7fb9a406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a415fa70 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a40ce6f0 con 0x7fb9a415ebf0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9ab80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a415f420 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50406/0 (socket says 172.21.15.106:50406) 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 -- 172.21.15.106:0/1181227579 learned_addr learned my addr 172.21.15.106:0/1181227579 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a9dac640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40b5bf0 0x7fb9a406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a95ab640 1 --2- 172.21.15.106:0/1181227579 >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 0x7fb9a406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 -- 172.21.15.106:0/1181227579 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40b5bf0 msgr2=0x7fb9a406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:05.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40b5bf0 0x7fb9a406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 -- 172.21.15.106:0/1181227579 >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 msgr2=0x7fb9a406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 --2- 172.21.15.106:0/1181227579 >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 0x7fb9a406f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a8daa640 1 -- 172.21.15.106:0/1181227579 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a415f700 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.141+0000 7fb9a95ab640 1 --2- 172.21.15.106:0/1181227579 >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 0x7fb9a406f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9a8daa640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4071c70 tx=0x7fb9a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0edfd38ced01 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb99bfff640 1 -- 172.21.15.106:0/1181227579 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0035020 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb99bfff640 1 -- 172.21.15.106:0/1181227579 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a0031870 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb99bfff640 1 -- 172.21.15.106:0/1181227579 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0031c40 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 -- 172.21.15.106:0/1181227579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40ce1f0 msgr2=0x7fb9a406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4071c70 tx=0x7fb9a002b910 comp rx=0 tx=0).stop 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 -- 172.21.15.106:0/1181227579 shutdown_connections 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40b5bf0 0x7fb9a406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/1181227579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40ce1f0 0x7fb9a406fcc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/1181227579 >> v2:172.21.15.114:3300/0 conn(0x7fb9a415ebf0 0x7fb9a406f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.142+0000 7fb9ab80f640 1 -- 172.21.15.106:0/1181227579 >> 172.21.15.106:0/1181227579 conn(0x7fb9a40cd6d0 msgr2=0x7fb9a40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.143+0000 7fb9ab80f640 1 -- 172.21.15.106:0/1181227579 shutdown_connections 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.143+0000 7fb9ab80f640 1 -- 172.21.15.106:0/1181227579 wait complete. 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 Processor -- start 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 -- start start 2024-04-13T03:11:05.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40ce1f0 0x7fb9a4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a415ebf0 0x7fb9a40725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a40b0650 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a40b04d0 con 0x7fb9a415ebf0 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9ab80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a415efd0 con 0x7fb9a40ce1f0 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9a95ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9a95ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50416/0 (socket says 172.21.15.106:50416) 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.144+0000 7fb9a95ab640 1 -- 172.21.15.106:0/3945957302 learned_addr learned my addr 172.21.15.106:0/3945957302 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 -- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40ce1f0 msgr2=0x7fb9a4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40ce1f0 0x7fb9a4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 -- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a415ebf0 msgr2=0x7fb9a40725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a415ebf0 0x7fb9a40725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 -- 172.21.15.106:0/3945957302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a415c690 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9a95ab640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c820 tx=0x7fb99400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994015070 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb994007a30 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a415c970 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994007de0 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.145+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a415cf30 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.146+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a4068a10 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.147+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb994018070 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.147+0000 7fb999ffb640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 0x7fb98806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.147+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb99408bb80 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.147+0000 7fb9a8daa640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 0x7fb98806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.147+0000 7fb9a8daa640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 0x7fb98806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0036000 tx=0x7fb9a0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.151+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb994059e20 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.373+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a40d0530 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.373+0000 7fb999ffb640 1 -- 172.21.15.106:0/3945957302 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fb994059720 con 0x7fb9a40b5bf0 2024-04-13T03:11:05.375 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:05.375 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:10:37.420915+0000","modified":"2024-04-13T03:10:39.426685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5059":{"gid":5059,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/1027593505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1027593505},{"type":"v1","addr":"172.21.15.114:6835","nonce":1027593505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5059,"qdb_cluster":[5059]},"id":2}]} 2024-04-13T03:11:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 msgr2=0x7fb98806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 0x7fb98806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0036000 tx=0x7fb9a0030000 comp rx=0 tx=0).stop 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 msgr2=0x7fb9a40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c820 tx=0x7fb99400cce0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 shutdown_connections 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb98806c8f0 0x7fb98806edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb9a40ce1f0 0x7fb9a4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a415ebf0 0x7fb9a40725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 --2- 172.21.15.106:0/3945957302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9a40b5bf0 0x7fb9a40b2120 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 >> 172.21.15.106:0/3945957302 conn(0x7fb9a40cd6d0 msgr2=0x7fb9a40ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 shutdown_connections 2024-04-13T03:11:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.376+0000 7fb9ab80f640 1 -- 172.21.15.106:0/3945957302 wait complete. 2024-04-13T03:11:05.388 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.487+0000 7fe13898c640 1 Processor -- start 2024-04-13T03:11:05.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.487+0000 7fe13898c640 1 -- start start 2024-04-13T03:11:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13898c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe13406cb80 0x7fe1340726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13898c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe134073090 con 0x7fe1340cdf50 2024-04-13T03:11:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13898c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe134073370 con 0x7fe134072c10 2024-04-13T03:11:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13898c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe134073650 con 0x7fe13406cb80 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe131d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe13406cb80 0x7fe1340726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe1340cdf50 0x7fe1340ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39440/0 (socket says 172.21.15.106:39440) 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13259c640 1 -- 172.21.15.106:0/2369446497 learned_addr learned my addr 172.21.15.106:0/2369446497 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe131d9b640 1 -- 172.21.15.106:0/2369446497 >> v1:172.21.15.114:6789/0 conn(0x7fe134072c10 legacy=0x7fe13406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe131d9b640 1 -- 172.21.15.106:0/2369446497 >> v1:172.21.15.106:6789/0 conn(0x7fe1340cdf50 legacy=0x7fe1340ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe13159a640 1 -- 172.21.15.106:0/2369446497 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950658867 0 0) 0x7fe134073090 con 0x7fe1340cdf50 2024-04-13T03:11:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.488+0000 7fe131d9b640 1 -- 172.21.15.106:0/2369446497 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe134073930 con 0x7fe13406cb80 2024-04-13T03:11:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.489+0000 7fe131d9b640 1 --2- 172.21.15.106:0/2369446497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13406cb80 0x7fe1340726d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097a0 tx=0x7fe12802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906925228c67c325 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.489+0000 7fe13159a640 1 -- 172.21.15.106:0/2369446497 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280315a0 con 0x7fe13406cb80 2024-04-13T03:11:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.489+0000 7fe13159a640 1 -- 172.21.15.106:0/2369446497 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe128037070 con 0x7fe13406cb80 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.489+0000 7fe13159a640 1 -- 172.21.15.106:0/2369446497 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128031960 con 0x7fe13406cb80 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 -- 172.21.15.106:0/2369446497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13406cb80 msgr2=0x7fe1340726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 --2- 172.21.15.106:0/2369446497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13406cb80 0x7fe1340726d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097a0 tx=0x7fe12802b840 comp rx=0 tx=0).stop 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 -- 172.21.15.106:0/2369446497 shutdown_connections 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 --2- 172.21.15.106:0/2369446497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13406cb80 0x7fe1340726d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 -- 172.21.15.106:0/2369446497 >> 172.21.15.106:0/2369446497 conn(0x7fe1340b1f10 msgr2=0x7fe1340b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 -- 172.21.15.106:0/2369446497 shutdown_connections 2024-04-13T03:11:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.490+0000 7fe13898c640 1 -- 172.21.15.106:0/2369446497 wait complete. 2024-04-13T03:11:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.491+0000 7fe13898c640 1 Processor -- start 2024-04-13T03:11:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.491+0000 7fe13898c640 1 -- start start 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 0x7fe134156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 0x7fe13415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe13406ea00 con 0x7fe1340cdf50 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe13406e880 con 0x7fe13406cb80 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13898c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe13406eb80 con 0x7fe134072c10 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50430/0 (socket says 172.21.15.106:50430) 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 -- 172.21.15.106:0/1026323859 learned_addr learned my addr 172.21.15.106:0/1026323859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13259c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 0x7fe134156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe131d9b640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 0x7fe13415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 -- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 msgr2=0x7fe13415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 0x7fe13415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 -- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 msgr2=0x7fe134156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 0x7fe134156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 -- 172.21.15.106:0/1026323859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1341568d0 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe13259c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 0x7fe134156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe131d9b640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 0x7fe13415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.492+0000 7fe132d9d640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c7b0 tx=0x7fe12c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c017070 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c003da0 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1341f4e70 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0052b0 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13415bc30 con 0x7fe1340cdf50 2024-04-13T03:11:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.493+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe134068a10 con 0x7fe1340cdf50 2024-04-13T03:11:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.494+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe12c01a070 con 0x7fe1340cdf50 2024-04-13T03:11:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.494+0000 7fe122ffd640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 0x7fe11406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.495+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fe12c08b2b0 con 0x7fe1340cdf50 2024-04-13T03:11:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.495+0000 7fe13259c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 0x7fe11406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.495+0000 7fe13259c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 0x7fe11406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe124000f60 tx=0x7fe124007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.499+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1341568d0 con 0x7fe1340cdf50 2024-04-13T03:11:05.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.675+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe134099aa0 con 0x7fe1340cdf50 2024-04-13T03:11:05.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.676+0000 7fe122ffd640 1 -- 172.21.15.106:0/1026323859 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17027 (secure 0 0 0) 0x7fe12c0595c0 con 0x7fe1340cdf50 2024-04-13T03:11:05.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:05.677 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":40,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:05.211999+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:10:35.635640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-13T03:10:36.371611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 msgr2=0x7fe11406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 0x7fe11406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe124000f60 tx=0x7fe124007ca0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 msgr2=0x7fe1341f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c7b0 tx=0x7fe12c00cc70 comp rx=0 tx=0).stop 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 shutdown_connections 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe11406c8f0 0x7fe11406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe134072c10 0x7fe13415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1340cdf50 0x7fe1341f4500 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 --2- 172.21.15.106:0/1026323859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13406cb80 0x7fe134156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 >> 172.21.15.106:0/1026323859 conn(0x7fe1340b1f10 msgr2=0x7fe13406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 shutdown_connections 2024-04-13T03:11:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.678+0000 7fe13898c640 1 -- 172.21.15.106:0/1026323859 wait complete. 2024-04-13T03:11:05.690 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:11:05.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.790+0000 7f265771e640 1 Processor -- start 2024-04-13T03:11:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f265771e640 1 -- start start 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f265771e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2650070bb0 0x7f26501697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f265771e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26500b5cf0 con 0x7f26500b1c30 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f265771e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26500b5fd0 con 0x7f265015ee40 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f265771e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f26500b62b0 con 0x7f2650070bb0 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f26554ba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f26500b1c30 0x7f26500b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39452/0 (socket says 172.21.15.106:39452) 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.791+0000 7f26554ba640 1 -- 172.21.15.106:0/1259665550 learned_addr learned my addr 172.21.15.106:0/1259665550 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2655cbb640 1 --2- 172.21.15.106:0/1259665550 >> v2:172.21.15.114:3300/0 conn(0x7f2650070bb0 0x7f26501697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312717654 0 0) 0x7f26500b5cf0 con 0x7f26500b1c30 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f263c003520 con 0x7f26500b1c30 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3558392730 0 0) 0x7f263c003520 con 0x7f26500b1c30 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2655cbb640 1 -- 172.21.15.106:0/1259665550 >> v1:172.21.15.114:6789/0 conn(0x7f265015ee40 legacy=0x7f26500704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2655cbb640 1 -- 172.21.15.106:0/1259665550 >> v1:172.21.15.106:6789/0 conn(0x7f26500b1c30 legacy=0x7f26500b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.792+0000 7f2655cbb640 1 -- 172.21.15.106:0/1259665550 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500b6590 con 0x7f2650070bb0 2024-04-13T03:11:05.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f2655cbb640 1 --2- 172.21.15.106:0/1259665550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f26501697c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6340cce659969a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648038070 con 0x7f2650070bb0 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648031500 con 0x7f2650070bb0 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f2647fff640 1 -- 172.21.15.106:0/1259665550 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0019f0 con 0x7f2650070bb0 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f265771e640 1 -- 172.21.15.106:0/1259665550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 msgr2=0x7f26501697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.793+0000 7f265771e640 1 --2- 172.21.15.106:0/1259665550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f26501697c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802b820 comp rx=0 tx=0).stop 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.794+0000 7f265771e640 1 -- 172.21.15.106:0/1259665550 shutdown_connections 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.794+0000 7f265771e640 1 --2- 172.21.15.106:0/1259665550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f26501697c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.794+0000 7f265771e640 1 -- 172.21.15.106:0/1259665550 >> 172.21.15.106:0/1259665550 conn(0x7f2650071870 msgr2=0x7f2650071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.794+0000 7f265771e640 1 -- 172.21.15.106:0/1259665550 shutdown_connections 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.794+0000 7f265771e640 1 -- 172.21.15.106:0/1259665550 wait complete. 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 Processor -- start 2024-04-13T03:11:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 -- start start 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f265014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26500b1c30 0x7f2650155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2650169d00 con 0x7f265015ee40 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f265006e880 con 0x7f2650070bb0 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.795+0000 7f265771e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2650169e80 con 0x7f26500b1c30 2024-04-13T03:11:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50436/0 (socket says 172.21.15.106:50436) 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 -- 172.21.15.106:0/2597769139 learned_addr learned my addr 172.21.15.106:0/2597769139 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f26554ba640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f265014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2654cb9640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26500b1c30 0x7f2650155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 -- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26500b1c30 msgr2=0x7f2650155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26500b1c30 0x7f2650155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 -- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 msgr2=0x7f265014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f265014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 -- 172.21.15.106:0/2597769139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2650156270 con 0x7f265015ee40 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f26554ba640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f265014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2655cbb640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f26480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648043070 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264803f870 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265013ab50 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648038040 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.796+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265013b110 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.797+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650068a10 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.798+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2648046070 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.798+0000 7f2645ffb640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 0x7f263406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.798+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f26480ad1e0 con 0x7f265015ee40 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.798+0000 7f26554ba640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 0x7f263406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.799+0000 7f26554ba640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 0x7f263406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f264c001560 tx=0x7f264c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:05.803+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f264804d020 con 0x7f265015ee40 2024-04-13T03:11:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.015+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f26500d2140 con 0x7f265015ee40 2024-04-13T03:11:06.016 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:11:06.015+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:11:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.017+0000 7f2645ffb640 1 -- 172.21.15.106:0/2597769139 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f2648030070 con 0x7f265015ee40 2024-04-13T03:11:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:11:06.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 msgr2=0x7f263406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 0x7f263406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f264c001560 tx=0x7f264c009040 comp rx=0 tx=0).stop 2024-04-13T03:11:06.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 msgr2=0x7f2650159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f26480046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 shutdown_connections 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f263406c920 0x7f263406ede0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26500b1c30 0x7f2650155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265015ee40 0x7f2650159ca0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 --2- 172.21.15.106:0/2597769139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2650070bb0 0x7f265014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 >> 172.21.15.106:0/2597769139 conn(0x7f2650071870 msgr2=0x7f265015c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 shutdown_connections 2024-04-13T03:11:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.020+0000 7f265771e640 1 -- 172.21.15.106:0/2597769139 wait complete. 2024-04-13T03:11:06.024 INFO:tasks.ceph.mds.d.smithi114.stderr: -6633> 2024-04-13T03:10:42.769+0000 7f7d439f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:11:06.033 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:11:06.057 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:11:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.133+0000 7fda5398c640 1 Processor -- start 2024-04-13T03:11:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.133+0000 7fda5398c640 1 -- start start 2024-04-13T03:11:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda5398c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda4c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda5398c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4c0b5060 con 0x7fda4c15eca0 2024-04-13T03:11:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda5398c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4c0b5340 con 0x7fda4c073520 2024-04-13T03:11:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda5398c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c06c500 con 0x7fda4c15e8c0 2024-04-13T03:11:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda50f27640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fda4c15eca0 0x7fda4c072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39464/0 (socket says 172.21.15.106:39464) 2024-04-13T03:11:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.134+0000 7fda50f27640 1 -- 172.21.15.106:0/3797672264 learned_addr learned my addr 172.21.15.106:0/3797672264 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda4c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944734578 0 0) 0x7fda4c0b5060 con 0x7fda4c15eca0 2024-04-13T03:11:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda38003520 con 0x7fda4c15eca0 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1491012132 0 0) 0x7fda38003520 con 0x7fda4c15eca0 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 msgr2=0x7fda4c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda4c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 >> v1:172.21.15.114:6789/0 conn(0x7fda4c073520 legacy=0x7fda4c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c06c7e0 con 0x7fda4c15eca0 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fda3c001670 con 0x7fda4c15eca0 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 >> v1:172.21.15.106:6789/0 conn(0x7fda4c15eca0 legacy=0x7fda4c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda380039e0 0x7fda38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda38004800 0x7fda38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c06c500 con 0x7fda380039e0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda3800bb00 con 0x7fda38004800 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda3800bd00 con 0x7fda4c15e8c0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda380039e0 0x7fda38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda50f27640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda38004800 0x7fda38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51f29640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 -- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 msgr2=0x7fda380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda4c15e8c0 0x7fda380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 -- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda38004800 msgr2=0x7fda38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda38004800 0x7fda38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.135+0000 7fda51728640 1 -- 172.21.15.106:0/3797672264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda38003520 con 0x7fda380039e0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.136+0000 7fda51728640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda380039e0 0x7fda38003e00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fda48009f90 tx=0x7fda38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c699b1080e10589 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.136+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48039070 con 0x7fda380039e0 2024-04-13T03:11:06.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.136+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4802cbd0 con 0x7fda380039e0 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.136+0000 7fda43fff640 1 -- 172.21.15.106:0/3797672264 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48009c10 con 0x7fda380039e0 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 -- 172.21.15.106:0/3797672264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda380039e0 msgr2=0x7fda38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 --2- 172.21.15.106:0/3797672264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda380039e0 0x7fda38003e00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fda48009f90 tx=0x7fda38005830 comp rx=0 tx=0).stop 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda51f29640 1 -- 172.21.15.106:0/3797672264 reap_dead start 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 -- 172.21.15.106:0/3797672264 shutdown_connections 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 -- 172.21.15.106:0/3797672264 >> 172.21.15.106:0/3797672264 conn(0x7fda4c06f0d0 msgr2=0x7fda4c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 -- 172.21.15.106:0/3797672264 shutdown_connections 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.137+0000 7fda5398c640 1 -- 172.21.15.106:0/3797672264 wait complete. 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.138+0000 7fda5398c640 1 Processor -- start 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.138+0000 7fda5398c640 1 -- start start 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda380039e0 0x7fda4c156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda4c1f4b30 0x7fda4c1f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c0ce250 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c0b4d50 con 0x7fda4c1f4b30 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c0ce3d0 con 0x7fda380039e0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50460/0 (socket says 172.21.15.106:50460) 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 -- 172.21.15.106:0/733632024 learned_addr learned my addr 172.21.15.106:0/733632024 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda50f27640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda4c1f4b30 0x7fda4c1f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51728640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda380039e0 0x7fda4c156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 -- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda380039e0 msgr2=0x7fda4c156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda380039e0 0x7fda4c156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 -- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda4c1f4b30 msgr2=0x7fda4c1f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda4c1f4b30 0x7fda4c1f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 -- 172.21.15.106:0/733632024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c157320 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda51f29640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0ce7c0 tx=0x7fda4400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:06.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44019070 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda44007e40 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c1613f0 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.139+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440035f0 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.140+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1619b0 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.141+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c068a10 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.141+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda4401c070 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.141+0000 7fda41ffb640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 0x7fda3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.142+0000 7fda51728640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 0x7fda3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.142+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fda4408afd0 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.142+0000 7fda51728640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 0x7fda3006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda48036000 tx=0x7fda4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:06.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.146+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda4c157320 con 0x7fda4c0cc7c0 2024-04-13T03:11:06.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:06.362+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fda4c0cdbf0 con 0x7fda4c0cc7c0 2024-04-13T03:11:07.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.018+0000 7fda41ffb640 1 -- 172.21.15.106:0/733632024 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fda44058c40 con 0x7fda4c0cc7c0 2024-04-13T03:11:07.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 msgr2=0x7fda3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 0x7fda3006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda48036000 tx=0x7fda4802e000 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 msgr2=0x7fda4c0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0ce7c0 tx=0x7fda4400e7e0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 shutdown_connections 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda3006c8f0 0x7fda3006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda4c0cc7c0 0x7fda4c0ccc40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda4c1f4b30 0x7fda4c1f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 --2- 172.21.15.106:0/733632024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda380039e0 0x7fda4c156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 >> 172.21.15.106:0/733632024 conn(0x7fda4c06f0d0 msgr2=0x7fda4c0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 shutdown_connections 2024-04-13T03:11:07.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.021+0000 7fda5398c640 1 -- 172.21.15.106:0/733632024 wait complete. 2024-04-13T03:11:07.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:11:07.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.143+0000 7fa2830d5640 1 Processor -- start 2024-04-13T03:11:07.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.143+0000 7fa2830d5640 1 -- start start 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa2830d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa27c1677f0 0x7fa27c169be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa2830d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27c06c700 con 0x7fa27c15e880 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa2830d5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27c0cfc60 con 0x7fa27c0cf7d0 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa2830d5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa27c0d30a0 con 0x7fa27c1677f0 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa280e71640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa27c15e880 0x7fa27c06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39466/0 (socket says 172.21.15.106:39466) 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa280e71640 1 -- 172.21.15.106:0/1217914859 learned_addr learned my addr 172.21.15.106:0/1217914859 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834402859 0 0) 0x7fa27c06c700 con 0x7fa27c15e880 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa281672640 1 --2- 172.21.15.106:0/1217914859 >> v2:172.21.15.114:3300/0 conn(0x7fa27c1677f0 0x7fa27c169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.144+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa264003520 con 0x7fa27c15e880 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2591688167 0 0) 0x7fa264003520 con 0x7fa27c15e880 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 >> v2:172.21.15.114:3300/0 conn(0x7fa27c1677f0 msgr2=0x7fa27c169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 --2- 172.21.15.106:0/1217914859 >> v2:172.21.15.114:3300/0 conn(0x7fa27c1677f0 0x7fa27c169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 >> v1:172.21.15.114:6789/0 conn(0x7fa27c0cf7d0 legacy=0x7fa27c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c0d2a50 con 0x7fa27c15e880 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa26c001670 con 0x7fa27c15e880 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 >> v1:172.21.15.106:6789/0 conn(0x7fa27c15e880 legacy=0x7fa27c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2640039b0 0x7fa264003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c0d30a0 con 0x7fa2640051c0 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa26400c160 con 0x7fa2640039b0 2024-04-13T03:11:07.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa26400c360 con 0x7fa264004800 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa280e71640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2640039b0 0x7fa264003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27bfff640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 -- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa264004800 msgr2=0x7fa264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 -- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2640039b0 msgr2=0x7fa264003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2640039b0 0x7fa264003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa281672640 1 -- 172.21.15.106:0/1217914859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa264003520 con 0x7fa2640051c0 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa280e71640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2640039b0 0x7fa264003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.145+0000 7fa27bfff640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.146+0000 7fa281672640 1 -- 172.21.15.106:0/1217914859 reap_dead start 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.146+0000 7fa281672640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 0x7fa26400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa274009f90 tx=0x7fa264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fa497b43f6eb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:07.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.146+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274009c10 con 0x7fa2640051c0 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.146+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27402cc20 con 0x7fa2640051c0 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.146+0000 7fa27b7fe640 1 -- 172.21.15.106:0/1217914859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274003990 con 0x7fa2640051c0 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 -- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 msgr2=0x7fa26400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 0x7fa26400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa274009f90 tx=0x7fa264005830 comp rx=0 tx=0).stop 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 -- 172.21.15.106:0/1217914859 shutdown_connections 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 --2- 172.21.15.106:0/1217914859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 -- 172.21.15.106:0/1217914859 >> 172.21.15.106:0/1217914859 conn(0x7fa27c0b6280 msgr2=0x7fa27c0b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 -- 172.21.15.106:0/1217914859 shutdown_connections 2024-04-13T03:11:07.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.147+0000 7fa2830d5640 1 -- 172.21.15.106:0/1217914859 wait complete. 2024-04-13T03:11:07.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.148+0000 7fa2830d5640 1 Processor -- start 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.148+0000 7fa2830d5640 1 -- start start 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2640051c0 0x7fa27c14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa27c160410 0x7fa27c160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c0b3a50 con 0x7fa27c15d8d0 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c15ec60 con 0x7fa27c160410 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa27c0b3bd0 con 0x7fa2640051c0 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50480/0 (socket says 172.21.15.106:50480) 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 -- 172.21.15.106:0/2521140434 learned_addr learned my addr 172.21.15.106:0/2521140434 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:07.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 -- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2640051c0 msgr2=0x7fa27c14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2640051c0 0x7fa27c14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 -- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa27c160410 msgr2=0x7fa27c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa27c160410 0x7fa27c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 -- 172.21.15.106:0/2521140434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c15ef10 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa27bfff640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e460 tx=0x7fa27000e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270019070 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2700052e0 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c15f1f0 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270003830 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.149+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c15f7b0 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.150+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c068a10 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.151+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa27001c070 con 0x7fa27c15d8d0 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.151+0000 7fa2797fa640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 0x7fa26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:07.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.151+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fa27008b1e0 con 0x7fa27c15d8d0 2024-04-13T03:11:07.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.151+0000 7fa280e71640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 0x7fa26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:07.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.152+0000 7fa280e71640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 0x7fa26806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa26c001eb0 tx=0x7fa26c001190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:07.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.156+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa270058dc0 con 0x7fa27c15d8d0 2024-04-13T03:11:07.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:07.332+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa27c155dd0 con 0x7fa27c15d8d0 2024-04-13T03:11:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.228+0000 7fa2797fa640 1 -- 172.21.15.106:0/2521140434 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7fa27005d0c0 con 0x7fa27c15d8d0 2024-04-13T03:11:08.230 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 msgr2=0x7fa26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 0x7fa26806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa26c001eb0 tx=0x7fa26c001190 comp rx=0 tx=0).stop 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 msgr2=0x7fa27c15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e460 tx=0x7fa27000e920 comp rx=0 tx=0).stop 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 shutdown_connections 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa26806c8f0 0x7fa26806edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa27c160410 0x7fa27c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa27c15d8d0 0x7fa27c15dd50 secure :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e460 tx=0x7fa27000e920 comp rx=0 tx=0).stop 2024-04-13T03:11:08.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 --2- 172.21.15.106:0/2521140434 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2640051c0 0x7fa27c14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 >> 172.21.15.106:0/2521140434 conn(0x7fa27c0b6280 msgr2=0x7fa27c158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:08.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 shutdown_connections 2024-04-13T03:11:08.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.232+0000 7fa2830d5640 1 -- 172.21.15.106:0/2521140434 wait complete. 2024-04-13T03:11:08.247 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:11:08.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 Processor -- start 2024-04-13T03:11:08.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 -- start start 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc2ac0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac161170 con 0x7fc2ac145c30 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac160bb0 con 0x7fc2ac146010 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.356+0000 7fc2b282d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac160950 con 0x7fc2ac0cd100 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2abfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc2ac145c30 0x7fc2ac160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39474/0 (socket says 172.21.15.106:39474) 2024-04-13T03:11:08.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2abfff640 1 -- 172.21.15.106:0/4195297100 learned_addr learned my addr 172.21.15.106:0/4195297100 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642055976 0 0) 0x7fc2ac161170 con 0x7fc2ac145c30 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc294003520 con 0x7fc2ac145c30 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2ab7fe640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc2ac0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3437152981 0 0) 0x7fc294003520 con 0x7fc2ac145c30 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 msgr2=0x7fc2ac0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc2ac0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 >> v1:172.21.15.114:6789/0 conn(0x7fc2ac146010 legacy=0x7fc2ac0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac160e90 con 0x7fc2ac145c30 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc29c001670 con 0x7fc2ac145c30 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 >> v1:172.21.15.106:6789/0 conn(0x7fc2ac145c30 legacy=0x7fc2ac160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2940039e0 0x7fc294003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc294004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc294004d40 0x7fc2940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac160950 con 0x7fc294004d40 2024-04-13T03:11:08.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400bb50 con 0x7fc2940039e0 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc29400bd20 con 0x7fc2ac0cd100 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc294004d40 0x7fc2940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2abfff640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2940039e0 0x7fc294003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.357+0000 7fc2ab7fe640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc294004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 -- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 msgr2=0x7fc294004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac0cd100 0x7fc294004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 -- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2940039e0 msgr2=0x7fc294003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2940039e0 0x7fc294003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 -- 172.21.15.106:0/4195297100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc294003520 con 0x7fc294004d40 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc294004d40 0x7fc2940051c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001e80 tx=0x7fc294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e778c93fb971c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4001a70 con 0x7fc294004d40 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a4003b70 con 0x7fc294004d40 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.358+0000 7fc2aaffd640 1 -- 172.21.15.106:0/4195297100 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40050d0 con 0x7fc294004d40 2024-04-13T03:11:08.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 -- 172.21.15.106:0/4195297100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc294004d40 msgr2=0x7fc2940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 --2- 172.21.15.106:0/4195297100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc294004d40 0x7fc2940051c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001e80 tx=0x7fc294005830 comp rx=0 tx=0).stop 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b0dca640 1 -- 172.21.15.106:0/4195297100 reap_dead start 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 -- 172.21.15.106:0/4195297100 shutdown_connections 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 -- 172.21.15.106:0/4195297100 >> 172.21.15.106:0/4195297100 conn(0x7fc2ac0d0370 msgr2=0x7fc2ac0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 -- 172.21.15.106:0/4195297100 shutdown_connections 2024-04-13T03:11:08.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.359+0000 7fc2b282d640 1 -- 172.21.15.106:0/4195297100 wait complete. 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.360+0000 7fc2b282d640 1 Processor -- start 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.360+0000 7fc2b282d640 1 -- start start 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2ac15ae20 0x7fc2ac06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac15b200 0x7fc2ac0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac06b8e0 con 0x7fc2ac15bc40 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0b3c00 con 0x7fc2ac15ae20 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b282d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac06ba60 con 0x7fc2ac15b200 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50490/0 (socket says 172.21.15.106:50490) 2024-04-13T03:11:08.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 -- 172.21.15.106:0/3565147735 learned_addr learned my addr 172.21.15.106:0/3565147735 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2abfff640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2ac15ae20 0x7fc2ac06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2ab7fe640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac15b200 0x7fc2ac0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 -- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac15b200 msgr2=0x7fc2ac0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac15b200 0x7fc2ac0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 -- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2ac15ae20 msgr2=0x7fc2ac06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.361+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2ac15ae20 0x7fc2ac06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2b0dca640 1 -- 172.21.15.106:0/3565147735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac06e310 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2b0dca640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001fa0 tx=0x7fc2a40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4001a70 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a4006b50 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4007db0 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac13d740 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.362+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac13dd00 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.363+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac068a10 con 0x7fc2ac15bc40 2024-04-13T03:11:08.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.363+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2a4007030 con 0x7fc2ac15bc40 2024-04-13T03:11:08.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.364+0000 7fc2a8ff9640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 0x7fc29806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:08.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.364+0000 7fc2abfff640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 0x7fc29806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:08.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.364+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5778+0+0 (secure 0 0 0) 0x7fc2a40acfd0 con 0x7fc2ac15bc40 2024-04-13T03:11:08.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.364+0000 7fc2abfff640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 0x7fc29806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000c00 tx=0x7fc29c0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:08.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.368+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a404a030 con 0x7fc2ac15bc40 2024-04-13T03:11:08.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:08.547+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc2ac144150 con 0x7fc2ac15bc40 2024-04-13T03:11:09.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.233+0000 7fc2a8ff9640 1 -- 172.21.15.106:0/3565147735 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7fc2a4033020 con 0x7fc2ac15bc40 2024-04-13T03:11:09.234 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.238+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 msgr2=0x7fc29806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.238+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 0x7fc29806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000c00 tx=0x7fc29c0078f0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.238+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 msgr2=0x7fc2ac0af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.238+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001fa0 tx=0x7fc2a40014a0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 shutdown_connections 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc29806c8f0 0x7fc29806edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc2ac15b200 0x7fc2ac0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ac15bc40 0x7fc2ac0af480 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 --2- 172.21.15.106:0/3565147735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2ac15ae20 0x7fc2ac06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 >> 172.21.15.106:0/3565147735 conn(0x7fc2ac0d0370 msgr2=0x7fc2ac0b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 shutdown_connections 2024-04-13T03:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.239+0000 7fc2b282d640 1 -- 172.21.15.106:0/3565147735 wait complete. 2024-04-13T03:11:09.257 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:11:09.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.363+0000 7f048a076640 1 Processor -- start 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.363+0000 7f048a076640 1 -- start start 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f048a076640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f04840af800 0x7f048415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f048a076640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f048a076640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048415c200 con 0x7f048406f200 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f048a076640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f04840b5090 con 0x7f04840af800 2024-04-13T03:11:09.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f048a076640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f048406fb00 con 0x7f048406f690 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f0483fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f04837fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f04840af800 0x7f048415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f0483fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:39992/0 (socket says 172.21.15.106:39992) 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f0482ffd640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f048406f200 0x7f048415e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:41212/0 (socket says 172.21.15.106:41212) 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f0483fff640 1 -- 172.21.15.106:0/3556586172 learned_addr learned my addr 172.21.15.106:0/3556586172 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f04837fe640 1 -- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 msgr2=0x7f048415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f04837fe640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f04837fe640 1 -- 172.21.15.106:0/3556586172 >> v1:172.21.15.114:6789/0 conn(0x7f048406f200 legacy=0x7f048415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f04837fe640 1 -- 172.21.15.106:0/3556586172 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04840b3ff0 con 0x7f04840af800 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.364+0000 7f0483fff640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.365+0000 7f04837fe640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04840af800 0x7f048415bcc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04740097a0 tx=0x7f047402dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910cc6ccf76ebe96 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.365+0000 7f04827fc640 1 -- 172.21.15.106:0/3556586172 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047402f8a0 con 0x7f04840af800 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.365+0000 7f04827fc640 1 -- 172.21.15.106:0/3556586172 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047402fa40 con 0x7f04840af800 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.365+0000 7f04827fc640 1 -- 172.21.15.106:0/3556586172 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047402fe10 con 0x7f04840af800 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 -- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04840af800 msgr2=0x7f048415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04840af800 0x7f048415bcc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04740097a0 tx=0x7f047402dae0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 -- 172.21.15.106:0/3556586172 shutdown_connections 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f048415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 --2- 172.21.15.106:0/3556586172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04840af800 0x7f048415bcc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 -- 172.21.15.106:0/3556586172 >> 172.21.15.106:0/3556586172 conn(0x7f04840b2600 msgr2=0x7f04840d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 -- 172.21.15.106:0/3556586172 shutdown_connections 2024-04-13T03:11:09.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.366+0000 7f048a076640 1 -- 172.21.15.106:0/3556586172 wait complete. 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.367+0000 7f048a076640 1 Processor -- start 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.367+0000 7f048a076640 1 -- start start 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f048406f200 0x7f04841f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f04841f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04840b4890 con 0x7f04840af800 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04840b45b0 con 0x7f048406f200 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f048a076640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f04840b4b70 con 0x7f048406f690 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f0483fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f0483fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50500/0 (socket says 172.21.15.106:50500) 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f0483fff640 1 -- 172.21.15.106:0/2976495174 learned_addr learned my addr 172.21.15.106:0/2976495174 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f04837fe640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f048406f200 0x7f04841f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.368+0000 7f0482ffd640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f04841f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 -- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 msgr2=0x7f04841f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f04841f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 -- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f048406f200 msgr2=0x7f04841f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f048406f200 0x7f04841f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 -- 172.21.15.106:0/2976495174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04841f4b50 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f0483fff640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f047c00efc0 tx=0x7f047c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00cbe0 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047c00cd80 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c003bb0 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04841f0bb0 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.369+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04841f1170 con 0x7f04840af800 2024-04-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.370+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0484068a10 con 0x7f04840af800 2024-04-13T03:11:09.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.370+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f047c01b030 con 0x7f04840af800 2024-04-13T03:11:09.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.371+0000 7f046bfff640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 0x7f047006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.371+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f047c08bc90 con 0x7f04840af800 2024-04-13T03:11:09.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.371+0000 7f04837fe640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 0x7f047006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.371+0000 7f04837fe640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 0x7f047006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0474038000 tx=0x7f0474008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.375+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f047c022020 con 0x7f04840af800 2024-04-13T03:11:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.550+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f04841f1510 con 0x7f04840af800 2024-04-13T03:11:09.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.554+0000 7f046bfff640 1 -- 172.21.15.106:0/2976495174 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f04841f4b50 con 0x7f04840af800 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 msgr2=0x7f047006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 0x7f047006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0474038000 tx=0x7f0474008030 comp rx=0 tx=0).stop 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 msgr2=0x7f04840cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f047c00efc0 tx=0x7f047c00c490 comp rx=0 tx=0).stop 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 shutdown_connections 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f047006c8f0 0x7f047006edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f048406f690 0x7f04841f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04840af800 0x7f04840cecc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 --2- 172.21.15.106:0/2976495174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f048406f200 0x7f04841f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 >> 172.21.15.106:0/2976495174 conn(0x7f04840b2600 msgr2=0x7f048406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:09.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 shutdown_connections 2024-04-13T03:11:09.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.556+0000 7f048a076640 1 -- 172.21.15.106:0/2976495174 wait complete. 2024-04-13T03:11:09.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-13T03:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.671+0000 7fbb69d8c640 1 Processor -- start 2024-04-13T03:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.671+0000 7fbb69d8c640 1 -- start start 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb69d8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb6415ce80 0x7fbb640d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb69d8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640b7f70 0x7fbb640d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb69d8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6406fd00 con 0x7fbb6415caa0 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb69d8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbb640714b0 con 0x7fbb6415ce80 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb69d8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406ffe0 con 0x7fbb640b7f70 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb62ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbb6415ce80 0x7fbb640d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb637fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbb6415caa0 0x7fbb64070da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39476/0 (socket says 172.21.15.106:39476) 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb637fe640 1 -- 172.21.15.106:0/2005036555 learned_addr learned my addr 172.21.15.106:0/2005036555 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb63fff640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640b7f70 0x7fbb640d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb62ffd640 1 -- 172.21.15.106:0/2005036555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640b7f70 msgr2=0x7fbb640d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb62ffd640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640b7f70 0x7fbb640d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb62ffd640 1 -- 172.21.15.106:0/2005036555 >> v1:172.21.15.106:6789/0 conn(0x7fbb6415caa0 legacy=0x7fbb64070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.672+0000 7fbb62ffd640 1 -- 172.21.15.106:0/2005036555 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb640702c0 con 0x7fbb6415ce80 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.673+0000 7fbb62ffd640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb6415ce80 0x7fbb640d14e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097a0 tx=0x7fbb5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390b957d6ad92c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.673+0000 7fbb627fc640 1 -- 172.21.15.106:0/2005036555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58035020 con 0x7fbb6415ce80 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.673+0000 7fbb627fc640 1 -- 172.21.15.106:0/2005036555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58031830 con 0x7fbb6415ce80 2024-04-13T03:11:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.673+0000 7fbb627fc640 1 -- 172.21.15.106:0/2005036555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031c00 con 0x7fbb6415ce80 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 -- 172.21.15.106:0/2005036555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb6415ce80 msgr2=0x7fbb640d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb6415ce80 0x7fbb640d14e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097a0 tx=0x7fbb5802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 -- 172.21.15.106:0/2005036555 shutdown_connections 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640b7f70 0x7fbb640d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/2005036555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb6415ce80 0x7fbb640d14e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 -- 172.21.15.106:0/2005036555 >> 172.21.15.106:0/2005036555 conn(0x7fbb640d25d0 msgr2=0x7fbb640d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 -- 172.21.15.106:0/2005036555 shutdown_connections 2024-04-13T03:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.674+0000 7fbb69d8c640 1 -- 172.21.15.106:0/2005036555 wait complete. 2024-04-13T03:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.675+0000 7fbb69d8c640 1 Processor -- start 2024-04-13T03:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.675+0000 7fbb69d8c640 1 -- start start 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb6415caa0 0x7fbb6415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640c5640 0x7fbb640ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6406f990 con 0x7fbb640b7f70 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64070880 con 0x7fbb6415caa0 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406f490 con 0x7fbb640c5640 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50522/0 (socket says 172.21.15.106:50522) 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 -- 172.21.15.106:0/3430693687 learned_addr learned my addr 172.21.15.106:0/3430693687 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 -- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640c5640 msgr2=0x7fbb640ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640c5640 0x7fbb640ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 -- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb6415caa0 msgr2=0x7fbb6415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb6415caa0 0x7fbb6415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 -- 172.21.15.106:0/3430693687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6415e630 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb637fe640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c430 tx=0x7fbb5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54017070 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb54005ce0 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540072f0 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6415e910 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.676+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6415eed0 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.677+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb64068a10 con 0x7fbb640b7f70 2024-04-13T03:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.678+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb5401a070 con 0x7fbb640b7f70 2024-04-13T03:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.678+0000 7fbb4bfff640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 0x7fbb5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.678+0000 7fbb62ffd640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 0x7fbb5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.679+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fbb5408ae00 con 0x7fbb640b7f70 2024-04-13T03:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.679+0000 7fbb62ffd640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 0x7fbb5006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb58036000 tx=0x7fbb58030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.683+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb540593a0 con 0x7fbb640b7f70 2024-04-13T03:11:09.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:09.925+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fbb6415be20 con 0x7fbb640b7f70 2024-04-13T03:11:10.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.238+0000 7fbb4bfff640 1 -- 172.21.15.106:0/3430693687 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v119) ==== 142+0+0 (secure 0 0 0) 0x7fbb54058ca0 con 0x7fbb640b7f70 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 msgr2=0x7fbb5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 0x7fbb5006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb58036000 tx=0x7fbb58030000 comp rx=0 tx=0).stop 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 msgr2=0x7fbb64151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c430 tx=0x7fbb5400c8f0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 shutdown_connections 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbb5006c8f0 0x7fbb5006edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbb640c5640 0x7fbb640ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb6415caa0 0x7fbb6415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 --2- 172.21.15.106:0/3430693687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb640b7f70 0x7fbb64151b80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 >> 172.21.15.106:0/3430693687 conn(0x7fbb640d25d0 msgr2=0x7fbb640b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 shutdown_connections 2024-04-13T03:11:10.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.240+0000 7fbb69d8c640 1 -- 172.21.15.106:0/3430693687 wait complete. 2024-04-13T03:11:10.252 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-13T03:11:10.252 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-13T03:11:10.252 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:11:10.253 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:10.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.351+0000 7f70ff660640 1 Processor -- start 2024-04-13T03:11:10.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 -- start start 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806cb10 0x7f70f8072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70f806e150 0x7f70f806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80998d0 con 0x7f70f80cdf50 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f70f8073020 con 0x7f70f806e150 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70ff660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f70f8073300 con 0x7f70f806cb10 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70fdbfd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70f806e150 0x7f70f806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70fdbfd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70f806e150 0x7f70f806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50528/0 (socket says 172.21.15.106:50528) 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70fd3fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f70f80cdf50 0x7f70f80ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39492/0 (socket says 172.21.15.106:39492) 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.352+0000 7f70fdbfd640 1 -- 172.21.15.106:0/474048015 learned_addr learned my addr 172.21.15.106:0/474048015 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:10.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fcbfb640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806cb10 0x7f70f8072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fdbfd640 1 -- 172.21.15.106:0/474048015 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806cb10 msgr2=0x7f70f8072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fdbfd640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806cb10 0x7f70f8072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fdbfd640 1 -- 172.21.15.106:0/474048015 >> v1:172.21.15.106:6789/0 conn(0x7f70f80cdf50 legacy=0x7f70f80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fdbfd640 1 -- 172.21.15.106:0/474048015 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80735e0 con 0x7f70f806e150 2024-04-13T03:11:10.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70effff640 1 -- 172.21.15.106:0/474048015 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484969064 0 0) 0x7f70f80998d0 con 0x7f70f80cdf50 2024-04-13T03:11:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.353+0000 7f70fdbfd640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806e150 0x7f70f806e600 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4646ac87fab8c5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70effff640 1 -- 172.21.15.106:0/474048015 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f002cb00 con 0x7f70f806e150 2024-04-13T03:11:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70effff640 1 -- 172.21.15.106:0/474048015 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f002cca0 con 0x7f70f806e150 2024-04-13T03:11:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70effff640 1 -- 172.21.15.106:0/474048015 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0034610 con 0x7f70f806e150 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 -- 172.21.15.106:0/474048015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806e150 msgr2=0x7f70f806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806e150 0x7f70f806e600 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002bf60 comp rx=0 tx=0).stop 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 -- 172.21.15.106:0/474048015 shutdown_connections 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806e150 0x7f70f806e600 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 --2- 172.21.15.106:0/474048015 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806cb10 0x7f70f8072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 -- 172.21.15.106:0/474048015 >> 172.21.15.106:0/474048015 conn(0x7f70f80b1f10 msgr2=0x7f70f80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 -- 172.21.15.106:0/474048015 shutdown_connections 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.354+0000 7f70ff660640 1 -- 172.21.15.106:0/474048015 wait complete. 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.355+0000 7f70ff660640 1 Processor -- start 2024-04-13T03:11:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.355+0000 7f70ff660640 1 -- start start 2024-04-13T03:11:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806e150 0x7f70f8098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70f80cdf50 0x7f70f80ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8072de0 con 0x7f70f806cb10 2024-04-13T03:11:10.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8072c60 con 0x7f70f80cdf50 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70ff660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f70f806eb40 con 0x7f70f806e150 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70fd3fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70fd3fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50538/0 (socket says 172.21.15.106:50538) 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.356+0000 7f70fd3fc640 1 -- 172.21.15.106:0/1237721561 learned_addr learned my addr 172.21.15.106:0/1237721561 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fcbfb640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806e150 0x7f70f8098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fdbfd640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70f80cdf50 0x7f70f80ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 -- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806e150 msgr2=0x7f70f8098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806e150 0x7f70f8098860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 -- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70f80cdf50 msgr2=0x7f70f80ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70f80cdf50 0x7f70f80ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 -- 172.21.15.106:0/1237721561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f815a070 con 0x7f70f806cb10 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70fd3fc640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f70e800c460 tx=0x7f70e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8018070 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e8007500 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f80c9d90 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8004a20 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.357+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f80ca350 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.358+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8068a10 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.358+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70e801b070 con 0x7f70f806cb10 2024-04-13T03:11:10.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.359+0000 7f70edffb640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.359+0000 7f70fcbfb640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.359+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f70e808ac40 con 0x7f70f806cb10 2024-04-13T03:11:10.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.359+0000 7f70fcbfb640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 0x7f70dc06edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70f80ce750 tx=0x7f70f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.363+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70e80591e0 con 0x7f70f806cb10 2024-04-13T03:11:10.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.540+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70f80cf5a0 con 0x7f70f806cb10 2024-04-13T03:11:10.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.540+0000 7f70edffb640 1 -- 172.21.15.106:0/1237721561 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13331 (secure 0 0 0) 0x7f70e8058ae0 con 0x7f70f806cb10 2024-04-13T03:11:10.541 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:10.542 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":43,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:09.229984+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6835/1027593505":"2024-04-14T03:11:06.016484+0000","172.21.15.114:6833/1027593505":"2024-04-14T03:11:06.016484+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 msgr2=0x7f70dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 0x7f70dc06edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70f80ce750 tx=0x7f70f4008040 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 msgr2=0x7f70f8159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f70e800c460 tx=0x7f70e800c920 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 shutdown_connections 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70dc06c8f0 0x7f70dc06edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70f806e150 0x7f70f8098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70f80cdf50 0x7f70f80ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 --2- 172.21.15.106:0/1237721561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f806cb10 0x7f70f8159b30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 >> 172.21.15.106:0/1237721561 conn(0x7f70f80b1f10 msgr2=0x7f70f80ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.542+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 shutdown_connections 2024-04-13T03:11:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.543+0000 7f70ff660640 1 -- 172.21.15.106:0/1237721561 wait complete. 2024-04-13T03:11:10.554 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:11:10.555 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:11:10.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.657+0000 7faeed91e640 1 Processor -- start 2024-04-13T03:11:10.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.657+0000 7faeed91e640 1 -- start start 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faeed91e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faee815ee40 0x7faee80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faeed91e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee80b5cf0 con 0x7faee80b1c30 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faeed91e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee80b5fd0 con 0x7faee80709e0 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faeed91e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7faee80b62b0 con 0x7faee815ee40 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faee6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7faee80b1c30 0x7faee80b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39498/0 (socket says 172.21.15.106:39498) 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faee6ffd640 1 -- 172.21.15.106:0/1862203346 learned_addr learned my addr 172.21.15.106:0/1862203346 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faee67fc640 1 --2- 172.21.15.106:0/1862203346 >> v2:172.21.15.114:3300/0 conn(0x7faee815ee40 0x7faee80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931520713 0 0) 0x7faee80b5cf0 con 0x7faee80b1c30 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.658+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed0003520 con 0x7faee80b1c30 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421170682 0 0) 0x7faee80b5fd0 con 0x7faee80709e0 2024-04-13T03:11:10.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faee80b5cf0 con 0x7faee80709e0 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 532403641 0 0) 0x7faed0003520 con 0x7faee80b1c30 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 >> v2:172.21.15.114:3300/0 conn(0x7faee815ee40 msgr2=0x7faee80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 --2- 172.21.15.106:0/1862203346 >> v2:172.21.15.114:3300/0 conn(0x7faee815ee40 0x7faee80704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 >> v1:172.21.15.114:6789/0 conn(0x7faee80709e0 legacy=0x7faee806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee80b6590 con 0x7faee80b1c30 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7faed80019f0 con 0x7faee80b1c30 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 >> v1:172.21.15.106:6789/0 conn(0x7faee80b1c30 legacy=0x7faee80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faed0003b90 0x7faed0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed0004800 0x7faed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 0x7faed000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faee80b62b0 con 0x7faed00051c0 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faed000c090 con 0x7faed0003b90 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7faed000c290 con 0x7faed0004800 2024-04-13T03:11:10.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 0x7faed000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee6ffd640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faed0003b90 0x7faed0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee67fc640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed0004800 0x7faed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 -- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed0004800 msgr2=0x7faed0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed0004800 0x7faed0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 -- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faed0003b90 msgr2=0x7faed0003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faed0003b90 0x7faed0003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.659+0000 7faee77fe640 1 -- 172.21.15.106:0/1862203346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed0003520 con 0x7faed00051c0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faee77fe640 1 -- 172.21.15.106:0/1862203346 reap_dead start 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faee77fe640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 0x7faed000b9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faee00018d0 tx=0x7faed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e7c13744dd9490 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee00014f0 con 0x7faed00051c0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee00052d0 con 0x7faed00051c0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faee5ffb640 1 -- 172.21.15.106:0/1862203346 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0003830 con 0x7faed00051c0 2024-04-13T03:11:10.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faeed91e640 1 -- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 msgr2=0x7faed000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faeed91e640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 0x7faed000b9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faee00018d0 tx=0x7faed0005830 comp rx=0 tx=0).stop 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faeed91e640 1 -- 172.21.15.106:0/1862203346 shutdown_connections 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faeed91e640 1 --2- 172.21.15.106:0/1862203346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faed00051c0 0x7faed000b9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.660+0000 7faeed91e640 1 -- 172.21.15.106:0/1862203346 >> 172.21.15.106:0/1862203346 conn(0x7faee8071870 msgr2=0x7faee8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.661+0000 7faeed91e640 1 -- 172.21.15.106:0/1862203346 shutdown_connections 2024-04-13T03:11:10.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.661+0000 7faeed91e640 1 -- 172.21.15.106:0/1862203346 wait complete. 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.662+0000 7faeed91e640 1 Processor -- start 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.662+0000 7faeed91e640 1 -- start start 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.662+0000 7faeed91e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed00051c0 0x7faee8146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faee815f220 0x7faee815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faee8070830 con 0x7faee80cd250 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faee80706b0 con 0x7faee815f220 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7faee80b54a0 con 0x7faed00051c0 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50554/0 (socket says 172.21.15.106:50554) 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 -- 172.21.15.106:0/2056777566 learned_addr learned my addr 172.21.15.106:0/2056777566 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 -- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed00051c0 msgr2=0x7faee8146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed00051c0 0x7faee8146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 -- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faee815f220 msgr2=0x7faee815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faee815f220 0x7faee815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 -- 172.21.15.106:0/2056777566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee81f4a30 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faee77fe640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faee00019f0 tx=0x7faee00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee00014f0 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0005480 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee80cdf40 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0005070 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.663+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee80d01e0 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.664+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee8068a10 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.665+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faee0044020 con 0x7faee80cd250 2024-04-13T03:11:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.665+0000 7faecffff640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 0x7faed406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.665+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7faee00acda0 con 0x7faee80cd250 2024-04-13T03:11:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.665+0000 7faee6ffd640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 0x7faed406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.665+0000 7faee6ffd640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 0x7faed406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed80013d0 tx=0x7faed8000f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.669+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faee0045370 con 0x7faee80cd250 2024-04-13T03:11:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.843+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faee80b17a0 con 0x7faee80cd250 2024-04-13T03:11:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.843+0000 7faecffff640 1 -- 172.21.15.106:0/2056777566 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7faee007b340 con 0x7faee80cd250 2024-04-13T03:11:10.845 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 msgr2=0x7faed406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 0x7faed406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed80013d0 tx=0x7faed8000f40 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 msgr2=0x7faee80cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faee00019f0 tx=0x7faee00012d0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 shutdown_connections 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7faed406c920 0x7faed406ede0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faee80cd250 0x7faee80cd6d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faee815f220 0x7faee815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 --2- 172.21.15.106:0/2056777566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7faed00051c0 0x7faee8146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 >> 172.21.15.106:0/2056777566 conn(0x7faee8071870 msgr2=0x7faee80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.845+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 shutdown_connections 2024-04-13T03:11:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.846+0000 7faeed91e640 1 -- 172.21.15.106:0/2056777566 wait complete. 2024-04-13T03:11:10.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-13T03:11:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.957+0000 7f1c49280640 1 Processor -- start 2024-04-13T03:11:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.957+0000 7f1c49280640 1 -- start start 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 0x7f1c440d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c4415ed90 0x7f1c440d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c440b3d60 0x7f1c440d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1c440d5070 con 0x7f1c4415ed90 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1c440cfd20 con 0x7f1c4415e9b0 2024-04-13T03:11:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c49280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c44162af0 con 0x7f1c440b3d60 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c4415ed90 0x7f1c440d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c4415ed90 0x7f1c440d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50560/0 (socket says 172.21.15.106:50560) 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 -- 172.21.15.106:0/433763993 learned_addr learned my addr 172.21.15.106:0/433763993 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c437fe640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c440b3d60 0x7f1c440d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/433763993 >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 0x7f1c440d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 -- 172.21.15.106:0/433763993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c440b3d60 msgr2=0x7f1c440d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c440b3d60 0x7f1c440d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 -- 172.21.15.106:0/433763993 >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 msgr2=0x7f1c440d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 --2- 172.21.15.106:0/433763993 >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 0x7f1c440d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c427fc640 1 -- 172.21.15.106:0/433763993 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c44162dd0 con 0x7f1c4415ed90 2024-04-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.958+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/433763993 >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 0x7f1c440d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.959+0000 7f1c427fc640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4415ed90 0x7f1c440d45f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c4408e4e0 tx=0x7f1c3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82cc90d07d67e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.959+0000 7f1c41ffb640 1 -- 172.21.15.106:0/433763993 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004230 con 0x7f1c4415ed90 2024-04-13T03:11:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.959+0000 7f1c41ffb640 1 -- 172.21.15.106:0/433763993 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c380043d0 con 0x7f1c4415ed90 2024-04-13T03:11:10.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.959+0000 7f1c41ffb640 1 -- 172.21.15.106:0/433763993 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38030840 con 0x7f1c4415ed90 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 -- 172.21.15.106:0/433763993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4415ed90 msgr2=0x7f1c440d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4415ed90 0x7f1c440d45f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c4408e4e0 tx=0x7f1c3802b980 comp rx=0 tx=0).stop 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 -- 172.21.15.106:0/433763993 shutdown_connections 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c440b3d60 0x7f1c440d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 --2- 172.21.15.106:0/433763993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4415ed90 0x7f1c440d45f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 --2- 172.21.15.106:0/433763993 >> v2:172.21.15.114:3300/0 conn(0x7f1c4415e9b0 0x7f1c440d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 -- 172.21.15.106:0/433763993 >> 172.21.15.106:0/433763993 conn(0x7f1c4408bf10 msgr2=0x7f1c440b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 -- 172.21.15.106:0/433763993 shutdown_connections 2024-04-13T03:11:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.960+0000 7f1c49280640 1 -- 172.21.15.106:0/433763993 wait complete. 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.961+0000 7f1c49280640 1 Processor -- start 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.961+0000 7f1c49280640 1 -- start start 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 0x7f1c440b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c4415e9b0 0x7f1c440b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44162680 con 0x7f1c4414b250 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44163390 con 0x7f1c440b3d60 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c44162800 con 0x7f1c4415e9b0 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50570/0 (socket says 172.21.15.106:50570) 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 -- 172.21.15.106:0/2900088560 learned_addr learned my addr 172.21.15.106:0/2900088560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 0x7f1c440b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c427fc640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c4415e9b0 0x7f1c440b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 -- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c4415e9b0 msgr2=0x7f1c440b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c4415e9b0 0x7f1c440b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 -- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 msgr2=0x7f1c440b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 0x7f1c440b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 -- 172.21.15.106:0/2900088560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c440cdce0 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 0x7f1c440b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c437fe640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009e30 tx=0x7f1c3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c018020 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3c014480 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c440cdfc0 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c014870 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.962+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4406be10 con 0x7f1c4414b250 2024-04-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.963+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c44068a10 con 0x7f1c4414b250 2024-04-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.964+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c3c005d10 con 0x7f1c4414b250 2024-04-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.964+0000 7f1c2b7fe640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 0x7f1c3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.964+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f1c3c08c760 con 0x7f1c4414b250 2024-04-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.964+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 0x7f1c3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.965+0000 7f1c42ffd640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 0x7f1c3006ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c34006fd0 tx=0x7f1c34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:10.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:10.969+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c3c022030 con 0x7f1c4414b250 2024-04-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.211+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f1c440d53a0 con 0x7f1c4414b250 2024-04-13T03:11:11.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.244+0000 7f1c2b7fe640 1 -- 172.21.15.106:0/2900088560 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v120) ==== 152+0+0 (secure 0 0 0) 0x7f1c3c010040 con 0x7f1c4414b250 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.246+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 msgr2=0x7f1c3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.246+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 0x7f1c3006ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c34006fd0 tx=0x7f1c34008040 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.246+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 msgr2=0x7f1c4414b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.246+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009e30 tx=0x7f1c3c00c490 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 shutdown_connections 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3006c920 0x7f1c3006ede0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c4415e9b0 0x7f1c440b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c4414b250 0x7f1c4414b6e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 --2- 172.21.15.106:0/2900088560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c440b3d60 0x7f1c440b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 >> 172.21.15.106:0/2900088560 conn(0x7f1c4408bf10 msgr2=0x7f1c440b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:11.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 shutdown_connections 2024-04-13T03:11:11.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.247+0000 7f1c49280640 1 -- 172.21.15.106:0/2900088560 wait complete. 2024-04-13T03:11:11.258 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:11:11.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 Processor -- start 2024-04-13T03:11:11.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 -- start start 2024-04-13T03:11:11.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c1413f510 0x7f6c1415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6c14160400 0x7f6c1406c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c14160890 con 0x7f6c1413f8f0 2024-04-13T03:11:11.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6c14160b70 con 0x7f6c1413f510 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.361+0000 7f6c1af55640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6c14160e50 con 0x7f6c14160400 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c1413f510 0x7f6c1415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c194f2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6c1413f8f0 0x7f6c140704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39504/0 (socket says 172.21.15.106:39504) 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c1413f510 0x7f6c1415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50572/0 (socket says 172.21.15.106:50572) 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c194f2640 1 -- 172.21.15.106:0/2851036978 learned_addr learned my addr 172.21.15.106:0/2851036978 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 -- 172.21.15.106:0/2851036978 >> v2:172.21.15.114:3300/0 conn(0x7f6c14160400 msgr2=0x7f6c1406c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 --2- 172.21.15.106:0/2851036978 >> v2:172.21.15.114:3300/0 conn(0x7f6c14160400 0x7f6c1406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 -- 172.21.15.106:0/2851036978 >> v1:172.21.15.106:6789/0 conn(0x7f6c1413f8f0 legacy=0x7f6c140704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c18cf1640 1 -- 172.21.15.106:0/2851036978 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c14161130 con 0x7f6c1413f510 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.362+0000 7f6c137fe640 1 -- 172.21.15.106:0/2851036978 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413142033 0 0) 0x7f6c14160890 con 0x7f6c1413f8f0 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c18cf1640 1 --2- 172.21.15.106:0/2851036978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c1415cdc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cde40 tx=0x7f6c0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f87d0c2dd515db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c137fe640 1 -- 172.21.15.106:0/2851036978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08035020 con 0x7f6c1413f510 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c137fe640 1 -- 172.21.15.106:0/2851036978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c08031830 con 0x7f6c1413f510 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c137fe640 1 -- 172.21.15.106:0/2851036978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08031c00 con 0x7f6c1413f510 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 -- 172.21.15.106:0/2851036978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 msgr2=0x7f6c1415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 --2- 172.21.15.106:0/2851036978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c1415cdc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cde40 tx=0x7f6c0802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 -- 172.21.15.106:0/2851036978 shutdown_connections 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 --2- 172.21.15.106:0/2851036978 >> v2:172.21.15.114:3300/0 conn(0x7f6c14160400 0x7f6c1406c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 --2- 172.21.15.106:0/2851036978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c1415cdc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 -- 172.21.15.106:0/2851036978 >> 172.21.15.106:0/2851036978 conn(0x7f6c140d13f0 msgr2=0x7f6c140d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 -- 172.21.15.106:0/2851036978 shutdown_connections 2024-04-13T03:11:11.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.363+0000 7f6c1af55640 1 -- 172.21.15.106:0/2851036978 wait complete. 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.364+0000 7f6c1af55640 1 Processor -- start 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.364+0000 7f6c1af55640 1 -- start start 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c1413f8f0 0x7f6c140cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 0x7f6c140d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1406c950 con 0x7f6c1413f510 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1406c7d0 con 0x7f6c141574d0 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c1af55640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6c14070c00 con 0x7f6c1413f8f0 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c18cf1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c18cf1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50578/0 (socket says 172.21.15.106:50578) 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.365+0000 7f6c18cf1640 1 -- 172.21.15.106:0/3807852186 learned_addr learned my addr 172.21.15.106:0/3807852186 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:11.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c194f2640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 0x7f6c140d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c13fff640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c1413f8f0 0x7f6c140cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 -- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c1413f8f0 msgr2=0x7f6c140cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c1413f8f0 0x7f6c140cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 -- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 msgr2=0x7f6c140d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 0x7f6c140d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 -- 172.21.15.106:0/3807852186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1415b5d0 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c194f2640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 0x7f6c140d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c18cf1640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009890 tx=0x7f6c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08004230 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c080043d0 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1415b8b0 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0803cad0 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.366+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1415a8a0 con 0x7f6c1413f510 2024-04-13T03:11:11.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.367+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c14069120 con 0x7f6c1413f510 2024-04-13T03:11:11.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.368+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c08038020 con 0x7f6c1413f510 2024-04-13T03:11:11.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.368+0000 7f6c117fa640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 0x7f6bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.368+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f6c080adc50 con 0x7f6c1413f510 2024-04-13T03:11:11.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.368+0000 7f6c13fff640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 0x7f6bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.368+0000 7f6c13fff640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 0x7f6bf806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6c140b2e80 tx=0x7f6c00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.372+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c080b3020 con 0x7f6c1413f510 2024-04-13T03:11:11.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.547+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6c14071160 con 0x7f6c1413f510 2024-04-13T03:11:11.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.550+0000 7f6c117fa640 1 -- 172.21.15.106:0/3807852186 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f6c08041070 con 0x7f6c1413f510 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 msgr2=0x7f6bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 0x7f6bf806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6c140b2e80 tx=0x7f6c00008040 comp rx=0 tx=0).stop 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 msgr2=0x7f6c141f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009890 tx=0x7f6c080046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 shutdown_connections 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6bf806c8f0 0x7f6bf806edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c1413f8f0 0x7f6c140cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c141574d0 0x7f6c140d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 --2- 172.21.15.106:0/3807852186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c1413f510 0x7f6c141f2c50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 >> 172.21.15.106:0/3807852186 conn(0x7f6c140d13f0 msgr2=0x7f6c140704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:11.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 shutdown_connections 2024-04-13T03:11:11.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.552+0000 7f6c1af55640 1 -- 172.21.15.106:0/3807852186 wait complete. 2024-04-13T03:11:11.565 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:11:11.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:11.603 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:11.652 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:11:11.652 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:11.652 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:11.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:11:11.667 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:11.667 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:11.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:11.685 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:11.685 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:11.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:11:11.723 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:11.723 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:11.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:11.742 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:11:11.742 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.861+0000 7f74e1315640 1 Processor -- start 2024-04-13T03:11:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.861+0000 7f74e1315640 1 -- start start 2024-04-13T03:11:11.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74e1315640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74dc0d0a80 0x7f74dc0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74e1315640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74dc0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74e1315640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74dc162820 con 0x7f74dc0d1900 2024-04-13T03:11:11.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74e1315640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f74dc1625d0 con 0x7f74dc0d0a80 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74e1315640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc163430 con 0x7f74dc0d4e00 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74da7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f74dc0d1900 0x7f74dc0d46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39506/0 (socket says 172.21.15.106:39506) 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74da7fc640 1 -- 172.21.15.106:0/2360908814 learned_addr learned my addr 172.21.15.106:0/2360908814 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300552865 0 0) 0x7f74dc162820 con 0x7f74dc0d1900 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c4003520 con 0x7f74dc0d1900 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74db7fe640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74dc0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74daffd640 1 --2- 172.21.15.106:0/2360908814 >> v2:172.21.15.114:3300/0 conn(0x7f74dc0d0a80 0x7f74dc0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 378738732 0 0) 0x7f74c4003520 con 0x7f74dc0d1900 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 msgr2=0x7f74dc0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74dc0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 >> v2:172.21.15.114:3300/0 conn(0x7f74dc0d0a80 msgr2=0x7f74dc0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 --2- 172.21.15.106:0/2360908814 >> v2:172.21.15.114:3300/0 conn(0x7f74dc0d0a80 0x7f74dc0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc162b00 con 0x7f74dc0d1900 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f74d0001670 con 0x7f74dc0d1900 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 >> v1:172.21.15.106:6789/0 conn(0x7f74dc0d1900 legacy=0x7f74dc0d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c4003fe0 0x7f74c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c4004d40 0x7f74c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1625d0 con 0x7f74c4003fe0 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc163430 con 0x7f74c4004d40 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.862+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74c400bac0 con 0x7f74dc0d4e00 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c4003fe0 0x7f74c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74daffd640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74db7fe640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c4004d40 0x7f74c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 -- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 msgr2=0x7f74c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74dc0d4e00 0x7f74c40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 -- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c4004d40 msgr2=0x7f74c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:11.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c4004d40 0x7f74c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 -- 172.21.15.106:0/2360908814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c4003520 con 0x7f74c4003fe0 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.863+0000 7f74da7fc640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c4003fe0 0x7f74c4004800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c240 tx=0x7f74c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b2ba44e8fcd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0007d30 con 0x7f74c4003fe0 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d0030070 con 0x7f74c4003fe0 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74d9ffb640 1 -- 172.21.15.106:0/2360908814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d002cdc0 con 0x7f74c4003fe0 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 -- 172.21.15.106:0/2360908814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c4003fe0 msgr2=0x7f74c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 --2- 172.21.15.106:0/2360908814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74c4003fe0 0x7f74c4004800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c240 tx=0x7f74c4005830 comp rx=0 tx=0).stop 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74db7fe640 1 -- 172.21.15.106:0/2360908814 reap_dead start 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 -- 172.21.15.106:0/2360908814 shutdown_connections 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 -- 172.21.15.106:0/2360908814 >> 172.21.15.106:0/2360908814 conn(0x7f74dc09da70 msgr2=0x7f74dc0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 -- 172.21.15.106:0/2360908814 shutdown_connections 2024-04-13T03:11:11.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.864+0000 7f74e1315640 1 -- 172.21.15.106:0/2360908814 wait complete. 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.865+0000 7f74e1315640 1 Processor -- start 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.865+0000 7f74e1315640 1 -- start start 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 0x7f74dc156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 0x7f74dc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1630c0 con 0x7f74dc0c8c80 2024-04-13T03:11:11.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0d41b0 con 0x7f74dc15e8b0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74e1315640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc0d0e60 con 0x7f74c4003fe0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74da7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74da7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50594/0 (socket says 172.21.15.106:50594) 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.866+0000 7f74da7fc640 1 -- 172.21.15.106:0/1211939125 learned_addr learned my addr 172.21.15.106:0/1211939125 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74daffd640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 0x7f74dc156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74db7fe640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 0x7f74dc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 -- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 msgr2=0x7f74dc156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 0x7f74dc156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 -- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 msgr2=0x7f74dc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 0x7f74dc0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 -- 172.21.15.106:0/1211939125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc0ca6e0 con 0x7f74dc0c8c80 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74db7fe640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 0x7f74dc0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74daffd640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 0x7f74dc156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74da7fc640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c360 tx=0x7f74dc06ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0001c70 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d0004bf0 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0007070 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc0cbba0 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.867+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc0cc160 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.868+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc069120 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.868+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74d0047020 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.869+0000 7f74c37fe640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 0x7f74c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.869+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f74d00acb60 con 0x7f74dc0c8c80 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.869+0000 7f74daffd640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 0x7f74c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:11.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.869+0000 7f74daffd640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 0x7f74c806ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f74dc1617c0 tx=0x7f74cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:11.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:11.873+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74d007b190 con 0x7f74dc0c8c80 2024-04-13T03:11:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.084+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74dc0d53d0 con 0x7f74dc0c8c80 2024-04-13T03:11:12.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.085+0000 7f74c37fe640 1 -- 172.21.15.106:0/1211939125 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3878 (secure 0 0 0) 0x7f74d007aa90 con 0x7f74dc0c8c80 2024-04-13T03:11:12.086 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:12.086 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/1699591174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1699591174},{"type":"v1","addr":"172.21.15.106:6836","nonce":1699591174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4811,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2224869290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2224869290},{"type":"v1","addr":"172.21.15.106:6837","nonce":2224869290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/467944963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":467944963},{"type":"v1","addr":"172.21.15.114:6835","nonce":467944963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1418340862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1418340862},{"type":"v1","addr":"172.21.15.114:6834","nonce":1418340862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-13T03:11:12.086 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 18 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 msgr2=0x7f74c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 0x7f74c806ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f74dc1617c0 tx=0x7f74cc008040 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 msgr2=0x7f74dc15e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c360 tx=0x7f74dc06ecd0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 shutdown_connections 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f74c806c920 0x7f74c806ede0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74dc15e8b0 0x7f74dc0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc0c8c80 0x7f74dc15e370 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 --2- 172.21.15.106:0/1211939125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74c4003fe0 0x7f74dc156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 >> 172.21.15.106:0/1211939125 conn(0x7f74dc09da70 msgr2=0x7f74dc0b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 shutdown_connections 2024-04-13T03:11:12.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.087+0000 7f74e1315640 1 -- 172.21.15.106:0/1211939125 wait complete. 2024-04-13T03:11:12.099 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:11:12.099 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:11:12.099 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:11:12.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:11:12.099 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:11:12.099 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:11:12.100 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:11:12.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:11:12.100 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:11:12.100 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:11:12.100 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:11:12.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:11:12.100 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:11:12.100 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:11:12.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:11:12.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:11:12.109 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:11:12.109 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:11:12.110 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:11:12.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:11:12.112 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:11:12.112 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:11:12.113 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:11:12.113 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:11:12.151 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:11:12.151 DEBUG:teuthology.parallel:result is None 2024-04-13T03:11:12.151 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:11:12.152 DEBUG:teuthology.parallel:result is None 2024-04-13T03:11:12.155 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:11:12.155 DEBUG:teuthology.parallel:result is None 2024-04-13T03:11:12.155 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:11:12.156 DEBUG:teuthology.parallel:result is None 2024-04-13T03:11:12.156 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:11:12.227 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:11:12.228 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:11:12.228 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:11:12.227+0000 7f97dcbfd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-13T03:11:12.228 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:11:12.227+0000 7fbbee5fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-13T03:11:12.240 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:11:12.240 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:11:12.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.262+0000 7f3a1dc60640 1 Processor -- start 2024-04-13T03:11:12.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 -- start start 2024-04-13T03:11:12.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a180d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a180b3d60 0x7f3a180d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a180cfd20 con 0x7f3a1815ed90 2024-04-13T03:11:12.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3a18162af0 con 0x7f3a180b3d60 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a1dc60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a18162dd0 con 0x7f3a1815e9b0 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a17fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a180b3d60 0x7f3a180d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.263+0000 7f3a17fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a180b3d60 0x7f3a180d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50614/0 (socket says 172.21.15.106:50614) 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 -- 172.21.15.106:0/2225907913 learned_addr learned my addr 172.21.15.106:0/2225907913 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a177fe640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a180d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 -- 172.21.15.106:0/2225907913 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 msgr2=0x7f3a180d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a180d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 -- 172.21.15.106:0/2225907913 >> v1:172.21.15.114:6789/0 conn(0x7f3a1815ed90 legacy=0x7f3a180d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 -- 172.21.15.106:0/2225907913 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a181630b0 con 0x7f3a180b3d60 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a167fc640 1 -- 172.21.15.106:0/2225907913 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938499723 0 0) 0x7f3a180cfd20 con 0x7f3a1815ed90 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.264+0000 7f3a17fff640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180d0200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009870 tx=0x7f3a1002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c702e68a069c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a167fc640 1 -- 172.21.15.106:0/2225907913 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1002f840 con 0x7f3a180b3d60 2024-04-13T03:11:12.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a167fc640 1 -- 172.21.15.106:0/2225907913 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1002f9e0 con 0x7f3a180b3d60 2024-04-13T03:11:12.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a167fc640 1 -- 172.21.15.106:0/2225907913 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1002fdb0 con 0x7f3a180b3d60 2024-04-13T03:11:12.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/2225907913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 msgr2=0x7f3a180d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180d0200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009870 tx=0x7f3a1002ba20 comp rx=0 tx=0).stop 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/2225907913 shutdown_connections 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180d0200 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/2225907913 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a180d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/2225907913 >> 172.21.15.106:0/2225907913 conn(0x7f3a1808bf10 msgr2=0x7f3a180b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/2225907913 shutdown_connections 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/2225907913 wait complete. 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.266+0000 7f3a1dc60640 1 Processor -- start 2024-04-13T03:11:12.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 -- start start 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a18159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 0x7f3a1815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18162800 con 0x7f3a180b3d60 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18162680 con 0x7f3a180c8c30 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a1dc60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a180d4d00 con 0x7f3a1815e9b0 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50620/0 (socket says 172.21.15.106:50620) 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.267+0000 7f3a177fe640 1 -- 172.21.15.106:0/3198895376 learned_addr learned my addr 172.21.15.106:0/3198895376 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a16ffd640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a18159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a17fff640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 0x7f3a1815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 -- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 msgr2=0x7f3a18159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a18159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 -- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 msgr2=0x7f3a1815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 0x7f3a1815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 -- 172.21.15.106:0/3198895376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a180cc200 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a16ffd640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a18159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a17fff640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 0x7f3a1815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a177fe640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800efc0 tx=0x7f3a0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08018020 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a08014480 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a180b4140 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08014850 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.268+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a180b4700 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.269+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a18068a10 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.269+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a08005d10 con 0x7f3a180b3d60 2024-04-13T03:11:12.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.270+0000 7f3a0ffff640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:12.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.270+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5335+0+0 (secure 0 0 0) 0x7f3a0808c510 con 0x7f3a180b3d60 2024-04-13T03:11:12.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.270+0000 7f3a16ffd640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:12.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.270+0000 7f3a16ffd640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 0x7f39f806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a181f86a0 tx=0x7f3a00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:12.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.274+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a08022070 con 0x7f3a180b3d60 2024-04-13T03:11:12.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:12.463+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3a180d53a0 con 0x7f3a180b3d60 2024-04-13T03:11:13.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.261+0000 7f3a0ffff640 1 -- 172.21.15.106:0/3198895376 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7f3a08010040 con 0x7f3a180b3d60 2024-04-13T03:11:13.263 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 msgr2=0x7f39f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 0x7f39f806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a181f86a0 tx=0x7f3a00009040 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 msgr2=0x7f3a180cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800efc0 tx=0x7f3a0800c490 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 shutdown_connections 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f39f806c8f0 0x7f39f806edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a1815e9b0 0x7f3a18159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a180c8c30 0x7f3a1815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 --2- 172.21.15.106:0/3198895376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a180b3d60 0x7f3a180cbcc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 >> 172.21.15.106:0/3198895376 conn(0x7f3a1808bf10 msgr2=0x7f3a180b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 shutdown_connections 2024-04-13T03:11:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.265+0000 7f3a1dc60640 1 -- 172.21.15.106:0/3198895376 wait complete. 2024-04-13T03:11:13.278 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:11:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.380+0000 7f29b7605640 1 Processor -- start 2024-04-13T03:11:13.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.380+0000 7f29b7605640 1 -- start start 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b7605640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29b00ce1f0 0x7f29b0162db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b7605640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b00ce6f0 con 0x7f29b006fe90 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b7605640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b015f420 con 0x7f29b015ebf0 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b7605640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f29b015fa70 con 0x7f29b00ce1f0 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b4ba0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f29b006fe90 0x7f29b01626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44974/0 (socket says 172.21.15.106:44974) 2024-04-13T03:11:13.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b4ba0640 1 -- 172.21.15.106:0/3776902629 learned_addr learned my addr 172.21.15.106:0/3776902629 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/3776902629 >> v2:172.21.15.114:3300/0 conn(0x7f29b00ce1f0 0x7f29b0162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.381+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965102475 0 0) 0x7f29b00ce6f0 con 0x7f29b006fe90 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c003520 con 0x7f29b006fe90 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4086693470 0 0) 0x7f299c003520 con 0x7f29b006fe90 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 >> v2:172.21.15.114:3300/0 conn(0x7f29b00ce1f0 msgr2=0x7f29b0162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 --2- 172.21.15.106:0/3776902629 >> v2:172.21.15.114:3300/0 conn(0x7f29b00ce1f0 0x7f29b0162db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 >> v1:172.21.15.114:6789/0 conn(0x7f29b015ebf0 legacy=0x7f29b006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b015f700 con 0x7f29b006fe90 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f29a0001670 con 0x7f29b006fe90 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 >> v1:172.21.15.106:6789/0 conn(0x7f29b006fe90 legacy=0x7f29b01626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0039b0 0x7f299c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f299c004800 0x7f299c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 0x7f299c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29b015fa70 con 0x7f299c0051c0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00c060 con 0x7f299c004800 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f299c00c260 con 0x7f299c0039b0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 0x7f299c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b4ba0640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f299c004800 0x7f299c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b53a1640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0039b0 0x7f299c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 -- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0039b0 msgr2=0x7f299c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0039b0 0x7f299c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 -- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f299c004800 msgr2=0x7f299c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f299c004800 0x7f299c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b5ba2640 1 -- 172.21.15.106:0/3776902629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c003520 con 0x7f299c0051c0 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b53a1640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0039b0 0x7f299c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:13.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.382+0000 7f29b4ba0640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f299c004800 0x7f299c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.383+0000 7f29b5ba2640 1 -- 172.21.15.106:0/3776902629 reap_dead start 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.383+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 0x7f299c00b9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f29b0067fd0 tx=0x7f299c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadb9e704591d8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.383+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8039070 con 0x7f299c0051c0 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.383+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a802cbd0 con 0x7f299c0051c0 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.383+0000 7f29a7fff640 1 -- 172.21.15.106:0/3776902629 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8009c10 con 0x7f299c0051c0 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 -- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 msgr2=0x7f299c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 0x7f299c00b9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f29b0067fd0 tx=0x7f299c005830 comp rx=0 tx=0).stop 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 -- 172.21.15.106:0/3776902629 shutdown_connections 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 --2- 172.21.15.106:0/3776902629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f299c0051c0 0x7f299c00b9c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 -- 172.21.15.106:0/3776902629 >> 172.21.15.106:0/3776902629 conn(0x7f29b00cd6d0 msgr2=0x7f29b00cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 -- 172.21.15.106:0/3776902629 shutdown_connections 2024-04-13T03:11:13.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.384+0000 7f29b7605640 1 -- 172.21.15.106:0/3776902629 wait complete. 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.385+0000 7f29b7605640 1 Processor -- start 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.385+0000 7f29b7605640 1 -- start start 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0051c0 0x7f29b0160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29b0149bb0 0x7f29b01f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00b0650 con 0x7f29b015cca0 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00b04d0 con 0x7f29b0149bb0 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f29b015efd0 con 0x7f299c0051c0 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45550/0 (socket says 172.21.15.106:45550) 2024-04-13T03:11:13.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 -- 172.21.15.106:0/657856997 learned_addr learned my addr 172.21.15.106:0/657856997 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 -- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0051c0 msgr2=0x7f29b0160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0051c0 0x7f29b0160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 -- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29b0149bb0 msgr2=0x7f29b01f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29b0149bb0 0x7f29b01f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 -- 172.21.15.106:0/657856997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b0162060 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b5ba2640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f29a802c640 tx=0x7f29a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8042070 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a803e480 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01617c0 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8039040 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.386+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b0161d80 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.387+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b0068a10 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.388+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29a8045070 con 0x7f29b015cca0 2024-04-13T03:11:13.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.388+0000 7f29a5ffb640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 0x7f299406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.388+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f29a80adc80 con 0x7f29b015cca0 2024-04-13T03:11:13.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.388+0000 7f29b53a1640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 0x7f299406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.389+0000 7f29b53a1640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 0x7f299406ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f29ac001510 tx=0x7f29ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.393+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29a807bcb0 con 0x7f29b015cca0 2024-04-13T03:11:13.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.635+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f29b00b58f0 con 0x7f29b015cca0 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.635+0000 7f29a5ffb640 1 -- 172.21.15.106:0/657856997 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f29a807ffb0 con 0x7f29b015cca0 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.636 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.638 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.639 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.640 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.641 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:11:13.642 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.643 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.644 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.645 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:11:13.646 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.647 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:11:13.648 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.649 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:13.649 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:11:13.649 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:13.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 msgr2=0x7f299406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 0x7f299406ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f29ac001510 tx=0x7f29ac009040 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 msgr2=0x7f29b015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f29a802c640 tx=0x7f29a8009750 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 shutdown_connections 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f299406c920 0x7f299406ede0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b015cca0 0x7f29b015d100 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29b0149bb0 0x7f29b01f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 --2- 172.21.15.106:0/657856997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f299c0051c0 0x7f29b0160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 >> 172.21.15.106:0/657856997 conn(0x7f29b00cd6d0 msgr2=0x7f29b00ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 shutdown_connections 2024-04-13T03:11:13.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.637+0000 7f29b7605640 1 -- 172.21.15.106:0/657856997 wait complete. 2024-04-13T03:11:13.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:13.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.751+0000 7f9fc620f640 1 Processor -- start 2024-04-13T03:11:13.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.751+0000 7f9fc620f640 1 -- start start 2024-04-13T03:11:13.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.751+0000 7f9fc620f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9fc00b8080 0x7f9fc00d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fc620f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc00d3530 con 0x7f9fc01622d0 2024-04-13T03:11:13.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fc620f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc00d0b60 con 0x7f9fc00d1c00 2024-04-13T03:11:13.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fc620f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc01626b0 con 0x7f9fc00b8080 2024-04-13T03:11:13.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbf7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9fc01622d0 0x7f9fc00b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44988/0 (socket says 172.21.15.106:44988) 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbf7fe640 1 -- 172.21.15.106:0/3367121034 learned_addr learned my addr 172.21.15.106:0/3367121034 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3367121034 >> v2:172.21.15.114:3300/0 conn(0x7f9fc00b8080 0x7f9fc00d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483275453 0 0) 0x7f9fc00d3530 con 0x7f9fc01622d0 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fa8003520 con 0x7f9fc01622d0 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936488930 0 0) 0x7f9fa8003520 con 0x7f9fc01622d0 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbffff640 1 -- 172.21.15.106:0/3367121034 >> v1:172.21.15.114:6789/0 conn(0x7f9fc00d1c00 legacy=0x7f9fc01668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbffff640 1 -- 172.21.15.106:0/3367121034 >> v1:172.21.15.106:6789/0 conn(0x7f9fc01622d0 legacy=0x7f9fc00b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.752+0000 7f9fbffff640 1 -- 172.21.15.106:0/3367121034 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc00d0e40 con 0x7f9fc00b8080 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.753+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3367121034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc00b8080 0x7f9fc00d04d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d90 tx=0x7f9fb802bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9918effb04053823 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.753+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8039070 con 0x7f9fc00b8080 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.753+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb802c7a0 con 0x7f9fc00b8080 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.753+0000 7f9fbe7fc640 1 -- 172.21.15.106:0/3367121034 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4001670 con 0x7f9fc00b8080 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 -- 172.21.15.106:0/3367121034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc00b8080 msgr2=0x7f9fc00d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 --2- 172.21.15.106:0/3367121034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc00b8080 0x7f9fc00d04d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d90 tx=0x7f9fb802bc70 comp rx=0 tx=0).stop 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 -- 172.21.15.106:0/3367121034 shutdown_connections 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 --2- 172.21.15.106:0/3367121034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc00b8080 0x7f9fc00d04d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 -- 172.21.15.106:0/3367121034 >> 172.21.15.106:0/3367121034 conn(0x7f9fc00d25d0 msgr2=0x7f9fc00d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 -- 172.21.15.106:0/3367121034 shutdown_connections 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.754+0000 7f9fc620f640 1 -- 172.21.15.106:0/3367121034 wait complete. 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.755+0000 7f9fc620f640 1 Processor -- start 2024-04-13T03:11:13.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.755+0000 7f9fc620f640 1 -- start start 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00b8080 0x7f9fc015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc01622d0 0x7f9fc00c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00d16e0 con 0x7f9fc00d1c00 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00d1400 con 0x7f9fc01622d0 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc00cc6f0 con 0x7f9fc00b8080 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45552/0 (socket says 172.21.15.106:45552) 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 -- 172.21.15.106:0/1450750358 learned_addr learned my addr 172.21.15.106:0/1450750358 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:13.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbffff640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc01622d0 0x7f9fc00c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbf7fe640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00b8080 0x7f9fc015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 -- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00b8080 msgr2=0x7f9fc015d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00b8080 0x7f9fc015d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 -- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc01622d0 msgr2=0x7f9fc00c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc01622d0 0x7f9fc00c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 -- 172.21.15.106:0/1450750358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc0145750 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fbeffd640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c7b0 tx=0x7f9fb000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0017070 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0005290 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc00c7f50 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000a880 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.756+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc00c7c00 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.757+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc0068a10 con 0x7f9fc00d1c00 2024-04-13T03:11:13.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.758+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fb001a070 con 0x7f9fc00d1c00 2024-04-13T03:11:13.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.758+0000 7f9fa7fff640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:13.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.758+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f9fb008add0 con 0x7f9fc00d1c00 2024-04-13T03:11:13.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.758+0000 7f9fbf7fe640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:13.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.759+0000 7f9fbf7fe640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001eb0 tx=0x7f9fb4001c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:13.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.763+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fb0059450 con 0x7f9fc00d1c00 2024-04-13T03:11:13.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.937+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fc00ce5a0 con 0x7f9fc00d1c00 2024-04-13T03:11:13.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.938+0000 7f9fa7fff640 1 -- 172.21.15.106:0/1450750358 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13200 (secure 0 0 0) 0x7f9fb008c070 con 0x7f9fc00d1c00 2024-04-13T03:11:13.939 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:13.939 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":48,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:13.259227+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:13.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 msgr2=0x7f9fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001eb0 tx=0x7f9fb4001c20 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 msgr2=0x7f9fc00c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c7b0 tx=0x7f9fb000cc70 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 shutdown_connections 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc01622d0 0x7f9fc00c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d1c00 0x7f9fc00c5d10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 --2- 172.21.15.106:0/1450750358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00b8080 0x7f9fc015d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 >> 172.21.15.106:0/1450750358 conn(0x7f9fc00d25d0 msgr2=0x7f9fc00b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 shutdown_connections 2024-04-13T03:11:13.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:13.940+0000 7f9fc620f640 1 -- 172.21.15.106:0/1450750358 wait complete. 2024-04-13T03:11:13.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:11:13.953 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.056+0000 7f47a6088640 1 Processor -- start 2024-04-13T03:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.056+0000 7f47a6088640 1 -- start start 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f47a6088640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47a015e8c0 0x7f47a00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f47a6088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 0x7f47a0072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f47a6088640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a00b5060 con 0x7f47a0073350 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f47a6088640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f47a00b5340 con 0x7f47a015e8c0 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f47a6088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f47a006c500 con 0x7f47a015eca0 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47a015e8c0 0x7f47a00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f47a015e8c0 0x7f47a00ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45562/0 (socket says 172.21.15.106:45562) 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 -- 172.21.15.106:0/1717350735 learned_addr learned my addr 172.21.15.106:0/1717350735 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479effd640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 0x7f47a0072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 -- 172.21.15.106:0/1717350735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 msgr2=0x7f47a0072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 0x7f47a0072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 -- 172.21.15.106:0/1717350735 >> v1:172.21.15.114:6789/0 conn(0x7f47a0073350 legacy=0x7f47a00b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479f7fe640 1 -- 172.21.15.106:0/1717350735 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a006c7e0 con 0x7f47a015e8c0 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.057+0000 7f479effd640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 0x7f47a0072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f479f7fe640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015e8c0 0x7f47a00ce250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4790009870 tx=0x7f479002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a802bf5978b8d5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f479e7fc640 1 -- 172.21.15.106:0/1717350735 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790034070 con 0x7f47a015e8c0 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f479e7fc640 1 -- 172.21.15.106:0/1717350735 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4790030840 con 0x7f47a015e8c0 2024-04-13T03:11:14.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f479e7fc640 1 -- 172.21.15.106:0/1717350735 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790030c10 con 0x7f47a015e8c0 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 -- 172.21.15.106:0/1717350735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015e8c0 msgr2=0x7f47a00ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015e8c0 0x7f47a00ce250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4790009870 tx=0x7f479002b980 comp rx=0 tx=0).stop 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 -- 172.21.15.106:0/1717350735 shutdown_connections 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015eca0 0x7f47a0072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 --2- 172.21.15.106:0/1717350735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015e8c0 0x7f47a00ce250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 -- 172.21.15.106:0/1717350735 >> 172.21.15.106:0/1717350735 conn(0x7f47a006f0d0 msgr2=0x7f47a006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 -- 172.21.15.106:0/1717350735 shutdown_connections 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.058+0000 7f47a6088640 1 -- 172.21.15.106:0/1717350735 wait complete. 2024-04-13T03:11:14.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 Processor -- start 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 -- start start 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47a0073350 0x7f47a0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 0x7f47a01e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47a006cd00 con 0x7f47a015eca0 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f47a00b4d50 con 0x7f47a0073350 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f47a6088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f47a006ce80 con 0x7f47a015e8c0 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.060+0000 7f479ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45570/0 (socket says 172.21.15.106:45570) 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 -- 172.21.15.106:0/327093696 learned_addr learned my addr 172.21.15.106:0/327093696 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479f7fe640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47a0073350 0x7f47a0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479effd640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 0x7f47a01e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 -- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 msgr2=0x7f47a01e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 0x7f47a01e8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 -- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47a0073350 msgr2=0x7f47a0159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47a0073350 0x7f47a0159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 -- 172.21.15.106:0/327093696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a00ccdd0 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479effd640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 0x7f47a01e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f479ffff640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f479800c7b0 tx=0x7f479800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004b60 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4798004d00 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a00cd0b0 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479800aa90 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.061+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01598d0 con 0x7f47a015eca0 2024-04-13T03:11:14.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.062+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0068a10 con 0x7f47a015eca0 2024-04-13T03:11:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.063+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f479800ac30 con 0x7f47a015eca0 2024-04-13T03:11:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.063+0000 7f4787fff640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 0x7f478c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.063+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f479808bc90 con 0x7f47a015eca0 2024-04-13T03:11:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.063+0000 7f479f7fe640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 0x7f478c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.063+0000 7f479f7fe640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 0x7f478c06edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f479003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.067+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4798020070 con 0x7f47a015eca0 2024-04-13T03:11:14.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.241+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47a0073dd0 con 0x7f47a015eca0 2024-04-13T03:11:14.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.241+0000 7f4787fff640 1 -- 172.21.15.106:0/327093696 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f479805a270 con 0x7f47a015eca0 2024-04-13T03:11:14.242 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:11:14.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 msgr2=0x7f478c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 0x7f478c06edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f479003b040 comp rx=0 tx=0).stop 2024-04-13T03:11:14.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 msgr2=0x7f47a01e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f479800c7b0 tx=0x7f479800cc70 comp rx=0 tx=0).stop 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 shutdown_connections 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f478c06c8f0 0x7f478c06edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47a015e8c0 0x7f47a01e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47a015eca0 0x7f47a01e8690 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 --2- 172.21.15.106:0/327093696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47a0073350 0x7f47a0159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 >> 172.21.15.106:0/327093696 conn(0x7f47a006f0d0 msgr2=0x7f47a006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 shutdown_connections 2024-04-13T03:11:14.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.243+0000 7f47a6088640 1 -- 172.21.15.106:0/327093696 wait complete. 2024-04-13T03:11:14.255 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:11:14.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.353+0000 7f42293b4640 1 Processor -- start 2024-04-13T03:11:14.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.353+0000 7f42293b4640 1 -- start start 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.354+0000 7f42293b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.354+0000 7f42293b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42240cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.354+0000 7f42293b4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42240d0880 con 0x7f4224073e10 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.354+0000 7f42293b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f422406d280 con 0x7f422415b6a0 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.354+0000 7f42293b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42240d0b60 con 0x7f42240734c0 2024-04-13T03:11:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f4222ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42237fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42240cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f4222ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:35758/0 (socket says 172.21.15.106:35758) 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f4222ffd640 1 -- 172.21.15.106:0/946802053 learned_addr learned my addr 172.21.15.106:0/946802053 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42227fc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4224073e10 0x7f4224072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36864/0 (socket says 172.21.15.106:36864) 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42237fe640 1 -- 172.21.15.106:0/946802053 >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 msgr2=0x7f42240d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42237fe640 1 --2- 172.21.15.106:0/946802053 >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42237fe640 1 -- 172.21.15.106:0/946802053 >> v1:172.21.15.114:6789/0 conn(0x7f4224073e10 legacy=0x7f4224072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f42237fe640 1 -- 172.21.15.106:0/946802053 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240d0e40 con 0x7f42240734c0 2024-04-13T03:11:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.355+0000 7f4222ffd640 1 --2- 172.21.15.106:0/946802053 >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42237fe640 1 --2- 172.21.15.106:0/946802053 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42240cbac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f421c0097a0 tx=0x7f421c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc3a186b6352635 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f4221ffb640 1 -- 172.21.15.106:0/946802053 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c035020 con 0x7f42240734c0 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f4221ffb640 1 -- 172.21.15.106:0/946802053 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421c031830 con 0x7f42240734c0 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f4221ffb640 1 -- 172.21.15.106:0/946802053 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c031c00 con 0x7f42240734c0 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 -- 172.21.15.106:0/946802053 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 msgr2=0x7f42240cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 --2- 172.21.15.106:0/946802053 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42240cbac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f421c0097a0 tx=0x7f421c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 -- 172.21.15.106:0/946802053 shutdown_connections 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 --2- 172.21.15.106:0/946802053 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42240cbac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 --2- 172.21.15.106:0/946802053 >> v2:172.21.15.114:3300/0 conn(0x7f422415b6a0 0x7f42240d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 -- 172.21.15.106:0/946802053 >> 172.21.15.106:0/946802053 conn(0x7f42240cdef0 msgr2=0x7f42240ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 -- 172.21.15.106:0/946802053 shutdown_connections 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.356+0000 7f42293b4640 1 -- 172.21.15.106:0/946802053 wait complete. 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 Processor -- start 2024-04-13T03:11:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 -- start start 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42241e8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f422415b6a0 0x7f42240caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42240cd760 con 0x7f4224073e10 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42240cd5e0 con 0x7f422415b6a0 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.358+0000 7f42293b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f42240cd8e0 con 0x7f42240734c0 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45574/0 (socket says 172.21.15.106:45574) 2024-04-13T03:11:14.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 -- 172.21.15.106:0/3342297612 learned_addr learned my addr 172.21.15.106:0/3342297612 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42237fe640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f422415b6a0 0x7f42240caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f4222ffd640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42241e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 -- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 msgr2=0x7f42241e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42241e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 -- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f422415b6a0 msgr2=0x7f42240caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f422415b6a0 0x7f42240caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 -- 172.21.15.106:0/3342297612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42241595e0 con 0x7f4224073e10 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42227fc640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f421800c4d0 tx=0x7f421800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218018070 con 0x7f4224073e10 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4218007480 con 0x7f4224073e10 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42241598c0 con 0x7f4224073e10 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180049e0 con 0x7f4224073e10 2024-04-13T03:11:14.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.359+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224159e80 con 0x7f4224073e10 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.360+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4224069120 con 0x7f4224073e10 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.361+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f421801b070 con 0x7f4224073e10 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.361+0000 7f420b7fe640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 0x7f421006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.361+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f421808acd0 con 0x7f4224073e10 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.361+0000 7f4222ffd640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 0x7f421006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.362+0000 7f4222ffd640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 0x7f421006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42140072c0 tx=0x7f421400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.366+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4218007670 con 0x7f4224073e10 2024-04-13T03:11:14.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.544+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f42240b2b70 con 0x7f4224073e10 2024-04-13T03:11:14.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.544+0000 7f420b7fe640 1 -- 172.21.15.106:0/3342297612 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) ==== 84+0+65 (secure 0 0 0) 0x7f4218058be0 con 0x7f4224073e10 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:14.546 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.546+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 msgr2=0x7f421006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.546+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 0x7f421006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42140072c0 tx=0x7f421400b040 comp rx=0 tx=0).stop 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.546+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 msgr2=0x7f4224158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.546+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f421800c4d0 tx=0x7f421800c990 comp rx=0 tx=0).stop 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 shutdown_connections 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f421006c8f0 0x7f421006edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f422415b6a0 0x7f42240caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4224073e10 0x7f4224158c70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 --2- 172.21.15.106:0/3342297612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f42240734c0 0x7f42241e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 >> 172.21.15.106:0/3342297612 conn(0x7f42240cdef0 msgr2=0x7f42240b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:14.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 shutdown_connections 2024-04-13T03:11:14.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.547+0000 7f42293b4640 1 -- 172.21.15.106:0/3342297612 wait complete. 2024-04-13T03:11:14.559 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:11:14.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:11:14.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.661+0000 7f41a7ec8640 1 Processor -- start 2024-04-13T03:11:14.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.662+0000 7f41a7ec8640 1 -- start start 2024-04-13T03:11:14.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a7ec8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41a00704a0 0x7f41a0160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a7ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 0x7f41a00b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a7ec8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a0161160 con 0x7f41a00b6b00 2024-04-13T03:11:14.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a7ec8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f41a00b7720 con 0x7f41a00704a0 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a7ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00b7160 con 0x7f41a00b5dd0 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a6465640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 0x7f41a00b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41a00704a0 0x7f41a0160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41a00704a0 0x7f41a0160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:35766/0 (socket says 172.21.15.106:35766) 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a6465640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 0x7f41a00b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:41204/0 (socket says 172.21.15.106:41204) 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 -- 172.21.15.106:0/2333114760 learned_addr learned my addr 172.21.15.106:0/2333114760 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 -- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 msgr2=0x7f41a00b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 --2- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 0x7f41a00b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 -- 172.21.15.106:0/2333114760 >> v1:172.21.15.114:6789/0 conn(0x7f41a00b6b00 legacy=0x7f41a0070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.663+0000 7f41a5c64640 1 -- 172.21.15.106:0/2333114760 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00b7440 con 0x7f41a00704a0 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.664+0000 7f41a5c64640 1 --2- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00704a0 0x7f41a0160c20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4190009870 tx=0x7f419002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37fed276955c367d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.664+0000 7f41a4c62640 1 -- 172.21.15.106:0/2333114760 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190034070 con 0x7f41a00704a0 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.664+0000 7f41a4c62640 1 -- 172.21.15.106:0/2333114760 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4190030840 con 0x7f41a00704a0 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.664+0000 7f41a4c62640 1 -- 172.21.15.106:0/2333114760 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190030c10 con 0x7f41a00704a0 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 -- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00704a0 msgr2=0x7f41a0160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00704a0 0x7f41a0160c20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4190009870 tx=0x7f419002b980 comp rx=0 tx=0).stop 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 -- 172.21.15.106:0/2333114760 shutdown_connections 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00b5dd0 0x7f41a00b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/2333114760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00704a0 0x7f41a0160c20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 -- 172.21.15.106:0/2333114760 >> 172.21.15.106:0/2333114760 conn(0x7f41a00717e0 msgr2=0x7f41a0071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:14.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 -- 172.21.15.106:0/2333114760 shutdown_connections 2024-04-13T03:11:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.665+0000 7f41a7ec8640 1 -- 172.21.15.106:0/2333114760 wait complete. 2024-04-13T03:11:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.666+0000 7f41a7ec8640 1 Processor -- start 2024-04-13T03:11:14.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.666+0000 7f41a7ec8640 1 -- start start 2024-04-13T03:11:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00704a0 0x7f41a01e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00b6b00 0x7f41a0155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41a015c9e0 con 0x7f41a00b5dd0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f41a00b58b0 con 0x7f41a00b6b00 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a7ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f41a015cb60 con 0x7f41a00704a0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45590/0 (socket says 172.21.15.106:45590) 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 -- 172.21.15.106:0/4270525314 learned_addr learned my addr 172.21.15.106:0/4270525314 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a6465640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00b6b00 0x7f41a0155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5c64640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00704a0 0x7f41a01e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 -- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00704a0 msgr2=0x7f41a01e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00704a0 0x7f41a01e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 -- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00b6b00 msgr2=0x7f41a0155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00b6b00 0x7f41a0155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.667+0000 7f41a5463640 1 -- 172.21.15.106:0/4270525314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a0158ff0 con 0x7f41a00b5dd0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41a5463640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c3f0 tx=0x7f419c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0052d0 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c005490 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a01592d0 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c007260 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.668+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a01e8930 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.669+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a0068a10 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.669+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f419c007400 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.669+0000 7f41967fc640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 0x7f418406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.670+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f419c08acf0 con 0x7f41a00b5dd0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.670+0000 7f41a5c64640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 0x7f418406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:14.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.670+0000 7f41a5c64640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 0x7f418406edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4190036000 tx=0x7f419003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:14.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.674+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f419c08f070 con 0x7f41a00b5dd0 2024-04-13T03:11:14.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:14.848+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f41a00725b0 con 0x7f41a00b5dd0 2024-04-13T03:11:15.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.270+0000 7f41967fc640 1 -- 172.21.15.106:0/4270525314 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) ==== 138+0+0 (secure 0 0 0) 0x7f419c05f030 con 0x7f41a00b5dd0 2024-04-13T03:11:15.271 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:11:15.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 msgr2=0x7f418406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 0x7f418406edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4190036000 tx=0x7f419003b040 comp rx=0 tx=0).stop 2024-04-13T03:11:15.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 msgr2=0x7f41a01555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c3f0 tx=0x7f419c00c8b0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 shutdown_connections 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f418406c8f0 0x7f418406edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41a00b6b00 0x7f41a0155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41a00b5dd0 0x7f41a01555c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 --2- 172.21.15.106:0/4270525314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f41a00704a0 0x7f41a01e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 >> 172.21.15.106:0/4270525314 conn(0x7f41a00717e0 msgr2=0x7f41a00b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 shutdown_connections 2024-04-13T03:11:15.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.272+0000 7f41a7ec8640 1 -- 172.21.15.106:0/4270525314 wait complete. 2024-04-13T03:11:15.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:11:15.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.395+0000 7f58d3c36640 1 Processor -- start 2024-04-13T03:11:15.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.395+0000 7f58d3c36640 1 -- start start 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d3c36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc0cf480 0x7f58cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d3c36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58cc0ba170 0x7f58cc0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d3c36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc15f380 con 0x7f58cc06b8e0 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d3c36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0d2d90 con 0x7f58cc0ba170 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d3c36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0cfda0 con 0x7f58cc0cf480 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58cc0ba170 0x7f58cc0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:15.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58cc0ba170 0x7f58cc0cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45592/0 (socket says 172.21.15.106:45592) 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 -- 172.21.15.106:0/3199190332 learned_addr learned my addr 172.21.15.106:0/3199190332 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d21d3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f58cc06b8e0 0x7f58cc06bd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44996/0 (socket says 172.21.15.106:44996) 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 -- 172.21.15.106:0/3199190332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc0cf480 msgr2=0x7f58cc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 --2- 172.21.15.106:0/3199190332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc0cf480 0x7f58cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 -- 172.21.15.106:0/3199190332 >> v1:172.21.15.106:6789/0 conn(0x7f58cc06b8e0 legacy=0x7f58cc06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d09d0640 1 -- 172.21.15.106:0/3199190332 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705853170 0 0) 0x7f58c4001490 con 0x7f58cc06b8e0 2024-04-13T03:11:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.396+0000 7f58d11d1640 1 -- 172.21.15.106:0/3199190332 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc15f600 con 0x7f58cc0ba170 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d11d1640 1 --2- 172.21.15.106:0/3199190332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc0cf860 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009870 tx=0x7f58bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6569f9188dce14 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d09d0640 1 -- 172.21.15.106:0/3199190332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc034070 con 0x7f58cc0ba170 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d09d0640 1 -- 172.21.15.106:0/3199190332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc030840 con 0x7f58cc0ba170 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d09d0640 1 -- 172.21.15.106:0/3199190332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc030c10 con 0x7f58cc0ba170 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d3c36640 1 -- 172.21.15.106:0/3199190332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 msgr2=0x7f58cc0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d3c36640 1 --2- 172.21.15.106:0/3199190332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc0cf860 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009870 tx=0x7f58bc02b980 comp rx=0 tx=0).stop 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.397+0000 7f58d3c36640 1 -- 172.21.15.106:0/3199190332 shutdown_connections 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.398+0000 7f58d3c36640 1 --2- 172.21.15.106:0/3199190332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc0cf860 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.398+0000 7f58d3c36640 1 --2- 172.21.15.106:0/3199190332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc0cf480 0x7f58cc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.398+0000 7f58d3c36640 1 -- 172.21.15.106:0/3199190332 >> 172.21.15.106:0/3199190332 conn(0x7f58cc0d47d0 msgr2=0x7f58cc0d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.398+0000 7f58d3c36640 1 -- 172.21.15.106:0/3199190332 shutdown_connections 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.398+0000 7f58d3c36640 1 -- 172.21.15.106:0/3199190332 wait complete. 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 Processor -- start 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 -- start start 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc06b8e0 0x7f58cc146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58cc0cf480 0x7f58cc160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc159170 con 0x7f58cc0ba170 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc15fbc0 con 0x7f58cc0cf480 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d3c36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc1592f0 con 0x7f58cc06b8e0 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.399+0000 7f58d11d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45598/0 (socket says 172.21.15.106:45598) 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 -- 172.21.15.106:0/2759211695 learned_addr learned my addr 172.21.15.106:0/2759211695 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d19d2640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc06b8e0 0x7f58cc146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d21d3640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58cc0cf480 0x7f58cc160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 -- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc06b8e0 msgr2=0x7f58cc146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc06b8e0 0x7f58cc146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 -- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58cc0cf480 msgr2=0x7f58cc160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58cc0cf480 0x7f58cc160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 -- 172.21.15.106:0/2759211695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc15ea00 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d11d1640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009870 tx=0x7f58bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc048070 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc030640 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc034040 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc0cba40 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.400+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0cd6e0 con 0x7f58cc0ba170 2024-04-13T03:11:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.401+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc068a10 con 0x7f58cc0ba170 2024-04-13T03:11:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.402+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58bc037070 con 0x7f58cc0ba170 2024-04-13T03:11:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.402+0000 7f58c27fc640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 0x7f58b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.402+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5619+0+0 (secure 0 0 0) 0x7f58bc0ada40 con 0x7f58cc0ba170 2024-04-13T03:11:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.402+0000 7f58d19d2640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 0x7f58b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:15.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.402+0000 7f58d19d2640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 0x7f58b006edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009890 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:15.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.406+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58cc15ea00 con 0x7f58cc0ba170 2024-04-13T03:11:15.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:15.582+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f58cc0d37e0 con 0x7f58cc0ba170 2024-04-13T03:11:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.272+0000 7f58c27fc640 1 -- 172.21.15.106:0/2759211695 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) ==== 170+0+0 (secure 0 0 0) 0x7f58bc07c070 con 0x7f58cc0ba170 2024-04-13T03:11:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:11:16.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 msgr2=0x7f58b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:16.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 0x7f58b006edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009890 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2024-04-13T03:11:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 msgr2=0x7f58cc15e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009870 tx=0x7f58bc004730 comp rx=0 tx=0).stop 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 shutdown_connections 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58b006c8f0 0x7f58b006edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58cc0cf480 0x7f58cc160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58cc0ba170 0x7f58cc15e4c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 --2- 172.21.15.106:0/2759211695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58cc06b8e0 0x7f58cc146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.276+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 >> 172.21.15.106:0/2759211695 conn(0x7f58cc0d47d0 msgr2=0x7f58cc0d3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.279+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 shutdown_connections 2024-04-13T03:11:16.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.280+0000 7f58d3c36640 1 -- 172.21.15.106:0/2759211695 wait complete. 2024-04-13T03:11:16.303 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:11:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.408+0000 7f76681f0640 1 Processor -- start 2024-04-13T03:11:16.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.408+0000 7f76681f0640 1 -- start start 2024-04-13T03:11:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 0x7f766015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600b54a0 0x7f76600704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f76600709e0 0x7f7660162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f76600b5cf0 con 0x7f76600709e0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f76600b5fd0 con 0x7f76600b1c30 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f76681f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f76600b62b0 con 0x7f76600b54a0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f76600709e0 0x7f7660162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f76600709e0 0x7f7660162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45612/0 (socket says 172.21.15.106:45612) 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 -- 172.21.15.106:0/2988976656 learned_addr learned my addr 172.21.15.106:0/2988976656 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766578b640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600b54a0 0x7f76600704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f7665f8c640 1 --2- 172.21.15.106:0/2988976656 >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 0x7f766015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 -- 172.21.15.106:0/2988976656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600b54a0 msgr2=0x7f76600704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600b54a0 0x7f76600704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 -- 172.21.15.106:0/2988976656 >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 msgr2=0x7f766015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 --2- 172.21.15.106:0/2988976656 >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 0x7f766015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f766678d640 1 -- 172.21.15.106:0/2988976656 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76600b6590 con 0x7f76600709e0 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.409+0000 7f7665f8c640 1 --2- 172.21.15.106:0/2988976656 >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 0x7f766015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:16.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f766678d640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600709e0 0x7f7660162430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca432680d4c8be server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f7664f8a640 1 -- 172.21.15.106:0/2988976656 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658035020 con 0x7f76600709e0 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f7664f8a640 1 -- 172.21.15.106:0/2988976656 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658031640 con 0x7f76600709e0 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f7664f8a640 1 -- 172.21.15.106:0/2988976656 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765802d040 con 0x7f76600709e0 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 -- 172.21.15.106:0/2988976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600709e0 msgr2=0x7f7660162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600709e0 0x7f7660162430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 -- 172.21.15.106:0/2988976656 shutdown_connections 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600709e0 0x7f7660162430 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 --2- 172.21.15.106:0/2988976656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600b54a0 0x7f76600704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 --2- 172.21.15.106:0/2988976656 >> v2:172.21.15.114:3300/0 conn(0x7f76600b1c30 0x7f766015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.410+0000 7f76681f0640 1 -- 172.21.15.106:0/2988976656 >> 172.21.15.106:0/2988976656 conn(0x7f7660071870 msgr2=0x7f7660071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.411+0000 7f76681f0640 1 -- 172.21.15.106:0/2988976656 shutdown_connections 2024-04-13T03:11:16.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.411+0000 7f76681f0640 1 -- 172.21.15.106:0/2988976656 wait complete. 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 Processor -- start 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 -- start start 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600709e0 0x7f7660151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 0x7f766015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7660162af0 con 0x7f76600b1c30 2024-04-13T03:11:16.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7660162970 con 0x7f76600b54a0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f76681f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7660162c70 con 0x7f76600709e0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f766578b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.412+0000 7f766578b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45614/0 (socket says 172.21.15.106:45614) 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 -- 172.21.15.106:0/1458361199 learned_addr learned my addr 172.21.15.106:0/1458361199 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766678d640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 0x7f766015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f7665f8c640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600709e0 0x7f7660151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 -- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600709e0 msgr2=0x7f7660151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600709e0 0x7f7660151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 -- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 msgr2=0x7f766015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 0x7f766015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 -- 172.21.15.106:0/1458361199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76600cd880 con 0x7f76600b1c30 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766678d640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 0x7f766015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f766578b640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f765400d5f0 tx=0x7f765400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:16.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76540125f0 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7654019070 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76600cdb60 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76540129d0 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.413+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76600ce120 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.414+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7660068a10 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.415+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7654012b70 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.415+0000 7f765e7fc640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 0x7f764c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.415+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6001+0+0 (secure 0 0 0) 0x7f765408c0d0 con 0x7f76600b1c30 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.415+0000 7f7665f8c640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 0x7f764c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:16.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.416+0000 7f7665f8c640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 0x7f764c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7650003fd0 tx=0x7f7650003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:16.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.420+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f765405a3d0 con 0x7f76600b1c30 2024-04-13T03:11:16.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:16.631+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f766015d8e0 con 0x7f76600b1c30 2024-04-13T03:11:17.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.292+0000 7f765e7fc640 1 -- 172.21.15.106:0/1458361199 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) ==== 172+0+0 (secure 0 0 0) 0x7f7654059cd0 con 0x7f76600b1c30 2024-04-13T03:11:17.294 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 7 and data pool 8 2024-04-13T03:11:17.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.299+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 msgr2=0x7f764c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.299+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 0x7f764c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7650003fd0 tx=0x7f7650003990 comp rx=0 tx=0).stop 2024-04-13T03:11:17.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.299+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 msgr2=0x7f76600cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.299+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f765400d5f0 tx=0x7f765400dab0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 shutdown_connections 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f764c06c8f0 0x7f764c06edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76600b54a0 0x7f766015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76600b1c30 0x7f76600cce30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 --2- 172.21.15.106:0/1458361199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f76600709e0 0x7f7660151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.301+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 >> 172.21.15.106:0/1458361199 conn(0x7f7660071870 msgr2=0x7f7660071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.302+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 shutdown_connections 2024-04-13T03:11:17.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.302+0000 7f76681f0640 1 -- 172.21.15.106:0/1458361199 wait complete. 2024-04-13T03:11:17.325 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:17.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 Processor -- start 2024-04-13T03:11:17.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 -- start start 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815ed90 0x7f3b180d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3b180b3d60 0x7f3b180d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b180cfd20 con 0x7f3b1815e9b0 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3b18162af0 con 0x7f3b180b3d60 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.427+0000 7f3b1cb8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b18162dd0 con 0x7f3b1815ed90 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3b180b3d60 0x7f3b180d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3b180b3d60 0x7f3b180d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45630/0 (socket says 172.21.15.106:45630) 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2179094296 learned_addr learned my addr 172.21.15.106:0/2179094296 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b0ffff640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815ed90 0x7f3b180d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2179094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815ed90 msgr2=0x7f3b180d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815ed90 0x7f3b180d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2179094296 >> v1:172.21.15.114:6789/0 conn(0x7f3b1815e9b0 legacy=0x7f3b180d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.428+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2179094296 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181630b0 con 0x7f3b180b3d60 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b16d9d640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b180b3d60 0x7f3b180d0200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009e50 tx=0x7f3b1002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe42b455a514bf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b15d9b640 1 -- 172.21.15.106:0/2179094296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1002cb90 con 0x7f3b180b3d60 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b15d9b640 1 -- 172.21.15.106:0/2179094296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1002cd30 con 0x7f3b180b3d60 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b15d9b640 1 -- 172.21.15.106:0/2179094296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10034610 con 0x7f3b180b3d60 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2179094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b180b3d60 msgr2=0x7f3b180d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b180b3d60 0x7f3b180d0200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009e50 tx=0x7f3b1002bff0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2179094296 shutdown_connections 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b180b3d60 0x7f3b180d0200 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2179094296 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815ed90 0x7f3b180d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2179094296 >> 172.21.15.106:0/2179094296 conn(0x7f3b1808bf10 msgr2=0x7f3b180b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2179094296 shutdown_connections 2024-04-13T03:11:17.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.429+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2179094296 wait complete. 2024-04-13T03:11:17.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 Processor -- start 2024-04-13T03:11:17.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 -- start start 2024-04-13T03:11:17.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b180b3d60 0x7f3b181f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815e9b0 0x7f3b1815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18162800 con 0x7f3b181f8540 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18162680 con 0x7f3b180b3d60 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b1cb8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180d4d00 con 0x7f3b1815e9b0 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b16d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b16d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45632/0 (socket says 172.21.15.106:45632) 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.431+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2814879822 learned_addr learned my addr 172.21.15.106:0/2814879822 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b0ffff640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815e9b0 0x7f3b1815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b1659c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b180b3d60 0x7f3b181f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815e9b0 msgr2=0x7f3b1815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815e9b0 0x7f3b1815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b180b3d60 msgr2=0x7f3b181f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b180b3d60 0x7f3b181f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 -- 172.21.15.106:0/2814879822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181f8a90 con 0x7f3b181f8540 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b16d9d640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b10036040 tx=0x7f3b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10043020 con 0x7f3b181f8540 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1003e450 con 0x7f3b181f8540 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b181f8d70 con 0x7f3b181f8540 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10009510 con 0x7f3b181f8540 2024-04-13T03:11:17.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.432+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181f9330 con 0x7f3b181f8540 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.433+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b18068a10 con 0x7f3b181f8540 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.433+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b10046020 con 0x7f3b181f8540 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.434+0000 7f3b0effd640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 0x7f3af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.434+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f3b100ae2b0 con 0x7f3b181f8540 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.434+0000 7f3b1659c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 0x7f3af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.434+0000 7f3b1659c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 0x7f3af806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000f60 tx=0x7f3b08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.438+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b1004d030 con 0x7f3b181f8540 2024-04-13T03:11:17.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.615+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b180d53a0 con 0x7f3b181f8540 2024-04-13T03:11:17.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.616+0000 7f3b0effd640 1 -- 172.21.15.106:0/2814879822 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16991 (secure 0 0 0) 0x7f3b1007bef0 con 0x7f3b181f8540 2024-04-13T03:11:17.617 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:17.617 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":51,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:16.632639+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 msgr2=0x7f3af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 0x7f3af806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000f60 tx=0x7f3b08009040 comp rx=0 tx=0).stop 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 msgr2=0x7f3b1806b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b10036040 tx=0x7f3b10004a40 comp rx=0 tx=0).stop 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 shutdown_connections 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3af806c8f0 0x7f3af806edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b1815e9b0 0x7f3b1815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b181f8540 0x7f3b1806b8e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 --2- 172.21.15.106:0/2814879822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b180b3d60 0x7f3b181f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 >> 172.21.15.106:0/2814879822 conn(0x7f3b1808bf10 msgr2=0x7f3b180b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:17.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 shutdown_connections 2024-04-13T03:11:17.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.618+0000 7f3b1cb8c640 1 -- 172.21.15.106:0/2814879822 wait complete. 2024-04-13T03:11:17.630 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:17.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.730+0000 7f9f7e805640 1 Processor -- start 2024-04-13T03:11:17.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.730+0000 7f9f7e805640 1 -- start start 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7e805640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f7806fe90 0x7f9f781626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7e805640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f78162be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7e805640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f780ce6f0 con 0x7f9f7815ebf0 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7e805640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9f7815fa70 con 0x7f9f7806fe90 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7e805640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7815f420 con 0x7f9f780ce1f0 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f77fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9f7815ebf0 0x7f9f7806f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45000/0 (socket says 172.21.15.106:45000) 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f77fff640 1 -- 172.21.15.106:0/3154181751 learned_addr learned my addr 172.21.15.106:0/3154181751 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f7cda2640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f78162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517977898 0 0) 0x7f9f780ce6f0 con 0x7f9f7815ebf0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f777fe640 1 --2- 172.21.15.106:0/3154181751 >> v2:172.21.15.114:3300/0 conn(0x7f9f7806fe90 0x7f9f781626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f60003520 con 0x7f9f7815ebf0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2578003706 0 0) 0x7f9f60003520 con 0x7f9f7815ebf0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 msgr2=0x7f9f78162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f78162be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 >> v2:172.21.15.114:3300/0 conn(0x7f9f7806fe90 msgr2=0x7f9f781626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 --2- 172.21.15.106:0/3154181751 >> v2:172.21.15.114:3300/0 conn(0x7f9f7806fe90 0x7f9f781626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7815f700 con 0x7f9f7815ebf0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9f68001670 con 0x7f9f7815ebf0 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 >> v1:172.21.15.106:6789/0 conn(0x7f9f7815ebf0 legacy=0x7f9f7806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f600039b0 0x7f9f60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.731+0000 7f9f76ffd640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f60004800 0x7f9f60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f76ffd640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7815fa70 con 0x7f9f600039b0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7815f420 con 0x7f9f60004800 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6000b9c0 con 0x7f9f780ce1f0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f600039b0 0x7f9f60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f777fe640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f60004800 0x7f9f60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f7cda2640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 -- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 msgr2=0x7f9f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f780ce1f0 0x7f9f600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 -- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f60004800 msgr2=0x7f9f60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f60004800 0x7f9f60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.732+0000 7f9f77fff640 1 -- 172.21.15.106:0/3154181751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f60003520 con 0x7f9f600039b0 2024-04-13T03:11:17.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f77fff640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f600039b0 0x7f9f60003dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c240 tx=0x7f9f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b70687a74290dc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68007d30 con 0x7f9f600039b0 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f68030070 con 0x7f9f600039b0 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f76ffd640 1 -- 172.21.15.106:0/3154181751 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6802cd60 con 0x7f9f600039b0 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 -- 172.21.15.106:0/3154181751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f600039b0 msgr2=0x7f9f60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 --2- 172.21.15.106:0/3154181751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f600039b0 0x7f9f60003dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c240 tx=0x7f9f60005830 comp rx=0 tx=0).stop 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7cda2640 1 -- 172.21.15.106:0/3154181751 reap_dead start 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 -- 172.21.15.106:0/3154181751 shutdown_connections 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 -- 172.21.15.106:0/3154181751 >> 172.21.15.106:0/3154181751 conn(0x7f9f780cd6d0 msgr2=0x7f9f780cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 -- 172.21.15.106:0/3154181751 shutdown_connections 2024-04-13T03:11:17.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.733+0000 7f9f7e805640 1 -- 172.21.15.106:0/3154181751 wait complete. 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.734+0000 7f9f7e805640 1 Processor -- start 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 -- start start 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 0x7f9f78158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f780aed80 0x7f9f78160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f780b0650 con 0x7f9f781508d0 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f780b04d0 con 0x7f9f780aed80 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f7e805640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7815efd0 con 0x7f9f600039b0 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.735+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45644/0 (socket says 172.21.15.106:45644) 2024-04-13T03:11:17.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 -- 172.21.15.106:0/4062972996 learned_addr learned my addr 172.21.15.106:0/4062972996 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f77fff640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 0x7f9f78158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f7cda2640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f780aed80 0x7f9f78160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 -- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 msgr2=0x7f9f78158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 0x7f9f78158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 -- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f780aed80 msgr2=0x7f9f78160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f780aed80 0x7f9f78160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 -- 172.21.15.106:0/4062972996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f78155220 con 0x7f9f781508d0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f77fff640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 0x7f9f78158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f777fe640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0118e0 tx=0x7f9f6c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00faf0 con 0x7f9f781508d0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c00fc90 con 0x7f9f781508d0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f78154f40 con 0x7f9f781508d0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c014440 con 0x7f9f781508d0 2024-04-13T03:11:17.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.736+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f780ae040 con 0x7f9f781508d0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.737+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f78068a10 con 0x7f9f781508d0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.737+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f6c012040 con 0x7f9f781508d0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.738+0000 7f9f74ff9640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 0x7f9f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.738+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f9f6c08c2d0 con 0x7f9f781508d0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.738+0000 7f9f77fff640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 0x7f9f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.738+0000 7f9f77fff640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 0x7f9f6406edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034000 tx=0x7f9f68007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:17.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.742+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f6c019030 con 0x7f9f781508d0 2024-04-13T03:11:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.919+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f78150cd0 con 0x7f9f781508d0 2024-04-13T03:11:17.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.920+0000 7f9f74ff9640 1 -- 172.21.15.106:0/4062972996 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16991 (secure 0 0 0) 0x7f9f6c05a580 con 0x7f9f781508d0 2024-04-13T03:11:17.921 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:17.921 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":51,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:16.632639+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 msgr2=0x7f9f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 0x7f9f6406edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034000 tx=0x7f9f68007000 comp rx=0 tx=0).stop 2024-04-13T03:11:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 msgr2=0x7f9f780ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0118e0 tx=0x7f9f6c011da0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 shutdown_connections 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6406c8f0 0x7f9f6406edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f780aed80 0x7f9f78160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f781508d0 0x7f9f780ae840 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 --2- 172.21.15.106:0/4062972996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f600039b0 0x7f9f78158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 >> 172.21.15.106:0/4062972996 conn(0x7f9f780cd6d0 msgr2=0x7f9f780725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 shutdown_connections 2024-04-13T03:11:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:17.922+0000 7f9f7e805640 1 -- 172.21.15.106:0/4062972996 wait complete. 2024-04-13T03:11:17.934 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:11:18.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.035+0000 7f9d943b4640 1 Processor -- start 2024-04-13T03:11:18.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 -- start start 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d8c15cb00 0x7f9d8c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9d8c0cd4c0 0x7f9d8c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8c0b54c0 con 0x7f9d8c0b81f0 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c071320 con 0x7f9d8c15cb00 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.036+0000 7f9d943b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c070d60 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d8c15cb00 0x7f9d8c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d8c15cb00 0x7f9d8c160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45656/0 (socket says 172.21.15.106:45656) 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 -- 172.21.15.106:0/3897621673 learned_addr learned my addr 172.21.15.106:0/3897621673 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92951640 1 --2- 172.21.15.106:0/3897621673 >> v2:172.21.15.114:3300/0 conn(0x7f9d8c0cd4c0 0x7f9d8c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 -- 172.21.15.106:0/3897621673 >> v2:172.21.15.114:3300/0 conn(0x7f9d8c0cd4c0 msgr2=0x7f9d8c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:18.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 --2- 172.21.15.106:0/3897621673 >> v2:172.21.15.114:3300/0 conn(0x7f9d8c0cd4c0 0x7f9d8c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 -- 172.21.15.106:0/3897621673 >> v1:172.21.15.106:6789/0 conn(0x7f9d8c0b81f0 legacy=0x7f9d8c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d9114e640 1 -- 172.21.15.106:0/3897621673 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270440313 0 0) 0x7f9d80001110 con 0x7f9d8c0b81f0 2024-04-13T03:11:18.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.037+0000 7f9d92150640 1 -- 172.21.15.106:0/3897621673 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c071040 con 0x7f9d8c15cb00 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d92150640 1 --2- 172.21.15.106:0/3897621673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c15cb00 0x7f9d8c160e90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c073e30 tx=0x7f9d8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61c49849dced5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d9114e640 1 -- 172.21.15.106:0/3897621673 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84034070 con 0x7f9d8c15cb00 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d9114e640 1 -- 172.21.15.106:0/3897621673 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d84030860 con 0x7f9d8c15cb00 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d9114e640 1 -- 172.21.15.106:0/3897621673 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84030c30 con 0x7f9d8c15cb00 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 -- 172.21.15.106:0/3897621673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c15cb00 msgr2=0x7f9d8c160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 --2- 172.21.15.106:0/3897621673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c15cb00 0x7f9d8c160e90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c073e30 tx=0x7f9d8402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 -- 172.21.15.106:0/3897621673 shutdown_connections 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 --2- 172.21.15.106:0/3897621673 >> v2:172.21.15.114:3300/0 conn(0x7f9d8c0cd4c0 0x7f9d8c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 --2- 172.21.15.106:0/3897621673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c15cb00 0x7f9d8c160e90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 -- 172.21.15.106:0/3897621673 >> 172.21.15.106:0/3897621673 conn(0x7f9d8c0d2530 msgr2=0x7f9d8c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 -- 172.21.15.106:0/3897621673 shutdown_connections 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.038+0000 7f9d943b4640 1 -- 172.21.15.106:0/3897621673 wait complete. 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.039+0000 7f9d943b4640 1 Processor -- start 2024-04-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 -- start start 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 0x7f9d8c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d8c15cb00 0x7f9d8c142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c0d1210 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c0d07b0 con 0x7f9d8c0b81f0 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d943b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8c0d1390 con 0x7f9d8c15cb00 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d9194f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d9194f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45666/0 (socket says 172.21.15.106:45666) 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d9194f640 1 -- 172.21.15.106:0/1524393194 learned_addr learned my addr 172.21.15.106:0/1524393194 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:18.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.040+0000 7f9d92150640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 0x7f9d8c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d92951640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d8c15cb00 0x7f9d8c142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 -- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d8c15cb00 msgr2=0x7f9d8c142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d8c15cb00 0x7f9d8c142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 -- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 msgr2=0x7f9d8c1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 0x7f9d8c1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 -- 172.21.15.106:0/1524393194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c15eb80 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d92150640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 0x7f9d8c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d9194f640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c7b0 tx=0x7f9d8000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80005070 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80005230 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c15f900 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000a640 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.041+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c15f100 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.042+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c068a10 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.042+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d8000a800 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.043+0000 7f9d8affd640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 0x7f9d7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.043+0000 7f9d92150640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 0x7f9d7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.043+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f9d8008b990 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.043+0000 7f9d92150640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 0x7f9d7806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9d84036000 tx=0x7f9d8403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.047+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d8c0b54c0 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.259+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9d8c0b7720 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.300+0000 7f9d8affd640 1 -- 172.21.15.106:0/1524393194 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7f9d80019300 con 0x7f9d8c0cd4c0 2024-04-13T03:11:18.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 msgr2=0x7f9d7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 0x7f9d7806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9d84036000 tx=0x7f9d8403b040 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 msgr2=0x7f9d8c142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c7b0 tx=0x7f9d8000cc70 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 shutdown_connections 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7806c8f0 0x7f9d7806edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d8c15cb00 0x7f9d8c142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d8c0cd4c0 0x7f9d8c142430 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 --2- 172.21.15.106:0/1524393194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d8c0b81f0 0x7f9d8c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.303+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 >> 172.21.15.106:0/1524393194 conn(0x7f9d8c0d2530 msgr2=0x7f9d8c0b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.304+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 shutdown_connections 2024-04-13T03:11:18.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.304+0000 7f9d943b4640 1 -- 172.21.15.106:0/1524393194 wait complete. 2024-04-13T03:11:18.315 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:11:18.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:11:18.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.419+0000 7f834e9f0640 1 Processor -- start 2024-04-13T03:11:18.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.419+0000 7f834e9f0640 1 -- start start 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834e9f0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f834815ebb0 0x7f834806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834e9f0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83480ce630 0x7f8348163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834e9f0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834806be20 con 0x7f83480ce1d0 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834e9f0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f834815ef90 con 0x7f83480ce630 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834e9f0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8348070410 con 0x7f834815ebb0 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f834cf8d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83480ce630 0x7f8348163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f83477fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f83480ce1d0 0x7f8348162970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45020/0 (socket says 172.21.15.106:45020) 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.420+0000 7f83477fe640 1 -- 172.21.15.106:0/2325931812 learned_addr learned my addr 172.21.15.106:0/2325931812 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:18.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f8347fff640 1 --2- 172.21.15.106:0/2325931812 >> v2:172.21.15.114:3300/0 conn(0x7f834815ebb0 0x7f834806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f834cf8d640 1 -- 172.21.15.106:0/2325931812 >> v2:172.21.15.114:3300/0 conn(0x7f834815ebb0 msgr2=0x7f834806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:18.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f834cf8d640 1 --2- 172.21.15.106:0/2325931812 >> v2:172.21.15.114:3300/0 conn(0x7f834815ebb0 0x7f834806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f834cf8d640 1 -- 172.21.15.106:0/2325931812 >> v1:172.21.15.106:6789/0 conn(0x7f83480ce1d0 legacy=0x7f8348162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f834cf8d640 1 -- 172.21.15.106:0/2325931812 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834806f890 con 0x7f83480ce630 2024-04-13T03:11:18.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f8346ffd640 1 -- 172.21.15.106:0/2325931812 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370317258 0 0) 0x7f834806be20 con 0x7f83480ce1d0 2024-04-13T03:11:18.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.421+0000 7f834cf8d640 1 --2- 172.21.15.106:0/2325931812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f8348163080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8348075d50 tx=0x7f834002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d396aa3542add64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f8346ffd640 1 -- 172.21.15.106:0/2325931812 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340039070 con 0x7f83480ce630 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f8346ffd640 1 -- 172.21.15.106:0/2325931812 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834002cc80 con 0x7f83480ce630 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f8346ffd640 1 -- 172.21.15.106:0/2325931812 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340034660 con 0x7f83480ce630 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 -- 172.21.15.106:0/2325931812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 msgr2=0x7f8348163080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 --2- 172.21.15.106:0/2325931812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f8348163080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8348075d50 tx=0x7f834002bed0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 -- 172.21.15.106:0/2325931812 shutdown_connections 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 --2- 172.21.15.106:0/2325931812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f8348163080 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 --2- 172.21.15.106:0/2325931812 >> v2:172.21.15.114:3300/0 conn(0x7f834815ebb0 0x7f834806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 -- 172.21.15.106:0/2325931812 >> 172.21.15.106:0/2325931812 conn(0x7f83480cd6b0 msgr2=0x7f83480cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 -- 172.21.15.106:0/2325931812 shutdown_connections 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.422+0000 7f834e9f0640 1 -- 172.21.15.106:0/2325931812 wait complete. 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.423+0000 7f834e9f0640 1 Processor -- start 2024-04-13T03:11:18.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 -- start start 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83480ce1d0 0x7f83481507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f834815ebb0 0x7f83480aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8348070130 con 0x7f83480ce630 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f834806fe50 con 0x7f834815ebb0 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f834e9f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83480b5100 con 0x7f83480ce1d0 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f83477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f83477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45686/0 (socket says 172.21.15.106:45686) 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.424+0000 7f83477fe640 1 -- 172.21.15.106:0/1356056345 learned_addr learned my addr 172.21.15.106:0/1356056345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:18.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f8347fff640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83480ce1d0 0x7f83481507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f834cf8d640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f834815ebb0 0x7f83480aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 -- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83480ce1d0 msgr2=0x7f83481507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83480ce1d0 0x7f83481507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 -- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f834815ebb0 msgr2=0x7f83480aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f834815ebb0 0x7f83480aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 -- 172.21.15.106:0/1356056345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834815c580 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f83477fe640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c4a0 tx=0x7f833c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c018070 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0074e0 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834815c860 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c004a20 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.425+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834815ce20 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.426+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348069120 con 0x7f83480ce630 2024-04-13T03:11:18.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.426+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f833c01b070 con 0x7f83480ce630 2024-04-13T03:11:18.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.427+0000 7f8344ff9640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 0x7f833406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:18.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.427+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f833c08afa0 con 0x7f83480ce630 2024-04-13T03:11:18.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.427+0000 7f8347fff640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 0x7f833406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:18.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.427+0000 7f8347fff640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 0x7f833406ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8338006fd0 tx=0x7f8338008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:18.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.431+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f833c059240 con 0x7f83480ce630 2024-04-13T03:11:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:18.643+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f83480cdd70 con 0x7f83480ce630 2024-04-13T03:11:19.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.308+0000 7f8344ff9640 1 -- 172.21.15.106:0/1356056345 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f833c058b40 con 0x7f83480ce630 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 msgr2=0x7f833406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 0x7f833406ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8338006fd0 tx=0x7f8338008040 comp rx=0 tx=0).stop 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 msgr2=0x7f834815d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c4a0 tx=0x7f833c00c960 comp rx=0 tx=0).stop 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 shutdown_connections 2024-04-13T03:11:19.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833406c920 0x7f833406ede0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f834815ebb0 0x7f83480aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83480ce630 0x7f834815d620 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 --2- 172.21.15.106:0/1356056345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83480ce1d0 0x7f83481507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 >> 172.21.15.106:0/1356056345 conn(0x7f83480cd6b0 msgr2=0x7f834806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 shutdown_connections 2024-04-13T03:11:19.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.311+0000 7f834e9f0640 1 -- 172.21.15.106:0/1356056345 wait complete. 2024-04-13T03:11:19.323 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:11:19.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.424+0000 7f138878c640 1 Processor -- start 2024-04-13T03:11:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.424+0000 7f138878c640 1 -- start start 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f138878c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f13800b68d0 0x7f13800ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f138878c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f13800d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f138878c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138015eda0 con 0x7f138015e9c0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f138878c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f13800d27f0 con 0x7f13800b68d0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f138878c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1380162c20 con 0x7f13801624b0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f1386528640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f138015e9c0 0x7f13800b61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45028/0 (socket says 172.21.15.106:45028) 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.425+0000 7f1386528640 1 -- 172.21.15.106:0/863265342 learned_addr learned my addr 172.21.15.106:0/863265342 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1386d29640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f13800d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 --2- 172.21.15.106:0/863265342 >> v2:172.21.15.114:3300/0 conn(0x7f13800b68d0 0x7f13800ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041056362 0 0) 0x7f138015eda0 con 0x7f138015e9c0 2024-04-13T03:11:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f136c003520 con 0x7f138015e9c0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3158987996 0 0) 0x7f136c003520 con 0x7f138015e9c0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 msgr2=0x7f13800d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f13800d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 >> v2:172.21.15.114:3300/0 conn(0x7f13800b68d0 msgr2=0x7f13800ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 --2- 172.21.15.106:0/863265342 >> v2:172.21.15.114:3300/0 conn(0x7f13800b68d0 0x7f13800ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1380162f00 con 0x7f138015e9c0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f137c0019f0 con 0x7f138015e9c0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 >> v1:172.21.15.106:6789/0 conn(0x7f138015e9c0 legacy=0x7f13800b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c0039b0 0x7f136c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f136c004800 0x7f136c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f136c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13800d27f0 con 0x7f136c004800 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1380162c20 con 0x7f136c0039b0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f136c00b9c0 con 0x7f13801624b0 2024-04-13T03:11:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f136c004800 0x7f136c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1386528640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c0039b0 0x7f136c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1386d29640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f136c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 -- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 msgr2=0x7f136c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13801624b0 0x7f136c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 -- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c0039b0 msgr2=0x7f136c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c0039b0 0x7f136c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.426+0000 7f1385d27640 1 -- 172.21.15.106:0/863265342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c003520 con 0x7f136c004800 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.427+0000 7f1385d27640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f136c004800 0x7f136c004c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1370009ac0 tx=0x7f136c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350c8d3ac1e6bdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.427+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370039070 con 0x7f136c004800 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.427+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137002c710 con 0x7f136c004800 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.427+0000 7f1385526640 1 -- 172.21.15.106:0/863265342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370009690 con 0x7f136c004800 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 -- 172.21.15.106:0/863265342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f136c004800 msgr2=0x7f136c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 --2- 172.21.15.106:0/863265342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f136c004800 0x7f136c004c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1370009ac0 tx=0x7f136c005830 comp rx=0 tx=0).stop 2024-04-13T03:11:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f1386d29640 1 -- 172.21.15.106:0/863265342 reap_dead start 2024-04-13T03:11:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 -- 172.21.15.106:0/863265342 shutdown_connections 2024-04-13T03:11:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 -- 172.21.15.106:0/863265342 >> 172.21.15.106:0/863265342 conn(0x7f13800d11c0 msgr2=0x7f13800d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 -- 172.21.15.106:0/863265342 shutdown_connections 2024-04-13T03:11:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.428+0000 7f138878c640 1 -- 172.21.15.106:0/863265342 wait complete. 2024-04-13T03:11:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.429+0000 7f138878c640 1 Processor -- start 2024-04-13T03:11:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.429+0000 7f138878c640 1 -- start start 2024-04-13T03:11:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 0x7f13801524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 0x7f13800cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1380162940 con 0x7f13800b5140 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f13800d2a40 con 0x7f136c004800 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f138878c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f13800d2f60 con 0x7f13800ca330 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45696/0 (socket says 172.21.15.106:45696) 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 -- 172.21.15.106:0/2846838185 learned_addr learned my addr 172.21.15.106:0/2846838185 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386528640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 0x7f13801524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1385d27640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 0x7f13800cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 -- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 msgr2=0x7f13800cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 0x7f13800cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 -- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 msgr2=0x7f13801524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 0x7f13801524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 -- 172.21.15.106:0/2846838185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13800ce3b0 con 0x7f13800b5140 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1385d27640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 0x7f13800cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386528640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 0x7f13801524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1386d29640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1380162890 tx=0x7f1378011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378019020 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.430+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1378004240 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.431+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13800c83a0 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.431+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137800f6f0 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.431+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13800c8960 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.431+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1380068a10 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.432+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f137801c020 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.432+0000 7f1376ffd640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 0x7f136406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.432+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f137808bde0 con 0x7f13800b5140 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.432+0000 7f1386528640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 0x7f136406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.433+0000 7f1386528640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 0x7f136406ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f137c002230 tx=0x7f137c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.437+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1378023020 con 0x7f13800b5140 2024-04-13T03:11:19.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.647+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13800ca710 con 0x7f13800b5140 2024-04-13T03:11:19.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.648+0000 7f1376ffd640 1 -- 172.21.15.106:0/2846838185 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f137805a110 con 0x7f13800b5140 2024-04-13T03:11:19.649 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:19.649 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:19.649 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 msgr2=0x7f136406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 0x7f136406ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f137c002230 tx=0x7f137c006040 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 msgr2=0x7f13800b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1380162890 tx=0x7f1378011cf0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 shutdown_connections 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f136406c920 0x7f136406ede0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f13800ca330 0x7f13800cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13800b5140 0x7f13800b55c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 --2- 172.21.15.106:0/2846838185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136c004800 0x7f13801524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.649+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 >> 172.21.15.106:0/2846838185 conn(0x7f13800d11c0 msgr2=0x7f138015dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.650+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 shutdown_connections 2024-04-13T03:11:19.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.650+0000 7f138878c640 1 -- 172.21.15.106:0/2846838185 wait complete. 2024-04-13T03:11:19.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:19.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.760+0000 7f8c4b8b7640 1 Processor -- start 2024-04-13T03:11:19.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.760+0000 7f8c4b8b7640 1 -- start start 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c4b8b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c4b8b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 0x7f8c441688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c4b8b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c440d3530 con 0x7f8c441622d0 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c4b8b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8c441626b0 con 0x7f8c440b8080 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c4b8b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8c440d0b60 con 0x7f8c440d1c00 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c48e52640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c49e54640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 0x7f8c441688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c48e52640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:41330/0 (socket says 172.21.15.106:41330) 2024-04-13T03:11:19.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c48e52640 1 -- 172.21.15.106:0/3091390719 learned_addr learned my addr 172.21.15.106:0/3091390719 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:19.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.761+0000 7f8c49e54640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 0x7f8c441688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:35914/0 (socket says 172.21.15.106:35914) 2024-04-13T03:11:19.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.762+0000 7f8c48e52640 1 -- 172.21.15.106:0/3091390719 >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 msgr2=0x7f8c441688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:19.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.762+0000 7f8c48e52640 1 --2- 172.21.15.106:0/3091390719 >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 0x7f8c441688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.762+0000 7f8c48e52640 1 -- 172.21.15.106:0/3091390719 >> v1:172.21.15.114:6789/0 conn(0x7f8c441622d0 legacy=0x7f8c440b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.762+0000 7f8c48e52640 1 -- 172.21.15.106:0/3091390719 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c440d0e40 con 0x7f8c440d1c00 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c48e52640 1 --2- 172.21.15.106:0/3091390719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009870 tx=0x7f8c3402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b442b8ec41b98f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c3bfff640 1 -- 172.21.15.106:0/3091390719 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34034070 con 0x7f8c440d1c00 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c3bfff640 1 -- 172.21.15.106:0/3091390719 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c34030840 con 0x7f8c440d1c00 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c3bfff640 1 -- 172.21.15.106:0/3091390719 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34030c10 con 0x7f8c440d1c00 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/3091390719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 msgr2=0x7f8c440d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/3091390719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009870 tx=0x7f8c3402b980 comp rx=0 tx=0).stop 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/3091390719 shutdown_connections 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/3091390719 >> v2:172.21.15.114:3300/0 conn(0x7f8c440b8080 0x7f8c441688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/3091390719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c440d04d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/3091390719 >> 172.21.15.106:0/3091390719 conn(0x7f8c440d25d0 msgr2=0x7f8c440d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/3091390719 shutdown_connections 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.763+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/3091390719 wait complete. 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 Processor -- start 2024-04-13T03:11:19.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 -- start start 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c440b8080 0x7f8c440c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c4414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c440d16e0 con 0x7f8c441622d0 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c440d1400 con 0x7f8c440b8080 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c4b8b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8c440cc6f0 con 0x7f8c440d1c00 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c49e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c49e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45710/0 (socket says 172.21.15.106:45710) 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.765+0000 7f8c49e54640 1 -- 172.21.15.106:0/1513282780 learned_addr learned my addr 172.21.15.106:0/1513282780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49653640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c440b8080 0x7f8c440c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c48e52640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c4414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 -- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 msgr2=0x7f8c4414bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c4414bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 -- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c440b8080 msgr2=0x7f8c440c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c440b8080 0x7f8c440c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 -- 172.21.15.106:0/1513282780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4415aef0 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c49e54640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e960 tx=0x7f8c3c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00c980 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c440c8c40 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00cd70 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.766+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c440ca020 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.767+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c44072370 con 0x7f8c441622d0 2024-04-13T03:11:19.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.768+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c3c01a070 con 0x7f8c441622d0 2024-04-13T03:11:19.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.768+0000 7f8c39ffb640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 0x7f8c2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:19.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.768+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f8c3c08c010 con 0x7f8c441622d0 2024-04-13T03:11:19.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.768+0000 7f8c49653640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 0x7f8c2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:19.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.768+0000 7f8c49653640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 0x7f8c2806ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001ca0 tx=0x7f8c40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:19.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.772+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c44072370 con 0x7f8c441622d0 2024-04-13T03:11:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.947+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c44068a10 con 0x7f8c441622d0 2024-04-13T03:11:19.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.948+0000 7f8c39ffb640 1 -- 172.21.15.106:0/1513282780 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16978 (secure 0 0 0) 0x7f8c4415aef0 con 0x7f8c441622d0 2024-04-13T03:11:19.949 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:19.949 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":52,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:18.293562+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:19.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 msgr2=0x7f8c2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 0x7f8c2806ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001ca0 tx=0x7f8c40009040 comp rx=0 tx=0).stop 2024-04-13T03:11:19.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 msgr2=0x7f8c4414b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e960 tx=0x7f8c3c00ee20 comp rx=0 tx=0).stop 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 shutdown_connections 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c2806c920 0x7f8c2806ede0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c440d1c00 0x7f8c4414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c441622d0 0x7f8c4414b380 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 --2- 172.21.15.106:0/1513282780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c440b8080 0x7f8c440c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 >> 172.21.15.106:0/1513282780 conn(0x7f8c440d25d0 msgr2=0x7f8c440ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 shutdown_connections 2024-04-13T03:11:19.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:19.950+0000 7f8c4b8b7640 1 -- 172.21.15.106:0/1513282780 wait complete. 2024-04-13T03:11:19.962 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:11:19.962 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:11:20.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.061+0000 7f5dbeb55640 1 Processor -- start 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.061+0000 7f5dbeb55640 1 -- start start 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5dbeb55640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5db80cdf50 0x7f5db815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5dbeb55640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5db806ef90 0x7f5db815e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5dbeb55640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db806f540 con 0x7f5db80b53a0 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5dbeb55640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5db806f820 con 0x7f5db80cdf50 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5dbeb55640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5db815b440 con 0x7f5db806ef90 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5db80cdf50 0x7f5db815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5db80cdf50 0x7f5db815bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45724/0 (socket says 172.21.15.106:45724) 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 -- 172.21.15.106:0/2506413262 learned_addr learned my addr 172.21.15.106:0/2506413262 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 -- 172.21.15.106:0/2506413262 >> v2:172.21.15.114:3300/0 conn(0x7f5db806ef90 msgr2=0x7f5db815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 --2- 172.21.15.106:0/2506413262 >> v2:172.21.15.114:3300/0 conn(0x7f5db806ef90 0x7f5db815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 -- 172.21.15.106:0/2506413262 >> v1:172.21.15.114:6789/0 conn(0x7f5db80b53a0 legacy=0x7f5db80b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:20.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.062+0000 7f5db7fff640 1 -- 172.21.15.106:0/2506413262 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db815b720 con 0x7f5db80cdf50 2024-04-13T03:11:20.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.063+0000 7f5db7fff640 1 --2- 172.21.15.106:0/2506413262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80cdf50 0x7f5db815bc40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5db80afdf0 tx=0x7f5dac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ddf2eded79552a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.063+0000 7f5db77fe640 1 -- 172.21.15.106:0/2506413262 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac035020 con 0x7f5db80cdf50 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.063+0000 7f5db77fe640 1 -- 172.21.15.106:0/2506413262 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac031870 con 0x7f5db80cdf50 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.063+0000 7f5db77fe640 1 -- 172.21.15.106:0/2506413262 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac031c40 con 0x7f5db80cdf50 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 -- 172.21.15.106:0/2506413262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80cdf50 msgr2=0x7f5db815bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/2506413262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80cdf50 0x7f5db815bc40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5db80afdf0 tx=0x7f5dac02b910 comp rx=0 tx=0).stop 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 -- 172.21.15.106:0/2506413262 shutdown_connections 2024-04-13T03:11:20.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/2506413262 >> v2:172.21.15.114:3300/0 conn(0x7f5db806ef90 0x7f5db815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/2506413262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80cdf50 0x7f5db815bc40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 -- 172.21.15.106:0/2506413262 >> 172.21.15.106:0/2506413262 conn(0x7f5db80b1f10 msgr2=0x7f5db80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 -- 172.21.15.106:0/2506413262 shutdown_connections 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.064+0000 7f5dbeb55640 1 -- 172.21.15.106:0/2506413262 wait complete. 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.065+0000 7f5dbeb55640 1 Processor -- start 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.065+0000 7f5dbeb55640 1 -- start start 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5db806ef90 0x7f5db8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db80999d0 0x7f5db80c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8158bb0 con 0x7f5db80b53a0 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5db815c180 con 0x7f5db80999d0 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbeb55640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5db8151870 con 0x7f5db806ef90 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45734/0 (socket says 172.21.15.106:45734) 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 -- 172.21.15.106:0/1737184832 learned_addr learned my addr 172.21.15.106:0/1737184832 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbd0f2640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db80999d0 0x7f5db80c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5dbc8f1640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5db806ef90 0x7f5db8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 -- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5db806ef90 msgr2=0x7f5db8154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5db806ef90 0x7f5db8154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 -- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db80999d0 msgr2=0x7f5db80c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db80999d0 0x7f5db80c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 -- 172.21.15.106:0/1737184832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db8152860 con 0x7f5db80b53a0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.066+0000 7f5db7fff640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5dac000c00 tx=0x7f5dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.067+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03ad60 con 0x7f5db80b53a0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.067+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac0443f0 con 0x7f5db80b53a0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.067+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db8152b40 con 0x7f5db80b53a0 2024-04-13T03:11:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.067+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0447c0 con 0x7f5db80b53a0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.067+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db80c4660 con 0x7f5db80b53a0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.068+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db8068a10 con 0x7f5db80b53a0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.068+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dac04b060 con 0x7f5db80b53a0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.068+0000 7f5db57fa640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.068+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f5dac0adf10 con 0x7f5db80b53a0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.068+0000 7f5dbc8f1640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.069+0000 7f5dbc8f1640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.073+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dac03f070 con 0x7f5db80b53a0 2024-04-13T03:11:20.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.245+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5db80c6ee0 con 0x7f5da406c8f0 2024-04-13T03:11:20.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.251+0000 7f5db57fa640 1 -- 172.21.15.106:0/1737184832 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251884 (secure 0 0 0) 0x7f5db80c6ee0 con 0x7f5da406c8f0 2024-04-13T03:11:20.255 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:20.256 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:11:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 msgr2=0x7f5da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8008040 comp rx=0 tx=0).stop 2024-04-13T03:11:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 msgr2=0x7f5db80c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5dac000c00 tx=0x7f5dac0046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 shutdown_connections 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db80999d0 0x7f5db80c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db80b53a0 0x7f5db80c4120 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 --2- 172.21.15.106:0/1737184832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5db806ef90 0x7f5db8154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 >> 172.21.15.106:0/1737184832 conn(0x7f5db80b1f10 msgr2=0x7f5db80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 shutdown_connections 2024-04-13T03:11:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.254+0000 7f5dbeb55640 1 -- 172.21.15.106:0/1737184832 wait complete. 2024-04-13T03:11:20.266 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-04-13T03:11:18.385647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222000,"kb_used_data":5828,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752720,"statfs":{"total":773094113280,"available":772866785280,"internally_reserved":0,"allocated":5967872,"data_stored":3293273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000576"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322024+0000","last_change":"2024-04-13T03:11:17.314443+0000","last_active":"2024-04-13T03:11:17.322024+0000","last_peered":"2024-04-13T03:11:17.322024+0000","last_clean":"2024-04-13T03:11:17.322024+0000","last_became_active":"2024-04-13T03:11:17.314184+0000","last_became_peered":"2024-04-13T03:11:17.314184+0000","last_unstale":"2024-04-13T03:11:17.322024+0000","last_undegraded":"2024-04-13T03:11:17.322024+0000","last_fullsized":"2024-04-13T03:11:17.322024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323859+0000","last_change":"2024-04-13T03:11:17.317238+0000","last_active":"2024-04-13T03:11:17.323859+0000","last_peered":"2024-04-13T03:11:17.323859+0000","last_clean":"2024-04-13T03:11:17.323859+0000","last_became_active":"2024-04-13T03:11:17.316740+0000","last_became_peered":"2024-04-13T03:11:17.316740+0000","last_unstale":"2024-04-13T03:11:17.323859+0000","last_undegraded":"2024-04-13T03:11:17.323859+0000","last_fullsized":"2024-04-13T03:11:17.323859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:33.423529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310254+0000","last_change":"2024-04-13T03:11:17.302928+0000","last_active":"2024-04-13T03:11:17.310254+0000","last_peered":"2024-04-13T03:11:17.310254+0000","last_clean":"2024-04-13T03:11:17.310254+0000","last_became_active":"2024-04-13T03:11:17.302637+0000","last_became_peered":"2024-04-13T03:11:17.302637+0000","last_unstale":"2024-04-13T03:11:17.310254+0000","last_undegraded":"2024-04-13T03:11:17.310254+0000","last_fullsized":"2024-04-13T03:11:17.310254+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:05.108643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323934+0000","last_change":"2024-04-13T03:11:17.318401+0000","last_active":"2024-04-13T03:11:17.323934+0000","last_peered":"2024-04-13T03:11:17.323934+0000","last_clean":"2024-04-13T03:11:17.323934+0000","last_became_active":"2024-04-13T03:11:17.318064+0000","last_became_peered":"2024-04-13T03:11:17.318064+0000","last_unstale":"2024-04-13T03:11:17.323934+0000","last_undegraded":"2024-04-13T03:11:17.323934+0000","last_fullsized":"2024-04-13T03:11:17.323934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.314479+0000","last_change":"2024-04-13T03:11:17.309144+0000","last_active":"2024-04-13T03:11:17.314479+0000","last_peered":"2024-04-13T03:11:17.314479+0000","last_clean":"2024-04-13T03:11:17.314479+0000","last_became_active":"2024-04-13T03:11:17.308876+0000","last_became_peered":"2024-04-13T03:11:17.308876+0000","last_unstale":"2024-04-13T03:11:17.314479+0000","last_undegraded":"2024-04-13T03:11:17.314479+0000","last_fullsized":"2024-04-13T03:11:17.314479+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311261+0000","last_change":"2024-04-13T03:11:17.308473+0000","last_active":"2024-04-13T03:11:17.311261+0000","last_peered":"2024-04-13T03:11:17.311261+0000","last_clean":"2024-04-13T03:11:17.311261+0000","last_became_active":"2024-04-13T03:11:17.308221+0000","last_became_peered":"2024-04-13T03:11:17.308221+0000","last_unstale":"2024-04-13T03:11:17.311261+0000","last_undegraded":"2024-04-13T03:11:17.311261+0000","last_fullsized":"2024-04-13T03:11:17.311261+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306843+0000","last_change":"2024-04-13T03:11:17.299596+0000","last_active":"2024-04-13T03:11:17.306843+0000","last_peered":"2024-04-13T03:11:17.306843+0000","last_clean":"2024-04-13T03:11:17.306843+0000","last_became_active":"2024-04-13T03:11:17.299354+0000","last_became_peered":"2024-04-13T03:11:17.299354+0000","last_unstale":"2024-04-13T03:11:17.306843+0000","last_undegraded":"2024-04-13T03:11:17.306843+0000","last_fullsized":"2024-04-13T03:11:17.306843+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322007+0000","last_change":"2024-04-13T03:11:17.311835+0000","last_active":"2024-04-13T03:11:17.322007+0000","last_peered":"2024-04-13T03:11:17.322007+0000","last_clean":"2024-04-13T03:11:17.322007+0000","last_became_active":"2024-04-13T03:11:17.311218+0000","last_became_peered":"2024-04-13T03:11:17.311218+0000","last_unstale":"2024-04-13T03:11:17.322007+0000","last_undegraded":"2024-04-13T03:11:17.322007+0000","last_fullsized":"2024-04-13T03:11:17.322007+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:32:31.447158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321999+0000","last_change":"2024-04-13T03:11:17.312212+0000","last_active":"2024-04-13T03:11:17.321999+0000","last_peered":"2024-04-13T03:11:17.321999+0000","last_clean":"2024-04-13T03:11:17.321999+0000","last_became_active":"2024-04-13T03:11:17.311958+0000","last_became_peered":"2024-04-13T03:11:17.311958+0000","last_unstale":"2024-04-13T03:11:17.321999+0000","last_undegraded":"2024-04-13T03:11:17.321999+0000","last_fullsized":"2024-04-13T03:11:17.321999+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:24.111294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311003+0000","last_change":"2024-04-13T03:11:17.307907+0000","last_active":"2024-04-13T03:11:17.311003+0000","last_peered":"2024-04-13T03:11:17.311003+0000","last_clean":"2024-04-13T03:11:17.311003+0000","last_became_active":"2024-04-13T03:11:17.307627+0000","last_became_peered":"2024-04-13T03:11:17.307627+0000","last_unstale":"2024-04-13T03:11:17.311003+0000","last_undegraded":"2024-04-13T03:11:17.311003+0000","last_fullsized":"2024-04-13T03:11:17.311003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:47.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313819+0000","last_change":"2024-04-13T03:11:17.310699+0000","last_active":"2024-04-13T03:11:17.313819+0000","last_peered":"2024-04-13T03:11:17.313819+0000","last_clean":"2024-04-13T03:11:17.313819+0000","last_became_active":"2024-04-13T03:11:17.310448+0000","last_became_peered":"2024-04-13T03:11:17.310448+0000","last_unstale":"2024-04-13T03:11:17.313819+0000","last_undegraded":"2024-04-13T03:11:17.313819+0000","last_fullsized":"2024-04-13T03:11:17.313819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:11.898938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322176+0000","last_change":"2024-04-13T03:11:17.306927+0000","last_active":"2024-04-13T03:11:17.322176+0000","last_peered":"2024-04-13T03:11:17.322176+0000","last_clean":"2024-04-13T03:11:17.322176+0000","last_became_active":"2024-04-13T03:11:17.306668+0000","last_became_peered":"2024-04-13T03:11:17.306668+0000","last_unstale":"2024-04-13T03:11:17.322176+0000","last_undegraded":"2024-04-13T03:11:17.322176+0000","last_fullsized":"2024-04-13T03:11:17.322176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:16.885740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310913+0000","last_change":"2024-04-13T03:11:17.306229+0000","last_active":"2024-04-13T03:11:17.310913+0000","last_peered":"2024-04-13T03:11:17.310913+0000","last_clean":"2024-04-13T03:11:17.310913+0000","last_became_active":"2024-04-13T03:11:17.305971+0000","last_became_peered":"2024-04-13T03:11:17.305971+0000","last_unstale":"2024-04-13T03:11:17.310913+0000","last_undegraded":"2024-04-13T03:11:17.310913+0000","last_fullsized":"2024-04-13T03:11:17.310913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:22.734271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309718+0000","last_change":"2024-04-13T03:11:17.303930+0000","last_active":"2024-04-13T03:11:17.309718+0000","last_peered":"2024-04-13T03:11:17.309718+0000","last_clean":"2024-04-13T03:11:17.309718+0000","last_became_active":"2024-04-13T03:11:17.303658+0000","last_became_peered":"2024-04-13T03:11:17.303658+0000","last_unstale":"2024-04-13T03:11:17.309718+0000","last_undegraded":"2024-04-13T03:11:17.309718+0000","last_fullsized":"2024-04-13T03:11:17.309718+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:09:18.552034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316279+0000","last_change":"2024-04-13T03:11:17.313250+0000","last_active":"2024-04-13T03:11:17.316279+0000","last_peered":"2024-04-13T03:11:17.316279+0000","last_clean":"2024-04-13T03:11:17.316279+0000","last_became_active":"2024-04-13T03:11:17.312996+0000","last_became_peered":"2024-04-13T03:11:17.312996+0000","last_unstale":"2024-04-13T03:11:17.316279+0000","last_undegraded":"2024-04-13T03:11:17.316279+0000","last_fullsized":"2024-04-13T03:11:17.316279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:34.277280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.305631+0000","last_change":"2024-04-13T03:11:17.302943+0000","last_active":"2024-04-13T03:11:17.305631+0000","last_peered":"2024-04-13T03:11:17.305631+0000","last_clean":"2024-04-13T03:11:17.305631+0000","last_became_active":"2024-04-13T03:11:17.302623+0000","last_became_peered":"2024-04-13T03:11:17.302623+0000","last_unstale":"2024-04-13T03:11:17.305631+0000","last_undegraded":"2024-04-13T03:11:17.305631+0000","last_fullsized":"2024-04-13T03:11:17.305631+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:10.323440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315512+0000","last_change":"2024-04-13T03:11:17.308404+0000","last_active":"2024-04-13T03:11:17.315512+0000","last_peered":"2024-04-13T03:11:17.315512+0000","last_clean":"2024-04-13T03:11:17.315512+0000","last_became_active":"2024-04-13T03:11:17.307783+0000","last_became_peered":"2024-04-13T03:11:17.307783+0000","last_unstale":"2024-04-13T03:11:17.315512+0000","last_undegraded":"2024-04-13T03:11:17.315512+0000","last_fullsized":"2024-04-13T03:11:17.315512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:47:22.417604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323895+0000","last_change":"2024-04-13T03:11:17.317241+0000","last_active":"2024-04-13T03:11:17.323895+0000","last_peered":"2024-04-13T03:11:17.323895+0000","last_clean":"2024-04-13T03:11:17.323895+0000","last_became_active":"2024-04-13T03:11:17.316966+0000","last_became_peered":"2024-04-13T03:11:17.316966+0000","last_unstale":"2024-04-13T03:11:17.323895+0000","last_undegraded":"2024-04-13T03:11:17.323895+0000","last_fullsized":"2024-04-13T03:11:17.323895+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306847+0000","last_change":"2024-04-13T03:11:17.298579+0000","last_active":"2024-04-13T03:11:17.306847+0000","last_peered":"2024-04-13T03:11:17.306847+0000","last_clean":"2024-04-13T03:11:17.306847+0000","last_became_active":"2024-04-13T03:11:17.298289+0000","last_became_peered":"2024-04-13T03:11:17.298289+0000","last_unstale":"2024-04-13T03:11:17.306847+0000","last_undegraded":"2024-04-13T03:11:17.306847+0000","last_fullsized":"2024-04-13T03:11:17.306847+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310546+0000","last_change":"2024-04-13T03:11:17.303977+0000","last_active":"2024-04-13T03:11:17.310546+0000","last_peered":"2024-04-13T03:11:17.310546+0000","last_clean":"2024-04-13T03:11:17.310546+0000","last_became_active":"2024-04-13T03:11:17.303716+0000","last_became_peered":"2024-04-13T03:11:17.303716+0000","last_unstale":"2024-04-13T03:11:17.310546+0000","last_undegraded":"2024-04-13T03:11:17.310546+0000","last_fullsized":"2024-04-13T03:11:17.310546+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:27.322536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315715+0000","last_change":"2024-04-13T03:11:17.311333+0000","last_active":"2024-04-13T03:11:17.315715+0000","last_peered":"2024-04-13T03:11:17.315715+0000","last_clean":"2024-04-13T03:11:17.315715+0000","last_became_active":"2024-04-13T03:11:17.311041+0000","last_became_peered":"2024-04-13T03:11:17.311041+0000","last_unstale":"2024-04-13T03:11:17.315715+0000","last_undegraded":"2024-04-13T03:11:17.315715+0000","last_fullsized":"2024-04-13T03:11:17.315715+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:10:16.686388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322714+0000","last_change":"2024-04-13T03:11:17.311077+0000","last_active":"2024-04-13T03:11:17.322714+0000","last_peered":"2024-04-13T03:11:17.322714+0000","last_clean":"2024-04-13T03:11:17.322714+0000","last_became_active":"2024-04-13T03:11:17.310774+0000","last_became_peered":"2024-04-13T03:11:17.310774+0000","last_unstale":"2024-04-13T03:11:17.322714+0000","last_undegraded":"2024-04-13T03:11:17.322714+0000","last_fullsized":"2024-04-13T03:11:17.322714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:17.647839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313833+0000","last_change":"2024-04-13T03:11:17.309828+0000","last_active":"2024-04-13T03:11:17.313833+0000","last_peered":"2024-04-13T03:11:17.313833+0000","last_clean":"2024-04-13T03:11:17.313833+0000","last_became_active":"2024-04-13T03:11:17.309510+0000","last_became_peered":"2024-04-13T03:11:17.309510+0000","last_unstale":"2024-04-13T03:11:17.313833+0000","last_undegraded":"2024-04-13T03:11:17.313833+0000","last_fullsized":"2024-04-13T03:11:17.313833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:45.002908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309711+0000","last_change":"2024-04-13T03:11:17.302009+0000","last_active":"2024-04-13T03:11:17.309711+0000","last_peered":"2024-04-13T03:11:17.309711+0000","last_clean":"2024-04-13T03:11:17.309711+0000","last_became_active":"2024-04-13T03:11:17.300543+0000","last_became_peered":"2024-04-13T03:11:17.300543+0000","last_unstale":"2024-04-13T03:11:17.309711+0000","last_undegraded":"2024-04-13T03:11:17.309711+0000","last_fullsized":"2024-04-13T03:11:17.309711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:05:40.358273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315491+0000","last_change":"2024-04-13T03:11:17.307345+0000","last_active":"2024-04-13T03:11:17.315491+0000","last_peered":"2024-04-13T03:11:17.315491+0000","last_clean":"2024-04-13T03:11:17.315491+0000","last_became_active":"2024-04-13T03:11:17.307092+0000","last_became_peered":"2024-04-13T03:11:17.307092+0000","last_unstale":"2024-04-13T03:11:17.315491+0000","last_undegraded":"2024-04-13T03:11:17.315491+0000","last_fullsized":"2024-04-13T03:11:17.315491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:42.115735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321875+0000","last_change":"2024-04-13T03:11:17.308657+0000","last_active":"2024-04-13T03:11:17.321875+0000","last_peered":"2024-04-13T03:11:17.321875+0000","last_clean":"2024-04-13T03:11:17.321875+0000","last_became_active":"2024-04-13T03:11:17.308328+0000","last_became_peered":"2024-04-13T03:11:17.308328+0000","last_unstale":"2024-04-13T03:11:17.321875+0000","last_undegraded":"2024-04-13T03:11:17.321875+0000","last_fullsized":"2024-04-13T03:11:17.321875+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:52:51.855563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.308229+0000","last_change":"2024-04-13T03:11:17.304159+0000","last_active":"2024-04-13T03:11:17.308229+0000","last_peered":"2024-04-13T03:11:17.308229+0000","last_clean":"2024-04-13T03:11:17.308229+0000","last_became_active":"2024-04-13T03:11:17.303904+0000","last_became_peered":"2024-04-13T03:11:17.303904+0000","last_unstale":"2024-04-13T03:11:17.308229+0000","last_undegraded":"2024-04-13T03:11:17.308229+0000","last_fullsized":"2024-04-13T03:11:17.308229+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:30.832276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310569+0000","last_change":"2024-04-13T03:11:17.307114+0000","last_active":"2024-04-13T03:11:17.310569+0000","last_peered":"2024-04-13T03:11:17.310569+0000","last_clean":"2024-04-13T03:11:17.310569+0000","last_became_active":"2024-04-13T03:11:17.306828+0000","last_became_peered":"2024-04-13T03:11:17.306828+0000","last_unstale":"2024-04-13T03:11:17.310569+0000","last_undegraded":"2024-04-13T03:11:17.310569+0000","last_fullsized":"2024-04-13T03:11:17.310569+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:36:09.772778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310223+0000","last_change":"2024-04-13T03:11:17.303616+0000","last_active":"2024-04-13T03:11:17.310223+0000","last_peered":"2024-04-13T03:11:17.310223+0000","last_clean":"2024-04-13T03:11:17.310223+0000","last_became_active":"2024-04-13T03:11:17.303358+0000","last_became_peered":"2024-04-13T03:11:17.303358+0000","last_unstale":"2024-04-13T03:11:17.310223+0000","last_undegraded":"2024-04-13T03:11:17.310223+0000","last_fullsized":"2024-04-13T03:11:17.310223+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:16:43.165248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316285+0000","last_change":"2024-04-13T03:11:17.311260+0000","last_active":"2024-04-13T03:11:17.316285+0000","last_peered":"2024-04-13T03:11:17.316285+0000","last_clean":"2024-04-13T03:11:17.316285+0000","last_became_active":"2024-04-13T03:11:17.311015+0000","last_became_peered":"2024-04-13T03:11:17.311015+0000","last_unstale":"2024-04-13T03:11:17.316285+0000","last_undegraded":"2024-04-13T03:11:17.316285+0000","last_fullsized":"2024-04-13T03:11:17.316285+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307056+0000","last_change":"2024-04-13T03:11:17.299746+0000","last_active":"2024-04-13T03:11:17.307056+0000","last_peered":"2024-04-13T03:11:17.307056+0000","last_clean":"2024-04-13T03:11:17.307056+0000","last_became_active":"2024-04-13T03:11:17.299120+0000","last_became_peered":"2024-04-13T03:11:17.299120+0000","last_unstale":"2024-04-13T03:11:17.307056+0000","last_undegraded":"2024-04-13T03:11:17.307056+0000","last_fullsized":"2024-04-13T03:11:17.307056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316497+0000","last_change":"2024-04-13T03:11:17.306889+0000","last_active":"2024-04-13T03:11:17.316497+0000","last_peered":"2024-04-13T03:11:17.316497+0000","last_clean":"2024-04-13T03:11:17.316497+0000","last_became_active":"2024-04-13T03:11:17.306456+0000","last_became_peered":"2024-04-13T03:11:17.306456+0000","last_unstale":"2024-04-13T03:11:17.316497+0000","last_undegraded":"2024-04-13T03:11:17.316497+0000","last_fullsized":"2024-04-13T03:11:17.316497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:33.423529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323890+0000","last_change":"2024-04-13T03:11:17.317468+0000","last_active":"2024-04-13T03:11:17.323890+0000","last_peered":"2024-04-13T03:11:17.323890+0000","last_clean":"2024-04-13T03:11:17.323890+0000","last_became_active":"2024-04-13T03:11:17.317158+0000","last_became_peered":"2024-04-13T03:11:17.317158+0000","last_unstale":"2024-04-13T03:11:17.323890+0000","last_undegraded":"2024-04-13T03:11:17.323890+0000","last_fullsized":"2024-04-13T03:11:17.323890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318758+0000","last_change":"2024-04-13T03:11:16.288090+0000","last_active":"2024-04-13T03:11:17.318758+0000","last_peered":"2024-04-13T03:11:17.318758+0000","last_clean":"2024-04-13T03:11:17.318758+0000","last_became_active":"2024-04-13T03:11:16.287829+0000","last_became_peered":"2024-04-13T03:11:16.287829+0000","last_unstale":"2024-04-13T03:11:17.318758+0000","last_undegraded":"2024-04-13T03:11:17.318758+0000","last_fullsized":"2024-04-13T03:11:17.318758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:14:53.458260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316408+0000","last_change":"2024-04-13T03:11:17.310642+0000","last_active":"2024-04-13T03:11:17.316408+0000","last_peered":"2024-04-13T03:11:17.316408+0000","last_clean":"2024-04-13T03:11:17.316408+0000","last_became_active":"2024-04-13T03:11:17.310332+0000","last_became_peered":"2024-04-13T03:11:17.310332+0000","last_unstale":"2024-04-13T03:11:17.316408+0000","last_undegraded":"2024-04-13T03:11:17.316408+0000","last_fullsized":"2024-04-13T03:11:17.316408+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322909+0000","last_change":"2024-04-13T03:11:16.290002+0000","last_active":"2024-04-13T03:11:17.322909+0000","last_peered":"2024-04-13T03:11:17.322909+0000","last_clean":"2024-04-13T03:11:17.322909+0000","last_became_active":"2024-04-13T03:11:16.289716+0000","last_became_peered":"2024-04-13T03:11:16.289716+0000","last_unstale":"2024-04-13T03:11:17.322909+0000","last_undegraded":"2024-04-13T03:11:17.322909+0000","last_fullsized":"2024-04-13T03:11:17.322909+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:10:15.680051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311020+0000","last_change":"2024-04-13T03:11:17.308034+0000","last_active":"2024-04-13T03:11:17.311020+0000","last_peered":"2024-04-13T03:11:17.311020+0000","last_clean":"2024-04-13T03:11:17.311020+0000","last_became_active":"2024-04-13T03:11:17.307741+0000","last_became_peered":"2024-04-13T03:11:17.307741+0000","last_unstale":"2024-04-13T03:11:17.311020+0000","last_undegraded":"2024-04-13T03:11:17.311020+0000","last_fullsized":"2024-04-13T03:11:17.311020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:57:34.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322017+0000","last_change":"2024-04-13T03:11:16.291777+0000","last_active":"2024-04-13T03:11:17.322017+0000","last_peered":"2024-04-13T03:11:17.322017+0000","last_clean":"2024-04-13T03:11:17.322017+0000","last_became_active":"2024-04-13T03:11:16.290999+0000","last_became_peered":"2024-04-13T03:11:16.290999+0000","last_unstale":"2024-04-13T03:11:17.322017+0000","last_undegraded":"2024-04-13T03:11:17.322017+0000","last_fullsized":"2024-04-13T03:11:17.322017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:24.983066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317523+0000","last_change":"2024-04-13T03:11:17.305054+0000","last_active":"2024-04-13T03:11:17.317523+0000","last_peered":"2024-04-13T03:11:17.317523+0000","last_clean":"2024-04-13T03:11:17.317523+0000","last_became_active":"2024-04-13T03:11:17.304628+0000","last_became_peered":"2024-04-13T03:11:17.304628+0000","last_unstale":"2024-04-13T03:11:17.317523+0000","last_undegraded":"2024-04-13T03:11:17.317523+0000","last_fullsized":"2024-04-13T03:11:17.317523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:35.617657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322811+0000","last_change":"2024-04-13T03:11:16.286797+0000","last_active":"2024-04-13T03:11:17.322811+0000","last_peered":"2024-04-13T03:11:17.322811+0000","last_clean":"2024-04-13T03:11:17.322811+0000","last_became_active":"2024-04-13T03:11:16.285784+0000","last_became_peered":"2024-04-13T03:11:16.285784+0000","last_unstale":"2024-04-13T03:11:17.322811+0000","last_undegraded":"2024-04-13T03:11:17.322811+0000","last_fullsized":"2024-04-13T03:11:17.322811+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:28.436370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.308263+0000","last_change":"2024-04-13T03:11:17.303820+0000","last_active":"2024-04-13T03:11:17.308263+0000","last_peered":"2024-04-13T03:11:17.308263+0000","last_clean":"2024-04-13T03:11:17.308263+0000","last_became_active":"2024-04-13T03:11:17.303574+0000","last_became_peered":"2024-04-13T03:11:17.303574+0000","last_unstale":"2024-04-13T03:11:17.308263+0000","last_undegraded":"2024-04-13T03:11:17.308263+0000","last_fullsized":"2024-04-13T03:11:17.308263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:25.706262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322422+0000","last_change":"2024-04-13T03:11:16.287747+0000","last_active":"2024-04-13T03:11:17.322422+0000","last_peered":"2024-04-13T03:11:17.322422+0000","last_clean":"2024-04-13T03:11:17.322422+0000","last_became_active":"2024-04-13T03:11:16.287189+0000","last_became_peered":"2024-04-13T03:11:16.287189+0000","last_unstale":"2024-04-13T03:11:17.322422+0000","last_undegraded":"2024-04-13T03:11:17.322422+0000","last_fullsized":"2024-04-13T03:11:17.322422+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316260+0000","last_change":"2024-04-13T03:11:17.311897+0000","last_active":"2024-04-13T03:11:17.316260+0000","last_peered":"2024-04-13T03:11:17.316260+0000","last_clean":"2024-04-13T03:11:17.316260+0000","last_became_active":"2024-04-13T03:11:17.311654+0000","last_became_peered":"2024-04-13T03:11:17.311654+0000","last_unstale":"2024-04-13T03:11:17.316260+0000","last_undegraded":"2024-04-13T03:11:17.316260+0000","last_fullsized":"2024-04-13T03:11:17.316260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315956+0000","last_change":"2024-04-13T03:11:16.289174+0000","last_active":"2024-04-13T03:11:17.315956+0000","last_peered":"2024-04-13T03:11:17.315956+0000","last_clean":"2024-04-13T03:11:17.315956+0000","last_became_active":"2024-04-13T03:11:16.288844+0000","last_became_peered":"2024-04-13T03:11:16.288844+0000","last_unstale":"2024-04-13T03:11:17.315956+0000","last_undegraded":"2024-04-13T03:11:17.315956+0000","last_fullsized":"2024-04-13T03:11:17.315956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:08:21.895972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317585+0000","last_change":"2024-04-13T03:11:17.306927+0000","last_active":"2024-04-13T03:11:17.317585+0000","last_peered":"2024-04-13T03:11:17.317585+0000","last_clean":"2024-04-13T03:11:17.317585+0000","last_became_active":"2024-04-13T03:11:17.306677+0000","last_became_peered":"2024-04-13T03:11:17.306677+0000","last_unstale":"2024-04-13T03:11:17.317585+0000","last_undegraded":"2024-04-13T03:11:17.317585+0000","last_fullsized":"2024-04-13T03:11:17.317585+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:22.833136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.297107+0000","last_change":"2024-04-13T03:11:16.292442+0000","last_active":"2024-04-13T03:11:17.297107+0000","last_peered":"2024-04-13T03:11:17.297107+0000","last_clean":"2024-04-13T03:11:17.297107+0000","last_became_active":"2024-04-13T03:11:16.292176+0000","last_became_peered":"2024-04-13T03:11:16.292176+0000","last_unstale":"2024-04-13T03:11:17.297107+0000","last_undegraded":"2024-04-13T03:11:17.297107+0000","last_fullsized":"2024-04-13T03:11:17.297107+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:12:59.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316241+0000","last_change":"2024-04-13T03:11:17.311970+0000","last_active":"2024-04-13T03:11:17.316241+0000","last_peered":"2024-04-13T03:11:17.316241+0000","last_clean":"2024-04-13T03:11:17.316241+0000","last_became_active":"2024-04-13T03:11:17.311722+0000","last_became_peered":"2024-04-13T03:11:17.311722+0000","last_unstale":"2024-04-13T03:11:17.316241+0000","last_undegraded":"2024-04-13T03:11:17.316241+0000","last_fullsized":"2024-04-13T03:11:17.316241+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.290766+0000","last_change":"2024-04-13T03:11:16.286594+0000","last_active":"2024-04-13T03:11:17.290766+0000","last_peered":"2024-04-13T03:11:17.290766+0000","last_clean":"2024-04-13T03:11:17.290766+0000","last_became_active":"2024-04-13T03:11:16.286337+0000","last_became_peered":"2024-04-13T03:11:16.286337+0000","last_unstale":"2024-04-13T03:11:17.290766+0000","last_undegraded":"2024-04-13T03:11:17.290766+0000","last_fullsized":"2024-04-13T03:11:17.290766+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:24.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.314227+0000","last_change":"2024-04-13T03:11:17.306492+0000","last_active":"2024-04-13T03:11:17.314227+0000","last_peered":"2024-04-13T03:11:17.314227+0000","last_clean":"2024-04-13T03:11:17.314227+0000","last_became_active":"2024-04-13T03:11:17.305879+0000","last_became_peered":"2024-04-13T03:11:17.305879+0000","last_unstale":"2024-04-13T03:11:17.314227+0000","last_undegraded":"2024-04-13T03:11:17.314227+0000","last_fullsized":"2024-04-13T03:11:17.314227+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:47.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.303307+0000","last_change":"2024-04-13T03:11:16.291313+0000","last_active":"2024-04-13T03:11:17.303307+0000","last_peered":"2024-04-13T03:11:17.303307+0000","last_clean":"2024-04-13T03:11:17.303307+0000","last_became_active":"2024-04-13T03:11:16.291043+0000","last_became_peered":"2024-04-13T03:11:16.291043+0000","last_unstale":"2024-04-13T03:11:17.303307+0000","last_undegraded":"2024-04-13T03:11:17.303307+0000","last_fullsized":"2024-04-13T03:11:17.303307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:29.825939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313929+0000","last_change":"2024-04-13T03:11:17.302490+0000","last_active":"2024-04-13T03:11:17.313929+0000","last_peered":"2024-04-13T03:11:17.313929+0000","last_clean":"2024-04-13T03:11:17.313929+0000","last_became_active":"2024-04-13T03:11:17.302219+0000","last_became_peered":"2024-04-13T03:11:17.302219+0000","last_unstale":"2024-04-13T03:11:17.313929+0000","last_undegraded":"2024-04-13T03:11:17.313929+0000","last_fullsized":"2024-04-13T03:11:17.313929+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293028+0000","last_change":"2024-04-13T03:11:16.293423+0000","last_active":"2024-04-13T03:11:17.293028+0000","last_peered":"2024-04-13T03:11:17.293028+0000","last_clean":"2024-04-13T03:11:17.293028+0000","last_became_active":"2024-04-13T03:11:16.293103+0000","last_became_peered":"2024-04-13T03:11:16.293103+0000","last_unstale":"2024-04-13T03:11:17.293028+0000","last_undegraded":"2024-04-13T03:11:17.293028+0000","last_fullsized":"2024-04-13T03:11:17.293028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:44.531785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309741+0000","last_change":"2024-04-13T03:11:17.303622+0000","last_active":"2024-04-13T03:11:17.309741+0000","last_peered":"2024-04-13T03:11:17.309741+0000","last_clean":"2024-04-13T03:11:17.309741+0000","last_became_active":"2024-04-13T03:11:17.303375+0000","last_became_peered":"2024-04-13T03:11:17.303375+0000","last_unstale":"2024-04-13T03:11:17.309741+0000","last_undegraded":"2024-04-13T03:11:17.309741+0000","last_fullsized":"2024-04-13T03:11:17.309741+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:24:11.365885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317076+0000","last_change":"2024-04-13T03:11:16.296242+0000","last_active":"2024-04-13T03:11:17.317076+0000","last_peered":"2024-04-13T03:11:17.317076+0000","last_clean":"2024-04-13T03:11:17.317076+0000","last_became_active":"2024-04-13T03:11:16.295965+0000","last_became_peered":"2024-04-13T03:11:16.295965+0000","last_unstale":"2024-04-13T03:11:17.317076+0000","last_undegraded":"2024-04-13T03:11:17.317076+0000","last_fullsized":"2024-04-13T03:11:17.317076+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:24.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304281+0000","last_change":"2024-04-13T03:11:17.301654+0000","last_active":"2024-04-13T03:11:17.304281+0000","last_peered":"2024-04-13T03:11:17.304281+0000","last_clean":"2024-04-13T03:11:17.304281+0000","last_became_active":"2024-04-13T03:11:17.301379+0000","last_became_peered":"2024-04-13T03:11:17.301379+0000","last_unstale":"2024-04-13T03:11:17.304281+0000","last_undegraded":"2024-04-13T03:11:17.304281+0000","last_fullsized":"2024-04-13T03:11:17.304281+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:22.734271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.326537+0000","last_change":"2024-04-13T03:11:16.291833+0000","last_active":"2024-04-13T03:11:17.326537+0000","last_peered":"2024-04-13T03:11:17.326537+0000","last_clean":"2024-04-13T03:11:17.326537+0000","last_became_active":"2024-04-13T03:11:16.291553+0000","last_became_peered":"2024-04-13T03:11:16.291553+0000","last_unstale":"2024-04-13T03:11:17.326537+0000","last_undegraded":"2024-04-13T03:11:17.326537+0000","last_fullsized":"2024-04-13T03:11:17.326537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:47:21.411267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316424+0000","last_change":"2024-04-13T03:11:17.310877+0000","last_active":"2024-04-13T03:11:17.316424+0000","last_peered":"2024-04-13T03:11:17.316424+0000","last_clean":"2024-04-13T03:11:17.316424+0000","last_became_active":"2024-04-13T03:11:17.309995+0000","last_became_peered":"2024-04-13T03:11:17.309995+0000","last_unstale":"2024-04-13T03:11:17.316424+0000","last_undegraded":"2024-04-13T03:11:17.316424+0000","last_fullsized":"2024-04-13T03:11:17.316424+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.324104+0000","last_change":"2024-04-13T03:11:16.287875+0000","last_active":"2024-04-13T03:11:17.324104+0000","last_peered":"2024-04-13T03:11:17.324104+0000","last_clean":"2024-04-13T03:11:17.324104+0000","last_became_active":"2024-04-13T03:11:16.287485+0000","last_became_peered":"2024-04-13T03:11:16.287485+0000","last_unstale":"2024-04-13T03:11:17.324104+0000","last_undegraded":"2024-04-13T03:11:17.324104+0000","last_fullsized":"2024-04-13T03:11:17.324104+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:28.436370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306907+0000","last_change":"2024-04-13T03:11:17.299767+0000","last_active":"2024-04-13T03:11:17.306907+0000","last_peered":"2024-04-13T03:11:17.306907+0000","last_clean":"2024-04-13T03:11:17.306907+0000","last_became_active":"2024-04-13T03:11:17.299514+0000","last_became_peered":"2024-04-13T03:11:17.299514+0000","last_unstale":"2024-04-13T03:11:17.306907+0000","last_undegraded":"2024-04-13T03:11:17.306907+0000","last_fullsized":"2024-04-13T03:11:17.306907+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:17:21.128440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.297750+0000","last_change":"2024-04-13T03:11:16.293489+0000","last_active":"2024-04-13T03:11:17.297750+0000","last_peered":"2024-04-13T03:11:17.297750+0000","last_clean":"2024-04-13T03:11:17.297750+0000","last_became_active":"2024-04-13T03:11:16.293189+0000","last_became_peered":"2024-04-13T03:11:16.293189+0000","last_unstale":"2024-04-13T03:11:17.297750+0000","last_undegraded":"2024-04-13T03:11:17.297750+0000","last_fullsized":"2024-04-13T03:11:17.297750+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:34:00.330011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317604+0000","last_change":"2024-04-13T03:11:17.307823+0000","last_active":"2024-04-13T03:11:17.317604+0000","last_peered":"2024-04-13T03:11:17.317604+0000","last_clean":"2024-04-13T03:11:17.317604+0000","last_became_active":"2024-04-13T03:11:17.307575+0000","last_became_peered":"2024-04-13T03:11:17.307575+0000","last_unstale":"2024-04-13T03:11:17.317604+0000","last_undegraded":"2024-04-13T03:11:17.317604+0000","last_fullsized":"2024-04-13T03:11:17.317604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:52:48.380574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304098+0000","last_change":"2024-04-13T03:11:16.290805+0000","last_active":"2024-04-13T03:11:17.304098+0000","last_peered":"2024-04-13T03:11:17.304098+0000","last_clean":"2024-04-13T03:11:17.304098+0000","last_became_active":"2024-04-13T03:11:16.290500+0000","last_became_peered":"2024-04-13T03:11:16.290500+0000","last_unstale":"2024-04-13T03:11:17.304098+0000","last_undegraded":"2024-04-13T03:11:17.304098+0000","last_fullsized":"2024-04-13T03:11:17.304098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:21.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317225+0000","last_change":"2024-04-13T03:11:17.310475+0000","last_active":"2024-04-13T03:11:17.317225+0000","last_peered":"2024-04-13T03:11:17.317225+0000","last_clean":"2024-04-13T03:11:17.317225+0000","last_became_active":"2024-04-13T03:11:17.310004+0000","last_became_peered":"2024-04-13T03:11:17.310004+0000","last_unstale":"2024-04-13T03:11:17.317225+0000","last_undegraded":"2024-04-13T03:11:17.317225+0000","last_fullsized":"2024-04-13T03:11:17.317225+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:35.617657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.324086+0000","last_change":"2024-04-13T03:11:16.287295+0000","last_active":"2024-04-13T03:11:17.324086+0000","last_peered":"2024-04-13T03:11:17.324086+0000","last_clean":"2024-04-13T03:11:17.324086+0000","last_became_active":"2024-04-13T03:11:16.287024+0000","last_became_peered":"2024-04-13T03:11:16.287024+0000","last_unstale":"2024-04-13T03:11:17.324086+0000","last_undegraded":"2024-04-13T03:11:17.324086+0000","last_fullsized":"2024-04-13T03:11:17.324086+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:05:13.291961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323643+0000","last_change":"2024-04-13T03:11:17.317185+0000","last_active":"2024-04-13T03:11:17.323643+0000","last_peered":"2024-04-13T03:11:17.323643+0000","last_clean":"2024-04-13T03:11:17.323643+0000","last_became_active":"2024-04-13T03:11:17.314734+0000","last_became_peered":"2024-04-13T03:11:17.314734+0000","last_unstale":"2024-04-13T03:11:17.323643+0000","last_undegraded":"2024-04-13T03:11:17.323643+0000","last_fullsized":"2024-04-13T03:11:17.323643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:07.922441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322922+0000","last_change":"2024-04-13T03:11:16.287839+0000","last_active":"2024-04-13T03:11:17.322922+0000","last_peered":"2024-04-13T03:11:17.322922+0000","last_clean":"2024-04-13T03:11:17.322922+0000","last_became_active":"2024-04-13T03:11:16.287582+0000","last_became_peered":"2024-04-13T03:11:16.287582+0000","last_unstale":"2024-04-13T03:11:17.322922+0000","last_undegraded":"2024-04-13T03:11:17.322922+0000","last_fullsized":"2024-04-13T03:11:17.322922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:32.417192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309481+0000","last_change":"2024-04-13T03:11:17.303324+0000","last_active":"2024-04-13T03:11:17.309481+0000","last_peered":"2024-04-13T03:11:17.309481+0000","last_clean":"2024-04-13T03:11:17.309481+0000","last_became_active":"2024-04-13T03:11:17.303015+0000","last_became_peered":"2024-04-13T03:11:17.303015+0000","last_unstale":"2024-04-13T03:11:17.309481+0000","last_undegraded":"2024-04-13T03:11:17.309481+0000","last_fullsized":"2024-04-13T03:11:17.309481+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:08:22.902309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317036+0000","last_change":"2024-04-13T03:11:16.291129+0000","last_active":"2024-04-13T03:11:17.317036+0000","last_peered":"2024-04-13T03:11:17.317036+0000","last_clean":"2024-04-13T03:11:17.317036+0000","last_became_active":"2024-04-13T03:11:16.290717+0000","last_became_peered":"2024-04-13T03:11:16.290717+0000","last_unstale":"2024-04-13T03:11:17.317036+0000","last_undegraded":"2024-04-13T03:11:17.317036+0000","last_fullsized":"2024-04-13T03:11:17.317036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:14:21.972935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.302593+0000","last_change":"2024-04-13T03:11:17.299719+0000","last_active":"2024-04-13T03:11:17.302593+0000","last_peered":"2024-04-13T03:11:17.302593+0000","last_clean":"2024-04-13T03:11:17.302593+0000","last_became_active":"2024-04-13T03:11:17.299425+0000","last_became_peered":"2024-04-13T03:11:17.299425+0000","last_unstale":"2024-04-13T03:11:17.302593+0000","last_undegraded":"2024-04-13T03:11:17.302593+0000","last_fullsized":"2024-04-13T03:11:17.302593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:30.832276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318740+0000","last_change":"2024-04-13T03:11:16.293749+0000","last_active":"2024-04-13T03:11:17.318740+0000","last_peered":"2024-04-13T03:11:17.318740+0000","last_clean":"2024-04-13T03:11:17.318740+0000","last_became_active":"2024-04-13T03:11:16.293341+0000","last_became_peered":"2024-04-13T03:11:16.293341+0000","last_unstale":"2024-04-13T03:11:17.318740+0000","last_undegraded":"2024-04-13T03:11:17.318740+0000","last_fullsized":"2024-04-13T03:11:17.318740+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:43:38.828201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322288+0000","last_change":"2024-04-13T03:11:17.314431+0000","last_active":"2024-04-13T03:11:17.322288+0000","last_peered":"2024-04-13T03:11:17.322288+0000","last_clean":"2024-04-13T03:11:17.322288+0000","last_became_active":"2024-04-13T03:11:17.314173+0000","last_became_peered":"2024-04-13T03:11:17.314173+0000","last_unstale":"2024-04-13T03:11:17.322288+0000","last_undegraded":"2024-04-13T03:11:17.322288+0000","last_fullsized":"2024-04-13T03:11:17.322288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.298869+0000","last_change":"2024-04-13T03:11:16.290092+0000","last_active":"2024-04-13T03:11:17.298869+0000","last_peered":"2024-04-13T03:11:17.298869+0000","last_clean":"2024-04-13T03:11:17.298869+0000","last_became_active":"2024-04-13T03:11:16.289797+0000","last_became_peered":"2024-04-13T03:11:16.289797+0000","last_unstale":"2024-04-13T03:11:17.298869+0000","last_undegraded":"2024-04-13T03:11:17.298869+0000","last_fullsized":"2024-04-13T03:11:17.298869+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:43.465157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313961+0000","last_change":"2024-04-13T03:11:17.303994+0000","last_active":"2024-04-13T03:11:17.313961+0000","last_peered":"2024-04-13T03:11:17.313961+0000","last_clean":"2024-04-13T03:11:17.313961+0000","last_became_active":"2024-04-13T03:11:17.303624+0000","last_became_peered":"2024-04-13T03:11:17.303624+0000","last_unstale":"2024-04-13T03:11:17.313961+0000","last_undegraded":"2024-04-13T03:11:17.313961+0000","last_fullsized":"2024-04-13T03:11:17.313961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322316+0000","last_change":"2024-04-13T03:11:16.294336+0000","last_active":"2024-04-13T03:11:17.322316+0000","last_peered":"2024-04-13T03:11:17.322316+0000","last_clean":"2024-04-13T03:11:17.322316+0000","last_became_active":"2024-04-13T03:11:16.294063+0000","last_became_peered":"2024-04-13T03:11:16.294063+0000","last_unstale":"2024-04-13T03:11:17.322316+0000","last_undegraded":"2024-04-13T03:11:17.322316+0000","last_fullsized":"2024-04-13T03:11:17.322316+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:57:28.027417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311345+0000","last_change":"2024-04-13T03:11:17.303393+0000","last_active":"2024-04-13T03:11:17.311345+0000","last_peered":"2024-04-13T03:11:17.311345+0000","last_clean":"2024-04-13T03:11:17.311345+0000","last_became_active":"2024-04-13T03:11:17.303109+0000","last_became_peered":"2024-04-13T03:11:17.303109+0000","last_unstale":"2024-04-13T03:11:17.311345+0000","last_undegraded":"2024-04-13T03:11:17.311345+0000","last_fullsized":"2024-04-13T03:11:17.311345+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:54:25.895429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.299995+0000","last_change":"2024-04-13T03:11:16.292326+0000","last_active":"2024-04-13T03:11:17.299995+0000","last_peered":"2024-04-13T03:11:17.299995+0000","last_clean":"2024-04-13T03:11:17.299995+0000","last_became_active":"2024-04-13T03:11:16.292079+0000","last_became_peered":"2024-04-13T03:11:16.292079+0000","last_unstale":"2024-04-13T03:11:17.299995+0000","last_undegraded":"2024-04-13T03:11:17.299995+0000","last_fullsized":"2024-04-13T03:11:17.299995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:24:10.359548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309812+0000","last_change":"2024-04-13T03:11:17.304267+0000","last_active":"2024-04-13T03:11:17.309812+0000","last_peered":"2024-04-13T03:11:17.309812+0000","last_clean":"2024-04-13T03:11:17.309812+0000","last_became_active":"2024-04-13T03:11:17.303881+0000","last_became_peered":"2024-04-13T03:11:17.303881+0000","last_unstale":"2024-04-13T03:11:17.309812+0000","last_undegraded":"2024-04-13T03:11:17.309812+0000","last_fullsized":"2024-04-13T03:11:17.309812+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:05:14.298298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.291208+0000","last_change":"2024-04-13T03:11:16.295696+0000","last_active":"2024-04-13T03:11:17.291208+0000","last_peered":"2024-04-13T03:11:17.291208+0000","last_clean":"2024-04-13T03:11:17.291208+0000","last_became_active":"2024-04-13T03:11:16.294249+0000","last_became_peered":"2024-04-13T03:11:16.294249+0000","last_unstale":"2024-04-13T03:11:17.291208+0000","last_undegraded":"2024-04-13T03:11:17.291208+0000","last_fullsized":"2024-04-13T03:11:17.291208+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:53.431976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311037+0000","last_change":"2024-04-13T03:11:17.307691+0000","last_active":"2024-04-13T03:11:17.311037+0000","last_peered":"2024-04-13T03:11:17.311037+0000","last_clean":"2024-04-13T03:11:17.311037+0000","last_became_active":"2024-04-13T03:11:17.307101+0000","last_became_peered":"2024-04-13T03:11:17.307101+0000","last_unstale":"2024-04-13T03:11:17.311037+0000","last_undegraded":"2024-04-13T03:11:17.311037+0000","last_fullsized":"2024-04-13T03:11:17.311037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:25.706262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.289416+0000","last_change":"2024-04-13T03:11:16.290729+0000","last_active":"2024-04-13T03:11:17.289416+0000","last_peered":"2024-04-13T03:11:17.289416+0000","last_clean":"2024-04-13T03:11:17.289416+0000","last_became_active":"2024-04-13T03:11:16.290454+0000","last_became_peered":"2024-04-13T03:11:16.290454+0000","last_unstale":"2024-04-13T03:11:17.289416+0000","last_undegraded":"2024-04-13T03:11:17.289416+0000","last_fullsized":"2024-04-13T03:11:17.289416+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:31.829634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316472+0000","last_change":"2024-04-13T03:11:17.306070+0000","last_active":"2024-04-13T03:11:17.316472+0000","last_peered":"2024-04-13T03:11:17.316472+0000","last_clean":"2024-04-13T03:11:17.316472+0000","last_became_active":"2024-04-13T03:11:17.305308+0000","last_became_peered":"2024-04-13T03:11:17.305308+0000","last_unstale":"2024-04-13T03:11:17.316472+0000","last_undegraded":"2024-04-13T03:11:17.316472+0000","last_fullsized":"2024-04-13T03:11:17.316472+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:34.277280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317674+0000","last_change":"2024-04-13T03:11:16.294574+0000","last_active":"2024-04-13T03:11:17.317674+0000","last_peered":"2024-04-13T03:11:17.317674+0000","last_clean":"2024-04-13T03:11:17.317674+0000","last_became_active":"2024-04-13T03:11:16.294353+0000","last_became_peered":"2024-04-13T03:11:16.294353+0000","last_unstale":"2024-04-13T03:11:17.317674+0000","last_undegraded":"2024-04-13T03:11:17.317674+0000","last_fullsized":"2024-04-13T03:11:17.317674+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:57:24.454175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307477+0000","last_change":"2024-04-13T03:11:17.303017+0000","last_active":"2024-04-13T03:11:17.307477+0000","last_peered":"2024-04-13T03:11:17.307477+0000","last_clean":"2024-04-13T03:11:17.307477+0000","last_became_active":"2024-04-13T03:11:17.302723+0000","last_became_peered":"2024-04-13T03:11:17.302723+0000","last_unstale":"2024-04-13T03:11:17.307477+0000","last_undegraded":"2024-04-13T03:11:17.307477+0000","last_fullsized":"2024-04-13T03:11:17.307477+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:45.538123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304809+0000","last_change":"2024-04-13T03:11:16.292685+0000","last_active":"2024-04-13T03:11:17.304809+0000","last_peered":"2024-04-13T03:11:17.304809+0000","last_clean":"2024-04-13T03:11:17.304809+0000","last_became_active":"2024-04-13T03:11:16.292391+0000","last_became_peered":"2024-04-13T03:11:16.292391+0000","last_unstale":"2024-04-13T03:11:17.304809+0000","last_undegraded":"2024-04-13T03:11:17.304809+0000","last_fullsized":"2024-04-13T03:11:17.304809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317545+0000","last_change":"2024-04-13T03:11:17.305645+0000","last_active":"2024-04-13T03:11:17.317545+0000","last_peered":"2024-04-13T03:11:17.317545+0000","last_clean":"2024-04-13T03:11:17.317545+0000","last_became_active":"2024-04-13T03:11:17.305355+0000","last_became_peered":"2024-04-13T03:11:17.305355+0000","last_unstale":"2024-04-13T03:11:17.317545+0000","last_undegraded":"2024-04-13T03:11:17.317545+0000","last_fullsized":"2024-04-13T03:11:17.317545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:42.136996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.292245+0000","last_change":"2024-04-13T03:11:16.288483+0000","last_active":"2024-04-13T03:11:17.292245+0000","last_peered":"2024-04-13T03:11:17.292245+0000","last_clean":"2024-04-13T03:11:17.292245+0000","last_became_active":"2024-04-13T03:11:16.288222+0000","last_became_peered":"2024-04-13T03:11:16.288222+0000","last_unstale":"2024-04-13T03:11:17.292245+0000","last_undegraded":"2024-04-13T03:11:17.292245+0000","last_fullsized":"2024-04-13T03:11:17.292245+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:42.019175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315466+0000","last_change":"2024-04-13T03:11:17.308242+0000","last_active":"2024-04-13T03:11:17.315466+0000","last_peered":"2024-04-13T03:11:17.315466+0000","last_clean":"2024-04-13T03:11:17.315466+0000","last_became_active":"2024-04-13T03:11:17.307966+0000","last_became_peered":"2024-04-13T03:11:17.307966+0000","last_unstale":"2024-04-13T03:11:17.315466+0000","last_undegraded":"2024-04-13T03:11:17.315466+0000","last_fullsized":"2024-04-13T03:11:17.315466+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:48:23.922890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300395+0000","last_change":"2024-04-13T03:11:16.283111+0000","last_active":"2024-04-13T03:11:17.300395+0000","last_peered":"2024-04-13T03:11:17.300395+0000","last_clean":"2024-04-13T03:11:17.300395+0000","last_became_active":"2024-04-13T03:11:16.282850+0000","last_became_peered":"2024-04-13T03:11:16.282850+0000","last_unstale":"2024-04-13T03:11:17.300395+0000","last_undegraded":"2024-04-13T03:11:17.300395+0000","last_fullsized":"2024-04-13T03:11:17.300395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:48:22.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316417+0000","last_change":"2024-04-13T03:11:17.310842+0000","last_active":"2024-04-13T03:11:17.316417+0000","last_peered":"2024-04-13T03:11:17.316417+0000","last_clean":"2024-04-13T03:11:17.316417+0000","last_became_active":"2024-04-13T03:11:17.310592+0000","last_became_peered":"2024-04-13T03:11:17.310592+0000","last_unstale":"2024-04-13T03:11:17.316417+0000","last_undegraded":"2024-04-13T03:11:17.316417+0000","last_fullsized":"2024-04-13T03:11:17.316417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300141+0000","last_change":"2024-04-13T03:11:16.294484+0000","last_active":"2024-04-13T03:11:17.300141+0000","last_peered":"2024-04-13T03:11:17.300141+0000","last_clean":"2024-04-13T03:11:17.300141+0000","last_became_active":"2024-04-13T03:11:16.294223+0000","last_became_peered":"2024-04-13T03:11:16.294223+0000","last_unstale":"2024-04-13T03:11:17.300141+0000","last_undegraded":"2024-04-13T03:11:17.300141+0000","last_fullsized":"2024-04-13T03:11:17.300141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:41.109398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307059+0000","last_change":"2024-04-13T03:11:17.301528+0000","last_active":"2024-04-13T03:11:17.307059+0000","last_peered":"2024-04-13T03:11:17.307059+0000","last_clean":"2024-04-13T03:11:17.307059+0000","last_became_active":"2024-04-13T03:11:17.301277+0000","last_became_peered":"2024-04-13T03:11:17.301277+0000","last_unstale":"2024-04-13T03:11:17.307059+0000","last_undegraded":"2024-04-13T03:11:17.307059+0000","last_fullsized":"2024-04-13T03:11:17.307059+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:53:58.099757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293006+0000","last_change":"2024-04-13T03:11:16.289098+0000","last_active":"2024-04-13T03:11:17.293006+0000","last_peered":"2024-04-13T03:11:17.293006+0000","last_clean":"2024-04-13T03:11:17.293006+0000","last_became_active":"2024-04-13T03:11:16.288450+0000","last_became_peered":"2024-04-13T03:11:16.288450+0000","last_unstale":"2024-04-13T03:11:17.293006+0000","last_undegraded":"2024-04-13T03:11:17.293006+0000","last_fullsized":"2024-04-13T03:11:17.293006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:09.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321974+0000","last_change":"2024-04-13T03:11:17.311576+0000","last_active":"2024-04-13T03:11:17.321974+0000","last_peered":"2024-04-13T03:11:17.321974+0000","last_clean":"2024-04-13T03:11:17.321974+0000","last_became_active":"2024-04-13T03:11:17.308969+0000","last_became_peered":"2024-04-13T03:11:17.308969+0000","last_unstale":"2024-04-13T03:11:17.321974+0000","last_undegraded":"2024-04-13T03:11:17.321974+0000","last_fullsized":"2024-04-13T03:11:17.321974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:49:58.487036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293431+0000","last_change":"2024-04-13T03:11:16.282731+0000","last_active":"2024-04-13T03:11:17.293431+0000","last_peered":"2024-04-13T03:11:17.293431+0000","last_clean":"2024-04-13T03:11:17.293431+0000","last_became_active":"2024-04-13T03:11:16.282448+0000","last_became_peered":"2024-04-13T03:11:16.282448+0000","last_unstale":"2024-04-13T03:11:17.293431+0000","last_undegraded":"2024-04-13T03:11:17.293431+0000","last_fullsized":"2024-04-13T03:11:17.293431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318781+0000","last_change":"2024-04-13T03:11:17.314498+0000","last_active":"2024-04-13T03:11:17.318781+0000","last_peered":"2024-04-13T03:11:17.318781+0000","last_clean":"2024-04-13T03:11:17.318781+0000","last_became_active":"2024-04-13T03:11:17.314184+0000","last_became_peered":"2024-04-13T03:11:17.314184+0000","last_unstale":"2024-04-13T03:11:17.318781+0000","last_undegraded":"2024-04-13T03:11:17.318781+0000","last_fullsized":"2024-04-13T03:11:17.318781+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:57:29.033754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.292464+0000","last_change":"2024-04-13T03:11:16.287541+0000","last_active":"2024-04-13T03:11:17.292464+0000","last_peered":"2024-04-13T03:11:17.292464+0000","last_clean":"2024-04-13T03:11:17.292464+0000","last_became_active":"2024-04-13T03:11:16.287126+0000","last_became_peered":"2024-04-13T03:11:16.287126+0000","last_unstale":"2024-04-13T03:11:17.292464+0000","last_undegraded":"2024-04-13T03:11:17.292464+0000","last_fullsized":"2024-04-13T03:11:17.292464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:33.270943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300427+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:11:17.300427+0000","last_peered":"2024-04-13T03:11:17.300427+0000","last_clean":"2024-04-13T03:11:17.300427+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:11:17.300427+0000","last_undegraded":"2024-04-13T03:11:17.300427+0000","last_fullsized":"2024-04-13T03:11:17.300427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1187840,"data_stored":853335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":271768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1187840,"data_stored":853335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:11:20.270 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:11:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.405+0000 7fd3bcc2d640 1 Processor -- start 2024-04-13T03:11:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.405+0000 7fd3bcc2d640 1 -- start start 2024-04-13T03:11:20.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3bcc2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd3b80b68d0 0x7fd3b80ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3bcc2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b81624b0 0x7fd3b80d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3bcc2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b815eda0 con 0x7fd3b815e9c0 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3bcc2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80d27f0 con 0x7fd3b80b68d0 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3bcc2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b8162c20 con 0x7fd3b81624b0 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b5d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd3b80b68d0 0x7fd3b80ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b5d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd3b80b68d0 0x7fd3b80ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45746/0 (socket says 172.21.15.106:45746) 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b5d9b640 1 -- 172.21.15.106:0/382322838 learned_addr learned my addr 172.21.15.106:0/382322838 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b6d9d640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b81624b0 0x7fd3b80d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b5d9b640 1 -- 172.21.15.106:0/382322838 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b81624b0 msgr2=0x7fd3b80d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.406+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b81624b0 0x7fd3b80d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b5d9b640 1 -- 172.21.15.106:0/382322838 >> v1:172.21.15.114:6789/0 conn(0x7fd3b815e9c0 legacy=0x7fd3b80b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b5d9b640 1 -- 172.21.15.106:0/382322838 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b8162f00 con 0x7fd3b80b68d0 2024-04-13T03:11:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b80ba290 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009870 tx=0x7fd3ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4b1f2f3eafd519 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b559a640 1 -- 172.21.15.106:0/382322838 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac034070 con 0x7fd3b80b68d0 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b559a640 1 -- 172.21.15.106:0/382322838 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac030840 con 0x7fd3b80b68d0 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.407+0000 7fd3b559a640 1 -- 172.21.15.106:0/382322838 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac030c10 con 0x7fd3b80b68d0 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/382322838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 msgr2=0x7fd3b80ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b80ba290 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009870 tx=0x7fd3ac02b980 comp rx=0 tx=0).stop 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/382322838 shutdown_connections 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b81624b0 0x7fd3b80d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/382322838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b80ba290 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/382322838 >> 172.21.15.106:0/382322838 conn(0x7fd3b80d11c0 msgr2=0x7fd3b80d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/382322838 shutdown_connections 2024-04-13T03:11:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.408+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/382322838 wait complete. 2024-04-13T03:11:20.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.409+0000 7fd3bcc2d640 1 Processor -- start 2024-04-13T03:11:20.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.409+0000 7fd3bcc2d640 1 -- start start 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 0x7fd3b815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 0x7fd3b80b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b8162940 con 0x7fd3b80b68d0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b80d2a40 con 0x7fd3b81624b0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3bcc2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b80d34a0 con 0x7fd3b815e9c0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45754/0 (socket says 172.21.15.106:45754) 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 -- 172.21.15.106:0/1360336990 learned_addr learned my addr 172.21.15.106:0/1360336990 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 0x7fd3b815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b6d9d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 0x7fd3b80b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 -- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 msgr2=0x7fd3b815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 0x7fd3b815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 -- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 msgr2=0x7fd3b80b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 0x7fd3b80b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b659c640 1 -- 172.21.15.106:0/1360336990 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80cdf30 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 0x7fd3b815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.410+0000 7fd3b6d9d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 0x7fd3b80b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3b659c640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c400 tx=0x7fd3a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80052d0 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a8005490 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8007260 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b80b6540 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.411+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b815f460 con 0x7fd3b80b68d0 2024-04-13T03:11:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.412+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b8068a10 con 0x7fd3b80b68d0 2024-04-13T03:11:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.412+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3a8007400 con 0x7fd3b80b68d0 2024-04-13T03:11:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.412+0000 7fd3a6ffd640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.413+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd3a808b240 con 0x7fd3b80b68d0 2024-04-13T03:11:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.413+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.413+0000 7fd3b5d9b640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac036000 tx=0x7fd3ac03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.417+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3a80594e0 con 0x7fd3b80b68d0 2024-04-13T03:11:20.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.588+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3b80b79e0 con 0x7fd39806c8f0 2024-04-13T03:11:20.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.594+0000 7fd3a6ffd640 1 -- 172.21.15.106:0/1360336990 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251884 (secure 0 0 0) 0x7fd3b80b79e0 con 0x7fd39806c8f0 2024-04-13T03:11:20.598 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:20.599 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:11:20.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 msgr2=0x7fd39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac036000 tx=0x7fd3ac03b040 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 msgr2=0x7fd3b8152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c400 tx=0x7fd3a800c8c0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 shutdown_connections 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd3b815e9c0 0x7fd3b815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3b81624b0 0x7fd3b80b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 --2- 172.21.15.106:0/1360336990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b80b68d0 0x7fd3b8152b50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 >> 172.21.15.106:0/1360336990 conn(0x7fd3b80d11c0 msgr2=0x7fd3b815cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 shutdown_connections 2024-04-13T03:11:20.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.597+0000 7fd3bcc2d640 1 -- 172.21.15.106:0/1360336990 wait complete. 2024-04-13T03:11:20.609 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-13T03:11:20.385983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222000,"kb_used_data":5828,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752720,"statfs":{"total":773094113280,"available":772866785280,"internally_reserved":0,"allocated":5967872,"data_stored":3293273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114221"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322024+0000","last_change":"2024-04-13T03:11:17.314443+0000","last_active":"2024-04-13T03:11:17.322024+0000","last_peered":"2024-04-13T03:11:17.322024+0000","last_clean":"2024-04-13T03:11:17.322024+0000","last_became_active":"2024-04-13T03:11:17.314184+0000","last_became_peered":"2024-04-13T03:11:17.314184+0000","last_unstale":"2024-04-13T03:11:17.322024+0000","last_undegraded":"2024-04-13T03:11:17.322024+0000","last_fullsized":"2024-04-13T03:11:17.322024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323859+0000","last_change":"2024-04-13T03:11:17.317238+0000","last_active":"2024-04-13T03:11:17.323859+0000","last_peered":"2024-04-13T03:11:17.323859+0000","last_clean":"2024-04-13T03:11:17.323859+0000","last_became_active":"2024-04-13T03:11:17.316740+0000","last_became_peered":"2024-04-13T03:11:17.316740+0000","last_unstale":"2024-04-13T03:11:17.323859+0000","last_undegraded":"2024-04-13T03:11:17.323859+0000","last_fullsized":"2024-04-13T03:11:17.323859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:33.423529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310254+0000","last_change":"2024-04-13T03:11:17.302928+0000","last_active":"2024-04-13T03:11:17.310254+0000","last_peered":"2024-04-13T03:11:17.310254+0000","last_clean":"2024-04-13T03:11:17.310254+0000","last_became_active":"2024-04-13T03:11:17.302637+0000","last_became_peered":"2024-04-13T03:11:17.302637+0000","last_unstale":"2024-04-13T03:11:17.310254+0000","last_undegraded":"2024-04-13T03:11:17.310254+0000","last_fullsized":"2024-04-13T03:11:17.310254+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:05.108643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323934+0000","last_change":"2024-04-13T03:11:17.318401+0000","last_active":"2024-04-13T03:11:17.323934+0000","last_peered":"2024-04-13T03:11:17.323934+0000","last_clean":"2024-04-13T03:11:17.323934+0000","last_became_active":"2024-04-13T03:11:17.318064+0000","last_became_peered":"2024-04-13T03:11:17.318064+0000","last_unstale":"2024-04-13T03:11:17.323934+0000","last_undegraded":"2024-04-13T03:11:17.323934+0000","last_fullsized":"2024-04-13T03:11:17.323934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.314479+0000","last_change":"2024-04-13T03:11:17.309144+0000","last_active":"2024-04-13T03:11:17.314479+0000","last_peered":"2024-04-13T03:11:17.314479+0000","last_clean":"2024-04-13T03:11:17.314479+0000","last_became_active":"2024-04-13T03:11:17.308876+0000","last_became_peered":"2024-04-13T03:11:17.308876+0000","last_unstale":"2024-04-13T03:11:17.314479+0000","last_undegraded":"2024-04-13T03:11:17.314479+0000","last_fullsized":"2024-04-13T03:11:17.314479+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311261+0000","last_change":"2024-04-13T03:11:17.308473+0000","last_active":"2024-04-13T03:11:17.311261+0000","last_peered":"2024-04-13T03:11:17.311261+0000","last_clean":"2024-04-13T03:11:17.311261+0000","last_became_active":"2024-04-13T03:11:17.308221+0000","last_became_peered":"2024-04-13T03:11:17.308221+0000","last_unstale":"2024-04-13T03:11:17.311261+0000","last_undegraded":"2024-04-13T03:11:17.311261+0000","last_fullsized":"2024-04-13T03:11:17.311261+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306843+0000","last_change":"2024-04-13T03:11:17.299596+0000","last_active":"2024-04-13T03:11:17.306843+0000","last_peered":"2024-04-13T03:11:17.306843+0000","last_clean":"2024-04-13T03:11:17.306843+0000","last_became_active":"2024-04-13T03:11:17.299354+0000","last_became_peered":"2024-04-13T03:11:17.299354+0000","last_unstale":"2024-04-13T03:11:17.306843+0000","last_undegraded":"2024-04-13T03:11:17.306843+0000","last_fullsized":"2024-04-13T03:11:17.306843+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322007+0000","last_change":"2024-04-13T03:11:17.311835+0000","last_active":"2024-04-13T03:11:17.322007+0000","last_peered":"2024-04-13T03:11:17.322007+0000","last_clean":"2024-04-13T03:11:17.322007+0000","last_became_active":"2024-04-13T03:11:17.311218+0000","last_became_peered":"2024-04-13T03:11:17.311218+0000","last_unstale":"2024-04-13T03:11:17.322007+0000","last_undegraded":"2024-04-13T03:11:17.322007+0000","last_fullsized":"2024-04-13T03:11:17.322007+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:32:31.447158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321999+0000","last_change":"2024-04-13T03:11:17.312212+0000","last_active":"2024-04-13T03:11:17.321999+0000","last_peered":"2024-04-13T03:11:17.321999+0000","last_clean":"2024-04-13T03:11:17.321999+0000","last_became_active":"2024-04-13T03:11:17.311958+0000","last_became_peered":"2024-04-13T03:11:17.311958+0000","last_unstale":"2024-04-13T03:11:17.321999+0000","last_undegraded":"2024-04-13T03:11:17.321999+0000","last_fullsized":"2024-04-13T03:11:17.321999+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:24.111294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311003+0000","last_change":"2024-04-13T03:11:17.307907+0000","last_active":"2024-04-13T03:11:17.311003+0000","last_peered":"2024-04-13T03:11:17.311003+0000","last_clean":"2024-04-13T03:11:17.311003+0000","last_became_active":"2024-04-13T03:11:17.307627+0000","last_became_peered":"2024-04-13T03:11:17.307627+0000","last_unstale":"2024-04-13T03:11:17.311003+0000","last_undegraded":"2024-04-13T03:11:17.311003+0000","last_fullsized":"2024-04-13T03:11:17.311003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:47.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313819+0000","last_change":"2024-04-13T03:11:17.310699+0000","last_active":"2024-04-13T03:11:17.313819+0000","last_peered":"2024-04-13T03:11:17.313819+0000","last_clean":"2024-04-13T03:11:17.313819+0000","last_became_active":"2024-04-13T03:11:17.310448+0000","last_became_peered":"2024-04-13T03:11:17.310448+0000","last_unstale":"2024-04-13T03:11:17.313819+0000","last_undegraded":"2024-04-13T03:11:17.313819+0000","last_fullsized":"2024-04-13T03:11:17.313819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:11.898938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322176+0000","last_change":"2024-04-13T03:11:17.306927+0000","last_active":"2024-04-13T03:11:17.322176+0000","last_peered":"2024-04-13T03:11:17.322176+0000","last_clean":"2024-04-13T03:11:17.322176+0000","last_became_active":"2024-04-13T03:11:17.306668+0000","last_became_peered":"2024-04-13T03:11:17.306668+0000","last_unstale":"2024-04-13T03:11:17.322176+0000","last_undegraded":"2024-04-13T03:11:17.322176+0000","last_fullsized":"2024-04-13T03:11:17.322176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:16.885740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310913+0000","last_change":"2024-04-13T03:11:17.306229+0000","last_active":"2024-04-13T03:11:17.310913+0000","last_peered":"2024-04-13T03:11:17.310913+0000","last_clean":"2024-04-13T03:11:17.310913+0000","last_became_active":"2024-04-13T03:11:17.305971+0000","last_became_peered":"2024-04-13T03:11:17.305971+0000","last_unstale":"2024-04-13T03:11:17.310913+0000","last_undegraded":"2024-04-13T03:11:17.310913+0000","last_fullsized":"2024-04-13T03:11:17.310913+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:22.734271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309718+0000","last_change":"2024-04-13T03:11:17.303930+0000","last_active":"2024-04-13T03:11:17.309718+0000","last_peered":"2024-04-13T03:11:17.309718+0000","last_clean":"2024-04-13T03:11:17.309718+0000","last_became_active":"2024-04-13T03:11:17.303658+0000","last_became_peered":"2024-04-13T03:11:17.303658+0000","last_unstale":"2024-04-13T03:11:17.309718+0000","last_undegraded":"2024-04-13T03:11:17.309718+0000","last_fullsized":"2024-04-13T03:11:17.309718+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:09:18.552034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316279+0000","last_change":"2024-04-13T03:11:17.313250+0000","last_active":"2024-04-13T03:11:17.316279+0000","last_peered":"2024-04-13T03:11:17.316279+0000","last_clean":"2024-04-13T03:11:17.316279+0000","last_became_active":"2024-04-13T03:11:17.312996+0000","last_became_peered":"2024-04-13T03:11:17.312996+0000","last_unstale":"2024-04-13T03:11:17.316279+0000","last_undegraded":"2024-04-13T03:11:17.316279+0000","last_fullsized":"2024-04-13T03:11:17.316279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:34.277280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.305631+0000","last_change":"2024-04-13T03:11:17.302943+0000","last_active":"2024-04-13T03:11:17.305631+0000","last_peered":"2024-04-13T03:11:17.305631+0000","last_clean":"2024-04-13T03:11:17.305631+0000","last_became_active":"2024-04-13T03:11:17.302623+0000","last_became_peered":"2024-04-13T03:11:17.302623+0000","last_unstale":"2024-04-13T03:11:17.305631+0000","last_undegraded":"2024-04-13T03:11:17.305631+0000","last_fullsized":"2024-04-13T03:11:17.305631+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:10.323440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315512+0000","last_change":"2024-04-13T03:11:17.308404+0000","last_active":"2024-04-13T03:11:17.315512+0000","last_peered":"2024-04-13T03:11:17.315512+0000","last_clean":"2024-04-13T03:11:17.315512+0000","last_became_active":"2024-04-13T03:11:17.307783+0000","last_became_peered":"2024-04-13T03:11:17.307783+0000","last_unstale":"2024-04-13T03:11:17.315512+0000","last_undegraded":"2024-04-13T03:11:17.315512+0000","last_fullsized":"2024-04-13T03:11:17.315512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:47:22.417604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323895+0000","last_change":"2024-04-13T03:11:17.317241+0000","last_active":"2024-04-13T03:11:17.323895+0000","last_peered":"2024-04-13T03:11:17.323895+0000","last_clean":"2024-04-13T03:11:17.323895+0000","last_became_active":"2024-04-13T03:11:17.316966+0000","last_became_peered":"2024-04-13T03:11:17.316966+0000","last_unstale":"2024-04-13T03:11:17.323895+0000","last_undegraded":"2024-04-13T03:11:17.323895+0000","last_fullsized":"2024-04-13T03:11:17.323895+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306847+0000","last_change":"2024-04-13T03:11:17.298579+0000","last_active":"2024-04-13T03:11:17.306847+0000","last_peered":"2024-04-13T03:11:17.306847+0000","last_clean":"2024-04-13T03:11:17.306847+0000","last_became_active":"2024-04-13T03:11:17.298289+0000","last_became_peered":"2024-04-13T03:11:17.298289+0000","last_unstale":"2024-04-13T03:11:17.306847+0000","last_undegraded":"2024-04-13T03:11:17.306847+0000","last_fullsized":"2024-04-13T03:11:17.306847+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310546+0000","last_change":"2024-04-13T03:11:17.303977+0000","last_active":"2024-04-13T03:11:17.310546+0000","last_peered":"2024-04-13T03:11:17.310546+0000","last_clean":"2024-04-13T03:11:17.310546+0000","last_became_active":"2024-04-13T03:11:17.303716+0000","last_became_peered":"2024-04-13T03:11:17.303716+0000","last_unstale":"2024-04-13T03:11:17.310546+0000","last_undegraded":"2024-04-13T03:11:17.310546+0000","last_fullsized":"2024-04-13T03:11:17.310546+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:27.322536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315715+0000","last_change":"2024-04-13T03:11:17.311333+0000","last_active":"2024-04-13T03:11:17.315715+0000","last_peered":"2024-04-13T03:11:17.315715+0000","last_clean":"2024-04-13T03:11:17.315715+0000","last_became_active":"2024-04-13T03:11:17.311041+0000","last_became_peered":"2024-04-13T03:11:17.311041+0000","last_unstale":"2024-04-13T03:11:17.315715+0000","last_undegraded":"2024-04-13T03:11:17.315715+0000","last_fullsized":"2024-04-13T03:11:17.315715+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:10:16.686388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322714+0000","last_change":"2024-04-13T03:11:17.311077+0000","last_active":"2024-04-13T03:11:17.322714+0000","last_peered":"2024-04-13T03:11:17.322714+0000","last_clean":"2024-04-13T03:11:17.322714+0000","last_became_active":"2024-04-13T03:11:17.310774+0000","last_became_peered":"2024-04-13T03:11:17.310774+0000","last_unstale":"2024-04-13T03:11:17.322714+0000","last_undegraded":"2024-04-13T03:11:17.322714+0000","last_fullsized":"2024-04-13T03:11:17.322714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:17.647839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313833+0000","last_change":"2024-04-13T03:11:17.309828+0000","last_active":"2024-04-13T03:11:17.313833+0000","last_peered":"2024-04-13T03:11:17.313833+0000","last_clean":"2024-04-13T03:11:17.313833+0000","last_became_active":"2024-04-13T03:11:17.309510+0000","last_became_peered":"2024-04-13T03:11:17.309510+0000","last_unstale":"2024-04-13T03:11:17.313833+0000","last_undegraded":"2024-04-13T03:11:17.313833+0000","last_fullsized":"2024-04-13T03:11:17.313833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:44:45.002908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309711+0000","last_change":"2024-04-13T03:11:17.302009+0000","last_active":"2024-04-13T03:11:17.309711+0000","last_peered":"2024-04-13T03:11:17.309711+0000","last_clean":"2024-04-13T03:11:17.309711+0000","last_became_active":"2024-04-13T03:11:17.300543+0000","last_became_peered":"2024-04-13T03:11:17.300543+0000","last_unstale":"2024-04-13T03:11:17.309711+0000","last_undegraded":"2024-04-13T03:11:17.309711+0000","last_fullsized":"2024-04-13T03:11:17.309711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:05:40.358273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315491+0000","last_change":"2024-04-13T03:11:17.307345+0000","last_active":"2024-04-13T03:11:17.315491+0000","last_peered":"2024-04-13T03:11:17.315491+0000","last_clean":"2024-04-13T03:11:17.315491+0000","last_became_active":"2024-04-13T03:11:17.307092+0000","last_became_peered":"2024-04-13T03:11:17.307092+0000","last_unstale":"2024-04-13T03:11:17.315491+0000","last_undegraded":"2024-04-13T03:11:17.315491+0000","last_fullsized":"2024-04-13T03:11:17.315491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:42.115735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321875+0000","last_change":"2024-04-13T03:11:17.308657+0000","last_active":"2024-04-13T03:11:17.321875+0000","last_peered":"2024-04-13T03:11:17.321875+0000","last_clean":"2024-04-13T03:11:17.321875+0000","last_became_active":"2024-04-13T03:11:17.308328+0000","last_became_peered":"2024-04-13T03:11:17.308328+0000","last_unstale":"2024-04-13T03:11:17.321875+0000","last_undegraded":"2024-04-13T03:11:17.321875+0000","last_fullsized":"2024-04-13T03:11:17.321875+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:52:51.855563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.308229+0000","last_change":"2024-04-13T03:11:17.304159+0000","last_active":"2024-04-13T03:11:17.308229+0000","last_peered":"2024-04-13T03:11:17.308229+0000","last_clean":"2024-04-13T03:11:17.308229+0000","last_became_active":"2024-04-13T03:11:17.303904+0000","last_became_peered":"2024-04-13T03:11:17.303904+0000","last_unstale":"2024-04-13T03:11:17.308229+0000","last_undegraded":"2024-04-13T03:11:17.308229+0000","last_fullsized":"2024-04-13T03:11:17.308229+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:30.832276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310569+0000","last_change":"2024-04-13T03:11:17.307114+0000","last_active":"2024-04-13T03:11:17.310569+0000","last_peered":"2024-04-13T03:11:17.310569+0000","last_clean":"2024-04-13T03:11:17.310569+0000","last_became_active":"2024-04-13T03:11:17.306828+0000","last_became_peered":"2024-04-13T03:11:17.306828+0000","last_unstale":"2024-04-13T03:11:17.310569+0000","last_undegraded":"2024-04-13T03:11:17.310569+0000","last_fullsized":"2024-04-13T03:11:17.310569+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:36:09.772778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.310223+0000","last_change":"2024-04-13T03:11:17.303616+0000","last_active":"2024-04-13T03:11:17.310223+0000","last_peered":"2024-04-13T03:11:17.310223+0000","last_clean":"2024-04-13T03:11:17.310223+0000","last_became_active":"2024-04-13T03:11:17.303358+0000","last_became_peered":"2024-04-13T03:11:17.303358+0000","last_unstale":"2024-04-13T03:11:17.310223+0000","last_undegraded":"2024-04-13T03:11:17.310223+0000","last_fullsized":"2024-04-13T03:11:17.310223+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:16:43.165248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316285+0000","last_change":"2024-04-13T03:11:17.311260+0000","last_active":"2024-04-13T03:11:17.316285+0000","last_peered":"2024-04-13T03:11:17.316285+0000","last_clean":"2024-04-13T03:11:17.316285+0000","last_became_active":"2024-04-13T03:11:17.311015+0000","last_became_peered":"2024-04-13T03:11:17.311015+0000","last_unstale":"2024-04-13T03:11:17.316285+0000","last_undegraded":"2024-04-13T03:11:17.316285+0000","last_fullsized":"2024-04-13T03:11:17.316285+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:29.442707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307056+0000","last_change":"2024-04-13T03:11:17.299746+0000","last_active":"2024-04-13T03:11:17.307056+0000","last_peered":"2024-04-13T03:11:17.307056+0000","last_clean":"2024-04-13T03:11:17.307056+0000","last_became_active":"2024-04-13T03:11:17.299120+0000","last_became_peered":"2024-04-13T03:11:17.299120+0000","last_unstale":"2024-04-13T03:11:17.307056+0000","last_undegraded":"2024-04-13T03:11:17.307056+0000","last_fullsized":"2024-04-13T03:11:17.307056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316497+0000","last_change":"2024-04-13T03:11:17.306889+0000","last_active":"2024-04-13T03:11:17.316497+0000","last_peered":"2024-04-13T03:11:17.316497+0000","last_clean":"2024-04-13T03:11:17.316497+0000","last_became_active":"2024-04-13T03:11:17.306456+0000","last_became_peered":"2024-04-13T03:11:17.306456+0000","last_unstale":"2024-04-13T03:11:17.316497+0000","last_undegraded":"2024-04-13T03:11:17.316497+0000","last_fullsized":"2024-04-13T03:11:17.316497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:33.423529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323890+0000","last_change":"2024-04-13T03:11:17.317468+0000","last_active":"2024-04-13T03:11:17.323890+0000","last_peered":"2024-04-13T03:11:17.323890+0000","last_clean":"2024-04-13T03:11:17.323890+0000","last_became_active":"2024-04-13T03:11:17.317158+0000","last_became_peered":"2024-04-13T03:11:17.317158+0000","last_unstale":"2024-04-13T03:11:17.323890+0000","last_undegraded":"2024-04-13T03:11:17.323890+0000","last_fullsized":"2024-04-13T03:11:17.323890+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318758+0000","last_change":"2024-04-13T03:11:16.288090+0000","last_active":"2024-04-13T03:11:17.318758+0000","last_peered":"2024-04-13T03:11:17.318758+0000","last_clean":"2024-04-13T03:11:17.318758+0000","last_became_active":"2024-04-13T03:11:16.287829+0000","last_became_peered":"2024-04-13T03:11:16.287829+0000","last_unstale":"2024-04-13T03:11:17.318758+0000","last_undegraded":"2024-04-13T03:11:17.318758+0000","last_fullsized":"2024-04-13T03:11:17.318758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:14:53.458260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316408+0000","last_change":"2024-04-13T03:11:17.310642+0000","last_active":"2024-04-13T03:11:17.316408+0000","last_peered":"2024-04-13T03:11:17.316408+0000","last_clean":"2024-04-13T03:11:17.316408+0000","last_became_active":"2024-04-13T03:11:17.310332+0000","last_became_peered":"2024-04-13T03:11:17.310332+0000","last_unstale":"2024-04-13T03:11:17.316408+0000","last_undegraded":"2024-04-13T03:11:17.316408+0000","last_fullsized":"2024-04-13T03:11:17.316408+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:14.281192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322909+0000","last_change":"2024-04-13T03:11:16.290002+0000","last_active":"2024-04-13T03:11:17.322909+0000","last_peered":"2024-04-13T03:11:17.322909+0000","last_clean":"2024-04-13T03:11:17.322909+0000","last_became_active":"2024-04-13T03:11:16.289716+0000","last_became_peered":"2024-04-13T03:11:16.289716+0000","last_unstale":"2024-04-13T03:11:17.322909+0000","last_undegraded":"2024-04-13T03:11:17.322909+0000","last_fullsized":"2024-04-13T03:11:17.322909+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:10:15.680051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311020+0000","last_change":"2024-04-13T03:11:17.308034+0000","last_active":"2024-04-13T03:11:17.311020+0000","last_peered":"2024-04-13T03:11:17.311020+0000","last_clean":"2024-04-13T03:11:17.311020+0000","last_became_active":"2024-04-13T03:11:17.307741+0000","last_became_peered":"2024-04-13T03:11:17.307741+0000","last_unstale":"2024-04-13T03:11:17.311020+0000","last_undegraded":"2024-04-13T03:11:17.311020+0000","last_fullsized":"2024-04-13T03:11:17.311020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:57:34.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322017+0000","last_change":"2024-04-13T03:11:16.291777+0000","last_active":"2024-04-13T03:11:17.322017+0000","last_peered":"2024-04-13T03:11:17.322017+0000","last_clean":"2024-04-13T03:11:17.322017+0000","last_became_active":"2024-04-13T03:11:16.290999+0000","last_became_peered":"2024-04-13T03:11:16.290999+0000","last_unstale":"2024-04-13T03:11:17.322017+0000","last_undegraded":"2024-04-13T03:11:17.322017+0000","last_fullsized":"2024-04-13T03:11:17.322017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:24.983066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317523+0000","last_change":"2024-04-13T03:11:17.305054+0000","last_active":"2024-04-13T03:11:17.317523+0000","last_peered":"2024-04-13T03:11:17.317523+0000","last_clean":"2024-04-13T03:11:17.317523+0000","last_became_active":"2024-04-13T03:11:17.304628+0000","last_became_peered":"2024-04-13T03:11:17.304628+0000","last_unstale":"2024-04-13T03:11:17.317523+0000","last_undegraded":"2024-04-13T03:11:17.317523+0000","last_fullsized":"2024-04-13T03:11:17.317523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:35.617657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322811+0000","last_change":"2024-04-13T03:11:16.286797+0000","last_active":"2024-04-13T03:11:17.322811+0000","last_peered":"2024-04-13T03:11:17.322811+0000","last_clean":"2024-04-13T03:11:17.322811+0000","last_became_active":"2024-04-13T03:11:16.285784+0000","last_became_peered":"2024-04-13T03:11:16.285784+0000","last_unstale":"2024-04-13T03:11:17.322811+0000","last_undegraded":"2024-04-13T03:11:17.322811+0000","last_fullsized":"2024-04-13T03:11:17.322811+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:28.436370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.308263+0000","last_change":"2024-04-13T03:11:17.303820+0000","last_active":"2024-04-13T03:11:17.308263+0000","last_peered":"2024-04-13T03:11:17.308263+0000","last_clean":"2024-04-13T03:11:17.308263+0000","last_became_active":"2024-04-13T03:11:17.303574+0000","last_became_peered":"2024-04-13T03:11:17.303574+0000","last_unstale":"2024-04-13T03:11:17.308263+0000","last_undegraded":"2024-04-13T03:11:17.308263+0000","last_fullsized":"2024-04-13T03:11:17.308263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:25.706262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322422+0000","last_change":"2024-04-13T03:11:16.287747+0000","last_active":"2024-04-13T03:11:17.322422+0000","last_peered":"2024-04-13T03:11:17.322422+0000","last_clean":"2024-04-13T03:11:17.322422+0000","last_became_active":"2024-04-13T03:11:16.287189+0000","last_became_peered":"2024-04-13T03:11:16.287189+0000","last_unstale":"2024-04-13T03:11:17.322422+0000","last_undegraded":"2024-04-13T03:11:17.322422+0000","last_fullsized":"2024-04-13T03:11:17.322422+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316260+0000","last_change":"2024-04-13T03:11:17.311897+0000","last_active":"2024-04-13T03:11:17.316260+0000","last_peered":"2024-04-13T03:11:17.316260+0000","last_clean":"2024-04-13T03:11:17.316260+0000","last_became_active":"2024-04-13T03:11:17.311654+0000","last_became_peered":"2024-04-13T03:11:17.311654+0000","last_unstale":"2024-04-13T03:11:17.316260+0000","last_undegraded":"2024-04-13T03:11:17.316260+0000","last_fullsized":"2024-04-13T03:11:17.316260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315956+0000","last_change":"2024-04-13T03:11:16.289174+0000","last_active":"2024-04-13T03:11:17.315956+0000","last_peered":"2024-04-13T03:11:17.315956+0000","last_clean":"2024-04-13T03:11:17.315956+0000","last_became_active":"2024-04-13T03:11:16.288844+0000","last_became_peered":"2024-04-13T03:11:16.288844+0000","last_unstale":"2024-04-13T03:11:17.315956+0000","last_undegraded":"2024-04-13T03:11:17.315956+0000","last_fullsized":"2024-04-13T03:11:17.315956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:08:21.895972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317585+0000","last_change":"2024-04-13T03:11:17.306927+0000","last_active":"2024-04-13T03:11:17.317585+0000","last_peered":"2024-04-13T03:11:17.317585+0000","last_clean":"2024-04-13T03:11:17.317585+0000","last_became_active":"2024-04-13T03:11:17.306677+0000","last_became_peered":"2024-04-13T03:11:17.306677+0000","last_unstale":"2024-04-13T03:11:17.317585+0000","last_undegraded":"2024-04-13T03:11:17.317585+0000","last_fullsized":"2024-04-13T03:11:17.317585+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:22.833136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.297107+0000","last_change":"2024-04-13T03:11:16.292442+0000","last_active":"2024-04-13T03:11:17.297107+0000","last_peered":"2024-04-13T03:11:17.297107+0000","last_clean":"2024-04-13T03:11:17.297107+0000","last_became_active":"2024-04-13T03:11:16.292176+0000","last_became_peered":"2024-04-13T03:11:16.292176+0000","last_unstale":"2024-04-13T03:11:17.297107+0000","last_undegraded":"2024-04-13T03:11:17.297107+0000","last_fullsized":"2024-04-13T03:11:17.297107+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:12:59.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316241+0000","last_change":"2024-04-13T03:11:17.311970+0000","last_active":"2024-04-13T03:11:17.316241+0000","last_peered":"2024-04-13T03:11:17.316241+0000","last_clean":"2024-04-13T03:11:17.316241+0000","last_became_active":"2024-04-13T03:11:17.311722+0000","last_became_peered":"2024-04-13T03:11:17.311722+0000","last_unstale":"2024-04-13T03:11:17.316241+0000","last_undegraded":"2024-04-13T03:11:17.316241+0000","last_fullsized":"2024-04-13T03:11:17.316241+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.290766+0000","last_change":"2024-04-13T03:11:16.286594+0000","last_active":"2024-04-13T03:11:17.290766+0000","last_peered":"2024-04-13T03:11:17.290766+0000","last_clean":"2024-04-13T03:11:17.290766+0000","last_became_active":"2024-04-13T03:11:16.286337+0000","last_became_peered":"2024-04-13T03:11:16.286337+0000","last_unstale":"2024-04-13T03:11:17.290766+0000","last_undegraded":"2024-04-13T03:11:17.290766+0000","last_fullsized":"2024-04-13T03:11:17.290766+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:24.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.314227+0000","last_change":"2024-04-13T03:11:17.306492+0000","last_active":"2024-04-13T03:11:17.314227+0000","last_peered":"2024-04-13T03:11:17.314227+0000","last_clean":"2024-04-13T03:11:17.314227+0000","last_became_active":"2024-04-13T03:11:17.305879+0000","last_became_peered":"2024-04-13T03:11:17.305879+0000","last_unstale":"2024-04-13T03:11:17.314227+0000","last_undegraded":"2024-04-13T03:11:17.314227+0000","last_fullsized":"2024-04-13T03:11:17.314227+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:47.397410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.303307+0000","last_change":"2024-04-13T03:11:16.291313+0000","last_active":"2024-04-13T03:11:17.303307+0000","last_peered":"2024-04-13T03:11:17.303307+0000","last_clean":"2024-04-13T03:11:17.303307+0000","last_became_active":"2024-04-13T03:11:16.291043+0000","last_became_peered":"2024-04-13T03:11:16.291043+0000","last_unstale":"2024-04-13T03:11:17.303307+0000","last_undegraded":"2024-04-13T03:11:17.303307+0000","last_fullsized":"2024-04-13T03:11:17.303307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:29.825939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313929+0000","last_change":"2024-04-13T03:11:17.302490+0000","last_active":"2024-04-13T03:11:17.313929+0000","last_peered":"2024-04-13T03:11:17.313929+0000","last_clean":"2024-04-13T03:11:17.313929+0000","last_became_active":"2024-04-13T03:11:17.302219+0000","last_became_peered":"2024-04-13T03:11:17.302219+0000","last_unstale":"2024-04-13T03:11:17.313929+0000","last_undegraded":"2024-04-13T03:11:17.313929+0000","last_fullsized":"2024-04-13T03:11:17.313929+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293028+0000","last_change":"2024-04-13T03:11:16.293423+0000","last_active":"2024-04-13T03:11:17.293028+0000","last_peered":"2024-04-13T03:11:17.293028+0000","last_clean":"2024-04-13T03:11:17.293028+0000","last_became_active":"2024-04-13T03:11:16.293103+0000","last_became_peered":"2024-04-13T03:11:16.293103+0000","last_unstale":"2024-04-13T03:11:17.293028+0000","last_undegraded":"2024-04-13T03:11:17.293028+0000","last_fullsized":"2024-04-13T03:11:17.293028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:44.531785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309741+0000","last_change":"2024-04-13T03:11:17.303622+0000","last_active":"2024-04-13T03:11:17.309741+0000","last_peered":"2024-04-13T03:11:17.309741+0000","last_clean":"2024-04-13T03:11:17.309741+0000","last_became_active":"2024-04-13T03:11:17.303375+0000","last_became_peered":"2024-04-13T03:11:17.303375+0000","last_unstale":"2024-04-13T03:11:17.309741+0000","last_undegraded":"2024-04-13T03:11:17.309741+0000","last_fullsized":"2024-04-13T03:11:17.309741+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:24:11.365885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317076+0000","last_change":"2024-04-13T03:11:16.296242+0000","last_active":"2024-04-13T03:11:17.317076+0000","last_peered":"2024-04-13T03:11:17.317076+0000","last_clean":"2024-04-13T03:11:17.317076+0000","last_became_active":"2024-04-13T03:11:16.295965+0000","last_became_peered":"2024-04-13T03:11:16.295965+0000","last_unstale":"2024-04-13T03:11:17.317076+0000","last_undegraded":"2024-04-13T03:11:17.317076+0000","last_fullsized":"2024-04-13T03:11:17.317076+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:24.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304281+0000","last_change":"2024-04-13T03:11:17.301654+0000","last_active":"2024-04-13T03:11:17.304281+0000","last_peered":"2024-04-13T03:11:17.304281+0000","last_clean":"2024-04-13T03:11:17.304281+0000","last_became_active":"2024-04-13T03:11:17.301379+0000","last_became_peered":"2024-04-13T03:11:17.301379+0000","last_unstale":"2024-04-13T03:11:17.304281+0000","last_undegraded":"2024-04-13T03:11:17.304281+0000","last_fullsized":"2024-04-13T03:11:17.304281+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:22.734271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.326537+0000","last_change":"2024-04-13T03:11:16.291833+0000","last_active":"2024-04-13T03:11:17.326537+0000","last_peered":"2024-04-13T03:11:17.326537+0000","last_clean":"2024-04-13T03:11:17.326537+0000","last_became_active":"2024-04-13T03:11:16.291553+0000","last_became_peered":"2024-04-13T03:11:16.291553+0000","last_unstale":"2024-04-13T03:11:17.326537+0000","last_undegraded":"2024-04-13T03:11:17.326537+0000","last_fullsized":"2024-04-13T03:11:17.326537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:47:21.411267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316424+0000","last_change":"2024-04-13T03:11:17.310877+0000","last_active":"2024-04-13T03:11:17.316424+0000","last_peered":"2024-04-13T03:11:17.316424+0000","last_clean":"2024-04-13T03:11:17.316424+0000","last_became_active":"2024-04-13T03:11:17.309995+0000","last_became_peered":"2024-04-13T03:11:17.309995+0000","last_unstale":"2024-04-13T03:11:17.316424+0000","last_undegraded":"2024-04-13T03:11:17.316424+0000","last_fullsized":"2024-04-13T03:11:17.316424+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:45:34.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.324104+0000","last_change":"2024-04-13T03:11:16.287875+0000","last_active":"2024-04-13T03:11:17.324104+0000","last_peered":"2024-04-13T03:11:17.324104+0000","last_clean":"2024-04-13T03:11:17.324104+0000","last_became_active":"2024-04-13T03:11:16.287485+0000","last_became_peered":"2024-04-13T03:11:16.287485+0000","last_unstale":"2024-04-13T03:11:17.324104+0000","last_undegraded":"2024-04-13T03:11:17.324104+0000","last_fullsized":"2024-04-13T03:11:17.324104+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:28.436370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.306907+0000","last_change":"2024-04-13T03:11:17.299767+0000","last_active":"2024-04-13T03:11:17.306907+0000","last_peered":"2024-04-13T03:11:17.306907+0000","last_clean":"2024-04-13T03:11:17.306907+0000","last_became_active":"2024-04-13T03:11:17.299514+0000","last_became_peered":"2024-04-13T03:11:17.299514+0000","last_unstale":"2024-04-13T03:11:17.306907+0000","last_undegraded":"2024-04-13T03:11:17.306907+0000","last_fullsized":"2024-04-13T03:11:17.306907+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:17:21.128440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.297750+0000","last_change":"2024-04-13T03:11:16.293489+0000","last_active":"2024-04-13T03:11:17.297750+0000","last_peered":"2024-04-13T03:11:17.297750+0000","last_clean":"2024-04-13T03:11:17.297750+0000","last_became_active":"2024-04-13T03:11:16.293189+0000","last_became_peered":"2024-04-13T03:11:16.293189+0000","last_unstale":"2024-04-13T03:11:17.297750+0000","last_undegraded":"2024-04-13T03:11:17.297750+0000","last_fullsized":"2024-04-13T03:11:17.297750+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:34:00.330011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317604+0000","last_change":"2024-04-13T03:11:17.307823+0000","last_active":"2024-04-13T03:11:17.317604+0000","last_peered":"2024-04-13T03:11:17.317604+0000","last_clean":"2024-04-13T03:11:17.317604+0000","last_became_active":"2024-04-13T03:11:17.307575+0000","last_became_peered":"2024-04-13T03:11:17.307575+0000","last_unstale":"2024-04-13T03:11:17.317604+0000","last_undegraded":"2024-04-13T03:11:17.317604+0000","last_fullsized":"2024-04-13T03:11:17.317604+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:52:48.380574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304098+0000","last_change":"2024-04-13T03:11:16.290805+0000","last_active":"2024-04-13T03:11:17.304098+0000","last_peered":"2024-04-13T03:11:17.304098+0000","last_clean":"2024-04-13T03:11:17.304098+0000","last_became_active":"2024-04-13T03:11:16.290500+0000","last_became_peered":"2024-04-13T03:11:16.290500+0000","last_unstale":"2024-04-13T03:11:17.304098+0000","last_undegraded":"2024-04-13T03:11:17.304098+0000","last_fullsized":"2024-04-13T03:11:17.304098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:29:21.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317225+0000","last_change":"2024-04-13T03:11:17.310475+0000","last_active":"2024-04-13T03:11:17.317225+0000","last_peered":"2024-04-13T03:11:17.317225+0000","last_clean":"2024-04-13T03:11:17.317225+0000","last_became_active":"2024-04-13T03:11:17.310004+0000","last_became_peered":"2024-04-13T03:11:17.310004+0000","last_unstale":"2024-04-13T03:11:17.317225+0000","last_undegraded":"2024-04-13T03:11:17.317225+0000","last_fullsized":"2024-04-13T03:11:17.317225+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:35.617657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.324086+0000","last_change":"2024-04-13T03:11:16.287295+0000","last_active":"2024-04-13T03:11:17.324086+0000","last_peered":"2024-04-13T03:11:17.324086+0000","last_clean":"2024-04-13T03:11:17.324086+0000","last_became_active":"2024-04-13T03:11:16.287024+0000","last_became_peered":"2024-04-13T03:11:16.287024+0000","last_unstale":"2024-04-13T03:11:17.324086+0000","last_undegraded":"2024-04-13T03:11:17.324086+0000","last_fullsized":"2024-04-13T03:11:17.324086+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:05:13.291961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.323643+0000","last_change":"2024-04-13T03:11:17.317185+0000","last_active":"2024-04-13T03:11:17.323643+0000","last_peered":"2024-04-13T03:11:17.323643+0000","last_clean":"2024-04-13T03:11:17.323643+0000","last_became_active":"2024-04-13T03:11:17.314734+0000","last_became_peered":"2024-04-13T03:11:17.314734+0000","last_unstale":"2024-04-13T03:11:17.323643+0000","last_undegraded":"2024-04-13T03:11:17.323643+0000","last_fullsized":"2024-04-13T03:11:17.323643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:07.922441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322922+0000","last_change":"2024-04-13T03:11:16.287839+0000","last_active":"2024-04-13T03:11:17.322922+0000","last_peered":"2024-04-13T03:11:17.322922+0000","last_clean":"2024-04-13T03:11:17.322922+0000","last_became_active":"2024-04-13T03:11:16.287582+0000","last_became_peered":"2024-04-13T03:11:16.287582+0000","last_unstale":"2024-04-13T03:11:17.322922+0000","last_undegraded":"2024-04-13T03:11:17.322922+0000","last_fullsized":"2024-04-13T03:11:17.322922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:32.417192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309481+0000","last_change":"2024-04-13T03:11:17.303324+0000","last_active":"2024-04-13T03:11:17.309481+0000","last_peered":"2024-04-13T03:11:17.309481+0000","last_clean":"2024-04-13T03:11:17.309481+0000","last_became_active":"2024-04-13T03:11:17.303015+0000","last_became_peered":"2024-04-13T03:11:17.303015+0000","last_unstale":"2024-04-13T03:11:17.309481+0000","last_undegraded":"2024-04-13T03:11:17.309481+0000","last_fullsized":"2024-04-13T03:11:17.309481+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:08:22.902309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317036+0000","last_change":"2024-04-13T03:11:16.291129+0000","last_active":"2024-04-13T03:11:17.317036+0000","last_peered":"2024-04-13T03:11:17.317036+0000","last_clean":"2024-04-13T03:11:17.317036+0000","last_became_active":"2024-04-13T03:11:16.290717+0000","last_became_peered":"2024-04-13T03:11:16.290717+0000","last_unstale":"2024-04-13T03:11:17.317036+0000","last_undegraded":"2024-04-13T03:11:17.317036+0000","last_fullsized":"2024-04-13T03:11:17.317036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:14:21.972935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.302593+0000","last_change":"2024-04-13T03:11:17.299719+0000","last_active":"2024-04-13T03:11:17.302593+0000","last_peered":"2024-04-13T03:11:17.302593+0000","last_clean":"2024-04-13T03:11:17.302593+0000","last_became_active":"2024-04-13T03:11:17.299425+0000","last_became_peered":"2024-04-13T03:11:17.299425+0000","last_unstale":"2024-04-13T03:11:17.302593+0000","last_undegraded":"2024-04-13T03:11:17.302593+0000","last_fullsized":"2024-04-13T03:11:17.302593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:30.832276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318740+0000","last_change":"2024-04-13T03:11:16.293749+0000","last_active":"2024-04-13T03:11:17.318740+0000","last_peered":"2024-04-13T03:11:17.318740+0000","last_clean":"2024-04-13T03:11:17.318740+0000","last_became_active":"2024-04-13T03:11:16.293341+0000","last_became_peered":"2024-04-13T03:11:16.293341+0000","last_unstale":"2024-04-13T03:11:17.318740+0000","last_undegraded":"2024-04-13T03:11:17.318740+0000","last_fullsized":"2024-04-13T03:11:17.318740+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:43:38.828201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322288+0000","last_change":"2024-04-13T03:11:17.314431+0000","last_active":"2024-04-13T03:11:17.322288+0000","last_peered":"2024-04-13T03:11:17.322288+0000","last_clean":"2024-04-13T03:11:17.322288+0000","last_became_active":"2024-04-13T03:11:17.314173+0000","last_became_peered":"2024-04-13T03:11:17.314173+0000","last_unstale":"2024-04-13T03:11:17.322288+0000","last_undegraded":"2024-04-13T03:11:17.322288+0000","last_fullsized":"2024-04-13T03:11:17.322288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:54.438313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.298869+0000","last_change":"2024-04-13T03:11:16.290092+0000","last_active":"2024-04-13T03:11:17.298869+0000","last_peered":"2024-04-13T03:11:17.298869+0000","last_clean":"2024-04-13T03:11:17.298869+0000","last_became_active":"2024-04-13T03:11:16.289797+0000","last_became_peered":"2024-04-13T03:11:16.289797+0000","last_unstale":"2024-04-13T03:11:17.298869+0000","last_undegraded":"2024-04-13T03:11:17.298869+0000","last_fullsized":"2024-04-13T03:11:17.298869+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:43.465157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.313961+0000","last_change":"2024-04-13T03:11:17.303994+0000","last_active":"2024-04-13T03:11:17.313961+0000","last_peered":"2024-04-13T03:11:17.313961+0000","last_clean":"2024-04-13T03:11:17.313961+0000","last_became_active":"2024-04-13T03:11:17.303624+0000","last_became_peered":"2024-04-13T03:11:17.303624+0000","last_unstale":"2024-04-13T03:11:17.313961+0000","last_undegraded":"2024-04-13T03:11:17.313961+0000","last_fullsized":"2024-04-13T03:11:17.313961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:43.025512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.322316+0000","last_change":"2024-04-13T03:11:16.294336+0000","last_active":"2024-04-13T03:11:17.322316+0000","last_peered":"2024-04-13T03:11:17.322316+0000","last_clean":"2024-04-13T03:11:17.322316+0000","last_became_active":"2024-04-13T03:11:16.294063+0000","last_became_peered":"2024-04-13T03:11:16.294063+0000","last_unstale":"2024-04-13T03:11:17.322316+0000","last_undegraded":"2024-04-13T03:11:17.322316+0000","last_fullsized":"2024-04-13T03:11:17.322316+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:57:28.027417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311345+0000","last_change":"2024-04-13T03:11:17.303393+0000","last_active":"2024-04-13T03:11:17.311345+0000","last_peered":"2024-04-13T03:11:17.311345+0000","last_clean":"2024-04-13T03:11:17.311345+0000","last_became_active":"2024-04-13T03:11:17.303109+0000","last_became_peered":"2024-04-13T03:11:17.303109+0000","last_unstale":"2024-04-13T03:11:17.311345+0000","last_undegraded":"2024-04-13T03:11:17.311345+0000","last_fullsized":"2024-04-13T03:11:17.311345+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:54:25.895429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.299995+0000","last_change":"2024-04-13T03:11:16.292326+0000","last_active":"2024-04-13T03:11:17.299995+0000","last_peered":"2024-04-13T03:11:17.299995+0000","last_clean":"2024-04-13T03:11:17.299995+0000","last_became_active":"2024-04-13T03:11:16.292079+0000","last_became_peered":"2024-04-13T03:11:16.292079+0000","last_unstale":"2024-04-13T03:11:17.299995+0000","last_undegraded":"2024-04-13T03:11:17.299995+0000","last_fullsized":"2024-04-13T03:11:17.299995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:24:10.359548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.309812+0000","last_change":"2024-04-13T03:11:17.304267+0000","last_active":"2024-04-13T03:11:17.309812+0000","last_peered":"2024-04-13T03:11:17.309812+0000","last_clean":"2024-04-13T03:11:17.309812+0000","last_became_active":"2024-04-13T03:11:17.303881+0000","last_became_peered":"2024-04-13T03:11:17.303881+0000","last_unstale":"2024-04-13T03:11:17.309812+0000","last_undegraded":"2024-04-13T03:11:17.309812+0000","last_fullsized":"2024-04-13T03:11:17.309812+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:05:14.298298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.291208+0000","last_change":"2024-04-13T03:11:16.295696+0000","last_active":"2024-04-13T03:11:17.291208+0000","last_peered":"2024-04-13T03:11:17.291208+0000","last_clean":"2024-04-13T03:11:17.291208+0000","last_became_active":"2024-04-13T03:11:16.294249+0000","last_became_peered":"2024-04-13T03:11:16.294249+0000","last_unstale":"2024-04-13T03:11:17.291208+0000","last_undegraded":"2024-04-13T03:11:17.291208+0000","last_fullsized":"2024-04-13T03:11:17.291208+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:53.431976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.311037+0000","last_change":"2024-04-13T03:11:17.307691+0000","last_active":"2024-04-13T03:11:17.311037+0000","last_peered":"2024-04-13T03:11:17.311037+0000","last_clean":"2024-04-13T03:11:17.311037+0000","last_became_active":"2024-04-13T03:11:17.307101+0000","last_became_peered":"2024-04-13T03:11:17.307101+0000","last_unstale":"2024-04-13T03:11:17.311037+0000","last_undegraded":"2024-04-13T03:11:17.311037+0000","last_fullsized":"2024-04-13T03:11:17.311037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:25.706262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.289416+0000","last_change":"2024-04-13T03:11:16.290729+0000","last_active":"2024-04-13T03:11:17.289416+0000","last_peered":"2024-04-13T03:11:17.289416+0000","last_clean":"2024-04-13T03:11:17.289416+0000","last_became_active":"2024-04-13T03:11:16.290454+0000","last_became_peered":"2024-04-13T03:11:16.290454+0000","last_unstale":"2024-04-13T03:11:17.289416+0000","last_undegraded":"2024-04-13T03:11:17.289416+0000","last_fullsized":"2024-04-13T03:11:17.289416+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:31.829634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316472+0000","last_change":"2024-04-13T03:11:17.306070+0000","last_active":"2024-04-13T03:11:17.316472+0000","last_peered":"2024-04-13T03:11:17.316472+0000","last_clean":"2024-04-13T03:11:17.316472+0000","last_became_active":"2024-04-13T03:11:17.305308+0000","last_became_peered":"2024-04-13T03:11:17.305308+0000","last_unstale":"2024-04-13T03:11:17.316472+0000","last_undegraded":"2024-04-13T03:11:17.316472+0000","last_fullsized":"2024-04-13T03:11:17.316472+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:34.277280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317674+0000","last_change":"2024-04-13T03:11:16.294574+0000","last_active":"2024-04-13T03:11:17.317674+0000","last_peered":"2024-04-13T03:11:17.317674+0000","last_clean":"2024-04-13T03:11:17.317674+0000","last_became_active":"2024-04-13T03:11:16.294353+0000","last_became_peered":"2024-04-13T03:11:16.294353+0000","last_unstale":"2024-04-13T03:11:17.317674+0000","last_undegraded":"2024-04-13T03:11:17.317674+0000","last_fullsized":"2024-04-13T03:11:17.317674+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:57:24.454175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307477+0000","last_change":"2024-04-13T03:11:17.303017+0000","last_active":"2024-04-13T03:11:17.307477+0000","last_peered":"2024-04-13T03:11:17.307477+0000","last_clean":"2024-04-13T03:11:17.307477+0000","last_became_active":"2024-04-13T03:11:17.302723+0000","last_became_peered":"2024-04-13T03:11:17.302723+0000","last_unstale":"2024-04-13T03:11:17.307477+0000","last_undegraded":"2024-04-13T03:11:17.307477+0000","last_fullsized":"2024-04-13T03:11:17.307477+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:45.538123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.304809+0000","last_change":"2024-04-13T03:11:16.292685+0000","last_active":"2024-04-13T03:11:17.304809+0000","last_peered":"2024-04-13T03:11:17.304809+0000","last_clean":"2024-04-13T03:11:17.304809+0000","last_became_active":"2024-04-13T03:11:16.292391+0000","last_became_peered":"2024-04-13T03:11:16.292391+0000","last_unstale":"2024-04-13T03:11:17.304809+0000","last_undegraded":"2024-04-13T03:11:17.304809+0000","last_fullsized":"2024-04-13T03:11:17.304809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.317545+0000","last_change":"2024-04-13T03:11:17.305645+0000","last_active":"2024-04-13T03:11:17.317545+0000","last_peered":"2024-04-13T03:11:17.317545+0000","last_clean":"2024-04-13T03:11:17.317545+0000","last_became_active":"2024-04-13T03:11:17.305355+0000","last_became_peered":"2024-04-13T03:11:17.305355+0000","last_unstale":"2024-04-13T03:11:17.317545+0000","last_undegraded":"2024-04-13T03:11:17.317545+0000","last_fullsized":"2024-04-13T03:11:17.317545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:34:42.136996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.292245+0000","last_change":"2024-04-13T03:11:16.288483+0000","last_active":"2024-04-13T03:11:17.292245+0000","last_peered":"2024-04-13T03:11:17.292245+0000","last_clean":"2024-04-13T03:11:17.292245+0000","last_became_active":"2024-04-13T03:11:16.288222+0000","last_became_peered":"2024-04-13T03:11:16.288222+0000","last_unstale":"2024-04-13T03:11:17.292245+0000","last_undegraded":"2024-04-13T03:11:17.292245+0000","last_fullsized":"2024-04-13T03:11:17.292245+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:42.019175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.315466+0000","last_change":"2024-04-13T03:11:17.308242+0000","last_active":"2024-04-13T03:11:17.315466+0000","last_peered":"2024-04-13T03:11:17.315466+0000","last_clean":"2024-04-13T03:11:17.315466+0000","last_became_active":"2024-04-13T03:11:17.307966+0000","last_became_peered":"2024-04-13T03:11:17.307966+0000","last_unstale":"2024-04-13T03:11:17.315466+0000","last_undegraded":"2024-04-13T03:11:17.315466+0000","last_fullsized":"2024-04-13T03:11:17.315466+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:48:23.922890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300395+0000","last_change":"2024-04-13T03:11:16.283111+0000","last_active":"2024-04-13T03:11:17.300395+0000","last_peered":"2024-04-13T03:11:17.300395+0000","last_clean":"2024-04-13T03:11:17.300395+0000","last_became_active":"2024-04-13T03:11:16.282850+0000","last_became_peered":"2024-04-13T03:11:16.282850+0000","last_unstale":"2024-04-13T03:11:17.300395+0000","last_undegraded":"2024-04-13T03:11:17.300395+0000","last_fullsized":"2024-04-13T03:11:17.300395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:48:22.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.316417+0000","last_change":"2024-04-13T03:11:17.310842+0000","last_active":"2024-04-13T03:11:17.316417+0000","last_peered":"2024-04-13T03:11:17.316417+0000","last_clean":"2024-04-13T03:11:17.316417+0000","last_became_active":"2024-04-13T03:11:17.310592+0000","last_became_peered":"2024-04-13T03:11:17.310592+0000","last_unstale":"2024-04-13T03:11:17.316417+0000","last_undegraded":"2024-04-13T03:11:17.316417+0000","last_fullsized":"2024-04-13T03:11:17.316417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:46:44.777716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300141+0000","last_change":"2024-04-13T03:11:16.294484+0000","last_active":"2024-04-13T03:11:17.300141+0000","last_peered":"2024-04-13T03:11:17.300141+0000","last_clean":"2024-04-13T03:11:17.300141+0000","last_became_active":"2024-04-13T03:11:16.294223+0000","last_became_peered":"2024-04-13T03:11:16.294223+0000","last_unstale":"2024-04-13T03:11:17.300141+0000","last_undegraded":"2024-04-13T03:11:17.300141+0000","last_fullsized":"2024-04-13T03:11:17.300141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:41.109398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.307059+0000","last_change":"2024-04-13T03:11:17.301528+0000","last_active":"2024-04-13T03:11:17.307059+0000","last_peered":"2024-04-13T03:11:17.307059+0000","last_clean":"2024-04-13T03:11:17.307059+0000","last_became_active":"2024-04-13T03:11:17.301277+0000","last_became_peered":"2024-04-13T03:11:17.301277+0000","last_unstale":"2024-04-13T03:11:17.307059+0000","last_undegraded":"2024-04-13T03:11:17.307059+0000","last_fullsized":"2024-04-13T03:11:17.307059+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:53:58.099757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293006+0000","last_change":"2024-04-13T03:11:16.289098+0000","last_active":"2024-04-13T03:11:17.293006+0000","last_peered":"2024-04-13T03:11:17.293006+0000","last_clean":"2024-04-13T03:11:17.293006+0000","last_became_active":"2024-04-13T03:11:16.288450+0000","last_became_peered":"2024-04-13T03:11:16.288450+0000","last_unstale":"2024-04-13T03:11:17.293006+0000","last_undegraded":"2024-04-13T03:11:17.293006+0000","last_fullsized":"2024-04-13T03:11:17.293006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:20:09.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.321974+0000","last_change":"2024-04-13T03:11:17.311576+0000","last_active":"2024-04-13T03:11:17.321974+0000","last_peered":"2024-04-13T03:11:17.321974+0000","last_clean":"2024-04-13T03:11:17.321974+0000","last_became_active":"2024-04-13T03:11:17.308969+0000","last_became_peered":"2024-04-13T03:11:17.308969+0000","last_unstale":"2024-04-13T03:11:17.321974+0000","last_undegraded":"2024-04-13T03:11:17.321974+0000","last_fullsized":"2024-04-13T03:11:17.321974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:49:58.487036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.293431+0000","last_change":"2024-04-13T03:11:16.282731+0000","last_active":"2024-04-13T03:11:17.293431+0000","last_peered":"2024-04-13T03:11:17.293431+0000","last_clean":"2024-04-13T03:11:17.293431+0000","last_became_active":"2024-04-13T03:11:16.282448+0000","last_became_peered":"2024-04-13T03:11:16.282448+0000","last_unstale":"2024-04-13T03:11:17.293431+0000","last_undegraded":"2024-04-13T03:11:17.293431+0000","last_fullsized":"2024-04-13T03:11:17.293431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:46.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.318781+0000","last_change":"2024-04-13T03:11:17.314498+0000","last_active":"2024-04-13T03:11:17.318781+0000","last_peered":"2024-04-13T03:11:17.318781+0000","last_clean":"2024-04-13T03:11:17.318781+0000","last_became_active":"2024-04-13T03:11:17.314184+0000","last_became_peered":"2024-04-13T03:11:17.314184+0000","last_unstale":"2024-04-13T03:11:17.318781+0000","last_undegraded":"2024-04-13T03:11:17.318781+0000","last_fullsized":"2024-04-13T03:11:17.318781+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:16.270837+0000","last_clean_scrub_stamp":"2024-04-13T03:11:16.270837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:57:29.033754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.292464+0000","last_change":"2024-04-13T03:11:16.287541+0000","last_active":"2024-04-13T03:11:17.292464+0000","last_peered":"2024-04-13T03:11:17.292464+0000","last_clean":"2024-04-13T03:11:17.292464+0000","last_became_active":"2024-04-13T03:11:16.287126+0000","last_became_peered":"2024-04-13T03:11:16.287126+0000","last_unstale":"2024-04-13T03:11:17.292464+0000","last_undegraded":"2024-04-13T03:11:17.292464+0000","last_fullsized":"2024-04-13T03:11:17.292464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:11:15.264500+0000","last_clean_scrub_stamp":"2024-04-13T03:11:15.264500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:33.270943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-13T03:11:17.300427+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:11:17.300427+0000","last_peered":"2024-04-13T03:11:17.300427+0000","last_clean":"2024-04-13T03:11:17.300427+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:11:17.300427+0000","last_undegraded":"2024-04-13T03:11:17.300427+0000","last_fullsized":"2024-04-13T03:11:17.300427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1187840,"data_stored":853335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":271768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1187840,"data_stored":853335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:11:20.612 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:11:20.613 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:11:20.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.754+0000 7f5f1f2b7640 1 Processor -- start 2024-04-13T03:11:20.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.754+0000 7f5f1f2b7640 1 -- start start 2024-04-13T03:11:20.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.755+0000 7f5f1f2b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f1815ed90 0x7f5f180d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.755+0000 7f5f1f2b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f180b3d60 0x7f5f180d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.755+0000 7f5f1f2b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f180cfd20 con 0x7f5f1815e9b0 2024-04-13T03:11:20.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.755+0000 7f5f1f2b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5f18162af0 con 0x7f5f180b3d60 2024-04-13T03:11:20.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.755+0000 7f5f1f2b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5f18162dd0 con 0x7f5f1815ed90 2024-04-13T03:11:20.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d854640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f180b3d60 0x7f5f180d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1c852640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f1815ed90 0x7f5f180d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d053640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5f1815e9b0 0x7f5f180d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36918/0 (socket says 172.21.15.106:36918) 2024-04-13T03:11:20.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d053640 1 -- 172.21.15.106:0/121029058 learned_addr learned my addr 172.21.15.106:0/121029058 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d854640 1 -- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f1815ed90 msgr2=0x7f5f180d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d854640 1 --2- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f1815ed90 0x7f5f180d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d854640 1 -- 172.21.15.106:0/121029058 >> v1:172.21.15.114:6789/0 conn(0x7f5f1815e9b0 legacy=0x7f5f180d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f1d854640 1 -- 172.21.15.106:0/121029058 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181630b0 con 0x7f5f180b3d60 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.756+0000 7f5f0ffff640 1 -- 172.21.15.106:0/121029058 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20846917 0 0) 0x7f5f180cfd20 con 0x7f5f1815e9b0 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1d854640 1 --2- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f180b3d60 0x7f5f180d0200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f100097a0 tx=0x7f5f1002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3daad69c3f34e14b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f0ffff640 1 -- 172.21.15.106:0/121029058 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10034070 con 0x7f5f180b3d60 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f0ffff640 1 -- 172.21.15.106:0/121029058 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f10030610 con 0x7f5f180b3d60 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f0ffff640 1 -- 172.21.15.106:0/121029058 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100309e0 con 0x7f5f180b3d60 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f180b3d60 msgr2=0x7f5f180d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f180b3d60 0x7f5f180d0200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f100097a0 tx=0x7f5f1002b960 comp rx=0 tx=0).stop 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/121029058 shutdown_connections 2024-04-13T03:11:20.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f180b3d60 0x7f5f180d0200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/121029058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f1815ed90 0x7f5f180d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.757+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/121029058 >> 172.21.15.106:0/121029058 conn(0x7f5f1808bf10 msgr2=0x7f5f180b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:20.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.758+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/121029058 shutdown_connections 2024-04-13T03:11:20.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.758+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/121029058 wait complete. 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 Processor -- start 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 -- start start 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 0x7f5f18155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 0x7f5f180caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18162800 con 0x7f5f180c9260 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18162680 con 0x7f5f1815e9b0 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.759+0000 7f5f1f2b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180d4d00 con 0x7f5f180b3d60 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45758/0 (socket says 172.21.15.106:45758) 2024-04-13T03:11:20.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 -- 172.21.15.106:0/3311811014 learned_addr learned my addr 172.21.15.106:0/3311811014 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d053640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 0x7f5f18155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1c852640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 0x7f5f180caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 -- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 msgr2=0x7f5f18155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 0x7f5f18155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 -- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 msgr2=0x7f5f180caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 0x7f5f180caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 -- 172.21.15.106:0/3311811014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181e9680 con 0x7f5f180c9260 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d053640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 0x7f5f18155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1c852640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 0x7f5f180caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1d854640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002b9a0 tx=0x7f5f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10034040 con 0x7f5f180c9260 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f10030410 con 0x7f5f180c9260 2024-04-13T03:11:20.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f180c9710 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10006e10 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.760+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f180c9cb0 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.761+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18068a10 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.762+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f1004c060 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.762+0000 7f5f0dffb640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 0x7f5efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.762+0000 7f5f1d053640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 0x7f5efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.762+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f5f100ad750 con 0x7f5f180c9260 2024-04-13T03:11:20.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.763+0000 7f5f1d053640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 0x7f5efc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f140012e0 tx=0x7f5f14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:20.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:20.767+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f1007b9f0 con 0x7f5f180c9260 2024-04-13T03:11:21.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.006+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5f180d53a0 con 0x7f5f180c9260 2024-04-13T03:11:21.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.008+0000 7f5f0dffb640 1 -- 172.21.15.106:0/3311811014 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f5f1007b2f0 con 0x7f5f180c9260 2024-04-13T03:11:21.009 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:11:21.009 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:11:21.010 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:11:21.010 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:11:21.010 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:11:21.010 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 msgr2=0x7f5efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 0x7f5efc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f140012e0 tx=0x7f5f14009040 comp rx=0 tx=0).stop 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 msgr2=0x7f5f181e9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002b9a0 tx=0x7f5f100046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 shutdown_connections 2024-04-13T03:11:21.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5efc06c8f0 0x7f5efc06edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f180c9260 0x7f5f181e9140 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f1815e9b0 0x7f5f180caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 --2- 172.21.15.106:0/3311811014 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f180b3d60 0x7f5f18155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 >> 172.21.15.106:0/3311811014 conn(0x7f5f1808bf10 msgr2=0x7f5f1815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 shutdown_connections 2024-04-13T03:11:21.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.010+0000 7f5f1f2b7640 1 -- 172.21.15.106:0/3311811014 wait complete. 2024-04-13T03:11:21.022 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:21.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.120+0000 7f2edcc20640 1 Processor -- start 2024-04-13T03:11:21.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.120+0000 7f2edcc20640 1 -- start start 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.121+0000 7f2edcc20640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 0x7f2ed80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.121+0000 7f2edcc20640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ed80b3d60 0x7f2ed80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.121+0000 7f2edcc20640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed80cfd20 con 0x7f2ed815e9b0 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.121+0000 7f2edcc20640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed8162af0 con 0x7f2ed80b3d60 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.121+0000 7f2edcc20640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed8162dd0 con 0x7f2ed815ed90 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ed80b3d60 0x7f2ed80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ecdd9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 0x7f2ed80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ed80b3d60 0x7f2ed80d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:35988/0 (socket says 172.21.15.106:35988) 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed659c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2ed815e9b0 0x7f2ed80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36930/0 (socket says 172.21.15.106:36930) 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2185175660 learned_addr learned my addr 172.21.15.106:0/2185175660 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 msgr2=0x7f2ed80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 0x7f2ed80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2185175660 >> v1:172.21.15.114:6789/0 conn(0x7f2ed815e9b0 legacy=0x7f2ed80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2185175660 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed81630b0 con 0x7f2ed80b3d60 2024-04-13T03:11:21.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.122+0000 7f2ecdd9b640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 0x7f2ed80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2ed6d9d640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80d0200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009e50 tx=0x7f2ed002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ada37246d4e086f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2ed5d9b640 1 -- 172.21.15.106:0/2185175660 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed002cb90 con 0x7f2ed80b3d60 2024-04-13T03:11:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2ed5d9b640 1 -- 172.21.15.106:0/2185175660 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed002cd30 con 0x7f2ed80b3d60 2024-04-13T03:11:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2ed5d9b640 1 -- 172.21.15.106:0/2185175660 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0034610 con 0x7f2ed80b3d60 2024-04-13T03:11:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 -- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 msgr2=0x7f2ed80d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80d0200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009e50 tx=0x7f2ed002bff0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 -- 172.21.15.106:0/2185175660 shutdown_connections 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80d0200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2185175660 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815ed90 0x7f2ed80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.123+0000 7f2edcc20640 1 -- 172.21.15.106:0/2185175660 >> 172.21.15.106:0/2185175660 conn(0x7f2ed808bf10 msgr2=0x7f2ed80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.124+0000 7f2edcc20640 1 -- 172.21.15.106:0/2185175660 shutdown_connections 2024-04-13T03:11:21.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.124+0000 7f2edcc20640 1 -- 172.21.15.106:0/2185175660 wait complete. 2024-04-13T03:11:21.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.125+0000 7f2edcc20640 1 Processor -- start 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.125+0000 7f2edcc20640 1 -- start start 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815e9b0 0x7f2ed80b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8162800 con 0x7f2ed8161ef0 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8162680 con 0x7f2ed80b3d60 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2edcc20640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed80d4d00 con 0x7f2ed815e9b0 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45770/0 (socket says 172.21.15.106:45770) 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2776620238 learned_addr learned my addr 172.21.15.106:0/2776620238 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed659c640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ecdd9b640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815e9b0 0x7f2ed80b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815e9b0 msgr2=0x7f2ed80b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815e9b0 0x7f2ed80b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 msgr2=0x7f2ed80b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 -- 172.21.15.106:0/2776620238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80b5d10 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.126+0000 7f2ed6d9d640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0036040 tx=0x7f2ed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0043020 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed003e450 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed80b5ff0 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0009510 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed80b65b0 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.127+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed8068a10 con 0x7f2ed8161ef0 2024-04-13T03:11:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.128+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ed0046020 con 0x7f2ed8161ef0 2024-04-13T03:11:21.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.128+0000 7f2ecf7fe640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.128+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2ed00ae270 con 0x7f2ed8161ef0 2024-04-13T03:11:21.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.128+0000 7f2ed659c640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.129+0000 7f2ed659c640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 0x7f2eb806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007fd0 tx=0x7f2ec8007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.133+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ed004d030 con 0x7f2ed8161ef0 2024-04-13T03:11:21.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.349+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ed80d53a0 con 0x7f2ed8161ef0 2024-04-13T03:11:21.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.349+0000 7f2ecf7fe640 1 -- 172.21.15.106:0/2776620238 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f2ed007c5a0 con 0x7f2ed8161ef0 2024-04-13T03:11:21.351 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:21.351 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:21.351 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 msgr2=0x7f2eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 0x7f2eb806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007fd0 tx=0x7f2ec8007c40 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 msgr2=0x7f2ed814f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0036040 tx=0x7f2ed0004a40 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 shutdown_connections 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ed815e9b0 0x7f2ed80b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ed8161ef0 0x7f2ed814f5e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 --2- 172.21.15.106:0/2776620238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ed80b3d60 0x7f2ed80b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.351+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 >> 172.21.15.106:0/2776620238 conn(0x7f2ed808bf10 msgr2=0x7f2ed80b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.352+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 shutdown_connections 2024-04-13T03:11:21.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.352+0000 7f2edcc20640 1 -- 172.21.15.106:0/2776620238 wait complete. 2024-04-13T03:11:21.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:11:16.632602+0000', 'modified': '2024-04-13T03:11:18.644896+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5856}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5856': {'gid': 5856, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6835/4219642587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6833', 'nonce': 4219642587}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 4219642587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5856, 'qdb_cluster': [5856]} 2024-04-13T03:11:21.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:11:21.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:11:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.869+0000 7fd87c1bb640 1 Processor -- start 2024-04-13T03:11:21.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.869+0000 7fd87c1bb640 1 -- start start 2024-04-13T03:11:21.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd87c1bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd87406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd87c1bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8740d02b0 con 0x7fd87415fbb0 2024-04-13T03:11:21.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd87c1bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87406ba60 con 0x7fd8740cf9b0 2024-04-13T03:11:21.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd87c1bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740d0590 con 0x7fd874072d40 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd879756640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd87415fbb0 0x7fd874073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45038/0 (socket says 172.21.15.106:45038) 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd879756640 1 -- 172.21.15.106:0/3030057303 learned_addr learned my addr 172.21.15.106:0/3030057303 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420233446 0 0) 0x7fd8740d02b0 con 0x7fd87415fbb0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd879f57640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd87406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd860003520 con 0x7fd87415fbb0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924181669 0 0) 0x7fd860003520 con 0x7fd87415fbb0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 msgr2=0x7fd87406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd87406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 >> v1:172.21.15.114:6789/0 conn(0x7fd8740cf9b0 legacy=0x7fd8740ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87406bd40 con 0x7fd87415fbb0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fd864001670 con 0x7fd87415fbb0 2024-04-13T03:11:21.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.870+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 >> v1:172.21.15.106:6789/0 conn(0x7fd87415fbb0 legacy=0x7fd874073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8600039e0 0x7fd860003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740d0590 con 0x7fd860004d40 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd86000ba70 con 0x7fd8600039e0 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd86000bc70 con 0x7fd874072d40 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd879f57640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8600039e0 0x7fd860003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd879756640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 -- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 msgr2=0x7fd860004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd874072d40 0x7fd860004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 -- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8600039e0 msgr2=0x7fd860003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8600039e0 0x7fd860003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.871+0000 7fd87a758640 1 -- 172.21.15.106:0/3030057303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd860003520 con 0x7fd860004d40 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87a758640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd8600051c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd86c001e20 tx=0x7fd860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6322ee6d88d859 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c001a70 con 0x7fd860004d40 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86c003b70 con 0x7fd860004d40 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd878f55640 1 -- 172.21.15.106:0/3030057303 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0050d0 con 0x7fd860004d40 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 -- 172.21.15.106:0/3030057303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 msgr2=0x7fd8600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/3030057303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd8600051c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd86c001e20 tx=0x7fd860005830 comp rx=0 tx=0).stop 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87a758640 1 -- 172.21.15.106:0/3030057303 reap_dead start 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 -- 172.21.15.106:0/3030057303 shutdown_connections 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 -- 172.21.15.106:0/3030057303 >> 172.21.15.106:0/3030057303 conn(0x7fd874072220 msgr2=0x7fd874072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 -- 172.21.15.106:0/3030057303 shutdown_connections 2024-04-13T03:11:21.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.872+0000 7fd87c1bb640 1 -- 172.21.15.106:0/3030057303 wait complete. 2024-04-13T03:11:21.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 Processor -- start 2024-04-13T03:11:21.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 -- start start 2024-04-13T03:11:21.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8741e9a90 0x7fd8741e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 0x7fd8740b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740cfe40 con 0x7fd860004d40 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741486b0 con 0x7fd8741e9a90 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd87c1bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740cffc0 con 0x7fd8740b52b0 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd879f57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd879f57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45782/0 (socket says 172.21.15.106:45782) 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.874+0000 7fd879f57640 1 -- 172.21.15.106:0/936114880 learned_addr learned my addr 172.21.15.106:0/936114880 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd87a758640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 0x7fd8740b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879756640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8741e9a90 0x7fd8741e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 -- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 msgr2=0x7fd8740b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 0x7fd8740b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 -- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8741e9a90 msgr2=0x7fd8741e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8741e9a90 0x7fd8741e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 -- 172.21.15.106:0/936114880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741ea430 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd87a758640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 0x7fd8740b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd879f57640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd8700119d0 tx=0x7fd870011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd86a7fc640 1 -- 172.21.15.106:0/936114880 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000fb50 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd86a7fc640 1 -- 172.21.15.106:0/936114880 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87000fcf0 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8740b6030 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd86a7fc640 1 -- 172.21.15.106:0/936114880 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000fea0 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.875+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd874131240 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.876+0000 7fd86a7fc640 1 -- 172.21.15.106:0/936114880 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd870012040 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.877+0000 7fd86a7fc640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.877+0000 7fd86a7fc640 1 -- 172.21.15.106:0/936114880 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd87008c3a0 con 0x7fd860004d40 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.877+0000 7fd879756640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.877+0000 7fd879756640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 0x7fd85806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd864001050 tx=0x7fd864009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.878+0000 7fd87efa5740 1 Processor -- start 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.878+0000 7fd87efa5740 1 -- start start 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.878+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616cc2c0 0x560d616fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87efa5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d61636990 con 0x560d616f91c0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d615dd620 con 0x560d616cc2c0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560d61632950 con 0x560d616f8de0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45790/0 (socket says 172.21.15.106:45790) 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 -- 172.21.15.106:0/1541150669 learned_addr learned my addr 172.21.15.106:0/1541150669 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879f57640 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 -- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 msgr2=0x560d616cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd87a758640 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616cc2c0 0x560d616fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 -- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616cc2c0 msgr2=0x560d616fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616cc2c0 0x560d616fc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 -- 172.21.15.106:0/1541150669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d613492d0 con 0x560d616f91c0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd879756640 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd864007cb0 tx=0x7fd864013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba776fb984b50494 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd8577fe640 1 -- 172.21.15.106:0/1541150669 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864026070 con 0x560d616f91c0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd8577fe640 1 -- 172.21.15.106:0/1541150669 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86401c050 con 0x560d616f91c0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.879+0000 7fd8577fe640 1 -- 172.21.15.106:0/1541150669 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864021410 con 0x560d616f91c0 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- 172.21.15.106:0/1541150669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 msgr2=0x560d616fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd864007cb0 tx=0x7fd864013000 comp rx=0 tx=0).stop 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- 172.21.15.106:0/1541150669 shutdown_connections 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616cc2c0 0x560d616fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616fbb70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 --2- 172.21.15.106:0/1541150669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- 172.21.15.106:0/1541150669 >> 172.21.15.106:0/1541150669 conn(0x560d616cdff0 msgr2=0x560d61716b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- 172.21.15.106:0/1541150669 shutdown_connections 2024-04-13T03:11:21.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- 172.21.15.106:0/1541150669 wait complete. 2024-04-13T03:11:21.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.880+0000 7fd87efa5740 1 -- start start 2024-04-13T03:11:21.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 Processor -- start 2024-04-13T03:11:21.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 0x560d61726bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 0x560d6170ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d616d0590 con 0x560d616f91c0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d615f09e0 con 0x560d616f8de0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.881+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560d616cf980 con 0x560d6170e5f0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87efa5740 10 client.0 _ll_register_callbacks cb 0x560d6162c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45794/0 (socket says 172.21.15.106:45794) 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 -- 172.21.15.106:0/2214858911 learned_addr learned my addr 172.21.15.106:0/2214858911 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879f57640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 0x560d61726bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 -- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 msgr2=0x560d6170ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87a758640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 0x560d6170ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 0x560d6170ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 -- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 msgr2=0x560d61726bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 0x560d61726bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 -- 172.21.15.106:0/2214858911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d614136b0 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87a758640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 0x560d6170ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879f57640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 0x560d61726bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd879756640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd864001eb0 tx=0x7fd864016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bbc7c6eb860e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864032060 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd864022770 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86401c050 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd864038050 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87efa5740 10 client.5973 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x560d617102b0 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.882+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86401c050 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fd86403b020 con 0x560d616f91c0 2024-04-13T03:11:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 10 client.5973 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 10 client.5973 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 20 client.5973 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d616fdd40 0x560d61734a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 4 client.5973 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d616443b0 con 0x560d616fdd40 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd879f57640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d616fdd40 0x560d61734a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd854ff9640 10 client.5973 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.883+0000 7fd879f57640 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d616fdd40 0x560d61734a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2eb7411f4782c0b server_cookie=ed30b0597a548bce in_seq=0 out_seq=0 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd854ff9640 1 -- 172.21.15.106:0/2214858911 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d616443b0 con 0x560d616fdd40 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd854ff9640 10 client.5973 handle_command_reply: tid=0 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 client.5973 shutdown 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 msgr2=0x560d616f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd864001eb0 tx=0x7fd864016000 comp rx=0 tx=0).stop 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 shutdown_connections 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d616fdd40 0x560d61734a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d6170e5f0 0x560d6170ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd854ff9640 0 client.5973 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d616f6a20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 --2- 172.21.15.106:0/2214858911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d616f8de0 0x560d61726bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 >> 172.21.15.106:0/2214858911 conn(0x560d616cc2c0 msgr2=0x560d616fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 shutdown_connections 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.884+0000 7fd87efa5740 1 -- 172.21.15.106:0/2214858911 wait complete. 2024-04-13T03:11:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.885+0000 7fd87efa5740 20 client.5973 trim_cache size 0 max 16384 2024-04-13T03:11:21.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 Processor -- start 2024-04-13T03:11:21.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 -- start start 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d61729ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d616ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd874200d20 con 0x560d616f91c0 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd874200080 con 0x560d6170e5f0 2024-04-13T03:11:21.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd8742006d0 con 0x560d616f8de0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45802/0 (socket says 172.21.15.106:45802) 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.903+0000 7fd879756640 1 -- 172.21.15.106:0/3808472430 learned_addr learned my addr 172.21.15.106:0/3808472430 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 -- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 msgr2=0x560d61729ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d61729ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 -- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 msgr2=0x560d616ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d616ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 -- 172.21.15.106:0/3808472430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d61726fa0 con 0x560d616f91c0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd879756640 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd864008cc0 tx=0x7fd864017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d152183f8856a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd856ffd640 1 -- 172.21.15.106:0/3808472430 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864026030 con 0x560d616f91c0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd856ffd640 1 -- 172.21.15.106:0/3808472430 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86401c040 con 0x560d616f91c0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd856ffd640 1 -- 172.21.15.106:0/3808472430 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864015d70 con 0x560d616f91c0 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 -- 172.21.15.106:0/3808472430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 msgr2=0x560d61717090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd864008cc0 tx=0x7fd864017000 comp rx=0 tx=0).stop 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 -- 172.21.15.106:0/3808472430 shutdown_connections 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d616ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3808472430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d61729ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.904+0000 7fd87efa5740 1 -- 172.21.15.106:0/3808472430 >> 172.21.15.106:0/3808472430 conn(0x560d616cc2c0 msgr2=0x560d617197c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.905+0000 7fd87efa5740 1 -- 172.21.15.106:0/3808472430 shutdown_connections 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.905+0000 7fd87efa5740 1 -- 172.21.15.106:0/3808472430 wait complete. 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.905+0000 7fd87efa5740 1 -- start start 2024-04-13T03:11:21.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.905+0000 7fd87efa5740 1 Processor -- start 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d617175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d616fdbc0 con 0x560d616f91c0 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d616d0590 con 0x560d6170e5f0 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560d616fe210 con 0x560d616f8de0 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87efa5740 10 client.0 _ll_register_callbacks cb 0x7fd864024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45814/0 (socket says 172.21.15.106:45814) 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 -- 172.21.15.106:0/3795108752 learned_addr learned my addr 172.21.15.106:0/3795108752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:21.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879f57640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87a758640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d617175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 -- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 msgr2=0x560d616d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 -- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 msgr2=0x560d617175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d617175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 -- 172.21.15.106:0/3795108752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d617180a0 con 0x560d616f91c0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd87a758640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d617175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd879756640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd864024040 tx=0x7fd864001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354703013d115ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.906+0000 7fd854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864026040 con 0x560d616f91c0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd864045e70 con 0x560d616f91c0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86401c040 con 0x560d616f91c0 2024-04-13T03:11:21.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd864032020 con 0x560d616f91c0 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 10 client.5979 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x560d61718c80 con 0x560d616f91c0 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86401c040 con 0x560d616f91c0 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fd86403b070 con 0x560d616f91c0 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 10 client.5979 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 10 client.5979 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 20 client.5979 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d61716850 0x560d6170c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 4 client.5979 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "status"}) -- 0x560d6163e220 con 0x560d61716850 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.907+0000 7fd879f57640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d61716850 0x560d6170c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd854ff9640 10 client.5979 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:21.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd879f57640 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d61716850 0x560d6170c8e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7499801844a5bb server_cookie=6be734b400cf0ef5 in_seq=0 out_seq=0 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd854ff9640 1 -- 172.21.15.106:0/3795108752 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x560d6163e220 con 0x560d61716850 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd854ff9640 10 client.5979 handle_command_reply: tid=0 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 client.5979 shutdown 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 msgr2=0x560d61717090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd864024040 tx=0x7fd864001a40 comp rx=0 tx=0).stop 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 shutdown_connections 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x560d61716850 0x560d6170c8e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d6170e5f0 0x560d617175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d616f91c0 0x560d61717090 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 --2- 172.21.15.106:0/3795108752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560d616f8de0 0x560d616d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd854ff9640 0 client.5979 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 >> 172.21.15.106:0/3795108752 conn(0x560d616cc2c0 msgr2=0x560d6171a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 shutdown_connections 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 1 -- 172.21.15.106:0/3795108752 wait complete. 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.908+0000 7fd87efa5740 20 client.5979 trim_cache size 0 max 16384 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5856, 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.614670383, 2024-04-13T03:11:21.916 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 22, 2024-04-13T03:11:21.917 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 52, 2024-04-13T03:11:21.917 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 52, 2024-04-13T03:11:21.917 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 9.6673389019999991 2024-04-13T03:11:21.917 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 msgr2=0x7fd85806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 0x7fd85806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd864001050 tx=0x7fd864009040 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 msgr2=0x7fd87415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd8700119d0 tx=0x7fd870011e90 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 shutdown_connections 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd8740b52b0 0x7fd8740b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8741e9a90 0x7fd8741e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 --2- 172.21.15.106:0/936114880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd860004d40 0x7fd87415e2e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 >> 172.21.15.106:0/936114880 conn(0x7fd874072220 msgr2=0x7fd8740ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 shutdown_connections 2024-04-13T03:11:21.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:21.909+0000 7fd87c1bb640 1 -- 172.21.15.106:0/936114880 wait complete. 2024-04-13T03:11:21.920 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:11:22.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 Processor -- start 2024-04-13T03:11:22.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 -- start start 2024-04-13T03:11:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1b80d22d0 0x7fa1b80ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80cf6f0 0x7fa1b80d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b80cf4b0 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b80b7830 con 0x7fa1b80d22d0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.019+0000 7fa1bd6de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b80cfe40 con 0x7fa1b80cf6f0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45828/0 (socket says 172.21.15.106:45828) 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 -- 172.21.15.106:0/3872375981 learned_addr learned my addr 172.21.15.106:0/3872375981 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b67fc640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80cf6f0 0x7fa1b80d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b6ffd640 1 --2- 172.21.15.106:0/3872375981 >> v2:172.21.15.114:3300/0 conn(0x7fa1b80d22d0 0x7fa1b80ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 -- 172.21.15.106:0/3872375981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80cf6f0 msgr2=0x7fa1b80d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80cf6f0 0x7fa1b80d26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 -- 172.21.15.106:0/3872375981 >> v2:172.21.15.114:3300/0 conn(0x7fa1b80d22d0 msgr2=0x7fa1b80ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:22.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 --2- 172.21.15.106:0/3872375981 >> v2:172.21.15.114:3300/0 conn(0x7fa1b80d22d0 0x7fa1b80ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.020+0000 7fa1b77fe640 1 -- 172.21.15.106:0/3872375981 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b80cfb80 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1b77fe640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80d4660 tx=0x7fa1b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9a55f8149c784e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:22.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1b5ffb640 1 -- 172.21.15.106:0/3872375981 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b002ca70 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1b5ffb640 1 -- 172.21.15.106:0/3872375981 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b002cc10 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1b5ffb640 1 -- 172.21.15.106:0/3872375981 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00343f0 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3872375981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d2bf0 msgr2=0x7fa1b806c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80d4660 tx=0x7fa1b002bed0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3872375981 shutdown_connections 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d2bf0 0x7fa1b806c2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3872375981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80cf6f0 0x7fa1b80d26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3872375981 >> v2:172.21.15.114:3300/0 conn(0x7fa1b80d22d0 0x7fa1b80ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3872375981 >> 172.21.15.106:0/3872375981 conn(0x7fa1b80d4700 msgr2=0x7fa1b80d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3872375981 shutdown_connections 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.021+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3872375981 wait complete. 2024-04-13T03:11:22.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 Processor -- start 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 -- start start 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 0x7fa1b814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80d2bf0 0x7fa1b815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80d3dd0 con 0x7fa1b80d22d0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80cf1d0 con 0x7fa1b80cf6f0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1bd6de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b80d3080 con 0x7fa1b80d2bf0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1b67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1b67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45844/0 (socket says 172.21.15.106:45844) 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.023+0000 7fa1b67fc640 1 -- 172.21.15.106:0/3734410261 learned_addr learned my addr 172.21.15.106:0/3734410261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b6ffd640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 0x7fa1b814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b77fe640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80d2bf0 0x7fa1b815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 -- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80d2bf0 msgr2=0x7fa1b815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:22.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80d2bf0 0x7fa1b815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 -- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 msgr2=0x7fa1b814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 0x7fa1b814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 -- 172.21.15.106:0/3734410261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b815f7f0 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b6ffd640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 0x7fa1b814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1b67fc640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00d620 tx=0x7fa1ac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00b620 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac019070 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b815fb30 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00ba00 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.024+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b815a170 con 0x7fa1b80d22d0 2024-04-13T03:11:22.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.025+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b8068a10 con 0x7fa1b80d22d0 2024-04-13T03:11:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.026+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1ac00bba0 con 0x7fa1b80d22d0 2024-04-13T03:11:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.026+0000 7fa19f7fe640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.026+0000 7fa1b6ffd640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.026+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa1ac08c0c0 con 0x7fa1b80d22d0 2024-04-13T03:11:22.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.026+0000 7fa1b6ffd640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 0x7fa1a406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:22.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.030+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1b815fb30 con 0x7fa1b80d22d0 2024-04-13T03:11:22.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.243+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1b806bd60 con 0x7fa1b80d22d0 2024-04-13T03:11:22.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.243+0000 7fa19f7fe640 1 -- 172.21.15.106:0/3734410261 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7fa1ac05a360 con 0x7fa1b80d22d0 2024-04-13T03:11:22.244 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.245+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 msgr2=0x7fa1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.245+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 0x7fa1a406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8008040 comp rx=0 tx=0).stop 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.245+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 msgr2=0x7fa1b8152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.245+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00d620 tx=0x7fa1ac00dae0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 shutdown_connections 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1b80d2bf0 0x7fa1b815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1b80d22d0 0x7fa1b8152540 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 --2- 172.21.15.106:0/3734410261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1b80cf6f0 0x7fa1b814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:22.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 >> 172.21.15.106:0/3734410261 conn(0x7fa1b80d4700 msgr2=0x7fa1b80b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:22.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 shutdown_connections 2024-04-13T03:11:22.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22.246+0000 7fa1bd6de640 1 -- 172.21.15.106:0/3734410261 wait complete. 2024-04-13T03:11:22.257 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:11:22.257 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:11:22.257 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:11:22.258 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:11:22.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.258 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:11:22.273 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:11:22.273 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:11:22.273 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:11:22.273 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.273 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:11:22.274 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42467sec preferred_lft 42467sec 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:11:22.275 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:11:22.276 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:11:22.276 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:11:22.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.276 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:11:22.339 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:11:22.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.340 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:11:22.395 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:11:22.396 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:11:22.396 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:11:22.396 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:11:22.396 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:11:22.396 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:11:22.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:11:22.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:11:22.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.531 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:11:22.545 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:11:22.545 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:11:22.545 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:11:22.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.546 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:11:22.557 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:11:22.556+0000 7f85d99f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:11:22.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:11:22.612 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:11:22.667 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:11:22.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:11:22.668 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:11:22.722 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:11:22.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.722 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:11:22.807 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:11:22.807 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:11:22.818 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:11:22.819 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:11:22.819 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:11:22.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:11:22.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:11:22.847 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:11:22.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:11:22.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:11:22.948 INFO:teuthology.orchestra.run.smithi106.stdout:client5988 2024-04-13T03:11:22.948 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:22.948 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:22.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:22.964 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-13T03:11:23.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:23.024 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-13T03:11:23.081 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:11:23.082 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:11:23.082 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:11:23.082 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:11:23.082 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-13T03:11:23.082 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:11:23.186 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:11:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:11:23.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:23.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.335+0000 7f9f91836640 1 Processor -- start 2024-04-13T03:11:23.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.335+0000 7f9f91836640 1 -- start start 2024-04-13T03:11:23.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f91836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f8c071390 0x7f9f8c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f91836640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c15f420 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f91836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c15f700 con 0x7f9f8c15ed80 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f91836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c06ecc0 con 0x7f9f8c071390 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f8affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9f8c06e8e0 0x7f9f8c0cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40630/0 (socket says 172.21.15.106:40630) 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f8affd640 1 -- 172.21.15.106:0/442803000 learned_addr learned my addr 172.21.15.106:0/442803000 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850427725 0 0) 0x7f9f8c15f420 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f74003520 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/442803000 >> v2:172.21.15.114:3300/0 conn(0x7f9f8c071390 0x7f9f8c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862787649 0 0) 0x7f9f74003520 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.336+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 >> v2:172.21.15.114:3300/0 conn(0x7f9f8c071390 msgr2=0x7f9f8c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 --2- 172.21.15.106:0/442803000 >> v2:172.21.15.114:3300/0 conn(0x7f9f8c071390 0x7f9f8c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 >> v1:172.21.15.114:6789/0 conn(0x7f9f8c15ed80 legacy=0x7f9f8c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15f9e0 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9f7c0019f0 con 0x7f9f8c06e8e0 2024-04-13T03:11:23.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 >> v1:172.21.15.106:6789/0 conn(0x7f9f8c06e8e0 legacy=0x7f9f8c0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740039b0 0x7f9f74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f74004800 0x7f9f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 0x7f9f7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7400c0e0 con 0x7f9f740051c0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7400c290 con 0x7f9f740039b0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7400c490 con 0x7f9f74004800 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 0x7f9f7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8affd640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740039b0 0x7f9f74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8a7fc640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f74004800 0x7f9f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 -- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f74004800 msgr2=0x7f9f74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f74004800 0x7f9f74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 -- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740039b0 msgr2=0x7f9f74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740039b0 0x7f9f74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8b7fe640 1 -- 172.21.15.106:0/442803000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74003520 con 0x7f9f740051c0 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.337+0000 7f9f8affd640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740039b0 0x7f9f74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f8b7fe640 1 -- 172.21.15.106:0/442803000 reap_dead start 2024-04-13T03:11:23.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 0x7f9f7400b9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400ba10 tx=0x7f9f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7a76c2642e9ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84036020 con 0x7f9f740051c0 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84032680 con 0x7f9f740051c0 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f89ffb640 1 -- 172.21.15.106:0/442803000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400b610 con 0x7f9f740051c0 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f91836640 1 -- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 msgr2=0x7f9f7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f91836640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 0x7f9f7400b9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400ba10 tx=0x7f9f74005830 comp rx=0 tx=0).stop 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f91836640 1 -- 172.21.15.106:0/442803000 shutdown_connections 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f91836640 1 --2- 172.21.15.106:0/442803000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f740051c0 0x7f9f7400b9c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.338+0000 7f9f91836640 1 -- 172.21.15.106:0/442803000 >> 172.21.15.106:0/442803000 conn(0x7f9f8c06f180 msgr2=0x7f9f8c06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.339+0000 7f9f91836640 1 -- 172.21.15.106:0/442803000 shutdown_connections 2024-04-13T03:11:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.339+0000 7f9f91836640 1 -- 172.21.15.106:0/442803000 wait complete. 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.340+0000 7f9f91836640 1 Processor -- start 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.340+0000 7f9f91836640 1 -- start start 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.340+0000 7f9f91836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 0x7f9f8c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.340+0000 7f9f91836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.340+0000 7f9f91836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 0x7f9f8c0b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f91836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0707f0 con 0x7f9f8c14a890 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f91836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15f1e0 con 0x7f9f740051c0 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f91836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c0cd770 con 0x7f9f8c0b6700 2024-04-13T03:11:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45358/0 (socket says 172.21.15.106:45358) 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 -- 172.21.15.106:0/2088893121 learned_addr learned my addr 172.21.15.106:0/2088893121 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 0x7f9f8c0b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8affd640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 0x7f9f8c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 -- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 msgr2=0x7f9f8c0b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 0x7f9f8c0b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 -- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 msgr2=0x7f9f8c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 0x7f9f8c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 -- 172.21.15.106:0/2088893121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c14ae50 con 0x7f9f8c14a890 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8b7fe640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 0x7f9f8c0b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8affd640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 0x7f9f8c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f8a7fc640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000f5f0 tx=0x7f9f8000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800052e0 con 0x7f9f8c14a890 2024-04-13T03:11:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80018070 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80003830 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c14b130 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.341+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c14b6f0 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.342+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c068a10 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.343+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f800039d0 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.343+0000 7f9f73fff640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.343+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f9f8008b5d0 con 0x7f9f8c14a890 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.343+0000 7f9f8affd640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.344+0000 7f9f8affd640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06dde0 tx=0x7f9f7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.347+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f80059880 con 0x7f9f8c14a890 2024-04-13T03:11:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.558+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f8c0b22d0 con 0x7f9f8c14a890 2024-04-13T03:11:23.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.559+0000 7f9f73fff640 1 -- 172.21.15.106:0/2088893121 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f9f80059180 con 0x7f9f8c14a890 2024-04-13T03:11:23.560 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:23.560 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:23.560 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 msgr2=0x7f9f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06dde0 tx=0x7f9f7c009040 comp rx=0 tx=0).stop 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 msgr2=0x7f9f8c0b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000f5f0 tx=0x7f9f8000fab0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 shutdown_connections 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8c0b6700 0x7f9f8c0b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c14a890 0x7f9f8c0b4c70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 --2- 172.21.15.106:0/2088893121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f740051c0 0x7f9f8c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 >> 172.21.15.106:0/2088893121 conn(0x7f9f8c06f180 msgr2=0x7f9f8c0cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 shutdown_connections 2024-04-13T03:11:23.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.561+0000 7f9f91836640 1 -- 172.21.15.106:0/2088893121 wait complete. 2024-04-13T03:11:23.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-13T03:11:23.714 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:11:23.714 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:11:23.714 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-13T03:11:23.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 Processor -- start 2024-04-13T03:11:23.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 -- start start 2024-04-13T03:11:23.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc9cf10 0x559efdc98f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdcb4060 0x559efdc98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559efdc70ee0 con 0x559efdc70380 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559efdc15d00 con 0x559efdcb4060 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.757+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x559efdcb4440 con 0x559efdc9cf10 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45374/0 (socket says 172.21.15.106:45374) 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 -- 172.21.15.106:0/1065302568 learned_addr learned my addr 172.21.15.106:0/1065302568 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaa64a640 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc9cf10 0x559efdc98f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aab64c640 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdcb4060 0x559efdc98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 -- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc9cf10 msgr2=0x559efdc98f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc9cf10 0x559efdc98f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 -- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdcb4060 msgr2=0x559efdc98520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdcb4060 0x559efdc98520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 -- 172.21.15.106:0/1065302568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efdc9bf30 con 0x559efdc70380 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009a00 tx=0x7f0a9c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cf3000f3782127 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aa9648640 1 -- 172.21.15.106:0/1065302568 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0087f0 con 0x559efdc70380 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aa9648640 1 -- 172.21.15.106:0/1065302568 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c008990 con 0x559efdc70380 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.758+0000 7f0aa9648640 1 -- 172.21.15.106:0/1065302568 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c008d40 con 0x559efdc70380 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1065302568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 msgr2=0x559efdd3bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009a00 tx=0x7f0a9c005a00 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1065302568 shutdown_connections 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc9cf10 0x559efdc98f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdcb4060 0x559efdc98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1065302568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdc70380 0x559efdd3bab0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1065302568 >> 172.21.15.106:0/1065302568 conn(0x559efdc6df30 msgr2=0x559efdd45050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1065302568 shutdown_connections 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1065302568 wait complete. 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 Processor -- start 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.759+0000 7f0aad57ff40 1 -- start start 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc70380 0x559efdc70790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdc9cf10 0x559efdd3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 0x559efdc99100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559efdc6d870 con 0x559efdcb4060 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559efdcb3970 con 0x559efdc9cf10 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aad57ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x559efdcb2d60 con 0x559efdc70380 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 0x559efdc99100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 0x559efdc99100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45378/0 (socket says 172.21.15.106:45378) 2024-04-13T03:11:23.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 -- 172.21.15.106:0/1509054428 learned_addr learned my addr 172.21.15.106:0/1509054428 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc70380 0x559efdc70790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aaa64a640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdc9cf10 0x559efdd3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 -- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc70380 msgr2=0x559efdc70790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559efdc70380 0x559efdc70790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 -- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdc9cf10 msgr2=0x559efdd3cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559efdc9cf10 0x559efdd3cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 -- 172.21.15.106:0/1509054428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559efdcb2470 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.760+0000 7f0aab64c640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 0x559efdc99100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0aa401f360 tx=0x7f0aa40033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.761+0000 7f0a93fff640 1 -- 172.21.15.106:0/1509054428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa406a070 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.761+0000 7f0a93fff640 1 -- 172.21.15.106:0/1509054428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa40655f0 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.761+0000 7f0a93fff640 1 -- 172.21.15.106:0/1509054428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4005a80 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.761+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559efdcb2190 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.761+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559efdd3bec0 con 0x559efdcb4060 2024-04-13T03:11:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.762+0000 7f0a93fff640 1 -- 172.21.15.106:0/1509054428 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0aa406d070 con 0x559efdcb4060 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.762+0000 7f0a93fff640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0a9806b2d0 0x7f0a9806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.762+0000 7f0a93fff640 1 -- 172.21.15.106:0/1509054428 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0aa40cea90 con 0x559efdcb4060 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.762+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0a9806b2d0 0x7f0a9806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559efdc99640 0x559efdd40ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) -- 0x559efdd3eb00 con 0x559efdc99640 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aaa64a640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559efdc99640 0x559efdd40ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aaae4b640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0a9806b2d0 0x7f0a9806d790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005a40 tx=0x7f0a9c012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aaa64a640 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559efdc99640 0x559efdd40ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aaa64a640 1 -- 172.21.15.106:0/1509054428 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f0aa0009040 con 0x559efdc99640 2024-04-13T03:11:23.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559efdc99640 msgr2=0x559efdd40ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559efdc99640 0x559efdd40ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0a9806b2d0 msgr2=0x7f0a9806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0a9806b2d0 0x7f0a9806d790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005a40 tx=0x7f0a9c012000 comp rx=0 tx=0).stop 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 msgr2=0x559efdc99100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 --2- 172.21.15.106:0/1509054428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559efdcb4060 0x559efdc99100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0aa401f360 tx=0x7f0aa40033a0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aab64c640 1 -- 172.21.15.106:0/1509054428 reap_dead start 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 shutdown_connections 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 >> 172.21.15.106:0/1509054428 conn(0x559efdc6df30 msgr2=0x559efdc6d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 shutdown_connections 2024-04-13T03:11:23.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.763+0000 7f0aad57ff40 1 -- 172.21.15.106:0/1509054428 wait complete. 2024-04-13T03:11:23.771 INFO:teuthology.orchestra.run.smithi106.stdout:Yalphaparent_a 2024-04-13T03:11:23.771 DEBUG:teuthology.orchestra.run.smithi106:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-13T03:11:23.903 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:23.903 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "ancestors": [ 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1099511627776, 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "alpha", 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 2 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1, 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "parent_a", 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 4 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-13T03:11:23.904 INFO:teuthology.orchestra.run.smithi106.stdout: "pool": 8, 2024-04-13T03:11:23.905 INFO:teuthology.orchestra.run.smithi106.stdout: "old_pools": [] 2024-04-13T03:11:23.905 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:23.905 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:23.907 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-13T03:11:23.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.949+0000 7f3bfa237f40 1 Processor -- start 2024-04-13T03:11:23.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.949+0000 7f3bfa237f40 1 -- start start 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x557474879bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487eb90 0x55747487b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5574747a9180 con 0x55747487ef70 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55747474fd00 con 0x55747487eb90 2024-04-13T03:11:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5574747a9420 con 0x557474863910 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf7a4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf7a4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45380/0 (socket says 172.21.15.106:45380) 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/4142075306 learned_addr learned my addr 172.21.15.106:0/4142075306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf724b640 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x557474879bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3beea4a640 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487eb90 0x55747487b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 msgr2=0x557474879bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.950+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x557474879bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487eb90 msgr2=0x55747487b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487eb90 0x55747487b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/4142075306 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557474862f70 con 0x55747487ef70 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000a790 tx=0x7f3bf0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d22df75646f70a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf6249640 1 -- 172.21.15.106:0/4142075306 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000ae30 con 0x55747487ef70 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf6249640 1 -- 172.21.15.106:0/4142075306 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf000d9c0 con 0x55747487ef70 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bf6249640 1 -- 172.21.15.106:0/4142075306 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000ddb0 con 0x55747487ef70 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 -- 172.21.15.106:0/4142075306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 msgr2=0x55747487b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000a790 tx=0x7f3bf0005d80 comp rx=0 tx=0).stop 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 -- 172.21.15.106:0/4142075306 shutdown_connections 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487ef70 0x55747487b940 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487eb90 0x55747487b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/4142075306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x557474879bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 -- 172.21.15.106:0/4142075306 >> 172.21.15.106:0/4142075306 conn(0x5574747d2ae0 msgr2=0x5574747d2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 -- 172.21.15.106:0/4142075306 shutdown_connections 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.951+0000 7f3bfa237f40 1 -- 172.21.15.106:0/4142075306 wait complete. 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.952+0000 7f3bfa237f40 1 Processor -- start 2024-04-13T03:11:23.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.952+0000 7f3bfa237f40 1 -- start start 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x55747490ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 0x5574747d6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487ef70 0x5574747d6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5574748797d0 con 0x55747487eb90 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55747487a430 con 0x55747487ef70 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5574747d33d0 con 0x557474863910 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 0x5574747d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bf724b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x55747490ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 0x5574747d6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45390/0 (socket says 172.21.15.106:45390) 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 -- 172.21.15.106:0/1888507613 learned_addr learned my addr 172.21.15.106:0/1888507613 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487ef70 0x5574747d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 -- 172.21.15.106:0/1888507613 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 msgr2=0x55747490ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557474863910 0x55747490ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 -- 172.21.15.106:0/1888507613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487ef70 msgr2=0x5574747d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55747487ef70 0x5574747d6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 -- 172.21.15.106:0/1888507613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574747d3620 con 0x55747487eb90 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3beea4a640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 0x5574747d6030 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3be000c580 tx=0x7f3be000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bf5247640 1 -- 172.21.15.106:0/1888507613 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0008070 con 0x55747487eb90 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bf5247640 1 -- 172.21.15.106:0/1888507613 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be00106a0 con 0x55747487eb90 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55747490e9f0 con 0x55747487eb90 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bf5247640 1 -- 172.21.15.106:0/1888507613 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0010a90 con 0x55747487eb90 2024-04-13T03:11:23.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.953+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747490e6a0 con 0x55747487eb90 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bf5247640 1 -- 172.21.15.106:0/1888507613 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3be0010c30 con 0x55747487eb90 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bf5247640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3bd806b2d0 0x7f3bd806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bf724b640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3bd806b2d0 0x7f3bd806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bf5247640 1 -- 172.21.15.106:0/1888507613 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3be008b100 con 0x55747487eb90 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55747490cf40 0x55747485ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.955+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x55747485d670 con 0x55747490cf40 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.956+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55747490cf40 0x55747485ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.956+0000 7f3bf724b640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3bd806b2d0 0x7f3bd806d790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3be8057000 tx=0x7f3be8052000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.956+0000 7f3bf7a4c640 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55747490cf40 0x55747485ffa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/1888507613 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v52'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bf0019020 con 0x55747490cf40 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55747490cf40 msgr2=0x55747485ffa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55747490cf40 0x55747485ffa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3bd806b2d0 msgr2=0x7f3bd806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3bd806b2d0 0x7f3bd806d790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3be8057000 tx=0x7f3be8052000 comp rx=0 tx=0).stop 2024-04-13T03:11:23.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 msgr2=0x5574747d6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 --2- 172.21.15.106:0/1888507613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55747487eb90 0x5574747d6030 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3be000c580 tx=0x7f3be000c980 comp rx=0 tx=0).stop 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bf7a4c640 1 -- 172.21.15.106:0/1888507613 reap_dead start 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 shutdown_connections 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 >> 172.21.15.106:0/1888507613 conn(0x5574747d2ae0 msgr2=0x557474879bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 shutdown_connections 2024-04-13T03:11:23.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:23.957+0000 7f3bfa237f40 1 -- 172.21.15.106:0/1888507613 wait complete. 2024-04-13T03:11:23.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-13T03:11:24.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:24.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-13T03:11:24.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.109+0000 7f3c45b95640 1 Processor -- start 2024-04-13T03:11:24.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.109+0000 7f3c45b95640 1 -- start start 2024-04-13T03:11:24.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c45b95640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3c400d1b90 0x7f3c400ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c45b95640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 0x7f3c400cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c45b95640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c400b2020 con 0x7f3c4015c6f0 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c45b95640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3c400b2c30 con 0x7f3c400d1b90 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c45b95640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3c400b2300 con 0x7f3c400d2020 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c3effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3c400d1b90 0x7f3c400ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c3f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3c4015c6f0 0x7f3c400b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40644/0 (socket says 172.21.15.106:40644) 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.110+0000 7f3c3f7fe640 1 -- 172.21.15.106:0/1154301819 learned_addr learned my addr 172.21.15.106:0/1154301819 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/1154301819 >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 0x7f3c400cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3effd640 1 -- 172.21.15.106:0/1154301819 >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 msgr2=0x7f3c400cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3effd640 1 --2- 172.21.15.106:0/1154301819 >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 0x7f3c400cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3effd640 1 -- 172.21.15.106:0/1154301819 >> v1:172.21.15.106:6789/0 conn(0x7f3c4015c6f0 legacy=0x7f3c400b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3effd640 1 -- 172.21.15.106:0/1154301819 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c400b25e0 con 0x7f3c400d1b90 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/1154301819 >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 0x7f3c400cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:24.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3e7fc640 1 -- 172.21.15.106:0/1154301819 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797024566 0 0) 0x7f3c400b2020 con 0x7f3c4015c6f0 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.111+0000 7f3c3effd640 1 --2- 172.21.15.106:0/1154301819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d1b90 0x7f3c400ccbc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ce2b0 tx=0x7f3c3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f2a222df1c28f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c3e7fc640 1 -- 172.21.15.106:0/1154301819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34035020 con 0x7f3c400d1b90 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c3e7fc640 1 -- 172.21.15.106:0/1154301819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c34031830 con 0x7f3c400d1b90 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c3e7fc640 1 -- 172.21.15.106:0/1154301819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34031c00 con 0x7f3c400d1b90 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 -- 172.21.15.106:0/1154301819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d1b90 msgr2=0x7f3c400ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 --2- 172.21.15.106:0/1154301819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d1b90 0x7f3c400ccbc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ce2b0 tx=0x7f3c3402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 -- 172.21.15.106:0/1154301819 shutdown_connections 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 --2- 172.21.15.106:0/1154301819 >> v2:172.21.15.114:3300/0 conn(0x7f3c400d2020 0x7f3c400cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 --2- 172.21.15.106:0/1154301819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d1b90 0x7f3c400ccbc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 -- 172.21.15.106:0/1154301819 >> 172.21.15.106:0/1154301819 conn(0x7f3c400b4060 msgr2=0x7f3c400b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 -- 172.21.15.106:0/1154301819 shutdown_connections 2024-04-13T03:11:24.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.112+0000 7f3c45b95640 1 -- 172.21.15.106:0/1154301819 wait complete. 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.113+0000 7f3c45b95640 1 Processor -- start 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 -- start start 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c400d1b90 0x7f3c400a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c4015c6f0 0x7f3c40148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4015cad0 con 0x7f3c400d2020 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40160ff0 con 0x7f3c4015c6f0 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c45b95640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4015eb50 con 0x7f3c400d1b90 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45404/0 (socket says 172.21.15.106:45404) 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.114+0000 7f3c3effd640 1 -- 172.21.15.106:0/2743052633 learned_addr learned my addr 172.21.15.106:0/2743052633 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c4015c6f0 0x7f3c40148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c400d1b90 0x7f3c400a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 -- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c400d1b90 msgr2=0x7f3c400a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c400d1b90 0x7f3c400a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 -- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c4015c6f0 msgr2=0x7f3c40148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c4015c6f0 0x7f3c40148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 -- 172.21.15.106:0/2743052633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4015df60 con 0x7f3c400d2020 2024-04-13T03:11:24.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009890 tx=0x7f3c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c27fff640 1 -- 172.21.15.106:0/2743052633 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34041070 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c27fff640 1 -- 172.21.15.106:0/2743052633 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3403aa30 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4015e220 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c27fff640 1 -- 172.21.15.106:0/2743052633 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34035040 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.115+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4015e7e0 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.117+0000 7f3c27fff640 1 -- 172.21.15.106:0/2743052633 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c34038020 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.117+0000 7f3c27fff640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 0x7f3c2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.117+0000 7f3c27fff640 1 -- 172.21.15.106:0/2743052633 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3c340ad690 con 0x7f3c400d2020 2024-04-13T03:11:24.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.117+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 0x7f3c2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.118+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 0x7f3c2c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000f60 tx=0x7f3c30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.118+0000 7f3c48977740 1 Processor -- start 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.118+0000 7f3c48977740 1 -- start start 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f4f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f29120 0x5619f9f282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619f9e8d0a0 con 0x5619f9f4fdb0 2024-04-13T03:11:24.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5619f9e33930 con 0x5619f9f28d40 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5619f9e89200 con 0x5619f9f29120 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45406/0 (socket says 172.21.15.106:45406) 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 -- 172.21.15.106:0/1707908450 learned_addr learned my addr 172.21.15.106:0/1707908450 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f29120 0x5619f9f282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f4f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 -- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f29120 msgr2=0x5619f9f282c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f29120 0x5619f9f282c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 -- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 msgr2=0x5619f9f4f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f4f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 -- 172.21.15.106:0/1707908450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f9c677e0 con 0x5619f9f4fdb0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.119+0000 7f3c3effd640 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3c340838e0 tx=0x7f3c34031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a0d3c1b6dc96aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c257fa640 1 -- 172.21.15.106:0/1707908450 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34083e10 con 0x5619f9f4fdb0 2024-04-13T03:11:24.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c257fa640 1 -- 172.21.15.106:0/1707908450 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c340abe50 con 0x5619f9f4fdb0 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c257fa640 1 -- 172.21.15.106:0/1707908450 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34079530 con 0x5619f9f4fdb0 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 -- 172.21.15.106:0/1707908450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 msgr2=0x5619f9f27d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3c340838e0 tx=0x7f3c34031f90 comp rx=0 tx=0).stop 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 -- 172.21.15.106:0/1707908450 shutdown_connections 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f29120 0x5619f9f282c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f4fdb0 0x5619f9f27d80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 --2- 172.21.15.106:0/1707908450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f4f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.120+0000 7f3c48977740 1 -- 172.21.15.106:0/1707908450 >> 172.21.15.106:0/1707908450 conn(0x5619f9eef6e0 msgr2=0x5619f9f6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.121+0000 7f3c48977740 1 -- 172.21.15.106:0/1707908450 shutdown_connections 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.121+0000 7f3c48977740 1 -- 172.21.15.106:0/1707908450 wait complete. 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.121+0000 7f3c48977740 1 -- start start 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.121+0000 7f3c48977740 1 Processor -- start 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f28d40 0x5619f9f46370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 0x5619f9f660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619f9e33930 con 0x5619f9f66600 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5619f9e46cf0 con 0x5619f9f29120 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5619f9e8d0a0 con 0x5619f9f28d40 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c48977740 10 client.0 _ll_register_callbacks cb 0x5619f9e466e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45410/0 (socket says 172.21.15.106:45410) 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 -- 172.21.15.106:0/2428072056 learned_addr learned my addr 172.21.15.106:0/2428072056 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f28d40 0x5619f9f46370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 0x5619f9f660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 -- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f28d40 msgr2=0x5619f9f46370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f28d40 0x5619f9f46370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 -- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 msgr2=0x5619f9f660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 0x5619f9f660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 -- 172.21.15.106:0/2428072056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f9cc7f70 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 0x5619f9f660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c38007b00 tx=0x7f3c38011230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afbe83cd4a90806 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.122+0000 7f3c1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801b020 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3c38017460 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c340abe50 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3c38025050 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 10 client.6024 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5619f9b5f170 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c340abe50 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f3c38028020 con 0x5619f9f66600 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 10 client.6024 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:24.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 10 client.6024 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 20 client.6024 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9f8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 4 client.6024 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619f9e9aac0 con 0x5619f9f88c90 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.123+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9f8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c1e7fc640 10 client.6024 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9f8b080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e870c258e41665d server_cookie=e01869e1c540b7af in_seq=0 out_seq=0 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2428072056 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619f9e9aac0 con 0x5619f9f88c90 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c1e7fc640 10 client.6024 handle_command_reply: tid=0 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 client.6024 shutdown 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 msgr2=0x5619f9f54040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c38007b00 tx=0x7f3c38011230 comp rx=0 tx=0).stop 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 shutdown_connections 2024-04-13T03:11:24.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9f8b080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c1e7fc640 0 client.6024 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.124+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f66600 0x5619f9f54040 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f29120 0x5619f9f660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 1 --2- 172.21.15.106:0/2428072056 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f28d40 0x5619f9f46370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 >> 172.21.15.106:0/2428072056 conn(0x5619f9eef6e0 msgr2=0x5619f9f52a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 shutdown_connections 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 1 -- 172.21.15.106:0/2428072056 wait complete. 2024-04-13T03:11:24.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.125+0000 7f3c48977740 20 client.6024 trim_cache size 0 max 16384 2024-04-13T03:11:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.129+0000 7f3dd0b08640 1 Processor -- start 2024-04-13T03:11:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.129+0000 7f3dd0b08640 1 -- start start 2024-04-13T03:11:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dd0b08640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3dcc15eaf0 0x7f3dcc0d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dd0b08640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc0d3270 con 0x7f3dcc162490 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dd0b08640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc162f30 con 0x7f3dcc0d4dd0 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dd0b08640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc0d28a0 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dc9d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3dcc162490 0x7f3dcc0d46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40654/0 (socket says 172.21.15.106:40654) 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dc9d9b640 1 -- 172.21.15.106:0/3937230952 learned_addr learned my addr 172.21.15.106:0/3937230952 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.130+0000 7f3dca59c640 1 --2- 172.21.15.106:0/3937230952 >> v2:172.21.15.114:3300/0 conn(0x7f3dcc15eaf0 0x7f3dcc0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20760208 0 0) 0x7f3dcc0d3270 con 0x7f3dcc162490 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3db4003520 con 0x7f3dcc162490 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617000547 0 0) 0x7f3dcc162f30 con 0x7f3dcc0d4dd0 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dcc0d3270 con 0x7f3dcc0d4dd0 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dca59c640 1 -- 172.21.15.106:0/3937230952 >> v1:172.21.15.114:6789/0 conn(0x7f3dcc0d4dd0 legacy=0x7f3dcc0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2196523268 0 0) 0x7f3db4003520 con 0x7f3dcc162490 2024-04-13T03:11:24.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dca59c640 1 -- 172.21.15.106:0/3937230952 >> v1:172.21.15.106:6789/0 conn(0x7f3dcc162490 legacy=0x7f3dcc0d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.131+0000 7f3dca59c640 1 -- 172.21.15.106:0/3937230952 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc163210 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.132+0000 7f3dca59c640 1 --2- 172.21.15.106:0/3937230952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc0d2d30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009790 tx=0x7f3dbc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5681c27f680ad6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.132+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc038070 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.132+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc031500 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.132+0000 7f3dc959a640 1 -- 172.21.15.106:0/3937230952 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00017f0 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 -- 172.21.15.106:0/3937230952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 msgr2=0x7f3dcc0d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 --2- 172.21.15.106:0/3937230952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc0d2d30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009790 tx=0x7f3dbc02b820 comp rx=0 tx=0).stop 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 -- 172.21.15.106:0/3937230952 shutdown_connections 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 --2- 172.21.15.106:0/3937230952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc0d2d30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 -- 172.21.15.106:0/3937230952 >> 172.21.15.106:0/3937230952 conn(0x7f3dcc08bf10 msgr2=0x7f3dcc0b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 -- 172.21.15.106:0/3937230952 shutdown_connections 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.133+0000 7f3dd0b08640 1 -- 172.21.15.106:0/3937230952 wait complete. 2024-04-13T03:11:24.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.134+0000 7f3dd0b08640 1 Processor -- start 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.134+0000 7f3dd0b08640 1 -- start start 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3dcc0d4dd0 0x7f3dcc0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dcc162490 0x7f3dcc161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc06ef60 con 0x7f3dcc162490 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0d2530 con 0x7f3dcc15eaf0 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dd0b08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc06f0e0 con 0x7f3dcc0d4dd0 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dcc162490 0x7f3dcc161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dcc162490 0x7f3dcc161130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45416/0 (socket says 172.21.15.106:45416) 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 -- 172.21.15.106:0/2141846133 learned_addr learned my addr 172.21.15.106:0/2141846133 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dc9d9b640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dca59c640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3dcc0d4dd0 0x7f3dcc0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 -- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3dcc0d4dd0 msgr2=0x7f3dcc0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3dcc0d4dd0 0x7f3dcc0b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 -- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 msgr2=0x7f3dcc1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 -- 172.21.15.106:0/2141846133 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0b4770 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dc9d9b640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3dcc15eaf0 0x7f3dcc1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dca59c640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3dcc0d4dd0 0x7f3dcc0b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.135+0000 7f3dcad9d640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dcc162490 0x7f3dcc161130 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c730 tx=0x7f3dc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4003b70 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4003d10 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dd0b08640 1 -- 172.21.15.106:0/2141846133 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc0b4a50 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400aa90 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dd0b08640 1 -- 172.21.15.106:0/2141846133 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc06b8e0 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.136+0000 7f3dd0b08640 1 -- 172.21.15.106:0/2141846133 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3dcc068a10 con 0x7f3dcc162490 2024-04-13T03:11:24.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.137+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dc400ac30 con 0x7f3dcc162490 2024-04-13T03:11:24.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dbaffd640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dca59c640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3dc408c670 con 0x7f3dcc162490 2024-04-13T03:11:24.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f3dc408cb20 con 0x7f3dcc162490 2024-04-13T03:11:24.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dd0b08640 1 -- 172.21.15.106:0/2141846133 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f3dcc0b9c40 con 0x7f3dcc162490 2024-04-13T03:11:24.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.138+0000 7f3dca59c640 1 --2- 172.21.15.106:0/2141846133 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3dac06c8f0 0x7f3dac06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc035000 tx=0x7f3dbc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.144+0000 7f3c48977740 1 Processor -- start 2024-04-13T03:11:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.144+0000 7f3c48977740 1 -- start start 2024-04-13T03:11:24.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f52f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401ff2a0 con 0x5619f9f29120 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401fe2a0 con 0x5619f9f28d40 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3c401f62c0 con 0x5619f9f66600 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45418/0 (socket says 172.21.15.106:45418) 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c3effd640 1 -- 172.21.15.106:0/2355685638 learned_addr learned my addr 172.21.15.106:0/2355685638 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.145+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f52f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 -- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 msgr2=0x5619f9f52f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f52f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 -- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 msgr2=0x5619f9f56a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f56a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 -- 172.21.15.106:0/2355685638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f9f242d0 con 0x5619f9f29120 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3c340aba80 tx=0x7f3c3403d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c12f4f6f34af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c24ff9640 1 -- 172.21.15.106:0/2355685638 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34083e10 con 0x5619f9f29120 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c24ff9640 1 -- 172.21.15.106:0/2355685638 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c340abe50 con 0x5619f9f29120 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.146+0000 7f3c24ff9640 1 -- 172.21.15.106:0/2355685638 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34079530 con 0x5619f9f29120 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- 172.21.15.106:0/2355685638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 msgr2=0x5619f9f52a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3c340aba80 tx=0x7f3c3403d070 comp rx=0 tx=0).stop 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- 172.21.15.106:0/2355685638 shutdown_connections 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f52f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f52a50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 --2- 172.21.15.106:0/2355685638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f56a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- 172.21.15.106:0/2355685638 >> 172.21.15.106:0/2355685638 conn(0x5619f9eef6e0 msgr2=0x5619f9f21a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- 172.21.15.106:0/2355685638 shutdown_connections 2024-04-13T03:11:24.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- 172.21.15.106:0/2355685638 wait complete. 2024-04-13T03:11:24.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 -- start start 2024-04-13T03:11:24.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.147+0000 7f3c48977740 1 Processor -- start 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f4c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619f9f50240 con 0x5619f9f29120 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401fdc80 con 0x5619f9f28d40 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5619f9f56120 con 0x5619f9f66600 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c48977740 10 client.0 _ll_register_callbacks cb 0x7f3c380073c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45422/0 (socket says 172.21.15.106:45422) 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c3effd640 1 -- 172.21.15.106:0/2629349875 learned_addr learned my addr 172.21.15.106:0/2629349875 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.148+0000 7f3c3ffff640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f4c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 -- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 msgr2=0x5619f9f4c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f4c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 -- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 msgr2=0x5619f9f90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f90500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 -- 172.21.15.106:0/2629349875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f9f90e80 con 0x5619f9f29120 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3effd640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c34000c00 tx=0x7f3c34006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c583bda784b51f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34099060 con 0x5619f9f29120 2024-04-13T03:11:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3c340778a0 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c340abe50 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3c34083e10 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 10 client.6036 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5619f9f90b00 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c340abe50 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f3c3407f120 con 0x5619f9f29120 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 10 client.6036 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 10 client.6036 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 20 client.6036 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9fe6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 4 client.6036 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5619f9e94930 con 0x5619f9f88c90 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.149+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9fe6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.150+0000 7f3c1e7fc640 10 client.6036 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.150+0000 7f3c3f7fe640 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9fe6b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325b84f96380d393 server_cookie=594dd9bea6ef106a in_seq=0 out_seq=0 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.150+0000 7f3c1e7fc640 1 -- 172.21.15.106:0/2629349875 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5619f9e94930 con 0x5619f9f88c90 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.150+0000 7f3c1e7fc640 10 client.6036 handle_command_reply: tid=0 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.150+0000 7f3c48977740 1 client.6036 shutdown 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 msgr2=0x5619f9f4c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c34000c00 tx=0x7f3c34006f40 comp rx=0 tx=0).stop 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 shutdown_connections 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5619f9f88c90 0x5619f9fe6b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5619f9f66600 0x5619f9f4c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619f9f29120 0x5619f9f4c380 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c1e7fc640 0 client.6036 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 --2- 172.21.15.106:0/2629349875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619f9f28d40 0x5619f9f90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 >> 172.21.15.106:0/2629349875 conn(0x5619f9eef6e0 msgr2=0x5619f9f457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 shutdown_connections 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 1 -- 172.21.15.106:0/2629349875 wait complete. 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c48977740 20 client.6036 trim_cache size 0 max 16384 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "fd0050df-ccd7-4843-82a6-8a62d21474d0", 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:11:24.159 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 msgr2=0x7f3c2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 0x7f3c2c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000f60 tx=0x7f3c30009040 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 msgr2=0x7f3c401f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009890 tx=0x7f3c340046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 shutdown_connections 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c2c06c8f0 0x7f3c2c06edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c4015c6f0 0x7f3c40148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c400d2020 0x7f3c401f29f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 --2- 172.21.15.106:0/2743052633 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c400d1b90 0x7f3c400a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 >> 172.21.15.106:0/2743052633 conn(0x7f3c400b4060 msgr2=0x7f3c400b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 shutdown_connections 2024-04-13T03:11:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.151+0000 7f3c45b95640 1 -- 172.21.15.106:0/2743052633 wait complete. 2024-04-13T03:11:24.161 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:11:24.150+0000 7f85d59e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5988=0-4194304@1} caps={5988=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5988 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55c48d954680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:11:24.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:24.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-13T03:11:24.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.264+0000 7fcb88636640 1 Processor -- start 2024-04-13T03:11:24.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.264+0000 7fcb88636640 1 -- start start 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb88636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb800cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb88636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb80072d40 con 0x7fcb8015ec10 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb88636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb80073020 con 0x7fcb800b42e0 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb88636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb80073300 con 0x7fcb80070c00 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb863d2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcb8015ec10 0x7fcb800704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40656/0 (socket says 172.21.15.106:40656) 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb863d2640 1 -- 172.21.15.106:0/3980329397 learned_addr learned my addr 172.21.15.106:0/3980329397 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.265+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/3980329397 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb800cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817936491 0 0) 0x7fcb80072d40 con 0x7fcb8015ec10 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb6c003520 con 0x7fcb8015ec10 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 803340659 0 0) 0x7fcb80073020 con 0x7fcb800b42e0 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb80072d40 con 0x7fcb800b42e0 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 557672003 0 0) 0x7fcb6c003520 con 0x7fcb8015ec10 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb85bd1640 1 -- 172.21.15.106:0/3980329397 >> v1:172.21.15.114:6789/0 conn(0x7fcb800b42e0 legacy=0x7fcb800cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb85bd1640 1 -- 172.21.15.106:0/3980329397 >> v1:172.21.15.106:6789/0 conn(0x7fcb8015ec10 legacy=0x7fcb800704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.266+0000 7fcb85bd1640 1 -- 172.21.15.106:0/3980329397 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb800735e0 con 0x7fcb80070c00 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/3980329397 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb800cf9b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb700097d0 tx=0x7fcb7002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2e023e2e7520cd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70037070 con 0x7fcb80070c00 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70030510 con 0x7fcb80070c00 2024-04-13T03:11:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb853d0640 1 -- 172.21.15.106:0/3980329397 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0019f0 con 0x7fcb80070c00 2024-04-13T03:11:24.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 -- 172.21.15.106:0/3980329397 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 msgr2=0x7fcb800cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 --2- 172.21.15.106:0/3980329397 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb800cf9b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb700097d0 tx=0x7fcb7002b800 comp rx=0 tx=0).stop 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 -- 172.21.15.106:0/3980329397 shutdown_connections 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 --2- 172.21.15.106:0/3980329397 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb800cf9b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 -- 172.21.15.106:0/3980329397 >> 172.21.15.106:0/3980329397 conn(0x7fcb800d13f0 msgr2=0x7fcb800d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 -- 172.21.15.106:0/3980329397 shutdown_connections 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.267+0000 7fcb88636640 1 -- 172.21.15.106:0/3980329397 wait complete. 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.268+0000 7fcb88636640 1 Processor -- start 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 -- start start 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb80143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb800b42e0 0x7fcb80157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8006e450 con 0x7fcb8015ec10 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800cdc20 con 0x7fcb800b42e0 2024-04-13T03:11:24.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb88636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8006e5d0 con 0x7fcb80070c00 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb86bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb86bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45430/0 (socket says 172.21.15.106:45430) 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb86bd3640 1 -- 172.21.15.106:0/2519664182 learned_addr learned my addr 172.21.15.106:0/2519664182 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb863d2640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb80143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.269+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb800b42e0 0x7fcb80157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 -- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 msgr2=0x7fcb80143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb80143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 -- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb800b42e0 msgr2=0x7fcb80157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb800b42e0 0x7fcb80157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 -- 172.21.15.106:0/2519664182 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb80142cf0 con 0x7fcb8015ec10 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c400 tx=0x7fcb7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb76ffd640 1 -- 172.21.15.106:0/2519664182 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78017070 con 0x7fcb8015ec10 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb76ffd640 1 -- 172.21.15.106:0/2519664182 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb780052d0 con 0x7fcb8015ec10 2024-04-13T03:11:24.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb80156cc0 con 0x7fcb8015ec10 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb76ffd640 1 -- 172.21.15.106:0/2519664182 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb780037f0 con 0x7fcb8015ec10 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.270+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb800aed30 con 0x7fcb8015ec10 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.271+0000 7fcb76ffd640 1 -- 172.21.15.106:0/2519664182 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb7801a070 con 0x7fcb8015ec10 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.272+0000 7fcb76ffd640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 0x7fcb6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.272+0000 7fcb76ffd640 1 -- 172.21.15.106:0/2519664182 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fcb7808af10 con 0x7fcb8015ec10 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.272+0000 7fcb863d2640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 0x7fcb6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.272+0000 7fcb863d2640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 0x7fcb6406edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001560 tx=0x7fcb7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.273+0000 7fcb8b431740 1 Processor -- start 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.273+0000 7fcb8b431740 1 -- start start 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3e8af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3e8b4d0 0x5620d3ebb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5620d3df6170 con 0x5620d3eb83d0 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5620d3d9c7f0 con 0x5620d3eb7ff0 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5620d3df2130 con 0x5620d3e8b4d0 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45440/0 (socket says 172.21.15.106:45440) 2024-04-13T03:11:24.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 -- 172.21.15.106:0/414715488 learned_addr learned my addr 172.21.15.106:0/414715488 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb863d2640 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3e8af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3e8b4d0 0x5620d3ebb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 -- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3e8b4d0 msgr2=0x5620d3ebb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3e8b4d0 0x5620d3ebb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 -- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 msgr2=0x5620d3e8af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3e8af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 -- 172.21.15.106:0/414715488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620d3bd27d0 con 0x5620d3eb83d0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009720 tx=0x7fcb7002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a70cf8d602652 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb63fff640 1 -- 172.21.15.106:0/414715488 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70037040 con 0x5620d3eb83d0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb63fff640 1 -- 172.21.15.106:0/414715488 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70004380 con 0x5620d3eb83d0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.274+0000 7fcb63fff640 1 -- 172.21.15.106:0/414715488 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003850 con 0x5620d3eb83d0 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 -- 172.21.15.106:0/414715488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 msgr2=0x5620d3ebad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009720 tx=0x7fcb7002f000 comp rx=0 tx=0).stop 2024-04-13T03:11:24.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 -- 172.21.15.106:0/414715488 shutdown_connections 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3e8b4d0 0x5620d3ebb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb83d0 0x5620d3ebad80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 --2- 172.21.15.106:0/414715488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3e8af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 -- 172.21.15.106:0/414715488 >> 172.21.15.106:0/414715488 conn(0x5620d3e8d200 msgr2=0x5620d3ed5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 -- 172.21.15.106:0/414715488 shutdown_connections 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.275+0000 7fcb8b431740 1 -- 172.21.15.106:0/414715488 wait complete. 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.276+0000 7fcb8b431740 1 -- start start 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.276+0000 7fcb8b431740 1 Processor -- start 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3ee5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3eb9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5620d3e8f8a0 con 0x5620d3eb03f0 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5620d3dafd00 con 0x5620d3eb7ff0 2024-04-13T03:11:24.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5620d3e920e0 con 0x5620d3eb83d0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb8b431740 10 client.0 _ll_register_callbacks cb 0x5620d3daf6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb863d2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3ee5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45450/0 (socket says 172.21.15.106:45450) 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 -- 172.21.15.106:0/229010336 learned_addr learned my addr 172.21.15.106:0/229010336 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3eb9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 -- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 msgr2=0x5620d3eb9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3eb9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 -- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 msgr2=0x5620d3ee5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3ee5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 -- 172.21.15.106:0/229010336 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620d3b083f0 con 0x5620d3eb03f0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb863d2640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3ee5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcb780892e0 tx=0x7fcb78005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0eb4deacd23b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb617fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.277+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78087060 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fcb78058040 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb70004380 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb78071070 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 10 client.6048 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5620d3ac80f0 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70004380 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fcb7805f0f0 con 0x5620d3eb03f0 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 10 client.6048 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 10 client.6048 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 20 client.6048 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebd390 0x5620d3ebd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 4 client.6048 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:11:24.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620d3e03b90 con 0x5620d3ebd390 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb863d2640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebd390 0x5620d3ebd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb617fa640 10 client.6048 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.278+0000 7fcb863d2640 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebd390 0x5620d3ebd790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ae9cc19cb30ea6 server_cookie=4dbcc9cd13d79298 in_seq=0 out_seq=0 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb617fa640 1 -- 172.21.15.106:0/229010336 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620d3e03b90 con 0x5620d3ebd390 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb617fa640 10 client.6048 handle_command_reply: tid=0 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 client.6048 shutdown 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 msgr2=0x5620d3ed41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcb780892e0 tx=0x7fcb78005540 comp rx=0 tx=0).stop 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 shutdown_connections 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebd390 0x5620d3ebd790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3eb9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3ed41b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb617fa640 0 client.6048 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 --2- 172.21.15.106:0/229010336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3ee5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 >> 172.21.15.106:0/229010336 conn(0x5620d3e8b4d0 msgr2=0x5620d3e90f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 shutdown_connections 2024-04-13T03:11:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 1 -- 172.21.15.106:0/229010336 wait complete. 2024-04-13T03:11:24.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.279+0000 7fcb8b431740 20 client.6048 trim_cache size 0 max 16384 2024-04-13T03:11:24.289 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:11:24.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:11:24.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.298+0000 7fcb8b431740 1 Processor -- start 2024-04-13T03:11:24.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.298+0000 7fcb8b431740 1 -- start start 2024-04-13T03:11:24.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb03f0 0x5620d3ebc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb83d0 0x5620d3eb87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb802013d0 con 0x5620d3eb7ff0 2024-04-13T03:11:24.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb802004a0 con 0x5620d3eb83d0 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb800b0160 con 0x5620d3eb03f0 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45456/0 (socket says 172.21.15.106:45456) 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 -- 172.21.15.106:0/4216943848 learned_addr learned my addr 172.21.15.106:0/4216943848 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb863d2640 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb03f0 0x5620d3ebc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb83d0 0x5620d3eb87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 -- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb03f0 msgr2=0x5620d3ebc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb03f0 0x5620d3ebc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 -- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb83d0 msgr2=0x5620d3eb87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb83d0 0x5620d3eb87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.299+0000 7fcb85bd1640 1 -- 172.21.15.106:0/4216943848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620d3e86cf0 con 0x5620d3eb7ff0 2024-04-13T03:11:24.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098f0 tx=0x7fcb70030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf223fc926bfd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb637fe640 1 -- 172.21.15.106:0/4216943848 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70037040 con 0x5620d3eb7ff0 2024-04-13T03:11:24.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb637fe640 1 -- 172.21.15.106:0/4216943848 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70004440 con 0x5620d3eb7ff0 2024-04-13T03:11:24.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb637fe640 1 -- 172.21.15.106:0/4216943848 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003850 con 0x5620d3eb7ff0 2024-04-13T03:11:24.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 -- 172.21.15.106:0/4216943848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 msgr2=0x5620d3ed9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098f0 tx=0x7fcb70030000 comp rx=0 tx=0).stop 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 -- 172.21.15.106:0/4216943848 shutdown_connections 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb83d0 0x5620d3eb87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb7ff0 0x5620d3ed9460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 --2- 172.21.15.106:0/4216943848 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb03f0 0x5620d3ebc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.300+0000 7fcb8b431740 1 -- 172.21.15.106:0/4216943848 >> 172.21.15.106:0/4216943848 conn(0x5620d3e8b4d0 msgr2=0x5620d3ee5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.301+0000 7fcb8b431740 1 -- 172.21.15.106:0/4216943848 shutdown_connections 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.301+0000 7fcb8b431740 1 -- 172.21.15.106:0/4216943848 wait complete. 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.301+0000 7fcb8b431740 1 -- start start 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.301+0000 7fcb8b431740 1 Processor -- start 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3eac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3e8f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80159730 con 0x5620d3eb03f0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5620d3ebe370 con 0x5620d3eb7ff0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5620d3e90830 con 0x5620d3eb83d0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb8b431740 10 client.0 _ll_register_callbacks cb 0x7fcb78058800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45462/0 (socket says 172.21.15.106:45462) 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 -- 172.21.15.106:0/1314954478 learned_addr learned my addr 172.21.15.106:0/1314954478 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb86bd3640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3e8f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3eac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 -- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 msgr2=0x5620d3e8f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3e8f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 -- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 msgr2=0x5620d3eac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3eac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.302+0000 7fcb863d2640 1 -- 172.21.15.106:0/1314954478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620d3eb6230 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb863d2640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f80 tx=0x7fcb7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89954b652b1bea server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c014d50 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fcb7c023920 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb70004440 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c039050 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 10 client.6054 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5620d3eabfd0 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70004440 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fcb7c03c020 con 0x5620d3eb03f0 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 10 client.6054 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 20 client.6054 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:24.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebc820 0x5620d3ed3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 4 client.6054 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620d3dfda00 con 0x5620d3ebc820 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.303+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebc820 0x5620d3ed3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb617fa640 10 client.6054 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb85bd1640 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebc820 0x5620d3ed3930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa11c3816047a86 server_cookie=9a5448afbd917cb9 in_seq=0 out_seq=0 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb617fa640 1 -- 172.21.15.106:0/1314954478 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5620d3dfda00 con 0x5620d3ebc820 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb617fa640 10 client.6054 handle_command_reply: tid=0 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 client.6054 shutdown 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 msgr2=0x5620d3f4fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f80 tx=0x7fcb7c012000 comp rx=0 tx=0).stop 2024-04-13T03:11:24.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 shutdown_connections 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x5620d3ebc820 0x5620d3ed3930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5620d3eb83d0 0x5620d3e8f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d3eb7ff0 0x5620d3eac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb617fa640 0 client.6054 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 --2- 172.21.15.106:0/1314954478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620d3eb03f0 0x5620d3f4fde0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 >> 172.21.15.106:0/1314954478 conn(0x5620d3e8b4d0 msgr2=0x5620d3ed9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 shutdown_connections 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 1 -- 172.21.15.106:0/1314954478 wait complete. 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.304+0000 7fcb8b431740 20 client.6054 trim_cache size 0 max 16384 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:24.313 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: "fd0050df-ccd7-4843-82a6-8a62d21474d0": { 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/", 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "fd0050df-ccd7-4843-82a6-8a62d21474d0", 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:24.314 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 msgr2=0x7fcb6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 0x7fcb6406edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001560 tx=0x7fcb7c009040 comp rx=0 tx=0).stop 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 msgr2=0x7fcb801427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c400 tx=0x7fcb7800c8c0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 shutdown_connections 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb6406c8f0 0x7fcb6406edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb8015ec10 0x7fcb801427b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb800b42e0 0x7fcb80157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 --2- 172.21.15.106:0/2519664182 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb80070c00 0x7fcb80143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:24.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 >> 172.21.15.106:0/2519664182 conn(0x7fcb800d13f0 msgr2=0x7fcb800b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:24.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 shutdown_connections 2024-04-13T03:11:24.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.305+0000 7fcb88636640 1 -- 172.21.15.106:0/2519664182 wait complete. 2024-04-13T03:11:24.317 INFO:tasks.cephfs.filesystem:scrub status for tag:fd0050df-ccd7-4843-82a6-8a62d21474d0 - {'path': '/', 'tag': 'fd0050df-ccd7-4843-82a6-8a62d21474d0', 'options': 'recursive,repair'} 2024-04-13T03:11:24.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:24.316+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-13T03:11:24.151314+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f3dc408ab20 con 0x7f3dcc162490 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: cluster: 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: id: d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: health: HEALTH_OK 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: services: 2024-04-13T03:11:24.318 INFO:teuthology.orchestra.run.smithi106.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: mgr: y(active, since 2m), standbys: x 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: mds: 1/1 daemons up, 3 standby 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: data: 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: volumes: 1/1 healthy 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: pools: 3 pools, 97 pgs 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: objects: 23 objects, 579 KiB 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: usage: 217 MiB used, 720 GiB / 720 GiB avail 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: pgs: 97 active+clean 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: io: 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: client: 2.3 KiB/s wr, 0 op/s rd, 7 op/s wr 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:24.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.151314+0000 mds.d [INF] scrub queued for path: / 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.151317+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.151323+0000 mds.d [INF] scrub summary: active paths [/] 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.151989+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.151997+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-13T03:11:24.320 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:24.152401+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:11:28.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:28.445+0000 7f3dbaffd640 1 -- 172.21.15.106:0/2141846133 <== mon.0 v2:172.21.15.106:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-13T03:11:27.244042+0000) ==== 189+0+0 (secure 0 0 0) 0x7f3dc40582b0 con 0x7f3dcc162490 2024-04-13T03:11:28.447 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:11:27.244042+0000 mds.d [INF] scrub summary: idle 2024-04-13T03:11:54.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:54.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-13T03:11:54.359 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:11:54.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:11:54.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.437+0000 7f221557b640 1 Processor -- start 2024-04-13T03:11:54.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.437+0000 7f221557b640 1 -- start start 2024-04-13T03:11:54.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f221557b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22100d1d40 0x7f22100d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f221557b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22100cfe70 con 0x7f221015ea80 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f221557b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221015ee60 con 0x7f2210162550 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f221557b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2210162af0 con 0x7f22100d1d40 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f220f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f22100d1d40 0x7f22100d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f220effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f221015ea80 0x7f22100b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34116/0 (socket says 172.21.15.106:34116) 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.438+0000 7f220effd640 1 -- 172.21.15.106:0/2404547409 learned_addr learned my addr 172.21.15.106:0/2404547409 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220f7fe640 1 -- 172.21.15.106:0/2404547409 >> v1:172.21.15.114:6789/0 conn(0x7f2210162550 legacy=0x7f22100d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220f7fe640 1 -- 172.21.15.106:0/2404547409 >> v1:172.21.15.106:6789/0 conn(0x7f221015ea80 legacy=0x7f22100b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220f7fe640 1 -- 172.21.15.106:0/2404547409 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2210162dd0 con 0x7f22100d1d40 2024-04-13T03:11:54.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220dffb640 1 -- 172.21.15.106:0/2404547409 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873536475 0 0) 0x7f22100cfe70 con 0x7f221015ea80 2024-04-13T03:11:54.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220f7fe640 1 --2- 172.21.15.106:0/2404547409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100d1d40 0x7f22100d22b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2208009d90 tx=0x7f220802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527d78a08d005b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220dffb640 1 -- 172.21.15.106:0/2404547409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208039070 con 0x7f22100d1d40 2024-04-13T03:11:54.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220dffb640 1 -- 172.21.15.106:0/2404547409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220802cd00 con 0x7f22100d1d40 2024-04-13T03:11:54.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.439+0000 7f220dffb640 1 -- 172.21.15.106:0/2404547409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208034660 con 0x7f22100d1d40 2024-04-13T03:11:54.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 -- 172.21.15.106:0/2404547409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100d1d40 msgr2=0x7f22100d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 --2- 172.21.15.106:0/2404547409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100d1d40 0x7f22100d22b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2208009d90 tx=0x7f220802bf50 comp rx=0 tx=0).stop 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 -- 172.21.15.106:0/2404547409 shutdown_connections 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 --2- 172.21.15.106:0/2404547409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100d1d40 0x7f22100d22b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 -- 172.21.15.106:0/2404547409 >> 172.21.15.106:0/2404547409 conn(0x7f221009da50 msgr2=0x7f22100b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 -- 172.21.15.106:0/2404547409 shutdown_connections 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.440+0000 7f221557b640 1 -- 172.21.15.106:0/2404547409 wait complete. 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.441+0000 7f221557b640 1 Processor -- start 2024-04-13T03:11:54.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.441+0000 7f221557b640 1 -- start start 2024-04-13T03:11:54.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22100d1d40 0x7f221015d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2210162550 0x7f221006d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22100d0320 con 0x7f221015ea80 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2210163390 con 0x7f22100d1d40 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f221557b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f22100d04a0 con 0x7f2210162550 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36250/0 (socket says 172.21.15.106:36250) 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f220e7fc640 1 -- 172.21.15.106:0/283903417 learned_addr learned my addr 172.21.15.106:0/283903417 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f220effd640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22100d1d40 0x7f221015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.442+0000 7f220f7fe640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2210162550 0x7f221006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 -- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2210162550 msgr2=0x7f221006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2210162550 0x7f221006d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 -- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22100d1d40 msgr2=0x7f221015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22100d1d40 0x7f221015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 -- 172.21.15.106:0/283903417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100c9fc0 con 0x7f221015ea80 2024-04-13T03:11:54.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f220e7fc640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f220000c400 tx=0x7f220000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f21f77fe640 1 -- 172.21.15.106:0/283903417 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200017070 con 0x7f221015ea80 2024-04-13T03:11:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f21f77fe640 1 -- 172.21.15.106:0/283903417 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000052d0 con 0x7f221015ea80 2024-04-13T03:11:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22100ca2a0 con 0x7f221015ea80 2024-04-13T03:11:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f21f77fe640 1 -- 172.21.15.106:0/283903417 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200003830 con 0x7f221015ea80 2024-04-13T03:11:54.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.443+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22100ca860 con 0x7f221015ea80 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.445+0000 7f21f77fe640 1 -- 172.21.15.106:0/283903417 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220001a070 con 0x7f221015ea80 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.445+0000 7f21f77fe640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 0x7f21fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.445+0000 7f21f77fe640 1 -- 172.21.15.106:0/283903417 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f220008af30 con 0x7f221015ea80 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.445+0000 7f220effd640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 0x7f21fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.445+0000 7f220effd640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 0x7f21fc06ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2204007f70 tx=0x7f2204007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.446+0000 7f2218367740 1 Processor -- start 2024-04-13T03:11:54.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.446+0000 7f2218367740 1 -- start start 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x564590533d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645905044d0 0x5645905342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56459046f170 con 0x564590530ff0 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645904157f0 con 0x5645905044d0 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56459046b130 con 0x5645905313d0 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f220effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f220effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36252/0 (socket says 172.21.15.106:36252) 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f220effd640 1 -- 172.21.15.106:0/2609155937 learned_addr learned my addr 172.21.15.106:0/2609155937 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f220e7fc640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x564590533d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.447+0000 7f220f7fe640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645905044d0 0x5645905342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 -- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 msgr2=0x564590533d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x564590533d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 -- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645905044d0 msgr2=0x5645905342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645905044d0 0x5645905342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 -- 172.21.15.106:0/2609155937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459024b7d0 con 0x564590530ff0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220e7fc640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x564590533d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f220effd640 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f22040018b0 tx=0x7f2204013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfd8d6bcaaa7bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f21f4ff9640 1 -- 172.21.15.106:0/2609155937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204026070 con 0x564590530ff0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f21f4ff9640 1 -- 172.21.15.106:0/2609155937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204013b80 con 0x564590530ff0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.448+0000 7f21f4ff9640 1 -- 172.21.15.106:0/2609155937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204012350 con 0x564590530ff0 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- 172.21.15.106:0/2609155937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 msgr2=0x564590503f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f22040018b0 tx=0x7f2204013000 comp rx=0 tx=0).stop 2024-04-13T03:11:54.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- 172.21.15.106:0/2609155937 shutdown_connections 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x564590533d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645905044d0 0x5645905342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 --2- 172.21.15.106:0/2609155937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590530ff0 0x564590503f90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- 172.21.15.106:0/2609155937 >> 172.21.15.106:0/2609155937 conn(0x564590506200 msgr2=0x56459054ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- 172.21.15.106:0/2609155937 shutdown_connections 2024-04-13T03:11:54.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- 172.21.15.106:0/2609155937 wait complete. 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.449+0000 7f2218367740 1 -- start start 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 Processor -- start 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x56459052f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 0x56459053b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5645905088a0 con 0x5645905313d0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564590428d00 con 0x564590530ff0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56459050b0e0 con 0x564590560330 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f2218367740 10 client.0 _ll_register_callbacks cb 0x5645904286f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36256/0 (socket says 172.21.15.106:36256) 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.450+0000 7f220e7fc640 1 -- 172.21.15.106:0/2361993620 learned_addr learned my addr 172.21.15.106:0/2361993620 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220effd640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x56459052f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220f7fe640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 0x56459053b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 -- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 msgr2=0x56459053b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 0x56459053b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 -- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 msgr2=0x56459052f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x56459052f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 -- 172.21.15.106:0/2361993620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645901813f0 con 0x5645905313d0 2024-04-13T03:11:54.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220f7fe640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 0x56459053b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f220e7fc640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2200058880 tx=0x7f220005cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ace9f60fb491230 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200057fd0 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2200058170 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2204013b80 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2200087050 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 10 client.6069 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5645901410f0 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204013b80 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f22000548d0 con 0x5645905313d0 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 10 client.6069 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 10 client.6069 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:54.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 20 client.6069 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x564590538e90 0x564590539290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 4 client.6069 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.451+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56459047cb90 con 0x564590538e90 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.452+0000 7f220effd640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x564590538e90 0x564590539290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.452+0000 7f21edffb640 10 client.6069 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.452+0000 7f220effd640 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x564590538e90 0x564590539290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82de21cb7fa7a4 server_cookie=895973c77a1ce699 in_seq=0 out_seq=0 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f21edffb640 1 -- 172.21.15.106:0/2361993620 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56459047cb90 con 0x564590538e90 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f21edffb640 10 client.6069 handle_command_reply: tid=0 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 client.6069 shutdown 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 msgr2=0x564590535740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2200058880 tx=0x7f220005cfc0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 shutdown_connections 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x564590538e90 0x564590539290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590560330 0x56459053b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f21edffb640 0 client.6069 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x564590535740 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 --2- 172.21.15.106:0/2361993620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x56459052f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 >> 172.21.15.106:0/2361993620 conn(0x5645905044d0 msgr2=0x564590507ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 shutdown_connections 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 1 -- 172.21.15.106:0/2361993620 wait complete. 2024-04-13T03:11:54.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.453+0000 7f2218367740 20 client.6069 trim_cache size 0 max 16384 2024-04-13T03:11:54.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.470+0000 7f2218367740 1 Processor -- start 2024-04-13T03:11:54.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 -- start start 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2210201420 con 0x564590560330 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f22102004e0 con 0x564590530ff0 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f22101e8e00 con 0x5645905313d0 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36264/0 (socket says 172.21.15.106:36264) 2024-04-13T03:11:54.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f220f7fe640 1 -- 172.21.15.106:0/1469390716 learned_addr learned my addr 172.21.15.106:0/1469390716 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.471+0000 7f220effd640 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220e7fc640 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 -- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 msgr2=0x5645905352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 -- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 msgr2=0x564590534d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 -- 172.21.15.106:0/1469390716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564590504950 con 0x564590560330 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f220f7fe640 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2208009eb0 tx=0x7f220802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f9649898a710b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f21effff640 1 -- 172.21.15.106:0/1469390716 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208039040 con 0x564590560330 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f21effff640 1 -- 172.21.15.106:0/1469390716 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220803a040 con 0x564590560330 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.472+0000 7f21effff640 1 -- 172.21.15.106:0/1469390716 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080418a0 con 0x564590560330 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- 172.21.15.106:0/1469390716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 msgr2=0x5645905508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2208009eb0 tx=0x7f220802e000 comp rx=0 tx=0).stop 2024-04-13T03:11:54.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- 172.21.15.106:0/1469390716 shutdown_connections 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564590560330 0x5645905508e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 --2- 172.21.15.106:0/1469390716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- 172.21.15.106:0/1469390716 >> 172.21.15.106:0/1469390716 conn(0x5645905044d0 msgr2=0x5645905337c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- 172.21.15.106:0/1469390716 shutdown_connections 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- 172.21.15.106:0/1469390716 wait complete. 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 -- start start 2024-04-13T03:11:54.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.473+0000 7f2218367740 1 Processor -- start 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590560330 0x564590572100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22101e9450 con 0x5645905313d0 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645905733c0 con 0x564590560330 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564590508bd0 con 0x564590530ff0 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f2218367740 10 client.0 _ll_register_callbacks cb 0x7f2200058ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36272/0 (socket says 172.21.15.106:36272) 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 -- 172.21.15.106:0/175904728 learned_addr learned my addr 172.21.15.106:0/175904728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220effd640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220f7fe640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590560330 0x564590572100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 -- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590530ff0 msgr2=0x564590534d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 -- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590560330 msgr2=0x564590572100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590560330 0x564590572100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.474+0000 7f220e7fc640 1 -- 172.21.15.106:0/175904728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645905ba170 con 0x5645905313d0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f220e7fc640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2200058aa0 tx=0x7f220005b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1018de41e0ec573e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220005f240 con 0x5645905313d0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2200074c40 con 0x5645905313d0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220803a040 con 0x5645905313d0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f220005f3e0 con 0x5645905313d0 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 10 client.6075 fetch_fsmap learned FSMap version 22 2024-04-13T03:11:54.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x564590552630 con 0x5645905313d0 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220803a040 con 0x5645905313d0 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f2200057a40 con 0x5645905313d0 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 10 client.6075 fetch_fsmap finished waiting for FSMap version 22 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 20 client.6075 populate_metadata read hostname 'smithi106' 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x56459052fa90 0x564590572640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 4 client.6075 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 --> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564590476a00 con 0x56459052fa90 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.475+0000 7f220effd640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x56459052fa90 0x564590572640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f21edffb640 10 client.6075 ms_handle_connect on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f220effd640 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x56459052fa90 0x564590572640 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ee35692633ea5d server_cookie=9484b943d92dbd8b in_seq=0 out_seq=0 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f21edffb640 1 -- 172.21.15.106:0/175904728 <== mds.0 v2:172.21.15.114:6833/4219642587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564590476a00 con 0x56459052fa90 2024-04-13T03:11:54.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f21edffb640 10 client.6075 handle_command_reply: tid=0 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 client.6075 shutdown 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 msgr2=0x5645905352c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2200058aa0 tx=0x7f220005b070 comp rx=0 tx=0).stop 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 shutdown_connections 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:6833/4219642587,v1:172.21.15.114:6835/4219642587] conn(0x56459052fa90 0x564590572640 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564590560330 0x564590572100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645905313d0 0x5645905352c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f21edffb640 0 client.6075 ms_handle_reset on v2:172.21.15.114:6833/4219642587 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 --2- 172.21.15.106:0/175904728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564590530ff0 0x564590534d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 >> 172.21.15.106:0/175904728 conn(0x5645905044d0 msgr2=0x5645905337c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 shutdown_connections 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 1 -- 172.21.15.106:0/175904728 wait complete. 2024-04-13T03:11:54.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.476+0000 7f2218367740 20 client.6075 trim_cache size 0 max 16384 2024-04-13T03:11:54.484 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:54.484 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:11:54.484 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:11:54.484 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 msgr2=0x7f21fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 0x7f21fc06ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2204007f70 tx=0x7f2204007f00 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 msgr2=0x7f221006d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f220000c400 tx=0x7f220000c8c0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 shutdown_connections 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21fc06c920 0x7f21fc06ede0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2210162550 0x7f221006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f221015ea80 0x7f221006d010 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 --2- 172.21.15.106:0/283903417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22100d1d40 0x7f221015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 >> 172.21.15.106:0/283903417 conn(0x7f221009da50 msgr2=0x7f22100b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 shutdown_connections 2024-04-13T03:11:54.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.477+0000 7f221557b640 1 -- 172.21.15.106:0/283903417 wait complete. 2024-04-13T03:11:54.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:11:54.589 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:54.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.690+0000 7f121ea05640 1 Processor -- start 2024-04-13T03:11:54.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.690+0000 7f121ea05640 1 -- start start 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121ea05640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f12180cefe0 0x7f121806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121ea05640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12180b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121ea05640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f121806bfe0 con 0x7f12180b3980 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121ea05640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f12180cfda0 con 0x7f12180cefe0 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121ea05640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f121806c2c0 con 0x7f12180b7810 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f12177fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f12180b3980 0x7f12180cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34122/0 (socket says 172.21.15.106:34122) 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f12177fe640 1 -- 172.21.15.106:0/1537560527 learned_addr learned my addr 172.21.15.106:0/1537560527 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:54.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3072869564 0 0) 0x7f121806bfe0 con 0x7f12180b3980 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1200003520 con 0x7f12180b3980 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f121cfa2640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12180b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1217fff640 1 --2- 172.21.15.106:0/1537560527 >> v2:172.21.15.114:3300/0 conn(0x7f12180cefe0 0x7f121806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3841761909 0 0) 0x7f1200003520 con 0x7f12180b3980 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 msgr2=0x7f12180b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12180b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 >> v2:172.21.15.114:3300/0 conn(0x7f12180cefe0 msgr2=0x7f121806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 --2- 172.21.15.106:0/1537560527 >> v2:172.21.15.114:3300/0 conn(0x7f12180cefe0 0x7f121806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121806c5a0 con 0x7f12180b3980 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f120c001670 con 0x7f12180b3980 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 >> v1:172.21.15.106:6789/0 conn(0x7f12180b3980 legacy=0x7f12180cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1200003f30 0x7f1200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1200004d40 0x7f12000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f121806c2c0 con 0x7f1200004d40 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f120000b9c0 con 0x7f1200003f30 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.691+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f120000bb40 con 0x7f12180b7810 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1200004d40 0x7f12000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f1217fff640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f12177fe640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1200003f30 0x7f1200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 -- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 msgr2=0x7f12000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 -- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1200003f30 msgr2=0x7f1200004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1200003f30 0x7f1200004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 -- 172.21.15.106:0/1537560527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1200003520 con 0x7f1200004d40 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f1217fff640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180b7810 0x7f12000039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.692+0000 7f121cfa2640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1200004d40 0x7f12000051c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1210009fc0 tx=0x7f1200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969c2361ef3fc65c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210039070 con 0x7f1200004d40 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121002cc00 con 0x7f1200004d40 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f1216ffd640 1 -- 172.21.15.106:0/1537560527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210009c10 con 0x7f1200004d40 2024-04-13T03:11:54.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 -- 172.21.15.106:0/1537560527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1200004d40 msgr2=0x7f12000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 --2- 172.21.15.106:0/1537560527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1200004d40 0x7f12000051c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1210009fc0 tx=0x7f1200005830 comp rx=0 tx=0).stop 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121cfa2640 1 -- 172.21.15.106:0/1537560527 reap_dead start 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 -- 172.21.15.106:0/1537560527 shutdown_connections 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 -- 172.21.15.106:0/1537560527 >> 172.21.15.106:0/1537560527 conn(0x7f12180d4780 msgr2=0x7f12180d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 -- 172.21.15.106:0/1537560527 shutdown_connections 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.693+0000 7f121ea05640 1 -- 172.21.15.106:0/1537560527 wait complete. 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.694+0000 7f121ea05640 1 Processor -- start 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 -- start start 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180cbdd0 0x7f12180ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12180ad3b0 0x7f12180ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f121806d080 con 0x7f121815c8c0 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f121806cb60 con 0x7f12180ad3b0 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f121ea05640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f12180ce9b0 con 0x7f12180cbdd0 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f1217fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f1217fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36292/0 (socket says 172.21.15.106:36292) 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.695+0000 7f1217fff640 1 -- 172.21.15.106:0/4004129619 learned_addr learned my addr 172.21.15.106:0/4004129619 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 -- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180cbdd0 msgr2=0x7f12180ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180cbdd0 0x7f12180ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f121cfa2640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12180ad3b0 0x7f12180ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 -- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12180ad3b0 msgr2=0x7f12180ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12180ad3b0 0x7f12180ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 -- 172.21.15.106:0/4004129619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12180c9e40 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1217fff640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1208011ca0 tx=0x7f120800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800fcc0 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120800fe60 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12180ab2c0 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208014680 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.696+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12180cc250 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.697+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1218068a10 con 0x7f121815c8c0 2024-04-13T03:11:54.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.697+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1208012040 con 0x7f121815c8c0 2024-04-13T03:11:54.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.698+0000 7f1214ff9640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 0x7f120406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:54.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.698+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f120808c450 con 0x7f121815c8c0 2024-04-13T03:11:54.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.698+0000 7f12177fe640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 0x7f120406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:54.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.698+0000 7f12177fe640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 0x7f120406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12180cb040 tx=0x7f120c0047e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:54.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.702+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1208019030 con 0x7f121815c8c0 2024-04-13T03:11:54.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.914+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12180cf3c0 con 0x7f121815c8c0 2024-04-13T03:11:54.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.915+0000 7f1214ff9640 1 -- 172.21.15.106:0/4004129619 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f120805a780 con 0x7f121815c8c0 2024-04-13T03:11:54.916 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:54.916 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:54.917 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 msgr2=0x7f120406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 0x7f120406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12180cb040 tx=0x7f120c0047e0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 msgr2=0x7f12180ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1208011ca0 tx=0x7f120800f610 comp rx=0 tx=0).stop 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 shutdown_connections 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f120406c8f0 0x7f120406edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12180cbdd0 0x7f12180ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12180ad3b0 0x7f12180ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 --2- 172.21.15.106:0/4004129619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f121815c8c0 0x7f12180ce360 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:54.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 >> 172.21.15.106:0/4004129619 conn(0x7f12180d4780 msgr2=0x7f12180b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:54.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 shutdown_connections 2024-04-13T03:11:54.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:54.917+0000 7f121ea05640 1 -- 172.21.15.106:0/4004129619 wait complete. 2024-04-13T03:11:54.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-13T03:11:55.071 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:11:55.071 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:11:55.071 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-13T03:11:55.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.113+0000 7f740c039f40 1 Processor -- start 2024-04-13T03:11:55.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.113+0000 7f740c039f40 1 -- start start 2024-04-13T03:11:55.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa3719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa410c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fffa2ebd00 con 0x55fffa40f050 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fffa2efd40 con 0x7f73f4002a10 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f740c039f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fffa40eeb0 con 0x55fffa40ea10 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36318/0 (socket says 172.21.15.106:36318) 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f7408fa3640 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa3719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 -- 172.21.15.106:0/3502050494 learned_addr learned my addr 172.21.15.106:0/3502050494 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74007a2640 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa410c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 -- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 msgr2=0x55fffa410c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa410c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 -- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 msgr2=0x55fffa3719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa3719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.114+0000 7f74097a4640 1 -- 172.21.15.106:0/3502050494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffa3704b0 con 0x55fffa40f050 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f74097a4640 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7404005fd0 tx=0x7f7404005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a574e7c926699202 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f74037fe640 1 -- 172.21.15.106:0/3502050494 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404017030 con 0x55fffa40f050 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f74037fe640 1 -- 172.21.15.106:0/3502050494 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74040106d0 con 0x55fffa40f050 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f74037fe640 1 -- 172.21.15.106:0/3502050494 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404010880 con 0x55fffa40f050 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 -- 172.21.15.106:0/3502050494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 msgr2=0x55fffa417f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7404005fd0 tx=0x7f7404005d80 comp rx=0 tx=0).stop 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 -- 172.21.15.106:0/3502050494 shutdown_connections 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa410c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa417f10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 --2- 172.21.15.106:0/3502050494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa3719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 -- 172.21.15.106:0/3502050494 >> 172.21.15.106:0/3502050494 conn(0x55fffa343f30 msgr2=0x55fffa370e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 -- 172.21.15.106:0/3502050494 shutdown_connections 2024-04-13T03:11:55.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.115+0000 7f740c039f40 1 -- 172.21.15.106:0/3502050494 wait complete. 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 Processor -- start 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 -- start start 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa402e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa36f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa417e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fffa3709f0 con 0x55fffa40f050 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fffa370750 con 0x7f73f4002a10 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.116+0000 7f740c039f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fffa417a10 con 0x55fffa40ea10 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa36f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f7408fa3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa402e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa36f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36322/0 (socket says 172.21.15.106:36322) 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 -- 172.21.15.106:0/3425943618 learned_addr learned my addr 172.21.15.106:0/3425943618 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74097a4640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa417e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 -- 172.21.15.106:0/3425943618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 msgr2=0x55fffa402e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fffa40ea10 0x55fffa402e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 -- 172.21.15.106:0/3425943618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 msgr2=0x55fffa417e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f4002a10 0x55fffa417e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 -- 172.21.15.106:0/3425943618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffa4082f0 con 0x55fffa40f050 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74007a2640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa36f570 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73f001ca10 tx=0x7f73f4004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74027fc640 1 -- 172.21.15.106:0/3425943618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0065070 con 0x55fffa40f050 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74027fc640 1 -- 172.21.15.106:0/3425943618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f004ab40 con 0x55fffa40f050 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f74027fc640 1 -- 172.21.15.106:0/3425943618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0054070 con 0x55fffa40f050 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fffa4085d0 con 0x55fffa40f050 2024-04-13T03:11:55.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.117+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffa408b90 con 0x55fffa40f050 2024-04-13T03:11:55.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f74027fc640 1 -- 172.21.15.106:0/3425943618 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73f0068070 con 0x55fffa40f050 2024-04-13T03:11:55.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f74027fc640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73e806b320 0x7f73e806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f74027fc640 1 -- 172.21.15.106:0/3425943618 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f73f00cb5a0 con 0x55fffa40f050 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f7408fa3640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73e806b320 0x7f73e806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f740c039f40 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55fffa36fab0 0x55fffa406d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) -- 0x55fffa407290 con 0x55fffa36fab0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f74097a4640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55fffa36fab0 0x55fffa406d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.119+0000 7f7408fa3640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73e806b320 0x7f73e806d7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55fffa38b090 tx=0x7f73f8012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f74097a4640 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55fffa36fab0 0x55fffa406d50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f74097a4640 1 -- 172.21.15.106:0/3425943618 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f7404010b20 con 0x55fffa36fab0 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55fffa36fab0 msgr2=0x55fffa406d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55fffa36fab0 0x55fffa406d50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73e806b320 msgr2=0x7f73e806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73e806b320 0x7f73e806d7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55fffa38b090 tx=0x7f73f8012040 comp rx=0 tx=0).stop 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 msgr2=0x55fffa36f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 --2- 172.21.15.106:0/3425943618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fffa40f050 0x55fffa36f570 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73f001ca10 tx=0x7f73f4004120 comp rx=0 tx=0).stop 2024-04-13T03:11:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f74097a4640 1 -- 172.21.15.106:0/3425943618 reap_dead start 2024-04-13T03:11:55.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 shutdown_connections 2024-04-13T03:11:55.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 >> 172.21.15.106:0/3425943618 conn(0x55fffa343f30 msgr2=0x55fffa40e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:55.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 shutdown_connections 2024-04-13T03:11:55.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.120+0000 7f740c039f40 1 -- 172.21.15.106:0/3425943618 wait complete. 2024-04-13T03:11:55.127 INFO:teuthology.orchestra.run.smithi106.stdout:Yalphaparent_a 2024-04-13T03:11:55.127 DEBUG:teuthology.orchestra.run.smithi106:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: "ancestors": [ 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1099511627776, 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "alpha", 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 4 2024-04-13T03:11:55.213 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1, 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "parent_a", 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 7 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: "pool": 8, 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: "old_pools": [] 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:11:55.214 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:55.215 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:11:55.216 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:55.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:11:55.344 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:11:55.344 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:55.344 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:55.344 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:55.359 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:55.360 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:55.415 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:55.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 Processor -- start 2024-04-13T03:11:55.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 -- start start 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec0d0190 con 0x7fe9ec0d0a80 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ec1625d0 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.554+0000 7fe9f0c1c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec163430 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9e3fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:51474/0 (socket says 172.21.15.106:51474) 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9e3fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:48260/0 (socket says 172.21.15.106:48260) 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ea59c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe9ec0d0a80 0x7fe9ec0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36172/0 (socket says 172.21.15.106:36172) 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 -- 172.21.15.106:0/38366545 learned_addr learned my addr 172.21.15.106:0/38366545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 -- 172.21.15.106:0/38366545 >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 msgr2=0x7fe9ec0d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 --2- 172.21.15.106:0/38366545 >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 -- 172.21.15.106:0/38366545 >> v1:172.21.15.114:6789/0 conn(0x7fe9ec0d0a80 legacy=0x7fe9ec0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9ead9d640 1 -- 172.21.15.106:0/38366545 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec162820 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.555+0000 7fe9e3fff640 1 --2- 172.21.15.106:0/38366545 >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.556+0000 7fe9ead9d640 1 --2- 172.21.15.106:0/38366545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009e50 tx=0x7fe9e402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1fa2016f0c5cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.556+0000 7fe9e9d9b640 1 -- 172.21.15.106:0/38366545 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e402cb00 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.556+0000 7fe9e9d9b640 1 -- 172.21.15.106:0/38366545 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e402cca0 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.556+0000 7fe9e9d9b640 1 -- 172.21.15.106:0/38366545 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4034610 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/38366545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 msgr2=0x7fe9ec0d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/38366545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009e50 tx=0x7fe9e402bf60 comp rx=0 tx=0).stop 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/38366545 shutdown_connections 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/38366545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d4c30 0x7fe9ec0d2530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/38366545 >> v2:172.21.15.114:3300/0 conn(0x7fe9ec0cfd30 0x7fe9ec0d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/38366545 >> 172.21.15.106:0/38366545 conn(0x7fe9ec09da70 msgr2=0x7fe9ec0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/38366545 shutdown_connections 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.557+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/38366545 wait complete. 2024-04-13T03:11:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.558+0000 7fe9f0c1c640 1 Processor -- start 2024-04-13T03:11:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.558+0000 7fe9f0c1c640 1 -- start start 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d0a80 0x7fe9ec156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9ec0d4c30 0x7fe9ec0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec162de0 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0d41b0 con 0x7fe9ec0d4c30 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9f0c1c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec1630c0 con 0x7fe9ec0d0a80 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36338/0 (socket says 172.21.15.106:36338) 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 -- 172.21.15.106:0/1559880983 learned_addr learned my addr 172.21.15.106:0/1559880983 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ead9d640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9ec0d4c30 0x7fe9ec0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9e3fff640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d0a80 0x7fe9ec156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 -- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d0a80 msgr2=0x7fe9ec156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d0a80 0x7fe9ec156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 -- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9ec0d4c30 msgr2=0x7fe9ec0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9ec0d4c30 0x7fe9ec0cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 -- 172.21.15.106:0/1559880983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec157160 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.559+0000 7fe9ea59c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c400 tx=0x7fe9dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.560+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc017070 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.560+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9dc005c70 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.560+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0c91f0 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.560+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc005e60 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.560+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0cbd60 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.561+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec069120 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.561+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9dc01a070 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.561+0000 7fe9e2ffd640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 0x7fe9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.562+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fe9dc08aed0 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.562+0000 7fe9e3fff640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 0x7fe9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.562+0000 7fe9e3fff640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 0x7fe9cc06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0cb390 tx=0x7fe9d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.566+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9ec157160 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.742+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9ec0b4c00 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.743+0000 7fe9e2ffd640 1 -- 172.21.15.106:0/1559880983 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16978 (secure 0 0 0) 0x7fe9dc058bb0 con 0x7fe9ec0cfd30 2024-04-13T03:11:55.744 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:55.744 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":52,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:18.293562+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 msgr2=0x7fe9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 0x7fe9cc06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0cb390 tx=0x7fe9d4009190 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 msgr2=0x7fe9ec09ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c400 tx=0x7fe9dc00c8c0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 shutdown_connections 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe9cc06c8f0 0x7fe9cc06edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe9ec0d0a80 0x7fe9ec156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9ec0d4c30 0x7fe9ec0cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 --2- 172.21.15.106:0/1559880983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9ec0cfd30 0x7fe9ec09ef10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 >> 172.21.15.106:0/1559880983 conn(0x7fe9ec09da70 msgr2=0x7fe9ec0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 shutdown_connections 2024-04-13T03:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.745+0000 7fe9f0c1c640 1 -- 172.21.15.106:0/1559880983 wait complete. 2024-04-13T03:11:55.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:11:14.853009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:11:15.586983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:11:55.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:11:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.859+0000 7f56ca8de640 1 Processor -- start 2024-04-13T03:11:55.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.859+0000 7f56ca8de640 1 -- start start 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56ca8de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56c4073520 0x7f56c40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56ca8de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 0x7f56c40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56ca8de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c40b5060 con 0x7f56c415e8c0 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56ca8de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40b5340 con 0x7f56c4073520 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56ca8de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f56c406c500 con 0x7f56c415eca0 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56c4073520 0x7f56c40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f56c415e8c0 0x7f56c4072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34144/0 (socket says 172.21.15.106:34144) 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c3fff640 1 -- 172.21.15.106:0/2752002978 learned_addr learned my addr 172.21.15.106:0/2752002978 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:55.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c2ffd640 1 -- 172.21.15.106:0/2752002978 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443928835 0 0) 0x7f56c40b5060 con 0x7f56c415e8c0 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c2ffd640 1 -- 172.21.15.106:0/2752002978 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56ac003520 con 0x7f56c415e8c0 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c37fe640 1 -- 172.21.15.106:0/2752002978 >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 msgr2=0x7f56c40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c8e7b640 1 --2- 172.21.15.106:0/2752002978 >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 0x7f56c40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c37fe640 1 --2- 172.21.15.106:0/2752002978 >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 0x7f56c40b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c37fe640 1 -- 172.21.15.106:0/2752002978 >> v1:172.21.15.106:6789/0 conn(0x7f56c415e8c0 legacy=0x7f56c4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c37fe640 1 -- 172.21.15.106:0/2752002978 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c406c7e0 con 0x7f56c4073520 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.860+0000 7f56c8e7b640 1 --2- 172.21.15.106:0/2752002978 >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 0x7f56c40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56c37fe640 1 --2- 172.21.15.106:0/2752002978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c4073520 0x7f56c40ce250 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f56c40d1430 tx=0x7f56b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea90c2d1e1a97e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56c2ffd640 1 -- 172.21.15.106:0/2752002978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8034070 con 0x7f56c4073520 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56c2ffd640 1 -- 172.21.15.106:0/2752002978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b8030610 con 0x7f56c4073520 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56c2ffd640 1 -- 172.21.15.106:0/2752002978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80309e0 con 0x7f56c4073520 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56ca8de640 1 -- 172.21.15.106:0/2752002978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c4073520 msgr2=0x7f56c40ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56ca8de640 1 --2- 172.21.15.106:0/2752002978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c4073520 0x7f56c40ce250 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f56c40d1430 tx=0x7f56b802b960 comp rx=0 tx=0).stop 2024-04-13T03:11:55.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56ca8de640 1 -- 172.21.15.106:0/2752002978 shutdown_connections 2024-04-13T03:11:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56ca8de640 1 --2- 172.21.15.106:0/2752002978 >> v2:172.21.15.114:3300/0 conn(0x7f56c415eca0 0x7f56c40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.861+0000 7f56ca8de640 1 --2- 172.21.15.106:0/2752002978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c4073520 0x7f56c40ce250 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.862+0000 7f56ca8de640 1 -- 172.21.15.106:0/2752002978 >> 172.21.15.106:0/2752002978 conn(0x7f56c406f0d0 msgr2=0x7f56c406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.862+0000 7f56ca8de640 1 -- 172.21.15.106:0/2752002978 shutdown_connections 2024-04-13T03:11:55.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.862+0000 7f56ca8de640 1 -- 172.21.15.106:0/2752002978 wait complete. 2024-04-13T03:11:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.863+0000 7f56ca8de640 1 Processor -- start 2024-04-13T03:11:55.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.863+0000 7f56ca8de640 1 -- start start 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.863+0000 7f56ca8de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56c4073520 0x7f56c41f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.863+0000 7f56ca8de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.863+0000 7f56ca8de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c415eca0 0x7f56c4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56ca8de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40b4d40 con 0x7f56c415e8c0 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56ca8de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40b4bc0 con 0x7f56c415eca0 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56ca8de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f56c406cd00 con 0x7f56c4073520 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36364/0 (socket says 172.21.15.106:36364) 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 -- 172.21.15.106:0/777754788 learned_addr learned my addr 172.21.15.106:0/777754788 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 -- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56c4073520 msgr2=0x7f56c41f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56c4073520 0x7f56c41f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 -- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c415eca0 msgr2=0x7f56c4156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c415eca0 0x7f56c4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 -- 172.21.15.106:0/777754788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c41f7b20 con 0x7f56c415e8c0 2024-04-13T03:11:55.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c37fe640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f56b8000c00 tx=0x7f56b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8034040 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b8030410 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c4160030 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8039c50 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.864+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41f4e10 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.865+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4068a10 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.865+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56b8037070 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.866+0000 7f56c0ff9640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.866+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f56b80add00 con 0x7f56c415e8c0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.866+0000 7f56c3fff640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:55.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.866+0000 7f56c3fff640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 0x7f56b006edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000f60 tx=0x7f56b40078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:55.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:55.870+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56c41f7b20 con 0x7f56c415e8c0 2024-04-13T03:11:56.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.043+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56c40cdbf0 con 0x7f56c415e8c0 2024-04-13T03:11:56.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.044+0000 7f56c0ff9640 1 -- 172.21.15.106:0/777754788 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f56b80305e0 con 0x7f56c415e8c0 2024-04-13T03:11:56.045 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 msgr2=0x7f56b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 0x7f56b006edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000f60 tx=0x7f56b40078f0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 msgr2=0x7f56c415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f56b8000c00 tx=0x7f56b80046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 shutdown_connections 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f56b006c8f0 0x7f56b006edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c415eca0 0x7f56c4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c415e8c0 0x7f56c415faf0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 --2- 172.21.15.106:0/777754788 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56c4073520 0x7f56c41f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 >> 172.21.15.106:0/777754788 conn(0x7f56c406f0d0 msgr2=0x7f56c40b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 shutdown_connections 2024-04-13T03:11:56.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.046+0000 7f56ca8de640 1 -- 172.21.15.106:0/777754788 wait complete. 2024-04-13T03:11:56.057 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:11:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.159+0000 7ff1e8008640 1 Processor -- start 2024-04-13T03:11:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.159+0000 7ff1e8008640 1 -- start start 2024-04-13T03:11:56.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e8008640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e8008640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff1e00b47e0 0x7ff1e015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e8008640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e00b3740 con 0x7ff1e00b4c20 2024-04-13T03:11:56.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e8008640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff1e00b3a20 con 0x7ff1e00b47e0 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e8008640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e00b3d00 con 0x7ff1e0145d20 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff1e00b47e0 0x7ff1e015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff1e00b47e0 0x7ff1e015cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36368/0 (socket says 172.21.15.106:36368) 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 -- 172.21.15.106:0/1579542940 learned_addr learned my addr 172.21.15.106:0/1579542940 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e5da4640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 -- 172.21.15.106:0/1579542940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 msgr2=0x7ff1e006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 -- 172.21.15.106:0/1579542940 >> v1:172.21.15.114:6789/0 conn(0x7ff1e00b4c20 legacy=0x7ff1e00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.160+0000 7ff1e55a3640 1 -- 172.21.15.106:0/1579542940 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e00b3fe0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.161+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e015cbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009870 tx=0x7ff1d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68117b9d0750c560 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.161+0000 7ff1e4da2640 1 -- 172.21.15.106:0/1579542940 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0034070 con 0x7ff1e00b47e0 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.161+0000 7ff1e4da2640 1 -- 172.21.15.106:0/1579542940 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d0030840 con 0x7ff1e00b47e0 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.161+0000 7ff1e4da2640 1 -- 172.21.15.106:0/1579542940 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0030c10 con 0x7ff1e00b47e0 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 -- 172.21.15.106:0/1579542940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 msgr2=0x7ff1e015cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e015cbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009870 tx=0x7ff1d002b980 comp rx=0 tx=0).stop 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 -- 172.21.15.106:0/1579542940 shutdown_connections 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e015cbc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1579542940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 -- 172.21.15.106:0/1579542940 >> 172.21.15.106:0/1579542940 conn(0x7ff1e00b2600 msgr2=0x7ff1e00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 -- 172.21.15.106:0/1579542940 shutdown_connections 2024-04-13T03:11:56.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.162+0000 7ff1e8008640 1 -- 172.21.15.106:0/1579542940 wait complete. 2024-04-13T03:11:56.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.163+0000 7ff1e8008640 1 Processor -- start 2024-04-13T03:11:56.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.163+0000 7ff1e8008640 1 -- start start 2024-04-13T03:11:56.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 0x7ff1e01e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e0146280 con 0x7ff1e00b47e0 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e0146100 con 0x7ff1e00b4c20 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e006d050 con 0x7ff1e0145d20 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36382/0 (socket says 172.21.15.106:36382) 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 -- 172.21.15.106:0/1833649478 learned_addr learned my addr 172.21.15.106:0/1833649478 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 -- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 msgr2=0x7ff1e015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 0x7ff1e01e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e65a5640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 -- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 msgr2=0x7ff1e01e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 0x7ff1e01e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 -- 172.21.15.106:0/1833649478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e01e99a0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 0x7ff1e01e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e65a5640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:56.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e5da4640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d620 tx=0x7ff1dc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00b620 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc019070 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e01e9c20 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00ba00 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.164+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e013d0a0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.165+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e0068a10 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.166+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1dc00bba0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.166+0000 7ff1d67fc640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 0x7ff1c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.166+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7ff1dc08c0d0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.166+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 0x7ff1c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.167+0000 7ff1e55a3640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 0x7ff1c406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0036000 tx=0x7ff1d003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.171+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1e01e99a0 con 0x7ff1e00b47e0 2024-04-13T03:11:56.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.347+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1e00b2e30 con 0x7ff1e00b47e0 2024-04-13T03:11:56.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.347+0000 7ff1d67fc640 1 -- 172.21.15.106:0/1833649478 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7ff1dc060070 con 0x7ff1e00b47e0 2024-04-13T03:11:56.348 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 msgr2=0x7ff1c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 0x7ff1c406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0036000 tx=0x7ff1d003b040 comp rx=0 tx=0).stop 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 msgr2=0x7ff1e013c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d620 tx=0x7ff1dc00dae0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 shutdown_connections 2024-04-13T03:11:56.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff1c406c8f0 0x7ff1c406edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff1e0145d20 0x7ff1e015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e00b4c20 0x7ff1e01e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 --2- 172.21.15.106:0/1833649478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1e00b47e0 0x7ff1e013c430 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 >> 172.21.15.106:0/1833649478 conn(0x7ff1e00b2600 msgr2=0x7ff1e00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 shutdown_connections 2024-04-13T03:11:56.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.349+0000 7ff1e8008640 1 -- 172.21.15.106:0/1833649478 wait complete. 2024-04-13T03:11:56.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:11:56.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.462+0000 7f8cf7a98640 1 Processor -- start 2024-04-13T03:11:56.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 -- start start 2024-04-13T03:11:56.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf015e9b0 0x7f8cf00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8cf015ed90 0x7f8cf00d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf00cfd20 con 0x7f8cf00b3d60 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf0162af0 con 0x7f8cf015ed90 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.463+0000 7f8cf7a98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf0162dd0 con 0x7f8cf015e9b0 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf6035640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8cf00b3d60 0x7f8cf00d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34154/0 (socket says 172.21.15.106:34154) 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf6035640 1 -- 172.21.15.106:0/191041604 learned_addr learned my addr 172.21.15.106:0/191041604 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472752663 0 0) 0x7f8cf00cfd20 con 0x7f8cf00b3d60 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cdc003520 con 0x7f8cf00b3d60 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5834640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf015e9b0 0x7f8cf00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> v2:172.21.15.114:3300/0 conn(0x7f8cf015ed90 0x7f8cf00d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2937417716 0 0) 0x7f8cdc003520 con 0x7f8cf00b3d60 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf015e9b0 msgr2=0x7f8cf00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf015e9b0 0x7f8cf00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 >> v2:172.21.15.114:3300/0 conn(0x7f8cf015ed90 msgr2=0x7f8cf00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 --2- 172.21.15.106:0/191041604 >> v2:172.21.15.114:3300/0 conn(0x7f8cf015ed90 0x7f8cf00d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01630b0 con 0x7f8cf00b3d60 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f8ce8001a10 con 0x7f8cf00b3d60 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 >> v1:172.21.15.106:6789/0 conn(0x7f8cf00b3d60 legacy=0x7f8cf00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cdc0039b0 0x7f8cdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cdc004800 0x7f8cdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc0051c0 0x7f8cdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0162af0 con 0x7f8cdc004800 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0162dd0 con 0x7f8cdc0051c0 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc00bfe0 con 0x7f8cdc0039b0 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5834640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf015e9b0 0x7f8cf00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:56.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> v2:172.21.15.114:3300/0 conn(0x7f8cf015ed90 0x7f8cf00d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cdc004800 0x7f8cdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf6035640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc0051c0 0x7f8cdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.464+0000 7f8cf5834640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cdc0039b0 0x7f8cdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 -- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cdc0039b0 msgr2=0x7f8cdc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cdc0039b0 0x7f8cdc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 -- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc0051c0 msgr2=0x7f8cdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc0051c0 0x7f8cdc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 -- 172.21.15.106:0/191041604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc003520 con 0x7f8cdc004800 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf5033640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cdc004800 0x7f8cdc004c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8cdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9683ec2f8400fa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000f040 con 0x7f8cdc004800 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce0037600 con 0x7f8cdc004800 2024-04-13T03:11:56.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.465+0000 7f8cf4832640 1 -- 172.21.15.106:0/191041604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00379b0 con 0x7f8cdc004800 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 -- 172.21.15.106:0/191041604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cdc004800 msgr2=0x7f8cdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/191041604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cdc004800 0x7f8cdc004c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8cdc005830 comp rx=0 tx=0).stop 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf6035640 1 -- 172.21.15.106:0/191041604 reap_dead start 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 -- 172.21.15.106:0/191041604 shutdown_connections 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 -- 172.21.15.106:0/191041604 >> 172.21.15.106:0/191041604 conn(0x7f8cf008bf10 msgr2=0x7f8cf00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 -- 172.21.15.106:0/191041604 shutdown_connections 2024-04-13T03:11:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.466+0000 7f8cf7a98640 1 -- 172.21.15.106:0/191041604 wait complete. 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.467+0000 7f8cf7a98640 1 Processor -- start 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.467+0000 7f8cf7a98640 1 -- start start 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc004800 0x7f8cf00b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 0x7f8cf01512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0162800 con 0x7f8cf00b66d0 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0162680 con 0x7f8cdc004800 2024-04-13T03:11:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf7a98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf006eb20 con 0x7f8cf01f8410 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36394/0 (socket says 172.21.15.106:36394) 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 -- 172.21.15.106:0/449777605 learned_addr learned my addr 172.21.15.106:0/449777605 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5834640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc004800 0x7f8cf00b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf6035640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 0x7f8cf01512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 -- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 msgr2=0x7f8cf01512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 0x7f8cf01512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 -- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc004800 msgr2=0x7f8cf00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc004800 0x7f8cf00b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 -- 172.21.15.106:0/449777605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf00b6c50 con 0x7f8cf00b66d0 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf6035640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 0x7f8cf01512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.468+0000 7f8cf5033640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.469+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0048070 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.469+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce0037e30 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.469+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0006260 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.469+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf00b6f30 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.469+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf006c3e0 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.470+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf01f8820 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.470+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ce004b070 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.470+0000 7f8ce67fc640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 0x7f8cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.471+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f8ce00ad820 con 0x7f8cf00b66d0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.471+0000 7f8cf5834640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 0x7f8cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.471+0000 7f8cf5834640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 0x7f8cd406edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8cf006db40 tx=0x7f8cec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.475+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cf01f8820 con 0x7f8cf00b66d0 2024-04-13T03:11:56.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.650+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8cf0068a10 con 0x7f8cf00b66d0 2024-04-13T03:11:56.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.651+0000 7f8ce67fc640 1 -- 172.21.15.106:0/449777605 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f8ce0033030 con 0x7f8cf00b66d0 2024-04-13T03:11:56.652 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:11:56.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 msgr2=0x7f8cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 0x7f8cd406edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8cf006db40 tx=0x7f8cec008040 comp rx=0 tx=0).stop 2024-04-13T03:11:56.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 msgr2=0x7f8cf01f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098a0 tx=0x7f8ce00046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 shutdown_connections 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8cd406c8f0 0x7f8cd406edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8cf01f8410 0x7f8cf01512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cf00b66d0 0x7f8cf01f7ed0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 --2- 172.21.15.106:0/449777605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cdc004800 0x7f8cf00b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 >> 172.21.15.106:0/449777605 conn(0x7f8cf008bf10 msgr2=0x7f8cf00b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 shutdown_connections 2024-04-13T03:11:56.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.653+0000 7f8cf7a98640 1 -- 172.21.15.106:0/449777605 wait complete. 2024-04-13T03:11:56.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:11:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.764+0000 7f688c420640 1 Processor -- start 2024-04-13T03:11:56.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.764+0000 7f688c420640 1 -- start start 2024-04-13T03:11:56.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.765+0000 7f688c420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f68840cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.765+0000 7f688c420640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688406ba10 con 0x7f68840cd580 2024-04-13T03:11:56.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.765+0000 7f688c420640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688406bcf0 con 0x7f68841660f0 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.765+0000 7f688c420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f688406bfd0 con 0x7f68840b32d0 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f688a9bd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f68840cd580 0x7f68840b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34170/0 (socket says 172.21.15.106:34170) 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f688a9bd640 1 -- 172.21.15.106:0/795691318 learned_addr learned my addr 172.21.15.106:0/795691318 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68899bb640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f68840cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859632007 0 0) 0x7f688406ba10 con 0x7f68840cd580 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6870003520 con 0x7f68840cd580 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463388524 0 0) 0x7f688406bcf0 con 0x7f68841660f0 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f688406ba10 con 0x7f68841660f0 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2123331130 0 0) 0x7f6870003520 con 0x7f68840cd580 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 msgr2=0x7f68840cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f68840cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 >> v1:172.21.15.114:6789/0 conn(0x7f68841660f0 legacy=0x7f688406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688406c2b0 con 0x7f68840cd580 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6880001670 con 0x7f68840cd580 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 >> v1:172.21.15.106:6789/0 conn(0x7f68840cd580 legacy=0x7f68840b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6870003b70 0x7f6870003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f6870004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6870004d40 0x7f68700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f688406bfd0 con 0x7f6870003b70 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f687000baf0 con 0x7f6870004d40 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f687000bcf0 con 0x7f68840b32d0 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f688a1bc640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6870003b70 0x7f6870003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f68899bb640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f6870004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.766+0000 7f688a9bd640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6870004d40 0x7f68700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 -- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 msgr2=0x7f6870004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68840b32d0 0x7f6870004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 -- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6870004d40 msgr2=0x7f68700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6870004d40 0x7f68700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 -- 172.21.15.106:0/795691318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6870003520 con 0x7f6870003b70 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a9bd640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6870004d40 0x7f68700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f688a1bc640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6870003b70 0x7f6870003f70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f687c001fd0 tx=0x7f6870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2bf329985e000 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c001bf0 con 0x7f6870003b70 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c003b70 con 0x7f6870003b70 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.767+0000 7f68891ba640 1 -- 172.21.15.106:0/795691318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0050d0 con 0x7f6870003b70 2024-04-13T03:11:56.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 -- 172.21.15.106:0/795691318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6870003b70 msgr2=0x7f6870003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 --2- 172.21.15.106:0/795691318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6870003b70 0x7f6870003f70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f687c001fd0 tx=0x7f6870005830 comp rx=0 tx=0).stop 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688a9bd640 1 -- 172.21.15.106:0/795691318 reap_dead start 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 -- 172.21.15.106:0/795691318 shutdown_connections 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 -- 172.21.15.106:0/795691318 >> 172.21.15.106:0/795691318 conn(0x7f68840b2600 msgr2=0x7f68840d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 -- 172.21.15.106:0/795691318 shutdown_connections 2024-04-13T03:11:56.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.768+0000 7f688c420640 1 -- 172.21.15.106:0/795691318 wait complete. 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.769+0000 7f688c420640 1 Processor -- start 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.769+0000 7f688c420640 1 -- start start 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6870003b70 0x7f688414b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f688415cd10 0x7f6884160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b4570 con 0x7f6884162280 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b43f0 con 0x7f688415cd10 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688c420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68840cd040 con 0x7f6870003b70 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36406/0 (socket says 172.21.15.106:36406) 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 -- 172.21.15.106:0/2701906726 learned_addr learned my addr 172.21.15.106:0/2701906726 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a1bc640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6870003b70 0x7f688414b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f68899bb640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f688415cd10 0x7f6884160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 -- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6870003b70 msgr2=0x7f688414b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6870003b70 0x7f688414b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 -- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f688415cd10 msgr2=0x7f6884160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f688415cd10 0x7f6884160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 -- 172.21.15.106:0/2701906726 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688414bb80 con 0x7f6884162280 2024-04-13T03:11:56.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.770+0000 7f688a9bd640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f68840b5760 tx=0x7f688000e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880010510 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68800106b0 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6884160bf0 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880010aa0 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688415d660 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.771+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688406fd50 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.772+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6880010c40 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.772+0000 7f687affd640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 0x7f686806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.773+0000 7f688a1bc640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 0x7f686806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.773+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f688008b170 con 0x7f6884162280 2024-04-13T03:11:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.773+0000 7f688a1bc640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 0x7f686806ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f687c034000 tx=0x7f687c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:56.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.777+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f688406fd50 con 0x7f6884162280 2024-04-13T03:11:56.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.962+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6884068a10 con 0x7f6884162280 2024-04-13T03:11:56.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.962+0000 7f687affd640 1 -- 172.21.15.106:0/2701906726 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f6880058dc0 con 0x7f6884162280 2024-04-13T03:11:56.963 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:11:56.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 msgr2=0x7f686806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 0x7f686806ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f687c034000 tx=0x7f687c009030 comp rx=0 tx=0).stop 2024-04-13T03:11:56.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 msgr2=0x7f6884162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f68840b5760 tx=0x7f688000e880 comp rx=0 tx=0).stop 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 shutdown_connections 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f686806c920 0x7f686806ede0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6884162280 0x7f6884162730 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f688415cd10 0x7f6884160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 --2- 172.21.15.106:0/2701906726 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6870003b70 0x7f688414b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 >> 172.21.15.106:0/2701906726 conn(0x7f68840b2600 msgr2=0x7f68840b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 shutdown_connections 2024-04-13T03:11:56.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:56.964+0000 7f688c420640 1 -- 172.21.15.106:0/2701906726 wait complete. 2024-04-13T03:11:56.976 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:11:56.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:56.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:11:57.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:11:57.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:11:57.021 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:11:57.045 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:57.046 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:57.046 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:57.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:57.063 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:57.063 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:11:57.063 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:57.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:11:57.101 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:11:57.101 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:11:57.101 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:11:57.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:11:57.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:57.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.240+0000 7ffabc31e640 1 Processor -- start 2024-04-13T03:11:57.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.240+0000 7ffabc31e640 1 -- start start 2024-04-13T03:11:57.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffabc31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffab40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffabc31e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffab40b3d60 0x7ffab40d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffabc31e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab40cfd20 con 0x7ffab415ed90 2024-04-13T03:11:57.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffabc31e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ffab4162af0 con 0x7ffab40b3d60 2024-04-13T03:11:57.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffabc31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffab4162dd0 con 0x7ffab415e9b0 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffab98b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffab415ed90 0x7ffab40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34180/0 (socket says 172.21.15.106:34180) 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.241+0000 7ffab98b9640 1 -- 172.21.15.106:0/64344458 learned_addr learned my addr 172.21.15.106:0/64344458 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba0ba640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffab40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 --2- 172.21.15.106:0/64344458 >> v2:172.21.15.114:3300/0 conn(0x7ffab40b3d60 0x7ffab40d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476386178 0 0) 0x7ffab40cfd20 con 0x7ffab415ed90 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa0003520 con 0x7ffab415ed90 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3498457213 0 0) 0x7ffaa0003520 con 0x7ffab415ed90 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 msgr2=0x7ffab40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffab40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 >> v2:172.21.15.114:3300/0 conn(0x7ffab40b3d60 msgr2=0x7ffab40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 --2- 172.21.15.106:0/64344458 >> v2:172.21.15.114:3300/0 conn(0x7ffab40b3d60 0x7ffab40d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41630b0 con 0x7ffab415ed90 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7ffab0001810 con 0x7ffab415ed90 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 >> v1:172.21.15.106:6789/0 conn(0x7ffab415ed90 legacy=0x7ffab40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa00039b0 0x7ffaa0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffaa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa0004d40 0x7ffaa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4162af0 con 0x7ffaa0004d40 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4162dd0 con 0x7ffaa00039b0 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa000b9c0 con 0x7ffab415e9b0 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa0004d40 0x7ffaa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffab98b9640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffaa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba0ba640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa00039b0 0x7ffaa0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 -- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 msgr2=0x7ffaa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab415e9b0 0x7ffaa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 -- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa00039b0 msgr2=0x7ffaa0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa00039b0 0x7ffaa0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.242+0000 7ffaba8bb640 1 -- 172.21.15.106:0/64344458 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa0003520 con 0x7ffaa0004d40 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.243+0000 7ffaba8bb640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa0004d40 0x7ffaa00051c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009f90 tx=0x7ffaa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c21300a1d2ea7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.243+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac039070 con 0x7ffaa0004d40 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.243+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaac02cc20 con 0x7ffaa0004d40 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.243+0000 7ffab90b8640 1 -- 172.21.15.106:0/64344458 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009c10 con 0x7ffaa0004d40 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 -- 172.21.15.106:0/64344458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa0004d40 msgr2=0x7ffaa00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 --2- 172.21.15.106:0/64344458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa0004d40 0x7ffaa00051c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009f90 tx=0x7ffaa0005830 comp rx=0 tx=0).stop 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffaba8bb640 1 -- 172.21.15.106:0/64344458 reap_dead start 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 -- 172.21.15.106:0/64344458 shutdown_connections 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 -- 172.21.15.106:0/64344458 >> 172.21.15.106:0/64344458 conn(0x7ffab408bf10 msgr2=0x7ffab40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 -- 172.21.15.106:0/64344458 shutdown_connections 2024-04-13T03:11:57.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.244+0000 7ffabc31e640 1 -- 172.21.15.106:0/64344458 wait complete. 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.245+0000 7ffabc31e640 1 Processor -- start 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.245+0000 7ffabc31e640 1 -- start start 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa0004d40 0x7ffab40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab40b5770 0x7ffab4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4162800 con 0x7ffab4152f40 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4162680 con 0x7ffaa0004d40 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffab40d4d00 con 0x7ffab40b5770 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36418/0 (socket says 172.21.15.106:36418) 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 -- 172.21.15.106:0/2352966904 learned_addr learned my addr 172.21.15.106:0/2352966904 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 -- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab40b5770 msgr2=0x7ffab4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab40b5770 0x7ffab4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 -- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa0004d40 msgr2=0x7ffab40caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa0004d40 0x7ffab40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 -- 172.21.15.106:0/2352966904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab40c9a40 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffab98b9640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffab000f940 tx=0x7ffab000fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00072e0 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab0018070 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab40c9d20 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00057a0 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.246+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab40ca2e0 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.247+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab4068a10 con 0x7ffab4152f40 2024-04-13T03:11:57.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.248+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffab0005940 con 0x7ffab4152f40 2024-04-13T03:11:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.248+0000 7ffaaaffd640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 0x7ffa9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.248+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7ffab008bdf0 con 0x7ffab4152f40 2024-04-13T03:11:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.248+0000 7ffaba0ba640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 0x7ffa9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.248+0000 7ffaba0ba640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 0x7ffa9806ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffab40c9520 tx=0x7ffaa4009320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.252+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffab001a310 con 0x7ffab4152f40 2024-04-13T03:11:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.470+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffab40d53a0 con 0x7ffab4152f40 2024-04-13T03:11:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.471+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2352966904 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7ffab005a090 con 0x7ffab4152f40 2024-04-13T03:11:57.472 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:57.472 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 msgr2=0x7ffa9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 0x7ffa9806ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffab40c9520 tx=0x7ffaa4009320 comp rx=0 tx=0).stop 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 msgr2=0x7ffab40b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffab000f940 tx=0x7ffab000fe00 comp rx=0 tx=0).stop 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 shutdown_connections 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffa9806c920 0x7ffa9806ede0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffab40b5770 0x7ffab4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab4152f40 0x7ffab40b5230 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 --2- 172.21.15.106:0/2352966904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffaa0004d40 0x7ffab40caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 >> 172.21.15.106:0/2352966904 conn(0x7ffab408bf10 msgr2=0x7ffab40d3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 shutdown_connections 2024-04-13T03:11:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.473+0000 7ffabc31e640 1 -- 172.21.15.106:0/2352966904 wait complete. 2024-04-13T03:11:57.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:57.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.583+0000 7fa1ea80f640 1 Processor -- start 2024-04-13T03:11:57.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.583+0000 7fa1ea80f640 1 -- start start 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 0x7fa1e41630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1e40d0bc0 0x7fa1e40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40cad60 0x7fa1e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e40cd4d0 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e40cb190 con 0x7fa1e4144f10 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1ea80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e40d1050 con 0x7fa1e40cad60 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1e40d0bc0 0x7fa1e40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa1e40d0bc0 0x7fa1e40ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36426/0 (socket says 172.21.15.106:36426) 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 -- 172.21.15.106:0/2267769394 learned_addr learned my addr 172.21.15.106:0/2267769394 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 -- 172.21.15.106:0/2267769394 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40cad60 msgr2=0x7fa1e40706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 --2- 172.21.15.106:0/2267769394 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40cad60 0x7fa1e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 -- 172.21.15.106:0/2267769394 >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 msgr2=0x7fa1e41630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e3fff640 1 --2- 172.21.15.106:0/2267769394 >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 0x7fa1e41630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 --2- 172.21.15.106:0/2267769394 >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 0x7fa1e41630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e37fe640 1 -- 172.21.15.106:0/2267769394 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e40cd700 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.584+0000 7fa1e3fff640 1 --2- 172.21.15.106:0/2267769394 >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 0x7fa1e41630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.585+0000 7fa1e37fe640 1 --2- 172.21.15.106:0/2267769394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e40d0bc0 0x7fa1e40ca820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4072b00 tx=0x7fa1d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80962892025f447 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.585+0000 7fa1e2ffd640 1 -- 172.21.15.106:0/2267769394 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8034070 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.585+0000 7fa1e2ffd640 1 -- 172.21.15.106:0/2267769394 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8030840 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.585+0000 7fa1e2ffd640 1 -- 172.21.15.106:0/2267769394 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8030c10 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 -- 172.21.15.106:0/2267769394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e40d0bc0 msgr2=0x7fa1e40ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/2267769394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e40d0bc0 0x7fa1e40ca820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4072b00 tx=0x7fa1d802b980 comp rx=0 tx=0).stop 2024-04-13T03:11:57.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 -- 172.21.15.106:0/2267769394 shutdown_connections 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/2267769394 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40cad60 0x7fa1e40706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/2267769394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e40d0bc0 0x7fa1e40ca820 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/2267769394 >> v2:172.21.15.114:3300/0 conn(0x7fa1e4144f10 0x7fa1e41630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 -- 172.21.15.106:0/2267769394 >> 172.21.15.106:0/2267769394 conn(0x7fa1e406f180 msgr2=0x7fa1e406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 -- 172.21.15.106:0/2267769394 shutdown_connections 2024-04-13T03:11:57.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.586+0000 7fa1ea80f640 1 -- 172.21.15.106:0/2267769394 wait complete. 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.587+0000 7fa1ea80f640 1 Processor -- start 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.587+0000 7fa1ea80f640 1 -- start start 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1e40cad60 0x7fa1e415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40d0bc0 0x7fa1e4145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40cf120 con 0x7fa1e4144f10 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40cdcc0 con 0x7fa1e40cad60 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1ea80f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e406c150 con 0x7fa1e40d0bc0 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36440/0 (socket says 172.21.15.106:36440) 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 -- 172.21.15.106:0/179901841 learned_addr learned my addr 172.21.15.106:0/179901841 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:57.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e37fe640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40d0bc0 0x7fa1e4145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e3fff640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1e40cad60 0x7fa1e415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 -- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40d0bc0 msgr2=0x7fa1e4145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40d0bc0 0x7fa1e4145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 -- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1e40cad60 msgr2=0x7fa1e415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1e40cad60 0x7fa1e415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 -- 172.21.15.106:0/179901841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e40b5990 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.588+0000 7fa1e8dac640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00b9a0 tx=0x7fa1dc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.589+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc019070 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.589+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1dc00cbc0 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.589+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc014440 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.589+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e40b5c70 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.589+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e40b6230 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.590+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e40cce30 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.590+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1dc01c070 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.590+0000 7fa1e0ff9640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.591+0000 7fa1e3fff640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.591+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa1dc08ba00 con 0x7fa1e4144f10 2024-04-13T03:11:57.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.591+0000 7fa1e3fff640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 0x7fa1d006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40b1990 tx=0x7fa1d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.595+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1e40cce30 con 0x7fa1e4144f10 2024-04-13T03:11:57.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.770+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1e4068a10 con 0x7fa1e4144f10 2024-04-13T03:11:57.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.771+0000 7fa1e0ff9640 1 -- 172.21.15.106:0/179901841 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16978 (secure 0 0 0) 0x7fa1dc059ca0 con 0x7fa1e4144f10 2024-04-13T03:11:57.772 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:57.773 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":52,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:18.293562+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 msgr2=0x7fa1d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 0x7fa1d006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40b1990 tx=0x7fa1d4008040 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 msgr2=0x7fa1e41f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00b9a0 tx=0x7fa1dc00be60 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 shutdown_connections 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1e40d0bc0 0x7fa1e4145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1e4144f10 0x7fa1e41f4ad0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 --2- 172.21.15.106:0/179901841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1e40cad60 0x7fa1e415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 >> 172.21.15.106:0/179901841 conn(0x7fa1e406f180 msgr2=0x7fa1e406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 shutdown_connections 2024-04-13T03:11:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.773+0000 7fa1ea80f640 1 -- 172.21.15.106:0/179901841 wait complete. 2024-04-13T03:11:57.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:11:14.853009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:11:15.586983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:11:57.786 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:11:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.888+0000 7f12b89fa640 1 Processor -- start 2024-04-13T03:11:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.888+0000 7f12b89fa640 1 -- start start 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b89fa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 0x7f12b415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b89fa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f12b40704a0 0x7f12b40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b89fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b415ddd0 con 0x7f12b40d1a60 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b89fa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f12b415e100 con 0x7f12b40704a0 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b89fa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f12b415d4d0 con 0x7f12b40cfea0 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b1d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f12b40704a0 0x7f12b40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f12b40d1a60 0x7f12b4160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34186/0 (socket says 172.21.15.106:34186) 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b2d9d640 1 -- 172.21.15.106:0/1283951975 learned_addr learned my addr 172.21.15.106:0/1283951975 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b259c640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 0x7f12b415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b1d9b640 1 -- 172.21.15.106:0/1283951975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 msgr2=0x7f12b415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b1d9b640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 0x7f12b415cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b1d9b640 1 -- 172.21.15.106:0/1283951975 >> v1:172.21.15.106:6789/0 conn(0x7f12b40d1a60 legacy=0x7f12b4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b1d9b640 1 -- 172.21.15.106:0/1283951975 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b415d7b0 con 0x7f12b40704a0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b159a640 1 -- 172.21.15.106:0/1283951975 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782000240 0 0) 0x7f12b415ddd0 con 0x7f12b40d1a60 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.889+0000 7f12b259c640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 0x7f12b415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.890+0000 7f12b1d9b640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40704a0 0x7f12b40d1520 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097a0 tx=0x7f12a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335a9019afeb6bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.890+0000 7f12b159a640 1 -- 172.21.15.106:0/1283951975 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8035020 con 0x7f12b40704a0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.890+0000 7f12b159a640 1 -- 172.21.15.106:0/1283951975 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a8031830 con 0x7f12b40704a0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.890+0000 7f12b159a640 1 -- 172.21.15.106:0/1283951975 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8031c00 con 0x7f12b40704a0 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 -- 172.21.15.106:0/1283951975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40704a0 msgr2=0x7f12b40d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40704a0 0x7f12b40d1520 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097a0 tx=0x7f12a802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 -- 172.21.15.106:0/1283951975 shutdown_connections 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40704a0 0x7f12b40d1520 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 --2- 172.21.15.106:0/1283951975 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40cfea0 0x7f12b415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 -- 172.21.15.106:0/1283951975 >> 172.21.15.106:0/1283951975 conn(0x7f12b4071870 msgr2=0x7f12b4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:57.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 -- 172.21.15.106:0/1283951975 shutdown_connections 2024-04-13T03:11:57.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.891+0000 7f12b89fa640 1 -- 172.21.15.106:0/1283951975 wait complete. 2024-04-13T03:11:57.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.892+0000 7f12b89fa640 1 Processor -- start 2024-04-13T03:11:57.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.892+0000 7f12b89fa640 1 -- start start 2024-04-13T03:11:57.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40704a0 0x7f12b4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 0x7f12b4143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f12b415d150 con 0x7f12b40d1a60 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40d1ef0 con 0x7f12b40cfea0 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b89fa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f12b415f5a0 con 0x7f12b40704a0 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36462/0 (socket says 172.21.15.106:36462) 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 -- 172.21.15.106:0/2045212638 learned_addr learned my addr 172.21.15.106:0/2045212638 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b259c640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40704a0 0x7f12b4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b1d9b640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 0x7f12b4143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 -- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40704a0 msgr2=0x7f12b4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40704a0 0x7f12b4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 -- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 msgr2=0x7f12b4143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 0x7f12b4143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 -- 172.21.15.106:0/2045212638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b4159ac0 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b1d9b640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 0x7f12b4143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.893+0000 7f12b2d9d640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f12ac001eb0 tx=0x7f12ac00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.894+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0074e0 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.894+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac0047e0 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.894+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b4144380 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.894+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac004bb0 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.894+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b4158060 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.895+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4068a10 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.895+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12ac011070 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.895+0000 7f12a2ffd640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 0x7f129406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.895+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f12ac08b7d0 con 0x7f12b40d1a60 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.895+0000 7f12b259c640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 0x7f129406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.896+0000 7f12b259c640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 0x7f129406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4003b80 tx=0x7f12a4009910 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:57.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:57.900+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12ac059a70 con 0x7f12b40d1a60 2024-04-13T03:11:58.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.072+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12b415b770 con 0x7f12b40d1a60 2024-04-13T03:11:58.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.072+0000 7f12a2ffd640 1 -- 172.21.15.106:0/2045212638 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f12ac059370 con 0x7f12b40d1a60 2024-04-13T03:11:58.073 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:11:58.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 msgr2=0x7f129406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 0x7f129406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4003b80 tx=0x7f12a4009910 comp rx=0 tx=0).stop 2024-04-13T03:11:58.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 msgr2=0x7f12b414f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f12ac001eb0 tx=0x7f12ac00c870 comp rx=0 tx=0).stop 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 shutdown_connections 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f129406c8f0 0x7f129406edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12b40d1a60 0x7f12b414f380 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b40cfea0 0x7f12b4143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 --2- 172.21.15.106:0/2045212638 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12b40704a0 0x7f12b4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 >> 172.21.15.106:0/2045212638 conn(0x7f12b4071870 msgr2=0x7f12b415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 shutdown_connections 2024-04-13T03:11:58.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.074+0000 7f12b89fa640 1 -- 172.21.15.106:0/2045212638 wait complete. 2024-04-13T03:11:58.086 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:11:58.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.187+0000 7fc61c555640 1 Processor -- start 2024-04-13T03:11:58.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.187+0000 7fc61c555640 1 -- start start 2024-04-13T03:11:58.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61c555640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc6140cd260 0x7fc6140c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61c555640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc6140d1c30 0x7fc614160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61c555640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140d21d0 con 0x7fc61415c5f0 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61c555640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140cd8b0 con 0x7fc6140d1c30 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61c555640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc614161330 con 0x7fc6140cd260 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61aaf2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc6140d1c30 0x7fc614160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61aaf2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc6140d1c30 0x7fc614160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36472/0 (socket says 172.21.15.106:36472) 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61aaf2640 1 -- 172.21.15.106:0/2111841609 learned_addr learned my addr 172.21.15.106:0/2111841609 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61aaf2640 1 -- 172.21.15.106:0/2111841609 >> v2:172.21.15.114:3300/0 conn(0x7fc6140cd260 msgr2=0x7fc6140c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.188+0000 7fc61aaf2640 1 --2- 172.21.15.106:0/2111841609 >> v2:172.21.15.114:3300/0 conn(0x7fc6140cd260 0x7fc6140c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc61aaf2640 1 -- 172.21.15.106:0/2111841609 >> v1:172.21.15.114:6789/0 conn(0x7fc61415c5f0 legacy=0x7fc6140ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:58.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc61aaf2640 1 -- 172.21.15.106:0/2111841609 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140b1f40 con 0x7fc6140d1c30 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc61aaf2640 1 --2- 172.21.15.106:0/2111841609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140d1c30 0x7fc614160df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b27e0 tx=0x7fc60c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda009e2755536cb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc6192ef640 1 -- 172.21.15.106:0/2111841609 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c02ca80 con 0x7fc6140d1c30 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc6192ef640 1 -- 172.21.15.106:0/2111841609 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c02cc20 con 0x7fc6140d1c30 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.189+0000 7fc6192ef640 1 -- 172.21.15.106:0/2111841609 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0343f0 con 0x7fc6140d1c30 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 -- 172.21.15.106:0/2111841609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140d1c30 msgr2=0x7fc614160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 --2- 172.21.15.106:0/2111841609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140d1c30 0x7fc614160df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b27e0 tx=0x7fc60c02bee0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 -- 172.21.15.106:0/2111841609 shutdown_connections 2024-04-13T03:11:58.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 --2- 172.21.15.106:0/2111841609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140d1c30 0x7fc614160df0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 --2- 172.21.15.106:0/2111841609 >> v2:172.21.15.114:3300/0 conn(0x7fc6140cd260 0x7fc6140c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 -- 172.21.15.106:0/2111841609 >> 172.21.15.106:0/2111841609 conn(0x7fc6140cef80 msgr2=0x7fc6140cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 -- 172.21.15.106:0/2111841609 shutdown_connections 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.190+0000 7fc61c555640 1 -- 172.21.15.106:0/2111841609 wait complete. 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.191+0000 7fc61c555640 1 Processor -- start 2024-04-13T03:11:58.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.191+0000 7fc61c555640 1 -- start start 2024-04-13T03:11:58.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 0x7fc6140cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 0x7fc614151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140b2500 con 0x7fc6140cd260 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140b2870 con 0x7fc61415c5f0 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61c555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc61415c9d0 con 0x7fc6140d1c30 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36482/0 (socket says 172.21.15.106:36482) 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 -- 172.21.15.106:0/1183968018 learned_addr learned my addr 172.21.15.106:0/1183968018 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc619af0640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 0x7fc6140cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61aaf2640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 0x7fc614151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 -- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 msgr2=0x7fc6140cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 0x7fc6140cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 -- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 msgr2=0x7fc614151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 0x7fc614151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 -- 172.21.15.106:0/1183968018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140cc990 con 0x7fc6140cd260 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc619af0640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 0x7fc6140cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61aaf2640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 0x7fc614151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:11:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.192+0000 7fc61a2f1640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b3e0 tx=0x7fc61000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.193+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610015020 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.193+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc610011850 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.193+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6140a9490 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.193+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610011c40 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.193+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6140c9e40 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.194+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614068a10 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.194+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc610018020 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.194+0000 7fc60affd640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 0x7fc5f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.195+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fc61008bc00 con 0x7fc6140cd260 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.195+0000 7fc619af0640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 0x7fc5f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.195+0000 7fc619af0640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 0x7fc5f806ede0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b3c60 tx=0x7fc604008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.199+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc610092080 con 0x7fc6140cd260 2024-04-13T03:11:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.371+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc614072900 con 0x7fc6140cd260 2024-04-13T03:11:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.371+0000 7fc60affd640 1 -- 172.21.15.106:0/1183968018 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7fc61001f020 con 0x7fc6140cd260 2024-04-13T03:11:58.372 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:11:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 msgr2=0x7fc5f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 0x7fc5f806ede0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b3c60 tx=0x7fc604008040 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 msgr2=0x7fc6141f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b3e0 tx=0x7fc61000b8a0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 shutdown_connections 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc5f806c920 0x7fc5f806ede0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6140d1c30 0x7fc6140cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc61415c5f0 0x7fc614151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 --2- 172.21.15.106:0/1183968018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6140cd260 0x7fc6141f6c20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 >> 172.21.15.106:0/1183968018 conn(0x7fc6140cef80 msgr2=0x7fc6140c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 shutdown_connections 2024-04-13T03:11:58.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.373+0000 7fc61c555640 1 -- 172.21.15.106:0/1183968018 wait complete. 2024-04-13T03:11:58.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:11:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.487+0000 7f83593a5640 1 Processor -- start 2024-04-13T03:11:58.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.487+0000 7f83593a5640 1 -- start start 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83593a5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f83540d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83593a5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83540d24d0 con 0x7f83540d1e50 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83593a5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835415eeb0 con 0x7f83540d4e00 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83593a5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83540d1060 con 0x7f835415ead0 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83527fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f83540d1e50 0x7f83540d46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34196/0 (socket says 172.21.15.106:34196) 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f83527fc640 1 -- 172.21.15.106:0/2911810910 learned_addr learned my addr 172.21.15.106:0/2911810910 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f8352ffd640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f83540d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123318142 0 0) 0x7f83540d24d0 con 0x7f83540d1e50 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f833c003520 con 0x7f83540d1e50 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.488+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2270253565 0 0) 0x7f833c003520 con 0x7f83540d1e50 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 msgr2=0x7f83540d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:58.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f83540d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 >> v1:172.21.15.114:6789/0 conn(0x7f83540d4e00 legacy=0x7f83540d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83540d2770 con 0x7f83540d1e50 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f8348001670 con 0x7f83540d1e50 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 >> v1:172.21.15.106:6789/0 conn(0x7f83540d1e50 legacy=0x7f83540d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f833c0039e0 0x7f833c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f833c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c004d40 0x7f833c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83540d1060 con 0x7f833c004d40 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f833c00b9c0 con 0x7f833c0039e0 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f833c00bbc0 con 0x7f835415ead0 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c004d40 0x7f833c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f8352ffd640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f833c0039e0 0x7f833c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83527fc640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f833c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 -- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 msgr2=0x7f833c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f835415ead0 0x7f833c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 -- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f833c0039e0 msgr2=0x7f833c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f833c0039e0 0x7f833c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.489+0000 7f83537fe640 1 -- 172.21.15.106:0/2911810910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c003520 con 0x7f833c004d40 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83537fe640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c004d40 0x7f833c0051c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f834c001910 tx=0x7f833c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0367fb7331e416d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c0014e0 con 0x7f833c004d40 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c037070 con 0x7f833c004d40 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f8351ffb640 1 -- 172.21.15.106:0/2911810910 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c032840 con 0x7f833c004d40 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83593a5640 1 -- 172.21.15.106:0/2911810910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c004d40 msgr2=0x7f833c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83593a5640 1 --2- 172.21.15.106:0/2911810910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c004d40 0x7f833c0051c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f834c001910 tx=0x7f833c005830 comp rx=0 tx=0).stop 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83537fe640 1 -- 172.21.15.106:0/2911810910 reap_dead start 2024-04-13T03:11:58.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83593a5640 1 -- 172.21.15.106:0/2911810910 shutdown_connections 2024-04-13T03:11:58.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.490+0000 7f83593a5640 1 -- 172.21.15.106:0/2911810910 >> 172.21.15.106:0/2911810910 conn(0x7f835409da70 msgr2=0x7f83540b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.491+0000 7f83593a5640 1 -- 172.21.15.106:0/2911810910 shutdown_connections 2024-04-13T03:11:58.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.491+0000 7f83593a5640 1 -- 172.21.15.106:0/2911810910 wait complete. 2024-04-13T03:11:58.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.492+0000 7f83593a5640 1 Processor -- start 2024-04-13T03:11:58.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.492+0000 7f83593a5640 1 -- start start 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83540a1cc0 0x7f83540ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83540a2140 0x7f83540ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83540b12f0 con 0x7f835409ff80 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83540d2d30 con 0x7f83540a2140 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83593a5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83540b1470 con 0x7f83540a1cc0 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36502/0 (socket says 172.21.15.106:36502) 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 -- 172.21.15.106:0/1046861918 learned_addr learned my addr 172.21.15.106:0/1046861918 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83537fe640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83540a2140 0x7f83540ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f83527fc640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83540a1cc0 0x7f83540ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 -- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83540a1cc0 msgr2=0x7f83540ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83540a1cc0 0x7f83540ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 -- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83540a2140 msgr2=0x7f83540ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83540a2140 0x7f83540ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 -- 172.21.15.106:0/1046861918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541f7e50 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f8352ffd640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8344011e30 tx=0x7f834400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.493+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344015630 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.494+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83440157d0 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.494+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344015ba0 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.494+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83541f8130 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.494+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541f86f0 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.494+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8354068a10 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.495+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f834401d400 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.496+0000 7f833b7fe640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 0x7f834006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.496+0000 7f83527fc640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 0x7f834006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.496+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f834408cd10 con 0x7f835409ff80 2024-04-13T03:11:58.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.496+0000 7f83527fc640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 0x7f834006ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83540ce530 tx=0x7f8348001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.501+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8344019030 con 0x7f835409ff80 2024-04-13T03:11:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.674+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f835415eeb0 con 0x7f835409ff80 2024-04-13T03:11:58.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.675+0000 7f833b7fe640 1 -- 172.21.15.106:0/1046861918 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f8344010040 con 0x7f835409ff80 2024-04-13T03:11:58.676 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 msgr2=0x7f834006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 0x7f834006ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83540ce530 tx=0x7f8348001170 comp rx=0 tx=0).stop 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 msgr2=0x7f83540a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8344011e30 tx=0x7f834400f580 comp rx=0 tx=0).stop 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 shutdown_connections 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f834006c920 0x7f834006ede0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83540a1cc0 0x7f83540ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83540a2140 0x7f83540ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 --2- 172.21.15.106:0/1046861918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f835409ff80 0x7f83540a0360 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 >> 172.21.15.106:0/1046861918 conn(0x7f835409da70 msgr2=0x7f835406d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 shutdown_connections 2024-04-13T03:11:58.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.677+0000 7f83593a5640 1 -- 172.21.15.106:0/1046861918 wait complete. 2024-04-13T03:11:58.688 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:58.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.787+0000 7f0b4cb5e640 1 Processor -- start 2024-04-13T03:11:58.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.787+0000 7f0b4cb5e640 1 -- start start 2024-04-13T03:11:58.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4cb5e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b4815ce90 0x7f0b480b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4cb5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480aefc0 con 0x7f0b480aebe0 2024-04-13T03:11:58.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4cb5e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480ad180 con 0x7f0b480b58e0 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4cb5e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480b5d30 con 0x7f0b4815ce90 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b45d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b4815ce90 0x7f0b480b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b480aebe0 0x7f0b481603a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34200/0 (socket says 172.21.15.106:34200) 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4659c640 1 -- 172.21.15.106:0/1602927004 learned_addr learned my addr 172.21.15.106:0/1602927004 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4559a640 1 -- 172.21.15.106:0/1602927004 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735894346 0 0) 0x7f0b38001110 con 0x7f0b480aebe0 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b4559a640 1 -- 172.21.15.106:0/1602927004 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b30003520 con 0x7f0b480aebe0 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b45d9b640 1 -- 172.21.15.106:0/1602927004 >> v1:172.21.15.114:6789/0 conn(0x7f0b480b58e0 legacy=0x7f0b4806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b45d9b640 1 -- 172.21.15.106:0/1602927004 >> v1:172.21.15.106:6789/0 conn(0x7f0b480aebe0 legacy=0x7f0b481603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.788+0000 7f0b45d9b640 1 -- 172.21.15.106:0/1602927004 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b48160ca0 con 0x7f0b4815ce90 2024-04-13T03:11:58.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.789+0000 7f0b45d9b640 1 --2- 172.21.15.106:0/1602927004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4815ce90 0x7f0b480b53a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409eec1980c294c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.789+0000 7f0b4559a640 1 -- 172.21.15.106:0/1602927004 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c035020 con 0x7f0b4815ce90 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.789+0000 7f0b4559a640 1 -- 172.21.15.106:0/1602927004 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c031830 con 0x7f0b4815ce90 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.789+0000 7f0b4559a640 1 -- 172.21.15.106:0/1602927004 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c031c00 con 0x7f0b4815ce90 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/1602927004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4815ce90 msgr2=0x7f0b480b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/1602927004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4815ce90 0x7f0b480b53a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/1602927004 shutdown_connections 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/1602927004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4815ce90 0x7f0b480b53a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/1602927004 >> 172.21.15.106:0/1602927004 conn(0x7f0b480b1f10 msgr2=0x7f0b480b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/1602927004 shutdown_connections 2024-04-13T03:11:58.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.790+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/1602927004 wait complete. 2024-04-13T03:11:58.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.791+0000 7f0b4cb5e640 1 Processor -- start 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.791+0000 7f0b4cb5e640 1 -- start start 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b480aebe0 0x7f0b48156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4815ce90 0x7f0b4815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b48161260 con 0x7f0b480b58e0 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4806e180 con 0x7f0b4815ce90 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4cb5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4806de10 con 0x7f0b480aebe0 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36506/0 (socket says 172.21.15.106:36506) 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 -- 172.21.15.106:0/3772025163 learned_addr learned my addr 172.21.15.106:0/3772025163 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b46d9d640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4815ce90 0x7f0b4815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b4659c640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b480aebe0 0x7f0b48156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 -- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b480aebe0 msgr2=0x7f0b48156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b480aebe0 0x7f0b48156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 -- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4815ce90 msgr2=0x7f0b4815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4815ce90 0x7f0b4815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 -- 172.21.15.106:0/3772025163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b480c92c0 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b45d9b640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.792+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c035040 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.793+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c004230 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.793+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b480c95a0 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.793+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c03a640 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.793+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481568d0 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.793+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b48068a10 con 0x7f0b480b58e0 2024-04-13T03:11:58.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.794+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b3c038020 con 0x7f0b480b58e0 2024-04-13T03:11:58.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.794+0000 7f0b36ffd640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 0x7f0b2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:58.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.794+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0b3c0ada20 con 0x7f0b480b58e0 2024-04-13T03:11:58.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.794+0000 7f0b4659c640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 0x7f0b2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:58.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.795+0000 7f0b4659c640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 0x7f0b2806edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b38007fd0 tx=0x7f0b38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:58.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:58.799+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b3c07bcc0 con 0x7f0b480b58e0 2024-04-13T03:11:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.009+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b4806ba20 con 0x7f0b480b58e0 2024-04-13T03:11:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.010+0000 7f0b36ffd640 1 -- 172.21.15.106:0/3772025163 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f0b3c07b5c0 con 0x7f0b480b58e0 2024-04-13T03:11:59.011 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 msgr2=0x7f0b2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 0x7f0b2806edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b38007fd0 tx=0x7f0b38009040 comp rx=0 tx=0).stop 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 msgr2=0x7f0b4815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 shutdown_connections 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2806c8f0 0x7f0b2806edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4815ce90 0x7f0b4815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b480b58e0 0x7f0b4815a3d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 --2- 172.21.15.106:0/3772025163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b480aebe0 0x7f0b48156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 >> 172.21.15.106:0/3772025163 conn(0x7f0b480b1f10 msgr2=0x7f0b4806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 shutdown_connections 2024-04-13T03:11:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.012+0000 7f0b4cb5e640 1 -- 172.21.15.106:0/3772025163 wait complete. 2024-04-13T03:11:59.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:11:59.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.122+0000 7f953c036640 1 Processor -- start 2024-04-13T03:11:59.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.122+0000 7f953c036640 1 -- start start 2024-04-13T03:11:59.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953c036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95340d1bf0 0x7f95340b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953c036640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953406fd00 con 0x7f953415caa0 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953c036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953406ffe0 con 0x7f953415ce80 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953c036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f95340714b0 con 0x7f95340d1bf0 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f9539dd2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f953415caa0 0x7f9534070da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34202/0 (socket says 172.21.15.106:34202) 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f9539dd2640 1 -- 172.21.15.106:0/3806759657 learned_addr learned my addr 172.21.15.106:0/3806759657 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953a5d3640 1 --2- 172.21.15.106:0/3806759657 >> v2:172.21.15.106:3300/0 conn(0x7f95340d1bf0 0x7f95340b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f9538dd0640 1 -- 172.21.15.106:0/3806759657 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551272001 0 0) 0x7f953406fd00 con 0x7f953415caa0 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f9538dd0640 1 -- 172.21.15.106:0/3806759657 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9520003520 con 0x7f953415caa0 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953a5d3640 1 -- 172.21.15.106:0/3806759657 >> v1:172.21.15.114:6789/0 conn(0x7f953415ce80 legacy=0x7f95340d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953a5d3640 1 -- 172.21.15.106:0/3806759657 >> v1:172.21.15.106:6789/0 conn(0x7f953415caa0 legacy=0x7f9534070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.123+0000 7f953a5d3640 1 -- 172.21.15.106:0/3806759657 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340702c0 con 0x7f95340d1bf0 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.124+0000 7f953a5d3640 1 --2- 172.21.15.106:0/3806759657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340d1bf0 0x7f95340b5780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9534071630 tx=0x7f952c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd8893f0f0affa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.124+0000 7f9538dd0640 1 -- 172.21.15.106:0/3806759657 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c039070 con 0x7f95340d1bf0 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.124+0000 7f9538dd0640 1 -- 172.21.15.106:0/3806759657 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c02cce0 con 0x7f95340d1bf0 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.124+0000 7f9538dd0640 1 -- 172.21.15.106:0/3806759657 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c034660 con 0x7f95340d1bf0 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 -- 172.21.15.106:0/3806759657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340d1bf0 msgr2=0x7f95340b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 --2- 172.21.15.106:0/3806759657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340d1bf0 0x7f95340b5780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9534071630 tx=0x7f952c02bf30 comp rx=0 tx=0).stop 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 -- 172.21.15.106:0/3806759657 shutdown_connections 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 --2- 172.21.15.106:0/3806759657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340d1bf0 0x7f95340b5780 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 -- 172.21.15.106:0/3806759657 >> 172.21.15.106:0/3806759657 conn(0x7f95340d25d0 msgr2=0x7f95340d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 -- 172.21.15.106:0/3806759657 shutdown_connections 2024-04-13T03:11:59.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.125+0000 7f953c036640 1 -- 172.21.15.106:0/3806759657 wait complete. 2024-04-13T03:11:59.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.126+0000 7f953c036640 1 Processor -- start 2024-04-13T03:11:59.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.126+0000 7f953c036640 1 -- start start 2024-04-13T03:11:59.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f95340d1bf0 0x7f953406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f953415caa0 0x7f95340b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f953406f990 con 0x7f95340b2da0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9534070880 con 0x7f953415caa0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953c036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f95340b7f70 con 0x7f95340d1bf0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36520/0 (socket says 172.21.15.106:36520) 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 -- 172.21.15.106:0/3940376164 learned_addr learned my addr 172.21.15.106:0/3940376164 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f9539dd2640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f95340d1bf0 0x7f953406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f95395d1640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f953415caa0 0x7f95340b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 -- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f95340d1bf0 msgr2=0x7f953406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f95340d1bf0 0x7f953406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 -- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f953415caa0 msgr2=0x7f95340b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f953415caa0 0x7f95340b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 -- 172.21.15.106:0/3940376164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340c84d0 con 0x7f95340b2da0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.127+0000 7f953a5d3640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e80 tx=0x7f952c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c039050 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c045070 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c009450 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340c87b0 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95340c81d0 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.128+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534068a10 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.129+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f952c03e850 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.129+0000 7f952a7fc640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.129+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f952c0ae870 con 0x7f95340b2da0 2024-04-13T03:11:59.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.130+0000 7f9539dd2640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.130+0000 7f9539dd2640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 0x7f951806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f95300011e0 tx=0x7f9530009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.134+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f952c07cba0 con 0x7f95340b2da0 2024-04-13T03:11:59.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.309+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f953406f5c0 con 0x7f95340b2da0 2024-04-13T03:11:59.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.310+0000 7f952a7fc640 1 -- 172.21.15.106:0/3940376164 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16978 (secure 0 0 0) 0x7f952c07c4a0 con 0x7f95340b2da0 2024-04-13T03:11:59.312 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:59.312 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":52,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:18.293562+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:11:59.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 msgr2=0x7f951806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 0x7f951806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f95300011e0 tx=0x7f9530009040 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 msgr2=0x7f95340c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e80 tx=0x7f952c004a40 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 shutdown_connections 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95340b2da0 0x7f95340c60c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f953415caa0 0x7f95340b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 --2- 172.21.15.106:0/3940376164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f95340d1bf0 0x7f953406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 >> 172.21.15.106:0/3940376164 conn(0x7f95340d25d0 msgr2=0x7f953415bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 shutdown_connections 2024-04-13T03:11:59.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.312+0000 7f953c036640 1 -- 172.21.15.106:0/3940376164 wait complete. 2024-04-13T03:11:59.324 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:11:59.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:59.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.424+0000 7f111d8c0640 1 Processor -- start 2024-04-13T03:11:59.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.424+0000 7f111d8c0640 1 -- start start 2024-04-13T03:11:59.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f111d8c0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180b5400 0x7f11180b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f111d8c0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f111d8c0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11180d0890 con 0x7f11180ce580 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f111d8c0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f11180d1be0 con 0x7f11180b5400 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f111d8c0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f11180d0b40 con 0x7f111815e220 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f11167fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180b5400 0x7f11180b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f11177fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f11167fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180b5400 0x7f11180b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:48488/0 (socket says 172.21.15.106:48488) 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f11177fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:51672/0 (socket says 172.21.15.106:51672) 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f11167fc640 1 -- 172.21.15.106:0/3452565919 learned_addr learned my addr 172.21.15.106:0/3452565919 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.425+0000 7f1116ffd640 1 --1- 172.21.15.106:0/3452565919 >> v1:172.21.15.114:6789/0 conn(0x7f11180ce580 0x7f11180d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36226/0 (socket says 172.21.15.106:36226) 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11167fc640 1 -- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 msgr2=0x7f111815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11167fc640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11167fc640 1 -- 172.21.15.106:0/3452565919 >> v1:172.21.15.114:6789/0 conn(0x7f11180ce580 legacy=0x7f11180d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11167fc640 1 -- 172.21.15.106:0/3452565919 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180d0e20 con 0x7f11180b5400 2024-04-13T03:11:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11177fe640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.426+0000 7f11167fc640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180b5400 0x7f11180b5890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f110c009870 tx=0x7f110c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09c58e4ceb247ab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f1115ffb640 1 -- 172.21.15.106:0/3452565919 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c034070 con 0x7f11180b5400 2024-04-13T03:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f1115ffb640 1 -- 172.21.15.106:0/3452565919 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c030840 con 0x7f11180b5400 2024-04-13T03:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f1115ffb640 1 -- 172.21.15.106:0/3452565919 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c030c10 con 0x7f11180b5400 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 -- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180b5400 msgr2=0x7f11180b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180b5400 0x7f11180b5890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f110c009870 tx=0x7f110c02b980 comp rx=0 tx=0).stop 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 -- 172.21.15.106:0/3452565919 shutdown_connections 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f111815e220 0x7f111815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 --2- 172.21.15.106:0/3452565919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180b5400 0x7f11180b5890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 -- 172.21.15.106:0/3452565919 >> 172.21.15.106:0/3452565919 conn(0x7f11180d25b0 msgr2=0x7f11180d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 -- 172.21.15.106:0/3452565919 shutdown_connections 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.427+0000 7f111d8c0640 1 -- 172.21.15.106:0/3452565919 wait complete. 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.428+0000 7f111d8c0640 1 Processor -- start 2024-04-13T03:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 -- start start 2024-04-13T03:11:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180b5400 0x7f1118151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f111815e220 0x7f11180c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11180d16c0 con 0x7f11180ce580 2024-04-13T03:11:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f11180d13e0 con 0x7f111815e220 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f111d8c0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f11180ce960 con 0x7f11180b5400 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f11167fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f11167fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36536/0 (socket says 172.21.15.106:36536) 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.429+0000 7f11167fc640 1 -- 172.21.15.106:0/1951848651 learned_addr learned my addr 172.21.15.106:0/1951848651 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f1116ffd640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180b5400 0x7f1118151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11177fe640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f111815e220 0x7f11180c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 -- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180b5400 msgr2=0x7f1118151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180b5400 0x7f1118151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 -- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f111815e220 msgr2=0x7f11180c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f111815e220 0x7f11180c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 -- 172.21.15.106:0/1951848651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180c60a0 con 0x7f11180ce580 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f11167fc640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c00 tx=0x7f110c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c048070 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118151980 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c030640 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c034040 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.430+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1118151f40 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.431+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1118074750 con 0x7f11180ce580 2024-04-13T03:11:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.431+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f110c037070 con 0x7f11180ce580 2024-04-13T03:11:59.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.432+0000 7f10fffff640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.432+0000 7f1116ffd640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.432+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f110c0ad930 con 0x7f11180ce580 2024-04-13T03:11:59.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.432+0000 7f1116ffd640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f11080012e0 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.436+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f110c07bd00 con 0x7f11180ce580 2024-04-13T03:11:59.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.649+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1118068a10 con 0x7f11180ce580 2024-04-13T03:11:59.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.650+0000 7f10fffff640 1 -- 172.21.15.106:0/1951848651 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f110c07b600 con 0x7f11180ce580 2024-04-13T03:11:59.651 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:59.651 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:59.651 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:59.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 msgr2=0x7f110406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f11080012e0 tx=0x7f1108009040 comp rx=0 tx=0).stop 2024-04-13T03:11:59.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 msgr2=0x7f11180c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c00 tx=0x7f110c004730 comp rx=0 tx=0).stop 2024-04-13T03:11:59.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 shutdown_connections 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f111815e220 0x7f11180c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180ce580 0x7f11180c6860 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 --2- 172.21.15.106:0/1951848651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180b5400 0x7f1118151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 >> 172.21.15.106:0/1951848651 conn(0x7f11180d25b0 msgr2=0x7f11180d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 shutdown_connections 2024-04-13T03:11:59.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.652+0000 7f111d8c0640 1 -- 172.21.15.106:0/1951848651 wait complete. 2024-04-13T03:11:59.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:11:59.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.762+0000 7f4d64ec8640 1 Processor -- start 2024-04-13T03:11:59.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.762+0000 7f4d64ec8640 1 -- start start 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d64ec8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d64ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d64ec8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d600ce6f0 con 0x7f4d600ce1f0 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d64ec8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6015fa70 con 0x7f4d6015ebf0 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d64ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6015f420 con 0x7f4d600b5bf0 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.763+0000 7f4d5e59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5e59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:48512/0 (socket says 172.21.15.106:48512) 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5dd9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4d600ce1f0 0x7f4d6006e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36240/0 (socket says 172.21.15.106:36240) 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:51700/0 (socket says 172.21.15.106:51700) 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5e59c640 1 -- 172.21.15.106:0/2391396576 learned_addr learned my addr 172.21.15.106:0/2391396576 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 -- 172.21.15.106:0/2391396576 >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 msgr2=0x7f4d6006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 --2- 172.21.15.106:0/2391396576 >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 -- 172.21.15.106:0/2391396576 >> v1:172.21.15.114:6789/0 conn(0x7f4d600ce1f0 legacy=0x7f4d6006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5ed9d640 1 -- 172.21.15.106:0/2391396576 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6015f700 con 0x7f4d600b5bf0 2024-04-13T03:11:59.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.764+0000 7f4d5e59c640 1 --2- 172.21.15.106:0/2391396576 >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d5ed9d640 1 --2- 172.21.15.106:0/2391396576 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009d90 tx=0x7f4d5802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a8e9f7068dc2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d5d59a640 1 -- 172.21.15.106:0/2391396576 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58039070 con 0x7f4d600b5bf0 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d5d59a640 1 -- 172.21.15.106:0/2391396576 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5802cbc0 con 0x7f4d600b5bf0 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d5d59a640 1 -- 172.21.15.106:0/2391396576 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58034440 con 0x7f4d600b5bf0 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d64ec8640 1 -- 172.21.15.106:0/2391396576 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 msgr2=0x7f4d6006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.765+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/2391396576 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009d90 tx=0x7f4d5802be10 comp rx=0 tx=0).stop 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 -- 172.21.15.106:0/2391396576 shutdown_connections 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/2391396576 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d600b5bf0 0x7f4d6006fcc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/2391396576 >> v2:172.21.15.114:3300/0 conn(0x7f4d6015ebf0 0x7f4d6006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 -- 172.21.15.106:0/2391396576 >> 172.21.15.106:0/2391396576 conn(0x7f4d600cd6d0 msgr2=0x7f4d600cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 -- 172.21.15.106:0/2391396576 shutdown_connections 2024-04-13T03:11:59.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.766+0000 7f4d64ec8640 1 -- 172.21.15.106:0/2391396576 wait complete. 2024-04-13T03:11:59.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.767+0000 7f4d64ec8640 1 Processor -- start 2024-04-13T03:11:59.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.767+0000 7f4d64ec8640 1 -- start start 2024-04-13T03:11:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d600ce1f0 0x7f4d601f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 0x7f4d60072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d600b0650 con 0x7f4d600b5bf0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d600b04d0 con 0x7f4d600ce1f0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6015efd0 con 0x7f4d6015ebf0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36538/0 (socket says 172.21.15.106:36538) 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 -- 172.21.15.106:0/1607447448 learned_addr learned my addr 172.21.15.106:0/1607447448 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5dd9b640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d600ce1f0 0x7f4d601f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5ed9d640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 0x7f4d60072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 -- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 msgr2=0x7f4d60072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 0x7f4d60072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 -- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d600ce1f0 msgr2=0x7f4d601f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d600ce1f0 0x7f4d601f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 -- 172.21.15.106:0/1607447448 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d601f8d20 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5ed9d640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 0x7f4d60072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d5e59c640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000e9b0 tx=0x7f4d5000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50017070 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5000c9a0 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5000cd50 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6006bc50 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.768+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d600adeb0 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.769+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d60068a10 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.770+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d5001a070 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.770+0000 7f4d4effd640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 0x7f4d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.770+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f4d5008bb70 con 0x7f4d600b5bf0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.770+0000 7f4d5dd9b640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 0x7f4d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:11:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.771+0000 7f4d5dd9b640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 0x7f4d4006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d600cc290 tx=0x7f4d54007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:11:59.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.775+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d601f8d20 con 0x7f4d600b5bf0 2024-04-13T03:11:59.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.990+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d600ced80 con 0x7f4d600b5bf0 2024-04-13T03:11:59.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.990+0000 7f4d4effd640 1 -- 172.21.15.106:0/1607447448 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5193 (secure 0 0 0) 0x7f4d50059e10 con 0x7f4d600b5bf0 2024-04-13T03:11:59.991 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:11:59.991 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:11:16.632602+0000","modified":"2024-04-13T03:11:18.644896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/4219642587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4219642587},{"type":"v1","addr":"172.21.15.114:6835","nonce":4219642587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-13T03:11:59.992 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 22 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 msgr2=0x7f4d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 0x7f4d4006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d600cc290 tx=0x7f4d54007cc0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 msgr2=0x7f4d60155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000e9b0 tx=0x7f4d5000ee70 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 shutdown_connections 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4d4006c8f0 0x7f4d4006edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d6015ebf0 0x7f4d60072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d600ce1f0 0x7f4d601f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 --2- 172.21.15.106:0/1607447448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d600b5bf0 0x7f4d60155240 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.992+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 >> 172.21.15.106:0/1607447448 conn(0x7f4d600cd6d0 msgr2=0x7f4d600cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.993+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 shutdown_connections 2024-04-13T03:11:59.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:11:59.993+0000 7f4d64ec8640 1 -- 172.21.15.106:0/1607447448 wait complete. 2024-04-13T03:12:00.004 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.103+0000 7fd636a29640 1 Processor -- start 2024-04-13T03:12:00.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 -- start start 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd63015eeb0 0x7fd630072db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 0x7fd6300cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd630073700 con 0x7fd63015c3f0 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd63006ec10 con 0x7fd63015eeb0 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.104+0000 7fd636a29640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd6300ccd00 con 0x7fd6300732f0 2024-04-13T03:12:00.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd62f7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd63015eeb0 0x7fd630072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd634fc6640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 0x7fd6300cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd62ffff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd63015c3f0 0x7fd6300cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:36244/0 (socket says 172.21.15.106:36244) 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd62ffff640 1 -- 172.21.15.106:0/1521390592 learned_addr learned my addr 172.21.15.106:0/1521390592 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd634fc6640 1 -- 172.21.15.106:0/1521390592 >> v2:172.21.15.114:3300/0 conn(0x7fd63015eeb0 msgr2=0x7fd630072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd634fc6640 1 --2- 172.21.15.106:0/1521390592 >> v2:172.21.15.114:3300/0 conn(0x7fd63015eeb0 0x7fd630072db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd62effd640 1 -- 172.21.15.106:0/1521390592 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448578759 0 0) 0x7fd624001110 con 0x7fd63015c3f0 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd634fc6640 1 -- 172.21.15.106:0/1521390592 >> v1:172.21.15.114:6789/0 conn(0x7fd63015c3f0 legacy=0x7fd6300cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.105+0000 7fd634fc6640 1 -- 172.21.15.106:0/1521390592 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6300ccfe0 con 0x7fd6300732f0 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.106+0000 7fd634fc6640 1 --2- 172.21.15.106:0/1521390592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 0x7fd6300cf530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd628009d90 tx=0x7fd62802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5333b3578333bd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.106+0000 7fd62effd640 1 -- 172.21.15.106:0/1521390592 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628039070 con 0x7fd6300732f0 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.106+0000 7fd62effd640 1 -- 172.21.15.106:0/1521390592 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62802cc50 con 0x7fd6300732f0 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.106+0000 7fd62effd640 1 -- 172.21.15.106:0/1521390592 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628034440 con 0x7fd6300732f0 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 -- 172.21.15.106:0/1521390592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 msgr2=0x7fd6300cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 --2- 172.21.15.106:0/1521390592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 0x7fd6300cf530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd628009d90 tx=0x7fd62802bea0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 -- 172.21.15.106:0/1521390592 shutdown_connections 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 --2- 172.21.15.106:0/1521390592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6300732f0 0x7fd6300cf530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 --2- 172.21.15.106:0/1521390592 >> v2:172.21.15.114:3300/0 conn(0x7fd63015eeb0 0x7fd630072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 -- 172.21.15.106:0/1521390592 >> 172.21.15.106:0/1521390592 conn(0x7fd6300cdef0 msgr2=0x7fd6300ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 -- 172.21.15.106:0/1521390592 shutdown_connections 2024-04-13T03:12:00.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.107+0000 7fd636a29640 1 -- 172.21.15.106:0/1521390592 wait complete. 2024-04-13T03:12:00.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.108+0000 7fd636a29640 1 Processor -- start 2024-04-13T03:12:00.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.108+0000 7fd636a29640 1 -- start start 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63015c3f0 0x7fd630157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd63015eeb0 0x7fd63015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd6300cd880 con 0x7fd6300732f0 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6300cd5a0 con 0x7fd63015c3f0 2024-04-13T03:12:00.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd63015c7d0 con 0x7fd63015eeb0 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36544/0 (socket says 172.21.15.106:36544) 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 -- 172.21.15.106:0/852620395 learned_addr learned my addr 172.21.15.106:0/852620395 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd634fc6640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd63015eeb0 0x7fd63015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 -- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd63015eeb0 msgr2=0x7fd63015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd63015eeb0 0x7fd63015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 -- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63015c3f0 msgr2=0x7fd630157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62f7fe640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63015c3f0 0x7fd630157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63015c3f0 0x7fd630157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 -- 172.21.15.106:0/852620395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63015a670 con 0x7fd6300732f0 2024-04-13T03:12:00.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62ffff640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c750 tx=0x7fd62400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624017070 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd624005ab0 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6300c95e0 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624007070 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.109+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6300c9ba0 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.111+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630069120 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.111+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd62401a070 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.111+0000 7fd62cff9640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 0x7fd61c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.111+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fd62408b060 con 0x7fd6300732f0 2024-04-13T03:12:00.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.111+0000 7fd62f7fe640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 0x7fd61c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.112+0000 7fd62f7fe640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 0x7fd61c06edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd6300b0cd0 tx=0x7fd620008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.116+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd630073700 con 0x7fd6300732f0 2024-04-13T03:12:00.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.291+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6300cc140 con 0x7fd6300732f0 2024-04-13T03:12:00.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.292+0000 7fd62cff9640 1 -- 172.21.15.106:0/852620395 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16978 (secure 0 0 0) 0x7fd624059310 con 0x7fd6300732f0 2024-04-13T03:12:00.293 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:00.293 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":52,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:11:18.293562+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:11:14.853009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-13T03:11:15.586983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 msgr2=0x7fd61c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 0x7fd61c06edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd6300b0cd0 tx=0x7fd620008040 comp rx=0 tx=0).stop 2024-04-13T03:12:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 msgr2=0x7fd6300ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c750 tx=0x7fd62400cc10 comp rx=0 tx=0).stop 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 shutdown_connections 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd61c06c8f0 0x7fd61c06edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd63015eeb0 0x7fd63015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63015c3f0 0x7fd630157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 --2- 172.21.15.106:0/852620395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6300732f0 0x7fd6300ca680 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 >> 172.21.15.106:0/852620395 conn(0x7fd6300cdef0 msgr2=0x7fd6300ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 shutdown_connections 2024-04-13T03:12:00.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.294+0000 7fd636a29640 1 -- 172.21.15.106:0/852620395 wait complete. 2024-04-13T03:12:00.306 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:12:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.407+0000 7f1664460640 1 Processor -- start 2024-04-13T03:12:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.407+0000 7f1664460640 1 -- start start 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f1664460640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f165c15cbd0 0x7f165c15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f1664460640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f165c06c330 0x7f165c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f1664460640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165c15d690 con 0x7f165c156b40 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f1664460640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f165c06b8e0 con 0x7f165c06c330 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f1664460640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f165c06bea0 con 0x7f165c15cbd0 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f165c06c330 0x7f165c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f165c06c330 0x7f165c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36554/0 (socket says 172.21.15.106:36554) 2024-04-13T03:12:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 -- 172.21.15.106:0/3300903968 learned_addr learned my addr 172.21.15.106:0/3300903968 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 -- 172.21.15.106:0/3300903968 >> v2:172.21.15.114:3300/0 conn(0x7f165c15cbd0 msgr2=0x7f165c15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 --2- 172.21.15.106:0/3300903968 >> v2:172.21.15.114:3300/0 conn(0x7f165c15cbd0 0x7f165c15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 -- 172.21.15.106:0/3300903968 >> v1:172.21.15.114:6789/0 conn(0x7f165c156b40 legacy=0x7f165c06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.408+0000 7f16629fd640 1 -- 172.21.15.106:0/3300903968 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c15d970 con 0x7f165c06c330 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f16629fd640 1 --2- 172.21.15.106:0/3300903968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c06c330 0x7f165c06c7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f165c0af3d0 tx=0x7f165402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e5a77eed18c1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f16611fa640 1 -- 172.21.15.106:0/3300903968 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165402ca80 con 0x7f165c06c330 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f16611fa640 1 -- 172.21.15.106:0/3300903968 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165402cc20 con 0x7f165c06c330 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f16611fa640 1 -- 172.21.15.106:0/3300903968 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16540343f0 con 0x7f165c06c330 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f1664460640 1 -- 172.21.15.106:0/3300903968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c06c330 msgr2=0x7f165c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.409+0000 7f1664460640 1 --2- 172.21.15.106:0/3300903968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c06c330 0x7f165c06c7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f165c0af3d0 tx=0x7f165402bee0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 -- 172.21.15.106:0/3300903968 shutdown_connections 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 --2- 172.21.15.106:0/3300903968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c06c330 0x7f165c06c7c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 --2- 172.21.15.106:0/3300903968 >> v2:172.21.15.114:3300/0 conn(0x7f165c15cbd0 0x7f165c15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 -- 172.21.15.106:0/3300903968 >> 172.21.15.106:0/3300903968 conn(0x7f165c0cd6a0 msgr2=0x7f165c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 -- 172.21.15.106:0/3300903968 shutdown_connections 2024-04-13T03:12:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.410+0000 7f1664460640 1 -- 172.21.15.106:0/3300903968 wait complete. 2024-04-13T03:12:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 Processor -- start 2024-04-13T03:12:00.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 -- start start 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f165c06c330 0x7f165c0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f165c15cbd0 0x7f165c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f165c06edc0 con 0x7f165c156b40 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f165c156f20 con 0x7f165c06c330 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f1664460640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f165c06d880 con 0x7f165c15cbd0 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f16619fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f16619fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36564/0 (socket says 172.21.15.106:36564) 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.411+0000 7f16619fb640 1 -- 172.21.15.106:0/2117990967 learned_addr learned my addr 172.21.15.106:0/2117990967 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 -- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f165c15cbd0 msgr2=0x7f165c0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f165c15cbd0 0x7f165c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 -- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f165c06c330 msgr2=0x7f165c0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f165c06c330 0x7f165c0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 -- 172.21.15.106:0/2117990967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c0cfd40 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f16619fb640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b410 tx=0x7f164c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c015020 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c011830 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c0cffe0 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c011c20 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.412+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c0d05a0 con 0x7f165c156b40 2024-04-13T03:12:00.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.413+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f165c068a10 con 0x7f165c156b40 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.414+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f164c018020 con 0x7f165c156b40 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.414+0000 7f1652ffd640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.414+0000 7f16621fc640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.414+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f164c08bbe0 con 0x7f165c156b40 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.414+0000 7f16621fc640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1658001560 tx=0x7f1658007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.419+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f164c01f020 con 0x7f165c156b40 2024-04-13T03:12:00.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.632+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f165c156360 con 0x7f165c156b40 2024-04-13T03:12:00.634 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:12:00.633+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:12:00.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.635+0000 7f1652ffd640 1 -- 172.21.15.106:0/2117990967 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f164c0172c0 con 0x7f165c156b40 2024-04-13T03:12:00.635 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 msgr2=0x7f164006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1658001560 tx=0x7f1658007990 comp rx=0 tx=0).stop 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 msgr2=0x7f165c0adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b410 tx=0x7f164c00b8d0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 shutdown_connections 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f165c15cbd0 0x7f165c0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f165c156b40 0x7f165c0adcc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 --2- 172.21.15.106:0/2117990967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f165c06c330 0x7f165c0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 >> 172.21.15.106:0/2117990967 conn(0x7f165c0cd6a0 msgr2=0x7f165c151660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.637+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 shutdown_connections 2024-04-13T03:12:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.638+0000 7f1664460640 1 -- 172.21.15.106:0/2117990967 wait complete. 2024-04-13T03:12:00.642 INFO:tasks.ceph.mds.d.smithi114.stderr: -5858> 2024-04-13T03:11:22.556+0000 7f85d99f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:12:00.645 INFO:tasks.ceph.mds.d.smithi114.stderr: -3186> 2024-04-13T03:11:24.150+0000 7f85d59e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5988=0-4194304@1} caps={5988=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5988 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55c48d954680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:12:00.649 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:12:00.674 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:12:00.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.750+0000 7f61ff11e640 1 Processor -- start 2024-04-13T03:12:00.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.750+0000 7f61ff11e640 1 -- start start 2024-04-13T03:12:00.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61ff11e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f61f80b1c30 0x7f61f815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61ff11e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f80b5cf0 con 0x7f61f8070bb0 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61ff11e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f80b5fd0 con 0x7f61f80b54a0 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61ff11e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f61f80b62b0 con 0x7f61f80b1c30 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61fceba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f61f80b1c30 0x7f61f815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61fd6bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f61f8070bb0 0x7f61f806d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34214/0 (socket says 172.21.15.106:34214) 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.751+0000 7f61fd6bb640 1 -- 172.21.15.106:0/4105248818 learned_addr learned my addr 172.21.15.106:0/4105248818 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:00.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61fceba640 1 -- 172.21.15.106:0/4105248818 >> v1:172.21.15.114:6789/0 conn(0x7f61f80b54a0 legacy=0x7f61f80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61fceba640 1 -- 172.21.15.106:0/4105248818 >> v1:172.21.15.106:6789/0 conn(0x7f61f8070bb0 legacy=0x7f61f806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61fceba640 1 -- 172.21.15.106:0/4105248818 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80b6590 con 0x7f61f80b1c30 2024-04-13T03:12:00.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61fceba640 1 --2- 172.21.15.106:0/4105248818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f815ee40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097a0 tx=0x7f61e802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52d9c10160f09d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61f77fe640 1 -- 172.21.15.106:0/4105248818 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8034070 con 0x7f61f80b1c30 2024-04-13T03:12:00.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61f77fe640 1 -- 172.21.15.106:0/4105248818 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e8030610 con 0x7f61f80b1c30 2024-04-13T03:12:00.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.752+0000 7f61f77fe640 1 -- 172.21.15.106:0/4105248818 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e80309e0 con 0x7f61f80b1c30 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 -- 172.21.15.106:0/4105248818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 msgr2=0x7f61f815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 --2- 172.21.15.106:0/4105248818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f815ee40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097a0 tx=0x7f61e802b960 comp rx=0 tx=0).stop 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 -- 172.21.15.106:0/4105248818 shutdown_connections 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 --2- 172.21.15.106:0/4105248818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f815ee40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 -- 172.21.15.106:0/4105248818 >> 172.21.15.106:0/4105248818 conn(0x7f61f8071870 msgr2=0x7f61f8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 -- 172.21.15.106:0/4105248818 shutdown_connections 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.753+0000 7f61ff11e640 1 -- 172.21.15.106:0/4105248818 wait complete. 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.754+0000 7f61ff11e640 1 Processor -- start 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.754+0000 7f61ff11e640 1 -- start start 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61f8070bb0 0x7f61f80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 0x7f61f8151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61f806def0 con 0x7f61f80b1c30 2024-04-13T03:12:00.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61f806dd70 con 0x7f61f80b54a0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61ff11e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f61f806e070 con 0x7f61f8070bb0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36580/0 (socket says 172.21.15.106:36580) 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 -- 172.21.15.106:0/3326137263 learned_addr learned my addr 172.21.15.106:0/3326137263 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61fd6bb640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 0x7f61f8151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61fceba640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61f8070bb0 0x7f61f80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 -- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61f8070bb0 msgr2=0x7f61f80d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61f8070bb0 0x7f61f80d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 -- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 msgr2=0x7f61f8151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 0x7f61f8151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61f7fff640 1 -- 172.21.15.106:0/3326137263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80ce8d0 con 0x7f61f80b1c30 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.755+0000 7f61fd6bb640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 0x7f61f8151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61f7fff640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c3c0 tx=0x7f61ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0052d0 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ec005490 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f8151ad0 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec007260 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.756+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f81622b0 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.757+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f8068a10 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.757+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61ec007400 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.758+0000 7f61f57fa640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.758+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6187+0+0 (secure 0 0 0) 0x7f61ec08b270 con 0x7f61f80b1c30 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.758+0000 7f61fceba640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:00.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.758+0000 7f61fceba640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 0x7f61e406edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61e8036000 tx=0x7f61e803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:00.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.762+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61ec059440 con 0x7f61f80b1c30 2024-04-13T03:12:00.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:00.974+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f61f8072140 con 0x7f61f80b1c30 2024-04-13T03:12:01.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.636+0000 7f61f57fa640 1 -- 172.21.15.106:0/3326137263 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f61ec058d40 con 0x7f61f80b1c30 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 msgr2=0x7f61e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 0x7f61e406edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61e8036000 tx=0x7f61e803b040 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 msgr2=0x7f61f80ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c3c0 tx=0x7f61ec00c880 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 shutdown_connections 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f61e406c8f0 0x7f61e406edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61f80b54a0 0x7f61f8151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f80b1c30 0x7f61f80ce140 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 --2- 172.21.15.106:0/3326137263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61f8070bb0 0x7f61f80d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.638+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 >> 172.21.15.106:0/3326137263 conn(0x7f61f8071870 msgr2=0x7f61f815b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.639+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 shutdown_connections 2024-04-13T03:12:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.639+0000 7f61ff11e640 1 -- 172.21.15.106:0/3326137263 wait complete. 2024-04-13T03:12:01.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:12:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.757+0000 7f19e76b7640 1 Processor -- start 2024-04-13T03:12:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.757+0000 7f19e76b7640 1 -- start start 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e76b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f19e015e900 0x7f19e006d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e76b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19e015fb20 0x7f19e00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e76b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e0070e00 con 0x7f19e00709e0 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e76b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f19e0160040 con 0x7f19e015fb20 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e76b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f19e015ece0 con 0x7f19e015e900 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e4c52640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19e015fb20 0x7f19e00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e5c54640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f19e00709e0 0x7f19e0162470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34220/0 (socket says 172.21.15.106:34220) 2024-04-13T03:12:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.758+0000 7f19e5c54640 1 -- 172.21.15.106:0/679799126 learned_addr learned my addr 172.21.15.106:0/679799126 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19e4c52640 1 -- 172.21.15.106:0/679799126 >> v2:172.21.15.114:3300/0 conn(0x7f19e015e900 msgr2=0x7f19e006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19e4c52640 1 --2- 172.21.15.106:0/679799126 >> v2:172.21.15.114:3300/0 conn(0x7f19e015e900 0x7f19e006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19e4c52640 1 -- 172.21.15.106:0/679799126 >> v1:172.21.15.106:6789/0 conn(0x7f19e00709e0 legacy=0x7f19e0162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19e4c52640 1 -- 172.21.15.106:0/679799126 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e006da90 con 0x7f19e015fb20 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19e4c52640 1 --2- 172.21.15.106:0/679799126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e015fb20 0x7f19e00704a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f19e00b86b0 tx=0x7f19d002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910704838358f8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19d7fff640 1 -- 172.21.15.106:0/679799126 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0038070 con 0x7f19e015fb20 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19d7fff640 1 -- 172.21.15.106:0/679799126 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d0031710 con 0x7f19e015fb20 2024-04-13T03:12:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.759+0000 7f19d7fff640 1 -- 172.21.15.106:0/679799126 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0031ae0 con 0x7f19e015fb20 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 -- 172.21.15.106:0/679799126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e015fb20 msgr2=0x7f19e00704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 --2- 172.21.15.106:0/679799126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e015fb20 0x7f19e00704a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f19e00b86b0 tx=0x7f19d002b7b0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 -- 172.21.15.106:0/679799126 shutdown_connections 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 --2- 172.21.15.106:0/679799126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e015fb20 0x7f19e00704a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 --2- 172.21.15.106:0/679799126 >> v2:172.21.15.114:3300/0 conn(0x7f19e015e900 0x7f19e006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 -- 172.21.15.106:0/679799126 >> 172.21.15.106:0/679799126 conn(0x7f19e00717e0 msgr2=0x7f19e0071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 -- 172.21.15.106:0/679799126 shutdown_connections 2024-04-13T03:12:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.760+0000 7f19e76b7640 1 -- 172.21.15.106:0/679799126 wait complete. 2024-04-13T03:12:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.761+0000 7f19e76b7640 1 Processor -- start 2024-04-13T03:12:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.761+0000 7f19e76b7640 1 -- start start 2024-04-13T03:12:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 0x7f19e0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 0x7f19e0156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006d550 con 0x7f19e00709e0 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006e050 con 0x7f19e015e900 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e76b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f19e006d6d0 con 0x7f19e015fb20 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36612/0 (socket says 172.21.15.106:36612) 2024-04-13T03:12:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 -- 172.21.15.106:0/338494349 learned_addr learned my addr 172.21.15.106:0/338494349 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5c54640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 0x7f19e0156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e4c52640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 0x7f19e0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 -- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 msgr2=0x7f19e0156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 0x7f19e0156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 -- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 msgr2=0x7f19e0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 0x7f19e0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 -- 172.21.15.106:0/338494349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e0153a10 con 0x7f19e00709e0 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5c54640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 0x7f19e0156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e4c52640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 0x7f19e0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.762+0000 7f19e5453640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00ba50 tx=0x7f19dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.763+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc019070 con 0x7f19e00709e0 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.763+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc00cc00 con 0x7f19e00709e0 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.763+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e0153cf0 con 0x7f19e00709e0 2024-04-13T03:12:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.763+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc014440 con 0x7f19e00709e0 2024-04-13T03:12:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.763+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01542b0 con 0x7f19e00709e0 2024-04-13T03:12:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.764+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e0068a10 con 0x7f19e00709e0 2024-04-13T03:12:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.764+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19dc01c070 con 0x7f19e00709e0 2024-04-13T03:12:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.765+0000 7f19d5ffb640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 0x7f19c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.765+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6187+0+0 (secure 0 0 0) 0x7f19dc08bac0 con 0x7f19e00709e0 2024-04-13T03:12:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.765+0000 7f19e4c52640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 0x7f19c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.765+0000 7f19e4c52640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 0x7f19c406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035000 tx=0x7f19d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.769+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19dc059d10 con 0x7f19e00709e0 2024-04-13T03:12:01.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:01.944+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19e015bb50 con 0x7f19e00709e0 2024-04-13T03:12:02.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.640+0000 7f19d5ffb640 1 -- 172.21.15.106:0/338494349 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) ==== 181+0+0 (secure 0 0 0) 0x7f19dc059610 con 0x7f19e00709e0 2024-04-13T03:12:02.641 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:12:02.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.644+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 msgr2=0x7f19c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.644+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 0x7f19c406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035000 tx=0x7f19d0008030 comp rx=0 tx=0).stop 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.644+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 msgr2=0x7f19e0152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.644+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00ba50 tx=0x7f19dc00bf10 comp rx=0 tx=0).stop 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 shutdown_connections 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19c406c8f0 0x7f19c406edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19e015fb20 0x7f19e0156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19e015e900 0x7f19e0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 --2- 172.21.15.106:0/338494349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19e00709e0 0x7f19e0152d40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 >> 172.21.15.106:0/338494349 conn(0x7f19e00717e0 msgr2=0x7f19e00b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:02.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 shutdown_connections 2024-04-13T03:12:02.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.645+0000 7f19e76b7640 1 -- 172.21.15.106:0/338494349 wait complete. 2024-04-13T03:12:02.659 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:12:02.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.766+0000 7f9ae955e640 1 Processor -- start 2024-04-13T03:12:02.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.766+0000 7f9ae955e640 1 -- start start 2024-04-13T03:12:02.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae955e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9ae40d2370 0x7f9ae40b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae955e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae415c870 con 0x7f9ae40d3490 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae955e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae415ee70 con 0x7f9ae40b3920 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae955e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae40d5420 con 0x7f9ae40d2370 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae2ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9ae40d3490 0x7f9ae40d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51868/0 (socket says 172.21.15.106:51868) 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/3302549467 learned_addr learned my addr 172.21.15.106:0/3302549467 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480466705 0 0) 0x7f9ae415c870 con 0x7f9ae40d3490 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9adbfff640 1 --2- 172.21.15.106:0/3302549467 >> v2:172.21.15.114:3300/0 conn(0x7f9ae40d2370 0x7f9ae40b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ad0003520 con 0x7f9ae40d3490 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3105727532 0 0) 0x7f9ad0003520 con 0x7f9ae40d3490 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 >> v2:172.21.15.114:3300/0 conn(0x7f9ae40d2370 msgr2=0x7f9ae40b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 --2- 172.21.15.106:0/3302549467 >> v2:172.21.15.114:3300/0 conn(0x7f9ae40d2370 0x7f9ae40b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 >> v1:172.21.15.114:6789/0 conn(0x7f9ae40b3920 legacy=0x7f9ae415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40d2800 con 0x7f9ae40d3490 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9ad4001670 con 0x7f9ae40d3490 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 >> v1:172.21.15.106:6789/0 conn(0x7f9ae40d3490 legacy=0x7f9ae40d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ad0005210 0x7f9ad000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40d5420 con 0x7f9ad0003a00 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad000c160 con 0x7f9ad0005210 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad000c360 con 0x7f9ad0004800 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.767+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae37fe640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ad0005210 0x7f9ad000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9adbfff640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ad0004800 msgr2=0x7f9ad0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ad0005210 msgr2=0x7f9ad000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ad0005210 0x7f9ad000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/3302549467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad0003520 con 0x7f9ad0003a00 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae37fe640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ad0005210 0x7f9ad000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae37fe640 1 -- 172.21.15.106:0/3302549467 reap_dead start 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.768+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c0e0 tx=0x7f9ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cc6f4b6a0c2656 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4007c40 con 0x7f9ad0003a00 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4030070 con 0x7f9ad0003a00 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae27fc640 1 -- 172.21.15.106:0/3302549467 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4005d70 con 0x7f9ad0003a00 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 -- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 msgr2=0x7f9ad0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c0e0 tx=0x7f9ad0005830 comp rx=0 tx=0).stop 2024-04-13T03:12:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 -- 172.21.15.106:0/3302549467 shutdown_connections 2024-04-13T03:12:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 --2- 172.21.15.106:0/3302549467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 -- 172.21.15.106:0/3302549467 >> 172.21.15.106:0/3302549467 conn(0x7f9ae40b9760 msgr2=0x7f9ae40b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 -- 172.21.15.106:0/3302549467 shutdown_connections 2024-04-13T03:12:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.769+0000 7f9ae955e640 1 -- 172.21.15.106:0/3302549467 wait complete. 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.770+0000 7f9ae955e640 1 Processor -- start 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.770+0000 7f9ae955e640 1 -- start start 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ae4158600 0x7f9ae406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 0x7f9ae4160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40d30a0 con 0x7f9ad0003a00 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40d2dc0 con 0x7f9ae4158600 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae955e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae40b3d50 con 0x7f9ae4158a60 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51676/0 (socket says 172.21.15.106:51676) 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/2217765800 learned_addr learned my addr 172.21.15.106:0/2217765800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae37fe640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 0x7f9ae4160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9adbfff640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ae4158600 0x7f9ae406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 msgr2=0x7f9ae4160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 0x7f9ae4160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ae4158600 msgr2=0x7f9ae406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ae4158600 0x7f9ae406d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 -- 172.21.15.106:0/2217765800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40c8d80 con 0x7f9ad0003a00 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae37fe640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 0x7f9ae4160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.771+0000 7f9ae2ffd640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c200 tx=0x7f9ad402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.772+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad402cbe0 con 0x7f9ad0003a00 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.772+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4000ed0 con 0x7f9ad0003a00 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.772+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae40c9020 con 0x7f9ad0003a00 2024-04-13T03:12:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.772+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4007070 con 0x7f9ad0003a00 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.772+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae40c95e0 con 0x7f9ad0003a00 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.773+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae4068a10 con 0x7f9ad0003a00 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.773+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ad4007210 con 0x7f9ad0003a00 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.773+0000 7f9adb7fe640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 0x7f9ac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.773+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5749+0+0 (secure 0 0 0) 0x7f9ad40ad080 con 0x7f9ad0003a00 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.774+0000 7f9adbfff640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 0x7f9ac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.774+0000 7f9adbfff640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 0x7f9ac406edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40b7130 tx=0x7f9acc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.778+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ad407b3f0 con 0x7f9ad0003a00 2024-04-13T03:12:02.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:02.951+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9ae4069120 con 0x7f9ad0003a00 2024-04-13T03:12:03.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.645+0000 7f9adb7fe640 1 -- 172.21.15.106:0/2217765800 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) ==== 169+0+0 (secure 0 0 0) 0x7f9ad407acf0 con 0x7f9ad0003a00 2024-04-13T03:12:03.646 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:12:03.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.648+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 msgr2=0x7f9ac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.648+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 0x7f9ac406edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40b7130 tx=0x7f9acc008040 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.648+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 msgr2=0x7f9ae40c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.648+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c200 tx=0x7f9ad402e070 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 shutdown_connections 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ac406c8f0 0x7f9ac406edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9ae4158a60 0x7f9ae4160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ae4158600 0x7f9ae406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 --2- 172.21.15.106:0/2217765800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ad0003a00 0x7f9ae40c9de0 secure :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c200 tx=0x7f9ad402e070 comp rx=0 tx=0).stop 2024-04-13T03:12:03.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 >> 172.21.15.106:0/2217765800 conn(0x7f9ae40b9760 msgr2=0x7f9ae40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:03.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 shutdown_connections 2024-04-13T03:12:03.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.649+0000 7f9ae955e640 1 -- 172.21.15.106:0/2217765800 wait complete. 2024-04-13T03:12:03.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:12:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.776+0000 7f26687f0640 1 Processor -- start 2024-04-13T03:12:03.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.776+0000 7f26687f0640 1 -- start start 2024-04-13T03:12:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f26687f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f26687f0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 0x7f266015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f26687f0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266015ddd0 con 0x7f26600704a0 2024-04-13T03:12:03.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f26687f0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f266015e100 con 0x7f26600d1c30 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f26687f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f266015d4d0 con 0x7f26600cfea0 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f2666d8d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 0x7f266015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:36088/0 (socket says 172.21.15.106:36088) 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 -- 172.21.15.106:0/80086431 learned_addr learned my addr 172.21.15.106:0/80086431 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f2666d8d640 1 --2- 172.21.15.106:0/80086431 >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 0x7f266015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:54522/0 (socket says 172.21.15.106:54522) 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 -- 172.21.15.106:0/80086431 >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 msgr2=0x7f266015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 --2- 172.21.15.106:0/80086431 >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 0x7f266015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 -- 172.21.15.106:0/80086431 >> v1:172.21.15.114:6789/0 conn(0x7f26600704a0 legacy=0x7f26600d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.777+0000 7f266658c640 1 -- 172.21.15.106:0/80086431 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266015d7b0 con 0x7f26600cfea0 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.778+0000 7f266658c640 1 --2- 172.21.15.106:0/80086431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d539e703c3f73e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.778+0000 7f266558a640 1 -- 172.21.15.106:0/80086431 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c035020 con 0x7f26600cfea0 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.778+0000 7f266558a640 1 -- 172.21.15.106:0/80086431 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c031640 con 0x7f26600cfea0 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.778+0000 7f266558a640 1 -- 172.21.15.106:0/80086431 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c02d040 con 0x7f26600cfea0 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 -- 172.21.15.106:0/80086431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 msgr2=0x7f266015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 --2- 172.21.15.106:0/80086431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c02b8f0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 -- 172.21.15.106:0/80086431 shutdown_connections 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 --2- 172.21.15.106:0/80086431 >> v2:172.21.15.114:3300/0 conn(0x7f26600d1c30 0x7f266015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 --2- 172.21.15.106:0/80086431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f266015cc10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 -- 172.21.15.106:0/80086431 >> 172.21.15.106:0/80086431 conn(0x7f2660071870 msgr2=0x7f2660071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 -- 172.21.15.106:0/80086431 shutdown_connections 2024-04-13T03:12:03.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.779+0000 7f26687f0640 1 -- 172.21.15.106:0/80086431 wait complete. 2024-04-13T03:12:03.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.780+0000 7f26687f0640 1 Processor -- start 2024-04-13T03:12:03.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.780+0000 7f26687f0640 1 -- start start 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f26601421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 0x7f2660153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f266015ead0 con 0x7f26600704a0 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f266015d150 con 0x7f26600d1c30 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f266015ec50 con 0x7f26600cfea0 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51688/0 (socket says 172.21.15.106:51688) 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 -- 172.21.15.106:0/1452420735 learned_addr learned my addr 172.21.15.106:0/1452420735 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:03.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2665d8b640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f26601421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2666d8d640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 0x7f2660153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 -- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 msgr2=0x7f26601421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f26601421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 -- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 msgr2=0x7f2660153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 0x7f2660153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 -- 172.21.15.106:0/1452420735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601545e0 con 0x7f26600704a0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2665d8b640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f26601421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2666d8d640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 0x7f2660153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f266658c640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f265c000c00 tx=0x7f265c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c040070 con 0x7f26600704a0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c0399f0 con 0x7f26600704a0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.781+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2660142720 con 0x7f26600704a0 2024-04-13T03:12:03.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.782+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c035040 con 0x7f26600704a0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.782+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601f6670 con 0x7f26600704a0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.782+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660068a10 con 0x7f26600704a0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.783+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f265c04b060 con 0x7f26600704a0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.783+0000 7f2656ffd640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.783+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f265c0ad250 con 0x7f26600704a0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.783+0000 7f2665d8b640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:03.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.784+0000 7f2665d8b640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 0x7f264406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26601596b0 tx=0x7f2650009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:03.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.788+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2660142720 con 0x7f26600704a0 2024-04-13T03:12:03.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.963+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2660072360 con 0x7f26600704a0 2024-04-13T03:12:03.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.966+0000 7f2656ffd640 1 -- 172.21.15.106:0/1452420735 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f265c07b7f0 con 0x7f26600704a0 2024-04-13T03:12:03.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 msgr2=0x7f264406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 0x7f264406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26601596b0 tx=0x7f2650009040 comp rx=0 tx=0).stop 2024-04-13T03:12:03.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 msgr2=0x7f26601f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:03.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f265c000c00 tx=0x7f265c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 shutdown_connections 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f264406c8f0 0x7f264406edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f26600cfea0 0x7f26601421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26600d1c30 0x7f2660153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 --2- 172.21.15.106:0/1452420735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26600704a0 0x7f26601f6090 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 >> 172.21.15.106:0/1452420735 conn(0x7f2660071870 msgr2=0x7f26600b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 shutdown_connections 2024-04-13T03:12:03.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:03.968+0000 7f26687f0640 1 -- 172.21.15.106:0/1452420735 wait complete. 2024-04-13T03:12:03.980 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-13T03:12:04.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.080+0000 7f5547555640 1 Processor -- start 2024-04-13T03:12:04.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.080+0000 7f5547555640 1 -- start start 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5547555640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f554015ebf0 0x7f554006f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5547555640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55400ce1f0 0x7f554006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5547555640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400ce6f0 con 0x7f55400b5bf0 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5547555640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f554015fa70 con 0x7f55400ce1f0 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5547555640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f554015f420 con 0x7f554015ebf0 2024-04-13T03:12:04.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5544af0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55400ce1f0 0x7f554006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5545af2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f55400b5bf0 0x7f554006e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51880/0 (socket says 172.21.15.106:51880) 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5545af2640 1 -- 172.21.15.106:0/258944703 learned_addr learned my addr 172.21.15.106:0/258944703 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f55452f1640 1 --2- 172.21.15.106:0/258944703 >> v2:172.21.15.114:3300/0 conn(0x7f554015ebf0 0x7f554006f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5544af0640 1 -- 172.21.15.106:0/258944703 >> v2:172.21.15.114:3300/0 conn(0x7f554015ebf0 msgr2=0x7f554006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5544af0640 1 --2- 172.21.15.106:0/258944703 >> v2:172.21.15.114:3300/0 conn(0x7f554015ebf0 0x7f554006f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5544af0640 1 -- 172.21.15.106:0/258944703 >> v1:172.21.15.106:6789/0 conn(0x7f55400b5bf0 legacy=0x7f554006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.081+0000 7f5544af0640 1 -- 172.21.15.106:0/258944703 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554015f700 con 0x7f55400ce1f0 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5544af0640 1 --2- 172.21.15.106:0/258944703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f554006fcc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5540071c70 tx=0x7f553c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bc23b56ca88dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5537fff640 1 -- 172.21.15.106:0/258944703 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0315a0 con 0x7f55400ce1f0 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5537fff640 1 -- 172.21.15.106:0/258944703 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c037070 con 0x7f55400ce1f0 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5537fff640 1 -- 172.21.15.106:0/258944703 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c031960 con 0x7f55400ce1f0 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 -- 172.21.15.106:0/258944703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 msgr2=0x7f554006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 --2- 172.21.15.106:0/258944703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f554006fcc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5540071c70 tx=0x7f553c02b840 comp rx=0 tx=0).stop 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 -- 172.21.15.106:0/258944703 shutdown_connections 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 --2- 172.21.15.106:0/258944703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f554006fcc0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 --2- 172.21.15.106:0/258944703 >> v2:172.21.15.114:3300/0 conn(0x7f554015ebf0 0x7f554006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.082+0000 7f5547555640 1 -- 172.21.15.106:0/258944703 >> 172.21.15.106:0/258944703 conn(0x7f55400cd6d0 msgr2=0x7f55400cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:04.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.083+0000 7f5547555640 1 -- 172.21.15.106:0/258944703 shutdown_connections 2024-04-13T03:12:04.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.083+0000 7f5547555640 1 -- 172.21.15.106:0/258944703 wait complete. 2024-04-13T03:12:04.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 Processor -- start 2024-04-13T03:12:04.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 -- start start 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55400b5bf0 0x7f55401568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 0x7f554006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55400b0650 con 0x7f55400ce1f0 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f55400b04d0 con 0x7f554015ebf0 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.084+0000 7f5547555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f554015efd0 con 0x7f55400b5bf0 2024-04-13T03:12:04.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51708/0 (socket says 172.21.15.106:51708) 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 -- 172.21.15.106:0/2592829367 learned_addr learned my addr 172.21.15.106:0/2592829367 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f55452f1640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55400b5bf0 0x7f55401568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5545af2640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 0x7f554006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 -- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55400b5bf0 msgr2=0x7f55401568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55400b5bf0 0x7f55401568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 -- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 msgr2=0x7f554006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 0x7f554006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 -- 172.21.15.106:0/2592829367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400aee20 con 0x7f55400ce1f0 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5545af2640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 0x7f554006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5544af0640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c032840 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c032a00 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55401f56f0 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c032df0 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.085+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55400aeb40 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.086+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5540068a10 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.087+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553c04d3f0 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.087+0000 7f5535ffb640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.087+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f553c02f030 con 0x7f55400ce1f0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.087+0000 7f55452f1640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.088+0000 7f55452f1640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 0x7f552406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.092+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55401f56f0 con 0x7f55400ce1f0 2024-04-13T03:12:04.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.336+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f55400726d0 con 0x7f55400ce1f0 2024-04-13T03:12:04.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.646+0000 7f5535ffb640 1 -- 172.21.15.106:0/2592829367 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v159) ==== 149+0+0 (secure 0 0 0) 0x7f55400aee20 con 0x7f55400ce1f0 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.648+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 msgr2=0x7f552406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.648+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 0x7f552406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530008040 comp rx=0 tx=0).stop 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.648+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 msgr2=0x7f55401f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.648+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 shutdown_connections 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f552406c8f0 0x7f552406edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f554015ebf0 0x7f554006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55400ce1f0 0x7f55401f51b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 --2- 172.21.15.106:0/2592829367 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55400b5bf0 0x7f55401568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 >> 172.21.15.106:0/2592829367 conn(0x7f55400cd6d0 msgr2=0x7f55400b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:04.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 shutdown_connections 2024-04-13T03:12:04.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.649+0000 7f5547555640 1 -- 172.21.15.106:0/2592829367 wait complete. 2024-04-13T03:12:04.661 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-13T03:12:04.661 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-13T03:12:04.661 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:12:04.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:04.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.761+0000 7fdfc95c4640 1 Processor -- start 2024-04-13T03:12:04.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.761+0000 7fdfc95c4640 1 -- start start 2024-04-13T03:12:04.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc95c4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfc40ce580 0x7fdfc40d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc95c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40b5b10 0x7fdfc415e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc95c4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc40d0b40 con 0x7fdfc40d06c0 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc95c4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc40d1be0 con 0x7fdfc40ce580 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc95c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40d0e20 con 0x7fdfc40b5b10 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfc40ce580 0x7fdfc40d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfc40ce580 0x7fdfc40d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51724/0 (socket says 172.21.15.106:51724) 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/4184800177 learned_addr learned my addr 172.21.15.106:0/4184800177 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc37fe640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40b5b10 0x7fdfc415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/4184800177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40b5b10 msgr2=0x7fdfc415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40b5b10 0x7fdfc415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/4184800177 >> v1:172.21.15.114:6789/0 conn(0x7fdfc40d06c0 legacy=0x7fdfc40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:04.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.762+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/4184800177 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc40d1100 con 0x7fdfc40ce580 2024-04-13T03:12:04.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.763+0000 7fdfc2ffd640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40ce580 0x7fdfc40d0180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009e50 tx=0x7fdfb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ed4da85d65c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.763+0000 7fdfc1ffb640 1 -- 172.21.15.106:0/4184800177 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb402ca70 con 0x7fdfc40ce580 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.763+0000 7fdfc1ffb640 1 -- 172.21.15.106:0/4184800177 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb402cc10 con 0x7fdfc40ce580 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.763+0000 7fdfc1ffb640 1 -- 172.21.15.106:0/4184800177 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40343f0 con 0x7fdfc40ce580 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 -- 172.21.15.106:0/4184800177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40ce580 msgr2=0x7fdfc40d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40ce580 0x7fdfc40d0180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009e50 tx=0x7fdfb402bed0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 -- 172.21.15.106:0/4184800177 shutdown_connections 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40b5b10 0x7fdfc415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/4184800177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40ce580 0x7fdfc40d0180 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 -- 172.21.15.106:0/4184800177 >> 172.21.15.106:0/4184800177 conn(0x7fdfc40d25b0 msgr2=0x7fdfc40d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 -- 172.21.15.106:0/4184800177 shutdown_connections 2024-04-13T03:12:04.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.764+0000 7fdfc95c4640 1 -- 172.21.15.106:0/4184800177 wait complete. 2024-04-13T03:12:04.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.765+0000 7fdfc95c4640 1 Processor -- start 2024-04-13T03:12:04.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.765+0000 7fdfc95c4640 1 -- start start 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40ce580 0x7fdfc4159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc40d06c0 0x7fdfc415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40ce960 con 0x7fdfc40b5b10 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40d16c0 con 0x7fdfc40d06c0 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc95c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40b7890 con 0x7fdfc40ce580 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51738/0 (socket says 172.21.15.106:51738) 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/3714797198 learned_addr learned my addr 172.21.15.106:0/3714797198 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc27fc640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40ce580 0x7fdfc4159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc37fe640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc40d06c0 0x7fdfc415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40ce580 msgr2=0x7fdfc4159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40ce580 0x7fdfc4159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc40d06c0 msgr2=0x7fdfc415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc40d06c0 0x7fdfc415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.766+0000 7fdfc2ffd640 1 -- 172.21.15.106:0/3714797198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc415d1a0 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfc2ffd640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009e50 tx=0x7fdfb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4043020 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb403e450 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc4159e50 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb403e840 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.767+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc40c6560 con 0x7fdfc40b5b10 2024-04-13T03:12:04.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.768+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4074750 con 0x7fdfc40b5b10 2024-04-13T03:12:04.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.768+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfb4046020 con 0x7fdfc40b5b10 2024-04-13T03:12:04.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.768+0000 7fdfab7fe640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:04.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.769+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7fdfb40ae350 con 0x7fdfc40b5b10 2024-04-13T03:12:04.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.769+0000 7fdfc27fc640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:04.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.769+0000 7fdfc27fc640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 0x7fdfb006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:04.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.773+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfb404d030 con 0x7fdfc40b5b10 2024-04-13T03:12:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.949+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfc4068a10 con 0x7fdfc40b5b10 2024-04-13T03:12:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.949+0000 7fdfab7fe640 1 -- 172.21.15.106:0/3714797198 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+13331 (secure 0 0 0) 0x7fdfb407c8f0 con 0x7fdfc40b5b10 2024-04-13T03:12:04.950 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:04.950 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":55,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:12:03.641295+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6835/4219642587":"2024-04-14T03:12:00.634178+0000","172.21.15.114:6833/4219642587":"2024-04-14T03:12:00.634178+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:04.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 msgr2=0x7fdfb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 0x7fdfb006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 msgr2=0x7fdfc415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009e50 tx=0x7fdfb4004a40 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 shutdown_connections 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdfc40ce580 0x7fdfc4159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc40d06c0 0x7fdfc415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 --2- 172.21.15.106:0/3714797198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc40b5b10 0x7fdfc415cc60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 >> 172.21.15.106:0/3714797198 conn(0x7fdfc40d25b0 msgr2=0x7fdfc40ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 shutdown_connections 2024-04-13T03:12:04.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:04.951+0000 7fdfc95c4640 1 -- 172.21.15.106:0/3714797198 wait complete. 2024-04-13T03:12:04.963 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:12:04.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:12:05.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.066+0000 7f4f5771e640 1 Processor -- start 2024-04-13T03:12:05.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.066+0000 7f4f5771e640 1 -- start start 2024-04-13T03:12:05.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f5771e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f500d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f5771e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f500d1630 0x7f4f500d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f5771e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f500cfe70 con 0x7f4f50162550 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f5771e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4f500d1fe0 con 0x7f4f500d1630 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f5771e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5015ee60 con 0x7f4f5015ea80 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f54cb9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4f50162550 0x7f4f500b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51896/0 (socket says 172.21.15.106:51896) 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f54cb9640 1 -- 172.21.15.106:0/2700114362 learned_addr learned my addr 172.21.15.106:0/2700114362 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f554ba640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f500d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f55cbb640 1 --2- 172.21.15.106:0/2700114362 >> v2:172.21.15.114:3300/0 conn(0x7f4f500d1630 0x7f4f500d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508751837 0 0) 0x7f4f500cfe70 con 0x7f4f50162550 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f3c003520 con 0x7f4f50162550 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173443648 0 0) 0x7f4f3c003520 con 0x7f4f50162550 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 msgr2=0x7f4f500d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f500d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 >> v2:172.21.15.114:3300/0 conn(0x7f4f500d1630 msgr2=0x7f4f500d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 --2- 172.21.15.106:0/2700114362 >> v2:172.21.15.114:3300/0 conn(0x7f4f500d1630 0x7f4f500d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f50162af0 con 0x7f4f50162550 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4f40001670 con 0x7f4f50162550 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 >> v1:172.21.15.106:6789/0 conn(0x7f4f50162550 legacy=0x7f4f500b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f3c0039b0 0x7f4f3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3c004d40 0x7f4f3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500d1fe0 con 0x7f4f3c0039b0 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5015ee60 con 0x7f4f3c004d40 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.067+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c00b9c0 con 0x7f4f5015ea80 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f3c0039b0 0x7f4f3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f54cb9640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f55cbb640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3c004d40 0x7f4f3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 -- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 msgr2=0x7f4f3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f5015ea80 0x7f4f3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 -- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3c004d40 msgr2=0x7f4f3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3c004d40 0x7f4f3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.068+0000 7f4f554ba640 1 -- 172.21.15.106:0/2700114362 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c003520 con 0x7f4f3c0039b0 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f554ba640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f3c0039b0 0x7f4f3c003dd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ac0 tx=0x7f4f3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495537bb4e281edf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c039070 con 0x7f4f3c0039b0 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c02c6b0 con 0x7f4f3c0039b0 2024-04-13T03:12:05.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f47fff640 1 -- 172.21.15.106:0/2700114362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c009690 con 0x7f4f3c0039b0 2024-04-13T03:12:05.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 -- 172.21.15.106:0/2700114362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f3c0039b0 msgr2=0x7f4f3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 --2- 172.21.15.106:0/2700114362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f3c0039b0 0x7f4f3c003dd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ac0 tx=0x7f4f3c005830 comp rx=0 tx=0).stop 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f55cbb640 1 -- 172.21.15.106:0/2700114362 reap_dead start 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 -- 172.21.15.106:0/2700114362 shutdown_connections 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 -- 172.21.15.106:0/2700114362 >> 172.21.15.106:0/2700114362 conn(0x7f4f5009da50 msgr2=0x7f4f500b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 -- 172.21.15.106:0/2700114362 shutdown_connections 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.069+0000 7f4f5771e640 1 -- 172.21.15.106:0/2700114362 wait complete. 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.070+0000 7f4f5771e640 1 Processor -- start 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 -- start start 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f3c0039b0 0x7f4f5015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f501e8c30 0x7f4f500c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50163390 con 0x7f4f500b5ae0 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501630b0 con 0x7f4f501e8c30 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f5771e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4f500d22b0 con 0x7f4f3c0039b0 2024-04-13T03:12:05.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f55cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f55cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51744/0 (socket says 172.21.15.106:51744) 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f55cbb640 1 -- 172.21.15.106:0/811794530 learned_addr learned my addr 172.21.15.106:0/811794530 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f54cb9640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f501e8c30 0x7f4f500c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.071+0000 7f4f554ba640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f3c0039b0 0x7f4f5015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 -- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f3c0039b0 msgr2=0x7f4f5015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f3c0039b0 0x7f4f5015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 -- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f501e8c30 msgr2=0x7f4f500c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f501e8c30 0x7f4f500c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 -- 172.21.15.106:0/811794530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f501e9130 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f55cbb640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f500d5320 tx=0x7f4f4800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800fde0 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f48015430 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48015820 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f501e9410 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.072+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f501e99d0 con 0x7f4f500b5ae0 2024-04-13T03:12:05.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.073+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f50068a10 con 0x7f4f500b5ae0 2024-04-13T03:12:05.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.073+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f480159c0 con 0x7f4f500b5ae0 2024-04-13T03:12:05.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.074+0000 7f4f45ffb640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.074+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f4f4808c740 con 0x7f4f500b5ae0 2024-04-13T03:12:05.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.074+0000 7f4f554ba640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.074+0000 7f4f554ba640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 0x7f4f3406ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c036000 tx=0x7f4f4c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.078+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f48019030 con 0x7f4f500b5ae0 2024-04-13T03:12:05.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.252+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f500d5350 con 0x7f4f500b5ae0 2024-04-13T03:12:05.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.252+0000 7f4f45ffb640 1 -- 172.21.15.106:0/811794530 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7f4f48010040 con 0x7f4f500b5ae0 2024-04-13T03:12:05.253 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:12:05.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 msgr2=0x7f4f3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 0x7f4f3406ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c036000 tx=0x7f4f4c030000 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 msgr2=0x7f4f500b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f500d5320 tx=0x7f4f4800f580 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 shutdown_connections 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f500b5ae0 0x7f4f500b5f70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f501e8c30 0x7f4f500c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 --2- 172.21.15.106:0/811794530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f3c0039b0 0x7f4f5015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 >> 172.21.15.106:0/811794530 conn(0x7f4f5009da50 msgr2=0x7f4f500b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 shutdown_connections 2024-04-13T03:12:05.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.254+0000 7f4f5771e640 1 -- 172.21.15.106:0/811794530 wait complete. 2024-04-13T03:12:05.266 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-13T03:12:05.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.365+0000 7f356b98c640 1 Processor -- start 2024-04-13T03:12:05.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.365+0000 7f356b98c640 1 -- start start 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f356b98c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3564160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f356b98c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3564070470 con 0x7f35640b0390 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f356b98c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f356415da00 con 0x7f35640cf5a0 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f356b98c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35640ceb50 con 0x7f3564070090 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f3568f27640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f35640b0390 0x7f3564160ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51906/0 (socket says 172.21.15.106:51906) 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f3568f27640 1 -- 172.21.15.106:0/2968684937 learned_addr learned my addr 172.21.15.106:0/2968684937 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f3569728640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3564160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507152363 0 0) 0x7f3564070470 con 0x7f35640b0390 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.366+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3550003520 con 0x7f35640b0390 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2659656539 0 0) 0x7f3550003520 con 0x7f35640b0390 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 msgr2=0x7f3564160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3564160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 >> v1:172.21.15.114:6789/0 conn(0x7f35640cf5a0 legacy=0x7f35640cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356415dcc0 con 0x7f35640b0390 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f3554001670 con 0x7f35640b0390 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 >> v1:172.21.15.106:6789/0 conn(0x7f35640b0390 legacy=0x7f3564160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35500039e0 0x7f3550003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3550004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3550004d40 0x7f35500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35640ceb50 con 0x7f3550004d40 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f355000ba70 con 0x7f35500039e0 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f355000bc70 con 0x7f3564070090 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3550004d40 0x7f35500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3568f27640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3550004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569728640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35500039e0 0x7f3550003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 -- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 msgr2=0x7f3550004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3550004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 -- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35500039e0 msgr2=0x7f3550003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35500039e0 0x7f3550003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569f29640 1 -- 172.21.15.106:0/2968684937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3550003520 con 0x7f3550004d40 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3569728640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35500039e0 0x7f3550003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.367+0000 7f3568f27640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3564070090 0x7f3550004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f3569f29640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3550004d40 0x7f35500051c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f355c001f60 tx=0x7f3550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b92d3571ac3da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c001b30 con 0x7f3550004d40 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c02d710 con 0x7f3550004d40 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f355bfff640 1 -- 172.21.15.106:0/2968684937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c02db00 con 0x7f3550004d40 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f356b98c640 1 -- 172.21.15.106:0/2968684937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3550004d40 msgr2=0x7f35500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f356b98c640 1 --2- 172.21.15.106:0/2968684937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3550004d40 0x7f35500051c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f355c001f60 tx=0x7f3550005830 comp rx=0 tx=0).stop 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f3569f29640 1 -- 172.21.15.106:0/2968684937 reap_dead start 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f356b98c640 1 -- 172.21.15.106:0/2968684937 shutdown_connections 2024-04-13T03:12:05.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.368+0000 7f356b98c640 1 -- 172.21.15.106:0/2968684937 >> 172.21.15.106:0/2968684937 conn(0x7f35640cd6d0 msgr2=0x7f35640cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.369+0000 7f356b98c640 1 -- 172.21.15.106:0/2968684937 shutdown_connections 2024-04-13T03:12:05.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.369+0000 7f356b98c640 1 -- 172.21.15.106:0/2968684937 wait complete. 2024-04-13T03:12:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 Processor -- start 2024-04-13T03:12:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 -- start start 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3550004d40 0x7f35641517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f356409e300 0x7f35640b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f356406e560 con 0x7f356409de80 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35640cf110 con 0x7f3550004d40 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.370+0000 7f356b98c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35640cd220 con 0x7f356409e300 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51766/0 (socket says 172.21.15.106:51766) 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 -- 172.21.15.106:0/1939690915 learned_addr learned my addr 172.21.15.106:0/1939690915 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3569728640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3550004d40 0x7f35641517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3569f29640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f356409e300 0x7f35640b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 -- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f356409e300 msgr2=0x7f35640b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f356409e300 0x7f35640b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 -- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3550004d40 msgr2=0x7f35641517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3550004d40 0x7f35641517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 -- 172.21.15.106:0/1939690915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3564151d00 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3568f27640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f355400e7e0 tx=0x7f355400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400c8d0 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3554018070 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35641f69f0 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400cc70 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.371+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356409d680 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.372+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3564068a10 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.373+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3554013420 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.373+0000 7f3559ffb640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.373+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f355408b5c0 con 0x7f356409de80 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.373+0000 7f3569728640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.373+0000 7f3569728640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3560009990 tx=0x7f3560008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.377+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3554059b60 con 0x7f356409de80 2024-04-13T03:12:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.622+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f356406ea60 con 0x7f356409de80 2024-04-13T03:12:05.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.648+0000 7f3559ffb640 1 -- 172.21.15.106:0/1939690915 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v160) ==== 172+0+0 (secure 0 0 0) 0x7f3554059460 con 0x7f356409de80 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 msgr2=0x7f354806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3560009990 tx=0x7f3560008040 comp rx=0 tx=0).stop 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 msgr2=0x7f35640b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f355400e7e0 tx=0x7f355400eca0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 shutdown_connections 2024-04-13T03:12:05.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f356409e300 0x7f35640b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f356409de80 0x7f35640b1cf0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 --2- 172.21.15.106:0/1939690915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3550004d40 0x7f35641517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 >> 172.21.15.106:0/1939690915 conn(0x7f35640cd6d0 msgr2=0x7f356406f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 shutdown_connections 2024-04-13T03:12:05.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.650+0000 7f356b98c640 1 -- 172.21.15.106:0/1939690915 wait complete. 2024-04-13T03:12:05.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:12:05.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.764+0000 7f9c12155640 1 Processor -- start 2024-04-13T03:12:05.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.764+0000 7f9c12155640 1 -- start start 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c12155640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c12155640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 0x7f9c0c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c12155640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c0d4110 con 0x7f9c0c0b69c0 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c12155640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c0cfbf0 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c12155640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0b3fd0 con 0x7f9c0c0d4c60 2024-04-13T03:12:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c0b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c0b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51772/0 (socket says 172.21.15.106:51772) 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/1021844480 learned_addr learned my addr 172.21.15.106:0/1021844480 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.765+0000 7f9c0bfff640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 0x7f9c0c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/1021844480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 msgr2=0x7f9c0c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0b7fe640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 0x7f9c0c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/1021844480 >> v1:172.21.15.114:6789/0 conn(0x7f9c0c0b69c0 legacy=0x7f9c0c0d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/1021844480 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c0d3500 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0bfff640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 0x7f9c0c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0b7fe640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009870 tx=0x7f9bfc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c06d63039af528 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/1021844480 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0317c0 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/1021844480 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bfc037070 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.766+0000 7f9c0a7fc640 1 -- 172.21.15.106:0/1021844480 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc031b80 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 -- 172.21.15.106:0/1021844480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 msgr2=0x7f9c0c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009870 tx=0x7f9bfc02b860 comp rx=0 tx=0).stop 2024-04-13T03:12:05.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 -- 172.21.15.106:0/1021844480 shutdown_connections 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0d4c60 0x7f9c0c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 --2- 172.21.15.106:0/1021844480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c0d2c70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 -- 172.21.15.106:0/1021844480 >> 172.21.15.106:0/1021844480 conn(0x7f9c0c0b96e0 msgr2=0x7f9c0c0b9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 -- 172.21.15.106:0/1021844480 shutdown_connections 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.767+0000 7f9c12155640 1 -- 172.21.15.106:0/1021844480 wait complete. 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.768+0000 7f9c12155640 1 Processor -- start 2024-04-13T03:12:05.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.768+0000 7f9c12155640 1 -- start start 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 0x7f9c0c06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0d4c60 0x7f9c0c0d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0d3da0 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0d3ac0 con 0x7f9c0c0d4c60 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c12155640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0d31b0 con 0x7f9c0c0b69c0 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51776/0 (socket says 172.21.15.106:51776) 2024-04-13T03:12:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/69590381 learned_addr learned my addr 172.21.15.106:0/69590381 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0bfff640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0d4c60 0x7f9c0c0d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0affd640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 0x7f9c0c06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 msgr2=0x7f9c0c06c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 0x7f9c0c06c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0d4c60 msgr2=0x7f9c0c0d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.769+0000 7f9c0b7fe640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0d4c60 0x7f9c0c0d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9c0b7fe640 1 -- 172.21.15.106:0/69590381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c157e00 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9c0affd640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 0x7f9c0c06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9c0b7fe640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc032650 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bfc032810 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c06d360 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc032c00 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.770+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c06d920 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.771+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c069120 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.771+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bfc04c060 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.772+0000 7f9bf3fff640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.772+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f9bfc0ad140 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.772+0000 7f9c0affd640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.772+0000 7f9c0affd640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0b3370 tx=0x7f9c00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:05.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.776+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bfc040070 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.950+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9c0c0d48d0 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.953+0000 7f9bf3fff640 1 -- 172.21.15.106:0/69590381 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f9bfc07b770 con 0x7f9c0c0b3bf0 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 msgr2=0x7f9bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0b3370 tx=0x7f9c00009040 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 msgr2=0x7f9c0c1578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc0046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 shutdown_connections 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9bf806c8f0 0x7f9bf806edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9c0c0b69c0 0x7f9c0c06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0c0d4c60 0x7f9c0c0d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 --2- 172.21.15.106:0/69590381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c0c0b3bf0 0x7f9c0c1578c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 >> 172.21.15.106:0/69590381 conn(0x7f9c0c0b96e0 msgr2=0x7f9c0c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 shutdown_connections 2024-04-13T03:12:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:05.955+0000 7f9c12155640 1 -- 172.21.15.106:0/69590381 wait complete. 2024-04-13T03:12:05.967 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:12:05.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:12:06.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:12:06.072 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:12:06.073 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:12:06.073 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:12:06.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:12:06.088 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:12:06.088 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:12:06.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:12:06.105 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:12:06.105 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:12:06.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:12:06.142 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:12:06.142 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:12:06.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:12:06.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:12:06.162 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:12:06.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.282+0000 7fe14d488640 1 Processor -- start 2024-04-13T03:12:06.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.282+0000 7fe14d488640 1 -- start start 2024-04-13T03:12:06.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe14d488640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe14815e9b0 0x7fe1480d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe14d488640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1480cfd20 con 0x7fe14815ed90 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe14d488640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe148162af0 con 0x7fe1480b3d60 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe14d488640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe148162dd0 con 0x7fe14815e9b0 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe1467fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe14815ed90 0x7fe1480d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51908/0 (socket says 172.21.15.106:51908) 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe1467fc640 1 -- 172.21.15.106:0/3407704558 learned_addr learned my addr 172.21.15.106:0/3407704558 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573888408 0 0) 0x7fe1480cfd20 con 0x7fe14815ed90 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe130003520 con 0x7fe14815ed90 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe146ffd640 1 --2- 172.21.15.106:0/3407704558 >> v2:172.21.15.114:3300/0 conn(0x7fe14815e9b0 0x7fe1480d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674373568 0 0) 0x7fe130003520 con 0x7fe14815ed90 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 >> v2:172.21.15.114:3300/0 conn(0x7fe14815e9b0 msgr2=0x7fe1480d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 --2- 172.21.15.106:0/3407704558 >> v2:172.21.15.114:3300/0 conn(0x7fe14815e9b0 0x7fe1480d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 >> v1:172.21.15.114:6789/0 conn(0x7fe1480b3d60 legacy=0x7fe1480d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1481630b0 con 0x7fe14815ed90 2024-04-13T03:12:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fe13c0017f0 con 0x7fe14815ed90 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 >> v1:172.21.15.106:6789/0 conn(0x7fe14815ed90 legacy=0x7fe1480d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe130003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.283+0000 7fe145ffb640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe145ffb640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1300051c0 0x7fe13000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe148162dd0 con 0x7fe1300039b0 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe13000c070 con 0x7fe1300051c0 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe13000c270 con 0x7fe130004800 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe130003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe1467fc640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe1477fe640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1300051c0 0x7fe13000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 -- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe130004800 msgr2=0x7fe130004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 -- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1300051c0 msgr2=0x7fe13000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1300051c0 0x7fe13000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe1477fe640 1 -- 172.21.15.106:0/3407704558 reap_dead start 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe146ffd640 1 -- 172.21.15.106:0/3407704558 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe130003520 con 0x7fe1300039b0 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.284+0000 7fe1467fc640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe146ffd640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe130003dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe138009ac0 tx=0x7fe130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d80b471038aada3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138039070 con 0x7fe1300039b0 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13802c6b0 con 0x7fe1300039b0 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe145ffb640 1 -- 172.21.15.106:0/3407704558 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138009690 con 0x7fe1300039b0 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 -- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 msgr2=0x7fe130003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe130003dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe138009ac0 tx=0x7fe130005830 comp rx=0 tx=0).stop 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 -- 172.21.15.106:0/3407704558 shutdown_connections 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 --2- 172.21.15.106:0/3407704558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe130003dd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 -- 172.21.15.106:0/3407704558 >> 172.21.15.106:0/3407704558 conn(0x7fe14808bf10 msgr2=0x7fe1480b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 -- 172.21.15.106:0/3407704558 shutdown_connections 2024-04-13T03:12:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.285+0000 7fe14d488640 1 -- 172.21.15.106:0/3407704558 wait complete. 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 Processor -- start 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 -- start start 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 0x7fe1480ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 0x7fe1480b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe148162800 con 0x7fe1300039b0 2024-04-13T03:12:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe148162680 con 0x7fe1480b4e30 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe14d488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe14806eb20 con 0x7fe1480cdd70 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51802/0 (socket says 172.21.15.106:51802) 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.287+0000 7fe146ffd640 1 -- 172.21.15.106:0/4159718930 learned_addr learned my addr 172.21.15.106:0/4159718930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe1477fe640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 0x7fe1480b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe1467fc640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 0x7fe1480ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 -- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 msgr2=0x7fe1480ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 0x7fe1480ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 -- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 msgr2=0x7fe1480b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 0x7fe1480b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 -- 172.21.15.106:0/4159718930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1480cba50 con 0x7fe1300039b0 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe1467fc640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 0x7fe1480ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe1477fe640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 0x7fe1480b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe146ffd640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe138037040 tx=0x7fe138004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138039050 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138045070 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1480cbd30 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380044e0 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.288+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1480cc2f0 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.289+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1480cb7e0 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.290+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe138003a50 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.290+0000 7fe12f7fe640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 0x7fe13406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.290+0000 7fe1467fc640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 0x7fe13406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.290+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7fe1380ad440 con 0x7fe1300039b0 2024-04-13T03:12:06.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.291+0000 7fe1467fc640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 0x7fe13406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001130 tx=0x7fe13c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.295+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe13804e020 con 0x7fe1300039b0 2024-04-13T03:12:06.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.506+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe148068a10 con 0x7fe1300039b0 2024-04-13T03:12:06.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.506+0000 7fe12f7fe640 1 -- 172.21.15.106:0/4159718930 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3881 (secure 0 0 0) 0x7fe13807ba70 con 0x7fe1300039b0 2024-04-13T03:12:06.507 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:06.507 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3104927434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":3104927434},{"type":"v1","addr":"172.21.15.114:6835","nonce":3104927434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2214173143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2214173143},{"type":"v1","addr":"172.21.15.114:6834","nonce":2214173143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6834/1257008338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1257008338},{"type":"v1","addr":"172.21.15.106:6834","nonce":1257008338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1429254575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1429254575},{"type":"v1","addr":"172.21.15.106:6837","nonce":1429254575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-13T03:12:06.508 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 24 2024-04-13T03:12:06.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 msgr2=0x7fe13406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 0x7fe13406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001130 tx=0x7fe13c009040 comp rx=0 tx=0).stop 2024-04-13T03:12:06.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 msgr2=0x7fe148159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe138037040 tx=0x7fe138004730 comp rx=0 tx=0).stop 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 shutdown_connections 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe13406c920 0x7fe13406ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1480cdd70 0x7fe1480ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1480b4e30 0x7fe1480b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 --2- 172.21.15.106:0/4159718930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1300039b0 0x7fe148159640 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 >> 172.21.15.106:0/4159718930 conn(0x7fe14808bf10 msgr2=0x7fe14815c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 shutdown_connections 2024-04-13T03:12:06.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.508+0000 7fe14d488640 1 -- 172.21.15.106:0/4159718930 wait complete. 2024-04-13T03:12:06.520 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:12:06.520 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:12:06.520 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:12:06.520 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:12:06.521 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:12:06.521 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:12:06.521 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:12:06.521 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:12:06.521 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:12:06.521 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:12:06.522 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:12:06.522 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:12:06.522 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:12:06.522 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:12:06.522 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:12:06.522 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:12:06.531 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:12:06.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:12:06.532 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:12:06.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:12:06.534 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:12:06.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:12:06.535 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:12:06.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:12:06.573 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:12:06.573 DEBUG:teuthology.parallel:result is None 2024-04-13T03:12:06.573 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:12:06.574 DEBUG:teuthology.parallel:result is None 2024-04-13T03:12:06.577 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:12:06.577 DEBUG:teuthology.parallel:result is None 2024-04-13T03:12:06.577 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:12:06.578 DEBUG:teuthology.parallel:result is None 2024-04-13T03:12:06.578 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:12:06.650 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:12:06.650 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:12:06.662 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:12:06.662 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:12:06.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.682+0000 7f220d100640 1 Processor -- start 2024-04-13T03:12:06.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.682+0000 7f220d100640 1 -- start start 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f220d100640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f220815b750 0x7f22080cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f220d100640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220806e200 con 0x7f2208070580 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f220d100640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22080c63d0 con 0x7f22080d0360 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f220d100640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f22080c7190 con 0x7f220815b750 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f2206d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f220815b750 0x7f22080cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f21fffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2208070580 0x7f22080709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51936/0 (socket says 172.21.15.106:51936) 2024-04-13T03:12:06.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f21fffff640 1 -- 172.21.15.106:0/2762716453 learned_addr learned my addr 172.21.15.106:0/2762716453 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f2206d9d640 1 --2- 172.21.15.106:0/2762716453 >> v2:172.21.15.106:3300/0 conn(0x7f220815b750 0x7f22080cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51838/0 (socket says 172.21.15.106:51838) 2024-04-13T03:12:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f2206d9d640 1 -- 172.21.15.106:0/2762716453 >> v1:172.21.15.114:6789/0 conn(0x7f22080d0360 legacy=0x7f22080d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f2206d9d640 1 -- 172.21.15.106:0/2762716453 >> v1:172.21.15.106:6789/0 conn(0x7f2208070580 legacy=0x7f22080709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.683+0000 7f2206d9d640 1 -- 172.21.15.106:0/2762716453 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22080c66b0 con 0x7f220815b750 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f2206d9d640 1 --2- 172.21.15.106:0/2762716453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220815b750 0x7f22080cd260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e4b72afdd9a431 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f220659c640 1 -- 172.21.15.106:0/2762716453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8035020 con 0x7f220815b750 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f220659c640 1 -- 172.21.15.106:0/2762716453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8031830 con 0x7f220815b750 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f220659c640 1 -- 172.21.15.106:0/2762716453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031c00 con 0x7f220815b750 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f220d100640 1 -- 172.21.15.106:0/2762716453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220815b750 msgr2=0x7f22080cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.684+0000 7f220d100640 1 --2- 172.21.15.106:0/2762716453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220815b750 0x7f22080cd260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.685+0000 7f220d100640 1 -- 172.21.15.106:0/2762716453 shutdown_connections 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.685+0000 7f220d100640 1 --2- 172.21.15.106:0/2762716453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220815b750 0x7f22080cd260 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.685+0000 7f220d100640 1 -- 172.21.15.106:0/2762716453 >> 172.21.15.106:0/2762716453 conn(0x7f22080d1650 msgr2=0x7f22080d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:06.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.685+0000 7f220d100640 1 -- 172.21.15.106:0/2762716453 shutdown_connections 2024-04-13T03:12:06.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.685+0000 7f220d100640 1 -- 172.21.15.106:0/2762716453 wait complete. 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.686+0000 7f220d100640 1 Processor -- start 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.686+0000 7f220d100640 1 -- start start 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080d0360 0x7f22080b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220815b750 0x7f22081f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f220815bb30 con 0x7f2208070580 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f22080c6c70 con 0x7f22080d0360 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f220806de20 con 0x7f220815b750 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51840/0 (socket says 172.21.15.106:51840) 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 -- 172.21.15.106:0/721057160 learned_addr learned my addr 172.21.15.106:0/721057160 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220759e640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220815b750 0x7f22081f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f21fffff640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080d0360 0x7f22080b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 -- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220815b750 msgr2=0x7f22081f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220815b750 0x7f22081f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 -- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080d0360 msgr2=0x7f22080b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080d0360 0x7f22080b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 -- 172.21.15.106:0/721057160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081f3710 con 0x7f2208070580 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f2206d9d640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f21f80098c0 tx=0x7f21f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f803ba80 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f803bc40 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081f39f0 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f80453f0 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.687+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081f3fb0 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.688+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208069120 con 0x7f2208070580 2024-04-13T03:12:06.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.689+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21f804c060 con 0x7f2208070580 2024-04-13T03:12:06.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.689+0000 7f21ff7fe640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:06.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.689+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5335+0+0 (secure 0 0 0) 0x7f21f80ad5a0 con 0x7f2208070580 2024-04-13T03:12:06.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.689+0000 7f21fffff640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:06.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.689+0000 7f21fffff640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 0x7f21e806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22081e6cc0 tx=0x7f21f0007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:06.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.693+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21f8040070 con 0x7f2208070580 2024-04-13T03:12:06.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:06.880+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2208070b90 con 0x7f2208070580 2024-04-13T03:12:07.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.964+0000 7f21ff7fe640 1 -- 172.21.15.106:0/721057160 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) ==== 97+0+0 (secure 0 0 0) 0x7f21f807b580 con 0x7f2208070580 2024-04-13T03:12:07.966 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:12:07.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 msgr2=0x7f21e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:07.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 0x7f21e806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22081e6cc0 tx=0x7f21f0007e80 comp rx=0 tx=0).stop 2024-04-13T03:12:07.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 msgr2=0x7f220815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f21f80098c0 tx=0x7f21f80046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 shutdown_connections 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220815b750 0x7f22081f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080d0360 0x7f22080b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 --2- 172.21.15.106:0/721057160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2208070580 0x7f220815e9b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 >> 172.21.15.106:0/721057160 conn(0x7f22080d1650 msgr2=0x7f22080ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 shutdown_connections 2024-04-13T03:12:07.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:07.967+0000 7f220d100640 1 -- 172.21.15.106:0/721057160 wait complete. 2024-04-13T03:12:07.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:12:08.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.086+0000 7f5c2c38c640 1 Processor -- start 2024-04-13T03:12:08.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.086+0000 7f5c2c38c640 1 -- start start 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.087+0000 7f5c2c38c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c24160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.087+0000 7f5c2c38c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c240d2310 con 0x7f5c2415c7d0 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.087+0000 7f5c2c38c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c240b1f30 con 0x7f5c240c60a0 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.087+0000 7f5c2c38c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5c240b2210 con 0x7f5c240cd2b0 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5c2415c7d0 0x7f5c240ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51946/0 (socket says 172.21.15.106:51946) 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 -- 172.21.15.106:0/2478685190 learned_addr learned my addr 172.21.15.106:0/2478685190 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:08.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785219786 0 0) 0x7f5c240d2310 con 0x7f5c2415c7d0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29927640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c24160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c10003520 con 0x7f5c2415c7d0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2244408587 0 0) 0x7f5c10003520 con 0x7f5c2415c7d0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 msgr2=0x7f5c24160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c24160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 >> v1:172.21.15.114:6789/0 conn(0x7f5c240c60a0 legacy=0x7f5c240b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c240b24f0 con 0x7f5c2415c7d0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5c20001670 con 0x7f5c2415c7d0 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 >> v1:172.21.15.106:6789/0 conn(0x7f5c2415c7d0 legacy=0x7f5c240ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c100039e0 0x7f5c10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c10004d40 0x7f5c100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5c240b2210 con 0x7f5c100039e0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1000ba70 con 0x7f5c10004d40 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1000bc70 con 0x7f5c240cd2b0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c100039e0 0x7f5c10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c29927640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a929640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c10004d40 0x7f5c100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 -- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 msgr2=0x7f5c10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c240cd2b0 0x7f5c10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 -- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c10004d40 msgr2=0x7f5c100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c10004d40 0x7f5c100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.088+0000 7f5c2a128640 1 -- 172.21.15.106:0/2478685190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c10003520 con 0x7f5c100039e0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.089+0000 7f5c2a128640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c100039e0 0x7f5c10003e00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5c20001eb0 tx=0x7f5c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8f5825798c13f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.089+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20007d30 con 0x7f5c100039e0 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.089+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c20030070 con 0x7f5c100039e0 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.089+0000 7f5c29126640 1 -- 172.21.15.106:0/2478685190 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20005d30 con 0x7f5c100039e0 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 -- 172.21.15.106:0/2478685190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c100039e0 msgr2=0x7f5c10003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/2478685190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c100039e0 0x7f5c10003e00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5c20001eb0 tx=0x7f5c10005830 comp rx=0 tx=0).stop 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2a929640 1 -- 172.21.15.106:0/2478685190 reap_dead start 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 -- 172.21.15.106:0/2478685190 shutdown_connections 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 -- 172.21.15.106:0/2478685190 >> 172.21.15.106:0/2478685190 conn(0x7f5c240d2550 msgr2=0x7f5c240d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 -- 172.21.15.106:0/2478685190 shutdown_connections 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.090+0000 7f5c2c38c640 1 -- 172.21.15.106:0/2478685190 wait complete. 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.091+0000 7f5c2c38c640 1 Processor -- start 2024-04-13T03:12:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.091+0000 7f5c2c38c640 1 -- start start 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 0x7f5c2415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 0x7f5c241e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2415cbb0 con 0x7f5c240b7160 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24160dd0 con 0x7f5c100039e0 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2c38c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5c241e6b90 con 0x7f5c241e60d0 2024-04-13T03:12:08.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51852/0 (socket says 172.21.15.106:51852) 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 -- 172.21.15.106:0/1357245003 learned_addr learned my addr 172.21.15.106:0/1357245003 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a128640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 0x7f5c2415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c29927640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 0x7f5c241e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 -- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 msgr2=0x7f5c241e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 0x7f5c241e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 -- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 msgr2=0x7f5c2415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 0x7f5c2415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 -- 172.21.15.106:0/1357245003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c241e6e70 con 0x7f5c240b7160 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c29927640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 0x7f5c241e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a128640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 0x7f5c2415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:08.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.092+0000 7f5c2a929640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5c24073a80 tx=0x7f5c1c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.093+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003970 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.093+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c1c018070 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.093+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c241e7150 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.093+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003d50 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.093+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c241e7710 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.094+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c241e7910 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.094+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c1c0050d0 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.094+0000 7f5c1affd640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 0x7f5c0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.094+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f5c1c08b600 con 0x7f5c240b7160 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.094+0000 7f5c2a128640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 0x7f5c0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.095+0000 7f5c2a128640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 0x7f5c0806ede0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5c20034000 tx=0x7f5c20009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.099+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c241e7910 con 0x7f5c240b7160 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.340+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5c24069120 con 0x7f5c240b7160 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.341+0000 7f5c1affd640 1 -- 172.21.15.106:0/1357245003 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f5c1c059bf0 con 0x7f5c240b7160 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:12:08.343 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:12:08.344 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.345 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.346 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.347 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.349 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.350 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:12:08.351 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:12:08.352 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.353 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.354 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.355 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:12:08.356 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 msgr2=0x7f5c0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 0x7f5c0806ede0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5c20034000 tx=0x7f5c20009030 comp rx=0 tx=0).stop 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 msgr2=0x7f5c240b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5c24073a80 tx=0x7f5c1c00ec10 comp rx=0 tx=0).stop 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 shutdown_connections 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5c0806c920 0x7f5c0806ede0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5c241e60d0 0x7f5c241e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c240b7160 0x7f5c240b75e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 --2- 172.21.15.106:0/1357245003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c100039e0 0x7f5c2415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 >> 172.21.15.106:0/1357245003 conn(0x7f5c240d2550 msgr2=0x7f5c240c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 shutdown_connections 2024-04-13T03:12:08.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.343+0000 7f5c2c38c640 1 -- 172.21.15.106:0/1357245003 wait complete. 2024-04-13T03:12:08.359 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:08.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 Processor -- start 2024-04-13T03:12:08.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 -- start start 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1d3815ed90 0x7f1d380d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d380cfd20 con 0x7f1d3815e9b0 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1d38162af0 con 0x7f1d3815ed90 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.459+0000 7f1d3cc2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1d38162dd0 con 0x7f1d380b3d60 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:36272/0 (socket says 172.21.15.106:36272) 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d2ffff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1d3815ed90 0x7f1d380d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 -- 172.21.15.106:0/1262745160 learned_addr learned my addr 172.21.15.106:0/1262745160 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:08.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 -- 172.21.15.106:0/1262745160 >> v2:172.21.15.114:3300/0 conn(0x7f1d3815ed90 msgr2=0x7f1d380d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 --2- 172.21.15.106:0/1262745160 >> v2:172.21.15.114:3300/0 conn(0x7f1d3815ed90 0x7f1d380d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 -- 172.21.15.106:0/1262745160 >> v1:172.21.15.114:6789/0 conn(0x7f1d3815e9b0 legacy=0x7f1d380d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.460+0000 7f1d36d9d640 1 -- 172.21.15.106:0/1262745160 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d381630b0 con 0x7f1d380b3d60 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.461+0000 7f1d36d9d640 1 --2- 172.21.15.106:0/1262745160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009e50 tx=0x7f1d3002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e172e4f6f666196 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.461+0000 7f1d35d9b640 1 -- 172.21.15.106:0/1262745160 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3002cb90 con 0x7f1d380b3d60 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.461+0000 7f1d35d9b640 1 -- 172.21.15.106:0/1262745160 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3002cd30 con 0x7f1d380b3d60 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.461+0000 7f1d35d9b640 1 -- 172.21.15.106:0/1262745160 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30034610 con 0x7f1d380b3d60 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/1262745160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 msgr2=0x7f1d380d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/1262745160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009e50 tx=0x7f1d3002bff0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/1262745160 shutdown_connections 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/1262745160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d380b3d60 0x7f1d380d0200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/1262745160 >> v2:172.21.15.114:3300/0 conn(0x7f1d3815ed90 0x7f1d380d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/1262745160 >> 172.21.15.106:0/1262745160 conn(0x7f1d3808bf10 msgr2=0x7f1d380b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/1262745160 shutdown_connections 2024-04-13T03:12:08.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.462+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/1262745160 wait complete. 2024-04-13T03:12:08.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.463+0000 7f1d3cc2d640 1 Processor -- start 2024-04-13T03:12:08.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.463+0000 7f1d3cc2d640 1 -- start start 2024-04-13T03:12:08.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 0x7f1d381597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d3815e9b0 0x7f1d380d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38162800 con 0x7f1d380d3150 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38162680 con 0x7f1d380b3d60 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1d380d4d00 con 0x7f1d3815e9b0 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51862/0 (socket says 172.21.15.106:51862) 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 -- 172.21.15.106:0/3541521798 learned_addr learned my addr 172.21.15.106:0/3541521798 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d2ffff640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d3815e9b0 0x7f1d380d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3659c640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 0x7f1d381597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 -- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d3815e9b0 msgr2=0x7f1d380d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d3815e9b0 0x7f1d380d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 -- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 msgr2=0x7f1d381597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 0x7f1d381597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 -- 172.21.15.106:0/3541521798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d380c9f40 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3659c640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 0x7f1d381597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d36d9d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1d30036040 tx=0x7f1d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30043020 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3003e450 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d380ca220 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30009510 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.464+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d380ca7e0 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.465+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3815bbf0 con 0x7f1d380d3150 2024-04-13T03:12:08.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.466+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d30046020 con 0x7f1d380d3150 2024-04-13T03:12:08.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.466+0000 7f1d2effd640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 0x7f1d1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.466+0000 7f1d3659c640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 0x7f1d1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.466+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f1d300ae000 con 0x7f1d380d3150 2024-04-13T03:12:08.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.467+0000 7f1d3659c640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 0x7f1d1806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d380d51e0 tx=0x7f1d28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.470+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d3815bbf0 con 0x7f1d380d3150 2024-04-13T03:12:08.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.645+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d38068a10 con 0x7f1d380d3150 2024-04-13T03:12:08.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.646+0000 7f1d2effd640 1 -- 172.21.15.106:0/3541521798 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13200 (secure 0 0 0) 0x7f1d3004d030 con 0x7f1d380d3150 2024-04-13T03:12:08.648 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:08.648 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":60,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:12:07.964661+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 msgr2=0x7f1d1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 0x7f1d1806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d380d51e0 tx=0x7f1d28009040 comp rx=0 tx=0).stop 2024-04-13T03:12:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 msgr2=0x7f1d380cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1d30036040 tx=0x7f1d30004a40 comp rx=0 tx=0).stop 2024-04-13T03:12:08.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 shutdown_connections 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1d1806c8f0 0x7f1d1806edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1d3815e9b0 0x7f1d380d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d380d3150 0x7f1d380cc390 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 --2- 172.21.15.106:0/3541521798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d380b3d60 0x7f1d381597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 >> 172.21.15.106:0/3541521798 conn(0x7f1d3808bf10 msgr2=0x7f1d380b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 shutdown_connections 2024-04-13T03:12:08.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.648+0000 7f1d3cc2d640 1 -- 172.21.15.106:0/3541521798 wait complete. 2024-04-13T03:12:08.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:12:08.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:12:08.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.763+0000 7f1a07498640 1 Processor -- start 2024-04-13T03:12:08.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 -- start start 2024-04-13T03:12:08.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f1a000b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1a000ceb70 0x7f1a000b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a000b6120 con 0x7f1a001626b0 2024-04-13T03:12:08.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1a000cde50 con 0x7f1a000ceb70 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.764+0000 7f1a07498640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1a000cd240 con 0x7f1a0015ec20 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a05a35640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1a001626b0 0x7f1a000b7590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51950/0 (socket says 172.21.15.106:51950) 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a05a35640 1 -- 172.21.15.106:0/2250861547 learned_addr learned my addr 172.21.15.106:0/2250861547 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236034028 0 0) 0x7f1a000b6120 con 0x7f1a001626b0 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 --2- 172.21.15.106:0/2250861547 >> v2:172.21.15.114:3300/0 conn(0x7f1a000ceb70 0x7f1a000b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a05234640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f1a000b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19ec003520 con 0x7f1a001626b0 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1322322143 0 0) 0x7f19ec003520 con 0x7f1a001626b0 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 msgr2=0x7f1a000b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f1a000b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 >> v2:172.21.15.114:3300/0 conn(0x7f1a000ceb70 msgr2=0x7f1a000b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 --2- 172.21.15.106:0/2250861547 >> v2:172.21.15.114:3300/0 conn(0x7f1a000ceb70 0x7f1a000b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a000cd520 con 0x7f1a001626b0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f19f80019f0 con 0x7f1a001626b0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 >> v1:172.21.15.106:6789/0 conn(0x7f1a001626b0 legacy=0x7f1a000b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f19ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cde50 con 0x7f19ec004800 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cd240 con 0x7f19ec0039b0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec00bb20 con 0x7f1a0015ec20 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a05234640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a05a35640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f19ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 -- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 msgr2=0x7f19ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a0015ec20 0x7f19ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 -- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19ec0039b0 msgr2=0x7f19ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.765+0000 7f1a04a33640 1 -- 172.21.15.106:0/2250861547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec003520 con 0x7f19ec004800 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.766+0000 7f1a04a33640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ba00 tx=0x7f19ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a05987cc2df86b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.766+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0036020 con 0x7f19ec004800 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.766+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f0032630 con 0x7f19ec004800 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.766+0000 7f19f7fff640 1 -- 172.21.15.106:0/2250861547 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f000b600 con 0x7f19ec004800 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 -- 172.21.15.106:0/2250861547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19ec004800 msgr2=0x7f19ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 --2- 172.21.15.106:0/2250861547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ba00 tx=0x7f19ec005830 comp rx=0 tx=0).stop 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a05a35640 1 -- 172.21.15.106:0/2250861547 reap_dead start 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 -- 172.21.15.106:0/2250861547 shutdown_connections 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 -- 172.21.15.106:0/2250861547 >> 172.21.15.106:0/2250861547 conn(0x7f1a000d2530 msgr2=0x7f1a000d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 -- 172.21.15.106:0/2250861547 shutdown_connections 2024-04-13T03:12:08.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.767+0000 7f1a07498640 1 -- 172.21.15.106:0/2250861547 wait complete. 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.768+0000 7f1a07498640 1 Processor -- start 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.768+0000 7f1a07498640 1 -- start start 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19ec004800 0x7f1a001f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a001f8cd0 0x7f1a0015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cced0 con 0x7f1a001f9350 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cdae0 con 0x7f1a001f8cd0 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a07498640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1a000d14d0 con 0x7f19ec004800 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51890/0 (socket says 172.21.15.106:51890) 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 -- 172.21.15.106:0/3991895976 learned_addr learned my addr 172.21.15.106:0/3991895976 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a05234640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19ec004800 0x7f1a001f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 -- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19ec004800 msgr2=0x7f1a001f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19ec004800 0x7f1a001f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 -- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a001f8cd0 msgr2=0x7f1a0015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a001f8cd0 0x7f1a0015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 -- 172.21.15.106:0/3991895976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a00158700 con 0x7f1a001f9350 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.769+0000 7f1a04a33640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f19f000bb20 tx=0x7f19f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.770+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0041070 con 0x7f1a001f9350 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.770+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f0030a40 con 0x7f1a001f9350 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.770+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a001589e0 con 0x7f1a001f9350 2024-04-13T03:12:08.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.770+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0036040 con 0x7f1a001f9350 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.770+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a00158fa0 con 0x7f1a001f9350 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.771+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a001f90f0 con 0x7f1a001f9350 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.771+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19f0039020 con 0x7f1a001f9350 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.771+0000 7f19f5ffb640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.772+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f19f00adad0 con 0x7f1a001f9350 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.772+0000 7f1a05234640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.772+0000 7f1a05234640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 0x7f19e406ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006fd0 tx=0x7f19fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:08.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.776+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19f007c150 con 0x7f1a001f9350 2024-04-13T03:12:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.948+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a00073da0 con 0x7f1a001f9350 2024-04-13T03:12:08.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.948+0000 7f19f5ffb640 1 -- 172.21.15.106:0/3991895976 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f1a00073da0 con 0x7f1a001f9350 2024-04-13T03:12:08.950 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:12:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 msgr2=0x7f19e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 0x7f19e406ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006fd0 tx=0x7f19fc008040 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 msgr2=0x7f1a001f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f19f000bb20 tx=0x7f19f0004730 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 shutdown_connections 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a001f8cd0 0x7f1a0015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a001f9350 0x7f1a001f8790 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 --2- 172.21.15.106:0/3991895976 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f19ec004800 0x7f1a001f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 >> 172.21.15.106:0/3991895976 conn(0x7f1a000d2530 msgr2=0x7f1a000c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 shutdown_connections 2024-04-13T03:12:08.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:08.950+0000 7f1a07498640 1 -- 172.21.15.106:0/3991895976 wait complete. 2024-04-13T03:12:08.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:12:09.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.061+0000 7f0b0fede640 1 Processor -- start 2024-04-13T03:12:09.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.061+0000 7f0b0fede640 1 -- start start 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0fede640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b081622d0 0x7f0b080d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0fede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 0x7f0b080b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0fede640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b080d3530 con 0x7f0b080cc6f0 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0fede640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b081626b0 con 0x7f0b081622d0 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0fede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b080d0b60 con 0x7f0b080d1c00 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0e47b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b080cc6f0 0x7f0b080ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51962/0 (socket says 172.21.15.106:51962) 2024-04-13T03:12:09.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0e47b640 1 -- 172.21.15.106:0/3321200282 learned_addr learned my addr 172.21.15.106:0/3321200282 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/3321200282 >> v2:172.21.15.114:3300/0 conn(0x7f0b081622d0 0x7f0b080d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0d479640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 0x7f0b080b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811184854 0 0) 0x7f0b080d3530 con 0x7f0b080cc6f0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b080cc6f0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430009512 0 0) 0x7f0af4003520 con 0x7f0b080cc6f0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0dc7a640 1 -- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 msgr2=0x7f0b080b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 0x7f0b080b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0dc7a640 1 -- 172.21.15.106:0/3321200282 >> v1:172.21.15.106:6789/0 conn(0x7f0b080cc6f0 legacy=0x7f0b080ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0dc7a640 1 -- 172.21.15.106:0/3321200282 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b080d0e40 con 0x7f0b081622d0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.062+0000 7f0b0d479640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 0x7f0b080b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.063+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b081622d0 0x7f0b080d04d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097a0 tx=0x7f0b0402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20b22baaa003d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.063+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04037070 con 0x7f0b081622d0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.063+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b04030510 con 0x7f0b081622d0 2024-04-13T03:12:09.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.063+0000 7f0b0cc78640 1 -- 172.21.15.106:0/3321200282 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000019f0 con 0x7f0b081622d0 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 -- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b081622d0 msgr2=0x7f0b080d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b081622d0 0x7f0b080d04d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097a0 tx=0x7f0b0402b790 comp rx=0 tx=0).stop 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 -- 172.21.15.106:0/3321200282 shutdown_connections 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b080d1c00 0x7f0b080b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 --2- 172.21.15.106:0/3321200282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b081622d0 0x7f0b080d04d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 -- 172.21.15.106:0/3321200282 >> 172.21.15.106:0/3321200282 conn(0x7f0b080d25d0 msgr2=0x7f0b080d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 -- 172.21.15.106:0/3321200282 shutdown_connections 2024-04-13T03:12:09.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.064+0000 7f0b0fede640 1 -- 172.21.15.106:0/3321200282 wait complete. 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.065+0000 7f0b0fede640 1 Processor -- start 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.065+0000 7f0b0fede640 1 -- start start 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b080cc6f0 0x7f0b080c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b081622d0 0x7f0b08153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b080d16e0 con 0x7f0b080d1c00 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b080d1400 con 0x7f0b080cc6f0 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0fede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b080b7eb0 con 0x7f0b081622d0 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51894/0 (socket says 172.21.15.106:51894) 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 -- 172.21.15.106:0/2431861867 learned_addr learned my addr 172.21.15.106:0/2431861867 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0e47b640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b081622d0 0x7f0b08153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b080cc6f0 0x7f0b080c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 -- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b081622d0 msgr2=0x7f0b08153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b081622d0 0x7f0b08153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 -- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b080cc6f0 msgr2=0x7f0b080c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b080cc6f0 0x7f0b080c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.066+0000 7f0b0d479640 1 -- 172.21.15.106:0/2431861867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b080c61c0 con 0x7f0b080d1c00 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0b0d479640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0af800ebe0 tx=0x7f0af800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8017070 con 0x7f0b080d1c00 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af800cc30 con 0x7f0b080d1c00 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b080c6460 con 0x7f0b080d1c00 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8013470 con 0x7f0b080d1c00 2024-04-13T03:12:09.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.067+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b080c6a20 con 0x7f0b080d1c00 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.068+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08068a10 con 0x7f0b080d1c00 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.068+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0af801a070 con 0x7f0b080d1c00 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.069+0000 7f0afe7fc640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.069+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.069+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f0af808bc00 con 0x7f0b080d1c00 2024-04-13T03:12:09.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.069+0000 7f0b0dc7a640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b04034000 tx=0x7f0b0402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.073+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0af8092080 con 0x7f0b080d1c00 2024-04-13T03:12:09.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.252+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0b080c7180 con 0x7f0b080d1c00 2024-04-13T03:12:09.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.252+0000 7f0afe7fc640 1 -- 172.21.15.106:0/2431861867 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) ==== 84+0+65 (secure 0 0 0) 0x7f0af8059ba0 con 0x7f0b080d1c00 2024-04-13T03:12:09.254 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:09.254 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:12:09.254 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:09.255 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:12:09.255 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:12:09.255 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:09.255 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 msgr2=0x7f0aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b04034000 tx=0x7f0b0402f000 comp rx=0 tx=0).stop 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 msgr2=0x7f0b080c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0af800ebe0 tx=0x7f0af800c520 comp rx=0 tx=0).stop 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 shutdown_connections 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b081622d0 0x7f0b08153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b080d1c00 0x7f0b080c5c80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 --2- 172.21.15.106:0/2431861867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b080cc6f0 0x7f0b080c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 >> 172.21.15.106:0/2431861867 conn(0x7f0b080d25d0 msgr2=0x7f0b080ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 shutdown_connections 2024-04-13T03:12:09.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.254+0000 7f0b0fede640 1 -- 172.21.15.106:0/2431861867 wait complete. 2024-04-13T03:12:09.267 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:12:09.267 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:12:09.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.369+0000 7f4b0e6de640 1 Processor -- start 2024-04-13T03:12:09.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.369+0000 7f4b0e6de640 1 -- start start 2024-04-13T03:12:09.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b0e6de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b080d05f0 0x7f4b080cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b0e6de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b080cedf0 con 0x7f4b080d0140 2024-04-13T03:12:09.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b0e6de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b081628e0 con 0x7f4b080cea10 2024-04-13T03:12:09.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b0e6de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4b08162bc0 con 0x7f4b080d05f0 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b077fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4b080d0140 0x7f4b080cce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51974/0 (socket says 172.21.15.106:51974) 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.370+0000 7f4b077fe640 1 -- 172.21.15.106:0/3014080710 learned_addr learned my addr 172.21.15.106:0/3014080710 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/3014080710 >> v2:172.21.15.114:3300/0 conn(0x7f4b080d05f0 0x7f4b080cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49480889 0 0) 0x7f4b080cedf0 con 0x7f4b080d0140 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4af0003520 con 0x7f4b080d0140 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132007247 0 0) 0x7f4b081628e0 con 0x7f4b080cea10 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b080cedf0 con 0x7f4b080cea10 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3104761202 0 0) 0x7f4af0003520 con 0x7f4b080d0140 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 >> v2:172.21.15.114:3300/0 conn(0x7f4b080d05f0 msgr2=0x7f4b080cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 --2- 172.21.15.106:0/3014080710 >> v2:172.21.15.114:3300/0 conn(0x7f4b080d05f0 0x7f4b080cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 >> v1:172.21.15.114:6789/0 conn(0x7f4b080cea10 legacy=0x7f4b0815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b08162ea0 con 0x7f4b080d0140 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4afc001670 con 0x7f4b080d0140 2024-04-13T03:12:09.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 >> v1:172.21.15.106:6789/0 conn(0x7f4b080d0140 legacy=0x7f4b080cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af0003b40 0x7f4af0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08162bc0 con 0x7f4af00051c0 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4af000c040 con 0x7f4af0003b40 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4af000c240 con 0x7f4af0004800 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b07fff640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af0003b40 0x7f4af0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b077fe640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4af0004800 msgr2=0x7f4af0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af0003b40 msgr2=0x7f4af0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af0003b40 0x7f4af0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/3014080710 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af0003520 con 0x7f4af00051c0 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.371+0000 7f4b077fe640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.372+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/3014080710 reap_dead start 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.372+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009fc0 tx=0x7f4af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90956f5fc332b19a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.372+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00009c10 con 0x7f4af00051c0 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.372+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0002cc60 con 0x7f4af00051c0 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.372+0000 7f4b06ffd640 1 -- 172.21.15.106:0/3014080710 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003bd0 con 0x7f4af00051c0 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 -- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 msgr2=0x7f4af000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009fc0 tx=0x7f4af0005830 comp rx=0 tx=0).stop 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 -- 172.21.15.106:0/3014080710 shutdown_connections 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/3014080710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 -- 172.21.15.106:0/3014080710 >> 172.21.15.106:0/3014080710 conn(0x7f4b080b4060 msgr2=0x7f4b080b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:09.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 -- 172.21.15.106:0/3014080710 shutdown_connections 2024-04-13T03:12:09.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.373+0000 7f4b0e6de640 1 -- 172.21.15.106:0/3014080710 wait complete. 2024-04-13T03:12:09.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.374+0000 7f4b0e6de640 1 Processor -- start 2024-04-13T03:12:09.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.374+0000 7f4b0e6de640 1 -- start start 2024-04-13T03:12:09.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af00051c0 0x7f4b080b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b080c69b0 0x7f4b0814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081625b0 con 0x7f4b0814b6f0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080d0a10 con 0x7f4af00051c0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0e6de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b08163460 con 0x7f4b080c69b0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51914/0 (socket says 172.21.15.106:51914) 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/1433025891 learned_addr learned my addr 172.21.15.106:0/1433025891 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b07fff640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af00051c0 0x7f4b080b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b077fe640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b080c69b0 0x7f4b0814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b080c69b0 msgr2=0x7f4b0814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b080c69b0 0x7f4b0814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af00051c0 msgr2=0x7f4b080b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af00051c0 0x7f4b080b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 -- 172.21.15.106:0/1433025891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080b31a0 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b0cc7b640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000e200 tx=0x7f4b00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.375+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003990 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.376+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00044070 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.376+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003d50 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.376+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0814bf20 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.376+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0814c4e0 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.377+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b08069120 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.377+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b00005d10 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.377+0000 7f4b04ff9640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.378+0000 7f4b07fff640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.378+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f4b000adfc0 con 0x7f4b0814b6f0 2024-04-13T03:12:09.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.378+0000 7f4b07fff640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 0x7f4af406edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4af8000c00 tx=0x7f4af8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:09.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.382+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b0007c640 con 0x7f4b0814b6f0 2024-04-13T03:12:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.555+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b080705c0 con 0x7f4b0814b6f0 2024-04-13T03:12:09.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.967+0000 7f4b04ff9640 1 -- 172.21.15.106:0/1433025891 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) ==== 138+0+0 (secure 0 0 0) 0x7f4b0007bf40 con 0x7f4b0814b6f0 2024-04-13T03:12:09.969 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:12:09.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.970+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 msgr2=0x7f4af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.970+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 0x7f4af406edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4af8000c00 tx=0x7f4af8009040 comp rx=0 tx=0).stop 2024-04-13T03:12:09.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.970+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 msgr2=0x7f4b0814cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:09.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.970+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000e200 tx=0x7f4b00009750 comp rx=0 tx=0).stop 2024-04-13T03:12:09.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 shutdown_connections 2024-04-13T03:12:09.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b080c69b0 0x7f4b0814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b0814b6f0 0x7f4b0814cfc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 --2- 172.21.15.106:0/1433025891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af00051c0 0x7f4b080b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:09.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 >> 172.21.15.106:0/1433025891 conn(0x7f4b080b4060 msgr2=0x7f4b080cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:09.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 shutdown_connections 2024-04-13T03:12:09.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:09.971+0000 7f4b0e6de640 1 -- 172.21.15.106:0/1433025891 wait complete. 2024-04-13T03:12:09.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:12:10.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.092+0000 7f4d14036640 1 Processor -- start 2024-04-13T03:12:10.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.092+0000 7f4d14036640 1 -- start start 2024-04-13T03:12:10.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d14036640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 0x7f4d0c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d14036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d14036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c15f380 con 0x7f4d0c06b8e0 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d14036640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0c0d2d90 con 0x7f4d0c0cf480 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d14036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c0cfda0 con 0x7f4d0c0ba170 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d11dd2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 0x7f4d0c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:36374/0 (socket says 172.21.15.106:36374) 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 -- 172.21.15.106:0/2440155133 learned_addr learned my addr 172.21.15.106:0/2440155133 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 -- 172.21.15.106:0/2440155133 >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 msgr2=0x7f4d0c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 --2- 172.21.15.106:0/2440155133 >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 0x7f4d0c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 -- 172.21.15.106:0/2440155133 >> v1:172.21.15.114:6789/0 conn(0x7f4d0c06b8e0 legacy=0x7f4d0c06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d115d1640 1 -- 172.21.15.106:0/2440155133 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c15f600 con 0x7f4d0c0ba170 2024-04-13T03:12:10.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d10dd0640 1 -- 172.21.15.106:0/2440155133 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718381476 0 0) 0x7f4d04001490 con 0x7f4d0c06b8e0 2024-04-13T03:12:10.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.093+0000 7f4d11dd2640 1 --2- 172.21.15.106:0/2440155133 >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 0x7f4d0c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.094+0000 7f4d115d1640 1 --2- 172.21.15.106:0/2440155133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009870 tx=0x7f4cfc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535a465a4a1a01c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.094+0000 7f4d10dd0640 1 -- 172.21.15.106:0/2440155133 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc034070 con 0x7f4d0c0ba170 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.094+0000 7f4d10dd0640 1 -- 172.21.15.106:0/2440155133 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc030840 con 0x7f4d0c0ba170 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.094+0000 7f4d10dd0640 1 -- 172.21.15.106:0/2440155133 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc030c10 con 0x7f4d0c0ba170 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 -- 172.21.15.106:0/2440155133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 msgr2=0x7f4d0c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 --2- 172.21.15.106:0/2440155133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009870 tx=0x7f4cfc02b980 comp rx=0 tx=0).stop 2024-04-13T03:12:10.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 -- 172.21.15.106:0/2440155133 shutdown_connections 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 --2- 172.21.15.106:0/2440155133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c0ba170 0x7f4d0c0cf860 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 --2- 172.21.15.106:0/2440155133 >> v2:172.21.15.114:3300/0 conn(0x7f4d0c0cf480 0x7f4d0c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 -- 172.21.15.106:0/2440155133 >> 172.21.15.106:0/2440155133 conn(0x7f4d0c0d47d0 msgr2=0x7f4d0c0d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 -- 172.21.15.106:0/2440155133 shutdown_connections 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.095+0000 7f4d14036640 1 -- 172.21.15.106:0/2440155133 wait complete. 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.096+0000 7f4d14036640 1 Processor -- start 2024-04-13T03:12:10.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.096+0000 7f4d14036640 1 -- start start 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 0x7f4d0c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0ba170 0x7f4d0c1f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c159170 con 0x7f4d0c0cf480 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c15fbc0 con 0x7f4d0c0ba170 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d14036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c1592f0 con 0x7f4d0c06b8e0 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51918/0 (socket says 172.21.15.106:51918) 2024-04-13T03:12:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 -- 172.21.15.106:0/2597409807 learned_addr learned my addr 172.21.15.106:0/2597409807 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d11dd2640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 0x7f4d0c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d115d1640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0ba170 0x7f4d0c1f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 -- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 msgr2=0x7f4d0c0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 0x7f4d0c0cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 -- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0ba170 msgr2=0x7f4d0c1f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0ba170 0x7f4d0c1f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 -- 172.21.15.106:0/2597409807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c0cd720 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d11dd2640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 0x7f4d0c0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.097+0000 7f4d125d3640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c780 tx=0x7f4d0400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.098+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04003b70 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.098+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d04003d10 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.098+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04005270 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.098+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c0cd990 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.098+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c0cbfb0 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.099+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c068a10 con 0x7f4d0c0cf480 2024-04-13T03:12:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.099+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d04005410 con 0x7f4d0c0cf480 2024-04-13T03:12:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.099+0000 7f4d027fc640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 0x7f4cf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.100+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5619+0+0 (secure 0 0 0) 0x7f4d0408b900 con 0x7f4d0c0cf480 2024-04-13T03:12:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.100+0000 7f4d11dd2640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 0x7f4cf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:10.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.100+0000 7f4d11dd2640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 0x7f4cf006ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009ba0 tx=0x7f4d08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.104+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d0c15f380 con 0x7f4d0c0cf480 2024-04-13T03:12:10.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.280+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4d0c158820 con 0x7f4d0c0cf480 2024-04-13T03:12:10.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.975+0000 7f4d027fc640 1 -- 172.21.15.106:0/2597409807 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) ==== 170+0+0 (secure 0 0 0) 0x7f4d04059d00 con 0x7f4d0c0cf480 2024-04-13T03:12:10.977 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:12:10.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.978+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 msgr2=0x7f4cf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.978+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 0x7f4cf006ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009ba0 tx=0x7f4d08008040 comp rx=0 tx=0).stop 2024-04-13T03:12:10.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.978+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 msgr2=0x7f4d0c15c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:10.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.978+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c780 tx=0x7f4d0400cb80 comp rx=0 tx=0).stop 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 shutdown_connections 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4cf006c920 0x7f4cf006ede0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0c0cf480 0x7f4d0c15c0c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0ba170 0x7f4d0c1f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 --2- 172.21.15.106:0/2597409807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4d0c06b8e0 0x7f4d0c0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 >> 172.21.15.106:0/2597409807 conn(0x7f4d0c0d47d0 msgr2=0x7f4d0c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.979+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 shutdown_connections 2024-04-13T03:12:10.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:10.980+0000 7f4d14036640 1 -- 172.21.15.106:0/2597409807 wait complete. 2024-04-13T03:12:11.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:12:11.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.110+0000 7f3d77e2d640 1 Processor -- start 2024-04-13T03:12:11.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.111+0000 7f3d77e2d640 1 -- start start 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.111+0000 7f3d77e2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d70073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.111+0000 7f3d77e2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7015bb70 con 0x7f3d7006e890 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.111+0000 7f3d77e2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d700cfea0 con 0x7f3d700704f0 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.111+0000 7f3d77e2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7006e3e0 con 0x7f3d7015b790 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d753c8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3d7006e890 0x7f3d7006dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51982/0 (socket says 172.21.15.106:51982) 2024-04-13T03:12:11.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d753c8640 1 -- 172.21.15.106:0/506196674 learned_addr learned my addr 172.21.15.106:0/506196674 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d70073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092638108 0 0) 0x7f3d7015bb70 con 0x7f3d7006e890 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d5c003520 con 0x7f3d7006e890 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 720556447 0 0) 0x7f3d5c003520 con 0x7f3d7006e890 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 msgr2=0x7f3d70073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d70073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 >> v1:172.21.15.114:6789/0 conn(0x7f3d700704f0 legacy=0x7f3d700b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d700d0180 con 0x7f3d7006e890 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f3d60001670 con 0x7f3d7006e890 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 >> v1:172.21.15.106:6789/0 conn(0x7f3d7006e890 legacy=0x7f3d7006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c004d40 0x7f3d5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7006e3e0 con 0x7f3d5c0039e0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00b9c0 con 0x7f3d5c004d40 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5c00bbc0 con 0x7f3d7015b790 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d753c8640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.112+0000 7f3d763ca640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c004d40 0x7f3d5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 -- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 msgr2=0x7f3d5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d7015b790 0x7f3d5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 -- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c004d40 msgr2=0x7f3d5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c004d40 0x7f3d5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 -- 172.21.15.106:0/506196674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c003520 con 0x7f3d5c0039e0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003dc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009ac0 tx=0x7f3d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe61700964bf19 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:11.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c039070 con 0x7f3d5c0039e0 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6c02c600 con 0x7f3d5c0039e0 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.113+0000 7f3d74bc7640 1 -- 172.21.15.106:0/506196674 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c009690 con 0x7f3d5c0039e0 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 -- 172.21.15.106:0/506196674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c0039e0 msgr2=0x7f3d5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/506196674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003dc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009ac0 tx=0x7f3d5c005830 comp rx=0 tx=0).stop 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d763ca640 1 -- 172.21.15.106:0/506196674 reap_dead start 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 -- 172.21.15.106:0/506196674 shutdown_connections 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 -- 172.21.15.106:0/506196674 >> 172.21.15.106:0/506196674 conn(0x7f3d700d13f0 msgr2=0x7f3d700d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 -- 172.21.15.106:0/506196674 shutdown_connections 2024-04-13T03:12:11.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.114+0000 7f3d77e2d640 1 -- 172.21.15.106:0/506196674 wait complete. 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.115+0000 7f3d77e2d640 1 Processor -- start 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.115+0000 7f3d77e2d640 1 -- start start 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c0039e0 0x7f3d70141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d701515f0 0x7f3d70151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700b4fb0 con 0x7f3d701503d0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7006d7b0 con 0x7f3d5c0039e0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d77e2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d700b5130 con 0x7f3d701515f0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51932/0 (socket says 172.21.15.106:51932) 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 -- 172.21.15.106:0/2424412438 learned_addr learned my addr 172.21.15.106:0/2424412438 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d763ca640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d701515f0 0x7f3d70151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c0039e0 0x7f3d70141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 -- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d701515f0 msgr2=0x7f3d70151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d701515f0 0x7f3d70151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 -- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c0039e0 msgr2=0x7f3d70141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c0039e0 0x7f3d70141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 -- 172.21.15.106:0/2424412438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7015e610 con 0x7f3d701503d0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.116+0000 7f3d753c8640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007310 tx=0x7f3d600077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.117+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000f960 con 0x7f3d701503d0 2024-04-13T03:12:11.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.117+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60018070 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.117+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7015e8f0 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.117+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000fd80 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.117+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7013f690 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.118+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7013f890 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.118+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d6001d460 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.118+0000 7f3d667fc640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 0x7f3d5406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.118+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6001+0+0 (secure 0 0 0) 0x7f3d6008bba0 con 0x7f3d701503d0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.118+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 0x7f3d5406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:11.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.119+0000 7f3d75bc9640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 0x7f3d5406ed50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c036000 tx=0x7f3d6c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:11.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.123+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d7013f890 con 0x7f3d701503d0 2024-04-13T03:12:11.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.336+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3d70069120 con 0x7f3d701503d0 2024-04-13T03:12:11.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.976+0000 7f3d667fc640 1 -- 172.21.15.106:0/2424412438 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v26) ==== 173+0+0 (secure 0 0 0) 0x7f3d600598e0 con 0x7f3d701503d0 2024-04-13T03:12:11.979 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 9 and data pool 10 2024-04-13T03:12:11.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 msgr2=0x7f3d5406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 0x7f3d5406ed50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c036000 tx=0x7f3d6c030000 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 msgr2=0x7f3d7013e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007310 tx=0x7f3d600077d0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 shutdown_connections 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d5406c890 0x7f3d5406ed50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d701515f0 0x7f3d70151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d701503d0 0x7f3d7013e9a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 --2- 172.21.15.106:0/2424412438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d5c0039e0 0x7f3d70141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:11.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.983+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 >> 172.21.15.106:0/2424412438 conn(0x7f3d700d13f0 msgr2=0x7f3d70154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:11.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.984+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 shutdown_connections 2024-04-13T03:12:11.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:11.984+0000 7f3d77e2d640 1 -- 172.21.15.106:0/2424412438 wait complete. 2024-04-13T03:12:12.012 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.114+0000 7fda79fcc640 1 Processor -- start 2024-04-13T03:12:12.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.114+0000 7fda79fcc640 1 -- start start 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda79fcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda741626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda79fcc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda740ce1f0 0x7fda74162be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda79fcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda740ce6f0 con 0x7fda7415ebf0 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda79fcc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fda7415fa70 con 0x7fda740ce1f0 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda79fcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda7415f420 con 0x7fda7406fe90 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda740ce1f0 0x7fda74162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda737fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fda7415ebf0 0x7fda7406f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51996/0 (socket says 172.21.15.106:51996) 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda740ce1f0 0x7fda74162be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51934/0 (socket says 172.21.15.106:51934) 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda737fe640 1 -- 172.21.15.106:0/177360651 learned_addr learned my addr 172.21.15.106:0/177360651 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda6bfff640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda741626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 -- 172.21.15.106:0/177360651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 msgr2=0x7fda741626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda741626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 -- 172.21.15.106:0/177360651 >> v1:172.21.15.106:6789/0 conn(0x7fda7415ebf0 legacy=0x7fda7406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda73fff640 1 -- 172.21.15.106:0/177360651 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7415f700 con 0x7fda740ce1f0 2024-04-13T03:12:12.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.115+0000 7fda72ffd640 1 -- 172.21.15.106:0/177360651 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666680440 0 0) 0x7fda740ce6f0 con 0x7fda7415ebf0 2024-04-13T03:12:12.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.116+0000 7fda73fff640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda74162be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009d90 tx=0x7fda6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa953ee7ae70418 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.116+0000 7fda72ffd640 1 -- 172.21.15.106:0/177360651 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c039070 con 0x7fda740ce1f0 2024-04-13T03:12:12.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.116+0000 7fda72ffd640 1 -- 172.21.15.106:0/177360651 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c02cc50 con 0x7fda740ce1f0 2024-04-13T03:12:12.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.116+0000 7fda72ffd640 1 -- 172.21.15.106:0/177360651 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c034440 con 0x7fda740ce1f0 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 -- 172.21.15.106:0/177360651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 msgr2=0x7fda74162be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda74162be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009d90 tx=0x7fda6c02bea0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 -- 172.21.15.106:0/177360651 shutdown_connections 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda74162be0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 --2- 172.21.15.106:0/177360651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda741626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 -- 172.21.15.106:0/177360651 >> 172.21.15.106:0/177360651 conn(0x7fda740cd6d0 msgr2=0x7fda740cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 -- 172.21.15.106:0/177360651 shutdown_connections 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.117+0000 7fda79fcc640 1 -- 172.21.15.106:0/177360651 wait complete. 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.118+0000 7fda79fcc640 1 Processor -- start 2024-04-13T03:12:12.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.118+0000 7fda79fcc640 1 -- start start 2024-04-13T03:12:12.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda740c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda7415ebf0 0x7fda7415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda740b0650 con 0x7fda740ce1f0 2024-04-13T03:12:12.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda740b04d0 con 0x7fda7415ebf0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda7415efd0 con 0x7fda7406fe90 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51948/0 (socket says 172.21.15.106:51948) 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 -- 172.21.15.106:0/2336398903 learned_addr learned my addr 172.21.15.106:0/2336398903 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda737fe640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda740c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda73fff640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda7415ebf0 0x7fda7415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 -- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 msgr2=0x7fda740c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda740c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 -- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda7415ebf0 msgr2=0x7fda7415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda7415ebf0 0x7fda7415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 -- 172.21.15.106:0/2336398903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7414fe20 con 0x7fda740ce1f0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda6bfff640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00d620 tx=0x7fda5c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00b620 con 0x7fda740ce1f0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c019070 con 0x7fda740ce1f0 2024-04-13T03:12:12.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda741602b0 con 0x7fda740ce1f0 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00ba00 con 0x7fda740ce1f0 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.119+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda74160870 con 0x7fda740ce1f0 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.120+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda74068a10 con 0x7fda740ce1f0 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.121+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda5c00bba0 con 0x7fda740ce1f0 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.121+0000 7fda70ff9640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.121+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fda5c08c040 con 0x7fda740ce1f0 2024-04-13T03:12:12.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.121+0000 7fda737fe640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.121+0000 7fda737fe640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda64000f60 tx=0x7fda64007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.126+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda7414fe20 con 0x7fda740ce1f0 2024-04-13T03:12:12.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.301+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda740d07d0 con 0x7fda740ce1f0 2024-04-13T03:12:12.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.303+0000 7fda70ff9640 1 -- 172.21.15.106:0/2336398903 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16933 (secure 0 0 0) 0x7fda5c060030 con 0x7fda740ce1f0 2024-04-13T03:12:12.305 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:12.305 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:12:11.338286+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 msgr2=0x7fda5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda64000f60 tx=0x7fda64007b50 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 msgr2=0x7fda7414c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00d620 tx=0x7fda5c00dae0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 shutdown_connections 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda7415ebf0 0x7fda7415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.304+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda740ce1f0 0x7fda7414c5f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.305+0000 7fda79fcc640 1 --2- 172.21.15.106:0/2336398903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda7406fe90 0x7fda740c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.305+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 >> 172.21.15.106:0/2336398903 conn(0x7fda740cd6d0 msgr2=0x7fda740cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:12.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.305+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 shutdown_connections 2024-04-13T03:12:12.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.305+0000 7fda79fcc640 1 -- 172.21.15.106:0/2336398903 wait complete. 2024-04-13T03:12:12.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:12.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.417+0000 7f5e795e0640 1 Processor -- start 2024-04-13T03:12:12.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.417+0000 7f5e795e0640 1 -- start start 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e795e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5e7415b800 0x7f5e740cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e795e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 0x7f5e7406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e795e0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e740ccd20 con 0x7f5e74072e80 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e795e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5e7415f790 con 0x7f5e7415b800 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e795e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7415fa70 con 0x7f5e7415f1f0 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5e7415b800 0x7f5e740cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e72ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5e7415b800 0x7f5e740cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51964/0 (socket says 172.21.15.106:51964) 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e72ffd640 1 -- 172.21.15.106:0/3275008709 learned_addr learned my addr 172.21.15.106:0/3275008709 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:12.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e727fc640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 0x7f5e7406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e72ffd640 1 -- 172.21.15.106:0/3275008709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 msgr2=0x7f5e7406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.418+0000 7f5e72ffd640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 0x7f5e7406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e72ffd640 1 -- 172.21.15.106:0/3275008709 >> v1:172.21.15.114:6789/0 conn(0x7f5e74072e80 legacy=0x7f5e740cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e72ffd640 1 -- 172.21.15.106:0/3275008709 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7415fd50 con 0x7f5e7415b800 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e727fc640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 0x7f5e7406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e72ffd640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cf030 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e50 tx=0x7f5e6402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209b2661517d5fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e71ffb640 1 -- 172.21.15.106:0/3275008709 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6402cb00 con 0x7f5e7415b800 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e71ffb640 1 -- 172.21.15.106:0/3275008709 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6402cca0 con 0x7f5e7415b800 2024-04-13T03:12:12.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.419+0000 7f5e71ffb640 1 -- 172.21.15.106:0/3275008709 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64034610 con 0x7f5e7415b800 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 -- 172.21.15.106:0/3275008709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 msgr2=0x7f5e740cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cf030 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e50 tx=0x7f5e6402bf60 comp rx=0 tx=0).stop 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 -- 172.21.15.106:0/3275008709 shutdown_connections 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e7415f1f0 0x7f5e7406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3275008709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cf030 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 -- 172.21.15.106:0/3275008709 >> 172.21.15.106:0/3275008709 conn(0x7f5e740cdee0 msgr2=0x7f5e740ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:12.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 -- 172.21.15.106:0/3275008709 shutdown_connections 2024-04-13T03:12:12.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.420+0000 7f5e795e0640 1 -- 172.21.15.106:0/3275008709 wait complete. 2024-04-13T03:12:12.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.421+0000 7f5e795e0640 1 Processor -- start 2024-04-13T03:12:12.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.421+0000 7f5e795e0640 1 -- start start 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e74072e80 0x7f5e74156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 0x7f5e7415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7406eb00 con 0x7f5e7415b800 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e740cf570 con 0x7f5e7415f1f0 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e795e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5e740cd8e0 con 0x7f5e74072e80 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51968/0 (socket says 172.21.15.106:51968) 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 -- 172.21.15.106:0/3576320837 learned_addr learned my addr 172.21.15.106:0/3576320837 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e72ffd640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e74072e80 0x7f5e74156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e737fe640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 0x7f5e7415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 -- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e74072e80 msgr2=0x7f5e74156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e74072e80 0x7f5e74156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 -- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 msgr2=0x7f5e7415bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 0x7f5e7415bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e727fc640 1 -- 172.21.15.106:0/3576320837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7415c120 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.422+0000 7f5e737fe640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 0x7f5e7415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e727fc640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800d7d0 tx=0x7f5e6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68004030 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e680041d0 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7415c3f0 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e680096a0 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.423+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7415c9b0 con 0x7f5e7415b800 2024-04-13T03:12:12.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.424+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e74069120 con 0x7f5e7415b800 2024-04-13T03:12:12.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.424+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e68009840 con 0x7f5e7415b800 2024-04-13T03:12:12.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.425+0000 7f5e5b7fe640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.425+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f5e6808be30 con 0x7f5e7415b800 2024-04-13T03:12:12.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.425+0000 7f5e72ffd640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.425+0000 7f5e72ffd640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 0x7f5e6006ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e64035000 tx=0x7f5e64030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.429+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e6805a160 con 0x7f5e7415b800 2024-04-13T03:12:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.606+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e74073ab0 con 0x7f5e7415b800 2024-04-13T03:12:12.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.607+0000 7f5e5b7fe640 1 -- 172.21.15.106:0/3576320837 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16933 (secure 0 0 0) 0x7f5e68059a60 con 0x7f5e7415b800 2024-04-13T03:12:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:12.610 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:12:11.338286+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:12.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 msgr2=0x7f5e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 0x7f5e6006ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e64035000 tx=0x7f5e64030000 comp rx=0 tx=0).stop 2024-04-13T03:12:12.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 msgr2=0x7f5e740cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800d7d0 tx=0x7f5e6800dc90 comp rx=0 tx=0).stop 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 shutdown_connections 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e7415f1f0 0x7f5e7415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e7415b800 0x7f5e740cae40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 --2- 172.21.15.106:0/3576320837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e74072e80 0x7f5e74156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 >> 172.21.15.106:0/3576320837 conn(0x7f5e740cdee0 msgr2=0x7f5e740ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 shutdown_connections 2024-04-13T03:12:12.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.609+0000 7f5e795e0640 1 -- 172.21.15.106:0/3576320837 wait complete. 2024-04-13T03:12:12.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 Processor -- start 2024-04-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 -- start start 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7e0413f510 0x7f7e0415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e04160890 con 0x7f7e04160400 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e04160b70 con 0x7f7e0413f8f0 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.724+0000 7f7e0c1f0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7e04160e50 con 0x7f7e0413f510 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e0978b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7e04160400 0x7f7e0406c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47440/0 (socket says 172.21.15.106:47440) 2024-04-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e0978b640 1 -- 172.21.15.106:0/848704112 learned_addr learned my addr 172.21.15.106:0/848704112 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/848704112 >> v2:172.21.15.114:3300/0 conn(0x7f7e0413f510 0x7f7e0415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063998357 0 0) 0x7f7e04160890 con 0x7f7e04160400 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7df0003520 con 0x7f7e04160400 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 547930760 0 0) 0x7f7df0003520 con 0x7f7e04160400 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 >> v2:172.21.15.114:3300/0 conn(0x7f7e0413f510 msgr2=0x7f7e0415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 --2- 172.21.15.106:0/848704112 >> v2:172.21.15.114:3300/0 conn(0x7f7e0413f510 0x7f7e0415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 >> v1:172.21.15.114:6789/0 conn(0x7f7e0413f8f0 legacy=0x7f7e040704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e04161130 con 0x7f7e04160400 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f7e00001670 con 0x7f7e04160400 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 >> v1:172.21.15.106:6789/0 conn(0x7f7e04160400 legacy=0x7f7e0406c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e04160e50 con 0x7f7df00039b0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7df000c060 con 0x7f7df0004800 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7df000c260 con 0x7f7df00051c0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e0978b640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.725+0000 7f7e0a78d640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 -- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7df00051c0 msgr2=0x7f7df000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 -- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df0004800 msgr2=0x7f7df0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 -- 172.21.15.106:0/848704112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0003520 con 0x7f7df00039b0 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e0a78d640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e0a78d640 1 -- 172.21.15.106:0/848704112 reap_dead start 2024-04-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ac0 tx=0x7f7df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cd8fcacc01270a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4039070 con 0x7f7df00039b0 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df402c6b0 con 0x7f7df00039b0 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.726+0000 7f7e08f8a640 1 -- 172.21.15.106:0/848704112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4009690 con 0x7f7df00039b0 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 msgr2=0x7f7df0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ac0 tx=0x7f7df0005830 comp rx=0 tx=0).stop 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/848704112 shutdown_connections 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/848704112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/848704112 >> 172.21.15.106:0/848704112 conn(0x7f7e040d13f0 msgr2=0x7f7e040d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/848704112 shutdown_connections 2024-04-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.727+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/848704112 wait complete. 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.728+0000 7f7e0c1f0640 1 Processor -- start 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.728+0000 7f7e0c1f0640 1 -- start start 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 0x7f7e041f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e04130750 0x7f7e04130bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0415d150 con 0x7f7e04153000 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0415cfd0 con 0x7f7df00039b0 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0c1f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0415d2d0 con 0x7f7e04130750 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47864/0 (socket says 172.21.15.106:47864) 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 -- 172.21.15.106:0/371816143 learned_addr learned my addr 172.21.15.106:0/371816143 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 0x7f7e041f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0978b640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e04130750 0x7f7e04130bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 -- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e04130750 msgr2=0x7f7e04130bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e04130750 0x7f7e04130bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 -- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 msgr2=0x7f7e041f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 0x7f7e041f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 -- 172.21.15.106:0/371816143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e04153ad0 con 0x7f7e04153000 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 0x7f7e041f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:12.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.729+0000 7f7e0a78d640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406cbb0 tx=0x7f7dfc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003970 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc018070 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003d50 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e041415c0 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e04141b80 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.730+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e04069120 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.731+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dfc0050d0 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.731+0000 7f7dfa7fc640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.731+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f7dfc08b630 con 0x7f7e04153000 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.731+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.732+0000 7f7e09f8c640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 0x7f7de806edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7df4036000 tx=0x7f7df4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:12.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.736+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7dfc022020 con 0x7f7e04153000 2024-04-13T03:12:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:12.947+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7e040d1a80 con 0x7f7e04153000 2024-04-13T03:12:13.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.001+0000 7f7dfa7fc640 1 -- 172.21.15.106:0/371816143 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f7dfc0598d0 con 0x7f7e04153000 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 msgr2=0x7f7de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 0x7f7de806edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7df4036000 tx=0x7f7df4030000 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 msgr2=0x7f7e04153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406cbb0 tx=0x7f7dfc00eb90 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 shutdown_connections 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7e04130750 0x7f7e04130bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e04153000 0x7f7e04153480 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 --2- 172.21.15.106:0/371816143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7df00039b0 0x7f7e041f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 >> 172.21.15.106:0/371816143 conn(0x7f7e040d13f0 msgr2=0x7f7e040cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:13.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 shutdown_connections 2024-04-13T03:12:13.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.003+0000 7f7e0c1f0640 1 -- 172.21.15.106:0/371816143 wait complete. 2024-04-13T03:12:13.015 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:12:13.015 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:12:13.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.117+0000 7f581e11e640 1 Processor -- start 2024-04-13T03:12:13.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.117+0000 7f581e11e640 1 -- start start 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f581e11e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f5818169840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f581e11e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58181632d0 con 0x7f581815ea80 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f581e11e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58180b3fc0 con 0x7f581806fe60 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f581e11e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58180b42a0 con 0x7f58180aeae0 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58177fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f581815ea80 0x7f58181624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47452/0 (socket says 172.21.15.106:47452) 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58177fe640 1 -- 172.21.15.106:0/1656216498 learned_addr learned my addr 172.21.15.106:0/1656216498 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f5817fff640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f5818169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820006852 0 0) 0x7f58181632d0 con 0x7f581815ea80 2024-04-13T03:12:13.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5800003520 con 0x7f581815ea80 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464436263 0 0) 0x7f5800003520 con 0x7f581815ea80 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 msgr2=0x7f5818169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f5818169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 >> v1:172.21.15.114:6789/0 conn(0x7f581806fe60 legacy=0x7f5818162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58180b4580 con 0x7f581815ea80 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.118+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5808001670 con 0x7f581815ea80 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 >> v1:172.21.15.106:6789/0 conn(0x7f581815ea80 legacy=0x7f58181624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f58000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5800003f60 0x7f5800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5800004d40 0x7f58000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58180b42a0 con 0x7f5800004d40 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f580000baa0 con 0x7f5800003f60 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f580000bca0 con 0x7f58180aeae0 2024-04-13T03:12:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5800004d40 0x7f58000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f58177fe640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f58000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5816ffd640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5800003f60 0x7f5800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 -- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 msgr2=0x7f58000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58180aeae0 0x7f58000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 -- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5800003f60 msgr2=0x7f5800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5800003f60 0x7f5800004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.119+0000 7f5817fff640 1 -- 172.21.15.106:0/1656216498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5800003520 con 0x7f5800004d40 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f5817fff640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5800004d40 0x7f58000051c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5810009fc0 tx=0x7f5800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4322a8e0a470fddb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810039070 con 0x7f5800004d40 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581002cc20 con 0x7f5800004d40 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f58167fc640 1 -- 172.21.15.106:0/1656216498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810009c10 con 0x7f5800004d40 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 -- 172.21.15.106:0/1656216498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5800004d40 msgr2=0x7f58000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 --2- 172.21.15.106:0/1656216498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5800004d40 0x7f58000051c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5810009fc0 tx=0x7f5800005830 comp rx=0 tx=0).stop 2024-04-13T03:12:13.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f5817fff640 1 -- 172.21.15.106:0/1656216498 reap_dead start 2024-04-13T03:12:13.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 -- 172.21.15.106:0/1656216498 shutdown_connections 2024-04-13T03:12:13.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 -- 172.21.15.106:0/1656216498 >> 172.21.15.106:0/1656216498 conn(0x7f58180b2600 msgr2=0x7f58180d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:13.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 -- 172.21.15.106:0/1656216498 shutdown_connections 2024-04-13T03:12:13.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.120+0000 7f581e11e640 1 -- 172.21.15.106:0/1656216498 wait complete. 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 Processor -- start 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 -- start start 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f581815a840 0x7f5818159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f581815ac20 0x7f58180ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5818167830 con 0x7f5818159940 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58180b3c90 con 0x7f581815a840 2024-04-13T03:12:13.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.122+0000 7f581e11e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58181679b0 con 0x7f581815ac20 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47888/0 (socket says 172.21.15.106:47888) 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 -- 172.21.15.106:0/133976517 learned_addr learned my addr 172.21.15.106:0/133976517 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5816ffd640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f581815ac20 0x7f58180ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f58177fe640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f581815a840 0x7f5818159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 -- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f581815ac20 msgr2=0x7f58180ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f581815ac20 0x7f58180ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 -- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f581815a840 msgr2=0x7f5818159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f581815a840 0x7f5818159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 -- 172.21.15.106:0/133976517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181543e0 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f5817fff640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f581002c690 tx=0x7f5810009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810045070 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581003a040 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58180cd490 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810039070 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.123+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58180cda50 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.124+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818068a10 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.125+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5810003bb0 con 0x7f5818159940 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.125+0000 7f57fffff640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 0x7f580406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:13.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.125+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f58100ad590 con 0x7f5818159940 2024-04-13T03:12:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.125+0000 7f58177fe640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 0x7f580406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.126+0000 7f58177fe640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 0x7f580406edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5808004840 tx=0x7f5808009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.129+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f581007b270 con 0x7f5818159940 2024-04-13T03:12:13.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:13.342+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f581806e2a0 con 0x7f5818159940 2024-04-13T03:12:14.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.003+0000 7f57fffff640 1 -- 172.21.15.106:0/133976517 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f581007f570 con 0x7f5818159940 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 msgr2=0x7f580406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 0x7f580406edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5808004840 tx=0x7f5808009040 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 msgr2=0x7f58180cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f581002c690 tx=0x7f5810009750 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 shutdown_connections 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f580406c8f0 0x7f580406edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f581815ac20 0x7f58180ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818159940 0x7f58180cf030 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 --2- 172.21.15.106:0/133976517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f581815a840 0x7f5818159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.005+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 >> 172.21.15.106:0/133976517 conn(0x7f58180b2600 msgr2=0x7f581806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.006+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 shutdown_connections 2024-04-13T03:12:14.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.006+0000 7f581e11e640 1 -- 172.21.15.106:0/133976517 wait complete. 2024-04-13T03:12:14.019 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:12:14.019 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:12:14.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.122+0000 7f3001a88640 1 Processor -- start 2024-04-13T03:12:14.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.122+0000 7f3001a88640 1 -- start start 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f3001a88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc15c940 0x7f2ffc06e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f3001a88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ffc06e780 0x7f2ffc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f3001a88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ffc15cd20 con 0x7f2ffc06cf10 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f3001a88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2ffc06ebe0 con 0x7f2ffc06e780 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f3001a88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc0c6330 con 0x7f2ffc15c940 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f2ffaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ffc06e780 0x7f2ffc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f2ffaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ffc06e780 0x7f2ffc06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47902/0 (socket says 172.21.15.106:47902) 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.123+0000 7f2ffaffd640 1 -- 172.21.15.106:0/1817667262 learned_addr learned my addr 172.21.15.106:0/1817667262 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffb7fe640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc15c940 0x7f2ffc06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffaffd640 1 -- 172.21.15.106:0/1817667262 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc15c940 msgr2=0x7f2ffc06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffaffd640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc15c940 0x7f2ffc06e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffaffd640 1 -- 172.21.15.106:0/1817667262 >> v1:172.21.15.114:6789/0 conn(0x7f2ffc06cf10 legacy=0x7f2ffc072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffaffd640 1 -- 172.21.15.106:0/1817667262 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc0cc700 con 0x7f2ffc06e780 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.124+0000 7f2ffaffd640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06e780 0x7f2ffc06c9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009870 tx=0x7f2ff002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54389de2d12a299b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f2ffa7fc640 1 -- 172.21.15.106:0/1817667262 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0035020 con 0x7f2ffc06e780 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f2ffa7fc640 1 -- 172.21.15.106:0/1817667262 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff0031640 con 0x7f2ffc06e780 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f2ffa7fc640 1 -- 172.21.15.106:0/1817667262 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff002d040 con 0x7f2ffc06e780 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 -- 172.21.15.106:0/1817667262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06e780 msgr2=0x7f2ffc06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06e780 0x7f2ffc06c9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009870 tx=0x7f2ff002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 -- 172.21.15.106:0/1817667262 shutdown_connections 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06e780 0x7f2ffc06c9d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 --2- 172.21.15.106:0/1817667262 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc15c940 0x7f2ffc06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 -- 172.21.15.106:0/1817667262 >> 172.21.15.106:0/1817667262 conn(0x7f2ffc0cdef0 msgr2=0x7f2ffc0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 -- 172.21.15.106:0/1817667262 shutdown_connections 2024-04-13T03:12:14.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.125+0000 7f3001a88640 1 -- 172.21.15.106:0/1817667262 wait complete. 2024-04-13T03:12:14.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.126+0000 7f3001a88640 1 Processor -- start 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 -- start start 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc06e780 0x7f2ffc147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ffc15c940 0x7f2ffc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc06dc70 con 0x7f2ffc06cf10 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc06d990 con 0x7f2ffc15c940 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f3001a88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc0c6580 con 0x7f2ffc06e780 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47912/0 (socket says 172.21.15.106:47912) 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f2ffb7fe640 1 -- 172.21.15.106:0/3972505408 learned_addr learned my addr 172.21.15.106:0/3972505408 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f2ffaffd640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc06e780 0x7f2ffc147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.127+0000 7f2ffbfff640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ffc15c940 0x7f2ffc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 -- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc06e780 msgr2=0x7f2ffc147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc06e780 0x7f2ffc147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 -- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ffc15c940 msgr2=0x7f2ffc0b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ffc15c940 0x7f2ffc0b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 -- 172.21.15.106:0/3972505408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc0ad7f0 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2ffb7fe640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d5f0 tx=0x7f2fec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0125f0 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec019070 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0129d0 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc1f8d70 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.128+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc148330 con 0x7f2ffc06cf10 2024-04-13T03:12:14.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.129+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc069120 con 0x7f2ffc06cf10 2024-04-13T03:12:14.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.129+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fec012b70 con 0x7f2ffc06cf10 2024-04-13T03:12:14.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.130+0000 7f2fe3fff640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 0x7f2fe806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.130+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f2fec08c0e0 con 0x7f2ffc06cf10 2024-04-13T03:12:14.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.130+0000 7f2ffaffd640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 0x7f2fe806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.130+0000 7f2ffaffd640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 0x7f2fe806ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0036000 tx=0x7f2ff003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.134+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fec05a380 con 0x7f2ffc06cf10 2024-04-13T03:12:14.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.346+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ffc148560 con 0x7f2ffc06cf10 2024-04-13T03:12:14.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.347+0000 7f2fe3fff640 1 -- 172.21.15.106:0/3972505408 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5185 (secure 0 0 0) 0x7f2fec059c80 con 0x7f2ffc06cf10 2024-04-13T03:12:14.348 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:14.348 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:13.343983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:12:14.348 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 28 2024-04-13T03:12:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 msgr2=0x7f2fe806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 0x7f2fe806ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0036000 tx=0x7f2ff003c040 comp rx=0 tx=0).stop 2024-04-13T03:12:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 msgr2=0x7f2ffc159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d5f0 tx=0x7f2fec00dab0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 shutdown_connections 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2fe806c920 0x7f2fe806ede0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ffc06e780 0x7f2ffc147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ffc15c940 0x7f2ffc0b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 --2- 172.21.15.106:0/3972505408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ffc06cf10 0x7f2ffc159680 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 >> 172.21.15.106:0/3972505408 conn(0x7f2ffc0cdef0 msgr2=0x7f2ffc159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 shutdown_connections 2024-04-13T03:12:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.349+0000 7f3001a88640 1 -- 172.21.15.106:0/3972505408 wait complete. 2024-04-13T03:12:14.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:12:14.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.460+0000 7f7af0636640 1 Processor -- start 2024-04-13T03:12:14.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.460+0000 7f7af0636640 1 -- start start 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ae815cbf0 0x7f7ae80b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 0x7f7ae80ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 0x7f7ae80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae8070640 con 0x7f7ae815cbf0 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae815cfd0 con 0x7f7ae8070ad0 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7af0636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae806fa30 con 0x7f7ae806f490 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ae815cbf0 0x7f7ae80b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ae815cbf0 0x7f7ae80b7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47926/0 (socket says 172.21.15.106:47926) 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 -- 172.21.15.106:0/3014695055 learned_addr learned my addr 172.21.15.106:0/3014695055 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 -- 172.21.15.106:0/3014695055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 msgr2=0x7f7ae80ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aedbd1640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 0x7f7ae80ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aeebd3640 1 --2- 172.21.15.106:0/3014695055 >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 0x7f7ae80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 0x7f7ae80ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 -- 172.21.15.106:0/3014695055 >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 msgr2=0x7f7ae80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/3014695055 >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 0x7f7ae80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aee3d2640 1 -- 172.21.15.106:0/3014695055 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae806fd10 con 0x7f7ae815cbf0 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aeebd3640 1 --2- 172.21.15.106:0/3014695055 >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 0x7f7ae80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.461+0000 7f7aedbd1640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 0x7f7ae80ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.462+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae815cbf0 0x7f7ae80b7f80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80b1f80 tx=0x7f7ae402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71062859526f328d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.462+0000 7f7aed3d0640 1 -- 172.21.15.106:0/3014695055 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4035020 con 0x7f7ae815cbf0 2024-04-13T03:12:14.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.462+0000 7f7aed3d0640 1 -- 172.21.15.106:0/3014695055 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae4031870 con 0x7f7ae815cbf0 2024-04-13T03:12:14.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.462+0000 7f7aed3d0640 1 -- 172.21.15.106:0/3014695055 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4031c40 con 0x7f7ae815cbf0 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 -- 172.21.15.106:0/3014695055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae815cbf0 msgr2=0x7f7ae80b7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae815cbf0 0x7f7ae80b7f80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80b1f80 tx=0x7f7ae402b910 comp rx=0 tx=0).stop 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 -- 172.21.15.106:0/3014695055 shutdown_connections 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 --2- 172.21.15.106:0/3014695055 >> v2:172.21.15.114:3300/0 conn(0x7f7ae8070ad0 0x7f7ae80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae806f490 0x7f7ae80ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 --2- 172.21.15.106:0/3014695055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae815cbf0 0x7f7ae80b7f80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 -- 172.21.15.106:0/3014695055 >> 172.21.15.106:0/3014695055 conn(0x7f7ae80d25b0 msgr2=0x7f7ae80d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 -- 172.21.15.106:0/3014695055 shutdown_connections 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.463+0000 7f7af0636640 1 -- 172.21.15.106:0/3014695055 wait complete. 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.464+0000 7f7af0636640 1 Processor -- start 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.464+0000 7f7af0636640 1 -- start start 2024-04-13T03:12:14.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 0x7f7ae8157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 0x7f7ae81f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80b84c0 con 0x7f7ae8070ad0 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80702d0 con 0x7f7ae806f490 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae80d0740 con 0x7f7ae815cbf0 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47934/0 (socket says 172.21.15.106:47934) 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 -- 172.21.15.106:0/1807832856 learned_addr learned my addr 172.21.15.106:0/1807832856 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aeebd3640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 0x7f7ae81f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 0x7f7ae8157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 -- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 msgr2=0x7f7ae81f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 0x7f7ae81f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 -- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 msgr2=0x7f7ae8157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 0x7f7ae8157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 -- 172.21.15.106:0/1807832856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae815de90 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aeebd3640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 0x7f7ae81f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 0x7f7ae8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7aedbd1640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c7c0 tx=0x7f7ad800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8015070 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad8007a30 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae80ca800 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8007e00 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.465+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae815db90 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.466+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae8074750 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.467+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad8018070 con 0x7f7ae8070ad0 2024-04-13T03:12:14.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.467+0000 7f7adeffd640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.467+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f7ad808bc60 con 0x7f7ae8070ad0 2024-04-13T03:12:14.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.467+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.468+0000 7f7aee3d2640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 0x7f7acc06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.472+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ad8059f10 con 0x7f7ae8070ad0 2024-04-13T03:12:14.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.648+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ae8068a10 con 0x7f7ae8070ad0 2024-04-13T03:12:14.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.649+0000 7f7adeffd640 1 -- 172.21.15.106:0/1807832856 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16920 (secure 0 0 0) 0x7f7ad8059810 con 0x7f7ae8070ad0 2024-04-13T03:12:14.650 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:14.650 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:12:12.978510+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 msgr2=0x7f7acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 0x7f7acc06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae403c040 comp rx=0 tx=0).stop 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 msgr2=0x7f7ae81efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c7c0 tx=0x7f7ad800cc80 comp rx=0 tx=0).stop 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 shutdown_connections 2024-04-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ae815cbf0 0x7f7ae81f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ae8070ad0 0x7f7ae81efbe0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 --2- 172.21.15.106:0/1807832856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ae806f490 0x7f7ae8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 >> 172.21.15.106:0/1807832856 conn(0x7f7ae80d25b0 msgr2=0x7f7ae80b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 shutdown_connections 2024-04-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.651+0000 7f7af0636640 1 -- 172.21.15.106:0/1807832856 wait complete. 2024-04-13T03:12:14.663 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:12:14.663 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:12:14.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.762+0000 7fa5ce35e640 1 Processor -- start 2024-04-13T03:12:14.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.762+0000 7fa5ce35e640 1 -- start start 2024-04-13T03:12:14.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5ce35e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 0x7fa5c8070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5ce35e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5ce35e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c806fd00 con 0x7fa5c815ce80 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5ce35e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c806ffe0 con 0x7fa5c815caa0 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5ce35e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c80702c0 con 0x7fa5c80b7f70 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5cc8fb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5c7fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 0x7fa5c8070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5cc8fb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:56458/0 (socket says 172.21.15.106:56458) 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.763+0000 7fa5cc8fb640 1 -- 172.21.15.106:0/2074260306 learned_addr learned my addr 172.21.15.106:0/2074260306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5cc8fb640 1 -- 172.21.15.106:0/2074260306 >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 msgr2=0x7fa5c8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5cc8fb640 1 --2- 172.21.15.106:0/2074260306 >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 0x7fa5c8070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5cc8fb640 1 -- 172.21.15.106:0/2074260306 >> v1:172.21.15.114:6789/0 conn(0x7fa5c815ce80 legacy=0x7fa5c80b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5cc8fb640 1 -- 172.21.15.106:0/2074260306 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80705a0 con 0x7fa5c80b7f70 2024-04-13T03:12:14.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5c7fff640 1 --2- 172.21.15.106:0/2074260306 >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 0x7fa5c8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.764+0000 7fa5cc8fb640 1 --2- 172.21.15.106:0/2074260306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009e50 tx=0x7fa5c002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0839976ddd0407 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5c6ffd640 1 -- 172.21.15.106:0/2074260306 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c002cb00 con 0x7fa5c80b7f70 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5c6ffd640 1 -- 172.21.15.106:0/2074260306 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c002cca0 con 0x7fa5c80b7f70 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5c6ffd640 1 -- 172.21.15.106:0/2074260306 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0034610 con 0x7fa5c80b7f70 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 -- 172.21.15.106:0/2074260306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 msgr2=0x7fa5c80b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/2074260306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009e50 tx=0x7fa5c002bf60 comp rx=0 tx=0).stop 2024-04-13T03:12:14.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 -- 172.21.15.106:0/2074260306 shutdown_connections 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/2074260306 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80b5e90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/2074260306 >> v2:172.21.15.114:3300/0 conn(0x7fa5c815caa0 0x7fa5c8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 -- 172.21.15.106:0/2074260306 >> 172.21.15.106:0/2074260306 conn(0x7fa5c80d25d0 msgr2=0x7fa5c80d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 -- 172.21.15.106:0/2074260306 shutdown_connections 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.765+0000 7fa5ce35e640 1 -- 172.21.15.106:0/2074260306 wait complete. 2024-04-13T03:12:14.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.766+0000 7fa5ce35e640 1 Processor -- start 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 -- start start 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 0x7fa5c814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80712e0 con 0x7fa5c815caa0 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c806f990 con 0x7fa5c814b380 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5ce35e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c8071460 con 0x7fa5c80b7f70 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.767+0000 7fa5c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47950/0 (socket says 172.21.15.106:47950) 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 -- 172.21.15.106:0/3311980198 learned_addr learned my addr 172.21.15.106:0/3311980198 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5cc8fb640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 0x7fa5c814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 -- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 msgr2=0x7fa5c80cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 -- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 msgr2=0x7fa5c814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 0x7fa5c814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 -- 172.21.15.106:0/3311980198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c814b9b0 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c7fff640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5cc8fb640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 0x7fa5c814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c77fe640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c3c0 tx=0x7fa5bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0052d0 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc005490 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c814bc90 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc007260 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.768+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c814c250 con 0x7fa5c815caa0 2024-04-13T03:12:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.769+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8068a10 con 0x7fa5c815caa0 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.770+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5bc007400 con 0x7fa5c815caa0 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.770+0000 7fa5c4ff9640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 0x7fa5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.770+0000 7fa5c7fff640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 0x7fa5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.770+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fa5bc08b130 con 0x7fa5c815caa0 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.771+0000 7fa5c7fff640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 0x7fa5b406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009560 tx=0x7fa5b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.775+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5bc058d80 con 0x7fa5c815caa0 2024-04-13T03:12:14.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.945+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5c80b1680 con 0x7fa5b406c8f0 2024-04-13T03:12:14.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.951+0000 7fa5c4ff9640 1 -- 172.21.15.106:0/3311980198 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7fa5c80b1680 con 0x7fa5b406c8f0 2024-04-13T03:12:14.954 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:14.956 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 msgr2=0x7fa5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 0x7fa5b406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009560 tx=0x7fa5b8009190 comp rx=0 tx=0).stop 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 msgr2=0x7fa5c80c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c3c0 tx=0x7fa5bc00c880 comp rx=0 tx=0).stop 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 shutdown_connections 2024-04-13T03:12:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa5b406c8f0 0x7fa5b406edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5c814b380 0x7fa5c814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c815caa0 0x7fa5c80c9990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 --2- 172.21.15.106:0/3311980198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa5c80b7f70 0x7fa5c80cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 >> 172.21.15.106:0/3311980198 conn(0x7fa5c80d25d0 msgr2=0x7fa5c80d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 shutdown_connections 2024-04-13T03:12:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:14.954+0000 7fa5ce35e640 1 -- 172.21.15.106:0/3311980198 wait complete. 2024-04-13T03:12:14.966 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-13T03:12:14.397288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223256,"kb_used_data":7072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751464,"statfs":{"total":773094113280,"available":772865499136,"internally_reserved":0,"allocated":7241728,"data_stored":3898884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.427633"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005952+0000","last_change":"2024-04-13T03:12:12.001940+0000","last_active":"2024-04-13T03:12:12.005952+0000","last_peered":"2024-04-13T03:12:12.005952+0000","last_clean":"2024-04-13T03:12:12.005952+0000","last_became_active":"2024-04-13T03:12:12.001677+0000","last_became_peered":"2024-04-13T03:12:12.001677+0000","last_unstale":"2024-04-13T03:12:12.005952+0000","last_undegraded":"2024-04-13T03:12:12.005952+0000","last_fullsized":"2024-04-13T03:12:12.005952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:27.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.994441+0000","last_change":"2024-04-13T03:12:11.991806+0000","last_active":"2024-04-13T03:12:11.994441+0000","last_peered":"2024-04-13T03:12:11.994441+0000","last_clean":"2024-04-13T03:12:11.994441+0000","last_became_active":"2024-04-13T03:12:11.991513+0000","last_became_peered":"2024-04-13T03:12:11.991513+0000","last_unstale":"2024-04-13T03:12:11.994441+0000","last_undegraded":"2024-04-13T03:12:11.994441+0000","last_fullsized":"2024-04-13T03:12:11.994441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:26.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004190+0000","last_change":"2024-04-13T03:12:11.998338+0000","last_active":"2024-04-13T03:12:12.004190+0000","last_peered":"2024-04-13T03:12:12.004190+0000","last_clean":"2024-04-13T03:12:12.004190+0000","last_became_active":"2024-04-13T03:12:11.997576+0000","last_became_peered":"2024-04-13T03:12:11.997576+0000","last_unstale":"2024-04-13T03:12:12.004190+0000","last_undegraded":"2024-04-13T03:12:12.004190+0000","last_fullsized":"2024-04-13T03:12:12.004190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:27.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011176+0000","last_change":"2024-04-13T03:12:12.006998+0000","last_active":"2024-04-13T03:12:12.011176+0000","last_peered":"2024-04-13T03:12:12.011176+0000","last_clean":"2024-04-13T03:12:12.011176+0000","last_became_active":"2024-04-13T03:12:12.006714+0000","last_became_peered":"2024-04-13T03:12:12.006714+0000","last_unstale":"2024-04-13T03:12:12.011176+0000","last_undegraded":"2024-04-13T03:12:12.011176+0000","last_fullsized":"2024-04-13T03:12:12.011176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:26.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011726+0000","last_change":"2024-04-13T03:12:12.009748+0000","last_active":"2024-04-13T03:12:12.011726+0000","last_peered":"2024-04-13T03:12:12.011726+0000","last_clean":"2024-04-13T03:12:12.011726+0000","last_became_active":"2024-04-13T03:12:12.009463+0000","last_became_peered":"2024-04-13T03:12:12.009463+0000","last_unstale":"2024-04-13T03:12:12.011726+0000","last_undegraded":"2024-04-13T03:12:12.011726+0000","last_fullsized":"2024-04-13T03:12:12.011726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:24.027177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004384+0000","last_change":"2024-04-13T03:12:12.001174+0000","last_active":"2024-04-13T03:12:12.004384+0000","last_peered":"2024-04-13T03:12:12.004384+0000","last_clean":"2024-04-13T03:12:12.004384+0000","last_became_active":"2024-04-13T03:12:12.000775+0000","last_became_peered":"2024-04-13T03:12:12.000775+0000","last_unstale":"2024-04-13T03:12:12.004384+0000","last_undegraded":"2024-04-13T03:12:12.004384+0000","last_fullsized":"2024-04-13T03:12:12.004384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:16.254907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003873+0000","last_change":"2024-04-13T03:12:11.992846+0000","last_active":"2024-04-13T03:12:12.003873+0000","last_peered":"2024-04-13T03:12:12.003873+0000","last_clean":"2024-04-13T03:12:12.003873+0000","last_became_active":"2024-04-13T03:12:11.992577+0000","last_became_peered":"2024-04-13T03:12:11.992577+0000","last_unstale":"2024-04-13T03:12:12.003873+0000","last_undegraded":"2024-04-13T03:12:12.003873+0000","last_fullsized":"2024-04-13T03:12:12.003873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:58:23.732544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010635+0000","last_change":"2024-04-13T03:12:12.008622+0000","last_active":"2024-04-13T03:12:12.010635+0000","last_peered":"2024-04-13T03:12:12.010635+0000","last_clean":"2024-04-13T03:12:12.010635+0000","last_became_active":"2024-04-13T03:12:12.008368+0000","last_became_peered":"2024-04-13T03:12:12.008368+0000","last_unstale":"2024-04-13T03:12:12.010635+0000","last_undegraded":"2024-04-13T03:12:12.010635+0000","last_fullsized":"2024-04-13T03:12:12.010635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:23:47.278596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013192+0000","last_change":"2024-04-13T03:12:12.009241+0000","last_active":"2024-04-13T03:12:12.013192+0000","last_peered":"2024-04-13T03:12:12.013192+0000","last_clean":"2024-04-13T03:12:12.013192+0000","last_became_active":"2024-04-13T03:12:12.008972+0000","last_became_peered":"2024-04-13T03:12:12.008972+0000","last_unstale":"2024-04-13T03:12:12.013192+0000","last_undegraded":"2024-04-13T03:12:12.013192+0000","last_fullsized":"2024-04-13T03:12:12.013192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:35.022725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008340+0000","last_change":"2024-04-13T03:12:11.996130+0000","last_active":"2024-04-13T03:12:12.008340+0000","last_peered":"2024-04-13T03:12:12.008340+0000","last_clean":"2024-04-13T03:12:12.008340+0000","last_became_active":"2024-04-13T03:12:11.995855+0000","last_became_peered":"2024-04-13T03:12:11.995855+0000","last_unstale":"2024-04-13T03:12:12.008340+0000","last_undegraded":"2024-04-13T03:12:12.008340+0000","last_fullsized":"2024-04-13T03:12:12.008340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011295+0000","last_change":"2024-04-13T03:12:12.006798+0000","last_active":"2024-04-13T03:12:12.011295+0000","last_peered":"2024-04-13T03:12:12.011295+0000","last_clean":"2024-04-13T03:12:12.011295+0000","last_became_active":"2024-04-13T03:12:12.006503+0000","last_became_peered":"2024-04-13T03:12:12.006503+0000","last_unstale":"2024-04-13T03:12:12.011295+0000","last_undegraded":"2024-04-13T03:12:12.011295+0000","last_fullsized":"2024-04-13T03:12:12.011295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:18.495277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010439+0000","last_change":"2024-04-13T03:12:12.002285+0000","last_active":"2024-04-13T03:12:12.010439+0000","last_peered":"2024-04-13T03:12:12.010439+0000","last_clean":"2024-04-13T03:12:12.010439+0000","last_became_active":"2024-04-13T03:12:12.001971+0000","last_became_peered":"2024-04-13T03:12:12.001971+0000","last_unstale":"2024-04-13T03:12:12.010439+0000","last_undegraded":"2024-04-13T03:12:12.010439+0000","last_fullsized":"2024-04-13T03:12:12.010439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:27.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010613+0000","last_change":"2024-04-13T03:12:12.007375+0000","last_active":"2024-04-13T03:12:12.010613+0000","last_peered":"2024-04-13T03:12:12.010613+0000","last_clean":"2024-04-13T03:12:12.010613+0000","last_became_active":"2024-04-13T03:12:12.007106+0000","last_became_peered":"2024-04-13T03:12:12.007106+0000","last_unstale":"2024-04-13T03:12:12.010613+0000","last_undegraded":"2024-04-13T03:12:12.010613+0000","last_fullsized":"2024-04-13T03:12:12.010613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:27.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005135+0000","last_change":"2024-04-13T03:12:11.997334+0000","last_active":"2024-04-13T03:12:12.005135+0000","last_peered":"2024-04-13T03:12:12.005135+0000","last_clean":"2024-04-13T03:12:12.005135+0000","last_became_active":"2024-04-13T03:12:11.996805+0000","last_became_peered":"2024-04-13T03:12:11.996805+0000","last_unstale":"2024-04-13T03:12:12.005135+0000","last_undegraded":"2024-04-13T03:12:12.005135+0000","last_fullsized":"2024-04-13T03:12:12.005135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:08.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004250+0000","last_change":"2024-04-13T03:12:11.998049+0000","last_active":"2024-04-13T03:12:12.004250+0000","last_peered":"2024-04-13T03:12:12.004250+0000","last_clean":"2024-04-13T03:12:12.004250+0000","last_became_active":"2024-04-13T03:12:11.996306+0000","last_became_peered":"2024-04-13T03:12:11.996306+0000","last_unstale":"2024-04-13T03:12:12.004250+0000","last_undegraded":"2024-04-13T03:12:12.004250+0000","last_fullsized":"2024-04-13T03:12:12.004250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:21.412437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004364+0000","last_change":"2024-04-13T03:12:12.002454+0000","last_active":"2024-04-13T03:12:12.004364+0000","last_peered":"2024-04-13T03:12:12.004364+0000","last_clean":"2024-04-13T03:12:12.004364+0000","last_became_active":"2024-04-13T03:12:12.002112+0000","last_became_peered":"2024-04-13T03:12:12.002112+0000","last_unstale":"2024-04-13T03:12:12.004364+0000","last_undegraded":"2024-04-13T03:12:12.004364+0000","last_fullsized":"2024-04-13T03:12:12.004364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:09:17.601100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012076+0000","last_change":"2024-04-13T03:12:12.009855+0000","last_active":"2024-04-13T03:12:12.012076+0000","last_peered":"2024-04-13T03:12:12.012076+0000","last_clean":"2024-04-13T03:12:12.012076+0000","last_became_active":"2024-04-13T03:12:12.009081+0000","last_became_peered":"2024-04-13T03:12:12.009081+0000","last_unstale":"2024-04-13T03:12:12.012076+0000","last_undegraded":"2024-04-13T03:12:12.012076+0000","last_fullsized":"2024-04-13T03:12:12.012076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011360+0000","last_change":"2024-04-13T03:12:12.007849+0000","last_active":"2024-04-13T03:12:12.011360+0000","last_peered":"2024-04-13T03:12:12.011360+0000","last_clean":"2024-04-13T03:12:12.011360+0000","last_became_active":"2024-04-13T03:12:12.007576+0000","last_became_peered":"2024-04-13T03:12:12.007576+0000","last_unstale":"2024-04-13T03:12:12.011360+0000","last_undegraded":"2024-04-13T03:12:12.011360+0000","last_fullsized":"2024-04-13T03:12:12.011360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:00.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.001231+0000","last_change":"2024-04-13T03:12:11.996165+0000","last_active":"2024-04-13T03:12:12.001231+0000","last_peered":"2024-04-13T03:12:12.001231+0000","last_clean":"2024-04-13T03:12:12.001231+0000","last_became_active":"2024-04-13T03:12:11.995522+0000","last_became_peered":"2024-04-13T03:12:11.995522+0000","last_unstale":"2024-04-13T03:12:12.001231+0000","last_undegraded":"2024-04-13T03:12:12.001231+0000","last_fullsized":"2024-04-13T03:12:12.001231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:24:31.981955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.001423+0000","last_change":"2024-04-13T03:12:11.997051+0000","last_active":"2024-04-13T03:12:12.001423+0000","last_peered":"2024-04-13T03:12:12.001423+0000","last_clean":"2024-04-13T03:12:12.001423+0000","last_became_active":"2024-04-13T03:12:11.996669+0000","last_became_peered":"2024-04-13T03:12:11.996669+0000","last_unstale":"2024-04-13T03:12:12.001423+0000","last_undegraded":"2024-04-13T03:12:12.001423+0000","last_fullsized":"2024-04-13T03:12:12.001423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:39:21.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010399+0000","last_change":"2024-04-13T03:12:12.001644+0000","last_active":"2024-04-13T03:12:12.010399+0000","last_peered":"2024-04-13T03:12:12.010399+0000","last_clean":"2024-04-13T03:12:12.010399+0000","last_became_active":"2024-04-13T03:12:12.001292+0000","last_became_peered":"2024-04-13T03:12:12.001292+0000","last_unstale":"2024-04-13T03:12:12.010399+0000","last_undegraded":"2024-04-13T03:12:12.010399+0000","last_fullsized":"2024-04-13T03:12:12.010399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:15.732922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998645+0000","last_change":"2024-04-13T03:12:11.994436+0000","last_active":"2024-04-13T03:12:11.998645+0000","last_peered":"2024-04-13T03:12:11.998645+0000","last_clean":"2024-04-13T03:12:11.998645+0000","last_became_active":"2024-04-13T03:12:11.994175+0000","last_became_peered":"2024-04-13T03:12:11.994175+0000","last_unstale":"2024-04-13T03:12:11.998645+0000","last_undegraded":"2024-04-13T03:12:11.998645+0000","last_fullsized":"2024-04-13T03:12:11.998645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:32:21.931044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.002983+0000","last_change":"2024-04-13T03:12:11.998967+0000","last_active":"2024-04-13T03:12:12.002983+0000","last_peered":"2024-04-13T03:12:12.002983+0000","last_clean":"2024-04-13T03:12:12.002983+0000","last_became_active":"2024-04-13T03:12:11.998654+0000","last_became_peered":"2024-04-13T03:12:11.998654+0000","last_unstale":"2024-04-13T03:12:12.002983+0000","last_undegraded":"2024-04-13T03:12:12.002983+0000","last_fullsized":"2024-04-13T03:12:12.002983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:18.501133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.007543+0000","last_change":"2024-04-13T03:12:11.998431+0000","last_active":"2024-04-13T03:12:12.007543+0000","last_peered":"2024-04-13T03:12:12.007543+0000","last_clean":"2024-04-13T03:12:12.007543+0000","last_became_active":"2024-04-13T03:12:11.997557+0000","last_became_peered":"2024-04-13T03:12:11.997557+0000","last_unstale":"2024-04-13T03:12:12.007543+0000","last_undegraded":"2024-04-13T03:12:12.007543+0000","last_fullsized":"2024-04-13T03:12:12.007543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:23:47.278596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011289+0000","last_change":"2024-04-13T03:12:12.005617+0000","last_active":"2024-04-13T03:12:12.011289+0000","last_peered":"2024-04-13T03:12:12.011289+0000","last_clean":"2024-04-13T03:12:12.011289+0000","last_became_active":"2024-04-13T03:12:12.005347+0000","last_became_peered":"2024-04-13T03:12:12.005347+0000","last_unstale":"2024-04-13T03:12:12.011289+0000","last_undegraded":"2024-04-13T03:12:12.011289+0000","last_fullsized":"2024-04-13T03:12:12.011289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:08.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010657+0000","last_change":"2024-04-13T03:12:12.004702+0000","last_active":"2024-04-13T03:12:12.010657+0000","last_peered":"2024-04-13T03:12:12.010657+0000","last_clean":"2024-04-13T03:12:12.010657+0000","last_became_active":"2024-04-13T03:12:12.004442+0000","last_became_peered":"2024-04-13T03:12:12.004442+0000","last_unstale":"2024-04-13T03:12:12.010657+0000","last_undegraded":"2024-04-13T03:12:12.010657+0000","last_fullsized":"2024-04-13T03:12:12.010657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:13.760383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011112+0000","last_change":"2024-04-13T03:12:12.006023+0000","last_active":"2024-04-13T03:12:12.011112+0000","last_peered":"2024-04-13T03:12:12.011112+0000","last_clean":"2024-04-13T03:12:12.011112+0000","last_became_active":"2024-04-13T03:12:12.005764+0000","last_became_peered":"2024-04-13T03:12:12.005764+0000","last_unstale":"2024-04-13T03:12:12.011112+0000","last_undegraded":"2024-04-13T03:12:12.011112+0000","last_fullsized":"2024-04-13T03:12:12.011112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:22.037533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.996322+0000","last_change":"2024-04-13T03:12:11.992842+0000","last_active":"2024-04-13T03:12:11.996322+0000","last_peered":"2024-04-13T03:12:11.996322+0000","last_clean":"2024-04-13T03:12:11.996322+0000","last_became_active":"2024-04-13T03:12:11.992572+0000","last_became_peered":"2024-04-13T03:12:11.992572+0000","last_unstale":"2024-04-13T03:12:11.996322+0000","last_undegraded":"2024-04-13T03:12:11.996322+0000","last_fullsized":"2024-04-13T03:12:11.996322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:00.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013220+0000","last_change":"2024-04-13T03:12:12.009920+0000","last_active":"2024-04-13T03:12:12.013220+0000","last_peered":"2024-04-13T03:12:12.013220+0000","last_clean":"2024-04-13T03:12:12.013220+0000","last_became_active":"2024-04-13T03:12:12.009674+0000","last_became_peered":"2024-04-13T03:12:12.009674+0000","last_unstale":"2024-04-13T03:12:12.013220+0000","last_undegraded":"2024-04-13T03:12:12.013220+0000","last_fullsized":"2024-04-13T03:12:12.013220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:14:59.480738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010757+0000","last_change":"2024-04-13T03:12:12.002921+0000","last_active":"2024-04-13T03:12:12.010757+0000","last_peered":"2024-04-13T03:12:12.010757+0000","last_clean":"2024-04-13T03:12:12.010757+0000","last_became_active":"2024-04-13T03:12:12.002575+0000","last_became_peered":"2024-04-13T03:12:12.002575+0000","last_unstale":"2024-04-13T03:12:12.010757+0000","last_undegraded":"2024-04-13T03:12:12.010757+0000","last_fullsized":"2024-04-13T03:12:12.010757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:42.179303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011153+0000","last_change":"2024-04-13T03:12:12.006815+0000","last_active":"2024-04-13T03:12:12.011153+0000","last_peered":"2024-04-13T03:12:12.011153+0000","last_clean":"2024-04-13T03:12:12.011153+0000","last_became_active":"2024-04-13T03:12:12.006548+0000","last_became_peered":"2024-04-13T03:12:12.006548+0000","last_unstale":"2024-04-13T03:12:12.011153+0000","last_undegraded":"2024-04-13T03:12:12.011153+0000","last_fullsized":"2024-04-13T03:12:12.011153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:15.810689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004360+0000","last_change":"2024-04-13T03:12:11.995517+0000","last_active":"2024-04-13T03:12:12.004360+0000","last_peered":"2024-04-13T03:12:12.004360+0000","last_clean":"2024-04-13T03:12:12.004360+0000","last_became_active":"2024-04-13T03:12:11.995072+0000","last_became_peered":"2024-04-13T03:12:11.995072+0000","last_unstale":"2024-04-13T03:12:12.004360+0000","last_undegraded":"2024-04-13T03:12:12.004360+0000","last_fullsized":"2024-04-13T03:12:12.004360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:31:13.521898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.991730+0000","last_change":"2024-04-13T03:12:11.987194+0000","last_active":"2024-04-13T03:12:11.991730+0000","last_peered":"2024-04-13T03:12:11.991730+0000","last_clean":"2024-04-13T03:12:11.991730+0000","last_became_active":"2024-04-13T03:12:11.986906+0000","last_became_peered":"2024-04-13T03:12:11.986906+0000","last_unstale":"2024-04-13T03:12:11.991730+0000","last_undegraded":"2024-04-13T03:12:11.991730+0000","last_fullsized":"2024-04-13T03:12:11.991730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:45:12.346630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013480+0000","last_change":"2024-04-13T03:12:11.000289+0000","last_active":"2024-04-13T03:12:12.013480+0000","last_peered":"2024-04-13T03:12:12.013480+0000","last_clean":"2024-04-13T03:12:12.013480+0000","last_became_active":"2024-04-13T03:12:11.000006+0000","last_became_peered":"2024-04-13T03:12:11.000006+0000","last_unstale":"2024-04-13T03:12:12.013480+0000","last_undegraded":"2024-04-13T03:12:12.013480+0000","last_fullsized":"2024-04-13T03:12:12.013480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:14.808497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010230+0000","last_change":"2024-04-13T03:12:12.005172+0000","last_active":"2024-04-13T03:12:12.010230+0000","last_peered":"2024-04-13T03:12:12.010230+0000","last_clean":"2024-04-13T03:12:12.010230+0000","last_became_active":"2024-04-13T03:12:12.004846+0000","last_became_peered":"2024-04-13T03:12:12.004846+0000","last_unstale":"2024-04-13T03:12:12.010230+0000","last_undegraded":"2024-04-13T03:12:12.010230+0000","last_fullsized":"2024-04-13T03:12:12.010230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:49.988398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012012+0000","last_change":"2024-04-13T03:12:10.993982+0000","last_active":"2024-04-13T03:12:12.012012+0000","last_peered":"2024-04-13T03:12:12.012012+0000","last_clean":"2024-04-13T03:12:12.012012+0000","last_became_active":"2024-04-13T03:12:10.993614+0000","last_became_peered":"2024-04-13T03:12:10.993614+0000","last_unstale":"2024-04-13T03:12:12.012012+0000","last_undegraded":"2024-04-13T03:12:12.012012+0000","last_fullsized":"2024-04-13T03:12:12.012012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:25.547973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004442+0000","last_change":"2024-04-13T03:12:11.996133+0000","last_active":"2024-04-13T03:12:12.004442+0000","last_peered":"2024-04-13T03:12:12.004442+0000","last_clean":"2024-04-13T03:12:12.004442+0000","last_became_active":"2024-04-13T03:12:11.995842+0000","last_became_peered":"2024-04-13T03:12:11.995842+0000","last_unstale":"2024-04-13T03:12:12.004442+0000","last_undegraded":"2024-04-13T03:12:12.004442+0000","last_fullsized":"2024-04-13T03:12:12.004442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:06:01.242240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011125+0000","last_change":"2024-04-13T03:12:10.987486+0000","last_active":"2024-04-13T03:12:12.011125+0000","last_peered":"2024-04-13T03:12:12.011125+0000","last_clean":"2024-04-13T03:12:12.011125+0000","last_became_active":"2024-04-13T03:12:10.987220+0000","last_became_peered":"2024-04-13T03:12:10.987220+0000","last_unstale":"2024-04-13T03:12:12.011125+0000","last_undegraded":"2024-04-13T03:12:12.011125+0000","last_fullsized":"2024-04-13T03:12:12.011125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:20:36.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005367+0000","last_change":"2024-04-13T03:12:11.998185+0000","last_active":"2024-04-13T03:12:12.005367+0000","last_peered":"2024-04-13T03:12:12.005367+0000","last_clean":"2024-04-13T03:12:12.005367+0000","last_became_active":"2024-04-13T03:12:11.997593+0000","last_became_peered":"2024-04-13T03:12:11.997593+0000","last_unstale":"2024-04-13T03:12:12.005367+0000","last_undegraded":"2024-04-13T03:12:12.005367+0000","last_fullsized":"2024-04-13T03:12:12.005367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:28.460426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009418+0000","last_change":"2024-04-13T03:12:10.995495+0000","last_active":"2024-04-13T03:12:12.009418+0000","last_peered":"2024-04-13T03:12:12.009418+0000","last_clean":"2024-04-13T03:12:12.009418+0000","last_became_active":"2024-04-13T03:12:10.995206+0000","last_became_peered":"2024-04-13T03:12:10.995206+0000","last_unstale":"2024-04-13T03:12:12.009418+0000","last_undegraded":"2024-04-13T03:12:12.009418+0000","last_fullsized":"2024-04-13T03:12:12.009418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:11:55.728873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004343+0000","last_change":"2024-04-13T03:12:11.995447+0000","last_active":"2024-04-13T03:12:12.004343+0000","last_peered":"2024-04-13T03:12:12.004343+0000","last_clean":"2024-04-13T03:12:12.004343+0000","last_became_active":"2024-04-13T03:12:11.995192+0000","last_became_peered":"2024-04-13T03:12:11.995192+0000","last_unstale":"2024-04-13T03:12:12.004343+0000","last_undegraded":"2024-04-13T03:12:12.004343+0000","last_fullsized":"2024-04-13T03:12:12.004343+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:05.022231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006523+0000","last_change":"2024-04-13T03:12:10.995555+0000","last_active":"2024-04-13T03:12:12.006523+0000","last_peered":"2024-04-13T03:12:12.006523+0000","last_clean":"2024-04-13T03:12:12.006523+0000","last_became_active":"2024-04-13T03:12:10.995315+0000","last_became_peered":"2024-04-13T03:12:10.995315+0000","last_unstale":"2024-04-13T03:12:12.006523+0000","last_undegraded":"2024-04-13T03:12:12.006523+0000","last_fullsized":"2024-04-13T03:12:12.006523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:31:46.289265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009030+0000","last_change":"2024-04-13T03:12:12.007004+0000","last_active":"2024-04-13T03:12:12.009030+0000","last_peered":"2024-04-13T03:12:12.009030+0000","last_clean":"2024-04-13T03:12:12.009030+0000","last_became_active":"2024-04-13T03:12:12.006660+0000","last_became_peered":"2024-04-13T03:12:12.006660+0000","last_unstale":"2024-04-13T03:12:12.009030+0000","last_undegraded":"2024-04-13T03:12:12.009030+0000","last_fullsized":"2024-04-13T03:12:12.009030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:42.179303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006359+0000","last_change":"2024-04-13T03:12:10.991165+0000","last_active":"2024-04-13T03:12:12.006359+0000","last_peered":"2024-04-13T03:12:12.006359+0000","last_clean":"2024-04-13T03:12:12.006359+0000","last_became_active":"2024-04-13T03:12:10.990606+0000","last_became_peered":"2024-04-13T03:12:10.990606+0000","last_unstale":"2024-04-13T03:12:12.006359+0000","last_undegraded":"2024-04-13T03:12:12.006359+0000","last_fullsized":"2024-04-13T03:12:12.006359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:06:00.240048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998725+0000","last_change":"2024-04-13T03:12:11.994449+0000","last_active":"2024-04-13T03:12:11.998725+0000","last_peered":"2024-04-13T03:12:11.998725+0000","last_clean":"2024-04-13T03:12:11.998725+0000","last_became_active":"2024-04-13T03:12:11.993141+0000","last_became_peered":"2024-04-13T03:12:11.993141+0000","last_unstale":"2024-04-13T03:12:11.998725+0000","last_undegraded":"2024-04-13T03:12:11.998725+0000","last_fullsized":"2024-04-13T03:12:11.998725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:03:18.810085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987951+0000","last_change":"2024-04-13T03:12:10.992458+0000","last_active":"2024-04-13T03:12:11.987951+0000","last_peered":"2024-04-13T03:12:11.987951+0000","last_clean":"2024-04-13T03:12:11.987951+0000","last_became_active":"2024-04-13T03:12:10.992194+0000","last_became_peered":"2024-04-13T03:12:10.992194+0000","last_unstale":"2024-04-13T03:12:11.987951+0000","last_undegraded":"2024-04-13T03:12:11.987951+0000","last_fullsized":"2024-04-13T03:12:11.987951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:16.165557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008781+0000","last_change":"2024-04-13T03:12:12.007408+0000","last_active":"2024-04-13T03:12:12.008781+0000","last_peered":"2024-04-13T03:12:12.008781+0000","last_clean":"2024-04-13T03:12:12.008781+0000","last_became_active":"2024-04-13T03:12:12.007107+0000","last_became_peered":"2024-04-13T03:12:12.007107+0000","last_unstale":"2024-04-13T03:12:12.008781+0000","last_undegraded":"2024-04-13T03:12:12.008781+0000","last_fullsized":"2024-04-13T03:12:12.008781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:38:33.202471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982644+0000","last_change":"2024-04-13T03:12:11.000011+0000","last_active":"2024-04-13T03:12:11.982644+0000","last_peered":"2024-04-13T03:12:11.982644+0000","last_clean":"2024-04-13T03:12:11.982644+0000","last_became_active":"2024-04-13T03:12:10.999734+0000","last_became_peered":"2024-04-13T03:12:10.999734+0000","last_unstale":"2024-04-13T03:12:11.982644+0000","last_undegraded":"2024-04-13T03:12:11.982644+0000","last_fullsized":"2024-04-13T03:12:11.982644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:53:42.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003159+0000","last_change":"2024-04-13T03:12:11.994287+0000","last_active":"2024-04-13T03:12:12.003159+0000","last_peered":"2024-04-13T03:12:12.003159+0000","last_clean":"2024-04-13T03:12:12.003159+0000","last_became_active":"2024-04-13T03:12:11.994031+0000","last_became_peered":"2024-04-13T03:12:11.994031+0000","last_unstale":"2024-04-13T03:12:12.003159+0000","last_undegraded":"2024-04-13T03:12:12.003159+0000","last_fullsized":"2024-04-13T03:12:12.003159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:39.898125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989152+0000","last_change":"2024-04-13T03:12:11.001540+0000","last_active":"2024-04-13T03:12:11.989152+0000","last_peered":"2024-04-13T03:12:11.989152+0000","last_clean":"2024-04-13T03:12:11.989152+0000","last_became_active":"2024-04-13T03:12:11.001298+0000","last_became_peered":"2024-04-13T03:12:11.001298+0000","last_unstale":"2024-04-13T03:12:11.989152+0000","last_undegraded":"2024-04-13T03:12:11.989152+0000","last_fullsized":"2024-04-13T03:12:11.989152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:12.758191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003295+0000","last_change":"2024-04-13T03:12:11.999580+0000","last_active":"2024-04-13T03:12:12.003295+0000","last_peered":"2024-04-13T03:12:12.003295+0000","last_clean":"2024-04-13T03:12:12.003295+0000","last_became_active":"2024-04-13T03:12:11.999325+0000","last_became_peered":"2024-04-13T03:12:11.999325+0000","last_unstale":"2024-04-13T03:12:12.003295+0000","last_undegraded":"2024-04-13T03:12:12.003295+0000","last_fullsized":"2024-04-13T03:12:12.003295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:43:24.721800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.994994+0000","last_change":"2024-04-13T03:12:10.996057+0000","last_active":"2024-04-13T03:12:11.994994+0000","last_peered":"2024-04-13T03:12:11.994994+0000","last_clean":"2024-04-13T03:12:11.994994+0000","last_became_active":"2024-04-13T03:12:10.995772+0000","last_became_peered":"2024-04-13T03:12:10.995772+0000","last_unstale":"2024-04-13T03:12:11.994994+0000","last_undegraded":"2024-04-13T03:12:11.994994+0000","last_fullsized":"2024-04-13T03:12:11.994994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:24:30.979763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.007423+0000","last_change":"2024-04-13T03:12:11.999989+0000","last_active":"2024-04-13T03:12:12.007423+0000","last_peered":"2024-04-13T03:12:12.007423+0000","last_clean":"2024-04-13T03:12:12.007423+0000","last_became_active":"2024-04-13T03:12:11.999638+0000","last_became_peered":"2024-04-13T03:12:11.999638+0000","last_unstale":"2024-04-13T03:12:12.007423+0000","last_undegraded":"2024-04-13T03:12:12.007423+0000","last_fullsized":"2024-04-13T03:12:12.007423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:02:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013212+0000","last_change":"2024-04-13T03:12:11.001477+0000","last_active":"2024-04-13T03:12:12.013212+0000","last_peered":"2024-04-13T03:12:12.013212+0000","last_clean":"2024-04-13T03:12:12.013212+0000","last_became_active":"2024-04-13T03:12:11.001262+0000","last_became_peered":"2024-04-13T03:12:11.001262+0000","last_unstale":"2024-04-13T03:12:12.013212+0000","last_undegraded":"2024-04-13T03:12:12.013212+0000","last_fullsized":"2024-04-13T03:12:12.013212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:26.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010057+0000","last_change":"2024-04-13T03:12:12.004097+0000","last_active":"2024-04-13T03:12:12.010057+0000","last_peered":"2024-04-13T03:12:12.010057+0000","last_clean":"2024-04-13T03:12:12.010057+0000","last_became_active":"2024-04-13T03:12:12.003694+0000","last_became_peered":"2024-04-13T03:12:12.003694+0000","last_unstale":"2024-04-13T03:12:12.010057+0000","last_undegraded":"2024-04-13T03:12:12.010057+0000","last_fullsized":"2024-04-13T03:12:12.010057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:24.027177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013867+0000","last_change":"2024-04-13T03:12:10.997926+0000","last_active":"2024-04-13T03:12:12.013867+0000","last_peered":"2024-04-13T03:12:12.013867+0000","last_clean":"2024-04-13T03:12:12.013867+0000","last_became_active":"2024-04-13T03:12:10.997507+0000","last_became_peered":"2024-04-13T03:12:10.997507+0000","last_unstale":"2024-04-13T03:12:12.013867+0000","last_undegraded":"2024-04-13T03:12:12.013867+0000","last_fullsized":"2024-04-13T03:12:12.013867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:21.035340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006872+0000","last_change":"2024-04-13T03:12:11.997061+0000","last_active":"2024-04-13T03:12:12.006872+0000","last_peered":"2024-04-13T03:12:12.006872+0000","last_clean":"2024-04-13T03:12:12.006872+0000","last_became_active":"2024-04-13T03:12:11.996792+0000","last_became_peered":"2024-04-13T03:12:11.996792+0000","last_unstale":"2024-04-13T03:12:12.006872+0000","last_undegraded":"2024-04-13T03:12:12.006872+0000","last_fullsized":"2024-04-13T03:12:12.006872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:02:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013006+0000","last_change":"2024-04-13T03:12:11.000799+0000","last_active":"2024-04-13T03:12:12.013006+0000","last_peered":"2024-04-13T03:12:12.013006+0000","last_clean":"2024-04-13T03:12:12.013006+0000","last_became_active":"2024-04-13T03:12:11.000493+0000","last_became_peered":"2024-04-13T03:12:11.000493+0000","last_unstale":"2024-04-13T03:12:12.013006+0000","last_undegraded":"2024-04-13T03:12:12.013006+0000","last_fullsized":"2024-04-13T03:12:12.013006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:25.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010560+0000","last_change":"2024-04-13T03:12:12.004796+0000","last_active":"2024-04-13T03:12:12.010560+0000","last_peered":"2024-04-13T03:12:12.010560+0000","last_clean":"2024-04-13T03:12:12.010560+0000","last_became_active":"2024-04-13T03:12:12.004537+0000","last_became_peered":"2024-04-13T03:12:12.004537+0000","last_unstale":"2024-04-13T03:12:12.010560+0000","last_undegraded":"2024-04-13T03:12:12.010560+0000","last_fullsized":"2024-04-13T03:12:12.010560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992254+0000","last_change":"2024-04-13T03:12:10.993159+0000","last_active":"2024-04-13T03:12:11.992254+0000","last_peered":"2024-04-13T03:12:11.992254+0000","last_clean":"2024-04-13T03:12:11.992254+0000","last_became_active":"2024-04-13T03:12:10.992887+0000","last_became_peered":"2024-04-13T03:12:10.992887+0000","last_unstale":"2024-04-13T03:12:11.992254+0000","last_undegraded":"2024-04-13T03:12:11.992254+0000","last_fullsized":"2024-04-13T03:12:11.992254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:14.808497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.000995+0000","last_change":"2024-04-13T03:12:11.995506+0000","last_active":"2024-04-13T03:12:12.000995+0000","last_peered":"2024-04-13T03:12:12.000995+0000","last_clean":"2024-04-13T03:12:12.000995+0000","last_became_active":"2024-04-13T03:12:11.995228+0000","last_became_peered":"2024-04-13T03:12:11.995228+0000","last_unstale":"2024-04-13T03:12:12.000995+0000","last_undegraded":"2024-04-13T03:12:12.000995+0000","last_fullsized":"2024-04-13T03:12:12.000995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:04:57.272523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.988049+0000","last_change":"2024-04-13T03:12:11.002952+0000","last_active":"2024-04-13T03:12:11.988049+0000","last_peered":"2024-04-13T03:12:11.988049+0000","last_clean":"2024-04-13T03:12:11.988049+0000","last_became_active":"2024-04-13T03:12:11.002568+0000","last_became_peered":"2024-04-13T03:12:11.002568+0000","last_unstale":"2024-04-13T03:12:11.988049+0000","last_undegraded":"2024-04-13T03:12:11.988049+0000","last_fullsized":"2024-04-13T03:12:11.988049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:14.730730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005858+0000","last_change":"2024-04-13T03:12:11.997983+0000","last_active":"2024-04-13T03:12:12.005858+0000","last_peered":"2024-04-13T03:12:12.005858+0000","last_clean":"2024-04-13T03:12:12.005858+0000","last_became_active":"2024-04-13T03:12:11.997191+0000","last_became_peered":"2024-04-13T03:12:11.997191+0000","last_unstale":"2024-04-13T03:12:12.005858+0000","last_undegraded":"2024-04-13T03:12:12.005858+0000","last_fullsized":"2024-04-13T03:12:12.005858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:15.732922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.015570+0000","last_change":"2024-04-13T03:12:10.992728+0000","last_active":"2024-04-13T03:12:12.015570+0000","last_peered":"2024-04-13T03:12:12.015570+0000","last_clean":"2024-04-13T03:12:12.015570+0000","last_became_active":"2024-04-13T03:12:10.992375+0000","last_became_peered":"2024-04-13T03:12:10.992375+0000","last_unstale":"2024-04-13T03:12:12.015570+0000","last_undegraded":"2024-04-13T03:12:12.015570+0000","last_fullsized":"2024-04-13T03:12:12.015570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:04:56.270330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013252+0000","last_change":"2024-04-13T03:12:12.010553+0000","last_active":"2024-04-13T03:12:12.013252+0000","last_peered":"2024-04-13T03:12:12.013252+0000","last_clean":"2024-04-13T03:12:12.013252+0000","last_became_active":"2024-04-13T03:12:12.010281+0000","last_became_peered":"2024-04-13T03:12:12.010281+0000","last_unstale":"2024-04-13T03:12:12.013252+0000","last_undegraded":"2024-04-13T03:12:12.013252+0000","last_fullsized":"2024-04-13T03:12:12.013252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:22.863303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011508+0000","last_change":"2024-04-13T03:12:10.993260+0000","last_active":"2024-04-13T03:12:12.011508+0000","last_peered":"2024-04-13T03:12:12.011508+0000","last_clean":"2024-04-13T03:12:12.011508+0000","last_became_active":"2024-04-13T03:12:10.992975+0000","last_became_peered":"2024-04-13T03:12:10.992975+0000","last_unstale":"2024-04-13T03:12:12.011508+0000","last_undegraded":"2024-04-13T03:12:12.011508+0000","last_fullsized":"2024-04-13T03:12:12.011508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:52:59.322006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012039+0000","last_change":"2024-04-13T03:12:12.010144+0000","last_active":"2024-04-13T03:12:12.012039+0000","last_peered":"2024-04-13T03:12:12.012039+0000","last_clean":"2024-04-13T03:12:12.012039+0000","last_became_active":"2024-04-13T03:12:12.009884+0000","last_became_peered":"2024-04-13T03:12:12.009884+0000","last_unstale":"2024-04-13T03:12:12.012039+0000","last_undegraded":"2024-04-13T03:12:12.012039+0000","last_fullsized":"2024-04-13T03:12:12.012039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:49.988398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011615+0000","last_change":"2024-04-13T03:12:11.003031+0000","last_active":"2024-04-13T03:12:12.011615+0000","last_peered":"2024-04-13T03:12:12.011615+0000","last_clean":"2024-04-13T03:12:12.011615+0000","last_became_active":"2024-04-13T03:12:11.002743+0000","last_became_peered":"2024-04-13T03:12:11.002743+0000","last_unstale":"2024-04-13T03:12:12.011615+0000","last_undegraded":"2024-04-13T03:12:12.011615+0000","last_fullsized":"2024-04-13T03:12:12.011615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:52:59.322006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010996+0000","last_change":"2024-04-13T03:12:12.005588+0000","last_active":"2024-04-13T03:12:12.010996+0000","last_peered":"2024-04-13T03:12:12.010996+0000","last_clean":"2024-04-13T03:12:12.010996+0000","last_became_active":"2024-04-13T03:12:12.003720+0000","last_became_peered":"2024-04-13T03:12:12.003720+0000","last_unstale":"2024-04-13T03:12:12.010996+0000","last_undegraded":"2024-04-13T03:12:12.010996+0000","last_fullsized":"2024-04-13T03:12:12.010996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:57.941380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012476+0000","last_change":"2024-04-13T03:12:11.002986+0000","last_active":"2024-04-13T03:12:12.012476+0000","last_peered":"2024-04-13T03:12:12.012476+0000","last_clean":"2024-04-13T03:12:12.012476+0000","last_became_active":"2024-04-13T03:12:11.002702+0000","last_became_peered":"2024-04-13T03:12:11.002702+0000","last_unstale":"2024-04-13T03:12:12.012476+0000","last_undegraded":"2024-04-13T03:12:12.012476+0000","last_fullsized":"2024-04-13T03:12:12.012476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:41.177111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998690+0000","last_change":"2024-04-13T03:12:11.994891+0000","last_active":"2024-04-13T03:12:11.998690+0000","last_peered":"2024-04-13T03:12:11.998690+0000","last_clean":"2024-04-13T03:12:11.998690+0000","last_became_active":"2024-04-13T03:12:11.994600+0000","last_became_peered":"2024-04-13T03:12:11.994600+0000","last_unstale":"2024-04-13T03:12:11.998690+0000","last_undegraded":"2024-04-13T03:12:11.998690+0000","last_fullsized":"2024-04-13T03:12:11.998690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:40:02.621231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987501+0000","last_change":"2024-04-13T03:12:10.988990+0000","last_active":"2024-04-13T03:12:11.987501+0000","last_peered":"2024-04-13T03:12:11.987501+0000","last_clean":"2024-04-13T03:12:11.987501+0000","last_became_active":"2024-04-13T03:12:10.988284+0000","last_became_peered":"2024-04-13T03:12:10.988284+0000","last_unstale":"2024-04-13T03:12:11.987501+0000","last_undegraded":"2024-04-13T03:12:11.987501+0000","last_fullsized":"2024-04-13T03:12:11.987501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:15.252715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004337+0000","last_change":"2024-04-13T03:12:11.995310+0000","last_active":"2024-04-13T03:12:12.004337+0000","last_peered":"2024-04-13T03:12:12.004337+0000","last_clean":"2024-04-13T03:12:12.004337+0000","last_became_active":"2024-04-13T03:12:11.994744+0000","last_became_peered":"2024-04-13T03:12:11.994744+0000","last_unstale":"2024-04-13T03:12:12.004337+0000","last_undegraded":"2024-04-13T03:12:12.004337+0000","last_fullsized":"2024-04-13T03:12:12.004337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:14:08.729277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009625+0000","last_change":"2024-04-13T03:12:10.992458+0000","last_active":"2024-04-13T03:12:12.009625+0000","last_peered":"2024-04-13T03:12:12.009625+0000","last_clean":"2024-04-13T03:12:12.009625+0000","last_became_active":"2024-04-13T03:12:10.992194+0000","last_became_peered":"2024-04-13T03:12:10.992194+0000","last_unstale":"2024-04-13T03:12:12.009625+0000","last_undegraded":"2024-04-13T03:12:12.009625+0000","last_fullsized":"2024-04-13T03:12:12.009625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:49.172143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010582+0000","last_change":"2024-04-13T03:12:12.006578+0000","last_active":"2024-04-13T03:12:12.010582+0000","last_peered":"2024-04-13T03:12:12.010582+0000","last_clean":"2024-04-13T03:12:12.010582+0000","last_became_active":"2024-04-13T03:12:12.005562+0000","last_became_peered":"2024-04-13T03:12:12.005562+0000","last_unstale":"2024-04-13T03:12:12.010582+0000","last_undegraded":"2024-04-13T03:12:12.010582+0000","last_fullsized":"2024-04-13T03:12:12.010582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:28.460426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992424+0000","last_change":"2024-04-13T03:12:10.994192+0000","last_active":"2024-04-13T03:12:11.992424+0000","last_peered":"2024-04-13T03:12:11.992424+0000","last_clean":"2024-04-13T03:12:11.992424+0000","last_became_active":"2024-04-13T03:12:10.993933+0000","last_became_peered":"2024-04-13T03:12:10.993933+0000","last_unstale":"2024-04-13T03:12:11.992424+0000","last_undegraded":"2024-04-13T03:12:11.992424+0000","last_fullsized":"2024-04-13T03:12:11.992424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:25.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989231+0000","last_change":"2024-04-13T03:12:11.986810+0000","last_active":"2024-04-13T03:12:11.989231+0000","last_peered":"2024-04-13T03:12:11.989231+0000","last_clean":"2024-04-13T03:12:11.989231+0000","last_became_active":"2024-04-13T03:12:11.986475+0000","last_became_peered":"2024-04-13T03:12:11.986475+0000","last_unstale":"2024-04-13T03:12:11.989231+0000","last_undegraded":"2024-04-13T03:12:11.989231+0000","last_fullsized":"2024-04-13T03:12:11.989231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:46.554354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982158+0000","last_change":"2024-04-13T03:12:10.993807+0000","last_active":"2024-04-13T03:12:11.982158+0000","last_peered":"2024-04-13T03:12:11.982158+0000","last_clean":"2024-04-13T03:12:11.982158+0000","last_became_active":"2024-04-13T03:12:10.993495+0000","last_became_peered":"2024-04-13T03:12:10.993495+0000","last_unstale":"2024-04-13T03:12:11.982158+0000","last_undegraded":"2024-04-13T03:12:11.982158+0000","last_fullsized":"2024-04-13T03:12:11.982158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:39:07.695986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010188+0000","last_change":"2024-04-13T03:12:12.007995+0000","last_active":"2024-04-13T03:12:12.010188+0000","last_peered":"2024-04-13T03:12:12.010188+0000","last_clean":"2024-04-13T03:12:12.010188+0000","last_became_active":"2024-04-13T03:12:12.007743+0000","last_became_peered":"2024-04-13T03:12:12.007743+0000","last_unstale":"2024-04-13T03:12:12.010188+0000","last_undegraded":"2024-04-13T03:12:12.010188+0000","last_fullsized":"2024-04-13T03:12:12.010188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:34:18.362959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010350+0000","last_change":"2024-04-13T03:12:10.995086+0000","last_active":"2024-04-13T03:12:12.010350+0000","last_peered":"2024-04-13T03:12:12.010350+0000","last_clean":"2024-04-13T03:12:12.010350+0000","last_became_active":"2024-04-13T03:12:10.994824+0000","last_became_peered":"2024-04-13T03:12:10.994824+0000","last_unstale":"2024-04-13T03:12:12.010350+0000","last_undegraded":"2024-04-13T03:12:12.010350+0000","last_fullsized":"2024-04-13T03:12:12.010350+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:38.895933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004057+0000","last_change":"2024-04-13T03:12:11.994087+0000","last_active":"2024-04-13T03:12:12.004057+0000","last_peered":"2024-04-13T03:12:12.004057+0000","last_clean":"2024-04-13T03:12:12.004057+0000","last_became_active":"2024-04-13T03:12:11.993830+0000","last_became_peered":"2024-04-13T03:12:11.993830+0000","last_unstale":"2024-04-13T03:12:12.004057+0000","last_undegraded":"2024-04-13T03:12:12.004057+0000","last_fullsized":"2024-04-13T03:12:12.004057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:30:20.688194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992023+0000","last_change":"2024-04-13T03:12:10.990141+0000","last_active":"2024-04-13T03:12:11.992023+0000","last_peered":"2024-04-13T03:12:11.992023+0000","last_clean":"2024-04-13T03:12:11.992023+0000","last_became_active":"2024-04-13T03:12:10.989849+0000","last_became_peered":"2024-04-13T03:12:10.989849+0000","last_unstale":"2024-04-13T03:12:11.992023+0000","last_undegraded":"2024-04-13T03:12:11.992023+0000","last_fullsized":"2024-04-13T03:12:11.992023+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:17.493085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010672+0000","last_change":"2024-04-13T03:12:12.006012+0000","last_active":"2024-04-13T03:12:12.010672+0000","last_peered":"2024-04-13T03:12:12.010672+0000","last_clean":"2024-04-13T03:12:12.010672+0000","last_became_active":"2024-04-13T03:12:12.005733+0000","last_became_peered":"2024-04-13T03:12:12.005733+0000","last_unstale":"2024-04-13T03:12:12.010672+0000","last_undegraded":"2024-04-13T03:12:12.010672+0000","last_fullsized":"2024-04-13T03:12:12.010672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:21.412437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982585+0000","last_change":"2024-04-13T03:12:10.985663+0000","last_active":"2024-04-13T03:12:11.982585+0000","last_peered":"2024-04-13T03:12:11.982585+0000","last_clean":"2024-04-13T03:12:11.982585+0000","last_became_active":"2024-04-13T03:12:10.985357+0000","last_became_peered":"2024-04-13T03:12:10.985357+0000","last_unstale":"2024-04-13T03:12:11.982585+0000","last_undegraded":"2024-04-13T03:12:11.982585+0000","last_fullsized":"2024-04-13T03:12:11.982585+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:28.708314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998409+0000","last_change":"2024-04-13T03:12:11.992737+0000","last_active":"2024-04-13T03:12:11.998409+0000","last_peered":"2024-04-13T03:12:11.998409+0000","last_clean":"2024-04-13T03:12:11.998409+0000","last_became_active":"2024-04-13T03:12:11.992460+0000","last_became_peered":"2024-04-13T03:12:11.992460+0000","last_unstale":"2024-04-13T03:12:11.998409+0000","last_undegraded":"2024-04-13T03:12:11.998409+0000","last_fullsized":"2024-04-13T03:12:11.998409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:33:26.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982314+0000","last_change":"2024-04-13T03:12:10.997885+0000","last_active":"2024-04-13T03:12:11.982314+0000","last_peered":"2024-04-13T03:12:11.982314+0000","last_clean":"2024-04-13T03:12:11.982314+0000","last_became_active":"2024-04-13T03:12:10.997484+0000","last_became_peered":"2024-04-13T03:12:10.997484+0000","last_unstale":"2024-04-13T03:12:11.982314+0000","last_undegraded":"2024-04-13T03:12:11.982314+0000","last_fullsized":"2024-04-13T03:12:11.982314+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:17.493085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989205+0000","last_change":"2024-04-13T03:12:11.987614+0000","last_active":"2024-04-13T03:12:11.989205+0000","last_peered":"2024-04-13T03:12:11.989205+0000","last_clean":"2024-04-13T03:12:11.989205+0000","last_became_active":"2024-04-13T03:12:11.987314+0000","last_became_peered":"2024-04-13T03:12:11.987314+0000","last_unstale":"2024-04-13T03:12:11.989205+0000","last_undegraded":"2024-04-13T03:12:11.989205+0000","last_fullsized":"2024-04-13T03:12:11.989205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:50:53.185826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987979+0000","last_change":"2024-04-13T03:12:11.001033+0000","last_active":"2024-04-13T03:12:11.987979+0000","last_peered":"2024-04-13T03:12:11.987979+0000","last_clean":"2024-04-13T03:12:11.987979+0000","last_became_active":"2024-04-13T03:12:11.000722+0000","last_became_peered":"2024-04-13T03:12:11.000722+0000","last_unstale":"2024-04-13T03:12:11.987979+0000","last_undegraded":"2024-04-13T03:12:11.987979+0000","last_fullsized":"2024-04-13T03:12:11.987979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:38.895933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011436+0000","last_change":"2024-04-13T03:12:12.009216+0000","last_active":"2024-04-13T03:12:12.011436+0000","last_peered":"2024-04-13T03:12:12.011436+0000","last_clean":"2024-04-13T03:12:12.011436+0000","last_became_active":"2024-04-13T03:12:12.008949+0000","last_became_peered":"2024-04-13T03:12:12.008949+0000","last_unstale":"2024-04-13T03:12:12.011436+0000","last_undegraded":"2024-04-13T03:12:12.011436+0000","last_fullsized":"2024-04-13T03:12:12.011436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:39.898125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987525+0000","last_change":"2024-04-13T03:12:10.995918+0000","last_active":"2024-04-13T03:12:11.987525+0000","last_peered":"2024-04-13T03:12:11.987525+0000","last_clean":"2024-04-13T03:12:11.987525+0000","last_became_active":"2024-04-13T03:12:10.994221+0000","last_became_peered":"2024-04-13T03:12:10.994221+0000","last_unstale":"2024-04-13T03:12:11.987525+0000","last_undegraded":"2024-04-13T03:12:11.987525+0000","last_fullsized":"2024-04-13T03:12:11.987525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:34:17.360767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008952+0000","last_change":"2024-04-13T03:12:12.007385+0000","last_active":"2024-04-13T03:12:12.008952+0000","last_peered":"2024-04-13T03:12:12.008952+0000","last_clean":"2024-04-13T03:12:12.008952+0000","last_became_active":"2024-04-13T03:12:12.007112+0000","last_became_peered":"2024-04-13T03:12:12.007112+0000","last_unstale":"2024-04-13T03:12:12.008952+0000","last_undegraded":"2024-04-13T03:12:12.008952+0000","last_fullsized":"2024-04-13T03:12:12.008952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:13.760383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.988102+0000","last_change":"2024-04-13T03:12:10.990048+0000","last_active":"2024-04-13T03:12:11.988102+0000","last_peered":"2024-04-13T03:12:11.988102+0000","last_clean":"2024-04-13T03:12:11.988102+0000","last_became_active":"2024-04-13T03:12:10.989765+0000","last_became_peered":"2024-04-13T03:12:10.989765+0000","last_unstale":"2024-04-13T03:12:11.988102+0000","last_undegraded":"2024-04-13T03:12:11.988102+0000","last_fullsized":"2024-04-13T03:12:11.988102+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:53:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004402+0000","last_change":"2024-04-13T03:12:12.000146+0000","last_active":"2024-04-13T03:12:12.004402+0000","last_peered":"2024-04-13T03:12:12.004402+0000","last_clean":"2024-04-13T03:12:12.004402+0000","last_became_active":"2024-04-13T03:12:11.999844+0000","last_became_peered":"2024-04-13T03:12:11.999844+0000","last_unstale":"2024-04-13T03:12:12.004402+0000","last_undegraded":"2024-04-13T03:12:12.004402+0000","last_fullsized":"2024-04-13T03:12:12.004402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:03:40.236913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.981444+0000","last_change":"2024-04-13T03:12:11.000775+0000","last_active":"2024-04-13T03:12:11.981444+0000","last_peered":"2024-04-13T03:12:11.981444+0000","last_clean":"2024-04-13T03:12:11.981444+0000","last_became_active":"2024-04-13T03:12:11.000448+0000","last_became_peered":"2024-04-13T03:12:11.000448+0000","last_unstale":"2024-04-13T03:12:11.981444+0000","last_undegraded":"2024-04-13T03:12:11.981444+0000","last_fullsized":"2024-04-13T03:12:11.981444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:56.939187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989232+0000","last_change":"2024-04-13T03:12:11.987151+0000","last_active":"2024-04-13T03:12:11.989232+0000","last_peered":"2024-04-13T03:12:11.989232+0000","last_clean":"2024-04-13T03:12:11.989232+0000","last_became_active":"2024-04-13T03:12:11.986818+0000","last_became_peered":"2024-04-13T03:12:11.986818+0000","last_unstale":"2024-04-13T03:12:11.989232+0000","last_undegraded":"2024-04-13T03:12:11.989232+0000","last_fullsized":"2024-04-13T03:12:11.989232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:56:11.584530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.986590+0000","last_change":"2024-04-13T03:12:10.995672+0000","last_active":"2024-04-13T03:12:11.986590+0000","last_peered":"2024-04-13T03:12:11.986590+0000","last_clean":"2024-04-13T03:12:11.986590+0000","last_became_active":"2024-04-13T03:12:10.995427+0000","last_became_peered":"2024-04-13T03:12:10.995427+0000","last_unstale":"2024-04-13T03:12:11.986590+0000","last_undegraded":"2024-04-13T03:12:11.986590+0000","last_fullsized":"2024-04-13T03:12:11.986590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:31:12.519706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987119+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:12:11.987119+0000","last_peered":"2024-04-13T03:12:11.987119+0000","last_clean":"2024-04-13T03:12:11.987119+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:12:11.987119+0000","last_undegraded":"2024-04-13T03:12:11.987119+0000","last_fullsized":"2024-04-13T03:12:11.987119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1343488,"data_stored":926546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":343379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":343367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":343345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":343345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1343488,"data_stored":926546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:12:14.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:12:15.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.105+0000 7f38a37f0640 1 Processor -- start 2024-04-13T03:12:15.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.105+0000 7f38a37f0640 1 -- start start 2024-04-13T03:12:15.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a37f0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f389c0cdf50 0x7f389c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a37f0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 0x7f389c072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a37f0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0ce870 con 0x7f389c06b8e0 2024-04-13T03:12:15.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a37f0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f389c0998d0 con 0x7f389c0cdf50 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a37f0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f389c073020 con 0x7f389c06cb10 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a158c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f389c0cdf50 0x7f389c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a1d8d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f389c06b8e0 0x7f389c06bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47466/0 (socket says 172.21.15.106:47466) 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a1d8d640 1 -- 172.21.15.106:0/1756418612 learned_addr learned my addr 172.21.15.106:0/1756418612 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a0d8b640 1 --2- 172.21.15.106:0/1756418612 >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 0x7f389c072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a158c640 1 -- 172.21.15.106:0/1756418612 >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 msgr2=0x7f389c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a158c640 1 --2- 172.21.15.106:0/1756418612 >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 0x7f389c072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a158c640 1 -- 172.21.15.106:0/1756418612 >> v1:172.21.15.106:6789/0 conn(0x7f389c06b8e0 legacy=0x7f389c06bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f3893fff640 1 -- 172.21.15.106:0/1756418612 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101829961 0 0) 0x7f389c0ce870 con 0x7f389c06b8e0 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a158c640 1 -- 172.21.15.106:0/1756418612 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c073300 con 0x7f389c0cdf50 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.106+0000 7f38a0d8b640 1 --2- 172.21.15.106:0/1756418612 >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 0x7f389c072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f38a158c640 1 --2- 172.21.15.106:0/1756418612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c0cdf50 0x7f389c0ce330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b64f0 tx=0x7f389802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af447d52b60160d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f3893fff640 1 -- 172.21.15.106:0/1756418612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898035020 con 0x7f389c0cdf50 2024-04-13T03:12:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f3893fff640 1 -- 172.21.15.106:0/1756418612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898031830 con 0x7f389c0cdf50 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f3893fff640 1 -- 172.21.15.106:0/1756418612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898031c00 con 0x7f389c0cdf50 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f38a37f0640 1 -- 172.21.15.106:0/1756418612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c0cdf50 msgr2=0x7f389c0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.107+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1756418612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c0cdf50 0x7f389c0ce330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b64f0 tx=0x7f389802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 -- 172.21.15.106:0/1756418612 shutdown_connections 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1756418612 >> v2:172.21.15.114:3300/0 conn(0x7f389c06cb10 0x7f389c072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1756418612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c0cdf50 0x7f389c0ce330 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 -- 172.21.15.106:0/1756418612 >> 172.21.15.106:0/1756418612 conn(0x7f389c0b1f10 msgr2=0x7f389c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 -- 172.21.15.106:0/1756418612 shutdown_connections 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.108+0000 7f38a37f0640 1 -- 172.21.15.106:0/1756418612 wait complete. 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 Processor -- start 2024-04-13T03:12:15.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 -- start start 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f389c06cb10 0x7f389c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f389c0cdf50 0x7f389c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f389c072c60 con 0x7f389c06b8e0 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0738c0 con 0x7f389c0cdf50 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.109+0000 7f38a37f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f389c072de0 con 0x7f389c06cb10 2024-04-13T03:12:15.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47968/0 (socket says 172.21.15.106:47968) 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 -- 172.21.15.106:0/1446458731 learned_addr learned my addr 172.21.15.106:0/1446458731 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 -- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f389c06cb10 msgr2=0x7f389c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f389c06cb10 0x7f389c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 -- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f389c0cdf50 msgr2=0x7f389c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f389c0cdf50 0x7f389c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 -- 172.21.15.106:0/1446458731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c1f42d0 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a158c640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3898009890 tx=0x7f38980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898004230 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980043d0 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c1f45b0 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389803ca50 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.110+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1f4b70 con 0x7f389c06b8e0 2024-04-13T03:12:15.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.111+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c068a10 con 0x7f389c06b8e0 2024-04-13T03:12:15.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.112+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3898038020 con 0x7f389c06b8e0 2024-04-13T03:12:15.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.112+0000 7f3891ffb640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 0x7f388006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.112+0000 7f38a0d8b640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 0x7f388006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.112+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f38980adf20 con 0x7f389c06b8e0 2024-04-13T03:12:15.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.113+0000 7f38a0d8b640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 0x7f388006edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f388c006fd0 tx=0x7f388c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.117+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3898041070 con 0x7f389c06b8e0 2024-04-13T03:12:15.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.288+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f389c098d80 con 0x7f388006c8f0 2024-04-13T03:12:15.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.294+0000 7f3891ffb640 1 -- 172.21.15.106:0/1446458731 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7f389c098d80 con 0x7f388006c8f0 2024-04-13T03:12:15.297 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:15.298 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 msgr2=0x7f388006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 0x7f388006edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f388c006fd0 tx=0x7f388c008040 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 msgr2=0x7f389c098840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3898009890 tx=0x7f38980046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 shutdown_connections 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388006c8f0 0x7f388006edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f389c06cb10 0x7f389c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f389c0cdf50 0x7f389c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 --2- 172.21.15.106:0/1446458731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f389c06b8e0 0x7f389c098840 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 >> 172.21.15.106:0/1446458731 conn(0x7f389c0b1f10 msgr2=0x7f389c06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 shutdown_connections 2024-04-13T03:12:15.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.297+0000 7f38a37f0640 1 -- 172.21.15.106:0/1446458731 wait complete. 2024-04-13T03:12:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-13T03:12:14.397288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223256,"kb_used_data":7072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751464,"statfs":{"total":773094113280,"available":772865499136,"internally_reserved":0,"allocated":7241728,"data_stored":3898884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.427633"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005952+0000","last_change":"2024-04-13T03:12:12.001940+0000","last_active":"2024-04-13T03:12:12.005952+0000","last_peered":"2024-04-13T03:12:12.005952+0000","last_clean":"2024-04-13T03:12:12.005952+0000","last_became_active":"2024-04-13T03:12:12.001677+0000","last_became_peered":"2024-04-13T03:12:12.001677+0000","last_unstale":"2024-04-13T03:12:12.005952+0000","last_undegraded":"2024-04-13T03:12:12.005952+0000","last_fullsized":"2024-04-13T03:12:12.005952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:27.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.994441+0000","last_change":"2024-04-13T03:12:11.991806+0000","last_active":"2024-04-13T03:12:11.994441+0000","last_peered":"2024-04-13T03:12:11.994441+0000","last_clean":"2024-04-13T03:12:11.994441+0000","last_became_active":"2024-04-13T03:12:11.991513+0000","last_became_peered":"2024-04-13T03:12:11.991513+0000","last_unstale":"2024-04-13T03:12:11.994441+0000","last_undegraded":"2024-04-13T03:12:11.994441+0000","last_fullsized":"2024-04-13T03:12:11.994441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:26.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004190+0000","last_change":"2024-04-13T03:12:11.998338+0000","last_active":"2024-04-13T03:12:12.004190+0000","last_peered":"2024-04-13T03:12:12.004190+0000","last_clean":"2024-04-13T03:12:12.004190+0000","last_became_active":"2024-04-13T03:12:11.997576+0000","last_became_peered":"2024-04-13T03:12:11.997576+0000","last_unstale":"2024-04-13T03:12:12.004190+0000","last_undegraded":"2024-04-13T03:12:12.004190+0000","last_fullsized":"2024-04-13T03:12:12.004190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:27.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011176+0000","last_change":"2024-04-13T03:12:12.006998+0000","last_active":"2024-04-13T03:12:12.011176+0000","last_peered":"2024-04-13T03:12:12.011176+0000","last_clean":"2024-04-13T03:12:12.011176+0000","last_became_active":"2024-04-13T03:12:12.006714+0000","last_became_peered":"2024-04-13T03:12:12.006714+0000","last_unstale":"2024-04-13T03:12:12.011176+0000","last_undegraded":"2024-04-13T03:12:12.011176+0000","last_fullsized":"2024-04-13T03:12:12.011176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:26.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011726+0000","last_change":"2024-04-13T03:12:12.009748+0000","last_active":"2024-04-13T03:12:12.011726+0000","last_peered":"2024-04-13T03:12:12.011726+0000","last_clean":"2024-04-13T03:12:12.011726+0000","last_became_active":"2024-04-13T03:12:12.009463+0000","last_became_peered":"2024-04-13T03:12:12.009463+0000","last_unstale":"2024-04-13T03:12:12.011726+0000","last_undegraded":"2024-04-13T03:12:12.011726+0000","last_fullsized":"2024-04-13T03:12:12.011726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:24.027177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004384+0000","last_change":"2024-04-13T03:12:12.001174+0000","last_active":"2024-04-13T03:12:12.004384+0000","last_peered":"2024-04-13T03:12:12.004384+0000","last_clean":"2024-04-13T03:12:12.004384+0000","last_became_active":"2024-04-13T03:12:12.000775+0000","last_became_peered":"2024-04-13T03:12:12.000775+0000","last_unstale":"2024-04-13T03:12:12.004384+0000","last_undegraded":"2024-04-13T03:12:12.004384+0000","last_fullsized":"2024-04-13T03:12:12.004384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:16.254907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003873+0000","last_change":"2024-04-13T03:12:11.992846+0000","last_active":"2024-04-13T03:12:12.003873+0000","last_peered":"2024-04-13T03:12:12.003873+0000","last_clean":"2024-04-13T03:12:12.003873+0000","last_became_active":"2024-04-13T03:12:11.992577+0000","last_became_peered":"2024-04-13T03:12:11.992577+0000","last_unstale":"2024-04-13T03:12:12.003873+0000","last_undegraded":"2024-04-13T03:12:12.003873+0000","last_fullsized":"2024-04-13T03:12:12.003873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:58:23.732544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010635+0000","last_change":"2024-04-13T03:12:12.008622+0000","last_active":"2024-04-13T03:12:12.010635+0000","last_peered":"2024-04-13T03:12:12.010635+0000","last_clean":"2024-04-13T03:12:12.010635+0000","last_became_active":"2024-04-13T03:12:12.008368+0000","last_became_peered":"2024-04-13T03:12:12.008368+0000","last_unstale":"2024-04-13T03:12:12.010635+0000","last_undegraded":"2024-04-13T03:12:12.010635+0000","last_fullsized":"2024-04-13T03:12:12.010635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:23:47.278596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013192+0000","last_change":"2024-04-13T03:12:12.009241+0000","last_active":"2024-04-13T03:12:12.013192+0000","last_peered":"2024-04-13T03:12:12.013192+0000","last_clean":"2024-04-13T03:12:12.013192+0000","last_became_active":"2024-04-13T03:12:12.008972+0000","last_became_peered":"2024-04-13T03:12:12.008972+0000","last_unstale":"2024-04-13T03:12:12.013192+0000","last_undegraded":"2024-04-13T03:12:12.013192+0000","last_fullsized":"2024-04-13T03:12:12.013192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:35.022725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008340+0000","last_change":"2024-04-13T03:12:11.996130+0000","last_active":"2024-04-13T03:12:12.008340+0000","last_peered":"2024-04-13T03:12:12.008340+0000","last_clean":"2024-04-13T03:12:12.008340+0000","last_became_active":"2024-04-13T03:12:11.995855+0000","last_became_peered":"2024-04-13T03:12:11.995855+0000","last_unstale":"2024-04-13T03:12:12.008340+0000","last_undegraded":"2024-04-13T03:12:12.008340+0000","last_fullsized":"2024-04-13T03:12:12.008340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011295+0000","last_change":"2024-04-13T03:12:12.006798+0000","last_active":"2024-04-13T03:12:12.011295+0000","last_peered":"2024-04-13T03:12:12.011295+0000","last_clean":"2024-04-13T03:12:12.011295+0000","last_became_active":"2024-04-13T03:12:12.006503+0000","last_became_peered":"2024-04-13T03:12:12.006503+0000","last_unstale":"2024-04-13T03:12:12.011295+0000","last_undegraded":"2024-04-13T03:12:12.011295+0000","last_fullsized":"2024-04-13T03:12:12.011295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:18.495277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010439+0000","last_change":"2024-04-13T03:12:12.002285+0000","last_active":"2024-04-13T03:12:12.010439+0000","last_peered":"2024-04-13T03:12:12.010439+0000","last_clean":"2024-04-13T03:12:12.010439+0000","last_became_active":"2024-04-13T03:12:12.001971+0000","last_became_peered":"2024-04-13T03:12:12.001971+0000","last_unstale":"2024-04-13T03:12:12.010439+0000","last_undegraded":"2024-04-13T03:12:12.010439+0000","last_fullsized":"2024-04-13T03:12:12.010439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:27.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010613+0000","last_change":"2024-04-13T03:12:12.007375+0000","last_active":"2024-04-13T03:12:12.010613+0000","last_peered":"2024-04-13T03:12:12.010613+0000","last_clean":"2024-04-13T03:12:12.010613+0000","last_became_active":"2024-04-13T03:12:12.007106+0000","last_became_peered":"2024-04-13T03:12:12.007106+0000","last_unstale":"2024-04-13T03:12:12.010613+0000","last_undegraded":"2024-04-13T03:12:12.010613+0000","last_fullsized":"2024-04-13T03:12:12.010613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:33:27.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005135+0000","last_change":"2024-04-13T03:12:11.997334+0000","last_active":"2024-04-13T03:12:12.005135+0000","last_peered":"2024-04-13T03:12:12.005135+0000","last_clean":"2024-04-13T03:12:12.005135+0000","last_became_active":"2024-04-13T03:12:11.996805+0000","last_became_peered":"2024-04-13T03:12:11.996805+0000","last_unstale":"2024-04-13T03:12:12.005135+0000","last_undegraded":"2024-04-13T03:12:12.005135+0000","last_fullsized":"2024-04-13T03:12:12.005135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:08.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004250+0000","last_change":"2024-04-13T03:12:11.998049+0000","last_active":"2024-04-13T03:12:12.004250+0000","last_peered":"2024-04-13T03:12:12.004250+0000","last_clean":"2024-04-13T03:12:12.004250+0000","last_became_active":"2024-04-13T03:12:11.996306+0000","last_became_peered":"2024-04-13T03:12:11.996306+0000","last_unstale":"2024-04-13T03:12:12.004250+0000","last_undegraded":"2024-04-13T03:12:12.004250+0000","last_fullsized":"2024-04-13T03:12:12.004250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:21.412437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004364+0000","last_change":"2024-04-13T03:12:12.002454+0000","last_active":"2024-04-13T03:12:12.004364+0000","last_peered":"2024-04-13T03:12:12.004364+0000","last_clean":"2024-04-13T03:12:12.004364+0000","last_became_active":"2024-04-13T03:12:12.002112+0000","last_became_peered":"2024-04-13T03:12:12.002112+0000","last_unstale":"2024-04-13T03:12:12.004364+0000","last_undegraded":"2024-04-13T03:12:12.004364+0000","last_fullsized":"2024-04-13T03:12:12.004364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:09:17.601100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012076+0000","last_change":"2024-04-13T03:12:12.009855+0000","last_active":"2024-04-13T03:12:12.012076+0000","last_peered":"2024-04-13T03:12:12.012076+0000","last_clean":"2024-04-13T03:12:12.012076+0000","last_became_active":"2024-04-13T03:12:12.009081+0000","last_became_peered":"2024-04-13T03:12:12.009081+0000","last_unstale":"2024-04-13T03:12:12.012076+0000","last_undegraded":"2024-04-13T03:12:12.012076+0000","last_fullsized":"2024-04-13T03:12:12.012076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011360+0000","last_change":"2024-04-13T03:12:12.007849+0000","last_active":"2024-04-13T03:12:12.011360+0000","last_peered":"2024-04-13T03:12:12.011360+0000","last_clean":"2024-04-13T03:12:12.011360+0000","last_became_active":"2024-04-13T03:12:12.007576+0000","last_became_peered":"2024-04-13T03:12:12.007576+0000","last_unstale":"2024-04-13T03:12:12.011360+0000","last_undegraded":"2024-04-13T03:12:12.011360+0000","last_fullsized":"2024-04-13T03:12:12.011360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:00.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.001231+0000","last_change":"2024-04-13T03:12:11.996165+0000","last_active":"2024-04-13T03:12:12.001231+0000","last_peered":"2024-04-13T03:12:12.001231+0000","last_clean":"2024-04-13T03:12:12.001231+0000","last_became_active":"2024-04-13T03:12:11.995522+0000","last_became_peered":"2024-04-13T03:12:11.995522+0000","last_unstale":"2024-04-13T03:12:12.001231+0000","last_undegraded":"2024-04-13T03:12:12.001231+0000","last_fullsized":"2024-04-13T03:12:12.001231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:24:31.981955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.001423+0000","last_change":"2024-04-13T03:12:11.997051+0000","last_active":"2024-04-13T03:12:12.001423+0000","last_peered":"2024-04-13T03:12:12.001423+0000","last_clean":"2024-04-13T03:12:12.001423+0000","last_became_active":"2024-04-13T03:12:11.996669+0000","last_became_peered":"2024-04-13T03:12:11.996669+0000","last_unstale":"2024-04-13T03:12:12.001423+0000","last_undegraded":"2024-04-13T03:12:12.001423+0000","last_fullsized":"2024-04-13T03:12:12.001423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:39:21.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010399+0000","last_change":"2024-04-13T03:12:12.001644+0000","last_active":"2024-04-13T03:12:12.010399+0000","last_peered":"2024-04-13T03:12:12.010399+0000","last_clean":"2024-04-13T03:12:12.010399+0000","last_became_active":"2024-04-13T03:12:12.001292+0000","last_became_peered":"2024-04-13T03:12:12.001292+0000","last_unstale":"2024-04-13T03:12:12.010399+0000","last_undegraded":"2024-04-13T03:12:12.010399+0000","last_fullsized":"2024-04-13T03:12:12.010399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:15.732922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998645+0000","last_change":"2024-04-13T03:12:11.994436+0000","last_active":"2024-04-13T03:12:11.998645+0000","last_peered":"2024-04-13T03:12:11.998645+0000","last_clean":"2024-04-13T03:12:11.998645+0000","last_became_active":"2024-04-13T03:12:11.994175+0000","last_became_peered":"2024-04-13T03:12:11.994175+0000","last_unstale":"2024-04-13T03:12:11.998645+0000","last_undegraded":"2024-04-13T03:12:11.998645+0000","last_fullsized":"2024-04-13T03:12:11.998645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:32:21.931044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.002983+0000","last_change":"2024-04-13T03:12:11.998967+0000","last_active":"2024-04-13T03:12:12.002983+0000","last_peered":"2024-04-13T03:12:12.002983+0000","last_clean":"2024-04-13T03:12:12.002983+0000","last_became_active":"2024-04-13T03:12:11.998654+0000","last_became_peered":"2024-04-13T03:12:11.998654+0000","last_unstale":"2024-04-13T03:12:12.002983+0000","last_undegraded":"2024-04-13T03:12:12.002983+0000","last_fullsized":"2024-04-13T03:12:12.002983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:18.501133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.007543+0000","last_change":"2024-04-13T03:12:11.998431+0000","last_active":"2024-04-13T03:12:12.007543+0000","last_peered":"2024-04-13T03:12:12.007543+0000","last_clean":"2024-04-13T03:12:12.007543+0000","last_became_active":"2024-04-13T03:12:11.997557+0000","last_became_peered":"2024-04-13T03:12:11.997557+0000","last_unstale":"2024-04-13T03:12:12.007543+0000","last_undegraded":"2024-04-13T03:12:12.007543+0000","last_fullsized":"2024-04-13T03:12:12.007543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:23:47.278596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011289+0000","last_change":"2024-04-13T03:12:12.005617+0000","last_active":"2024-04-13T03:12:12.011289+0000","last_peered":"2024-04-13T03:12:12.011289+0000","last_clean":"2024-04-13T03:12:12.011289+0000","last_became_active":"2024-04-13T03:12:12.005347+0000","last_became_peered":"2024-04-13T03:12:12.005347+0000","last_unstale":"2024-04-13T03:12:12.011289+0000","last_undegraded":"2024-04-13T03:12:12.011289+0000","last_fullsized":"2024-04-13T03:12:12.011289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:08.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010657+0000","last_change":"2024-04-13T03:12:12.004702+0000","last_active":"2024-04-13T03:12:12.010657+0000","last_peered":"2024-04-13T03:12:12.010657+0000","last_clean":"2024-04-13T03:12:12.010657+0000","last_became_active":"2024-04-13T03:12:12.004442+0000","last_became_peered":"2024-04-13T03:12:12.004442+0000","last_unstale":"2024-04-13T03:12:12.010657+0000","last_undegraded":"2024-04-13T03:12:12.010657+0000","last_fullsized":"2024-04-13T03:12:12.010657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:13.760383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011112+0000","last_change":"2024-04-13T03:12:12.006023+0000","last_active":"2024-04-13T03:12:12.011112+0000","last_peered":"2024-04-13T03:12:12.011112+0000","last_clean":"2024-04-13T03:12:12.011112+0000","last_became_active":"2024-04-13T03:12:12.005764+0000","last_became_peered":"2024-04-13T03:12:12.005764+0000","last_unstale":"2024-04-13T03:12:12.011112+0000","last_undegraded":"2024-04-13T03:12:12.011112+0000","last_fullsized":"2024-04-13T03:12:12.011112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:22.037533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.996322+0000","last_change":"2024-04-13T03:12:11.992842+0000","last_active":"2024-04-13T03:12:11.996322+0000","last_peered":"2024-04-13T03:12:11.996322+0000","last_clean":"2024-04-13T03:12:11.996322+0000","last_became_active":"2024-04-13T03:12:11.992572+0000","last_became_peered":"2024-04-13T03:12:11.992572+0000","last_unstale":"2024-04-13T03:12:11.996322+0000","last_undegraded":"2024-04-13T03:12:11.996322+0000","last_fullsized":"2024-04-13T03:12:11.996322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:53:00.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013220+0000","last_change":"2024-04-13T03:12:12.009920+0000","last_active":"2024-04-13T03:12:12.013220+0000","last_peered":"2024-04-13T03:12:12.013220+0000","last_clean":"2024-04-13T03:12:12.013220+0000","last_became_active":"2024-04-13T03:12:12.009674+0000","last_became_peered":"2024-04-13T03:12:12.009674+0000","last_unstale":"2024-04-13T03:12:12.013220+0000","last_undegraded":"2024-04-13T03:12:12.013220+0000","last_fullsized":"2024-04-13T03:12:12.013220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:14:59.480738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010757+0000","last_change":"2024-04-13T03:12:12.002921+0000","last_active":"2024-04-13T03:12:12.010757+0000","last_peered":"2024-04-13T03:12:12.010757+0000","last_clean":"2024-04-13T03:12:12.010757+0000","last_became_active":"2024-04-13T03:12:12.002575+0000","last_became_peered":"2024-04-13T03:12:12.002575+0000","last_unstale":"2024-04-13T03:12:12.010757+0000","last_undegraded":"2024-04-13T03:12:12.010757+0000","last_fullsized":"2024-04-13T03:12:12.010757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:42.179303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011153+0000","last_change":"2024-04-13T03:12:12.006815+0000","last_active":"2024-04-13T03:12:12.011153+0000","last_peered":"2024-04-13T03:12:12.011153+0000","last_clean":"2024-04-13T03:12:12.011153+0000","last_became_active":"2024-04-13T03:12:12.006548+0000","last_became_peered":"2024-04-13T03:12:12.006548+0000","last_unstale":"2024-04-13T03:12:12.011153+0000","last_undegraded":"2024-04-13T03:12:12.011153+0000","last_fullsized":"2024-04-13T03:12:12.011153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:15.810689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004360+0000","last_change":"2024-04-13T03:12:11.995517+0000","last_active":"2024-04-13T03:12:12.004360+0000","last_peered":"2024-04-13T03:12:12.004360+0000","last_clean":"2024-04-13T03:12:12.004360+0000","last_became_active":"2024-04-13T03:12:11.995072+0000","last_became_peered":"2024-04-13T03:12:11.995072+0000","last_unstale":"2024-04-13T03:12:12.004360+0000","last_undegraded":"2024-04-13T03:12:12.004360+0000","last_fullsized":"2024-04-13T03:12:12.004360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:31:13.521898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.991730+0000","last_change":"2024-04-13T03:12:11.987194+0000","last_active":"2024-04-13T03:12:11.991730+0000","last_peered":"2024-04-13T03:12:11.991730+0000","last_clean":"2024-04-13T03:12:11.991730+0000","last_became_active":"2024-04-13T03:12:11.986906+0000","last_became_peered":"2024-04-13T03:12:11.986906+0000","last_unstale":"2024-04-13T03:12:11.991730+0000","last_undegraded":"2024-04-13T03:12:11.991730+0000","last_fullsized":"2024-04-13T03:12:11.991730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:45:12.346630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013480+0000","last_change":"2024-04-13T03:12:11.000289+0000","last_active":"2024-04-13T03:12:12.013480+0000","last_peered":"2024-04-13T03:12:12.013480+0000","last_clean":"2024-04-13T03:12:12.013480+0000","last_became_active":"2024-04-13T03:12:11.000006+0000","last_became_peered":"2024-04-13T03:12:11.000006+0000","last_unstale":"2024-04-13T03:12:12.013480+0000","last_undegraded":"2024-04-13T03:12:12.013480+0000","last_fullsized":"2024-04-13T03:12:12.013480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:14.808497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010230+0000","last_change":"2024-04-13T03:12:12.005172+0000","last_active":"2024-04-13T03:12:12.010230+0000","last_peered":"2024-04-13T03:12:12.010230+0000","last_clean":"2024-04-13T03:12:12.010230+0000","last_became_active":"2024-04-13T03:12:12.004846+0000","last_became_peered":"2024-04-13T03:12:12.004846+0000","last_unstale":"2024-04-13T03:12:12.010230+0000","last_undegraded":"2024-04-13T03:12:12.010230+0000","last_fullsized":"2024-04-13T03:12:12.010230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:49.988398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012012+0000","last_change":"2024-04-13T03:12:10.993982+0000","last_active":"2024-04-13T03:12:12.012012+0000","last_peered":"2024-04-13T03:12:12.012012+0000","last_clean":"2024-04-13T03:12:12.012012+0000","last_became_active":"2024-04-13T03:12:10.993614+0000","last_became_peered":"2024-04-13T03:12:10.993614+0000","last_unstale":"2024-04-13T03:12:12.012012+0000","last_undegraded":"2024-04-13T03:12:12.012012+0000","last_fullsized":"2024-04-13T03:12:12.012012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:25.547973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004442+0000","last_change":"2024-04-13T03:12:11.996133+0000","last_active":"2024-04-13T03:12:12.004442+0000","last_peered":"2024-04-13T03:12:12.004442+0000","last_clean":"2024-04-13T03:12:12.004442+0000","last_became_active":"2024-04-13T03:12:11.995842+0000","last_became_peered":"2024-04-13T03:12:11.995842+0000","last_unstale":"2024-04-13T03:12:12.004442+0000","last_undegraded":"2024-04-13T03:12:12.004442+0000","last_fullsized":"2024-04-13T03:12:12.004442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:06:01.242240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011125+0000","last_change":"2024-04-13T03:12:10.987486+0000","last_active":"2024-04-13T03:12:12.011125+0000","last_peered":"2024-04-13T03:12:12.011125+0000","last_clean":"2024-04-13T03:12:12.011125+0000","last_became_active":"2024-04-13T03:12:10.987220+0000","last_became_peered":"2024-04-13T03:12:10.987220+0000","last_unstale":"2024-04-13T03:12:12.011125+0000","last_undegraded":"2024-04-13T03:12:12.011125+0000","last_fullsized":"2024-04-13T03:12:12.011125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:20:36.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005367+0000","last_change":"2024-04-13T03:12:11.998185+0000","last_active":"2024-04-13T03:12:12.005367+0000","last_peered":"2024-04-13T03:12:12.005367+0000","last_clean":"2024-04-13T03:12:12.005367+0000","last_became_active":"2024-04-13T03:12:11.997593+0000","last_became_peered":"2024-04-13T03:12:11.997593+0000","last_unstale":"2024-04-13T03:12:12.005367+0000","last_undegraded":"2024-04-13T03:12:12.005367+0000","last_fullsized":"2024-04-13T03:12:12.005367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:28.460426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009418+0000","last_change":"2024-04-13T03:12:10.995495+0000","last_active":"2024-04-13T03:12:12.009418+0000","last_peered":"2024-04-13T03:12:12.009418+0000","last_clean":"2024-04-13T03:12:12.009418+0000","last_became_active":"2024-04-13T03:12:10.995206+0000","last_became_peered":"2024-04-13T03:12:10.995206+0000","last_unstale":"2024-04-13T03:12:12.009418+0000","last_undegraded":"2024-04-13T03:12:12.009418+0000","last_fullsized":"2024-04-13T03:12:12.009418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:11:55.728873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004343+0000","last_change":"2024-04-13T03:12:11.995447+0000","last_active":"2024-04-13T03:12:12.004343+0000","last_peered":"2024-04-13T03:12:12.004343+0000","last_clean":"2024-04-13T03:12:12.004343+0000","last_became_active":"2024-04-13T03:12:11.995192+0000","last_became_peered":"2024-04-13T03:12:11.995192+0000","last_unstale":"2024-04-13T03:12:12.004343+0000","last_undegraded":"2024-04-13T03:12:12.004343+0000","last_fullsized":"2024-04-13T03:12:12.004343+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:05.022231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006523+0000","last_change":"2024-04-13T03:12:10.995555+0000","last_active":"2024-04-13T03:12:12.006523+0000","last_peered":"2024-04-13T03:12:12.006523+0000","last_clean":"2024-04-13T03:12:12.006523+0000","last_became_active":"2024-04-13T03:12:10.995315+0000","last_became_peered":"2024-04-13T03:12:10.995315+0000","last_unstale":"2024-04-13T03:12:12.006523+0000","last_undegraded":"2024-04-13T03:12:12.006523+0000","last_fullsized":"2024-04-13T03:12:12.006523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:31:46.289265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009030+0000","last_change":"2024-04-13T03:12:12.007004+0000","last_active":"2024-04-13T03:12:12.009030+0000","last_peered":"2024-04-13T03:12:12.009030+0000","last_clean":"2024-04-13T03:12:12.009030+0000","last_became_active":"2024-04-13T03:12:12.006660+0000","last_became_peered":"2024-04-13T03:12:12.006660+0000","last_unstale":"2024-04-13T03:12:12.009030+0000","last_undegraded":"2024-04-13T03:12:12.009030+0000","last_fullsized":"2024-04-13T03:12:12.009030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:42.179303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006359+0000","last_change":"2024-04-13T03:12:10.991165+0000","last_active":"2024-04-13T03:12:12.006359+0000","last_peered":"2024-04-13T03:12:12.006359+0000","last_clean":"2024-04-13T03:12:12.006359+0000","last_became_active":"2024-04-13T03:12:10.990606+0000","last_became_peered":"2024-04-13T03:12:10.990606+0000","last_unstale":"2024-04-13T03:12:12.006359+0000","last_undegraded":"2024-04-13T03:12:12.006359+0000","last_fullsized":"2024-04-13T03:12:12.006359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:06:00.240048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998725+0000","last_change":"2024-04-13T03:12:11.994449+0000","last_active":"2024-04-13T03:12:11.998725+0000","last_peered":"2024-04-13T03:12:11.998725+0000","last_clean":"2024-04-13T03:12:11.998725+0000","last_became_active":"2024-04-13T03:12:11.993141+0000","last_became_peered":"2024-04-13T03:12:11.993141+0000","last_unstale":"2024-04-13T03:12:11.998725+0000","last_undegraded":"2024-04-13T03:12:11.998725+0000","last_fullsized":"2024-04-13T03:12:11.998725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:03:18.810085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987951+0000","last_change":"2024-04-13T03:12:10.992458+0000","last_active":"2024-04-13T03:12:11.987951+0000","last_peered":"2024-04-13T03:12:11.987951+0000","last_clean":"2024-04-13T03:12:11.987951+0000","last_became_active":"2024-04-13T03:12:10.992194+0000","last_became_peered":"2024-04-13T03:12:10.992194+0000","last_unstale":"2024-04-13T03:12:11.987951+0000","last_undegraded":"2024-04-13T03:12:11.987951+0000","last_fullsized":"2024-04-13T03:12:11.987951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:16.165557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008781+0000","last_change":"2024-04-13T03:12:12.007408+0000","last_active":"2024-04-13T03:12:12.008781+0000","last_peered":"2024-04-13T03:12:12.008781+0000","last_clean":"2024-04-13T03:12:12.008781+0000","last_became_active":"2024-04-13T03:12:12.007107+0000","last_became_peered":"2024-04-13T03:12:12.007107+0000","last_unstale":"2024-04-13T03:12:12.008781+0000","last_undegraded":"2024-04-13T03:12:12.008781+0000","last_fullsized":"2024-04-13T03:12:12.008781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:38:33.202471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982644+0000","last_change":"2024-04-13T03:12:11.000011+0000","last_active":"2024-04-13T03:12:11.982644+0000","last_peered":"2024-04-13T03:12:11.982644+0000","last_clean":"2024-04-13T03:12:11.982644+0000","last_became_active":"2024-04-13T03:12:10.999734+0000","last_became_peered":"2024-04-13T03:12:10.999734+0000","last_unstale":"2024-04-13T03:12:11.982644+0000","last_undegraded":"2024-04-13T03:12:11.982644+0000","last_fullsized":"2024-04-13T03:12:11.982644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:53:42.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003159+0000","last_change":"2024-04-13T03:12:11.994287+0000","last_active":"2024-04-13T03:12:12.003159+0000","last_peered":"2024-04-13T03:12:12.003159+0000","last_clean":"2024-04-13T03:12:12.003159+0000","last_became_active":"2024-04-13T03:12:11.994031+0000","last_became_peered":"2024-04-13T03:12:11.994031+0000","last_unstale":"2024-04-13T03:12:12.003159+0000","last_undegraded":"2024-04-13T03:12:12.003159+0000","last_fullsized":"2024-04-13T03:12:12.003159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:39.898125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989152+0000","last_change":"2024-04-13T03:12:11.001540+0000","last_active":"2024-04-13T03:12:11.989152+0000","last_peered":"2024-04-13T03:12:11.989152+0000","last_clean":"2024-04-13T03:12:11.989152+0000","last_became_active":"2024-04-13T03:12:11.001298+0000","last_became_peered":"2024-04-13T03:12:11.001298+0000","last_unstale":"2024-04-13T03:12:11.989152+0000","last_undegraded":"2024-04-13T03:12:11.989152+0000","last_fullsized":"2024-04-13T03:12:11.989152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:12.758191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.003295+0000","last_change":"2024-04-13T03:12:11.999580+0000","last_active":"2024-04-13T03:12:12.003295+0000","last_peered":"2024-04-13T03:12:12.003295+0000","last_clean":"2024-04-13T03:12:12.003295+0000","last_became_active":"2024-04-13T03:12:11.999325+0000","last_became_peered":"2024-04-13T03:12:11.999325+0000","last_unstale":"2024-04-13T03:12:12.003295+0000","last_undegraded":"2024-04-13T03:12:12.003295+0000","last_fullsized":"2024-04-13T03:12:12.003295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:43:24.721800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.994994+0000","last_change":"2024-04-13T03:12:10.996057+0000","last_active":"2024-04-13T03:12:11.994994+0000","last_peered":"2024-04-13T03:12:11.994994+0000","last_clean":"2024-04-13T03:12:11.994994+0000","last_became_active":"2024-04-13T03:12:10.995772+0000","last_became_peered":"2024-04-13T03:12:10.995772+0000","last_unstale":"2024-04-13T03:12:11.994994+0000","last_undegraded":"2024-04-13T03:12:11.994994+0000","last_fullsized":"2024-04-13T03:12:11.994994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:24:30.979763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.007423+0000","last_change":"2024-04-13T03:12:11.999989+0000","last_active":"2024-04-13T03:12:12.007423+0000","last_peered":"2024-04-13T03:12:12.007423+0000","last_clean":"2024-04-13T03:12:12.007423+0000","last_became_active":"2024-04-13T03:12:11.999638+0000","last_became_peered":"2024-04-13T03:12:11.999638+0000","last_unstale":"2024-04-13T03:12:12.007423+0000","last_undegraded":"2024-04-13T03:12:12.007423+0000","last_fullsized":"2024-04-13T03:12:12.007423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:02:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013212+0000","last_change":"2024-04-13T03:12:11.001477+0000","last_active":"2024-04-13T03:12:12.013212+0000","last_peered":"2024-04-13T03:12:12.013212+0000","last_clean":"2024-04-13T03:12:12.013212+0000","last_became_active":"2024-04-13T03:12:11.001262+0000","last_became_peered":"2024-04-13T03:12:11.001262+0000","last_unstale":"2024-04-13T03:12:12.013212+0000","last_undegraded":"2024-04-13T03:12:12.013212+0000","last_fullsized":"2024-04-13T03:12:12.013212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:26.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010057+0000","last_change":"2024-04-13T03:12:12.004097+0000","last_active":"2024-04-13T03:12:12.010057+0000","last_peered":"2024-04-13T03:12:12.010057+0000","last_clean":"2024-04-13T03:12:12.010057+0000","last_became_active":"2024-04-13T03:12:12.003694+0000","last_became_peered":"2024-04-13T03:12:12.003694+0000","last_unstale":"2024-04-13T03:12:12.010057+0000","last_undegraded":"2024-04-13T03:12:12.010057+0000","last_fullsized":"2024-04-13T03:12:12.010057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:24.027177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013867+0000","last_change":"2024-04-13T03:12:10.997926+0000","last_active":"2024-04-13T03:12:12.013867+0000","last_peered":"2024-04-13T03:12:12.013867+0000","last_clean":"2024-04-13T03:12:12.013867+0000","last_became_active":"2024-04-13T03:12:10.997507+0000","last_became_peered":"2024-04-13T03:12:10.997507+0000","last_unstale":"2024-04-13T03:12:12.013867+0000","last_undegraded":"2024-04-13T03:12:12.013867+0000","last_fullsized":"2024-04-13T03:12:12.013867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:21.035340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.006872+0000","last_change":"2024-04-13T03:12:11.997061+0000","last_active":"2024-04-13T03:12:12.006872+0000","last_peered":"2024-04-13T03:12:12.006872+0000","last_clean":"2024-04-13T03:12:12.006872+0000","last_became_active":"2024-04-13T03:12:11.996792+0000","last_became_peered":"2024-04-13T03:12:11.996792+0000","last_unstale":"2024-04-13T03:12:12.006872+0000","last_undegraded":"2024-04-13T03:12:12.006872+0000","last_fullsized":"2024-04-13T03:12:12.006872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:02:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013006+0000","last_change":"2024-04-13T03:12:11.000799+0000","last_active":"2024-04-13T03:12:12.013006+0000","last_peered":"2024-04-13T03:12:12.013006+0000","last_clean":"2024-04-13T03:12:12.013006+0000","last_became_active":"2024-04-13T03:12:11.000493+0000","last_became_peered":"2024-04-13T03:12:11.000493+0000","last_unstale":"2024-04-13T03:12:12.013006+0000","last_undegraded":"2024-04-13T03:12:12.013006+0000","last_fullsized":"2024-04-13T03:12:12.013006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:25.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010560+0000","last_change":"2024-04-13T03:12:12.004796+0000","last_active":"2024-04-13T03:12:12.010560+0000","last_peered":"2024-04-13T03:12:12.010560+0000","last_clean":"2024-04-13T03:12:12.010560+0000","last_became_active":"2024-04-13T03:12:12.004537+0000","last_became_peered":"2024-04-13T03:12:12.004537+0000","last_unstale":"2024-04-13T03:12:12.010560+0000","last_undegraded":"2024-04-13T03:12:12.010560+0000","last_fullsized":"2024-04-13T03:12:12.010560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:22.313658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992254+0000","last_change":"2024-04-13T03:12:10.993159+0000","last_active":"2024-04-13T03:12:11.992254+0000","last_peered":"2024-04-13T03:12:11.992254+0000","last_clean":"2024-04-13T03:12:11.992254+0000","last_became_active":"2024-04-13T03:12:10.992887+0000","last_became_peered":"2024-04-13T03:12:10.992887+0000","last_unstale":"2024-04-13T03:12:11.992254+0000","last_undegraded":"2024-04-13T03:12:11.992254+0000","last_fullsized":"2024-04-13T03:12:11.992254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:14.808497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.000995+0000","last_change":"2024-04-13T03:12:11.995506+0000","last_active":"2024-04-13T03:12:12.000995+0000","last_peered":"2024-04-13T03:12:12.000995+0000","last_clean":"2024-04-13T03:12:12.000995+0000","last_became_active":"2024-04-13T03:12:11.995228+0000","last_became_peered":"2024-04-13T03:12:11.995228+0000","last_unstale":"2024-04-13T03:12:12.000995+0000","last_undegraded":"2024-04-13T03:12:12.000995+0000","last_fullsized":"2024-04-13T03:12:12.000995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:04:57.272523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.988049+0000","last_change":"2024-04-13T03:12:11.002952+0000","last_active":"2024-04-13T03:12:11.988049+0000","last_peered":"2024-04-13T03:12:11.988049+0000","last_clean":"2024-04-13T03:12:11.988049+0000","last_became_active":"2024-04-13T03:12:11.002568+0000","last_became_peered":"2024-04-13T03:12:11.002568+0000","last_unstale":"2024-04-13T03:12:11.988049+0000","last_undegraded":"2024-04-13T03:12:11.988049+0000","last_fullsized":"2024-04-13T03:12:11.988049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:14.730730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.005858+0000","last_change":"2024-04-13T03:12:11.997983+0000","last_active":"2024-04-13T03:12:12.005858+0000","last_peered":"2024-04-13T03:12:12.005858+0000","last_clean":"2024-04-13T03:12:12.005858+0000","last_became_active":"2024-04-13T03:12:11.997191+0000","last_became_peered":"2024-04-13T03:12:11.997191+0000","last_unstale":"2024-04-13T03:12:12.005858+0000","last_undegraded":"2024-04-13T03:12:12.005858+0000","last_fullsized":"2024-04-13T03:12:12.005858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:15.732922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.015570+0000","last_change":"2024-04-13T03:12:10.992728+0000","last_active":"2024-04-13T03:12:12.015570+0000","last_peered":"2024-04-13T03:12:12.015570+0000","last_clean":"2024-04-13T03:12:12.015570+0000","last_became_active":"2024-04-13T03:12:10.992375+0000","last_became_peered":"2024-04-13T03:12:10.992375+0000","last_unstale":"2024-04-13T03:12:12.015570+0000","last_undegraded":"2024-04-13T03:12:12.015570+0000","last_fullsized":"2024-04-13T03:12:12.015570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:04:56.270330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.013252+0000","last_change":"2024-04-13T03:12:12.010553+0000","last_active":"2024-04-13T03:12:12.013252+0000","last_peered":"2024-04-13T03:12:12.013252+0000","last_clean":"2024-04-13T03:12:12.013252+0000","last_became_active":"2024-04-13T03:12:12.010281+0000","last_became_peered":"2024-04-13T03:12:12.010281+0000","last_unstale":"2024-04-13T03:12:12.013252+0000","last_undegraded":"2024-04-13T03:12:12.013252+0000","last_fullsized":"2024-04-13T03:12:12.013252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:22.863303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011508+0000","last_change":"2024-04-13T03:12:10.993260+0000","last_active":"2024-04-13T03:12:12.011508+0000","last_peered":"2024-04-13T03:12:12.011508+0000","last_clean":"2024-04-13T03:12:12.011508+0000","last_became_active":"2024-04-13T03:12:10.992975+0000","last_became_peered":"2024-04-13T03:12:10.992975+0000","last_unstale":"2024-04-13T03:12:12.011508+0000","last_undegraded":"2024-04-13T03:12:12.011508+0000","last_fullsized":"2024-04-13T03:12:12.011508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:52:59.322006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012039+0000","last_change":"2024-04-13T03:12:12.010144+0000","last_active":"2024-04-13T03:12:12.012039+0000","last_peered":"2024-04-13T03:12:12.012039+0000","last_clean":"2024-04-13T03:12:12.012039+0000","last_became_active":"2024-04-13T03:12:12.009884+0000","last_became_peered":"2024-04-13T03:12:12.009884+0000","last_unstale":"2024-04-13T03:12:12.012039+0000","last_undegraded":"2024-04-13T03:12:12.012039+0000","last_fullsized":"2024-04-13T03:12:12.012039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:49.988398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011615+0000","last_change":"2024-04-13T03:12:11.003031+0000","last_active":"2024-04-13T03:12:12.011615+0000","last_peered":"2024-04-13T03:12:12.011615+0000","last_clean":"2024-04-13T03:12:12.011615+0000","last_became_active":"2024-04-13T03:12:11.002743+0000","last_became_peered":"2024-04-13T03:12:11.002743+0000","last_unstale":"2024-04-13T03:12:12.011615+0000","last_undegraded":"2024-04-13T03:12:12.011615+0000","last_fullsized":"2024-04-13T03:12:12.011615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:52:59.322006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010996+0000","last_change":"2024-04-13T03:12:12.005588+0000","last_active":"2024-04-13T03:12:12.010996+0000","last_peered":"2024-04-13T03:12:12.010996+0000","last_clean":"2024-04-13T03:12:12.010996+0000","last_became_active":"2024-04-13T03:12:12.003720+0000","last_became_peered":"2024-04-13T03:12:12.003720+0000","last_unstale":"2024-04-13T03:12:12.010996+0000","last_undegraded":"2024-04-13T03:12:12.010996+0000","last_fullsized":"2024-04-13T03:12:12.010996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:57.941380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.012476+0000","last_change":"2024-04-13T03:12:11.002986+0000","last_active":"2024-04-13T03:12:12.012476+0000","last_peered":"2024-04-13T03:12:12.012476+0000","last_clean":"2024-04-13T03:12:12.012476+0000","last_became_active":"2024-04-13T03:12:11.002702+0000","last_became_peered":"2024-04-13T03:12:11.002702+0000","last_unstale":"2024-04-13T03:12:12.012476+0000","last_undegraded":"2024-04-13T03:12:12.012476+0000","last_fullsized":"2024-04-13T03:12:12.012476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:07:41.177111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998690+0000","last_change":"2024-04-13T03:12:11.994891+0000","last_active":"2024-04-13T03:12:11.998690+0000","last_peered":"2024-04-13T03:12:11.998690+0000","last_clean":"2024-04-13T03:12:11.998690+0000","last_became_active":"2024-04-13T03:12:11.994600+0000","last_became_peered":"2024-04-13T03:12:11.994600+0000","last_unstale":"2024-04-13T03:12:11.998690+0000","last_undegraded":"2024-04-13T03:12:11.998690+0000","last_fullsized":"2024-04-13T03:12:11.998690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:40:02.621231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987501+0000","last_change":"2024-04-13T03:12:10.988990+0000","last_active":"2024-04-13T03:12:11.987501+0000","last_peered":"2024-04-13T03:12:11.987501+0000","last_clean":"2024-04-13T03:12:11.987501+0000","last_became_active":"2024-04-13T03:12:10.988284+0000","last_became_peered":"2024-04-13T03:12:10.988284+0000","last_unstale":"2024-04-13T03:12:11.987501+0000","last_undegraded":"2024-04-13T03:12:11.987501+0000","last_fullsized":"2024-04-13T03:12:11.987501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:15.252715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004337+0000","last_change":"2024-04-13T03:12:11.995310+0000","last_active":"2024-04-13T03:12:12.004337+0000","last_peered":"2024-04-13T03:12:12.004337+0000","last_clean":"2024-04-13T03:12:12.004337+0000","last_became_active":"2024-04-13T03:12:11.994744+0000","last_became_peered":"2024-04-13T03:12:11.994744+0000","last_unstale":"2024-04-13T03:12:12.004337+0000","last_undegraded":"2024-04-13T03:12:12.004337+0000","last_fullsized":"2024-04-13T03:12:12.004337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:14:08.729277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.009625+0000","last_change":"2024-04-13T03:12:10.992458+0000","last_active":"2024-04-13T03:12:12.009625+0000","last_peered":"2024-04-13T03:12:12.009625+0000","last_clean":"2024-04-13T03:12:12.009625+0000","last_became_active":"2024-04-13T03:12:10.992194+0000","last_became_peered":"2024-04-13T03:12:10.992194+0000","last_unstale":"2024-04-13T03:12:12.009625+0000","last_undegraded":"2024-04-13T03:12:12.009625+0000","last_fullsized":"2024-04-13T03:12:12.009625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:49.172143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010582+0000","last_change":"2024-04-13T03:12:12.006578+0000","last_active":"2024-04-13T03:12:12.010582+0000","last_peered":"2024-04-13T03:12:12.010582+0000","last_clean":"2024-04-13T03:12:12.010582+0000","last_became_active":"2024-04-13T03:12:12.005562+0000","last_became_peered":"2024-04-13T03:12:12.005562+0000","last_unstale":"2024-04-13T03:12:12.010582+0000","last_undegraded":"2024-04-13T03:12:12.010582+0000","last_fullsized":"2024-04-13T03:12:12.010582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:28.460426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992424+0000","last_change":"2024-04-13T03:12:10.994192+0000","last_active":"2024-04-13T03:12:11.992424+0000","last_peered":"2024-04-13T03:12:11.992424+0000","last_clean":"2024-04-13T03:12:11.992424+0000","last_became_active":"2024-04-13T03:12:10.993933+0000","last_became_peered":"2024-04-13T03:12:10.993933+0000","last_unstale":"2024-04-13T03:12:11.992424+0000","last_undegraded":"2024-04-13T03:12:11.992424+0000","last_fullsized":"2024-04-13T03:12:11.992424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:25.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989231+0000","last_change":"2024-04-13T03:12:11.986810+0000","last_active":"2024-04-13T03:12:11.989231+0000","last_peered":"2024-04-13T03:12:11.989231+0000","last_clean":"2024-04-13T03:12:11.989231+0000","last_became_active":"2024-04-13T03:12:11.986475+0000","last_became_peered":"2024-04-13T03:12:11.986475+0000","last_unstale":"2024-04-13T03:12:11.989231+0000","last_undegraded":"2024-04-13T03:12:11.989231+0000","last_fullsized":"2024-04-13T03:12:11.989231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:46.554354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982158+0000","last_change":"2024-04-13T03:12:10.993807+0000","last_active":"2024-04-13T03:12:11.982158+0000","last_peered":"2024-04-13T03:12:11.982158+0000","last_clean":"2024-04-13T03:12:11.982158+0000","last_became_active":"2024-04-13T03:12:10.993495+0000","last_became_peered":"2024-04-13T03:12:10.993495+0000","last_unstale":"2024-04-13T03:12:11.982158+0000","last_undegraded":"2024-04-13T03:12:11.982158+0000","last_fullsized":"2024-04-13T03:12:11.982158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:39:07.695986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010188+0000","last_change":"2024-04-13T03:12:12.007995+0000","last_active":"2024-04-13T03:12:12.010188+0000","last_peered":"2024-04-13T03:12:12.010188+0000","last_clean":"2024-04-13T03:12:12.010188+0000","last_became_active":"2024-04-13T03:12:12.007743+0000","last_became_peered":"2024-04-13T03:12:12.007743+0000","last_unstale":"2024-04-13T03:12:12.010188+0000","last_undegraded":"2024-04-13T03:12:12.010188+0000","last_fullsized":"2024-04-13T03:12:12.010188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:34:18.362959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010350+0000","last_change":"2024-04-13T03:12:10.995086+0000","last_active":"2024-04-13T03:12:12.010350+0000","last_peered":"2024-04-13T03:12:12.010350+0000","last_clean":"2024-04-13T03:12:12.010350+0000","last_became_active":"2024-04-13T03:12:10.994824+0000","last_became_peered":"2024-04-13T03:12:10.994824+0000","last_unstale":"2024-04-13T03:12:12.010350+0000","last_undegraded":"2024-04-13T03:12:12.010350+0000","last_fullsized":"2024-04-13T03:12:12.010350+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:38.895933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004057+0000","last_change":"2024-04-13T03:12:11.994087+0000","last_active":"2024-04-13T03:12:12.004057+0000","last_peered":"2024-04-13T03:12:12.004057+0000","last_clean":"2024-04-13T03:12:12.004057+0000","last_became_active":"2024-04-13T03:12:11.993830+0000","last_became_peered":"2024-04-13T03:12:11.993830+0000","last_unstale":"2024-04-13T03:12:12.004057+0000","last_undegraded":"2024-04-13T03:12:12.004057+0000","last_fullsized":"2024-04-13T03:12:12.004057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:30:20.688194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.992023+0000","last_change":"2024-04-13T03:12:10.990141+0000","last_active":"2024-04-13T03:12:11.992023+0000","last_peered":"2024-04-13T03:12:11.992023+0000","last_clean":"2024-04-13T03:12:11.992023+0000","last_became_active":"2024-04-13T03:12:10.989849+0000","last_became_peered":"2024-04-13T03:12:10.989849+0000","last_unstale":"2024-04-13T03:12:11.992023+0000","last_undegraded":"2024-04-13T03:12:11.992023+0000","last_fullsized":"2024-04-13T03:12:11.992023+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:17.493085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.010672+0000","last_change":"2024-04-13T03:12:12.006012+0000","last_active":"2024-04-13T03:12:12.010672+0000","last_peered":"2024-04-13T03:12:12.010672+0000","last_clean":"2024-04-13T03:12:12.010672+0000","last_became_active":"2024-04-13T03:12:12.005733+0000","last_became_peered":"2024-04-13T03:12:12.005733+0000","last_unstale":"2024-04-13T03:12:12.010672+0000","last_undegraded":"2024-04-13T03:12:12.010672+0000","last_fullsized":"2024-04-13T03:12:12.010672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:21.412437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982585+0000","last_change":"2024-04-13T03:12:10.985663+0000","last_active":"2024-04-13T03:12:11.982585+0000","last_peered":"2024-04-13T03:12:11.982585+0000","last_clean":"2024-04-13T03:12:11.982585+0000","last_became_active":"2024-04-13T03:12:10.985357+0000","last_became_peered":"2024-04-13T03:12:10.985357+0000","last_unstale":"2024-04-13T03:12:11.982585+0000","last_undegraded":"2024-04-13T03:12:11.982585+0000","last_fullsized":"2024-04-13T03:12:11.982585+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:28.708314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.998409+0000","last_change":"2024-04-13T03:12:11.992737+0000","last_active":"2024-04-13T03:12:11.998409+0000","last_peered":"2024-04-13T03:12:11.998409+0000","last_clean":"2024-04-13T03:12:11.998409+0000","last_became_active":"2024-04-13T03:12:11.992460+0000","last_became_peered":"2024-04-13T03:12:11.992460+0000","last_unstale":"2024-04-13T03:12:11.998409+0000","last_undegraded":"2024-04-13T03:12:11.998409+0000","last_fullsized":"2024-04-13T03:12:11.998409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:33:26.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.982314+0000","last_change":"2024-04-13T03:12:10.997885+0000","last_active":"2024-04-13T03:12:11.982314+0000","last_peered":"2024-04-13T03:12:11.982314+0000","last_clean":"2024-04-13T03:12:11.982314+0000","last_became_active":"2024-04-13T03:12:10.997484+0000","last_became_peered":"2024-04-13T03:12:10.997484+0000","last_unstale":"2024-04-13T03:12:11.982314+0000","last_undegraded":"2024-04-13T03:12:11.982314+0000","last_fullsized":"2024-04-13T03:12:11.982314+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:10:17.493085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989205+0000","last_change":"2024-04-13T03:12:11.987614+0000","last_active":"2024-04-13T03:12:11.989205+0000","last_peered":"2024-04-13T03:12:11.989205+0000","last_clean":"2024-04-13T03:12:11.989205+0000","last_became_active":"2024-04-13T03:12:11.987314+0000","last_became_peered":"2024-04-13T03:12:11.987314+0000","last_unstale":"2024-04-13T03:12:11.989205+0000","last_undegraded":"2024-04-13T03:12:11.989205+0000","last_fullsized":"2024-04-13T03:12:11.989205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:50:53.185826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987979+0000","last_change":"2024-04-13T03:12:11.001033+0000","last_active":"2024-04-13T03:12:11.987979+0000","last_peered":"2024-04-13T03:12:11.987979+0000","last_clean":"2024-04-13T03:12:11.987979+0000","last_became_active":"2024-04-13T03:12:11.000722+0000","last_became_peered":"2024-04-13T03:12:11.000722+0000","last_unstale":"2024-04-13T03:12:11.987979+0000","last_undegraded":"2024-04-13T03:12:11.987979+0000","last_fullsized":"2024-04-13T03:12:11.987979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:38.895933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.011436+0000","last_change":"2024-04-13T03:12:12.009216+0000","last_active":"2024-04-13T03:12:12.011436+0000","last_peered":"2024-04-13T03:12:12.011436+0000","last_clean":"2024-04-13T03:12:12.011436+0000","last_became_active":"2024-04-13T03:12:12.008949+0000","last_became_peered":"2024-04-13T03:12:12.008949+0000","last_unstale":"2024-04-13T03:12:12.011436+0000","last_undegraded":"2024-04-13T03:12:12.011436+0000","last_fullsized":"2024-04-13T03:12:12.011436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:09:39.898125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987525+0000","last_change":"2024-04-13T03:12:10.995918+0000","last_active":"2024-04-13T03:12:11.987525+0000","last_peered":"2024-04-13T03:12:11.987525+0000","last_clean":"2024-04-13T03:12:11.987525+0000","last_became_active":"2024-04-13T03:12:10.994221+0000","last_became_peered":"2024-04-13T03:12:10.994221+0000","last_unstale":"2024-04-13T03:12:11.987525+0000","last_undegraded":"2024-04-13T03:12:11.987525+0000","last_fullsized":"2024-04-13T03:12:11.987525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:34:17.360767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.008952+0000","last_change":"2024-04-13T03:12:12.007385+0000","last_active":"2024-04-13T03:12:12.008952+0000","last_peered":"2024-04-13T03:12:12.008952+0000","last_clean":"2024-04-13T03:12:12.008952+0000","last_became_active":"2024-04-13T03:12:12.007112+0000","last_became_peered":"2024-04-13T03:12:12.007112+0000","last_unstale":"2024-04-13T03:12:12.008952+0000","last_undegraded":"2024-04-13T03:12:12.008952+0000","last_fullsized":"2024-04-13T03:12:12.008952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:23:13.760383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.988102+0000","last_change":"2024-04-13T03:12:10.990048+0000","last_active":"2024-04-13T03:12:11.988102+0000","last_peered":"2024-04-13T03:12:11.988102+0000","last_clean":"2024-04-13T03:12:11.988102+0000","last_became_active":"2024-04-13T03:12:10.989765+0000","last_became_peered":"2024-04-13T03:12:10.989765+0000","last_unstale":"2024-04-13T03:12:11.988102+0000","last_undegraded":"2024-04-13T03:12:11.988102+0000","last_fullsized":"2024-04-13T03:12:11.988102+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:53:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:12.004402+0000","last_change":"2024-04-13T03:12:12.000146+0000","last_active":"2024-04-13T03:12:12.004402+0000","last_peered":"2024-04-13T03:12:12.004402+0000","last_clean":"2024-04-13T03:12:12.004402+0000","last_became_active":"2024-04-13T03:12:11.999844+0000","last_became_peered":"2024-04-13T03:12:11.999844+0000","last_unstale":"2024-04-13T03:12:12.004402+0000","last_undegraded":"2024-04-13T03:12:12.004402+0000","last_fullsized":"2024-04-13T03:12:12.004402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:03:40.236913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.981444+0000","last_change":"2024-04-13T03:12:11.000775+0000","last_active":"2024-04-13T03:12:11.981444+0000","last_peered":"2024-04-13T03:12:11.981444+0000","last_clean":"2024-04-13T03:12:11.981444+0000","last_became_active":"2024-04-13T03:12:11.000448+0000","last_became_peered":"2024-04-13T03:12:11.000448+0000","last_unstale":"2024-04-13T03:12:11.981444+0000","last_undegraded":"2024-04-13T03:12:11.981444+0000","last_fullsized":"2024-04-13T03:12:11.981444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:56.939187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.989232+0000","last_change":"2024-04-13T03:12:11.987151+0000","last_active":"2024-04-13T03:12:11.989232+0000","last_peered":"2024-04-13T03:12:11.989232+0000","last_clean":"2024-04-13T03:12:11.989232+0000","last_became_active":"2024-04-13T03:12:11.986818+0000","last_became_peered":"2024-04-13T03:12:11.986818+0000","last_unstale":"2024-04-13T03:12:11.989232+0000","last_undegraded":"2024-04-13T03:12:11.989232+0000","last_fullsized":"2024-04-13T03:12:11.989232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:10.969627+0000","last_clean_scrub_stamp":"2024-04-13T03:12:10.969627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:56:11.584530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.986590+0000","last_change":"2024-04-13T03:12:10.995672+0000","last_active":"2024-04-13T03:12:11.986590+0000","last_peered":"2024-04-13T03:12:11.986590+0000","last_clean":"2024-04-13T03:12:11.986590+0000","last_became_active":"2024-04-13T03:12:10.995427+0000","last_became_peered":"2024-04-13T03:12:10.995427+0000","last_unstale":"2024-04-13T03:12:11.986590+0000","last_undegraded":"2024-04-13T03:12:11.986590+0000","last_fullsized":"2024-04-13T03:12:11.986590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:12:09.967435+0000","last_clean_scrub_stamp":"2024-04-13T03:12:09.967435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:31:12.519706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-13T03:12:11.987119+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:12:11.987119+0000","last_peered":"2024-04-13T03:12:11.987119+0000","last_clean":"2024-04-13T03:12:11.987119+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:12:11.987119+0000","last_undegraded":"2024-04-13T03:12:11.987119+0000","last_fullsized":"2024-04-13T03:12:11.987119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1343488,"data_stored":926546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":343379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":343367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":343345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":343345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1343488,"data_stored":926546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:12:15.312 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:12:15.312 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.456+0000 7f6c3211e640 1 Processor -- start 2024-04-13T03:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.456+0000 7f6c3211e640 1 -- start start 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c3211e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6c2c15c5f0 0x7f6c2c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c3211e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 0x7f6c2c160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c3211e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2c0d21d0 con 0x7f6c2c15c9d0 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c3211e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2c0cd8b0 con 0x7f6c2c15c5f0 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c3211e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c161330 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6c2c15c9d0 0x7f6c2c0ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47482/0 (socket says 172.21.15.106:47482) 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2bfff640 1 -- 172.21.15.106:0/1415826028 learned_addr learned my addr 172.21.15.106:0/1415826028 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1415826028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 0x7f6c2c160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2b7fe640 1 --2- 172.21.15.106:0/1415826028 >> v2:172.21.15.114:3300/0 conn(0x7f6c2c15c5f0 0x7f6c2c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2a7fc640 1 -- 172.21.15.106:0/1415826028 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244977691 0 0) 0x7f6c2c0d21d0 con 0x7f6c2c15c9d0 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2a7fc640 1 -- 172.21.15.106:0/1415826028 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c14003520 con 0x7f6c2c15c9d0 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2affd640 1 -- 172.21.15.106:0/1415826028 >> v2:172.21.15.114:3300/0 conn(0x7f6c2c15c5f0 msgr2=0x7f6c2c0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1415826028 >> v2:172.21.15.114:3300/0 conn(0x7f6c2c15c5f0 0x7f6c2c0c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2affd640 1 -- 172.21.15.106:0/1415826028 >> v1:172.21.15.106:6789/0 conn(0x7f6c2c15c9d0 legacy=0x7f6c2c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.457+0000 7f6c2affd640 1 -- 172.21.15.106:0/1415826028 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c0b1f40 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.458+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1415826028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 0x7f6c2c160df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c200097a0 tx=0x7f6c2002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=696de625368b2a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.458+0000 7f6c2a7fc640 1 -- 172.21.15.106:0/1415826028 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20037070 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.458+0000 7f6c2a7fc640 1 -- 172.21.15.106:0/1415826028 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c20030710 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.458+0000 7f6c2a7fc640 1 -- 172.21.15.106:0/1415826028 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20030ac0 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 -- 172.21.15.106:0/1415826028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 msgr2=0x7f6c2c160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1415826028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 0x7f6c2c160df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c200097a0 tx=0x7f6c2002b790 comp rx=0 tx=0).stop 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 -- 172.21.15.106:0/1415826028 shutdown_connections 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1415826028 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c0d1c30 0x7f6c2c160df0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1415826028 >> v2:172.21.15.114:3300/0 conn(0x7f6c2c15c5f0 0x7f6c2c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 -- 172.21.15.106:0/1415826028 >> 172.21.15.106:0/1415826028 conn(0x7f6c2c0cef80 msgr2=0x7f6c2c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 -- 172.21.15.106:0/1415826028 shutdown_connections 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.459+0000 7f6c3211e640 1 -- 172.21.15.106:0/1415826028 wait complete. 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.460+0000 7f6c3211e640 1 Processor -- start 2024-04-13T03:12:15.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.460+0000 7f6c3211e640 1 -- start start 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c15c5f0 0x7f6c2c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c2c15c9d0 0x7f6c2c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c0b2500 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c0b2870 con 0x7f6c2c15c9d0 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c3211e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c0cd260 con 0x7f6c2c15c5f0 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47982/0 (socket says 172.21.15.106:47982) 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 -- 172.21.15.106:0/1556870469 learned_addr learned my addr 172.21.15.106:0/1556870469 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c15c5f0 0x7f6c2c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2bfff640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c2c15c9d0 0x7f6c2c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 -- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c15c5f0 msgr2=0x7f6c2c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c15c5f0 0x7f6c2c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 -- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c2c15c9d0 msgr2=0x7f6c2c0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c2c15c9d0 0x7f6c2c0b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 -- 172.21.15.106:0/1556870469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c0c9b10 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.461+0000 7f6c2b7fe640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00e9b0 tx=0x7f6c1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.462+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c017070 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.462+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00c9a0 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.462+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c0b67e0 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.462+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00cd90 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.462+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c15e180 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.463+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c0ce340 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.463+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c1c01a070 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.463+0000 7f6c13fff640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 0x7f6c1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.464+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f6c1c08bf40 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.464+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 0x7f6c1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.464+0000 7f6c2affd640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 0x7f6c1806edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c20034000 tx=0x7f6c2002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.468+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c2c0ce340 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.711+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6c2c072900 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.713+0000 7f6c13fff640 1 -- 172.21.15.106:0/1556870469 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f6c1c059c20 con 0x7f6c2c0d1c30 2024-04-13T03:12:15.714 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:12:15.714 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:12:15.714 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:12:15.714 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:12:15.715 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:12:15.715 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:12:15.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 msgr2=0x7f6c1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 0x7f6c1806edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c20034000 tx=0x7f6c2002f000 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 msgr2=0x7f6c2c0c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00e9b0 tx=0x7f6c1c00ee70 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 shutdown_connections 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6c1806c8f0 0x7f6c1806edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6c2c15c5f0 0x7f6c2c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c2c15c9d0 0x7f6c2c0b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 --2- 172.21.15.106:0/1556870469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c2c0d1c30 0x7f6c2c0c95d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 >> 172.21.15.106:0/1556870469 conn(0x7f6c2c0cef80 msgr2=0x7f6c2c0c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 shutdown_connections 2024-04-13T03:12:15.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.715+0000 7f6c3211e640 1 -- 172.21.15.106:0/1556870469 wait complete. 2024-04-13T03:12:15.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:12:15.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.826+0000 7fe6659bb640 1 Processor -- start 2024-04-13T03:12:15.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.826+0000 7fe6659bb640 1 -- start start 2024-04-13T03:12:15.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.826+0000 7fe6659bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.826+0000 7fe6659bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 0x7fe6600d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe6659bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6600cfd20 con 0x7fe66015e9b0 2024-04-13T03:12:15.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe6659bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe660162af0 con 0x7fe66015ed90 2024-04-13T03:12:15.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe6659bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe660162dd0 con 0x7fe6600b3d60 2024-04-13T03:12:15.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65f7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 0x7fe6600d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65e7fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65effd640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe66015e9b0 0x7fe6600d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:48984/0 (socket says 172.21.15.106:48984) 2024-04-13T03:12:15.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65e7fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:49150/0 (socket says 172.21.15.106:49150) 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65effd640 1 -- 172.21.15.106:0/1437751698 learned_addr learned my addr 172.21.15.106:0/1437751698 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65f7fe640 1 -- 172.21.15.106:0/1437751698 >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 msgr2=0x7fe6600d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65f7fe640 1 --2- 172.21.15.106:0/1437751698 >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65f7fe640 1 -- 172.21.15.106:0/1437751698 >> v1:172.21.15.114:6789/0 conn(0x7fe66015e9b0 legacy=0x7fe6600d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65f7fe640 1 -- 172.21.15.106:0/1437751698 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601630b0 con 0x7fe6600b3d60 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.827+0000 7fe65e7fc640 1 --2- 172.21.15.106:0/1437751698 >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.828+0000 7fe65f7fe640 1 --2- 172.21.15.106:0/1437751698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 0x7fe6600d0200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9e869e5ab9d98d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.828+0000 7fe65dffb640 1 -- 172.21.15.106:0/1437751698 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658039070 con 0x7fe6600b3d60 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.828+0000 7fe65dffb640 1 -- 172.21.15.106:0/1437751698 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65802cce0 con 0x7fe6600b3d60 2024-04-13T03:12:15.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.828+0000 7fe65dffb640 1 -- 172.21.15.106:0/1437751698 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658034660 con 0x7fe6600b3d60 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 -- 172.21.15.106:0/1437751698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 msgr2=0x7fe6600d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 --2- 172.21.15.106:0/1437751698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 0x7fe6600d0200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65802bf30 comp rx=0 tx=0).stop 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 -- 172.21.15.106:0/1437751698 shutdown_connections 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 --2- 172.21.15.106:0/1437751698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600b3d60 0x7fe6600d0200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 --2- 172.21.15.106:0/1437751698 >> v2:172.21.15.114:3300/0 conn(0x7fe66015ed90 0x7fe6600d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 -- 172.21.15.106:0/1437751698 >> 172.21.15.106:0/1437751698 conn(0x7fe66008bf10 msgr2=0x7fe6600b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 -- 172.21.15.106:0/1437751698 shutdown_connections 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.829+0000 7fe6659bb640 1 -- 172.21.15.106:0/1437751698 wait complete. 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.830+0000 7fe6659bb640 1 Processor -- start 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.830+0000 7fe6659bb640 1 -- start start 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6600b3d60 0x7fe6600ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 0x7fe6600cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe660162800 con 0x7fe66015e9b0 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe660162680 con 0x7fe6600b3d60 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe6659bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d4d00 con 0x7fe6600cd1d0 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47998/0 (socket says 172.21.15.106:47998) 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65f7fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 0x7fe6600cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 -- 172.21.15.106:0/961413166 learned_addr learned my addr 172.21.15.106:0/961413166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65effd640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6600b3d60 0x7fe6600ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 -- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 msgr2=0x7fe6600cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 0x7fe6600cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 -- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6600b3d60 msgr2=0x7fe6600ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6600b3d60 0x7fe6600ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65e7fc640 1 -- 172.21.15.106:0/961413166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600c9df0 con 0x7fe66015e9b0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.831+0000 7fe65f7fe640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 0x7fe6600cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe65e7fc640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9b0 tx=0x7fe65400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654005990 con 0x7fe66015e9b0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe654005b30 con 0x7fe66015e9b0 2024-04-13T03:12:15.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6600ca0d0 con 0x7fe66015e9b0 2024-04-13T03:12:15.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400c9a0 con 0x7fe66015e9b0 2024-04-13T03:12:15.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.832+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600ca690 con 0x7fe66015e9b0 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.833+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6600d13c0 con 0x7fe66015e9b0 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.833+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe65400cb40 con 0x7fe66015e9b0 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.834+0000 7fe647fff640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 0x7fe64c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.834+0000 7fe65effd640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 0x7fe64c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.834+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fe65408bfb0 con 0x7fe66015e9b0 2024-04-13T03:12:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.834+0000 7fe65effd640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 0x7fe64c06edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe650007fd0 tx=0x7fe650007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:15.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:15.838+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6600d13c0 con 0x7fe66015e9b0 2024-04-13T03:12:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.052+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe660068a10 con 0x7fe66015e9b0 2024-04-13T03:12:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.053+0000 7fe647fff640 1 -- 172.21.15.106:0/961413166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5185 (secure 0 0 0) 0x7fe654017020 con 0x7fe66015e9b0 2024-04-13T03:12:16.054 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:16.054 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:13.343983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:12:16.054 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 28 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 msgr2=0x7fe64c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 0x7fe64c06edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe650007fd0 tx=0x7fe650007e80 comp rx=0 tx=0).stop 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 msgr2=0x7fe66006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9b0 tx=0x7fe65400ee70 comp rx=0 tx=0).stop 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 shutdown_connections 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe64c06c8f0 0x7fe64c06edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe6600cd1d0 0x7fe6600cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66015e9b0 0x7fe66006e1e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 --2- 172.21.15.106:0/961413166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6600b3d60 0x7fe6600ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 >> 172.21.15.106:0/961413166 conn(0x7fe66008bf10 msgr2=0x7fe6600d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 shutdown_connections 2024-04-13T03:12:16.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.055+0000 7fe6659bb640 1 -- 172.21.15.106:0/961413166 wait complete. 2024-04-13T03:12:16.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:12:11.338248+0000', 'modified': '2024-04-13T03:12:13.343983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6273': {'gid': 6273, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6836/536113082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 536113082}, {'type': 'v1', 'addr': '172.21.15.106:6836', 'nonce': 536113082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6273, 'qdb_cluster': [6273]} 2024-04-13T03:12:16.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:12:16.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-13T03:12:16.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.572+0000 7f1c580c8640 1 Processor -- start 2024-04-13T03:12:16.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.572+0000 7f1c580c8640 1 -- start start 2024-04-13T03:12:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.573+0000 7f1c580c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5015ed90 0x7f1c500d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.573+0000 7f1c580c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.573+0000 7f1c580c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c500cfd20 con 0x7f1c5015e9b0 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.573+0000 7f1c580c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1c50162af0 con 0x7f1c5015ed90 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.573+0000 7f1c580c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c50162dd0 con 0x7f1c500b3d60 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5015ed90 0x7f1c500d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c56665640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c56665640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:56514/0 (socket says 172.21.15.106:56514) 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5015ed90 0x7f1c500d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:49170/0 (socket says 172.21.15.106:49170) 2024-04-13T03:12:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c56665640 1 -- 172.21.15.106:0/3686658238 learned_addr learned my addr 172.21.15.106:0/3686658238 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 -- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 msgr2=0x7f1c500d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 -- 172.21.15.106:0/3686658238 >> v1:172.21.15.114:6789/0 conn(0x7f1c5015e9b0 legacy=0x7f1c500d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c55663640 1 -- 172.21.15.106:0/3686658238 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c501630b0 con 0x7f1c5015ed90 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.574+0000 7f1c56665640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.575+0000 7f1c55663640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015ed90 0x7f1c500d2300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009790 tx=0x7f1c4002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bd5463d4e997d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.575+0000 7f1c54e62640 1 -- 172.21.15.106:0/3686658238 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40035020 con 0x7f1c5015ed90 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.575+0000 7f1c54e62640 1 -- 172.21.15.106:0/3686658238 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c40031890 con 0x7f1c5015ed90 2024-04-13T03:12:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.575+0000 7f1c54e62640 1 -- 172.21.15.106:0/3686658238 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40031c60 con 0x7f1c5015ed90 2024-04-13T03:12:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 -- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015ed90 msgr2=0x7f1c500d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015ed90 0x7f1c500d2300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009790 tx=0x7f1c4002b930 comp rx=0 tx=0).stop 2024-04-13T03:12:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 -- 172.21.15.106:0/3686658238 shutdown_connections 2024-04-13T03:12:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b3d60 0x7f1c500d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 --2- 172.21.15.106:0/3686658238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015ed90 0x7f1c500d2300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 -- 172.21.15.106:0/3686658238 >> 172.21.15.106:0/3686658238 conn(0x7f1c5008bf10 msgr2=0x7f1c500b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 -- 172.21.15.106:0/3686658238 shutdown_connections 2024-04-13T03:12:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.576+0000 7f1c580c8640 1 -- 172.21.15.106:0/3686658238 wait complete. 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 Processor -- start 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 -- start start 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015e9b0 0x7f1c501ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b5c90 0x7f1c5015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50162800 con 0x7f1c500b3d60 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50162680 con 0x7f1c5015e9b0 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.577+0000 7f1c580c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c500d4d00 con 0x7f1c500b5c90 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48012/0 (socket says 172.21.15.106:48012) 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 -- 172.21.15.106:0/934810265 learned_addr learned my addr 172.21.15.106:0/934810265 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c56665640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b5c90 0x7f1c5015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55663640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015e9b0 0x7f1c501ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 -- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b5c90 msgr2=0x7f1c5015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b5c90 0x7f1c5015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 -- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015e9b0 msgr2=0x7f1c501ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015e9b0 0x7f1c501ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 -- 172.21.15.106:0/934810265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c500cd6e0 con 0x7f1c500b3d60 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c55e64640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c790 tx=0x7f1c4c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c467fc640 1 -- 172.21.15.106:0/934810265 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c016020 con 0x7f1c500b3d60 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c467fc640 1 -- 172.21.15.106:0/934810265 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4c006d50 con 0x7f1c500b3d60 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c500b6120 con 0x7f1c500b3d60 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c467fc640 1 -- 172.21.15.106:0/934810265 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c003bb0 con 0x7f1c500b3d60 2024-04-13T03:12:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.578+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501eb130 con 0x7f1c500b3d60 2024-04-13T03:12:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.580+0000 7f1c467fc640 1 -- 172.21.15.106:0/934810265 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c4c00a850 con 0x7f1c500b3d60 2024-04-13T03:12:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.580+0000 7f1c467fc640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 0x7f1c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.581+0000 7f1c467fc640 1 -- 172.21.15.106:0/934810265 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1c4c08b820 con 0x7f1c500b3d60 2024-04-13T03:12:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.581+0000 7f1c55663640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 0x7f1c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.581+0000 7f1c55663640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 0x7f1c3406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c40036000 tx=0x7f1c4003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.581+0000 7f1c5af85740 1 Processor -- start 2024-04-13T03:12:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.581+0000 7f1c5af85740 1 -- start start 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb522990 con 0x55f2eb5b82c0 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb4c9620 con 0x55f2eb5e51c0 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2eb51e950 con 0x55f2eb5e4de0 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48020/0 (socket says 172.21.15.106:48020) 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 -- 172.21.15.106:0/88818663 learned_addr learned my addr 172.21.15.106:0/88818663 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c55e64640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c55663640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 -- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 msgr2=0x55f2eb5b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 -- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 msgr2=0x55f2eb5e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c56665640 1 -- 172.21.15.106:0/88818663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2eb2352d0 con 0x55f2eb5b82c0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.582+0000 7f1c55663640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c56665640 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009bb0 tx=0x7f1c48011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a13b64f801379a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c337fe640 1 -- 172.21.15.106:0/88818663 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48004020 con 0x55f2eb5b82c0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c337fe640 1 -- 172.21.15.106:0/88818663 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c480041c0 con 0x55f2eb5b82c0 2024-04-13T03:12:16.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c337fe640 1 -- 172.21.15.106:0/88818663 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48003610 con 0x55f2eb5b82c0 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 -- 172.21.15.106:0/88818663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 msgr2=0x55f2eb5e80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009bb0 tx=0x7f1c48011690 comp rx=0 tx=0).stop 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 -- 172.21.15.106:0/88818663 shutdown_connections 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5b82c0 0x55f2eb5e80b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 --2- 172.21.15.106:0/88818663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 -- 172.21.15.106:0/88818663 >> 172.21.15.106:0/88818663 conn(0x55f2eb5b9ff0 msgr2=0x55f2eb602b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.583+0000 7f1c5af85740 1 -- 172.21.15.106:0/88818663 shutdown_connections 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.584+0000 7f1c5af85740 1 -- 172.21.15.106:0/88818663 wait complete. 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.584+0000 7f1c5af85740 1 -- start start 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.584+0000 7f1c5af85740 1 Processor -- start 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e9350 0x55f2eb5fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb5bc590 con 0x55f2eb5e4de0 2024-04-13T03:12:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb4dc9e0 con 0x55f2eb5e51c0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2eb5bb980 con 0x55f2eb5e9350 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c5af85740 10 client.0 _ll_register_callbacks cb 0x55f2eb518220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48024/0 (socket says 172.21.15.106:48024) 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 -- 172.21.15.106:0/1396246393 learned_addr learned my addr 172.21.15.106:0/1396246393 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55663640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c56665640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e9350 0x55f2eb5fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 -- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e9350 msgr2=0x55f2eb5fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e9350 0x55f2eb5fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 -- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 msgr2=0x55f2eb5e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.585+0000 7f1c55e64640 1 -- 172.21.15.106:0/1396246393 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2eb2ff6b0 con 0x55f2eb5e4de0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c55e64640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c055960 tx=0x7f1c4c05d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8c94cd44ca9a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:16.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c071020 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1c4c00ad40 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c480041c0 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c4c06de60 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 10 client.6402 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55f2eb5ea330 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c480041c0 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f1c4c058950 con 0x55f2eb5e4de0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 10 client.6402 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 10 client.6402 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 20 client.6402 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb616af0 0x55f2eb5d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 4 client.6402 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2eb5303b0 con 0x55f2eb616af0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.586+0000 7f1c55663640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb616af0 0x55f2eb5d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c30ff9640 10 client.6402 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c55663640 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb616af0 0x55f2eb5d29a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b3263eca5ca59d server_cookie=c09024c76c7fcc88 in_seq=0 out_seq=0 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c30ff9640 1 -- 172.21.15.106:0/1396246393 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2eb5303b0 con 0x55f2eb616af0 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c30ff9640 10 client.6402 handle_command_reply: tid=0 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 client.6402 shutdown 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 msgr2=0x55f2eb603590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c055960 tx=0x7f1c4c05d940 comp rx=0 tx=0).stop 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 shutdown_connections 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb616af0 0x55f2eb5d29a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e9350 0x55f2eb5fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c30ff9640 0 client.6402 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1396246393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e4de0 0x55f2eb603590 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 >> 172.21.15.106:0/1396246393 conn(0x55f2eb5b82c0 msgr2=0x55f2eb5e7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 shutdown_connections 2024-04-13T03:12:16.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.587+0000 7f1c5af85740 1 -- 172.21.15.106:0/1396246393 wait complete. 2024-04-13T03:12:16.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.588+0000 7f1c5af85740 20 client.6402 trim_cache size 0 max 16384 2024-04-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.605+0000 7f1c5af85740 1 Processor -- start 2024-04-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 -- start start 2024-04-13T03:12:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 0x55f2eb5fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50200660 con 0x55f2eb5e51c0 2024-04-13T03:12:16.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c500cd3f0 con 0x55f2eb5e9350 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1c501e9390 con 0x55f2eb5e4de0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c55663640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c55663640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48030/0 (socket says 172.21.15.106:48030) 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c55663640 1 -- 172.21.15.106:0/1470760705 learned_addr learned my addr 172.21.15.106:0/1470760705 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c55e64640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c56665640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 0x55f2eb5fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.606+0000 7f1c55663640 1 -- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 msgr2=0x55f2eb5beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c55663640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c55663640 1 -- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 msgr2=0x55f2eb5fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c55663640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 0x55f2eb5fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c55663640 1 -- 172.21.15.106:0/1470760705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2eb5b3d70 con 0x55f2eb5e51c0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c56665640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 0x55f2eb5fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c55663640 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1c400098b0 tx=0x7f1c40030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f2cd2d572f58e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c32ffd640 1 -- 172.21.15.106:0/1470760705 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40035040 con 0x55f2eb5e51c0 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c32ffd640 1 -- 172.21.15.106:0/1470760705 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4005a070 con 0x55f2eb5e51c0 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c32ffd640 1 -- 172.21.15.106:0/1470760705 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400543c0 con 0x55f2eb5e51c0 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 -- 172.21.15.106:0/1470760705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 msgr2=0x55f2eb5fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1c400098b0 tx=0x7f1c40030000 comp rx=0 tx=0).stop 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 -- 172.21.15.106:0/1470760705 shutdown_connections 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e9350 0x55f2eb5fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e51c0 0x55f2eb5fba20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 --2- 172.21.15.106:0/1470760705 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e4de0 0x55f2eb5beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 -- 172.21.15.106:0/1470760705 >> 172.21.15.106:0/1470760705 conn(0x55f2eb5b82c0 msgr2=0x55f2eb5e8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 -- 172.21.15.106:0/1470760705 shutdown_connections 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.607+0000 7f1c5af85740 1 -- 172.21.15.106:0/1470760705 wait complete. 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.608+0000 7f1c5af85740 1 -- start start 2024-04-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.608+0000 7f1c5af85740 1 Processor -- start 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 0x55f2eb625ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e51c0 0x55f2eb674cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb616970 con 0x55f2eb5e9350 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2eb5d3050 con 0x55f2eb5e4de0 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2eb616f30 con 0x55f2eb5e51c0 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c5af85740 10 client.0 _ll_register_callbacks cb 0x7f1c4c00cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48036/0 (socket says 172.21.15.106:48036) 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 -- 172.21.15.106:0/3854399658 learned_addr learned my addr 172.21.15.106:0/3854399658 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c55663640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e51c0 0x55f2eb674cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c55e64640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 0x55f2eb625ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 -- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e51c0 msgr2=0x55f2eb674cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e51c0 0x55f2eb674cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 -- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 msgr2=0x55f2eb625ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 0x55f2eb625ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 -- 172.21.15.106:0/3854399658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2eb6256f0 con 0x55f2eb5e9350 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c55e64640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 0x55f2eb625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c56665640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1c480090b0 tx=0x7f1c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65c922f9593587a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.609+0000 7f1c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480094c0 con 0x55f2eb5e9350 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1c48003810 con 0x55f2eb5e9350 2024-04-13T03:12:16.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4005a070 con 0x55f2eb5e9350 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c48003a10 con 0x55f2eb5e9350 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 10 client.6408 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55f2eb626a50 con 0x55f2eb5e9350 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4005a070 con 0x55f2eb5e9350 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f1c48003bb0 con 0x55f2eb5e9350 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 10 client.6408 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 10 client.6408 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 20 client.6408 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb615150 0x55f2eb615530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 4 client.6408 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "status"}) -- 0x55f2eb52a220 con 0x55f2eb615150 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c55e64640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb615150 0x55f2eb615530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.610+0000 7f1c30ff9640 10 client.6408 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c55e64640 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb615150 0x55f2eb615530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25a83d69fa447c3 server_cookie=323caea192e7d603 in_seq=0 out_seq=0 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c30ff9640 1 -- 172.21.15.106:0/3854399658 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f2eb52a220 con 0x55f2eb615150 2024-04-13T03:12:16.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c30ff9640 10 client.6408 handle_command_reply: tid=0 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 client.6408 shutdown 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 msgr2=0x55f2eb672580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1c480090b0 tx=0x7f1c48004730 comp rx=0 tx=0).stop 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 shutdown_connections 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55f2eb615150 0x55f2eb615530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2eb5e51c0 0x55f2eb674cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2eb5e9350 0x55f2eb672580 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 --2- 172.21.15.106:0/3854399658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2eb5e4de0 0x55f2eb625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 >> 172.21.15.106:0/3854399658 conn(0x55f2eb5b82c0 msgr2=0x55f2eb603230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c30ff9640 0 client.6408 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 shutdown_connections 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 1 -- 172.21.15.106:0/3854399658 wait complete. 2024-04-13T03:12:16.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.611+0000 7f1c5af85740 20 client.6408 trim_cache size 0 max 16384 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 6273, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.6326691359999996, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 28, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 64, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 64, 2024-04-13T03:12:16.619 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 9.9600090320000003 2024-04-13T03:12:16.620 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 msgr2=0x7f1c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 0x7f1c3406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c40036000 tx=0x7f1c4003c040 comp rx=0 tx=0).stop 2024-04-13T03:12:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 msgr2=0x7f1c5015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c790 tx=0x7f1c4c00cc50 comp rx=0 tx=0).stop 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 shutdown_connections 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1c3406c8f0 0x7f1c3406edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1c500b5c90 0x7f1c5015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5015e9b0 0x7f1c501ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 --2- 172.21.15.106:0/934810265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c500b3d60 0x7f1c5015dca0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 >> 172.21.15.106:0/934810265 conn(0x7f1c5008bf10 msgr2=0x7f1c5015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 shutdown_connections 2024-04-13T03:12:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.612+0000 7f1c580c8640 1 -- 172.21.15.106:0/934810265 wait complete. 2024-04-13T03:12:16.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:12:16.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.722+0000 7f08955dc640 1 Processor -- start 2024-04-13T03:12:16.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.722+0000 7f08955dc640 1 -- start start 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f08955dc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f08900b0b10 0x7f089015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f08955dc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f089015f260 0x7f089015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f08955dc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f089015d370 con 0x7f0890072e10 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f08955dc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f089015c510 con 0x7f089015f260 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f08955dc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f089015c7f0 con 0x7f08900b0b10 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f088e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f089015f260 0x7f089015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f088f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0890072e10 0x7f08900cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47488/0 (socket says 172.21.15.106:47488) 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f088f7fe640 1 -- 172.21.15.106:0/1093588586 learned_addr learned my addr 172.21.15.106:0/1093588586 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.723+0000 7f088effd640 1 --2- 172.21.15.106:0/1093588586 >> v2:172.21.15.114:3300/0 conn(0x7f08900b0b10 0x7f089015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088e7fc640 1 -- 172.21.15.106:0/1093588586 >> v2:172.21.15.114:3300/0 conn(0x7f08900b0b10 msgr2=0x7f089015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088e7fc640 1 --2- 172.21.15.106:0/1093588586 >> v2:172.21.15.114:3300/0 conn(0x7f08900b0b10 0x7f089015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088e7fc640 1 -- 172.21.15.106:0/1093588586 >> v1:172.21.15.106:6789/0 conn(0x7f0890072e10 legacy=0x7f08900cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088e7fc640 1 -- 172.21.15.106:0/1093588586 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089015cad0 con 0x7f089015f260 2024-04-13T03:12:16.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088dffb640 1 -- 172.21.15.106:0/1093588586 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560618849 0 0) 0x7f089015d370 con 0x7f0890072e10 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088e7fc640 1 --2- 172.21.15.106:0/1093588586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f089015f260 0x7f089015bc70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f08840097a0 tx=0x7f088402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1ed14299fe3754 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088dffb640 1 -- 172.21.15.106:0/1093588586 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884035020 con 0x7f089015f260 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088dffb640 1 -- 172.21.15.106:0/1093588586 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884031830 con 0x7f089015f260 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.724+0000 7f088dffb640 1 -- 172.21.15.106:0/1093588586 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884031c00 con 0x7f089015f260 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 -- 172.21.15.106:0/1093588586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f089015f260 msgr2=0x7f089015bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 --2- 172.21.15.106:0/1093588586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f089015f260 0x7f089015bc70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f08840097a0 tx=0x7f088402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 -- 172.21.15.106:0/1093588586 shutdown_connections 2024-04-13T03:12:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 --2- 172.21.15.106:0/1093588586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f089015f260 0x7f089015bc70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 --2- 172.21.15.106:0/1093588586 >> v2:172.21.15.114:3300/0 conn(0x7f08900b0b10 0x7f089015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 -- 172.21.15.106:0/1093588586 >> 172.21.15.106:0/1093588586 conn(0x7f089006f290 msgr2=0x7f089006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 -- 172.21.15.106:0/1093588586 shutdown_connections 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.725+0000 7f08955dc640 1 -- 172.21.15.106:0/1093588586 wait complete. 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.726+0000 7f08955dc640 1 Processor -- start 2024-04-13T03:12:16.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.726+0000 7f08955dc640 1 -- start start 2024-04-13T03:12:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f08900b0b10 0x7f089006c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 0x7f089006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f08900b0ef0 con 0x7f0890072e10 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f089015c1b0 con 0x7f089015f260 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f08955dc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f08900b1070 con 0x7f08900b0b10 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48062/0 (socket says 172.21.15.106:48062) 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 -- 172.21.15.106:0/1158054120 learned_addr learned my addr 172.21.15.106:0/1158054120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088e7fc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f08900b0b10 0x7f089006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088f7fe640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 0x7f089006c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 -- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f08900b0b10 msgr2=0x7f089006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f08900b0b10 0x7f089006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 -- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 msgr2=0x7f089006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 0x7f089006c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088effd640 1 -- 172.21.15.106:0/1158054120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089015a510 con 0x7f0890072e10 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.727+0000 7f088f7fe640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 0x7f089006c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f088effd640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0880007f20 tx=0x7f088000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880014620 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0880019070 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0890151000 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880014a00 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.728+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089015a7b0 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.729+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0890151520 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.729+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0880014ba0 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.729+0000 7f08777fe640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 0x7f087c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.730+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f088008c4c0 con 0x7f0890072e10 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.730+0000 7f088e7fc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 0x7f087c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.730+0000 7f088e7fc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 0x7f087c06ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0884036000 tx=0x7f088403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:16.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.734+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f088005a7f0 con 0x7f0890072e10 2024-04-13T03:12:16.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.944+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0890069120 con 0x7f0890072e10 2024-04-13T03:12:16.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.945+0000 7f08777fe640 1 -- 172.21.15.106:0/1158054120 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f088005a0f0 con 0x7f0890072e10 2024-04-13T03:12:16.946 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 msgr2=0x7f087c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 0x7f087c06ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0884036000 tx=0x7f088403c040 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 msgr2=0x7f0890138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0880007f20 tx=0x7f088000e570 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 shutdown_connections 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f087c06c920 0x7f087c06ede0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f08900b0b10 0x7f089006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f089015f260 0x7f089006c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 --2- 172.21.15.106:0/1158054120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0890072e10 0x7f0890138240 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 >> 172.21.15.106:0/1158054120 conn(0x7f089006f290 msgr2=0x7f08900b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 shutdown_connections 2024-04-13T03:12:16.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:16.947+0000 7f08955dc640 1 -- 172.21.15.106:0/1158054120 wait complete. 2024-04-13T03:12:16.958 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:12:16.958 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:12:16.959 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:12:16.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:16.959 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:12:16.974 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:12:16.974 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:12:16.974 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:12:16.974 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.974 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:12:16.975 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:12:16.976 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:12:16.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:16.977 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:12:17.043 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:12:17.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.045 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:12:17.100 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:12:17.100 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:12:17.100 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:12:17.101 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:12:17.101 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:12:17.101 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:12:17.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:12:17.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:12:17.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.235 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:12:17.250 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:12:17.250 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:12:17.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:12:17.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.250 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:12:17.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:12:17.318 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:12:17.373 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:12:17.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:12:17.374 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:12:17.428 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:12:17.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:12:17.513 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:12:17.513 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:12:17.520 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:12:17.520 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:12:17.520 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:12:17.520 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:12:17.520 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:12:17.521 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:12:17.521 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:12:17.521 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:12:17.521 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:12:17.521 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:12:17.523 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:12:17.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:12:17.556 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:12:17.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi106.stdout:client6420 2024-04-13T03:12:17.658 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:12:17.658 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:12:17.660 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:12:17.660+0000 7fbb993f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:12:17.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.673 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-13T03:12:17.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:17.733 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-13T03:12:17.789 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:12:17.894 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:12:17.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:12:17.903 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:12:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.043+0000 7ffb60436640 1 Processor -- start 2024-04-13T03:12:18.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.043+0000 7ffb60436640 1 -- start start 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb60436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb5815ed80 0x7ffb580cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb60436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb5815f420 con 0x7ffb58070c80 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb60436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb5815f700 con 0x7ffb5806e8e0 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb60436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ffb5806ecc0 con 0x7ffb5815ed80 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb5e9d3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffb58070c80 0x7ffb580ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47504/0 (socket says 172.21.15.106:47504) 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/4219375180 learned_addr learned my addr 172.21.15.106:0/4219375180 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.044+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330972713 0 0) 0x7ffb5815f420 con 0x7ffb58070c80 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb44003520 con 0x7ffb58070c80 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/4219375180 >> v2:172.21.15.114:3300/0 conn(0x7ffb5815ed80 0x7ffb580cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2717624896 0 0) 0x7ffb44003520 con 0x7ffb58070c80 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 >> v2:172.21.15.114:3300/0 conn(0x7ffb5815ed80 msgr2=0x7ffb580cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 --2- 172.21.15.106:0/4219375180 >> v2:172.21.15.114:3300/0 conn(0x7ffb5815ed80 0x7ffb580cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 >> v1:172.21.15.114:6789/0 conn(0x7ffb5806e8e0 legacy=0x7ffb580cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5815f9e0 con 0x7ffb58070c80 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7ffb50001670 con 0x7ffb58070c80 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 >> v1:172.21.15.106:6789/0 conn(0x7ffb58070c80 legacy=0x7ffb580ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb440039b0 0x7ffb44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb440051c0 0x7ffb4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4400c030 con 0x7ffb44004800 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4400c1e0 con 0x7ffb440051c0 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4400c3e0 con 0x7ffb440039b0 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5e1d2640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb440039b0 0x7ffb44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5e9d3640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb440051c0 0x7ffb4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 -- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb440039b0 msgr2=0x7ffb44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb440039b0 0x7ffb44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 -- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb440051c0 msgr2=0x7ffb4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb440051c0 0x7ffb4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/4219375180 reap_dead start 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.045+0000 7ffb5d9d1640 1 -- 172.21.15.106:0/4219375180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb44003520 con 0x7ffb44004800 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.046+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009ac0 tx=0x7ffb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97884d610d016804 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.046+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48039070 con 0x7ffb44004800 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.046+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4802c6b0 con 0x7ffb44004800 2024-04-13T03:12:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.046+0000 7ffb5d1d0640 1 -- 172.21.15.106:0/4219375180 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48009690 con 0x7ffb44004800 2024-04-13T03:12:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 -- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 msgr2=0x7ffb44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009ac0 tx=0x7ffb44005830 comp rx=0 tx=0).stop 2024-04-13T03:12:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 -- 172.21.15.106:0/4219375180 shutdown_connections 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 --2- 172.21.15.106:0/4219375180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 -- 172.21.15.106:0/4219375180 >> 172.21.15.106:0/4219375180 conn(0x7ffb5806f180 msgr2=0x7ffb5806f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 -- 172.21.15.106:0/4219375180 shutdown_connections 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.047+0000 7ffb60436640 1 -- 172.21.15.106:0/4219375180 wait complete. 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.048+0000 7ffb60436640 1 Processor -- start 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.048+0000 7ffb60436640 1 -- start start 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb44004800 0x7ffb58148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb5815c950 0x7ffb580b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580707f0 con 0x7ffb580b6780 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5815f1e0 con 0x7ffb5815c950 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ffb580cd770 con 0x7ffb44004800 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48086/0 (socket says 172.21.15.106:48086) 2024-04-13T03:12:18.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/2793411814 learned_addr learned my addr 172.21.15.106:0/2793411814 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e1d2640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb44004800 0x7ffb58148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5d9d1640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb5815c950 0x7ffb580b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb44004800 msgr2=0x7ffb58148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb44004800 0x7ffb58148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb5815c950 msgr2=0x7ffb580b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb5815c950 0x7ffb580b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 -- 172.21.15.106:0/2793411814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb58149490 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb5e9d3640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffb580cb720 tx=0x7ffb5000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50010450 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb500105f0 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb580b4c00 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500109e0 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.049+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb580b51c0 con 0x7ffb580b6780 2024-04-13T03:12:18.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.050+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb58068a10 con 0x7ffb580b6780 2024-04-13T03:12:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.051+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb5001b020 con 0x7ffb580b6780 2024-04-13T03:12:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.051+0000 7ffb4effd640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 0x7ffb3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.051+0000 7ffb5e1d2640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 0x7ffb3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.051+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7ffb5008ade0 con 0x7ffb580b6780 2024-04-13T03:12:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.052+0000 7ffb5e1d2640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 0x7ffb3c06ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffb54003fd0 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.056+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb58149490 con 0x7ffb580b6780 2024-04-13T03:12:18.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.268+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb580b7780 con 0x7ffb580b6780 2024-04-13T03:12:18.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.269+0000 7ffb4effd640 1 -- 172.21.15.106:0/2793411814 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5185 (secure 0 0 0) 0x7ffb50059080 con 0x7ffb580b6780 2024-04-13T03:12:18.270 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:18.270 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:13.343983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:12:18.270 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 28 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 msgr2=0x7ffb3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 0x7ffb3c06ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffb54003fd0 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 msgr2=0x7ffb580b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffb580cb720 tx=0x7ffb5000e890 comp rx=0 tx=0).stop 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 shutdown_connections 2024-04-13T03:12:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ffb3c06c920 0x7ffb3c06ede0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb580b6780 0x7ffb580b6c30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb5815c950 0x7ffb580b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 --2- 172.21.15.106:0/2793411814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ffb44004800 0x7ffb58148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 >> 172.21.15.106:0/2793411814 conn(0x7ffb5806f180 msgr2=0x7ffb580b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 shutdown_connections 2024-04-13T03:12:18.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.271+0000 7ffb60436640 1 -- 172.21.15.106:0/2793411814 wait complete. 2024-04-13T03:12:18.283 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-13T03:12:18.411 INFO:teuthology.orchestra.run.smithi106.stdout:{"message":"","return_code":0} 2024-04-13T03:12:18.411 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:12:18.412 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-13T03:12:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.452+0000 7f7691e42f40 1 Processor -- start 2024-04-13T03:12:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 -- start start 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc167743e0 0x55fc168b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc16747490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc166edd00 con 0x55fc16745340 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fc1678eb10 con 0x55fc1681b170 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f7691e42f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fc16771db0 con 0x55fc167743e0 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f768ee4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f768ee4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48098/0 (socket says 172.21.15.106:48098) 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.453+0000 7f768ee4b640 1 -- 172.21.15.106:0/2463942572 learned_addr learned my addr 172.21.15.106:0/2463942572 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 -- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc167743e0 msgr2=0x55fc168b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc167743e0 0x55fc168b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 -- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 msgr2=0x55fc16747490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc16747490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 -- 172.21.15.106:0/2463942572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc16770e80 con 0x55fc16745340 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768ee4b640 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7680058040 tx=0x7f7680051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c29036ae0a9165 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768d648640 1 -- 172.21.15.106:0/2463942572 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680068070 con 0x55fc16745340 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768d648640 1 -- 172.21.15.106:0/2463942572 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768004ed50 con 0x55fc16745340 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f768d648640 1 -- 172.21.15.106:0/2463942572 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680060650 con 0x55fc16745340 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 -- 172.21.15.106:0/2463942572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 msgr2=0x55fc168afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7680058040 tx=0x7f7680051000 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 -- 172.21.15.106:0/2463942572 shutdown_connections 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc167743e0 0x55fc168b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc16747490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 --2- 172.21.15.106:0/2463942572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc16745340 0x55fc168afcf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 -- 172.21.15.106:0/2463942572 >> 172.21.15.106:0/2463942572 conn(0x55fc16745f30 msgr2=0x55fc167724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 -- 172.21.15.106:0/2463942572 shutdown_connections 2024-04-13T03:12:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.454+0000 7f7691e42f40 1 -- 172.21.15.106:0/2463942572 wait complete. 2024-04-13T03:12:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.455+0000 7f7691e42f40 1 Processor -- start 2024-04-13T03:12:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.455+0000 7f7691e42f40 1 -- start start 2024-04-13T03:12:18.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc16745340 0x55fc167724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 0x55fc1680e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc168ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc16813320 con 0x55fc167743e0 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fc16813080 con 0x55fc1681b170 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f7691e42f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fc167704a0 con 0x55fc16745340 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 0x55fc1680e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 0x55fc1680e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48102/0 (socket says 172.21.15.106:48102) 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 -- 172.21.15.106:0/3022893226 learned_addr learned my addr 172.21.15.106:0/3022893226 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768ee4b640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc16745340 0x55fc167724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768f64c640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc168ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 -- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc16745340 msgr2=0x55fc167724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc16745340 0x55fc167724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 -- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 msgr2=0x55fc168ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc168ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 -- 172.21.15.106:0/3022893226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc16801010 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768f64c640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1681b170 0x55fc168ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768ee4b640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fc16745340 0x55fc167724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.456+0000 7f768e64a640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 0x55fc1680e520 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f768400bdb0 tx=0x7f7684009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.457+0000 7f7677fff640 1 -- 172.21.15.106:0/3022893226 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684008070 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.457+0000 7f7677fff640 1 -- 172.21.15.106:0/3022893226 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768400f650 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.457+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc16800d30 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.457+0000 7f7677fff640 1 -- 172.21.15.106:0/3022893226 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768400fa40 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.457+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc168ac880 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.458+0000 7f7677fff640 1 -- 172.21.15.106:0/3022893226 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f768400fbe0 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.458+0000 7f7677fff640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f767c06b2d0 0x7f767c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.458+0000 7f7677fff640 1 -- 172.21.15.106:0/3022893226 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f768408aa30 con 0x55fc167743e0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.458+0000 7f768ee4b640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f767c06b2d0 0x7f767c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.459+0000 7f7691e42f40 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55fc167ff0f0 0x55fc167ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.459+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) -- 0x55fc167ffa10 con 0x55fc167ff0f0 2024-04-13T03:12:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.459+0000 7f768ee4b640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f767c06b2d0 0x7f767c06d790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7680065000 tx=0x7f7680053000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.459+0000 7f768f64c640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55fc167ff0f0 0x55fc167ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.459+0000 7f768f64c640 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55fc167ff0f0 0x55fc167ff4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f768f64c640 1 -- 172.21.15.106:0/3022893226 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f768800c5c0 con 0x55fc167ff0f0 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55fc167ff0f0 msgr2=0x55fc167ff4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55fc167ff0f0 0x55fc167ff4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f767c06b2d0 msgr2=0x7f767c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f767c06b2d0 0x7f767c06d790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7680065000 tx=0x7f7680053000 comp rx=0 tx=0).stop 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 msgr2=0x55fc1680e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 --2- 172.21.15.106:0/3022893226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc167743e0 0x55fc1680e520 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f768400bdb0 tx=0x7f7684009560 comp rx=0 tx=0).stop 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f768f64c640 1 -- 172.21.15.106:0/3022893226 reap_dead start 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 shutdown_connections 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 >> 172.21.15.106:0/3022893226 conn(0x55fc16745f30 msgr2=0x55fc168185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 shutdown_connections 2024-04-13T03:12:18.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.460+0000 7f7691e42f40 1 -- 172.21.15.106:0/3022893226 wait complete. 2024-04-13T03:12:18.465 INFO:teuthology.orchestra.run.smithi106.stdout:Xfiledir_test 2024-04-13T03:12:18.466 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:18.466 DEBUG:teuthology.orchestra.run.smithi106:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: "ancestors": [ 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1099511627776, 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "file", 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 2 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:12:18.552 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: "dirino": 1, 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "dir_test", 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 4 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: "pool": 10, 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: "old_pools": [] 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:18.553 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:18.554 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-13T03:12:18.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.637+0000 7fe94663bf40 1 Processor -- start 2024-04-13T03:12:18.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.637+0000 7fe94663bf40 1 -- start start 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d025e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d025ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5584d025d1d0 con 0x5584d0215ec0 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5584d01bdd00 con 0x5584d02ea070 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94663bf40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5584d02e4bc0 con 0x5584d0217570 2024-04-13T03:12:18.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48116/0 (socket says 172.21.15.106:48116) 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 -- 172.21.15.106:0/2209531223 learned_addr learned my addr 172.21.15.106:0/2209531223 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe942e4a640 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d025e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe943e4c640 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d025ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 -- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 msgr2=0x5584d025e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d025e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 -- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 msgr2=0x5584d025ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d025ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.638+0000 7fe94364b640 1 -- 172.21.15.106:0/2209531223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d02e9340 con 0x5584d0215ec0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94364b640 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9340097e0 tx=0x7fe934005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f207b1cfe813c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe941e48640 1 -- 172.21.15.106:0/2209531223 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934008870 con 0x5584d0215ec0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe941e48640 1 -- 172.21.15.106:0/2209531223 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934018070 con 0x5584d0215ec0 2024-04-13T03:12:18.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe941e48640 1 -- 172.21.15.106:0/2209531223 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934008c50 con 0x5584d0215ec0 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 -- 172.21.15.106:0/2209531223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 msgr2=0x5584d0255b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9340097e0 tx=0x7fe934005ac0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 -- 172.21.15.106:0/2209531223 shutdown_connections 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d025e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d025ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 --2- 172.21.15.106:0/2209531223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0255b20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 -- 172.21.15.106:0/2209531223 >> 172.21.15.106:0/2209531223 conn(0x5584d0241360 msgr2=0x5584d02450e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 -- 172.21.15.106:0/2209531223 shutdown_connections 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.639+0000 7fe94663bf40 1 -- 172.21.15.106:0/2209531223 wait complete. 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.640+0000 7fe94663bf40 1 Processor -- start 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.640+0000 7fe94663bf40 1 -- start start 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0377580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d0255b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d0256060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5584d02e5630 con 0x5584d0215ec0 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5584d02e4e20 con 0x5584d02ea070 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5584d02e58d0 con 0x5584d0217570 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0377580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0377580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48124/0 (socket says 172.21.15.106:48124) 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 -- 172.21.15.106:0/688957814 learned_addr learned my addr 172.21.15.106:0/688957814 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe942e4a640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d0255b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe943e4c640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d0256060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 -- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 msgr2=0x5584d0255b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5584d0217570 0x5584d0255b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 -- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 msgr2=0x5584d0256060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5584d02ea070 0x5584d0256060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 -- 172.21.15.106:0/688957814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d0381f10 con 0x5584d0215ec0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94364b640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0377580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe9340057c0 tx=0x7fe934003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe940e46640 1 -- 172.21.15.106:0/688957814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934026670 con 0x5584d0215ec0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe940e46640 1 -- 172.21.15.106:0/688957814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934021070 con 0x5584d0215ec0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5584d02155e0 con 0x5584d0215ec0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe940e46640 1 -- 172.21.15.106:0/688957814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934026a70 con 0x5584d0215ec0 2024-04-13T03:12:18.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.641+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d0215ba0 con 0x5584d0215ec0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe940e46640 1 -- 172.21.15.106:0/688957814 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe934013420 con 0x5584d0215ec0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe940e46640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe93006b300 0x7fe93006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe940e46640 1 -- 172.21.15.106:0/688957814 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fe93408c790 con 0x5584d0215ec0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe942e4a640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe93006b300 0x7fe93006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe94663bf40 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5584d03817d0 0x5584d0382920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.643+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) -- 0x5584d0377ac0 con 0x5584d03817d0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.644+0000 7fe942e4a640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe93006b300 0x7fe93006d7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe938002fd0 tx=0x7fe938001810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.644+0000 7fe943e4c640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5584d03817d0 0x5584d0382920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.644+0000 7fe943e4c640 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5584d03817d0 0x5584d0382920 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe943e4c640 1 -- 172.21.15.106:0/688957814 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v64'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe93c05e040 con 0x5584d03817d0 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5584d03817d0 msgr2=0x5584d0382920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5584d03817d0 0x5584d0382920 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe93006b300 msgr2=0x7fe93006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe93006b300 0x7fe93006d7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe938002fd0 tx=0x7fe938001810 comp rx=0 tx=0).stop 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 msgr2=0x5584d0377580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 --2- 172.21.15.106:0/688957814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5584d0215ec0 0x5584d0377580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe9340057c0 tx=0x7fe934003020 comp rx=0 tx=0).stop 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe943e4c640 1 -- 172.21.15.106:0/688957814 reap_dead start 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 shutdown_connections 2024-04-13T03:12:18.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 >> 172.21.15.106:0/688957814 conn(0x5584d0241360 msgr2=0x5584d02e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 shutdown_connections 2024-04-13T03:12:18.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.645+0000 7fe94663bf40 1 -- 172.21.15.106:0/688957814 wait complete. 2024-04-13T03:12:18.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:18.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-13T03:12:18.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.794+0000 7fd9d14ad640 1 Processor -- start 2024-04-13T03:12:18.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.794+0000 7fd9d14ad640 1 -- start start 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.794+0000 7fd9d14ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0d3490 0x7fd9cc0d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.794+0000 7fd9d14ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9cc0d2370 0x7fd9cc0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9d14ad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9cc15c870 con 0x7fd9cc0b3920 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9d14ad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd9cc0d5420 con 0x7fd9cc0d2370 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9d14ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc15ee70 con 0x7fd9cc0d3490 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9ca7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9cc0d2370 0x7fd9cc0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9cb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd9cc0b3920 0x7fd9cc15f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47522/0 (socket says 172.21.15.106:47522) 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9cb7fe640 1 -- 172.21.15.106:0/1937077730 learned_addr learned my addr 172.21.15.106:0/1937077730 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9caffd640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0d3490 0x7fd9cc0d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9c9ffb640 1 -- 172.21.15.106:0/1937077730 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196767460 0 0) 0x7fd9cc15c870 con 0x7fd9cc0b3920 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9c9ffb640 1 -- 172.21.15.106:0/1937077730 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9b4003520 con 0x7fd9cc0b3920 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/1937077730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0d3490 msgr2=0x7fd9cc0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0d3490 0x7fd9cc0d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/1937077730 >> v1:172.21.15.106:6789/0 conn(0x7fd9cc0b3920 legacy=0x7fd9cc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.795+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/1937077730 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0d3db0 con 0x7fd9cc0d2370 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d2370 0x7fd9cc0b13f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009790 tx=0x7fd9c002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add3b3901417571e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9c9ffb640 1 -- 172.21.15.106:0/1937077730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0031770 con 0x7fd9cc0d2370 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9c9ffb640 1 -- 172.21.15.106:0/1937077730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c0037070 con 0x7fd9cc0d2370 2024-04-13T03:12:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9c9ffb640 1 -- 172.21.15.106:0/1937077730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0031b30 con 0x7fd9cc0d2370 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 -- 172.21.15.106:0/1937077730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d2370 msgr2=0x7fd9cc0b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d2370 0x7fd9cc0b13f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009790 tx=0x7fd9c002b810 comp rx=0 tx=0).stop 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 -- 172.21.15.106:0/1937077730 shutdown_connections 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d2370 0x7fd9cc0b13f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/1937077730 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0d3490 0x7fd9cc0d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 -- 172.21.15.106:0/1937077730 >> 172.21.15.106:0/1937077730 conn(0x7fd9cc0b9760 msgr2=0x7fd9cc0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.796+0000 7fd9d14ad640 1 -- 172.21.15.106:0/1937077730 shutdown_connections 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.797+0000 7fd9d14ad640 1 -- 172.21.15.106:0/1937077730 wait complete. 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 Processor -- start 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 -- start start 2024-04-13T03:12:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0b3920 0x7fd9cc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 0x7fd9cc160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0d2dc0 con 0x7fd9cc0d3490 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0d2ae0 con 0x7fd9cc0cd130 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.798+0000 7fd9d14ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc0d30a0 con 0x7fd9cc0b3920 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48140/0 (socket says 172.21.15.106:48140) 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/57246793 learned_addr learned my addr 172.21.15.106:0/57246793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9caffd640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0b3920 0x7fd9cc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 0x7fd9cc160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0b3920 msgr2=0x7fd9cc158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0b3920 0x7fd9cc158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 msgr2=0x7fd9cc160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 0x7fd9cc160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/57246793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc158d20 con 0x7fd9cc0d3490 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 0x7fd9cc160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0000c00 tx=0x7fd9c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9b37fe640 1 -- 172.21.15.106:0/57246793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0032600 con 0x7fd9cc0d3490 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9b37fe640 1 -- 172.21.15.106:0/57246793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c0031ec0 con 0x7fd9cc0d3490 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc06d670 con 0x7fd9cc0d3490 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9b37fe640 1 -- 172.21.15.106:0/57246793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00329c0 con 0x7fd9cc0d3490 2024-04-13T03:12:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.799+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc0cc930 con 0x7fd9cc0d3490 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.801+0000 7fd9b37fe640 1 -- 172.21.15.106:0/57246793 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9c004c060 con 0x7fd9cc0d3490 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.801+0000 7fd9b37fe640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 0x7fd9b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.801+0000 7fd9b37fe640 1 -- 172.21.15.106:0/57246793 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd9c00ad470 con 0x7fd9cc0d3490 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.801+0000 7fd9caffd640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 0x7fd9b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.802+0000 7fd9caffd640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 0x7fd9b806ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009b40 tx=0x7fd9bc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.802+0000 7fd9d436c740 1 Processor -- start 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.802+0000 7fd9d436c740 1 -- start start 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c9a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c6ac90 0x55a592c9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a592b93200 con 0x55a592c977b0 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a592bd4d60 con 0x55a592c6ac90 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a592b71a10 con 0x55a592c97b90 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48144/0 (socket says 172.21.15.106:48144) 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 -- 172.21.15.106:0/2388573147 learned_addr learned my addr 172.21.15.106:0/2388573147 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c6ac90 0x55a592c9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c9a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 -- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 msgr2=0x55a592c9a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c9a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 -- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c6ac90 msgr2=0x55a592c9aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c6ac90 0x55a592c9aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.803+0000 7fd9caffd640 1 -- 172.21.15.106:0/2388573147 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a592b9b930 con 0x55a592c977b0 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc011ee0 tx=0x7fd9bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9909bcc238a80be server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9b0ff9640 1 -- 172.21.15.106:0/2388573147 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc01b070 con 0x55a592c977b0 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9b0ff9640 1 -- 172.21.15.106:0/2388573147 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9bc027070 con 0x55a592c977b0 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9b0ff9640 1 -- 172.21.15.106:0/2388573147 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc003a40 con 0x55a592c977b0 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 -- 172.21.15.106:0/2388573147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 msgr2=0x55a592c6a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc011ee0 tx=0x7fd9bc014000 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 -- 172.21.15.106:0/2388573147 shutdown_connections 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c9a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c6ac90 0x55a592c9aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2388573147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c6a750 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.804+0000 7fd9d436c740 1 -- 172.21.15.106:0/2388573147 >> 172.21.15.106:0/2388573147 conn(0x55a592c6c9c0 msgr2=0x55a592cb5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.805+0000 7fd9d436c740 1 -- 172.21.15.106:0/2388573147 shutdown_connections 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.805+0000 7fd9d436c740 1 -- 172.21.15.106:0/2388573147 wait complete. 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.805+0000 7fd9d436c740 1 -- start start 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.805+0000 7fd9d436c740 1 Processor -- start 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a592b6f6e0 con 0x55a592c977b0 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a592bd0d20 con 0x55a592c9dc70 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a592c6ef60 con 0x55a592c97b90 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9d436c740 10 client.0 _ll_register_callbacks cb 0x55a592be1b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48146/0 (socket says 172.21.15.106:48146) 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 -- 172.21.15.106:0/2176863204 learned_addr learned my addr 172.21.15.106:0/2176863204 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 -- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 msgr2=0x55a592c808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 -- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 msgr2=0x55a592c80e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c80e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 -- 172.21.15.106:0/2176863204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5929b37b0 con 0x55a592c977b0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc024040 tx=0x7fd9bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ab54049b871ab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.806+0000 7fd9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc01b040 con 0x55a592c977b0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd9bc036c70 con 0x55a592c977b0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9bc027070 con 0x55a592c977b0 2024-04-13T03:12:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc03d050 con 0x55a592c977b0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 10 client.6459 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55a592a13f40 con 0x55a592c977b0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9bc027070 con 0x55a592c977b0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fd9bc040020 con 0x55a592c977b0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 10 client.6459 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 10 client.6459 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 20 client.6459 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592cd08c0 0x55a592cd2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 4 client.6459 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a592be8d10 con 0x55a592cd08c0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592cd08c0 0x55a592cd2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9a9ffb640 10 client.6459 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.807+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592cd08c0 0x55a592cd2cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db43a386fe5769b server_cookie=876e20ea80afd9bf in_seq=0 out_seq=0 2024-04-13T03:12:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2176863204 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a592be8d10 con 0x55a592cd08c0 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9a9ffb640 10 client.6459 handle_command_reply: tid=0 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 client.6459 shutdown 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 msgr2=0x55a592c93190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc024040 tx=0x7fd9bc004730 comp rx=0 tx=0).stop 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 shutdown_connections 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592cd08c0 0x55a592cd2cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c97b90 0x55a592c808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9a9ffb640 0 client.6459 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2176863204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c977b0 0x55a592c93190 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 >> 172.21.15.106:0/2176863204 conn(0x55a592c6ac90 msgr2=0x55a592c6e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 shutdown_connections 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 1 -- 172.21.15.106:0/2176863204 wait complete. 2024-04-13T03:12:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.808+0000 7fd9d436c740 20 client.6459 trim_cache size 0 max 16384 2024-04-13T03:12:18.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.826+0000 7fd9d436c740 1 Processor -- start 2024-04-13T03:12:18.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.826+0000 7fd9d436c740 1 -- start start 2024-04-13T03:12:18.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592cb7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a592ccf5c0 con 0x55a592c97b90 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc2011c0 con 0x55a592c9dc70 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc2001a0 con 0x55a592c977b0 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48156/0 (socket says 172.21.15.106:48156) 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2885563236 learned_addr learned my addr 172.21.15.106:0/2885563236 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592cb7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 msgr2=0x55a592cb7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592cb7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 msgr2=0x55a592c70480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c70480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2885563236 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5928a70a0 con 0x55a592c97b90 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592cb7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd9c002be20 tx=0x7fd9c004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cb08c7463000d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9abfff640 1 -- 172.21.15.106:0/2885563236 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0083c70 con 0x55a592c97b90 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9abfff640 1 -- 172.21.15.106:0/2885563236 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c0037040 con 0x55a592c97b90 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.827+0000 7fd9abfff640 1 -- 172.21.15.106:0/2885563236 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00ab880 con 0x55a592c97b90 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- 172.21.15.106:0/2885563236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 msgr2=0x55a592cb83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd9c002be20 tx=0x7fd9c004b000 comp rx=0 tx=0).stop 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- 172.21.15.106:0/2885563236 shutdown_connections 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c70480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592cb83e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2885563236 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592cb7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- 172.21.15.106:0/2885563236 >> 172.21.15.106:0/2885563236 conn(0x55a592c6ac90 msgr2=0x55a592cb8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- 172.21.15.106:0/2885563236 shutdown_connections 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- 172.21.15.106:0/2885563236 wait complete. 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 -- start start 2024-04-13T03:12:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.828+0000 7fd9d436c740 1 Processor -- start 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592d31b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c8f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1ffd70 con 0x55a592c97b90 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1f4e90 con 0x55a592c9dc70 2024-04-13T03:12:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a592ccf5c0 con 0x55a592c977b0 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9d436c740 10 client.0 _ll_register_callbacks cb 0x7fd9bc011de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48170/0 (socket says 172.21.15.106:48170) 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2926344719 learned_addr learned my addr 172.21.15.106:0/2926344719 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592d31b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.829+0000 7fd9cb7fe640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c8f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 msgr2=0x55a592d31b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592d31b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 msgr2=0x55a592c8f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c8f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 -- 172.21.15.106:0/2926344719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a592c9e760 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9ca7fc640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009790 tx=0x7fd9c00ab640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c91e6fe4dbf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00ab3c0 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd9c00a8070 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c0037040 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9c00ab7f0 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 10 client.6465 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x55a592d325b0 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c0037040 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7fd9c007f0b0 con 0x55a592c97b90 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 10 client.6465 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 10 client.6465 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:18.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 20 client.6465 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592d32170 0x55a592c703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 4 client.6465 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55a592c6bdd0 con 0x55a592d32170 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.830+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592d32170 0x55a592c703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9a9ffb640 10 client.6465 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9caffd640 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592d32170 0x55a592c703d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142ef8fb5efd018e server_cookie=faf0c0be3c3d061b in_seq=0 out_seq=0 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9a9ffb640 1 -- 172.21.15.106:0/2926344719 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a592c6bdd0 con 0x55a592d32170 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9a9ffb640 10 client.6465 handle_command_reply: tid=0 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 client.6465 shutdown 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 msgr2=0x55a592c8ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009790 tx=0x7fd9c00ab640 comp rx=0 tx=0).stop 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 shutdown_connections 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55a592d32170 0x55a592c703d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a592c9dc70 0x55a592c8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a592c97b90 0x55a592c8ed30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9a9ffb640 0 client.6465 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 --2- 172.21.15.106:0/2926344719 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a592c977b0 0x55a592d31b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.831+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 >> 172.21.15.106:0/2926344719 conn(0x55a592c6ac90 msgr2=0x55a592cb74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 shutdown_connections 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d436c740 1 -- 172.21.15.106:0/2926344719 wait complete. 2024-04-13T03:12:18.839 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fbb953e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6420=0-4194304@1} caps={6420=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6420 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55e633d34680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:12:18.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d436c740 20 client.6465 trim_cache size 0 max 16384 2024-04-13T03:12:18.840 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:18.840 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:12:18.840 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "ec4f527a-702a-4123-9094-9a468e0fa285", 2024-04-13T03:12:18.840 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:12:18.840 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 msgr2=0x7fd9b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 0x7fd9b806ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009b40 tx=0x7fd9bc009190 comp rx=0 tx=0).stop 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 msgr2=0x7fd9cc06cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0000c00 tx=0x7fd9c00046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 shutdown_connections 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd9b806c920 0x7fd9b806ede0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc0cd130 0x7fd9cc160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc0d3490 0x7fd9cc06cfe0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 --2- 172.21.15.106:0/57246793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9cc0b3920 0x7fd9cc158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 >> 172.21.15.106:0/57246793 conn(0x7fd9cc0b9760 msgr2=0x7fd9cc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 shutdown_connections 2024-04-13T03:12:18.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.832+0000 7fd9d14ad640 1 -- 172.21.15.106:0/57246793 wait complete. 2024-04-13T03:12:18.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:18.844 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-13T03:12:18.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.945+0000 7f38a5ba5640 1 Processor -- start 2024-04-13T03:12:18.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.945+0000 7f38a5ba5640 1 -- start start 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f38a5ba5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38a00cfea0 0x7f38a015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f38a5ba5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00d1c30 0x7f38a015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f38a5ba5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a015ddd0 con 0x7f38a00704a0 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f38a5ba5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f38a015e100 con 0x7f38a00cfea0 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f38a5ba5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38a015d4d0 con 0x7f38a00d1c30 2024-04-13T03:12:18.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f38a00704a0 0x7f38a00d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47532/0 (socket says 172.21.15.106:47532) 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389effd640 1 -- 172.21.15.106:0/3156774331 learned_addr learned my addr 172.21.15.106:0/3156774331 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389ffff640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00d1c30 0x7f38a015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3156774331 >> v2:172.21.15.114:3300/0 conn(0x7f38a00cfea0 0x7f38a015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409730437 0 0) 0x7f38a015ddd0 con 0x7f38a00704a0 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3888003520 con 0x7f38a00704a0 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 969479504 0 0) 0x7f3888003520 con 0x7f38a00704a0 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389f7fe640 1 -- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00d1c30 msgr2=0x7f38a015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00d1c30 0x7f38a015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389f7fe640 1 -- 172.21.15.106:0/3156774331 >> v1:172.21.15.106:6789/0 conn(0x7f38a00704a0 legacy=0x7f38a00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.946+0000 7f389f7fe640 1 -- 172.21.15.106:0/3156774331 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a015d7b0 con 0x7f38a00cfea0 2024-04-13T03:12:18.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.947+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00cfea0 0x7f38a015cc10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38a015e280 tx=0x7f389002b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af2cf140a277513 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.947+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890037070 con 0x7f38a00cfea0 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.947+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3890030510 con 0x7f38a00cfea0 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.947+0000 7f389e7fc640 1 -- 172.21.15.106:0/3156774331 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894001670 con 0x7f38a00cfea0 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 -- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00cfea0 msgr2=0x7f38a015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00cfea0 0x7f38a015cc10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38a015e280 tx=0x7f389002b6f0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 -- 172.21.15.106:0/3156774331 shutdown_connections 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00d1c30 0x7f38a015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/3156774331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00cfea0 0x7f38a015cc10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 -- 172.21.15.106:0/3156774331 >> 172.21.15.106:0/3156774331 conn(0x7f38a0071870 msgr2=0x7f38a0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 -- 172.21.15.106:0/3156774331 shutdown_connections 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.948+0000 7f38a5ba5640 1 -- 172.21.15.106:0/3156774331 wait complete. 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.949+0000 7f38a5ba5640 1 Processor -- start 2024-04-13T03:12:18.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.949+0000 7f38a5ba5640 1 -- start start 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00704a0 0x7f38a01e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00d1c30 0x7f38a00b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38a015ead0 con 0x7f38a00cfea0 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38a015d150 con 0x7f38a00d1c30 2024-04-13T03:12:18.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f38a5ba5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38a015ec50 con 0x7f38a00704a0 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48174/0 (socket says 172.21.15.106:48174) 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 -- 172.21.15.106:0/2444235047 learned_addr learned my addr 172.21.15.106:0/2444235047 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389ffff640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00d1c30 0x7f38a00b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389f7fe640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00704a0 0x7f38a01e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 -- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00704a0 msgr2=0x7f38a01e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00704a0 0x7f38a01e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 -- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00d1c30 msgr2=0x7f38a00b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00d1c30 0x7f38a00b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 -- 172.21.15.106:0/2444235047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01e5570 con 0x7f38a00cfea0 2024-04-13T03:12:18.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f389effd640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f389400c750 tx=0x7f389400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.950+0000 7f3887fff640 1 -- 172.21.15.106:0/2444235047 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894005ad0 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.951+0000 7f3887fff640 1 -- 172.21.15.106:0/2444235047 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3894005c90 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.951+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a01f5c80 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.951+0000 7f3887fff640 1 -- 172.21.15.106:0/2444235047 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940072b0 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.951+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01f6240 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.952+0000 7f3887fff640 1 -- 172.21.15.106:0/2444235047 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3894007450 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.952+0000 7f3887fff640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 0x7f388c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.952+0000 7f3887fff640 1 -- 172.21.15.106:0/2444235047 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f389408b930 con 0x7f38a00cfea0 2024-04-13T03:12:18.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.952+0000 7f389f7fe640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 0x7f388c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.953+0000 7f389f7fe640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 0x7f388c06edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3890034000 tx=0x7f3890008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.953+0000 7f38a898c740 1 Processor -- start 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.953+0000 7f38a898c740 1 -- start start 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x5564562ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562d04d0 0x5564563002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55645623b170 con 0x5564562fcff0 2024-04-13T03:12:18.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564561e17f0 con 0x5564562d04d0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556456237130 con 0x5564562fd3d0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48184/0 (socket says 172.21.15.106:48184) 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.954+0000 7f389f7fe640 1 -- 172.21.15.106:0/3336352766 learned_addr learned my addr 172.21.15.106:0/3336352766 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 -- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 msgr2=0x5564562ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389effd640 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x5564562ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x5564562ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 -- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562d04d0 msgr2=0x5564563002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562d04d0 0x5564563002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 -- 172.21.15.106:0/3336352766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564560177d0 con 0x5564562fcff0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3890009860 tx=0x7f389002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb7d546494a7fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f38857fa640 1 -- 172.21.15.106:0/3336352766 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890055070 con 0x5564562fcff0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f38857fa640 1 -- 172.21.15.106:0/3336352766 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3890030510 con 0x5564562fcff0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.955+0000 7f38857fa640 1 -- 172.21.15.106:0/3336352766 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900308c0 con 0x5564562fcff0 2024-04-13T03:12:18.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- 172.21.15.106:0/3336352766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 msgr2=0x5564562cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3890009860 tx=0x7f389002f000 comp rx=0 tx=0).stop 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- 172.21.15.106:0/3336352766 shutdown_connections 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x5564562ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562d04d0 0x5564563002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 --2- 172.21.15.106:0/3336352766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x5564562cff90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- 172.21.15.106:0/3336352766 >> 172.21.15.106:0/3336352766 conn(0x5564562d2200 msgr2=0x55645631ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- 172.21.15.106:0/3336352766 shutdown_connections 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- 172.21.15.106:0/3336352766 wait complete. 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 -- start start 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.956+0000 7f38a898c740 1 Processor -- start 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fd3d0 0x5564562fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5564562d48a0 con 0x5564562fcff0 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564561f4d00 con 0x5564562fd3d0 2024-04-13T03:12:18.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5564562d70e0 con 0x5564562f5d90 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f38a898c740 10 client.0 _ll_register_callbacks cb 0x5564561f46f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48198/0 (socket says 172.21.15.106:48198) 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 -- 172.21.15.106:0/3098613460 learned_addr learned my addr 172.21.15.106:0/3098613460 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389ffff640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389effd640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fd3d0 0x5564562fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 -- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 msgr2=0x5564562f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 -- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fd3d0 msgr2=0x5564562fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fd3d0 0x5564562fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.957+0000 7f389f7fe640 1 -- 172.21.15.106:0/3098613460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556455f4d3f0 con 0x5564562fcff0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3890000c00 tx=0x7f3890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87164cf2f5d4ed5a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389005d070 con 0x5564562fcff0 2024-04-13T03:12:18.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f3890030c50 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3890030510 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3890068050 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 10 client.6477 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x556455f0d0f0 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3890030510 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f389006b020 con 0x5564562fcff0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 10 client.6477 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 10 client.6477 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 20 client.6477 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632ea50 0x55645632eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 4 client.6477 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556456248b90 con 0x55645632ea50 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f389effd640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632ea50 0x55645632eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.958+0000 7f387e7fc640 10 client.6477 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f389effd640 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632ea50 0x55645632eeb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699e01df62a39db0 server_cookie=414fbdc6af2ab878 in_seq=0 out_seq=0 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f387e7fc640 1 -- 172.21.15.106:0/3098613460 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556456248b90 con 0x55645632ea50 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f387e7fc640 10 client.6477 handle_command_reply: tid=0 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 client.6477 shutdown 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 msgr2=0x55645631afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3890000c00 tx=0x7f3890004730 comp rx=0 tx=0).stop 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 shutdown_connections 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632ea50 0x55645632eeb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fd3d0 0x5564562fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 --2- 172.21.15.106:0/3098613460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fcff0 0x55645631afd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f387e7fc640 0 client.6477 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 >> 172.21.15.106:0/3098613460 conn(0x5564562d04d0 msgr2=0x5564562d40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 shutdown_connections 2024-04-13T03:12:18.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 1 -- 172.21.15.106:0/3098613460 wait complete. 2024-04-13T03:12:18.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.959+0000 7f38a898c740 20 client.6477 trim_cache size 0 max 16384 2024-04-13T03:12:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 Processor -- start 2024-04-13T03:12:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 -- start start 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x5564562cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01fcef0 con 0x5564562fd3d0 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38a006e5e0 con 0x5564562fcff0 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38a01f43c0 con 0x5564562f5d90 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.977+0000 7f389ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48212/0 (socket says 172.21.15.106:48212) 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 -- 172.21.15.106:0/2770159552 learned_addr learned my addr 172.21.15.106:0/2770159552 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 -- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 msgr2=0x5564562d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 -- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 msgr2=0x5564562cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389effd640 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x5564562cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x5564562cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 -- 172.21.15.106:0/2770159552 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645631aa00 con 0x5564562fd3d0 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f389ffff640 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3898004a80 tx=0x7f38980126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1099eec7976a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f3884ff9640 1 -- 172.21.15.106:0/2770159552 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801a070 con 0x5564562fd3d0 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f3884ff9640 1 -- 172.21.15.106:0/2770159552 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898016620 con 0x5564562fd3d0 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.978+0000 7f3884ff9640 1 -- 172.21.15.106:0/2770159552 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898016a10 con 0x5564562fd3d0 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- 172.21.15.106:0/2770159552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 msgr2=0x5564562cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3898004a80 tx=0x7f38980126d0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- 172.21.15.106:0/2770159552 shutdown_connections 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562fd3d0 0x5564562cf3c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x5564562cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 --2- 172.21.15.106:0/2770159552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562f5d90 0x5564562d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- 172.21.15.106:0/2770159552 >> 172.21.15.106:0/2770159552 conn(0x5564562d04d0 msgr2=0x556456302c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- 172.21.15.106:0/2770159552 shutdown_connections 2024-04-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- 172.21.15.106:0/2770159552 wait complete. 2024-04-13T03:12:18.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 -- start start 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.979+0000 7f38a898c740 1 Processor -- start 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x55645638dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x55645632d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5564562f5a60 con 0x5564562f5d90 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01f4a10 con 0x5564562fcff0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5564562d02f0 con 0x5564562fd3d0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f38a898c740 10 client.0 _ll_register_callbacks cb 0x7f3890009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48214/0 (socket says 172.21.15.106:48214) 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 -- 172.21.15.106:0/3240446231 learned_addr learned my addr 172.21.15.106:0/3240446231 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389effd640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x55645638dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389ffff640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x55645632d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 -- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 msgr2=0x55645632d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x55645632d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 -- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 msgr2=0x55645638dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.980+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x55645638dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f389f7fe640 1 -- 172.21.15.106:0/3240446231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564563354a0 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f389f7fe640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f389003a940 tx=0x7f38900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28512659ae158f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389005d040 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f3890030c50 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3898016620 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3890055040 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 10 client.6483 fetch_fsmap learned FSMap version 28 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x556456394fe0 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898016620 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7f389006b030 con 0x5564562f5d90 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 10 client.6483 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T03:12:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 10 client.6483 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 20 client.6483 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632dde0 0x556456393f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 4 client.6483 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556456242a00 con 0x55645632dde0 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.981+0000 7f389effd640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632dde0 0x556456393f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f387e7fc640 10 client.6483 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f389effd640 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632dde0 0x556456393f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20e54b4ad7f8d18 server_cookie=3bc79b7cec44a9c0 in_seq=0 out_seq=0 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f387e7fc640 1 -- 172.21.15.106:0/3240446231 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x556456242a00 con 0x55645632dde0 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f387e7fc640 10 client.6483 handle_command_reply: tid=0 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 client.6483 shutdown 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 msgr2=0x5564562f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f389003a940 tx=0x7f38900046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 shutdown_connections 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55645632dde0 0x556456393f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5564562fd3d0 0x55645632d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f387e7fc640 0 client.6483 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564562fcff0 0x55645638dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 --2- 172.21.15.106:0/3240446231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5564562f5d90 0x5564562f61c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 >> 172.21.15.106:0/3240446231 conn(0x5564562d04d0 msgr2=0x5564562cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 shutdown_connections 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 1 -- 172.21.15.106:0/3240446231 wait complete. 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.982+0000 7f38a898c740 20 client.6483 trim_cache size 0 max 16384 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout: "ec4f527a-702a-4123-9094-9a468e0fa285": { 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/", 2024-04-13T03:12:18.989 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "ec4f527a-702a-4123-9094-9a468e0fa285", 2024-04-13T03:12:18.990 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive" 2024-04-13T03:12:18.990 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:18.990 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:18.990 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 msgr2=0x7f388c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 0x7f388c06edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3890034000 tx=0x7f3890008030 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 msgr2=0x7f38a0139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f389400c750 tx=0x7f389400cc10 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 shutdown_connections 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f388c06c8f0 0x7f388c06edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38a00d1c30 0x7f38a00b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38a00cfea0 0x7f38a0139e40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 --2- 172.21.15.106:0/2444235047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38a00704a0 0x7f38a01e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 >> 172.21.15.106:0/2444235047 conn(0x7f38a0071870 msgr2=0x7f38a00b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 shutdown_connections 2024-04-13T03:12:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:18.983+0000 7f38a5ba5640 1 -- 172.21.15.106:0/2444235047 wait complete. 2024-04-13T03:12:18.995 INFO:tasks.cephfs.filesystem:scrub status for tag:ec4f527a-702a-4123-9094-9a468e0fa285 - {'path': '/', 'tag': 'ec4f527a-702a-4123-9094-9a468e0fa285', 'options': 'recursive'} 2024-04-13T03:12:24.038 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:12:24.037+0000 7f707b3f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-13T03:12:24.714 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:12:24.718 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:12:48.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:48.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-13T03:12:49.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.111+0000 7fec4b78c640 1 Processor -- start 2024-04-13T03:12:49.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.111+0000 7fec4b78c640 1 -- start start 2024-04-13T03:12:49.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec4b78c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec4415e8c0 0x7fec440ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec4b78c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec440b5060 con 0x7fec4415eca0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec4b78c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec440b5340 con 0x7fec44073520 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec4b78c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec4406c500 con 0x7fec4415e8c0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec48d27640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fec4415eca0 0x7fec44072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51672/0 (socket says 172.21.15.106:51672) 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec48d27640 1 -- 172.21.15.106:0/150934550 learned_addr learned my addr 172.21.15.106:0/150934550 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec49528640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec4415e8c0 0x7fec440ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2617971355 0 0) 0x7fec440b5060 con 0x7fec4415eca0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec30003520 con 0x7fec4415eca0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155608502 0 0) 0x7fec440b5340 con 0x7fec44073520 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.112+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec440b5060 con 0x7fec44073520 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2971736092 0 0) 0x7fec30003520 con 0x7fec4415eca0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec4415e8c0 msgr2=0x7fec440ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec4415e8c0 0x7fec440ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 >> v1:172.21.15.114:6789/0 conn(0x7fec44073520 legacy=0x7fec440b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4406c7e0 con 0x7fec4415eca0 2024-04-13T03:12:49.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fec34001670 con 0x7fec4415eca0 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 >> v1:172.21.15.106:6789/0 conn(0x7fec4415eca0 legacy=0x7fec44072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec30003b70 0x7fec30003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec30004800 0x7fec30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec300051c0 0x7fec3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec4406c500 con 0x7fec30004800 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec3000bf30 con 0x7fec300051c0 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec3000c130 con 0x7fec30003b70 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec49528640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec4415e8c0 0x7fec440ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec30004800 0x7fec30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec49d29640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec300051c0 0x7fec3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec49528640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec30003b70 0x7fec30003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 -- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec30003b70 msgr2=0x7fec30003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec30003b70 0x7fec30003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 -- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec300051c0 msgr2=0x7fec3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec300051c0 0x7fec3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.113+0000 7fec48d27640 1 -- 172.21.15.106:0/150934550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec30003520 con 0x7fec30004800 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec48d27640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec30004800 0x7fec30004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c210 tx=0x7fec30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2a71bcc679079d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34007ce0 con 0x7fec30004800 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34030070 con 0x7fec30004800 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec3bfff640 1 -- 172.21.15.106:0/150934550 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34005d90 con 0x7fec30004800 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec4b78c640 1 -- 172.21.15.106:0/150934550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec30004800 msgr2=0x7fec30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec4b78c640 1 --2- 172.21.15.106:0/150934550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec30004800 0x7fec30004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c210 tx=0x7fec30005830 comp rx=0 tx=0).stop 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.114+0000 7fec49d29640 1 -- 172.21.15.106:0/150934550 reap_dead start 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.115+0000 7fec4b78c640 1 -- 172.21.15.106:0/150934550 shutdown_connections 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.115+0000 7fec4b78c640 1 -- 172.21.15.106:0/150934550 >> 172.21.15.106:0/150934550 conn(0x7fec4406f0d0 msgr2=0x7fec4406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.115+0000 7fec4b78c640 1 -- 172.21.15.106:0/150934550 shutdown_connections 2024-04-13T03:12:49.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.115+0000 7fec4b78c640 1 -- 172.21.15.106:0/150934550 wait complete. 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.116+0000 7fec4b78c640 1 Processor -- start 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.116+0000 7fec4b78c640 1 -- start start 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 0x7fec44157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec441534a0 0x7fec44161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec440ce250 con 0x7fec44161550 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec440b4d50 con 0x7fec30004800 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec440ce3d0 con 0x7fec441534a0 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38080/0 (socket says 172.21.15.106:38080) 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 -- 172.21.15.106:0/3811370309 learned_addr learned my addr 172.21.15.106:0/3811370309 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec48d27640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec441534a0 0x7fec44161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 -- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec441534a0 msgr2=0x7fec44161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49528640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 0x7fec44157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec441534a0 0x7fec44161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 -- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 msgr2=0x7fec44157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 0x7fec44157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 -- 172.21.15.106:0/3811370309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44161a60 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49528640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 0x7fec44157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec49d29640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00fae0 tx=0x7fec3c0020d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec39ffb640 1 -- 172.21.15.106:0/3811370309 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c019070 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec39ffb640 1 -- 172.21.15.106:0/3811370309 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c006e20 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4415a7d0 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.117+0000 7fec39ffb640 1 -- 172.21.15.106:0/3811370309 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003bb0 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.118+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4415ad90 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.119+0000 7fec39ffb640 1 -- 172.21.15.106:0/3811370309 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec3c01c070 con 0x7fec44161550 2024-04-13T03:12:49.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.119+0000 7fec39ffb640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.119+0000 7fec39ffb640 1 -- 172.21.15.106:0/3811370309 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fec3c08af90 con 0x7fec44161550 2024-04-13T03:12:49.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.119+0000 7fec49528640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.120+0000 7fec49528640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 0x7fec2806edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec4406bb40 tx=0x7fec40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 Processor -- start 2024-04-13T03:12:49.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 -- start start 2024-04-13T03:12:49.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b6899f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b689a4d0 0x5555b68ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5555b6805170 con 0x5555b68c73d0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5555b67ab7f0 con 0x5555b68c6ff0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5555b6801130 con 0x5555b689a4d0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38082/0 (socket says 172.21.15.106:38082) 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.121+0000 7fec48d27640 1 -- 172.21.15.106:0/1160771752 learned_addr learned my addr 172.21.15.106:0/1160771752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec49d29640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b689a4d0 0x5555b68ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec49528640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b6899f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 -- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b689a4d0 msgr2=0x5555b68ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b689a4d0 0x5555b68ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 -- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 msgr2=0x5555b6899f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b6899f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 -- 172.21.15.106:0/1160771752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b65e17d0 con 0x5555b68c73d0 2024-04-13T03:12:49.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec49528640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b6899f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec48d27640 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec3402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54495963e5f35ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec2effd640 1 -- 172.21.15.106:0/1160771752 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3402c850 con 0x5555b68c73d0 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec2effd640 1 -- 172.21.15.106:0/1160771752 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34030040 con 0x5555b68c73d0 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.122+0000 7fec2effd640 1 -- 172.21.15.106:0/1160771752 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3402cbf0 con 0x5555b68c73d0 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- 172.21.15.106:0/1160771752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 msgr2=0x5555b68c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec3402e000 comp rx=0 tx=0).stop 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- 172.21.15.106:0/1160771752 shutdown_connections 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b689a4d0 0x5555b68ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68c9d80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 --2- 172.21.15.106:0/1160771752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b6899f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- 172.21.15.106:0/1160771752 >> 172.21.15.106:0/1160771752 conn(0x5555b689c200 msgr2=0x5555b68e4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- 172.21.15.106:0/1160771752 shutdown_connections 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- 172.21.15.106:0/1160771752 wait complete. 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 -- start start 2024-04-13T03:12:49.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.123+0000 7fec4e580740 1 Processor -- start 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b68c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5555b689e8a0 con 0x5555b68c73d0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5555b67bed00 con 0x5555b68c9810 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5555b68a10e0 con 0x5555b68c6ff0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec4e580740 10 client.0 _ll_register_callbacks cb 0x5555b67be6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38096/0 (socket says 172.21.15.106:38096) 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.124+0000 7fec48d27640 1 -- 172.21.15.106:0/2429279184 learned_addr learned my addr 172.21.15.106:0/2429279184 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec49528640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b68c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec49d29640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 -- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 msgr2=0x5555b68c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b68c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 -- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 msgr2=0x5555b68e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 -- 172.21.15.106:0/2429279184 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b65173f0 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec49d29640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec48d27640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fec3402c4c0 tx=0x7fec440b5230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24742ba5fc12b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3404f020 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fec340047e0 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec34030040 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec34059050 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 10 client.6495 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x5555b64d70f0 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34030040 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fec3405c020 con 0x5555b68c73d0 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 10 client.6495 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 10 client.6495 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:49.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 20 client.6495 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68cbab0 0x5555b68cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 4 client.6495 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.125+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b6812b90 con 0x5555b68cbab0 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec49528640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68cbab0 0x5555b68cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec23fff640 10 client.6495 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec49528640 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68cbab0 0x5555b68cbe90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03b5a48d05e3610 server_cookie=9ce8f2818f710c13 in_seq=0 out_seq=0 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec23fff640 1 -- 172.21.15.106:0/2429279184 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555b6812b90 con 0x5555b68cbab0 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec23fff640 10 client.6495 handle_command_reply: tid=0 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec4e580740 1 client.6495 shutdown 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 msgr2=0x5555b68bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.126+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fec3402c4c0 tx=0x7fec440b5230 comp rx=0 tx=0).stop 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 shutdown_connections 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68cbab0 0x5555b68cbe90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b68bf120 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 --2- 172.21.15.106:0/2429279184 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b68c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec23fff640 0 client.6495 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 >> 172.21.15.106:0/2429279184 conn(0x5555b689a4d0 msgr2=0x5555b689e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 shutdown_connections 2024-04-13T03:12:49.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 1 -- 172.21.15.106:0/2429279184 wait complete. 2024-04-13T03:12:49.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.127+0000 7fec4e580740 20 client.6495 trim_cache size 0 max 16384 2024-04-13T03:12:49.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.144+0000 7fec4e580740 1 Processor -- start 2024-04-13T03:12:49.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.144+0000 7fec4e580740 1 -- start start 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b68c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c73d0 0x5555b68f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec44201d40 con 0x5555b68c9810 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec44200d60 con 0x5555b68c6ff0 2024-04-13T03:12:49.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec442000c0 con 0x5555b68c73d0 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38106/0 (socket says 172.21.15.106:38106) 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 -- 172.21.15.106:0/36070778 learned_addr learned my addr 172.21.15.106:0/36070778 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49528640 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b68c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec48d27640 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c73d0 0x5555b68f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 -- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c73d0 msgr2=0x5555b68f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c73d0 0x5555b68f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 -- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 msgr2=0x5555b68c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b68c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 -- 172.21.15.106:0/36070778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b68e4fd0 con 0x5555b68c9810 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec49d29640 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00d520 tx=0x7fec3c089200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a5b5f8780f2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec2e7fc640 1 -- 172.21.15.106:0/36070778 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c056e30 con 0x5555b68c9810 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec2e7fc640 1 -- 172.21.15.106:0/36070778 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c0551a0 con 0x5555b68c9810 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.145+0000 7fec2e7fc640 1 -- 172.21.15.106:0/36070778 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c058180 con 0x5555b68c9810 2024-04-13T03:12:49.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- 172.21.15.106:0/36070778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 msgr2=0x5555b68f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00d520 tx=0x7fec3c089200 comp rx=0 tx=0).stop 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- 172.21.15.106:0/36070778 shutdown_connections 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c73d0 0x5555b68f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c9810 0x5555b68f4b60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 --2- 172.21.15.106:0/36070778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c6ff0 0x5555b68c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- 172.21.15.106:0/36070778 >> 172.21.15.106:0/36070778 conn(0x5555b689a4d0 msgr2=0x5555b68cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- 172.21.15.106:0/36070778 shutdown_connections 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- 172.21.15.106:0/36070778 wait complete. 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.146+0000 7fec4e580740 1 -- start start 2024-04-13T03:12:49.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 Processor -- start 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b689ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec44200710 con 0x5555b68c73d0 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5555b68e44d0 con 0x5555b68c9810 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.147+0000 7fec4e580740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5555b6906c90 con 0x5555b68c6ff0 2024-04-13T03:12:49.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec4e580740 10 client.0 _ll_register_callbacks cb 0x7fec30003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38112/0 (socket says 172.21.15.106:38112) 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 -- 172.21.15.106:0/2264497691 learned_addr learned my addr 172.21.15.106:0/2264497691 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec49528640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b689ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec49d29640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 -- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 msgr2=0x5555b689ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b689ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 -- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 msgr2=0x5555b68cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 -- 172.21.15.106:0/2264497691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b689b580 con 0x5555b68c73d0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec49d29640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec48d27640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec34001eb0 tx=0x7fec34007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e37b8af034fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec23fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3404f030 con 0x5555b68c73d0 2024-04-13T03:12:49.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fec3404cc30 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c0551a0 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec34004e60 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec4e580740 10 client.6501 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x5555b6906a80 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.148+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c0551a0 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fec3405c040 con 0x5555b68c73d0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 10 client.6501 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 10 client.6501 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 20 client.6501 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68ccb60 0x5555b68ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 4 client.6501 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555b680ca00 con 0x5555b68ccb60 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec49528640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68ccb60 0x5555b68ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec23fff640 10 client.6501 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec49528640 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68ccb60 0x5555b68ccf40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78505d11e6dce0dc server_cookie=e865a3f5304b8456 in_seq=0 out_seq=0 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec23fff640 1 -- 172.21.15.106:0/2264497691 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555b680ca00 con 0x5555b68ccb60 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec23fff640 10 client.6501 handle_command_reply: tid=0 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 client.6501 shutdown 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 msgr2=0x5555b689f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec34001eb0 tx=0x7fec34007000 comp rx=0 tx=0).stop 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 shutdown_connections 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x5555b68ccb60 0x5555b68ccf40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5555b68c9810 0x5555b68cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5555b68c73d0 0x5555b689f150 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 --2- 172.21.15.106:0/2264497691 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5555b68c6ff0 0x5555b689ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 >> 172.21.15.106:0/2264497691 conn(0x5555b689a4d0 msgr2=0x5555b689e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec23fff640 0 client.6501 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 shutdown_connections 2024-04-13T03:12:49.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.149+0000 7fec4e580740 1 -- 172.21.15.106:0/2264497691 wait complete. 2024-04-13T03:12:49.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4e580740 20 client.6501 trim_cache size 0 max 16384 2024-04-13T03:12:49.157 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:49.157 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:12:49.157 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:12:49.157 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 msgr2=0x7fec2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 0x7fec2806edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec4406bb40 tx=0x7fec40008040 comp rx=0 tx=0).stop 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 msgr2=0x7fec4415cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00fae0 tx=0x7fec3c0020d0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 shutdown_connections 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec441534a0 0x7fec44161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec44161550 0x7fec4415cd80 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 --2- 172.21.15.106:0/3811370309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec30004800 0x7fec44157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 >> 172.21.15.106:0/3811370309 conn(0x7fec4406f0d0 msgr2=0x7fec440cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 shutdown_connections 2024-04-13T03:12:49.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.150+0000 7fec4b78c640 1 -- 172.21.15.106:0/3811370309 wait complete. 2024-04-13T03:12:49.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:12:49.162 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:12:49.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.260+0000 7fcdf3280640 1 Processor -- start 2024-04-13T03:12:49.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.260+0000 7fcdf3280640 1 -- start start 2024-04-13T03:12:49.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf3280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf3280640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcdec06e150 0x7fcdec06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf3280640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdec0ce870 con 0x7fcdec06cb10 2024-04-13T03:12:49.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf3280640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcdec0998d0 con 0x7fcdec06e150 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf3280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec073020 con 0x7fcdec0cdf50 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcdec06e150 0x7fcdec06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcdec06e150 0x7fcdec06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38128/0 (socket says 172.21.15.106:38128) 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 -- 172.21.15.106:0/1609195218 learned_addr learned my addr 172.21.15.106:0/1609195218 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf101c640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 -- 172.21.15.106:0/1609195218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 msgr2=0x7fcdec0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 -- 172.21.15.106:0/1609195218 >> v1:172.21.15.114:6789/0 conn(0x7fcdec06cb10 legacy=0x7fcdec072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.261+0000 7fcdf181d640 1 -- 172.21.15.106:0/1609195218 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec073300 con 0x7fcdec06e150 2024-04-13T03:12:49.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.262+0000 7fcdf181d640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06e150 0x7fcdec06e600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d150 tx=0x7fcde402e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2257a5744700d60e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.262+0000 7fcde3fff640 1 -- 172.21.15.106:0/1609195218 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde400b0d0 con 0x7fcdec06e150 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.262+0000 7fcde3fff640 1 -- 172.21.15.106:0/1609195218 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde400b270 con 0x7fcdec06e150 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.262+0000 7fcde3fff640 1 -- 172.21.15.106:0/1609195218 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4009600 con 0x7fcdec06e150 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 -- 172.21.15.106:0/1609195218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06e150 msgr2=0x7fcdec06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06e150 0x7fcdec06e600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d150 tx=0x7fcde402e160 comp rx=0 tx=0).stop 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 -- 172.21.15.106:0/1609195218 shutdown_connections 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06e150 0x7fcdec06e600 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1609195218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 -- 172.21.15.106:0/1609195218 >> 172.21.15.106:0/1609195218 conn(0x7fcdec0b1f10 msgr2=0x7fcdec0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 -- 172.21.15.106:0/1609195218 shutdown_connections 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.263+0000 7fcdf3280640 1 -- 172.21.15.106:0/1609195218 wait complete. 2024-04-13T03:12:49.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.264+0000 7fcdf3280640 1 Processor -- start 2024-04-13T03:12:49.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.264+0000 7fcdf3280640 1 -- start start 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.264+0000 7fcdf3280640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.264+0000 7fcdf3280640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdec06e150 0x7fcdec0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.264+0000 7fcdf3280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf3280640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06eb40 con 0x7fcdec06cb10 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf3280640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0738c0 con 0x7fcdec06e150 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf3280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec06ecc0 con 0x7fcdec0cdf50 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38138/0 (socket says 172.21.15.106:38138) 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 -- 172.21.15.106:0/1273720285 learned_addr learned my addr 172.21.15.106:0/1273720285 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 -- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 msgr2=0x7fcdec0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 -- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdec06e150 msgr2=0x7fcdec0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdec06e150 0x7fcdec0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 -- 172.21.15.106:0/1273720285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec0c8d80 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf101c640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcde800d620 tx=0x7fcde800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80125f0 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde8019070 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec0c7800 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80129d0 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.265+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec0c8040 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.266+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec068a10 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.267+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcde8012b70 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.267+0000 7fcde1ffb640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.267+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fcde808c0c0 con 0x7fcdec06cb10 2024-04-13T03:12:49.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.267+0000 7fcdf081b640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.268+0000 7fcdf081b640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 0x7fcdd006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000f60 tx=0x7fcddc0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.272+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdec0c7800 con 0x7fcdec06cb10 2024-04-13T03:12:49.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.482+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdec0b19d0 con 0x7fcdec06cb10 2024-04-13T03:12:49.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.483+0000 7fcde1ffb640 1 -- 172.21.15.106:0/1273720285 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5185 (secure 0 0 0) 0x7fcde805a360 con 0x7fcdec06cb10 2024-04-13T03:12:49.485 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:49.485 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:24.015761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:12:49.485 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 29 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 msgr2=0x7fcdd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 0x7fcdd006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000f60 tx=0x7fcddc0079d0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 msgr2=0x7fcdec0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcde800d620 tx=0x7fcde800dae0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 shutdown_connections 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcdec0cdf50 0x7fcdec0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdec06e150 0x7fcdec0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 --2- 172.21.15.106:0/1273720285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec06cb10 0x7fcdec0c8840 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 >> 172.21.15.106:0/1273720285 conn(0x7fcdec0b1f10 msgr2=0x7fcdec15a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 shutdown_connections 2024-04-13T03:12:49.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.485+0000 7fcdf3280640 1 -- 172.21.15.106:0/1273720285 wait complete. 2024-04-13T03:12:49.497 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-13T03:12:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.597+0000 7fef02a36640 1 Processor -- start 2024-04-13T03:12:49.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.597+0000 7fef02a36640 1 -- start start 2024-04-13T03:12:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef02a36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feefc145d20 0x7feefc06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef02a36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feefc0b4c20 0x7feefc0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef02a36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc0b3740 con 0x7feefc0b47e0 2024-04-13T03:12:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef02a36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feefc0b3a20 con 0x7feefc0b4c20 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef02a36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7feefc0b3d00 con 0x7feefc145d20 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feefc0b4c20 0x7feefc0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feefc0b4c20 0x7feefc0cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38146/0 (socket says 172.21.15.106:38146) 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 -- 172.21.15.106:0/2717319744 learned_addr learned my addr 172.21.15.106:0/2717319744 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7feefbfff640 1 --2- 172.21.15.106:0/2717319744 >> v2:172.21.15.114:3300/0 conn(0x7feefc145d20 0x7feefc06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 -- 172.21.15.106:0/2717319744 >> v2:172.21.15.114:3300/0 conn(0x7feefc145d20 msgr2=0x7feefc06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 --2- 172.21.15.106:0/2717319744 >> v2:172.21.15.114:3300/0 conn(0x7feefc145d20 0x7feefc06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 -- 172.21.15.106:0/2717319744 >> v1:172.21.15.114:6789/0 conn(0x7feefc0b47e0 legacy=0x7feefc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.598+0000 7fef00fd3640 1 -- 172.21.15.106:0/2717319744 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc0b3fe0 con 0x7feefc0b4c20 2024-04-13T03:12:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.599+0000 7fef00fd3640 1 --2- 172.21.15.106:0/2717319744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b4c20 0x7feefc0cf1a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b2b20 tx=0x7feef402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365969bb76f43dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.599+0000 7feefaffd640 1 -- 172.21.15.106:0/2717319744 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef402c9f0 con 0x7feefc0b4c20 2024-04-13T03:12:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.599+0000 7feefaffd640 1 -- 172.21.15.106:0/2717319744 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef402cb90 con 0x7feefc0b4c20 2024-04-13T03:12:49.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.599+0000 7feefaffd640 1 -- 172.21.15.106:0/2717319744 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40343f0 con 0x7feefc0b4c20 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 -- 172.21.15.106:0/2717319744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b4c20 msgr2=0x7feefc0cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 --2- 172.21.15.106:0/2717319744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b4c20 0x7feefc0cf1a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b2b20 tx=0x7feef402be50 comp rx=0 tx=0).stop 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 -- 172.21.15.106:0/2717319744 shutdown_connections 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 --2- 172.21.15.106:0/2717319744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b4c20 0x7feefc0cf1a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 --2- 172.21.15.106:0/2717319744 >> v2:172.21.15.114:3300/0 conn(0x7feefc145d20 0x7feefc06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 -- 172.21.15.106:0/2717319744 >> 172.21.15.106:0/2717319744 conn(0x7feefc0b2600 msgr2=0x7feefc0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 -- 172.21.15.106:0/2717319744 shutdown_connections 2024-04-13T03:12:49.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.600+0000 7fef02a36640 1 -- 172.21.15.106:0/2717319744 wait complete. 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.601+0000 7fef02a36640 1 Processor -- start 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.601+0000 7fef02a36640 1 -- start start 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 0x7feefc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feefc145d20 0x7feefc15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feefc146280 con 0x7feefc0b47e0 2024-04-13T03:12:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feefc146100 con 0x7feefc0b4c20 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef02a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feefc06d050 con 0x7feefc145d20 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38160/0 (socket says 172.21.15.106:38160) 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 -- 172.21.15.106:0/297240248 learned_addr learned my addr 172.21.15.106:0/297240248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7fef00fd3640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feefc145d20 0x7feefc15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefb7fe640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 0x7feefc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 -- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feefc145d20 msgr2=0x7feefc15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feefc145d20 0x7feefc15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 -- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 msgr2=0x7feefc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 0x7feefc15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefbfff640 1 -- 172.21.15.106:0/297240248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc158800 con 0x7feefc0b47e0 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.602+0000 7feefb7fe640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 0x7feefc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7feefbfff640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c7f0 tx=0x7feeec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7feef8ff9640 1 -- 172.21.15.106:0/297240248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec015070 con 0x7feefc0b47e0 2024-04-13T03:12:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7feef8ff9640 1 -- 172.21.15.106:0/297240248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec004240 con 0x7feefc0b47e0 2024-04-13T03:12:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc13f6f0 con 0x7feefc0b47e0 2024-04-13T03:12:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7feef8ff9640 1 -- 172.21.15.106:0/297240248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec007620 con 0x7feefc0b47e0 2024-04-13T03:12:49.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.603+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc13f130 con 0x7feefc0b47e0 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.604+0000 7feef8ff9640 1 -- 172.21.15.106:0/297240248 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeec0079d0 con 0x7feefc0b47e0 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.605+0000 7feef8ff9640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 0x7feee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.605+0000 7feefb7fe640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 0x7feee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.605+0000 7feef8ff9640 1 -- 172.21.15.106:0/297240248 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7feeec08c200 con 0x7feefc0b47e0 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.605+0000 7feefb7fe640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 0x7feee806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feef0006fd0 tx=0x7feef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 Processor -- start 2024-04-13T03:12:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 -- start start 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561f9400cfe0 con 0x561f940d26d0 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f93fa9860 con 0x561f940cfaf0 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f94008fa0 con 0x561f940a7800 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.606+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38172/0 (socket says 172.21.15.106:38172) 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 -- 172.21.15.106:0/2773744930 learned_addr learned my addr 172.21.15.106:0/2773744930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 -- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 msgr2=0x561f940cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 -- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 msgr2=0x561f940d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 -- 172.21.15.106:0/2773744930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f93fd81a0 con 0x561f940d26d0 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feefbfff640 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feeec062330 tx=0x7feeec059e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b723f3fe579f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feee1ffb640 1 -- 172.21.15.106:0/2773744930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec01e040 con 0x561f940d26d0 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feee1ffb640 1 -- 172.21.15.106:0/2773744930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec057fa0 con 0x561f940d26d0 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.607+0000 7feee1ffb640 1 -- 172.21.15.106:0/2773744930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec007cc0 con 0x561f940d26d0 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- 172.21.15.106:0/2773744930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 msgr2=0x561f940a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feeec062330 tx=0x7feeec059e20 comp rx=0 tx=0).stop 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- 172.21.15.106:0/2773744930 shutdown_connections 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 --2- 172.21.15.106:0/2773744930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940d26d0 0x561f940a9560 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- 172.21.15.106:0/2773744930 >> 172.21.15.106:0/2773744930 conn(0x561f9403bdc0 msgr2=0x561f940ed4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- 172.21.15.106:0/2773744930 shutdown_connections 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- 172.21.15.106:0/2773744930 wait complete. 2024-04-13T03:12:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.608+0000 7fef0581f740 1 -- start start 2024-04-13T03:12:49.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 Processor -- start 2024-04-13T03:12:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 0x561f940d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561f940a2b00 con 0x561f940cfaf0 2024-04-13T03:12:49.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f93fa71d0 con 0x561f940d26d0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f940edc90 con 0x561f940a7800 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7fef0581f740 10 client.0 _ll_register_callbacks cb 0x561f93fca010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7feefb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7feefb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38176/0 (socket says 172.21.15.106:38176) 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.609+0000 7feefb7fe640 1 -- 172.21.15.106:0/3104203230 learned_addr learned my addr 172.21.15.106:0/3104203230 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefbfff640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef00fd3640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 0x561f940d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 -- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 msgr2=0x561f940c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 -- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 msgr2=0x561f940d5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 0x561f940d5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 -- 172.21.15.106:0/3104203230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f93deb7c0 con 0x561f940cfaf0 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefbfff640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef00fd3640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 0x561f940d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:49.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b34b0 tx=0x7feef0014800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a7085a47912d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0024060 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7feef001f400 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec057fa0 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feef0004020 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 10 client.6519 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x561f93e4bf50 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec057fa0 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7feef0013070 con 0x561f940cfaf0 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 10 client.6519 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 10 client.6519 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 20 client.6519 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f94106f20 0x561f9410b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 4 client.6519 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:49.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.610+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f93fac4a0 con 0x561f94106f20 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7feefbfff640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f94106f20 0x561f9410b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7feefbfff640 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f94106f20 0x561f9410b1e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea051e0cc3ded74b server_cookie=dfff836f8021c4a2 in_seq=0 out_seq=0 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7feedaffd640 10 client.6519 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7feedaffd640 1 -- 172.21.15.106:0/3104203230 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f93fac4a0 con 0x561f94106f20 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7feedaffd640 10 client.6519 handle_command_reply: tid=0 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.611+0000 7fef0581f740 1 client.6519 shutdown 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 msgr2=0x561f940d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b34b0 tx=0x7feef0014800 comp rx=0 tx=0).stop 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 shutdown_connections 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f94106f20 0x561f9410b1e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940d26d0 0x561f940d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940cfaf0 0x561f940d3e70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 --2- 172.21.15.106:0/3104203230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940a7800 0x561f940c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7feedaffd640 0 client.6519 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 >> 172.21.15.106:0/3104203230 conn(0x561f9403bdc0 msgr2=0x561f940a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 shutdown_connections 2024-04-13T03:12:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 1 -- 172.21.15.106:0/3104203230 wait complete. 2024-04-13T03:12:49.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.612+0000 7fef0581f740 20 client.6519 trim_cache size 0 max 16384 2024-04-13T03:12:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.625+0000 7fef0581f740 1 Processor -- start 2024-04-13T03:12:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.625+0000 7fef0581f740 1 -- start start 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f94110720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f940a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1fe340 con 0x561f940a7800 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feefc13e800 con 0x561f940cfaf0 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feefc1fdcf0 con 0x561f940d26d0 2024-04-13T03:12:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38184/0 (socket says 172.21.15.106:38184) 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 -- 172.21.15.106:0/1000693790 learned_addr learned my addr 172.21.15.106:0/1000693790 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 -- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 msgr2=0x561f940a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f940a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 -- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 msgr2=0x561f94110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f94110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 -- 172.21.15.106:0/1000693790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f940d2500 con 0x561f940a7800 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feefbfff640 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feeec08a7f0 tx=0x7feeec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4478df73a5a06b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feee17fa640 1 -- 172.21.15.106:0/1000693790 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec083e60 con 0x561f940a7800 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feee17fa640 1 -- 172.21.15.106:0/1000693790 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec056310 con 0x561f940a7800 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.626+0000 7feee17fa640 1 -- 172.21.15.106:0/1000693790 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec01e040 con 0x561f940a7800 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- 172.21.15.106:0/1000693790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 msgr2=0x561f940cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feeec08a7f0 tx=0x7feeec01a000 comp rx=0 tx=0).stop 2024-04-13T03:12:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- 172.21.15.106:0/1000693790 shutdown_connections 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f940a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f94110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 --2- 172.21.15.106:0/1000693790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940cf150 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- 172.21.15.106:0/1000693790 >> 172.21.15.106:0/1000693790 conn(0x561f9403bdc0 msgr2=0x561f940c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- 172.21.15.106:0/1000693790 shutdown_connections 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- 172.21.15.106:0/1000693790 wait complete. 2024-04-13T03:12:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.627+0000 7fef0581f740 1 -- start start 2024-04-13T03:12:49.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 Processor -- start 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f9410f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561f94106da0 con 0x561f940a7800 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f940d45c0 con 0x561f940cfaf0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f941073c0 con 0x561f940d26d0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7fef0581f740 10 client.0 _ll_register_callbacks cb 0x7feef0009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7feefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38190/0 (socket says 172.21.15.106:38190) 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.628+0000 7feefbfff640 1 -- 172.21.15.106:0/3937394764 learned_addr learned my addr 172.21.15.106:0/3937394764 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef00fd3640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f9410f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 -- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 msgr2=0x561f9410f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f9410f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 -- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 msgr2=0x561f940a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 -- 172.21.15.106:0/3937394764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f93cdf0b0 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef00fd3640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f9410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feefbfff640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7feeec004630 tx=0x7feeec006f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9504faba9fc04dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec083e60 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7feeec057fa0 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec056310 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feeec07ae60 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 10 client.6525 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x561f940c8180 con 0x561f940a7800 2024-04-13T03:12:49.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec056310 con 0x561f940a7800 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7feeec007cc0 con 0x561f940a7800 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 10 client.6525 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 10 client.6525 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 20 client.6525 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f9410f950 0x561f940c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 4 client.6525 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.629+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x561f9401aa00 con 0x561f9410f950 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f9410f950 0x561f940c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feefb7fe640 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f9410f950 0x561f940c5e10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90f5bbc3a6472f2 server_cookie=4dcd1a8c71e0e884 in_seq=0 out_seq=0 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feedaffd640 10 client.6525 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feedaffd640 1 -- 172.21.15.106:0/3937394764 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x561f9401aa00 con 0x561f9410f950 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feedaffd640 10 client.6525 handle_command_reply: tid=0 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 client.6525 shutdown 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 msgr2=0x561f940a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7feeec004630 tx=0x7feeec006f30 comp rx=0 tx=0).stop 2024-04-13T03:12:49.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 shutdown_connections 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x561f9410f950 0x561f940c5e10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f940d26d0 0x561f9410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f940cfaf0 0x561f940a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 --2- 172.21.15.106:0/3937394764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f940a7800 0x561f940a6250 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7feedaffd640 0 client.6525 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 >> 172.21.15.106:0/3937394764 conn(0x561f9403bdc0 msgr2=0x561f940ed9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 shutdown_connections 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 1 -- 172.21.15.106:0/3937394764 wait complete. 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.630+0000 7fef0581f740 20 client.6525 trim_cache size 0 max 16384 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:49.638 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "backtrace", 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 3651295286, 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir_test/file" 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:49.639 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 msgr2=0x7feee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 0x7feee806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feef0006fd0 tx=0x7feef0008040 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 msgr2=0x7feefc1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c7f0 tx=0x7feeec00ccb0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 shutdown_connections 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feee806c8f0 0x7feee806edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feefc145d20 0x7feefc15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0b4c20 0x7feefc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 --2- 172.21.15.106:0/297240248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feefc0b47e0 0x7feefc1582c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 >> 172.21.15.106:0/297240248 conn(0x7feefc0b2600 msgr2=0x7feefc0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 shutdown_connections 2024-04-13T03:12:49.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.631+0000 7fef02a36640 1 -- 172.21.15.106:0/297240248 wait complete. 2024-04-13T03:12:49.643 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:12:49.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.741+0000 7fdd85315640 1 Processor -- start 2024-04-13T03:12:49.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.741+0000 7fdd85315640 1 -- start start 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd85315640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdd8015b6a0 0x7fdd800d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd85315640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd800cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd85315640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd800d0880 con 0x7fdd800cd5e0 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd85315640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdd8006d280 con 0x7fdd8015b6a0 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd85315640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdd800d0b60 con 0x7fdd80073e10 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd7f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdd800cd5e0 0x7fdd80072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51686/0 (socket says 172.21.15.106:51686) 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd7f7fe640 1 -- 172.21.15.106:0/1919879029 learned_addr learned my addr 172.21.15.106:0/1919879029 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1919879029 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd800cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.742+0000 7fdd7effd640 1 --2- 172.21.15.106:0/1919879029 >> v2:172.21.15.114:3300/0 conn(0x7fdd8015b6a0 0x7fdd800d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710759169 0 0) 0x7fdd800d0880 con 0x7fdd800cd5e0 2024-04-13T03:12:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd68003520 con 0x7fdd800cd5e0 2024-04-13T03:12:49.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1919879029 >> v2:172.21.15.114:3300/0 conn(0x7fdd8015b6a0 msgr2=0x7fdd800d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:49.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1919879029 >> v2:172.21.15.114:3300/0 conn(0x7fdd8015b6a0 0x7fdd800d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3079220437 0 0) 0x7fdd68003520 con 0x7fdd800cd5e0 2024-04-13T03:12:49.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1919879029 >> v1:172.21.15.106:6789/0 conn(0x7fdd800cd5e0 legacy=0x7fdd80072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.743+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1919879029 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800d0e40 con 0x7fdd80073e10 2024-04-13T03:12:49.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1919879029 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd800cbac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd740097a0 tx=0x7fdd7402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a51780cac334cc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74037070 con 0x7fdd80073e10 2024-04-13T03:12:49.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd74030510 con 0x7fdd80073e10 2024-04-13T03:12:49.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd7dffb640 1 -- 172.21.15.106:0/1919879029 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78001670 con 0x7fdd80073e10 2024-04-13T03:12:49.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 -- 172.21.15.106:0/1919879029 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 msgr2=0x7fdd800cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 --2- 172.21.15.106:0/1919879029 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd800cbac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd740097a0 tx=0x7fdd7402b790 comp rx=0 tx=0).stop 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 -- 172.21.15.106:0/1919879029 shutdown_connections 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 --2- 172.21.15.106:0/1919879029 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd800cbac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 --2- 172.21.15.106:0/1919879029 >> v2:172.21.15.114:3300/0 conn(0x7fdd8015b6a0 0x7fdd800d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 -- 172.21.15.106:0/1919879029 >> 172.21.15.106:0/1919879029 conn(0x7fdd800cdef0 msgr2=0x7fdd800ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 -- 172.21.15.106:0/1919879029 shutdown_connections 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.744+0000 7fdd85315640 1 -- 172.21.15.106:0/1919879029 wait complete. 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 Processor -- start 2024-04-13T03:12:49.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 -- start start 2024-04-13T03:12:49.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd801447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd8015b6a0 0x7fdd8015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80073640 con 0x7fdd800cd5e0 2024-04-13T03:12:49.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800734c0 con 0x7fdd8015b6a0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd85315640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8006d710 con 0x7fdd80073e10 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd7e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd7e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38192/0 (socket says 172.21.15.106:38192) 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.746+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1657474907 learned_addr learned my addr 172.21.15.106:0/1657474907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7effd640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd801447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7f7fe640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd8015b6a0 0x7fdd8015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 msgr2=0x7fdd801447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd801447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd8015b6a0 msgr2=0x7fdd8015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd8015b6a0 0x7fdd8015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 -- 172.21.15.106:0/1657474907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8015e970 con 0x7fdd800cd5e0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd7e7fc640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdd7402b7d0 tx=0x7fdd740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74041030 con 0x7fdd800cd5e0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7403c670 con 0x7fdd800cd5e0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8015a6a0 con 0x7fdd800cd5e0 2024-04-13T03:12:49.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74037040 con 0x7fdd800cd5e0 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.747+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd800b45a0 con 0x7fdd800cd5e0 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.748+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd80069120 con 0x7fdd800cd5e0 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.748+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd74044070 con 0x7fdd800cd5e0 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.749+0000 7fdd677fe640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.749+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fdd740ad450 con 0x7fdd800cd5e0 2024-04-13T03:12:49.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.749+0000 7fdd7effd640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:49.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.749+0000 7fdd7effd640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b73d0 tx=0x7fdd7000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:49.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.753+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd8015e970 con 0x7fdd800cd5e0 2024-04-13T03:12:49.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.967+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd8015c340 con 0x7fdd800cd5e0 2024-04-13T03:12:49.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.967+0000 7fdd677fe640 1 -- 172.21.15.106:0/1657474907 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fdd7407b780 con 0x7fdd800cd5e0 2024-04-13T03:12:49.969 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:12:49.969 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 msgr2=0x7fdd6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b73d0 tx=0x7fdd7000b040 comp rx=0 tx=0).stop 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 msgr2=0x7fdd8015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdd7402b7d0 tx=0x7fdd740046c0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 shutdown_connections 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd8015b6a0 0x7fdd8015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd800cd5e0 0x7fdd8015e430 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 --2- 172.21.15.106:0/1657474907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdd80073e10 0x7fdd801447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:49.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 >> 172.21.15.106:0/1657474907 conn(0x7fdd800cdef0 msgr2=0x7fdd800b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:49.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 shutdown_connections 2024-04-13T03:12:49.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:49.969+0000 7fdd85315640 1 -- 172.21.15.106:0/1657474907 wait complete. 2024-04-13T03:12:49.982 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:12:49.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:49.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-13T03:12:50.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.083+0000 7f26a3f5e640 1 Processor -- start 2024-04-13T03:12:50.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a3f5e640 1 -- start start 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a3f5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f269c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a3f5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c15c870 con 0x7f269c0d3490 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a3f5e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c15ee70 con 0x7f269c0d2370 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a3f5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0b1b00 con 0x7f269c0b3920 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a1cfa640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f269c0d3490 0x7f269c0d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51688/0 (socket says 172.21.15.106:51688) 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.084+0000 7f26a1cfa640 1 -- 172.21.15.106:0/723190153 learned_addr learned my addr 172.21.15.106:0/723190153 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:50.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f269c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663183856 0 0) 0x7f269c15c870 con 0x7f269c0d3490 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2688003520 con 0x7f269c0d3490 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3254997569 0 0) 0x7f2688003520 con 0x7f269c0d3490 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 msgr2=0x7f269c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f269c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 >> v1:172.21.15.114:6789/0 conn(0x7f269c0d2370 legacy=0x7f269c0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0d2800 con 0x7f269c0d3490 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f268c001670 con 0x7f269c0d3490 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 >> v1:172.21.15.106:6789/0 conn(0x7f269c0d3490 legacy=0x7f269c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f26880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688003f60 0x7f2688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2688004d40 0x7f26880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0b1b00 con 0x7f2688004d40 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f268800bb50 con 0x7f2688003f60 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f268800bd50 con 0x7f269c0b3920 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2688004d40 0x7f26880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f26880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a14f9640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688003f60 0x7f2688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 -- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 msgr2=0x7f26880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c0b3920 0x7f26880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 -- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688003f60 msgr2=0x7f2688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688003f60 0x7f2688004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a24fb640 1 -- 172.21.15.106:0/723190153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2688003520 con 0x7f2688004d40 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.085+0000 7f26a14f9640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688003f60 0x7f2688004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.086+0000 7f26a24fb640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2688004d40 0x7f26880051c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2694009fc0 tx=0x7f2688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6148e85c6f61a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.086+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694039070 con 0x7f2688004d40 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.086+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269402cc20 con 0x7f2688004d40 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.086+0000 7f26a0cf8640 1 -- 172.21.15.106:0/723190153 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694009c10 con 0x7f2688004d40 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 -- 172.21.15.106:0/723190153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2688004d40 msgr2=0x7f26880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/723190153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2688004d40 0x7f26880051c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2694009fc0 tx=0x7f2688005830 comp rx=0 tx=0).stop 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a24fb640 1 -- 172.21.15.106:0/723190153 reap_dead start 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 -- 172.21.15.106:0/723190153 shutdown_connections 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 -- 172.21.15.106:0/723190153 >> 172.21.15.106:0/723190153 conn(0x7f269c0b9760 msgr2=0x7f269c0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 -- 172.21.15.106:0/723190153 shutdown_connections 2024-04-13T03:12:50.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.087+0000 7f26a3f5e640 1 -- 172.21.15.106:0/723190153 wait complete. 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.088+0000 7f26a3f5e640 1 Processor -- start 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.088+0000 7f26a3f5e640 1 -- start start 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 0x7f269c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c1f5780 0x7f269c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0d2dc0 con 0x7f269c1592b0 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0d5420 con 0x7f2688004d40 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a3f5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0d30a0 con 0x7f269c1f5780 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38208/0 (socket says 172.21.15.106:38208) 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 -- 172.21.15.106:0/69167117 learned_addr learned my addr 172.21.15.106:0/69167117 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 0x7f269c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a24fb640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c1f5780 0x7f269c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 -- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c1f5780 msgr2=0x7f269c1f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c1f5780 0x7f269c1f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 -- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 msgr2=0x7f269c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 0x7f269c06cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 -- 172.21.15.106:0/69167117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c06b8e0 con 0x7f269c1592b0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 0x7f269c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26a14f9640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f269800e580 tx=0x7f269800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.089+0000 7f26927fc640 1 -- 172.21.15.106:0/69167117 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980052e0 con 0x7f269c1592b0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.090+0000 7f26927fc640 1 -- 172.21.15.106:0/69167117 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2698018070 con 0x7f269c1592b0 2024-04-13T03:12:50.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.090+0000 7f26927fc640 1 -- 172.21.15.106:0/69167117 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698003860 con 0x7f269c1592b0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.090+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c06bbc0 con 0x7f269c1592b0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.090+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c06c180 con 0x7f269c1592b0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.091+0000 7f26927fc640 1 -- 172.21.15.106:0/69167117 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2698003a00 con 0x7f269c1592b0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.092+0000 7f26927fc640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 0x7f268006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.092+0000 7f26927fc640 1 -- 172.21.15.106:0/69167117 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f269808ba20 con 0x7f269c1592b0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.092+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 0x7f268006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.092+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 0x7f268006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f269c0b1700 tx=0x7f268c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 Processor -- start 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 -- start start 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a3dbb0 0x557828a3f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a698f0 0x557828a69d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578289a4e80 con 0x557828a39b60 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55782894b8a0 con 0x557828a3dbb0 2024-04-13T03:12:50.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5578289a0e40 con 0x557828a698f0 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a1cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a1cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38210/0 (socket says 172.21.15.106:38210) 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.093+0000 7f26a1cfa640 1 -- 172.21.15.106:0/862749074 learned_addr learned my addr 172.21.15.106:0/862749074 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 -- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a698f0 msgr2=0x557828a69d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a698f0 0x557828a69d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 -- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a3dbb0 msgr2=0x557828a3f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a3dbb0 0x557828a3f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 -- 172.21.15.106:0/862749074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782895c200 con 0x557828a39b60 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f268c004da0 tx=0x7f268c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82762d362f4e04f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f267f7fe640 1 -- 172.21.15.106:0/862749074 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c021450 con 0x557828a39b60 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f267f7fe640 1 -- 172.21.15.106:0/862749074 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c026070 con 0x557828a39b60 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.094+0000 7f267f7fe640 1 -- 172.21.15.106:0/862749074 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0217f0 con 0x557828a39b60 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- 172.21.15.106:0/862749074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 msgr2=0x557828a39f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f268c004da0 tx=0x7f268c014000 comp rx=0 tx=0).stop 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- 172.21.15.106:0/862749074 shutdown_connections 2024-04-13T03:12:50.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a698f0 0x557828a69d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a3dbb0 0x557828a3f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 --2- 172.21.15.106:0/862749074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a39b60 0x557828a39f40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- 172.21.15.106:0/862749074 >> 172.21.15.106:0/862749074 conn(0x557828a3bbe0 msgr2=0x557828a3d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- 172.21.15.106:0/862749074 shutdown_connections 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- 172.21.15.106:0/862749074 wait complete. 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 -- start start 2024-04-13T03:12:50.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.095+0000 7f26a6e34740 1 Processor -- start 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557828a3e040 con 0x557828a6d7f0 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55782895ec60 con 0x557828a698f0 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557828a3e320 con 0x557828a6afd0 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a6e34740 10 client.0 _ll_register_callbacks cb 0x55782895e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a24fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a24fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38216/0 (socket says 172.21.15.106:38216) 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a24fb640 1 -- 172.21.15.106:0/976594087 learned_addr learned my addr 172.21.15.106:0/976594087 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.096+0000 7f26a14f9640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 -- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 msgr2=0x557828a6d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a6d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 -- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 msgr2=0x557828a6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a6aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 -- 172.21.15.106:0/976594087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557828783940 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a24fb640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2694037040 tx=0x7f2694009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b5202873d1b8db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694043020 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f2694003b90 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268c026070 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2694039040 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 10 client.6543 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x5578287e40d0 con 0x557828a6d7f0 2024-04-13T03:12:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c026070 con 0x557828a6d7f0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f2694004040 con 0x557828a6d7f0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 10 client.6543 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 10 client.6543 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 20 client.6543 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a7f8c0 0x557828a7fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 4 client.6543 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.097+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578289b28a0 con 0x557828a7f8c0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a7f8c0 0x557828a7fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f267cff9640 10 client.6543 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a7f8c0 0x557828a7fca0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c02251b11cfa5d server_cookie=a75a38fc0b4025b2 in_seq=0 out_seq=0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f267cff9640 1 -- 172.21.15.106:0/976594087 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578289b28a0 con 0x557828a7f8c0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f267cff9640 10 client.6543 handle_command_reply: tid=0 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f26a6e34740 1 client.6543 shutdown 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 msgr2=0x557828a84fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.098+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2694037040 tx=0x7f2694009750 comp rx=0 tx=0).stop 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 shutdown_connections 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a7f8c0 0x557828a7fca0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a6d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a84fe0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f267cff9640 0 client.6543 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 --2- 172.21.15.106:0/976594087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 >> 172.21.15.106:0/976594087 conn(0x557828a39b60 msgr2=0x557828a678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 shutdown_connections 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 1 -- 172.21.15.106:0/976594087 wait complete. 2024-04-13T03:12:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.099+0000 7f26a6e34740 20 client.6543 trim_cache size 0 max 16384 2024-04-13T03:12:50.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.116+0000 7f26a6e34740 1 Processor -- start 2024-04-13T03:12:50.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.116+0000 7f26a6e34740 1 -- start start 2024-04-13T03:12:50.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a7e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a7ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f269c201ab0 con 0x557828a6d7f0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f269c200ad0 con 0x557828a698f0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f269c1ffe30 con 0x557828a6afd0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38224/0 (socket says 172.21.15.106:38224) 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 -- 172.21.15.106:0/392828030 learned_addr learned my addr 172.21.15.106:0/392828030 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 -- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 msgr2=0x557828a7ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a7ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 -- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 msgr2=0x557828a7e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a7e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 -- 172.21.15.106:0/392828030 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557828677230 con 0x557828a6d7f0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f26a24fb640 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2694009e60 tx=0x7f269403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd65e867056545f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f267effd640 1 -- 172.21.15.106:0/392828030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694043040 con 0x557828a6d7f0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f267effd640 1 -- 172.21.15.106:0/392828030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694004590 con 0x557828a6d7f0 2024-04-13T03:12:50.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.117+0000 7f267effd640 1 -- 172.21.15.106:0/392828030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694004740 con 0x557828a6d7f0 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 -- 172.21.15.106:0/392828030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 msgr2=0x557828a38a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2694009e60 tx=0x7f269403c000 comp rx=0 tx=0).stop 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 -- 172.21.15.106:0/392828030 shutdown_connections 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828a7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a6d7f0 0x557828a38a50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 --2- 172.21.15.106:0/392828030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a698f0 0x557828a7e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 -- 172.21.15.106:0/392828030 >> 172.21.15.106:0/392828030 conn(0x557828a39b60 msgr2=0x557828a3e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 -- 172.21.15.106:0/392828030 shutdown_connections 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.118+0000 7f26a6e34740 1 -- 172.21.15.106:0/392828030 wait complete. 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.119+0000 7f26a6e34740 1 -- start start 2024-04-13T03:12:50.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.119+0000 7f26a6e34740 1 Processor -- start 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.119+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828aa6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a6d7f0 0x557828a6a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f269c200480 con 0x557828a698f0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557828a3cfe0 con 0x557828a6d7f0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557828aa6f10 con 0x557828a6afd0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 10 client.0 _ll_register_callbacks cb 0x7f2688004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38236/0 (socket says 172.21.15.106:38236) 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 -- 172.21.15.106:0/3429350293 learned_addr learned my addr 172.21.15.106:0/3429350293 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a24fb640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a6d7f0 0x557828a6a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a14f9640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828aa6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 -- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 msgr2=0x557828aa6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828aa6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 -- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a6d7f0 msgr2=0x557828a6a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a6d7f0 0x557828a6a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 -- 172.21.15.106:0/3429350293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557828a9dc80 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a1cfa640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f268c023040 tx=0x7f268c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c8c27d538f7b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f267cff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0218e0 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f268c021a80 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2694004590 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f268c021c80 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 10 client.6549 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x557828aa6c50 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.120+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694004590 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f268c034070 con 0x557828a698f0 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 10 client.6549 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 10 client.6549 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:50.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 20 client.6549 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a97650 0x557828a3ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 4 client.6549 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x5578289ac710 con 0x557828a97650 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a14f9640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a97650 0x557828a3ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f267cff9640 10 client.6549 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.121+0000 7f26a14f9640 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a97650 0x557828a3ea10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d867cfa02d44945d server_cookie=3cb8e0bda872f282 in_seq=0 out_seq=0 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f267cff9640 1 -- 172.21.15.106:0/3429350293 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578289ac710 con 0x557828a97650 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f267cff9640 10 client.6549 handle_command_reply: tid=0 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 client.6549 shutdown 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 msgr2=0x557828afeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f268c023040 tx=0x7f268c0078f0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 shutdown_connections 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x557828a97650 0x557828a3ea10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557828a6afd0 0x557828aa6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557828a6d7f0 0x557828a6a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f267cff9640 0 client.6549 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 --2- 172.21.15.106:0/3429350293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557828a698f0 0x557828afeaa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 >> 172.21.15.106:0/3429350293 conn(0x557828a39b60 msgr2=0x557828a7e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 shutdown_connections 2024-04-13T03:12:50.129 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:12:50.122+0000 7fbb953e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6420=0-4194304@1} caps={6420=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6420 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55e633d34680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:12:50.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 1 -- 172.21.15.106:0/3429350293 wait complete. 2024-04-13T03:12:50.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.122+0000 7f26a6e34740 20 client.6549 trim_cache size 0 max 16384 2024-04-13T03:12:50.130 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:50.130 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:12:50.130 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "2884081e-0444-46b9-b7e0-cbc61d97c2c8", 2024-04-13T03:12:50.130 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:12:50.130 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 msgr2=0x7f268006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 0x7f268006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f269c0b1700 tx=0x7f268c009040 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 msgr2=0x7f269c159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f269800e580 tx=0x7f269800ea40 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 shutdown_connections 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f268006c920 0x7f268006ede0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f269c1f5780 0x7f269c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c1592b0 0x7f269c159730 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 --2- 172.21.15.106:0/69167117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2688004d40 0x7f269c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 >> 172.21.15.106:0/69167117 conn(0x7f269c0b9760 msgr2=0x7f269c15dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 shutdown_connections 2024-04-13T03:12:50.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.123+0000 7f26a3f5e640 1 -- 172.21.15.106:0/69167117 wait complete. 2024-04-13T03:12:50.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:12:50.135 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-13T03:12:50.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.236+0000 7fa2d151e640 1 Processor -- start 2024-04-13T03:12:50.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.236+0000 7fa2d151e640 1 -- start start 2024-04-13T03:12:50.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2d151e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa2cc1593b0 0x7fa2cc0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2d151e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2cc070a30 0x7fa2cc15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2d151e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2cc160170 con 0x7fa2cc15f930 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2d151e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc070e90 con 0x7fa2cc070a30 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2d151e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc073d50 con 0x7fa2cc1593b0 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2ca7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2cc070a30 0x7fa2cc15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2cb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa2cc15f930 0x7fa2cc06d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51696/0 (socket says 172.21.15.106:51696) 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2cb7fe640 1 -- 172.21.15.106:0/2230338824 learned_addr learned my addr 172.21.15.106:0/2230338824 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2230338824 >> v2:172.21.15.114:3300/0 conn(0x7fa2cc1593b0 msgr2=0x7fa2cc0704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2230338824 >> v2:172.21.15.114:3300/0 conn(0x7fa2cc1593b0 0x7fa2cc0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2230338824 >> v1:172.21.15.106:6789/0 conn(0x7fa2cc15f930 legacy=0x7fa2cc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.237+0000 7fa2c9ffb640 1 -- 172.21.15.106:0/2230338824 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1224521192 0 0) 0x7fa2cc160170 con 0x7fa2cc15f930 2024-04-13T03:12:50.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.238+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2230338824 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc0cf390 con 0x7fa2cc070a30 2024-04-13T03:12:50.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.238+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2230338824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc15f3f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc074540 tx=0x7fa2c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938a039cc5c0c544 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.238+0000 7fa2c9ffb640 1 -- 172.21.15.106:0/2230338824 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0035020 con 0x7fa2cc070a30 2024-04-13T03:12:50.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.238+0000 7fa2c9ffb640 1 -- 172.21.15.106:0/2230338824 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0031830 con 0x7fa2cc070a30 2024-04-13T03:12:50.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.238+0000 7fa2c9ffb640 1 -- 172.21.15.106:0/2230338824 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0031c00 con 0x7fa2cc070a30 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 -- 172.21.15.106:0/2230338824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 msgr2=0x7fa2cc15f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2230338824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc15f3f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc074540 tx=0x7fa2c002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 -- 172.21.15.106:0/2230338824 shutdown_connections 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2230338824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc15f3f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2230338824 >> v2:172.21.15.114:3300/0 conn(0x7fa2cc1593b0 0x7fa2cc0704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 -- 172.21.15.106:0/2230338824 >> 172.21.15.106:0/2230338824 conn(0x7fa2cc0d13f0 msgr2=0x7fa2cc0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 -- 172.21.15.106:0/2230338824 shutdown_connections 2024-04-13T03:12:50.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.239+0000 7fa2d151e640 1 -- 172.21.15.106:0/2230338824 wait complete. 2024-04-13T03:12:50.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.240+0000 7fa2d151e640 1 Processor -- start 2024-04-13T03:12:50.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.240+0000 7fa2d151e640 1 -- start start 2024-04-13T03:12:50.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2cc1593b0 0x7fa2cc14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 0x7fa2cc150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0cfc30 con 0x7fa2cc070a30 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0cf950 con 0x7fa2cc15f930 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2d151e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc15fd80 con 0x7fa2cc1593b0 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38246/0 (socket says 172.21.15.106:38246) 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 -- 172.21.15.106:0/2531839368 learned_addr learned my addr 172.21.15.106:0/2531839368 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2cc1593b0 0x7fa2cc14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 0x7fa2cc150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 -- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2cc1593b0 msgr2=0x7fa2cc14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2cc1593b0 0x7fa2cc14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 -- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 msgr2=0x7fa2cc150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 0x7fa2cc150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2caffd640 1 -- 172.21.15.106:0/2531839368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc14fb10 con 0x7fa2cc070a30 2024-04-13T03:12:50.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.241+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 0x7fa2cc150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ba80 tx=0x7fa2bc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2b37fe640 1 -- 172.21.15.106:0/2531839368 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc019070 con 0x7fa2cc070a30 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2b37fe640 1 -- 172.21.15.106:0/2531839368 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00cc20 con 0x7fa2cc070a30 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc14fdf0 con 0x7fa2cc070a30 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2b37fe640 1 -- 172.21.15.106:0/2531839368 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc014440 con 0x7fa2cc070a30 2024-04-13T03:12:50.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.242+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1503b0 con 0x7fa2cc070a30 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.243+0000 7fa2b37fe640 1 -- 172.21.15.106:0/2531839368 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2bc01c070 con 0x7fa2cc070a30 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.244+0000 7fa2b37fe640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 0x7fa2b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.244+0000 7fa2b37fe640 1 -- 172.21.15.106:0/2531839368 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc08b920 con 0x7fa2cc070a30 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.244+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 0x7fa2b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.244+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 0x7fa2b806ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0036000 tx=0x7fa2c003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 Processor -- start 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 -- start start 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f30664ff0 0x558f30637f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306384d0 0x558f306682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558f305a3170 con 0x558f306653d0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558f305497f0 con 0x558f306384d0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.245+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558f3059f130 con 0x558f30664ff0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38258/0 (socket says 172.21.15.106:38258) 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2200294259 learned_addr learned my addr 172.21.15.106:0/2200294259 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f30664ff0 0x558f30637f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306384d0 0x558f306682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f30664ff0 msgr2=0x558f30637f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f30664ff0 0x558f30637f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306384d0 msgr2=0x558f306682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306384d0 0x558f306682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 -- 172.21.15.106:0/2200294259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3037f7d0 con 0x558f306653d0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031c10 tx=0x7fa2c0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9544ad6350da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2b0ff9640 1 -- 172.21.15.106:0/2200294259 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0057070 con 0x558f306653d0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2b0ff9640 1 -- 172.21.15.106:0/2200294259 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0031c40 con 0x558f306653d0 2024-04-13T03:12:50.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.246+0000 7fa2b0ff9640 1 -- 172.21.15.106:0/2200294259 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0052420 con 0x558f306653d0 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2200294259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 msgr2=0x558f30667d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031c10 tx=0x7fa2c0030000 comp rx=0 tx=0).stop 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2200294259 shutdown_connections 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306384d0 0x558f306682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f306653d0 0x558f30667d80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2200294259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f30664ff0 0x558f30637f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2200294259 >> 172.21.15.106:0/2200294259 conn(0x558f3063a200 msgr2=0x558f30682d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2200294259 shutdown_connections 2024-04-13T03:12:50.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.247+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2200294259 wait complete. 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.248+0000 7fa2d43ec740 1 -- start start 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.248+0000 7fa2d43ec740 1 Processor -- start 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.248+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.248+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 0x558f3066bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 0x558f3066c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558f3063c8a0 con 0x558f30664ff0 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558f3055cd00 con 0x558f306653d0 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558f3063f0e0 con 0x558f3066db10 2024-04-13T03:12:50.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 10 client.0 _ll_register_callbacks cb 0x558f3055c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38274/0 (socket says 172.21.15.106:38274) 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 -- 172.21.15.106:0/2494295934 learned_addr learned my addr 172.21.15.106:0/2494295934 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 0x558f3066c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 0x558f3066bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 -- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 msgr2=0x558f3066c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 0x558f3066c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 -- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 msgr2=0x558f3066bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 0x558f3066bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 -- 172.21.15.106:0/2494295934 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f302b53f0 con 0x558f30664ff0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 0x558f3066bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 0x558f3066c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ba50 tx=0x7fa2bc062130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f18d1bc10be761 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc058a50 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc058bf0 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c0031c40 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc055350 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 10 client.6564 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x558f302750f0 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.249+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0031c40 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fa2bc0554f0 con 0x558f30664ff0 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 10 client.6564 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 10 client.6564 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 20 client.6564 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f3066ec90 0x558f30696450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 4 client.6564 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:12:50.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f305b0b90 con 0x558f3066ec90 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f3066ec90 0x558f30696450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2a9ffb640 10 client.6564 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.250+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f3066ec90 0x558f30696450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc14b2e975b3800 server_cookie=a621a0c23afe75ad in_seq=0 out_seq=0 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/2494295934 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f305b0b90 con 0x558f3066ec90 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2a9ffb640 10 client.6564 handle_command_reply: tid=0 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 client.6564 shutdown 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 msgr2=0x558f306938b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ba50 tx=0x7fa2bc062130 comp rx=0 tx=0).stop 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 shutdown_connections 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f3066ec90 0x558f30696450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f3066db10 0x558f3066c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f306653d0 0x558f3066bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2a9ffb640 0 client.6564 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2494295934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f306938b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 >> 172.21.15.106:0/2494295934 conn(0x558f306384d0 msgr2=0x558f3063c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 shutdown_connections 2024-04-13T03:12:50.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2494295934 wait complete. 2024-04-13T03:12:50.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.251+0000 7fa2d43ec740 20 client.6564 trim_cache size 0 max 16384 2024-04-13T03:12:50.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.268+0000 7fa2d43ec740 1 Processor -- start 2024-04-13T03:12:50.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.268+0000 7fa2d43ec740 1 -- start start 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f3069c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f3066db10 0x558f3069c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1fee10 con 0x558f30664ff0 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1fdfc0 con 0x558f3066db10 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc1573a0 con 0x558f306653d0 2024-04-13T03:12:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38276/0 (socket says 172.21.15.106:38276) 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 -- 172.21.15.106:0/2896140888 learned_addr learned my addr 172.21.15.106:0/2896140888 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f3069c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f3066db10 0x558f3069c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 -- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 msgr2=0x558f3069c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f3069c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 -- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f3066db10 msgr2=0x558f3069c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f3066db10 0x558f3069c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 -- 172.21.15.106:0/2896140888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3066a7b0 con 0x558f30664ff0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f3069c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2caffd640 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc055a10 tx=0x7fa2bc05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1292280c28ebf34 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2abfff640 1 -- 172.21.15.106:0/2896140888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0090a0 con 0x558f30664ff0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2abfff640 1 -- 172.21.15.106:0/2896140888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc0554f0 con 0x558f30664ff0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.269+0000 7fa2abfff640 1 -- 172.21.15.106:0/2896140888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0556a0 con 0x558f30664ff0 2024-04-13T03:12:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2896140888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 msgr2=0x558f3069d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc055a10 tx=0x7fa2bc05b000 comp rx=0 tx=0).stop 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2896140888 shutdown_connections 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f3069c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f3066db10 0x558f3069c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/2896140888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f30664ff0 0x558f3069d050 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2896140888 >> 172.21.15.106:0/2896140888 conn(0x558f306384d0 msgr2=0x558f30667820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2896140888 shutdown_connections 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- 172.21.15.106:0/2896140888 wait complete. 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 -- start start 2024-04-13T03:12:50.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.270+0000 7fa2d43ec740 1 Processor -- start 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f30664ff0 0x558f306ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f30685f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1579f0 con 0x558f3066db10 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558f3066d920 con 0x558f30664ff0 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558f306fda90 con 0x558f306653d0 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2d43ec740 10 client.0 _ll_register_callbacks cb 0x7fa2bc0596d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38282/0 (socket says 172.21.15.106:38282) 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2cb7fe640 1 -- 172.21.15.106:0/1572356989 learned_addr learned my addr 172.21.15.106:0/1572356989 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2ca7fc640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f30685f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.271+0000 7fa2caffd640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f30664ff0 0x558f306ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 -- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 msgr2=0x558f30685f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f30685f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 -- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f30664ff0 msgr2=0x558f306ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f30664ff0 0x558f306ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 -- 172.21.15.106:0/1572356989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f30637020 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2cb7fe640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005e90 tx=0x7fa2c4013640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1151e659ee01f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c401a070 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fa2c40163f0 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2bc0554f0 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4025050 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 10 client.6570 fetch_fsmap learned FSMap version 29 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x558f306fdc30 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc0554f0 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7fa2c4028020 con 0x558f3066db10 2024-04-13T03:12:50.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 10 client.6570 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 10 client.6570 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 20 client.6570 populate_metadata read hostname 'smithi106' 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f30661750 0x558f306fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 4 client.6570 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f3066d560 con 0x558f30661750 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.272+0000 7fa2caffd640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f30661750 0x558f306fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2a9ffb640 10 client.6570 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2caffd640 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f30661750 0x558f306fcb60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f170b62a77d03194 server_cookie=9597ecd17c1c0e0c in_seq=0 out_seq=0 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2a9ffb640 1 -- 172.21.15.106:0/1572356989 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x558f3066d560 con 0x558f30661750 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2a9ffb640 10 client.6570 handle_command_reply: tid=0 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 client.6570 shutdown 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 msgr2=0x558f30686490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005e90 tx=0x7fa2c4013640 comp rx=0 tx=0).stop 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 shutdown_connections 2024-04-13T03:12:50.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x558f30661750 0x558f306fcb60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558f306653d0 0x558f30685f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558f3066db10 0x558f30686490 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 --2- 172.21.15.106:0/1572356989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f30664ff0 0x558f306ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 >> 172.21.15.106:0/1572356989 conn(0x558f306384d0 msgr2=0x558f30662b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2a9ffb640 0 client.6570 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 shutdown_connections 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 1 -- 172.21.15.106:0/1572356989 wait complete. 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d43ec740 20 client.6570 trim_cache size 0 max 16384 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:12:50.281 INFO:teuthology.orchestra.run.smithi106.stdout: "2884081e-0444-46b9-b7e0-cbc61d97c2c8": { 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/", 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "2884081e-0444-46b9-b7e0-cbc61d97c2c8", 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair,force" 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:12:50.282 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 msgr2=0x7fa2b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.273+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 0x7fa2b806ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0036000 tx=0x7fa2c003d040 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 msgr2=0x7fa2cc148020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00ba80 tx=0x7fa2bc00bf40 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 shutdown_connections 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2b806c920 0x7fa2b806ede0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2cc1593b0 0x7fa2cc14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc15f930 0x7fa2cc150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 --2- 172.21.15.106:0/2531839368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2cc070a30 0x7fa2cc148020 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 >> 172.21.15.106:0/2531839368 conn(0x7fa2cc0d13f0 msgr2=0x7fa2cc15c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:12:50.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 shutdown_connections 2024-04-13T03:12:50.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:12:50.274+0000 7fa2d151e640 1 -- 172.21.15.106:0/2531839368 wait complete. 2024-04-13T03:12:50.286 INFO:tasks.cephfs.filesystem:scrub status for tag:2884081e-0444-46b9-b7e0-cbc61d97c2c8 - {'path': '/', 'tag': '2884081e-0444-46b9-b7e0-cbc61d97c2c8', 'options': 'recursive,repair,force'} 2024-04-13T03:12:54.846 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:12:54.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:13:20.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:20.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-13T03:13:20.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.400+0000 7f57efda5640 1 Processor -- start 2024-04-13T03:13:20.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.400+0000 7f57efda5640 1 -- start start 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e815adb0 0x7f57e806d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f57e806fc50 0x7f57e80700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f57e80cf170 0x7f57e80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f57e815b190 con 0x7f57e806fc50 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f57e806c170 con 0x7f57e80cf170 2024-04-13T03:13:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57efda5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57e806c3c0 con 0x7f57e815adb0 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ed340640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f57e806fc50 0x7f57e80700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ed340640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f57e806fc50 0x7f57e80700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50376/0 (socket says 172.21.15.106:50376) 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ed340640 1 -- 172.21.15.106:0/2444285424 learned_addr learned my addr 172.21.15.106:0/2444285424 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57edb41640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e815adb0 0x7f57e806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ee342640 1 --2- 172.21.15.106:0/2444285424 >> v2:172.21.15.114:3300/0 conn(0x7f57e80cf170 0x7f57e80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ed340640 1 -- 172.21.15.106:0/2444285424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e815adb0 msgr2=0x7f57e806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.401+0000 7f57ed340640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e815adb0 0x7f57e806d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ed340640 1 -- 172.21.15.106:0/2444285424 >> v2:172.21.15.114:3300/0 conn(0x7f57e80cf170 msgr2=0x7f57e80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ed340640 1 --2- 172.21.15.106:0/2444285424 >> v2:172.21.15.114:3300/0 conn(0x7f57e80cf170 0x7f57e80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ed340640 1 -- 172.21.15.106:0/2444285424 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e806c6a0 con 0x7f57e806fc50 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ed340640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e80700b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46242d344997fb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ecb3f640 1 -- 172.21.15.106:0/2444285424 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8035020 con 0x7f57e806fc50 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ecb3f640 1 -- 172.21.15.106:0/2444285424 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8031640 con 0x7f57e806fc50 2024-04-13T03:13:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.402+0000 7f57ecb3f640 1 -- 172.21.15.106:0/2444285424 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d802d040 con 0x7f57e806fc50 2024-04-13T03:13:20.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 -- 172.21.15.106:0/2444285424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 msgr2=0x7f57e80700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e80700b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 -- 172.21.15.106:0/2444285424 shutdown_connections 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 --2- 172.21.15.106:0/2444285424 >> v2:172.21.15.114:3300/0 conn(0x7f57e80cf170 0x7f57e80cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e80700b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 --2- 172.21.15.106:0/2444285424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e815adb0 0x7f57e806d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 -- 172.21.15.106:0/2444285424 >> 172.21.15.106:0/2444285424 conn(0x7f57e80cd6a0 msgr2=0x7f57e80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 -- 172.21.15.106:0/2444285424 shutdown_connections 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.403+0000 7f57efda5640 1 -- 172.21.15.106:0/2444285424 wait complete. 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.404+0000 7f57efda5640 1 Processor -- start 2024-04-13T03:13:20.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.404+0000 7f57efda5640 1 -- start start 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e80cf170 0x7f57e80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e815adb0 0x7f57e8149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57e806cf40 con 0x7f57e806fc50 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e806cc60 con 0x7f57e815adb0 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57efda5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57e806d9a0 con 0x7f57e80cf170 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50378/0 (socket says 172.21.15.106:50378) 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 -- 172.21.15.106:0/3002409476 learned_addr learned my addr 172.21.15.106:0/3002409476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57ed340640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e80cf170 0x7f57e80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57ee342640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e815adb0 0x7f57e8149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 -- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e80cf170 msgr2=0x7f57e80ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e80cf170 0x7f57e80ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 -- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e815adb0 msgr2=0x7f57e8149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e815adb0 0x7f57e8149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.405+0000 7f57edb41640 1 -- 172.21.15.106:0/3002409476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e81482a0 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57edb41640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57e400d700 tx=0x7f57e400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57de7fc640 1 -- 172.21.15.106:0/3002409476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4015070 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57de7fc640 1 -- 172.21.15.106:0/3002409476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e4007900 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e8148580 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57de7fc640 1 -- 172.21.15.106:0/3002409476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e400b040 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.406+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e8148b40 con 0x7f57e806fc50 2024-04-13T03:13:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.407+0000 7f57de7fc640 1 -- 172.21.15.106:0/3002409476 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57e4018070 con 0x7f57e806fc50 2024-04-13T03:13:20.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.408+0000 7f57de7fc640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 0x7f57cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.408+0000 7f57de7fc640 1 -- 172.21.15.106:0/3002409476 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f57e408bd80 con 0x7f57e806fc50 2024-04-13T03:13:20.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.408+0000 7f57ed340640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 0x7f57cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.408+0000 7f57ed340640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 0x7f57cc06edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57d8036000 tx=0x7f57d803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.409+0000 7f57f2b8f740 1 Processor -- start 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.409+0000 7f57f2b8f740 1 -- start start 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213033d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559212f6f170 con 0x5592130044d0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559212f157f0 con 0x559213030ff0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x559212f6b130 con 0x5592130313d0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50392/0 (socket says 172.21.15.106:50392) 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 -- 172.21.15.106:0/791404706 learned_addr learned my addr 172.21.15.106:0/791404706 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ed340640 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213033d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57edb41640 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 -- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 msgr2=0x559213033d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213033d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 -- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 msgr2=0x559213003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 -- 172.21.15.106:0/791404706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559212d4b7d0 con 0x5592130044d0 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57ee342640 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009eb0 tx=0x7f57e0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6b219b552b5abc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57cb7fe640 1 -- 172.21.15.106:0/791404706 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e00047d0 con 0x5592130044d0 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57cb7fe640 1 -- 172.21.15.106:0/791404706 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e001a070 con 0x5592130044d0 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.410+0000 7f57cb7fe640 1 -- 172.21.15.106:0/791404706 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0015640 con 0x5592130044d0 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 -- 172.21.15.106:0/791404706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 msgr2=0x5592130342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009eb0 tx=0x7f57e0003d60 comp rx=0 tx=0).stop 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 -- 172.21.15.106:0/791404706 shutdown_connections 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213033d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130044d0 0x5592130342c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/791404706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 -- 172.21.15.106:0/791404706 >> 172.21.15.106:0/791404706 conn(0x559213006200 msgr2=0x55921304ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 -- 172.21.15.106:0/791404706 shutdown_connections 2024-04-13T03:13:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.411+0000 7f57f2b8f740 1 -- 172.21.15.106:0/791404706 wait complete. 2024-04-13T03:13:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.412+0000 7f57f2b8f740 1 -- start start 2024-04-13T03:13:20.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.412+0000 7f57f2b8f740 1 Processor -- start 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213036540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5592130088a0 con 0x559213062a90 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559212f28d00 con 0x559213030ff0 2024-04-13T03:13:20.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55921300b0e0 con 0x5592130313d0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57f2b8f740 10 client.0 _ll_register_callbacks cb 0x559212f286f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50402/0 (socket says 172.21.15.106:50402) 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 -- 172.21.15.106:0/87888696 learned_addr learned my addr 172.21.15.106:0/87888696 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ed340640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213036540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57edb41640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 -- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 msgr2=0x559213036540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213036540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 -- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 msgr2=0x559213036000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 -- 172.21.15.106:0/87888696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559212c813f0 con 0x559213062a90 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57ee342640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f57e0016000 tx=0x7f57e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b60339007e5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.413+0000 7f57c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0022070 con 0x559213062a90 2024-04-13T03:13:20.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f57e00159e0 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e001a070 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57e00048a0 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 10 client.6582 fetch_fsmap learned FSMap version 30 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x559212c410f0 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e001a070 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7f57e0015be0 con 0x559213062a90 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 10 client.6582 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 10 client.6582 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 20 client.6582 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213020d60 0x559213021170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 4 client.6582 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559212f7cb90 con 0x559213020d60 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57edb41640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213020d60 0x559213021170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.414+0000 7f57c8ff9640 10 client.6582 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57edb41640 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213020d60 0x559213021170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dc3de85d5a3fde server_cookie=be32775e4daead1a in_seq=0 out_seq=0 2024-04-13T03:13:20.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57c8ff9640 1 -- 172.21.15.106:0/87888696 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559212f7cb90 con 0x559213020d60 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57c8ff9640 10 client.6582 handle_command_reply: tid=0 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 client.6582 shutdown 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 msgr2=0x559213062f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f57e0016000 tx=0x7f57e0004ab0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 shutdown_connections 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213020d60 0x559213021170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592130313d0 0x559213036540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57c8ff9640 0 client.6582 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559213062a90 0x559213062f10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/87888696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 >> 172.21.15.106:0/87888696 conn(0x5592130044d0 msgr2=0x559213008010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 shutdown_connections 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.415+0000 7f57f2b8f740 1 -- 172.21.15.106:0/87888696 wait complete. 2024-04-13T03:13:20.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.416+0000 7f57f2b8f740 20 client.6582 trim_cache size 0 max 16384 2024-04-13T03:13:20.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.433+0000 7f57f2b8f740 1 Processor -- start 2024-04-13T03:13:20.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.433+0000 7f57f2b8f740 1 -- start start 2024-04-13T03:13:20.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x559213062e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x55921303a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81ff6d0 con 0x5592130313d0 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81fe6b0 con 0x559213030ff0 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57e814ad30 con 0x559213020d60 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50416/0 (socket says 172.21.15.106:50416) 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 -- 172.21.15.106:0/1364751929 learned_addr learned my addr 172.21.15.106:0/1364751929 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57edb41640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x559213062e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ed340640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x55921303a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 -- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 msgr2=0x559213062e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x559213062e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 -- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 msgr2=0x55921303a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x55921303a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 -- 172.21.15.106:0/1364751929 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559213033f90 con 0x5592130313d0 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ed340640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x55921303a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57ee342640 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f57e0003da0 tx=0x7f57e001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a266d4ef90ccb60 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57caffd640 1 -- 172.21.15.106:0/1364751929 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0038070 con 0x5592130313d0 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57caffd640 1 -- 172.21.15.106:0/1364751929 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e001a070 con 0x5592130313d0 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.434+0000 7f57caffd640 1 -- 172.21.15.106:0/1364751929 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0022040 con 0x5592130313d0 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1364751929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 msgr2=0x55921303ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f57e0003da0 tx=0x7f57e001f000 comp rx=0 tx=0).stop 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1364751929 shutdown_connections 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921303ac00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x55921303a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1364751929 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x559213062e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1364751929 >> 172.21.15.106:0/1364751929 conn(0x5592130044d0 msgr2=0x559213033880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1364751929 shutdown_connections 2024-04-13T03:13:20.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.435+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1364751929 wait complete. 2024-04-13T03:13:20.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.436+0000 7f57f2b8f740 1 -- start start 2024-04-13T03:13:20.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.436+0000 7f57f2b8f740 1 Processor -- start 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x55921303ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5592130348e0 con 0x5592130313d0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55921304fa10 con 0x559213030ff0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x559212ffff10 con 0x559213020d60 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 10 client.0 _ll_register_callbacks cb 0x7f57e00047d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50422/0 (socket says 172.21.15.106:50422) 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 -- 172.21.15.106:0/1110522792 learned_addr learned my addr 172.21.15.106:0/1110522792 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57edb41640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x55921303ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ed340640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 -- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 msgr2=0x55921303ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x55921303ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 -- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 msgr2=0x559213036560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 -- 172.21.15.106:0/1110522792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592130cc260 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57ee342640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009eb0 tx=0x7f57e00096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cc3812d5bd0c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0022070 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f57e0027e00 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e001a070 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57e0043050 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 10 client.6588 fetch_fsmap learned FSMap version 30 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.437+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x559213005400 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e001a070 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7f57e0046020 con 0x5592130313d0 2024-04-13T03:13:20.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 10 client.6588 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 10 client.6588 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 20 client.6588 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213062a90 0x55921305d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 4 client.6588 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559212f76a00 con 0x559213062a90 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57edb41640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213062a90 0x55921305d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57c8ff9640 10 client.6588 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57edb41640 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213062a90 0x55921305d9e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4435588703e48a4 server_cookie=381da704e50a934e in_seq=0 out_seq=0 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57c8ff9640 1 -- 172.21.15.106:0/1110522792 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559212f76a00 con 0x559213062a90 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57c8ff9640 10 client.6588 handle_command_reply: tid=0 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 1 client.6588 shutdown 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.438+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 msgr2=0x55921305d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009eb0 tx=0x7f57e00096f0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 shutdown_connections 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x559213062a90 0x55921305d9e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592130313d0 0x55921305d4a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559213030ff0 0x559213036560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57c8ff9640 0 client.6588 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 --2- 172.21.15.106:0/1110522792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559213020d60 0x55921303ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 >> 172.21.15.106:0/1110522792 conn(0x5592130044d0 msgr2=0x55921302f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 shutdown_connections 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 1 -- 172.21.15.106:0/1110522792 wait complete. 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57f2b8f740 20 client.6588 trim_cache size 0 max 16384 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:13:20.446 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:20.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 msgr2=0x7f57cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 0x7f57cc06edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57d8036000 tx=0x7f57d803c040 comp rx=0 tx=0).stop 2024-04-13T03:13:20.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 msgr2=0x7f57e8147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57e400d700 tx=0x7f57e400dbc0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 shutdown_connections 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57cc06c8f0 0x7f57cc06edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57e80cf170 0x7f57e80ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e815adb0 0x7f57e8149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 --2- 172.21.15.106:0/3002409476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e806fc50 0x7f57e8147610 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 >> 172.21.15.106:0/3002409476 conn(0x7f57e80cd6a0 msgr2=0x7f57e8157120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 shutdown_connections 2024-04-13T03:13:20.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.439+0000 7f57efda5640 1 -- 172.21.15.106:0/3002409476 wait complete. 2024-04-13T03:13:20.451 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:13:20.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:20.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.550+0000 7f1333755640 1 Processor -- start 2024-04-13T03:13:20.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.550+0000 7f1333755640 1 -- start start 2024-04-13T03:13:20.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1333755640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 0x7f132c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1333755640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c0d3530 con 0x7f132c0b8080 2024-04-13T03:13:20.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1333755640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c0d0b60 con 0x7f132c1622d0 2024-04-13T03:13:20.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1333755640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f132c1626b0 con 0x7f132c0d1c00 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1331cf2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f132c0b8080 0x7f132c1688e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59638/0 (socket says 172.21.15.106:59638) 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1331cf2640 1 -- 172.21.15.106:0/3248176332 learned_addr learned my addr 172.21.15.106:0/3248176332 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1330cf0640 1 --2- 172.21.15.106:0/3248176332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 0x7f132c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432757250 0 0) 0x7f132c0d3530 con 0x7f132c0b8080 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1314003520 con 0x7f132c0b8080 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f1330cf0640 1 -- 172.21.15.106:0/3248176332 >> v1:172.21.15.114:6789/0 conn(0x7f132c1622d0 legacy=0x7f132c0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f1330cf0640 1 -- 172.21.15.106:0/3248176332 >> v1:172.21.15.106:6789/0 conn(0x7f132c0b8080 legacy=0x7f132c1688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.551+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362784570 0 0) 0x7f132c0d0b60 con 0x7f132c1622d0 2024-04-13T03:13:20.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f1330cf0640 1 -- 172.21.15.106:0/3248176332 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c0d0e40 con 0x7f132c0d1c00 2024-04-13T03:13:20.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f1330cf0640 1 --2- 172.21.15.106:0/3248176332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 0x7f132c0d04d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13200097d0 tx=0x7f132002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9569d07656705f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320035020 con 0x7f132c0d1c00 2024-04-13T03:13:20.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320031890 con 0x7f132c0d1c00 2024-04-13T03:13:20.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.552+0000 7f132bfff640 1 -- 172.21.15.106:0/3248176332 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320031c40 con 0x7f132c0d1c00 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 -- 172.21.15.106:0/3248176332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 msgr2=0x7f132c0d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 --2- 172.21.15.106:0/3248176332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 0x7f132c0d04d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13200097d0 tx=0x7f132002b930 comp rx=0 tx=0).stop 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 -- 172.21.15.106:0/3248176332 shutdown_connections 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 --2- 172.21.15.106:0/3248176332 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c0d1c00 0x7f132c0d04d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 -- 172.21.15.106:0/3248176332 >> 172.21.15.106:0/3248176332 conn(0x7f132c0d25d0 msgr2=0x7f132c0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 -- 172.21.15.106:0/3248176332 shutdown_connections 2024-04-13T03:13:20.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.553+0000 7f1333755640 1 -- 172.21.15.106:0/3248176332 wait complete. 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.554+0000 7f1333755640 1 Processor -- start 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.554+0000 7f1333755640 1 -- start start 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f132c0d1c00 0x7f132c1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 0x7f132c1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f132c0d16e0 con 0x7f132c0b8080 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f132c0d1400 con 0x7f132c0d1c00 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1333755640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f132c0cc6f0 con 0x7f132c1622d0 2024-04-13T03:13:20.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50438/0 (socket says 172.21.15.106:50438) 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 -- 172.21.15.106:0/3568482694 learned_addr learned my addr 172.21.15.106:0/3568482694 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1330cf0640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f132c0d1c00 0x7f132c1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1331cf2640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 0x7f132c1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 -- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 msgr2=0x7f132c1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 0x7f132c1f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 -- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f132c0d1c00 msgr2=0x7f132c1f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f132c0d1c00 0x7f132c1f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 -- 172.21.15.106:0/3568482694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c0c97d0 con 0x7f132c0b8080 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f1331cf2640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 0x7f132c1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.555+0000 7f13314f1640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f131c00c7f0 tx=0x7f131c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.556+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c017070 con 0x7f132c0b8080 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.556+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c003da0 con 0x7f132c0b8080 2024-04-13T03:13:20.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.556+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c0c9ab0 con 0x7f132c0b8080 2024-04-13T03:13:20.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.556+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c005290 con 0x7f132c0b8080 2024-04-13T03:13:20.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.556+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c0c83f0 con 0x7f132c0b8080 2024-04-13T03:13:20.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.557+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c068a10 con 0x7f132c0b8080 2024-04-13T03:13:20.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.557+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f131c01a070 con 0x7f132c0b8080 2024-04-13T03:13:20.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.557+0000 7f1329ffb640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 0x7f131806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.557+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f131c08b350 con 0x7f132c0b8080 2024-04-13T03:13:20.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.558+0000 7f1330cf0640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 0x7f131806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.558+0000 7f1330cf0640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 0x7f131806edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1320036000 tx=0x7f1320008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.562+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f131c0595f0 con 0x7f132c0b8080 2024-04-13T03:13:20.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.774+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f132c0b6560 con 0x7f132c0b8080 2024-04-13T03:13:20.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.774+0000 7f1329ffb640 1 -- 172.21.15.106:0/3568482694 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5186 (secure 0 0 0) 0x7f131c058ef0 con 0x7f132c0b8080 2024-04-13T03:13:20.775 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:20.775 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:52.017572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:13:20.775 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 30 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 msgr2=0x7f131806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 0x7f131806edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1320036000 tx=0x7f1320008030 comp rx=0 tx=0).stop 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 msgr2=0x7f132c15dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f131c00c7f0 tx=0x7f131c00ccb0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 shutdown_connections 2024-04-13T03:13:20.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f131806c8f0 0x7f131806edb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f132c1622d0 0x7f132c1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f132c0d1c00 0x7f132c1f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 --2- 172.21.15.106:0/3568482694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f132c0b8080 0x7f132c15dba0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 >> 172.21.15.106:0/3568482694 conn(0x7f132c0d25d0 msgr2=0x7f132c0ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 shutdown_connections 2024-04-13T03:13:20.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.776+0000 7f1333755640 1 -- 172.21.15.106:0/3568482694 wait complete. 2024-04-13T03:13:20.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-13T03:13:20.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.888+0000 7fde578c8640 1 Processor -- start 2024-04-13T03:13:20.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.888+0000 7fde578c8640 1 -- start start 2024-04-13T03:13:20.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 0x7fde50160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde500b81f0 0x7fde500ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fde500cd2f0 0x7fde5006f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fde500b54c0 con 0x7fde500cd2f0 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fde50071320 con 0x7fde5015cb00 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde578c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fde50070d60 con 0x7fde500b81f0 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde55e65640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fde500cd2f0 0x7fde5006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde55e65640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fde500cd2f0 0x7fde5006f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50454/0 (socket says 172.21.15.106:50454) 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.889+0000 7fde55e65640 1 -- 172.21.15.106:0/555222493 learned_addr learned my addr 172.21.15.106:0/555222493 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde54e63640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde500b81f0 0x7fde500ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55664640 1 --2- 172.21.15.106:0/555222493 >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 0x7fde50160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 -- 172.21.15.106:0/555222493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde500b81f0 msgr2=0x7fde500ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde500b81f0 0x7fde500ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 -- 172.21.15.106:0/555222493 >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 msgr2=0x7fde50160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 --2- 172.21.15.106:0/555222493 >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 0x7fde50160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 -- 172.21.15.106:0/555222493 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde50071040 con 0x7fde500cd2f0 2024-04-13T03:13:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55664640 1 --2- 172.21.15.106:0/555222493 >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 0x7fde50160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.890+0000 7fde55e65640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500cd2f0 0x7fde5006f920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fde500ccd80 tx=0x7fde4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67ed751c8dc4e27 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde47fff640 1 -- 172.21.15.106:0/555222493 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48035020 con 0x7fde500cd2f0 2024-04-13T03:13:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde47fff640 1 -- 172.21.15.106:0/555222493 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48031870 con 0x7fde500cd2f0 2024-04-13T03:13:20.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde47fff640 1 -- 172.21.15.106:0/555222493 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48031c40 con 0x7fde500cd2f0 2024-04-13T03:13:20.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 -- 172.21.15.106:0/555222493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500cd2f0 msgr2=0x7fde5006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500cd2f0 0x7fde5006f920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fde500ccd80 tx=0x7fde4802b910 comp rx=0 tx=0).stop 2024-04-13T03:13:20.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 -- 172.21.15.106:0/555222493 shutdown_connections 2024-04-13T03:13:20.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500cd2f0 0x7fde5006f920 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 --2- 172.21.15.106:0/555222493 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde500b81f0 0x7fde500ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 --2- 172.21.15.106:0/555222493 >> v2:172.21.15.114:3300/0 conn(0x7fde5015cb00 0x7fde50160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 -- 172.21.15.106:0/555222493 >> 172.21.15.106:0/555222493 conn(0x7fde500d2530 msgr2=0x7fde500d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 -- 172.21.15.106:0/555222493 shutdown_connections 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.891+0000 7fde578c8640 1 -- 172.21.15.106:0/555222493 wait complete. 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.892+0000 7fde578c8640 1 Processor -- start 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.892+0000 7fde578c8640 1 -- start start 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 0x7fde501f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde5015cb00 0x7fde500c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde5006fe60 con 0x7fde500b81f0 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde500d07b0 con 0x7fde500cd2f0 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde578c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fde5006ffe0 con 0x7fde5015cb00 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50464/0 (socket says 172.21.15.106:50464) 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 -- 172.21.15.106:0/2581240850 learned_addr learned my addr 172.21.15.106:0/2581240850 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde54e63640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 0x7fde501f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55e65640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde5015cb00 0x7fde500c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 -- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde5015cb00 msgr2=0x7fde500c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde5015cb00 0x7fde500c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 -- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 msgr2=0x7fde501f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 0x7fde501f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde55664640 1 -- 172.21.15.106:0/2581240850 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde501f6910 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.893+0000 7fde54e63640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 0x7fde501f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde55664640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00ce00 tx=0x7fde4c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde45ffb640 1 -- 172.21.15.106:0/2581240850 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00f890 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde45ffb640 1 -- 172.21.15.106:0/2581240850 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c00fa30 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde501f6bf0 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde45ffb640 1 -- 172.21.15.106:0/2581240850 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00fe40 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.894+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde500cb150 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.896+0000 7fde45ffb640 1 -- 172.21.15.106:0/2581240850 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde4c018400 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.896+0000 7fde45ffb640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 0x7fde3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.896+0000 7fde45ffb640 1 -- 172.21.15.106:0/2581240850 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fde4c08cb40 con 0x7fde500b81f0 2024-04-13T03:13:20.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.896+0000 7fde54e63640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 0x7fde3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.896+0000 7fde54e63640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 0x7fde3406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde500c9060 tx=0x7fde40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.897+0000 7fde5a785740 1 Processor -- start 2024-04-13T03:13:20.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.897+0000 7fde5a785740 1 -- start start 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ebec624fe0 con 0x55ebec6bf800 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ebec5c1860 con 0x55ebec6ea6d0 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ebec620fa0 con 0x55ebec6e7af0 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50478/0 (socket says 172.21.15.106:50478) 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 -- 172.21.15.106:0/754705431 learned_addr learned my addr 172.21.15.106:0/754705431 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde54e63640 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55664640 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 -- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 msgr2=0x55ebec6eaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6eaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 -- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 msgr2=0x55ebec6c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6c1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 -- 172.21.15.106:0/754705431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebec5f01a0 con 0x55ebec6bf800 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde55e65640 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fde4802b950 tx=0x7fde48008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f544e527286580 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.898+0000 7fde3affd640 1 -- 172.21.15.106:0/754705431 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48031a70 con 0x55ebec6bf800 2024-04-13T03:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde3affd640 1 -- 172.21.15.106:0/754705431 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48049070 con 0x55ebec6bf800 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde3affd640 1 -- 172.21.15.106:0/754705431 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48031e10 con 0x55ebec6bf800 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 -- 172.21.15.106:0/754705431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 msgr2=0x55ebec6e6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fde4802b950 tx=0x7fde48008030 comp rx=0 tx=0).stop 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 -- 172.21.15.106:0/754705431 shutdown_connections 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6eaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec6e6f90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 --2- 172.21.15.106:0/754705431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 -- 172.21.15.106:0/754705431 >> 172.21.15.106:0/754705431 conn(0x55ebec653dc0 msgr2=0x55ebec7054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 -- 172.21.15.106:0/754705431 shutdown_connections 2024-04-13T03:13:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.899+0000 7fde5a785740 1 -- 172.21.15.106:0/754705431 wait complete. 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.900+0000 7fde5a785740 1 -- start start 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.900+0000 7fde5a785740 1 Processor -- start 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec707660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ebec6bab00 con 0x55ebec6e7af0 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ebec5bf1d0 con 0x55ebec6bf800 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ebec705c90 con 0x55ebec6ea6d0 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 10 client.0 _ll_register_callbacks cb 0x55ebec5e2010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50484/0 (socket says 172.21.15.106:50484) 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 -- 172.21.15.106:0/3286137954 learned_addr learned my addr 172.21.15.106:0/3286137954 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde55664640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde55e65640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec707660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 -- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 msgr2=0x55ebec707660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec707660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 -- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 msgr2=0x55ebec6dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 -- 172.21.15.106:0/3286137954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebec4037c0 con 0x55ebec6e7af0 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde55664640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde54e63640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fde501f5e60 tx=0x7fde40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f041dd5d2ad084 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40025020 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fde40021830 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde48049070 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde4002f050 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 10 client.6606 fetch_fsmap learned FSMap version 30 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.901+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55ebec463f50 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48049070 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7fde40032020 con 0x55ebec6e7af0 2024-04-13T03:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 10 client.6606 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 10 client.6606 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 20 client.6606 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec715a20 0x55ebec715e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 4 client.6606 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebec5c44a0 con 0x55ebec715a20 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde55664640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec715a20 0x55ebec715e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde2ffff640 10 client.6606 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.902+0000 7fde55664640 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec715a20 0x55ebec715e00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643c0d29b419b98e server_cookie=4d0b5f84172b7dc8 in_seq=0 out_seq=0 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde2ffff640 1 -- 172.21.15.106:0/3286137954 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebec5c44a0 con 0x55ebec715a20 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde2ffff640 10 client.6606 handle_command_reply: tid=0 2024-04-13T03:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 client.6606 shutdown 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 msgr2=0x55ebec6e3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fde501f5e60 tx=0x7fde40004730 comp rx=0 tx=0).stop 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 shutdown_connections 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec715a20 0x55ebec715e00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec707660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6e3030 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 --2- 172.21.15.106:0/3286137954 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde2ffff640 0 client.6606 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 >> 172.21.15.106:0/3286137954 conn(0x55ebec653dc0 msgr2=0x55ebec6b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 shutdown_connections 2024-04-13T03:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 1 -- 172.21.15.106:0/3286137954 wait complete. 2024-04-13T03:13:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.903+0000 7fde5a785740 20 client.6606 trim_cache size 0 max 16384 2024-04-13T03:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.916+0000 7fde5a785740 1 Processor -- start 2024-04-13T03:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.916+0000 7fde5a785740 1 -- start start 2024-04-13T03:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec6ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde501fe560 con 0x55ebec6e7af0 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde500cc120 con 0x55ebec6bf800 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fde500cc770 con 0x55ebec6ea6d0 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50492/0 (socket says 172.21.15.106:50492) 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 -- 172.21.15.106:0/521154150 learned_addr learned my addr 172.21.15.106:0/521154150 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 -- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 msgr2=0x55ebec6ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec6ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 -- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 msgr2=0x55ebec6e6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 -- 172.21.15.106:0/521154150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebec2f70b0 con 0x55ebec6e7af0 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde54e63640 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde4000fc30 tx=0x7fde40020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6578b3021ad84c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde3a7fc640 1 -- 172.21.15.106:0/521154150 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40037070 con 0x55ebec6e7af0 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde3a7fc640 1 -- 172.21.15.106:0/521154150 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4001c040 con 0x55ebec6e7af0 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.917+0000 7fde3a7fc640 1 -- 172.21.15.106:0/521154150 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40025050 con 0x55ebec6e7af0 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 -- 172.21.15.106:0/521154150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 msgr2=0x55ebec6de850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde4000fc30 tx=0x7fde40020000 comp rx=0 tx=0).stop 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 -- 172.21.15.106:0/521154150 shutdown_connections 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6ea6d0 0x55ebec6ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6e7af0 0x55ebec6de850 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 --2- 172.21.15.106:0/521154150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6bf800 0x55ebec6e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 -- 172.21.15.106:0/521154150 >> 172.21.15.106:0/521154150 conn(0x55ebec653dc0 msgr2=0x55ebec6ee480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 -- 172.21.15.106:0/521154150 shutdown_connections 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.918+0000 7fde5a785740 1 -- 172.21.15.106:0/521154150 wait complete. 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 -- start start 2024-04-13T03:13:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 Processor -- start 2024-04-13T03:13:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ebec6e5e70 con 0x55ebec6bf800 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ebec6e5480 con 0x55ebec6ea6d0 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.919+0000 7fde5a785740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ebec6e1b20 con 0x55ebec6e7af0 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 10 client.0 _ll_register_callbacks cb 0x7fde4000fcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50508/0 (socket says 172.21.15.106:50508) 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 -- 172.21.15.106:0/4107745457 learned_addr learned my addr 172.21.15.106:0/4107745457 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde54e63640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55e65640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 -- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 msgr2=0x55ebec6e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 -- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 msgr2=0x55ebec6e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 -- 172.21.15.106:0/4107745457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebec6bfd80 con 0x55ebec6bf800 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55e65640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde55664640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fde4c05a690 tx=0x7fde4c05a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47351bd8c2d55bf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c089060 con 0x55ebec6bf800 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fde4c08aad0 con 0x55ebec6bf800 2024-04-13T03:13:20.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4001c040 con 0x55ebec6bf800 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde4c07e050 con 0x55ebec6bf800 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 10 client.6612 fetch_fsmap learned FSMap version 30 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55ebec77f1e0 con 0x55ebec6bf800 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4001c040 con 0x55ebec6bf800 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7fde4c081020 con 0x55ebec6bf800 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 10 client.6612 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 10 client.6612 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 20 client.6612 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec707340 0x55ebec713730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 4 client.6612 mds_command: new command op to 6273 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.920+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 --> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ebec6bbc90 con 0x55ebec707340 2024-04-13T03:13:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde54e63640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec707340 0x55ebec713730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde2ffff640 10 client.6612 ms_handle_connect on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde54e63640 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec707340 0x55ebec713730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a901e4aad2cf5e server_cookie=b5d31682138bcf46 in_seq=0 out_seq=0 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde2ffff640 1 -- 172.21.15.106:0/4107745457 <== mds.0 v2:172.21.15.106:6834/536113082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebec6bbc90 con 0x55ebec707340 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde2ffff640 10 client.6612 handle_command_reply: tid=0 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 client.6612 shutdown 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 msgr2=0x55ebec77c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fde4c05a690 tx=0x7fde4c05a4d0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 shutdown_connections 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:6834/536113082,v1:172.21.15.106:6836/536113082] conn(0x55ebec707340 0x55ebec713730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ebec6e7af0 0x55ebec6e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ebec6ea6d0 0x55ebec6e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 --2- 172.21.15.106:0/4107745457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ebec6bf800 0x55ebec77c530 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde2ffff640 0 client.6612 ms_handle_reset on v2:172.21.15.106:6834/536113082 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 >> 172.21.15.106:0/4107745457 conn(0x55ebec653dc0 msgr2=0x55ebec6deaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 shutdown_connections 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 1 -- 172.21.15.106:0/4107745457 wait complete. 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.921+0000 7fde5a785740 20 client.6612 trim_cache size 0 max 16384 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:20.931 INFO:teuthology.orchestra.run.smithi106.stdout:[] 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 msgr2=0x7fde3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 0x7fde3406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde500c9060 tx=0x7fde40008040 comp rx=0 tx=0).stop 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 msgr2=0x7fde5014e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00ce00 tx=0x7fde4c00a650 comp rx=0 tx=0).stop 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 shutdown_connections 2024-04-13T03:13:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3406c8f0 0x7fde3406edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde5015cb00 0x7fde500c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde500cd2f0 0x7fde501f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 --2- 172.21.15.106:0/2581240850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde500b81f0 0x7fde5014e900 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 >> 172.21.15.106:0/2581240850 conn(0x7fde500d2530 msgr2=0x7fde500b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 shutdown_connections 2024-04-13T03:13:20.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:20.922+0000 7fde578c8640 1 -- 172.21.15.106:0/2581240850 wait complete. 2024-04-13T03:13:20.934 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:13:21.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.032+0000 7f8445676640 1 Processor -- start 2024-04-13T03:13:21.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.032+0000 7f8445676640 1 -- start start 2024-04-13T03:13:21.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f8445676640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f84400ba290 0x7f84400cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f8445676640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84400cf4b0 con 0x7f84400d22d0 2024-04-13T03:13:21.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f8445676640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84400b7830 con 0x7f84400d2dc0 2024-04-13T03:13:21.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f8445676640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f84400ba6e0 con 0x7f84400ba290 2024-04-13T03:13:21.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f84400ba290 0x7f84400cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f84400d22d0 0x7f84400d26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59642/0 (socket says 172.21.15.106:59642) 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843effd640 1 -- 172.21.15.106:0/3855985646 learned_addr learned my addr 172.21.15.106:0/3855985646 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843f7fe640 1 -- 172.21.15.106:0/3855985646 >> v1:172.21.15.114:6789/0 conn(0x7f84400d2dc0 legacy=0x7f84401663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843f7fe640 1 -- 172.21.15.106:0/3855985646 >> v1:172.21.15.106:6789/0 conn(0x7f84400d22d0 legacy=0x7f84400d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843f7fe640 1 -- 172.21.15.106:0/3855985646 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84400cfe40 con 0x7f84400ba290 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.033+0000 7f843dffb640 1 -- 172.21.15.106:0/3855985646 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978322361 0 0) 0x7f84400cf4b0 con 0x7f84400d22d0 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.034+0000 7f843f7fe640 1 --2- 172.21.15.106:0/3855985646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f84400cf6f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f84380097a0 tx=0x7f843802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad7a322bad1854f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.034+0000 7f843dffb640 1 -- 172.21.15.106:0/3855985646 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035020 con 0x7f84400ba290 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.034+0000 7f843dffb640 1 -- 172.21.15.106:0/3855985646 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8438031830 con 0x7f84400ba290 2024-04-13T03:13:21.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.034+0000 7f843dffb640 1 -- 172.21.15.106:0/3855985646 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438031c00 con 0x7f84400ba290 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 -- 172.21.15.106:0/3855985646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 msgr2=0x7f84400cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 --2- 172.21.15.106:0/3855985646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f84400cf6f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f84380097a0 tx=0x7f843802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 -- 172.21.15.106:0/3855985646 shutdown_connections 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 --2- 172.21.15.106:0/3855985646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f84400cf6f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 -- 172.21.15.106:0/3855985646 >> 172.21.15.106:0/3855985646 conn(0x7f84400d4700 msgr2=0x7f84400d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 -- 172.21.15.106:0/3855985646 shutdown_connections 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.035+0000 7f8445676640 1 -- 172.21.15.106:0/3855985646 wait complete. 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.036+0000 7f8445676640 1 Processor -- start 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.036+0000 7f8445676640 1 -- start start 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 0x7f84400cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84400d2dc0 0x7f84400b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84400d3dd0 con 0x7f84400ba290 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84400cf1d0 con 0x7f84400d2dc0 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f8445676640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84400d42f0 con 0x7f84400d22d0 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50522/0 (socket says 172.21.15.106:50522) 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 -- 172.21.15.106:0/2439575967 learned_addr learned my addr 172.21.15.106:0/2439575967 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843e7fc640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 0x7f84400cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843f7fe640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84400d2dc0 0x7f84400b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 -- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 msgr2=0x7f84400cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 0x7f84400cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 -- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84400d2dc0 msgr2=0x7f84400b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84400d2dc0 0x7f84400b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 -- 172.21.15.106:0/2439575967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401f8e90 con 0x7f84400ba290 2024-04-13T03:13:21.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843e7fc640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 0x7f84400cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.037+0000 7f843effd640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f843000c8c0 tx=0x7f843000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.038+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430018070 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.038+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843000a640 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.038+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8440160b90 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.038+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000aa30 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.038+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8440161150 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.039+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440068a10 con 0x7f84400ba290 2024-04-13T03:13:21.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.039+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f843001b070 con 0x7f84400ba290 2024-04-13T03:13:21.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.040+0000 7f84277fe640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.040+0000 7f843e7fc640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.040+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f843008b3c0 con 0x7f84400ba290 2024-04-13T03:13:21.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.040+0000 7f843e7fc640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 0x7f842c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8440160520 tx=0x7f8434009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.044+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84300595d0 con 0x7f84400ba290 2024-04-13T03:13:21.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.258+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f84400b7100 con 0x7f84400ba290 2024-04-13T03:13:21.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.259+0000 7f84277fe640 1 -- 172.21.15.106:0/2439575967 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8430058ed0 con 0x7f84400ba290 2024-04-13T03:13:21.260 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:21.260 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 msgr2=0x7f842c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 0x7f842c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8440160520 tx=0x7f8434009040 comp rx=0 tx=0).stop 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 msgr2=0x7f8440155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f843000c8c0 tx=0x7f843000cd80 comp rx=0 tx=0).stop 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 shutdown_connections 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84400d22d0 0x7f84400cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84400d2dc0 0x7f84400b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 --2- 172.21.15.106:0/2439575967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84400ba290 0x7f8440155ef0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 >> 172.21.15.106:0/2439575967 conn(0x7f84400d4700 msgr2=0x7f84400b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 shutdown_connections 2024-04-13T03:13:21.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.261+0000 7f8445676640 1 -- 172.21.15.106:0/2439575967 wait complete. 2024-04-13T03:13:21.272 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:13:21.272 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:13:21.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:13:21.316 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:13:21.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:13:21.370 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:13:21.370 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:21.371 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:21.386 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:21.386 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:21.441 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:21.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.579+0000 7f948c5b4640 1 Processor -- start 2024-04-13T03:13:21.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.579+0000 7f948c5b4640 1 -- start start 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948c5b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f948415ed90 0x7f94840d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948c5b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f94840b3d60 0x7f94840d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948c5b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94840cfd20 con 0x7f948415e9b0 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948c5b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9484162af0 con 0x7f94840b3d60 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948c5b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9484162dd0 con 0x7f948415ed90 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.580+0000 7f948a350640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f948415e9b0 0x7f94840d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59652/0 (socket says 172.21.15.106:59652) 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948a350640 1 -- 172.21.15.106:0/1564162370 learned_addr learned my addr 172.21.15.106:0/1564162370 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104980959 0 0) 0x7f94840cfd20 con 0x7f948415e9b0 2024-04-13T03:13:21.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f9489b4f640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f948415ed90 0x7f94840d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f946c003520 con 0x7f948415e9b0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948ab51640 1 --2- 172.21.15.106:0/1564162370 >> v2:172.21.15.114:3300/0 conn(0x7f94840b3d60 0x7f94840d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2146625220 0 0) 0x7f946c003520 con 0x7f948415e9b0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f948415ed90 msgr2=0x7f94840d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f948415ed90 0x7f94840d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 >> v2:172.21.15.114:3300/0 conn(0x7f94840b3d60 msgr2=0x7f94840d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 --2- 172.21.15.106:0/1564162370 >> v2:172.21.15.114:3300/0 conn(0x7f94840b3d60 0x7f94840d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94841630b0 con 0x7f948415e9b0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f947c001a10 con 0x7f948415e9b0 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 >> v1:172.21.15.106:6789/0 conn(0x7f948415e9b0 legacy=0x7f94840d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f946c0039b0 0x7f946c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f946c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f946c004e50 0x7f946c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9484162af0 con 0x7f946c004310 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9484162dd0 con 0x7f946c004e50 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f946c00bdd0 con 0x7f946c0039b0 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f9489b4f640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f946c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948a350640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f946c0039b0 0x7f946c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.581+0000 7f948ab51640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f946c004e50 0x7f946c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 -- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f946c0039b0 msgr2=0x7f946c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f946c0039b0 0x7f946c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 -- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f946c004e50 msgr2=0x7f946c0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f946c004e50 0x7f946c0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 -- 172.21.15.106:0/1564162370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c003520 con 0x7f946c004310 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f9489b4f640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f946c004800 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9478009ab0 tx=0x7f946c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed24277a0ce2426 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478009680 con 0x7f946c004310 2024-04-13T03:13:21.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9478005990 con 0x7f946c004310 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.582+0000 7f948934e640 1 -- 172.21.15.106:0/1564162370 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478005d80 con 0x7f946c004310 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 -- 172.21.15.106:0/1564162370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 msgr2=0x7f946c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 --2- 172.21.15.106:0/1564162370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f946c004800 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9478009ab0 tx=0x7f946c005830 comp rx=0 tx=0).stop 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948ab51640 1 -- 172.21.15.106:0/1564162370 reap_dead start 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 -- 172.21.15.106:0/1564162370 shutdown_connections 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 -- 172.21.15.106:0/1564162370 >> 172.21.15.106:0/1564162370 conn(0x7f948408bf10 msgr2=0x7f94840b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 -- 172.21.15.106:0/1564162370 shutdown_connections 2024-04-13T03:13:21.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.583+0000 7f948c5b4640 1 -- 172.21.15.106:0/1564162370 wait complete. 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.584+0000 7f948c5b4640 1 Processor -- start 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.584+0000 7f948c5b4640 1 -- start start 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94840c9fb0 0x7f94840cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f948406c920 0x7f94840b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9484162800 con 0x7f946c004310 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9484162680 con 0x7f948406c920 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948c5b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f94840d4d00 con 0x7f94840c9fb0 2024-04-13T03:13:21.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50546/0 (socket says 172.21.15.106:50546) 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 -- 172.21.15.106:0/2180908893 learned_addr learned my addr 172.21.15.106:0/2180908893 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f9489b4f640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94840c9fb0 0x7f94840cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948ab51640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f948406c920 0x7f94840b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 -- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94840c9fb0 msgr2=0x7f94840cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94840c9fb0 0x7f94840cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 -- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f948406c920 msgr2=0x7f94840b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f948406c920 0x7f94840b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 -- 172.21.15.106:0/2180908893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94840ce4a0 con 0x7f946c004310 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f948a350640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f947c0079d0 tx=0x7f947c00a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c017020 con 0x7f946c004310 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.585+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c007ac0 con 0x7f946c004310 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.586+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94840cb930 con 0x7f946c004310 2024-04-13T03:13:21.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.586+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c013610 con 0x7f946c004310 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.586+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94840cbef0 con 0x7f946c004310 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.586+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9484068a10 con 0x7f946c004310 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.587+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f947c01a020 con 0x7f946c004310 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.587+0000 7f9482ffd640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 0x7f947006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.587+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f947c08c790 con 0x7f946c004310 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.587+0000 7f9489b4f640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 0x7f947006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.588+0000 7f9489b4f640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 0x7f947006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9478035000 tx=0x7f9478030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.592+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f947c021020 con 0x7f946c004310 2024-04-13T03:13:21.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.769+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94840d53a0 con 0x7f946c004310 2024-04-13T03:13:21.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.770+0000 7f9482ffd640 1 -- 172.21.15.106:0/2180908893 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17160 (secure 0 0 0) 0x7f947c05aa30 con 0x7f946c004310 2024-04-13T03:13:21.771 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:21.772 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":66,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:07.122095+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:21.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 msgr2=0x7f947006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 0x7f947006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9478035000 tx=0x7f9478030000 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 msgr2=0x7f948415aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f947c0079d0 tx=0x7f947c00a770 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 shutdown_connections 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f947006c920 0x7f947006ede0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f94840c9fb0 0x7f94840cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f948406c920 0x7f94840b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 --2- 172.21.15.106:0/2180908893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f946c004310 0x7f948415aa90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 >> 172.21.15.106:0/2180908893 conn(0x7f948408bf10 msgr2=0x7f94840b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 shutdown_connections 2024-04-13T03:13:21.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.772+0000 7f948c5b4640 1 -- 172.21.15.106:0/2180908893 wait complete. 2024-04-13T03:13:21.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:12:09.560834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:12:10.285668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:21.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:21.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 Processor -- start 2024-04-13T03:13:21.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 -- start start 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f46cc15c820 0x7f46cc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46cc0b7f00 0x7f46cc0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc0cd4b0 con 0x7f46cc0cd0a0 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc0706d0 con 0x7f46cc0b7f00 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.888+0000 7f46d2bcc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc0709b0 con 0x7f46cc15c820 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46cbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46cc0b7f00 0x7f46cc0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46d1169640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f46cc0cd0a0 0x7f46cc160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59668/0 (socket says 172.21.15.106:59668) 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46d1169640 1 -- 172.21.15.106:0/286207559 learned_addr learned my addr 172.21.15.106:0/286207559 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46cbfff640 1 -- 172.21.15.106:0/286207559 >> v2:172.21.15.114:3300/0 conn(0x7f46cc15c820 msgr2=0x7f46cc06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:21.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46cbfff640 1 --2- 172.21.15.106:0/286207559 >> v2:172.21.15.114:3300/0 conn(0x7f46cc15c820 0x7f46cc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46cbfff640 1 -- 172.21.15.106:0/286207559 >> v1:172.21.15.106:6789/0 conn(0x7f46cc0cd0a0 legacy=0x7f46cc160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.889+0000 7f46cbfff640 1 -- 172.21.15.106:0/286207559 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc070c90 con 0x7f46cc0b7f00 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46cbfff640 1 --2- 172.21.15.106:0/286207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc0ccb60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0d3610 tx=0x7f46b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fee8d67cd517a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46cb7fe640 1 -- 172.21.15.106:0/286207559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8035020 con 0x7f46cc0b7f00 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46cb7fe640 1 -- 172.21.15.106:0/286207559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b8031830 con 0x7f46cc0b7f00 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46cb7fe640 1 -- 172.21.15.106:0/286207559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8031c00 con 0x7f46cc0b7f00 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 -- 172.21.15.106:0/286207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 msgr2=0x7f46cc0ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/286207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc0ccb60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0d3610 tx=0x7f46b802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 -- 172.21.15.106:0/286207559 shutdown_connections 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/286207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc0ccb60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/286207559 >> v2:172.21.15.114:3300/0 conn(0x7f46cc15c820 0x7f46cc06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 -- 172.21.15.106:0/286207559 >> 172.21.15.106:0/286207559 conn(0x7f46cc0d2510 msgr2=0x7f46cc0d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 -- 172.21.15.106:0/286207559 shutdown_connections 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.890+0000 7f46d2bcc640 1 -- 172.21.15.106:0/286207559 wait complete. 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.891+0000 7f46d2bcc640 1 Processor -- start 2024-04-13T03:13:21.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.891+0000 7f46d2bcc640 1 -- start start 2024-04-13T03:13:21.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 0x7f46cc15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f46cc15c820 0x7f46cc15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0703e0 con 0x7f46cc0b7f00 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0d03c0 con 0x7f46cc0cd0a0 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d2bcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc0d18f0 con 0x7f46cc15c820 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d0968640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d0968640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50568/0 (socket says 172.21.15.106:50568) 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.892+0000 7f46d0968640 1 -- 172.21.15.106:0/240980463 learned_addr learned my addr 172.21.15.106:0/240980463 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d1169640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f46cc15c820 0x7f46cc15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46cbfff640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 0x7f46cc15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 -- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f46cc15c820 msgr2=0x7f46cc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f46cc15c820 0x7f46cc15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 -- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 msgr2=0x7f46cc15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 0x7f46cc15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 -- 172.21.15.106:0/240980463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc15f740 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46cbfff640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 0x7f46cc15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d0968640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b410 tx=0x7f46c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0015020 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0011850 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc15fa20 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011c40 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.893+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc0c69b0 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.894+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc069120 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.894+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46c0018020 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.895+0000 7f46c97fa640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 0x7f46b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.895+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f46c008bc50 con 0x7f46cc0b7f00 2024-04-13T03:13:21.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.895+0000 7f46cbfff640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 0x7f46b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:21.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.895+0000 7f46cbfff640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 0x7f46b006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46b8036000 tx=0x7f46b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:21.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:21.899+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46c0091020 con 0x7f46cc0b7f00 2024-04-13T03:13:22.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.072+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46cc15fff0 con 0x7f46cc0b7f00 2024-04-13T03:13:22.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.073+0000 7f46c97fa640 1 -- 172.21.15.106:0/240980463 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f46c001f020 con 0x7f46cc0b7f00 2024-04-13T03:13:22.074 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:22.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 msgr2=0x7f46b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 0x7f46b006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46b8036000 tx=0x7f46b8008030 comp rx=0 tx=0).stop 2024-04-13T03:13:22.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 msgr2=0x7f46cc15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b410 tx=0x7f46c000b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 shutdown_connections 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f46b006c8f0 0x7f46b006edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f46cc15c820 0x7f46cc15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46cc0cd0a0 0x7f46cc15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 --2- 172.21.15.106:0/240980463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46cc0b7f00 0x7f46cc15ec60 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 >> 172.21.15.106:0/240980463 conn(0x7f46cc0d2510 msgr2=0x7f46cc0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 shutdown_connections 2024-04-13T03:13:22.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.075+0000 7f46d2bcc640 1 -- 172.21.15.106:0/240980463 wait complete. 2024-04-13T03:13:22.087 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:13:22.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.190+0000 7fe2cc5aa640 1 Processor -- start 2024-04-13T03:13:22.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.190+0000 7fe2cc5aa640 1 -- start start 2024-04-13T03:13:22.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.191+0000 7fe2cc5aa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe2c40b7f00 0x7fe2c40ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.191+0000 7fe2cc5aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2c40d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.191+0000 7fe2cc5aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c40cd4b0 con 0x7fe2c415c820 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.191+0000 7fe2cc5aa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c40706d0 con 0x7fe2c40b7f00 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.191+0000 7fe2cc5aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c40709b0 con 0x7fe2c40cd0a0 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2ca346640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe2c415c820 0x7fe2c406fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59680/0 (socket says 172.21.15.106:59680) 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2ca346640 1 -- 172.21.15.106:0/1651123955 learned_addr learned my addr 172.21.15.106:0/1651123955 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/1651123955 >> v2:172.21.15.114:3300/0 conn(0x7fe2c40b7f00 0x7fe2c40ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2cab47640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2c40d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890071602 0 0) 0x7fe2bc001490 con 0x7fe2c415c820 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2ac003520 con 0x7fe2c415c820 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2054261098 0 0) 0x7fe2ac003520 con 0x7fe2c415c820 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 msgr2=0x7fe2c40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2c40d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 >> v2:172.21.15.114:3300/0 conn(0x7fe2c40b7f00 msgr2=0x7fe2c40ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 --2- 172.21.15.106:0/1651123955 >> v2:172.21.15.114:3300/0 conn(0x7fe2c40b7f00 0x7fe2c40ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c4070c90 con 0x7fe2c415c820 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fe2bc001be0 con 0x7fe2c415c820 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 >> v1:172.21.15.106:6789/0 conn(0x7fe2c415c820 legacy=0x7fe2c406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac0039b0 0x7fe2ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40706d0 con 0x7fe2ac004d40 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40709b0 con 0x7fe2ac0039b0 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac00b9c0 con 0x7fe2c40cd0a0 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2cab47640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.192+0000 7fe2ca346640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac0039b0 0x7fe2ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 -- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 msgr2=0x7fe2ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c40cd0a0 0x7fe2ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 -- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac0039b0 msgr2=0x7fe2ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac0039b0 0x7fe2ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 -- 172.21.15.106:0/1651123955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac003520 con 0x7fe2ac004d40 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2cab47640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2ac0051c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009af0 tx=0x7fe2ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c86f4ebee6513a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4039070 con 0x7fe2ac004d40 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b402c6b0 con 0x7fe2ac004d40 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.193+0000 7fe2c9344640 1 -- 172.21.15.106:0/1651123955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009690 con 0x7fe2ac004d40 2024-04-13T03:13:22.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/1651123955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 msgr2=0x7fe2ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/1651123955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2ac0051c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009af0 tx=0x7fe2ac005830 comp rx=0 tx=0).stop 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cab47640 1 -- 172.21.15.106:0/1651123955 reap_dead start 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/1651123955 shutdown_connections 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/1651123955 >> 172.21.15.106:0/1651123955 conn(0x7fe2c40d2510 msgr2=0x7fe2c40d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/1651123955 shutdown_connections 2024-04-13T03:13:22.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.194+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/1651123955 wait complete. 2024-04-13T03:13:22.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.195+0000 7fe2cc5aa640 1 Processor -- start 2024-04-13T03:13:22.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.195+0000 7fe2cc5aa640 1 -- start start 2024-04-13T03:13:22.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 0x7fe2c415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 0x7fe2c40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40d08e0 con 0x7fe2ac004d40 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40d03c0 con 0x7fe2c406ef80 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cc5aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c40d0a60 con 0x7fe2c415d140 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50582/0 (socket says 172.21.15.106:50582) 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 -- 172.21.15.106:0/3056380416 learned_addr learned my addr 172.21.15.106:0/3056380416 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cab47640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 0x7fe2c40c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 0x7fe2c415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 -- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 msgr2=0x7fe2c40c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 0x7fe2c40c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 -- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 msgr2=0x7fe2c415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 0x7fe2c415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 -- 172.21.15.106:0/3056380416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c406d9a0 con 0x7fe2ac004d40 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2cab47640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 0x7fe2c40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 0x7fe2c415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.196+0000 7fe2ca346640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc001d50 tx=0x7fe2bc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.197+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc017020 con 0x7fe2ac004d40 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.197+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc0133f0 con 0x7fe2ac004d40 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.197+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0137e0 con 0x7fe2ac004d40 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.197+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c406dc80 con 0x7fe2ac004d40 2024-04-13T03:13:22.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.197+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c406e240 con 0x7fe2ac004d40 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.198+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c4069120 con 0x7fe2ac004d40 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.198+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2bc01a020 con 0x7fe2ac004d40 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.199+0000 7fe2c2ffd640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 0x7fe2b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.199+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fe2bc08c830 con 0x7fe2ac004d40 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.199+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 0x7fe2b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.199+0000 7fe2c9b45640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 0x7fe2b006ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4071310 tx=0x7fe2b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.203+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2bc021020 con 0x7fe2ac004d40 2024-04-13T03:13:22.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.379+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2c415fff0 con 0x7fe2ac004d40 2024-04-13T03:13:22.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.379+0000 7fe2c2ffd640 1 -- 172.21.15.106:0/3056380416 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7fe2c40cd4b0 con 0x7fe2ac004d40 2024-04-13T03:13:22.380 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:13:22.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 msgr2=0x7fe2b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 0x7fe2b006ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4071310 tx=0x7fe2b8008040 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 msgr2=0x7fe2c406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc001d50 tx=0x7fe2bc007430 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 shutdown_connections 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2b006c920 0x7fe2b006ede0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2c415d140 0x7fe2c40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2c406ef80 0x7fe2c415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 --2- 172.21.15.106:0/3056380416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2ac004d40 0x7fe2c406ea40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 >> 172.21.15.106:0/3056380416 conn(0x7fe2c40d2510 msgr2=0x7fe2c40b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 shutdown_connections 2024-04-13T03:13:22.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.381+0000 7fe2cc5aa640 1 -- 172.21.15.106:0/3056380416 wait complete. 2024-04-13T03:13:22.393 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:13:22.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.496+0000 7f2a11da5640 1 Processor -- start 2024-04-13T03:13:22.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.496+0000 7f2a11da5640 1 -- start start 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a11da5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a0c0b1c30 0x7f2a0c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a11da5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 0x7f2a0c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a11da5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0c0b5cf0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a11da5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c0b5fd0 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a11da5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c0b62b0 con 0x7f2a0c0b54a0 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a0c0b1c30 0x7f2a0c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2a0c0709e0 0x7f2a0c162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59686/0 (socket says 172.21.15.106:59686) 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0bfff640 1 -- 172.21.15.106:0/638506562 learned_addr learned my addr 172.21.15.106:0/638506562 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0affd640 1 --2- 172.21.15.106:0/638506562 >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 0x7f2a0c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/638506562 >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 msgr2=0x7f2a0c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0b7fe640 1 --2- 172.21.15.106:0/638506562 >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 0x7f2a0c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/638506562 >> v1:172.21.15.106:6789/0 conn(0x7f2a0c0709e0 legacy=0x7f2a0c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/638506562 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0b6590 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0a7fc640 1 -- 172.21.15.106:0/638506562 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198823253 0 0) 0x7f2a0c0b5cf0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.497+0000 7f2a0affd640 1 --2- 172.21.15.106:0/638506562 >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 0x7f2a0c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.498+0000 7f2a0b7fe640 1 --2- 172.21.15.106:0/638506562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0b1c30 0x7f2a0c15ee40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c06c900 tx=0x7f29fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8e5dea9137ec44 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.498+0000 7f2a0a7fc640 1 -- 172.21.15.106:0/638506562 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc035020 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.498+0000 7f2a0a7fc640 1 -- 172.21.15.106:0/638506562 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc031830 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.498+0000 7f2a0a7fc640 1 -- 172.21.15.106:0/638506562 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc031c00 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 -- 172.21.15.106:0/638506562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0b1c30 msgr2=0x7f2a0c15ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 --2- 172.21.15.106:0/638506562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0b1c30 0x7f2a0c15ee40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c06c900 tx=0x7f29fc02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 -- 172.21.15.106:0/638506562 shutdown_connections 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 --2- 172.21.15.106:0/638506562 >> v2:172.21.15.114:3300/0 conn(0x7f2a0c0b54a0 0x7f2a0c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 --2- 172.21.15.106:0/638506562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0b1c30 0x7f2a0c15ee40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 -- 172.21.15.106:0/638506562 >> 172.21.15.106:0/638506562 conn(0x7f2a0c071870 msgr2=0x7f2a0c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 -- 172.21.15.106:0/638506562 shutdown_connections 2024-04-13T03:13:22.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.499+0000 7f2a11da5640 1 -- 172.21.15.106:0/638506562 wait complete. 2024-04-13T03:13:22.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.500+0000 7f2a11da5640 1 Processor -- start 2024-04-13T03:13:22.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.500+0000 7f2a11da5640 1 -- start start 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 0x7f2a0c0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0c0b54a0 0x7f2a0c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c06e7f0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c06e670 con 0x7f2a0c0b54a0 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a11da5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c06e970 con 0x7f2a0c0b1c30 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50592/0 (socket says 172.21.15.106:50592) 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/1510121380 learned_addr learned my addr 172.21.15.106:0/1510121380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0affd640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 0x7f2a0c0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0bfff640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0c0b54a0 0x7f2a0c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 msgr2=0x7f2a0c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 0x7f2a0c0cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0c0b54a0 msgr2=0x7f2a0c150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0c0b54a0 0x7f2a0c150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 -- 172.21.15.106:0/1510121380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0cf9e0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0affd640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 0x7f2a0c0cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.501+0000 7f2a0b7fe640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009890 tx=0x7f29fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.502+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc041070 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.502+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc03aa30 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.502+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c0cfcc0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.502+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc035040 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.502+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c0d0280 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.503+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c068a10 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.503+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29fc038020 con 0x7f2a0c0709e0 2024-04-13T03:13:22.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.503+0000 7f29f3fff640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 0x7f29f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.504+0000 7f2a0affd640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 0x7f29f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.504+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f29fc0ad830 con 0x7f2a0c0709e0 2024-04-13T03:13:22.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.504+0000 7f2a0affd640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 0x7f29f806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006fd0 tx=0x7f2a00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.508+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29fc07ba40 con 0x7f2a0c0709e0 2024-04-13T03:13:22.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.682+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a0c0711f0 con 0x7f2a0c0709e0 2024-04-13T03:13:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.682+0000 7f29f3fff640 1 -- 172.21.15.106:0/1510121380 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f29fc07b340 con 0x7f2a0c0709e0 2024-04-13T03:13:22.683 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:13:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 msgr2=0x7f29f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 0x7f29f806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006fd0 tx=0x7f2a00008040 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 msgr2=0x7f2a0c162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009890 tx=0x7f29fc0046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 shutdown_connections 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29f806c8f0 0x7f29f806edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0c0b1c30 0x7f2a0c0cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0c0b54a0 0x7f2a0c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 --2- 172.21.15.106:0/1510121380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a0c0709e0 0x7f2a0c162200 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 >> 172.21.15.106:0/1510121380 conn(0x7f2a0c071870 msgr2=0x7f2a0c15c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 shutdown_connections 2024-04-13T03:13:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.684+0000 7f2a11da5640 1 -- 172.21.15.106:0/1510121380 wait complete. 2024-04-13T03:13:22.696 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:13:22.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.795+0000 7fc0e4a0f640 1 Processor -- start 2024-04-13T03:13:22.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.795+0000 7fc0e4a0f640 1 -- start start 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.796+0000 7fc0e4a0f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc0e00cf8f0 0x7fc0e013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.796+0000 7fc0e4a0f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0e0141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.796+0000 7fc0e4a0f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e01429b0 con 0x7fc0e015d400 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.796+0000 7fc0e4a0f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e006c0d0 con 0x7fc0e00cf8f0 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.796+0000 7fc0e4a0f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e006ba80 con 0x7fc0e015d7e0 2024-04-13T03:13:22.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc0e015d400 0x7fc0e00cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41272/0 (socket says 172.21.15.106:41272) 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 -- 172.21.15.106:0/2800771096 learned_addr learned my addr 172.21.15.106:0/2800771096 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ddd9b640 1 --2- 172.21.15.106:0/2800771096 >> v2:172.21.15.114:3300/0 conn(0x7fc0e00cf8f0 0x7fc0e013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0e0141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110987621 0 0) 0x7fc0e01429b0 con 0x7fc0e015d400 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0cc003520 con 0x7fc0e015d400 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1928832564 0 0) 0x7fc0cc003520 con 0x7fc0e015d400 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 msgr2=0x7fc0e0141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0e0141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 >> v2:172.21.15.114:3300/0 conn(0x7fc0e00cf8f0 msgr2=0x7fc0e013fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 --2- 172.21.15.106:0/2800771096 >> v2:172.21.15.114:3300/0 conn(0x7fc0e00cf8f0 0x7fc0e013fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e006bd60 con 0x7fc0e015d400 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc0d8001690 con 0x7fc0e015d400 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ddd9b640 1 --2- 172.21.15.106:0/2800771096 >> v2:172.21.15.114:3300/0 conn(0x7fc0e00cf8f0 0x7fc0e013fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0e0141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:22.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 >> v1:172.21.15.106:6789/0 conn(0x7fc0e015d400 legacy=0x7fc0e00cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0cc0039b0 0x7fc0cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0cc004d40 0x7fc0cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e006c0d0 con 0x7fc0cc0039b0 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e006ba80 con 0x7fc0cc004d40 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc00b9c0 con 0x7fc0e015d7e0 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0cc0039b0 0x7fc0cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ddd9b640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0cc004d40 0x7fc0cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 -- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 msgr2=0x7fc0cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0e015d7e0 0x7fc0cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 -- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0cc004d40 msgr2=0x7fc0cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:22.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.797+0000 7fc0de59c640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0cc004d40 0x7fc0cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.798+0000 7fc0de59c640 1 -- 172.21.15.106:0/2800771096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc003520 con 0x7fc0cc0039b0 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.798+0000 7fc0de59c640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0cc0039b0 0x7fc0cc003dd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8007fd0 tx=0x7fc0cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2037e07fe808c7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.798+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8007d50 con 0x7fc0cc0039b0 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.798+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d8030070 con 0x7fc0cc0039b0 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.798+0000 7fc0dd59a640 1 -- 172.21.15.106:0/2800771096 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8005d10 con 0x7fc0cc0039b0 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/2800771096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0cc0039b0 msgr2=0x7fc0cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/2800771096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0cc0039b0 0x7fc0cc003dd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8007fd0 tx=0x7fc0cc005830 comp rx=0 tx=0).stop 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0ded9d640 1 -- 172.21.15.106:0/2800771096 reap_dead start 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/2800771096 shutdown_connections 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/2800771096 >> 172.21.15.106:0/2800771096 conn(0x7fc0e00b2600 msgr2=0x7fc0e00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/2800771096 shutdown_connections 2024-04-13T03:13:22.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.799+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/2800771096 wait complete. 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.800+0000 7fc0e4a0f640 1 Processor -- start 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.800+0000 7fc0e4a0f640 1 -- start start 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0cc0039b0 0x7fc0e00ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 0x7fc0e00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0142050 con 0x7fc0e00b5530 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0141ed0 con 0x7fc0e00ad390 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0e4a0f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e01421d0 con 0x7fc0cc0039b0 2024-04-13T03:13:22.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34300/0 (socket says 172.21.15.106:34300) 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 -- 172.21.15.106:0/1785768195 learned_addr learned my addr 172.21.15.106:0/1785768195 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0de59c640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0cc0039b0 0x7fc0e00ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ddd9b640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 0x7fc0e00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 -- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0cc0039b0 msgr2=0x7fc0e00ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0cc0039b0 0x7fc0e00ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 -- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 msgr2=0x7fc0e00b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 0x7fc0e00b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ded9d640 1 -- 172.21.15.106:0/1785768195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e00ad810 con 0x7fc0e00b5530 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.801+0000 7fc0ddd9b640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 0x7fc0e00b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0ded9d640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0070100 tx=0x7fc0d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000fb70 con 0x7fc0e00b5530 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d000fd50 con 0x7fc0e00b5530 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0003bd0 con 0x7fc0e00b5530 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e00adad0 con 0x7fc0e00b5530 2024-04-13T03:13:22.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.802+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e00ae090 con 0x7fc0e00b5530 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.803+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e0068a10 con 0x7fc0e00b5530 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.803+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0d001f020 con 0x7fc0e00b5530 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.803+0000 7fc0d6ffd640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.804+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fc0d008bfd0 con 0x7fc0e00b5530 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.804+0000 7fc0de59c640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.804+0000 7fc0de59c640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 0x7fc0c006edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8034000 tx=0x7fc0d8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:22.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.808+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0d0010040 con 0x7fc0e00b5530 2024-04-13T03:13:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.996+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0e015a4d0 con 0x7fc0e00b5530 2024-04-13T03:13:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.996+0000 7fc0d6ffd640 1 -- 172.21.15.106:0/1785768195 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7fc0d005a1e0 con 0x7fc0e00b5530 2024-04-13T03:13:22.997 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:13:22.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 msgr2=0x7fc0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 0x7fc0c006edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8034000 tx=0x7fc0d8007000 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 msgr2=0x7fc0e015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0070100 tx=0x7fc0d0009f90 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 shutdown_connections 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0e00b5530 0x7fc0e015ec40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0e00ad390 0x7fc0e00b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 --2- 172.21.15.106:0/1785768195 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0cc0039b0 0x7fc0e00ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 >> 172.21.15.106:0/1785768195 conn(0x7fc0e00b2600 msgr2=0x7fc0e013c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 shutdown_connections 2024-04-13T03:13:23.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:22.998+0000 7fc0e4a0f640 1 -- 172.21.15.106:0/1785768195 wait complete. 2024-04-13T03:13:23.010 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:13:23.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:23.011 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:23.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:13:23.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:23.050 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:13:23.106 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:13:23.107 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:13:23.107 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:23.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:13:23.124 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:13:23.124 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:23.124 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:23.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:13:23.161 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:13:23.162 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:13:23.162 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:23.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:13:23.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.301+0000 7f20405ed640 1 Processor -- start 2024-04-13T03:13:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.301+0000 7f20405ed640 1 -- start start 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f20405ed640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f20405ed640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203806fd00 con 0x7f203815caa0 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f20405ed640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203806ffe0 con 0x7f20380b7f70 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f20405ed640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20380714b0 con 0x7f203815ce80 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f203e389640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f203815caa0 0x7f2038070da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41284/0 (socket says 172.21.15.106:41284) 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f203e389640 1 -- 172.21.15.106:0/965795526 learned_addr learned my addr 172.21.15.106:0/965795526 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f203db88640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339768215 0 0) 0x7f203806fd00 con 0x7f203815caa0 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.302+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2024003520 con 0x7f203815caa0 2024-04-13T03:13:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145050238 0 0) 0x7f2024003520 con 0x7f203815caa0 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 msgr2=0x7f20380d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 >> v1:172.21.15.114:6789/0 conn(0x7f20380b7f70 legacy=0x7f20380b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380702c0 con 0x7f203815caa0 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203db88640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2034001670 con 0x7f203815caa0 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 >> v1:172.21.15.106:6789/0 conn(0x7f203815caa0 legacy=0x7f2038070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20240039e0 0x7f2024003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2024004300 0x7f2024004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f202400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20380714b0 con 0x7f2024004f90 2024-04-13T03:13:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f202400bf00 con 0x7f2024004300 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f202400c080 con 0x7f20240039e0 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f202400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203e389640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20240039e0 0x7f2024003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203db88640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2024004300 0x7f2024004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 -- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20240039e0 msgr2=0x7f2024003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20240039e0 0x7f2024003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 -- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2024004300 msgr2=0x7f2024004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2024004300 0x7f2024004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.303+0000 7f203eb8a640 1 -- 172.21.15.106:0/965795526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024003520 con 0x7f2024004f90 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f203eb8a640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f202400b9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2030002010 tx=0x7f2024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7607fb5cb3f10133 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030039070 con 0x7f2024004f90 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203002c750 con 0x7f2024004f90 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f203d387640 1 -- 172.21.15.106:0/965795526 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030001be0 con 0x7f2024004f90 2024-04-13T03:13:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f20405ed640 1 -- 172.21.15.106:0/965795526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 msgr2=0x7f202400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.304+0000 7f20405ed640 1 --2- 172.21.15.106:0/965795526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f202400b9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2030002010 tx=0x7f2024005830 comp rx=0 tx=0).stop 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.305+0000 7f203eb8a640 1 -- 172.21.15.106:0/965795526 reap_dead start 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.305+0000 7f20405ed640 1 -- 172.21.15.106:0/965795526 shutdown_connections 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.305+0000 7f20405ed640 1 -- 172.21.15.106:0/965795526 >> 172.21.15.106:0/965795526 conn(0x7f20380d25d0 msgr2=0x7f20380d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.305+0000 7f20405ed640 1 -- 172.21.15.106:0/965795526 shutdown_connections 2024-04-13T03:13:23.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.305+0000 7f20405ed640 1 -- 172.21.15.106:0/965795526 wait complete. 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.306+0000 7f20405ed640 1 Processor -- start 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.306+0000 7f20405ed640 1 -- start start 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.306+0000 7f20405ed640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.306+0000 7f20405ed640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.306+0000 7f20405ed640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20380c6640 0x7f20380cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f20405ed640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f203806f990 con 0x7f2024004f90 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f20405ed640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2038070880 con 0x7f20380c6640 2024-04-13T03:13:23.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f20405ed640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f203806f490 con 0x7f203815ce80 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34312/0 (socket says 172.21.15.106:34312) 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 -- 172.21.15.106:0/1664047297 learned_addr learned my addr 172.21.15.106:0/1664047297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203eb8a640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20380c6640 0x7f20380cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 -- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 msgr2=0x7f20380c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 -- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20380c6640 msgr2=0x7f20380cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20380c6640 0x7f20380cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 -- 172.21.15.106:0/1664047297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038158300 con 0x7f2024004f90 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f203e389640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2034007340 tx=0x7f2034007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203400f860 con 0x7f2024004f90 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034018070 con 0x7f2024004f90 2024-04-13T03:13:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203400fc60 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203815d340 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.307+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203815d900 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.308+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2038068a10 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.309+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f203401d460 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.309+0000 7f202effd640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 0x7f201c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.309+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f203408bd20 con 0x7f2024004f90 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.309+0000 7f203db88640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 0x7f201c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.309+0000 7f203db88640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 0x7f201c06ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f20380b6950 tx=0x7f20280093a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.313+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2038158300 con 0x7f2024004f90 2024-04-13T03:13:23.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.525+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20380cac70 con 0x7f2024004f90 2024-04-13T03:13:23.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.526+0000 7f202effd640 1 -- 172.21.15.106:0/1664047297 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5186 (secure 0 0 0) 0x7f2034059f30 con 0x7f2024004f90 2024-04-13T03:13:23.527 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:23.527 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:52.017572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:13:23.527 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 30 2024-04-13T03:13:23.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 msgr2=0x7f201c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 0x7f201c06ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f20380b6950 tx=0x7f20280093a0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 msgr2=0x7f2038157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2034007340 tx=0x7f2034007800 comp rx=0 tx=0).stop 2024-04-13T03:13:23.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 shutdown_connections 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f201c06c920 0x7f201c06ede0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f203815ce80 0x7f20380c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20380c6640 0x7f20380cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 --2- 172.21.15.106:0/1664047297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2024004f90 0x7f2038157dc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 >> 172.21.15.106:0/1664047297 conn(0x7f20380d25d0 msgr2=0x7f20380b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 shutdown_connections 2024-04-13T03:13:23.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.528+0000 7f20405ed640 1 -- 172.21.15.106:0/1664047297 wait complete. 2024-04-13T03:13:23.540 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:23.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.639+0000 7f10e2005640 1 Processor -- start 2024-04-13T03:13:23.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.639+0000 7f10e2005640 1 -- start start 2024-04-13T03:13:23.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.639+0000 7f10e2005640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10e2005640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f10dc06cb10 0x7f10dc072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10e2005640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10dc06b8e0 0x7f10dc06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10e2005640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0ce870 con 0x7f10dc06b8e0 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10e2005640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0998d0 con 0x7f10dc06cb10 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10e2005640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc073020 con 0x7f10dc0cdf50 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10dc06b8e0 0x7f10dc06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10dc06b8e0 0x7f10dc06bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34322/0 (socket says 172.21.15.106:34322) 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 -- 172.21.15.106:0/3699319606 learned_addr learned my addr 172.21.15.106:0/3699319606 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10d3fff640 1 --2- 172.21.15.106:0/3699319606 >> v2:172.21.15.114:3300/0 conn(0x7f10dc06cb10 0x7f10dc072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10db7fe640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 -- 172.21.15.106:0/3699319606 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 msgr2=0x7f10dc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 -- 172.21.15.106:0/3699319606 >> v2:172.21.15.114:3300/0 conn(0x7f10dc06cb10 msgr2=0x7f10dc072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 --2- 172.21.15.106:0/3699319606 >> v2:172.21.15.114:3300/0 conn(0x7f10dc06cb10 0x7f10dc072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10dbfff640 1 -- 172.21.15.106:0/3699319606 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc073300 con 0x7f10dc06b8e0 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.640+0000 7f10db7fe640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10dbfff640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc06bd90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0b2fd0 tx=0x7f10d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b116c973d9af5a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10daffd640 1 -- 172.21.15.106:0/3699319606 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d402ca70 con 0x7f10dc06b8e0 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10daffd640 1 -- 172.21.15.106:0/3699319606 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d402cc10 con 0x7f10dc06b8e0 2024-04-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10daffd640 1 -- 172.21.15.106:0/3699319606 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d40343f0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 -- 172.21.15.106:0/3699319606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 msgr2=0x7f10dc06bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc06bd90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0b2fd0 tx=0x7f10d402bed0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 -- 172.21.15.106:0/3699319606 shutdown_connections 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc06bd90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 --2- 172.21.15.106:0/3699319606 >> v2:172.21.15.114:3300/0 conn(0x7f10dc06cb10 0x7f10dc072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 --2- 172.21.15.106:0/3699319606 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.641+0000 7f10e2005640 1 -- 172.21.15.106:0/3699319606 >> 172.21.15.106:0/3699319606 conn(0x7f10dc0b1f10 msgr2=0x7f10dc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.642+0000 7f10e2005640 1 -- 172.21.15.106:0/3699319606 shutdown_connections 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.642+0000 7f10e2005640 1 -- 172.21.15.106:0/3699319606 wait complete. 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 Processor -- start 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 -- start start 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 0x7f10dc0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc072c60 con 0x7f10dc06b8e0 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0738c0 con 0x7f10dc06cb10 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10e2005640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc072de0 con 0x7f10dc0cdf50 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34330/0 (socket says 172.21.15.106:34330) 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.643+0000 7f10db7fe640 1 -- 172.21.15.106:0/3279674409 learned_addr learned my addr 172.21.15.106:0/3279674409 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10dbfff640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10d3fff640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 0x7f10dc0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 -- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 msgr2=0x7f10dc0ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 -- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 msgr2=0x7f10dc0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 0x7f10dc0ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 -- 172.21.15.106:0/3279674409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc0c6680 con 0x7f10dc06b8e0 2024-04-13T03:13:23.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10d3fff640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 0x7f10dc0ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10db7fe640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00e7e0 tx=0x7f10cc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00f040 con 0x7f10dc06b8e0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc004030 con 0x7f10dc06b8e0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc15b7e0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004400 con 0x7f10dc06b8e0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.644+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1594c0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.645+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc068a10 con 0x7f10dc06b8e0 2024-04-13T03:13:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.646+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10cc00c6d0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.646+0000 7f10d8ff9640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 0x7f10bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.646+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f10cc08ba10 con 0x7f10dc06b8e0 2024-04-13T03:13:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.646+0000 7f10d3fff640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 0x7f10bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.646+0000 7f10d3fff640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 0x7f10bc06ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f10dc153c30 tx=0x7f10c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.650+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10cc017020 con 0x7f10dc06b8e0 2024-04-13T03:13:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.826+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10dc0ceda0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.827+0000 7f10d8ff9640 1 -- 172.21.15.106:0/3279674409 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17160 (secure 0 0 0) 0x7f10cc0595d0 con 0x7f10dc06b8e0 2024-04-13T03:13:23.828 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:23.828 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":66,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:07.122095+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:23.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 msgr2=0x7f10bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 0x7f10bc06ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f10dc153c30 tx=0x7f10c4008040 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 msgr2=0x7f10dc0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00e7e0 tx=0x7f10cc00eca0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 shutdown_connections 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f10bc06c920 0x7f10bc06ede0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10dc0cdf50 0x7f10dc0ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10dc06cb10 0x7f10dc0ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 --2- 172.21.15.106:0/3279674409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10dc06b8e0 0x7f10dc0c6140 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 >> 172.21.15.106:0/3279674409 conn(0x7f10dc0b1f10 msgr2=0x7f10dc0af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 shutdown_connections 2024-04-13T03:13:23.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.829+0000 7f10e2005640 1 -- 172.21.15.106:0/3279674409 wait complete. 2024-04-13T03:13:23.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:12:09.560834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:12:10.285668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:23.842 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.944+0000 7ff3da088640 1 Processor -- start 2024-04-13T03:13:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.944+0000 7ff3da088640 1 -- start start 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3da088640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3d4073520 0x7ff3d40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3da088640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3d415eca0 0x7ff3d40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3da088640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d40b5060 con 0x7ff3d415e8c0 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3da088640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d40b5340 con 0x7ff3d415eca0 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3da088640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d406c500 con 0x7ff3d4073520 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3d415eca0 0x7ff3d40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff3d415e8c0 0x7ff3d4072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41300/0 (socket says 172.21.15.106:41300) 2024-04-13T03:13:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d37fe640 1 -- 172.21.15.106:0/271567802 learned_addr learned my addr 172.21.15.106:0/271567802 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3cbfff640 1 --2- 172.21.15.106:0/271567802 >> v2:172.21.15.114:3300/0 conn(0x7ff3d4073520 0x7ff3d40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d3fff640 1 -- 172.21.15.106:0/271567802 >> v2:172.21.15.114:3300/0 conn(0x7ff3d4073520 msgr2=0x7ff3d40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d3fff640 1 --2- 172.21.15.106:0/271567802 >> v2:172.21.15.114:3300/0 conn(0x7ff3d4073520 0x7ff3d40ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d3fff640 1 -- 172.21.15.106:0/271567802 >> v1:172.21.15.106:6789/0 conn(0x7ff3d415e8c0 legacy=0x7ff3d4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d3fff640 1 -- 172.21.15.106:0/271567802 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d406c7e0 con 0x7ff3d415eca0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.945+0000 7ff3d2ffd640 1 -- 172.21.15.106:0/271567802 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593479203 0 0) 0x7ff3d40b5060 con 0x7ff3d415e8c0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.946+0000 7ff3d3fff640 1 --2- 172.21.15.106:0/271567802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d415eca0 0x7ff3d40b4430 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4072ae0 tx=0x7ff3cc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c147a82789e8c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.946+0000 7ff3d2ffd640 1 -- 172.21.15.106:0/271567802 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc039070 con 0x7ff3d415eca0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.946+0000 7ff3d2ffd640 1 -- 172.21.15.106:0/271567802 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3cc02cd00 con 0x7ff3d415eca0 2024-04-13T03:13:23.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.946+0000 7ff3d2ffd640 1 -- 172.21.15.106:0/271567802 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc034660 con 0x7ff3d415eca0 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 -- 172.21.15.106:0/271567802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d415eca0 msgr2=0x7ff3d40b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 --2- 172.21.15.106:0/271567802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d415eca0 0x7ff3d40b4430 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4072ae0 tx=0x7ff3cc02bf50 comp rx=0 tx=0).stop 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 -- 172.21.15.106:0/271567802 shutdown_connections 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 --2- 172.21.15.106:0/271567802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d415eca0 0x7ff3d40b4430 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 --2- 172.21.15.106:0/271567802 >> v2:172.21.15.114:3300/0 conn(0x7ff3d4073520 0x7ff3d40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 -- 172.21.15.106:0/271567802 >> 172.21.15.106:0/271567802 conn(0x7ff3d406f0d0 msgr2=0x7ff3d406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 -- 172.21.15.106:0/271567802 shutdown_connections 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.947+0000 7ff3da088640 1 -- 172.21.15.106:0/271567802 wait complete. 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.948+0000 7ff3da088640 1 Processor -- start 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.948+0000 7ff3da088640 1 -- start start 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3d415e8c0 0x7ff3d4160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3d415eca0 0x7ff3d4149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d40b4d40 con 0x7ff3d4073520 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d40b4bc0 con 0x7ff3d415eca0 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d406cd00 con 0x7ff3d415e8c0 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34348/0 (socket says 172.21.15.106:34348) 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 -- 172.21.15.106:0/1610570599 learned_addr learned my addr 172.21.15.106:0/1610570599 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 -- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3d415e8c0 msgr2=0x7ff3d4160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3d415e8c0 0x7ff3d4160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 -- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3d415eca0 msgr2=0x7ff3d4149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3d415eca0 0x7ff3d4149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 -- 172.21.15.106:0/1610570599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d41613f0 con 0x7ff3d4073520 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d37fe640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c500 tx=0x7ff3c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4018070 con 0x7ff3d4073520 2024-04-13T03:13:23.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c4007580 con 0x7ff3d4073520 2024-04-13T03:13:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4004a20 con 0x7ff3d4073520 2024-04-13T03:13:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d41616d0 con 0x7ff3d4073520 2024-04-13T03:13:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.949+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d40cab30 con 0x7ff3d4073520 2024-04-13T03:13:23.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.950+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d4068a10 con 0x7ff3d4073520 2024-04-13T03:13:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.951+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3c401b070 con 0x7ff3d4073520 2024-04-13T03:13:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.951+0000 7ff3d0ff9640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 0x7ff3b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.951+0000 7ff3cbfff640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 0x7ff3b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.951+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7ff3c408afe0 con 0x7ff3d4073520 2024-04-13T03:13:23.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.951+0000 7ff3cbfff640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 0x7ff3b406ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc006fd0 tx=0x7ff3bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:23.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:23.955+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3c4059280 con 0x7ff3d4073520 2024-04-13T03:13:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.134+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3d40cdbf0 con 0x7ff3d4073520 2024-04-13T03:13:24.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.134+0000 7ff3d0ff9640 1 -- 172.21.15.106:0/1610570599 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7ff3c4058b80 con 0x7ff3d4073520 2024-04-13T03:13:24.135 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 msgr2=0x7ff3b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 0x7ff3b406ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc006fd0 tx=0x7ff3bc008040 comp rx=0 tx=0).stop 2024-04-13T03:13:24.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 msgr2=0x7ff3d4158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c500 tx=0x7ff3c400c9c0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 shutdown_connections 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3b406c920 0x7ff3b406ede0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3d415e8c0 0x7ff3d4160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3d415eca0 0x7ff3d4149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 --2- 172.21.15.106:0/1610570599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3d4073520 0x7ff3d4158bd0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 >> 172.21.15.106:0/1610570599 conn(0x7ff3d406f0d0 msgr2=0x7ff3d40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 shutdown_connections 2024-04-13T03:13:24.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.136+0000 7ff3da088640 1 -- 172.21.15.106:0/1610570599 wait complete. 2024-04-13T03:13:24.148 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:13:24.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.250+0000 7f4534436640 1 Processor -- start 2024-04-13T03:13:24.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.250+0000 7f4534436640 1 -- start start 2024-04-13T03:13:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f4534436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f452c156b40 0x7f452c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f4534436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 0x7f452c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f4534436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f452c15d690 con 0x7f452c06ebf0 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f4534436640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f452c06b8e0 con 0x7f452c156b40 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f4534436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f452c06bea0 con 0x7f452c06c330 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f45321d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f452c156b40 0x7f452c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f45321d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f452c156b40 0x7f452c06e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34356/0 (socket says 172.21.15.106:34356) 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f45321d2640 1 -- 172.21.15.106:0/1031593394 learned_addr learned my addr 172.21.15.106:0/1031593394 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.251+0000 7f45329d3640 1 --2- 172.21.15.106:0/1031593394 >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 0x7f452c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45321d2640 1 -- 172.21.15.106:0/1031593394 >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 msgr2=0x7f452c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45321d2640 1 --2- 172.21.15.106:0/1031593394 >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 0x7f452c06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45321d2640 1 -- 172.21.15.106:0/1031593394 >> v1:172.21.15.114:6789/0 conn(0x7f452c06ebf0 legacy=0x7f452c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45321d2640 1 -- 172.21.15.106:0/1031593394 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c15d970 con 0x7f452c156b40 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45329d3640 1 --2- 172.21.15.106:0/1031593394 >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 0x7f452c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:24.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45321d2640 1 --2- 172.21.15.106:0/1031593394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c06e6b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f452c09efe0 tx=0x7f452802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec5b88a52b05e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45311d0640 1 -- 172.21.15.106:0/1031593394 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528035020 con 0x7f452c156b40 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45311d0640 1 -- 172.21.15.106:0/1031593394 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4528031870 con 0x7f452c156b40 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.252+0000 7f45311d0640 1 -- 172.21.15.106:0/1031593394 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528031c40 con 0x7f452c156b40 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 -- 172.21.15.106:0/1031593394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 msgr2=0x7f452c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 --2- 172.21.15.106:0/1031593394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c06e6b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f452c09efe0 tx=0x7f452802b910 comp rx=0 tx=0).stop 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 -- 172.21.15.106:0/1031593394 shutdown_connections 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 --2- 172.21.15.106:0/1031593394 >> v2:172.21.15.114:3300/0 conn(0x7f452c06c330 0x7f452c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 --2- 172.21.15.106:0/1031593394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c06e6b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 -- 172.21.15.106:0/1031593394 >> 172.21.15.106:0/1031593394 conn(0x7f452c0cd6a0 msgr2=0x7f452c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 -- 172.21.15.106:0/1031593394 shutdown_connections 2024-04-13T03:13:24.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.253+0000 7f4534436640 1 -- 172.21.15.106:0/1031593394 wait complete. 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.254+0000 7f4534436640 1 Processor -- start 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.254+0000 7f4534436640 1 -- start start 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c06c330 0x7f452c0b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f452c06ebf0 0x7f452c146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f452c15d2e0 con 0x7f452c156b40 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f452c156f20 con 0x7f452c06c330 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f4534436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f452c1586d0 con 0x7f452c06ebf0 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34372/0 (socket says 172.21.15.106:34372) 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 -- 172.21.15.106:0/1247305524 learned_addr learned my addr 172.21.15.106:0/1247305524 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45319d1640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f452c06ebf0 0x7f452c146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45321d2640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c06c330 0x7f452c0b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 -- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f452c06ebf0 msgr2=0x7f452c146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f452c06ebf0 0x7f452c146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 -- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c06c330 msgr2=0x7f452c0b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c06c330 0x7f452c0b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.255+0000 7f45329d3640 1 -- 172.21.15.106:0/1247305524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c1589b0 con 0x7f452c156b40 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f45329d3640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f452400cef0 tx=0x7f452400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452401a070 con 0x7f452c156b40 2024-04-13T03:13:24.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452400d860 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c158c90 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400dc50 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.256+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c159250 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.257+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c159450 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.257+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f452401d070 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.258+0000 7f4522ffd640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 0x7f451006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.258+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f452408c600 con 0x7f452c156b40 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.258+0000 7f45321d2640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 0x7f451006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.258+0000 7f45321d2640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 0x7f451006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4528036000 tx=0x7f452803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.262+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f452c159450 con 0x7f452c156b40 2024-04-13T03:13:24.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.435+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f452c068a10 con 0x7f452c156b40 2024-04-13T03:13:24.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.435+0000 7f4522ffd640 1 -- 172.21.15.106:0/1247305524 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f452405a8d0 con 0x7f452c156b40 2024-04-13T03:13:24.437 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:13:24.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 msgr2=0x7f451006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 0x7f451006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4528036000 tx=0x7f452803c040 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 msgr2=0x7f452c159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f452400cef0 tx=0x7f452400a710 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 shutdown_connections 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f451006c8f0 0x7f451006edb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f452c06ebf0 0x7f452c146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c156b40 0x7f452c159770 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 --2- 172.21.15.106:0/1247305524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c06c330 0x7f452c0b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 >> 172.21.15.106:0/1247305524 conn(0x7f452c0cd6a0 msgr2=0x7f452c1515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 shutdown_connections 2024-04-13T03:13:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.437+0000 7f4534436640 1 -- 172.21.15.106:0/1247305524 wait complete. 2024-04-13T03:13:24.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:13:24.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.551+0000 7f2150f5e640 1 Processor -- start 2024-04-13T03:13:24.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.551+0000 7f2150f5e640 1 -- start start 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f214c0b1c30 0x7f214c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f214c0709e0 0x7f214c162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f214c0b5cf0 con 0x7f214c0709e0 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f214c0b5fd0 con 0x7f214c0b1c30 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2150f5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f214c0b62b0 con 0x7f214c0b54a0 2024-04-13T03:13:24.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f214c0709e0 0x7f214c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f214c0709e0 0x7f214c162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34386/0 (socket says 172.21.15.106:34386) 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 -- 172.21.15.106:0/943052459 learned_addr learned my addr 172.21.15.106:0/943052459 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f2149d9b640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 -- 172.21.15.106:0/943052459 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 msgr2=0x7f214c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c0704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 -- 172.21.15.106:0/943052459 >> v2:172.21.15.114:3300/0 conn(0x7f214c0b1c30 msgr2=0x7f214c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214a59c640 1 --2- 172.21.15.106:0/943052459 >> v2:172.21.15.114:3300/0 conn(0x7f214c0b1c30 0x7f214c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 --2- 172.21.15.106:0/943052459 >> v2:172.21.15.114:3300/0 conn(0x7f214c0b1c30 0x7f214c15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.552+0000 7f214ad9d640 1 -- 172.21.15.106:0/943052459 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c0b6590 con 0x7f214c0709e0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.553+0000 7f214ad9d640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0709e0 0x7f214c162430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f213c009870 tx=0x7f213c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f51ad4e994d50ae server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.553+0000 7f214959a640 1 -- 172.21.15.106:0/943052459 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c035020 con 0x7f214c0709e0 2024-04-13T03:13:24.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.553+0000 7f214959a640 1 -- 172.21.15.106:0/943052459 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c031640 con 0x7f214c0709e0 2024-04-13T03:13:24.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.553+0000 7f214959a640 1 -- 172.21.15.106:0/943052459 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c02d040 con 0x7f214c0709e0 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 -- 172.21.15.106:0/943052459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0709e0 msgr2=0x7f214c162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0709e0 0x7f214c162430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f213c009870 tx=0x7f213c02b8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 -- 172.21.15.106:0/943052459 shutdown_connections 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0709e0 0x7f214c162430 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 --2- 172.21.15.106:0/943052459 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 --2- 172.21.15.106:0/943052459 >> v2:172.21.15.114:3300/0 conn(0x7f214c0b1c30 0x7f214c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 -- 172.21.15.106:0/943052459 >> 172.21.15.106:0/943052459 conn(0x7f214c071870 msgr2=0x7f214c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 -- 172.21.15.106:0/943052459 shutdown_connections 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.554+0000 7f2150f5e640 1 -- 172.21.15.106:0/943052459 wait complete. 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.555+0000 7f2150f5e640 1 Processor -- start 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.555+0000 7f2150f5e640 1 -- start start 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f214c0709e0 0x7f214c153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f214c162af0 con 0x7f214c0b1c30 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f214c162970 con 0x7f214c0709e0 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2150f5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f214c162c70 con 0x7f214c0b54a0 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34398/0 (socket says 172.21.15.106:34398) 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 -- 172.21.15.106:0/2805997003 learned_addr learned my addr 172.21.15.106:0/2805997003 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f214ad9d640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f214a59c640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f214c0709e0 0x7f214c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 -- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 msgr2=0x7f214c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 -- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f214c0709e0 msgr2=0x7f214c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f214c0709e0 0x7f214c153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.556+0000 7f2149d9b640 1 -- 172.21.15.106:0/2805997003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c0ccc50 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f2149d9b640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f214000c820 tx=0x7f214000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140015070 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2140004020 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c0ccf30 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400043f0 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.557+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c0cd4f0 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.558+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c068a10 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.558+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2140007830 con 0x7f214c0b1c30 2024-04-13T03:13:24.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.559+0000 7f213affd640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 0x7f212c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.559+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f214008bb80 con 0x7f214c0b1c30 2024-04-13T03:13:24.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.559+0000 7f214a59c640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 0x7f212c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.559+0000 7f214a59c640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 0x7f212c06edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2144007fd0 tx=0x7f2144007560 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.563+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2140059d90 con 0x7f214c0b1c30 2024-04-13T03:13:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.735+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f214c0711f0 con 0x7f214c0b1c30 2024-04-13T03:13:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.736+0000 7f213affd640 1 -- 172.21.15.106:0/2805997003 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f2140059690 con 0x7f214c0b1c30 2024-04-13T03:13:24.737 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 msgr2=0x7f212c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 0x7f212c06edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2144007fd0 tx=0x7f2144007560 comp rx=0 tx=0).stop 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 msgr2=0x7f214c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f214000c820 tx=0x7f214000cce0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 shutdown_connections 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f212c06c8f0 0x7f212c06edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f214c0b54a0 0x7f214c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c0b1c30 0x7f214c153b20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.737+0000 7f2150f5e640 1 --2- 172.21.15.106:0/2805997003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f214c0709e0 0x7f214c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.738+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 >> 172.21.15.106:0/2805997003 conn(0x7f214c071870 msgr2=0x7f214c0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.738+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 shutdown_connections 2024-04-13T03:13:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.738+0000 7f2150f5e640 1 -- 172.21.15.106:0/2805997003 wait complete. 2024-04-13T03:13:24.750 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:24.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.848+0000 7f0b77711640 1 Processor -- start 2024-04-13T03:13:24.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 -- start start 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 0x7f0b70072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b7006e150 0x7f0b7006e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b700998d0 con 0x7f0b700cdf50 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0b70073020 con 0x7f0b7006e150 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.849+0000 7f0b77711640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b70073300 con 0x7f0b7006cb10 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b75cae640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b7006e150 0x7f0b7006e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b754ad640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b700cdf50 0x7f0b700ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41306/0 (socket says 172.21.15.106:41306) 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b754ad640 1 -- 172.21.15.106:0/1325191373 learned_addr learned my addr 172.21.15.106:0/1325191373 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b74cac640 1 --2- 172.21.15.106:0/1325191373 >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 0x7f0b70072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b75cae640 1 -- 172.21.15.106:0/1325191373 >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 msgr2=0x7f0b70072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b75cae640 1 --2- 172.21.15.106:0/1325191373 >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 0x7f0b70072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b75cae640 1 -- 172.21.15.106:0/1325191373 >> v1:172.21.15.106:6789/0 conn(0x7f0b700cdf50 legacy=0x7f0b700ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b75cae640 1 -- 172.21.15.106:0/1325191373 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b700735e0 con 0x7f0b7006e150 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b67fff640 1 -- 172.21.15.106:0/1325191373 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109170430 0 0) 0x7f0b700998d0 con 0x7f0b700cdf50 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.850+0000 7f0b74cac640 1 --2- 172.21.15.106:0/1325191373 >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 0x7f0b70072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b75cae640 1 --2- 172.21.15.106:0/1325191373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006e150 0x7f0b7006e600 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b64f0 tx=0x7f0b6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4325d7fdca4c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b67fff640 1 -- 172.21.15.106:0/1325191373 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68039070 con 0x7f0b7006e150 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b67fff640 1 -- 172.21.15.106:0/1325191373 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6802cc80 con 0x7f0b7006e150 2024-04-13T03:13:24.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b67fff640 1 -- 172.21.15.106:0/1325191373 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68034660 con 0x7f0b7006e150 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 -- 172.21.15.106:0/1325191373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006e150 msgr2=0x7f0b7006e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 --2- 172.21.15.106:0/1325191373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006e150 0x7f0b7006e600 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b64f0 tx=0x7f0b6802bed0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 -- 172.21.15.106:0/1325191373 shutdown_connections 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 --2- 172.21.15.106:0/1325191373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006e150 0x7f0b7006e600 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 --2- 172.21.15.106:0/1325191373 >> v2:172.21.15.114:3300/0 conn(0x7f0b7006cb10 0x7f0b70072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 -- 172.21.15.106:0/1325191373 >> 172.21.15.106:0/1325191373 conn(0x7f0b700b1f10 msgr2=0x7f0b700b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.851+0000 7f0b77711640 1 -- 172.21.15.106:0/1325191373 shutdown_connections 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.852+0000 7f0b77711640 1 -- 172.21.15.106:0/1325191373 wait complete. 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 Processor -- start 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 -- start start 2024-04-13T03:13:24.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 0x7f0b70157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b700cdf50 0x7f0b701e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70072de0 con 0x7f0b7006cb10 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70072c60 con 0x7f0b7006e150 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b77711640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7006eb40 con 0x7f0b700cdf50 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b754ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b754ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34418/0 (socket says 172.21.15.106:34418) 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.853+0000 7f0b754ad640 1 -- 172.21.15.106:0/3982578893 learned_addr learned my addr 172.21.15.106:0/3982578893 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b75cae640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b700cdf50 0x7f0b701e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b74cac640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 0x7f0b70157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 -- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b700cdf50 msgr2=0x7f0b701e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b700cdf50 0x7f0b701e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 -- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 msgr2=0x7f0b70157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 0x7f0b70157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:24.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 -- 172.21.15.106:0/3982578893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b70154820 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b74cac640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 0x7f0b70157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b754ad640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c430 tx=0x7f0b6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600074e0 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b600047e0 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60004bd0 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b70154dc0 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.854+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b700adb80 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.855+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b70068a10 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.855+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b60011070 con 0x7f0b7006cb10 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.856+0000 7f0b65ffb640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.856+0000 7f0b74cac640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:24.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.856+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f0b6008b8e0 con 0x7f0b7006cb10 2024-04-13T03:13:24.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.856+0000 7f0b74cac640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b701e8f00 tx=0x7f0b6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:24.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:24.860+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b70154820 con 0x7f0b7006cb10 2024-04-13T03:13:24.986 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:13:24.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:13:25.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.077+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b700cd900 con 0x7f0b7006cb10 2024-04-13T03:13:25.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.077+0000 7f0b65ffb640 1 -- 172.21.15.106:0/3982578893 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5186 (secure 0 0 0) 0x7f0b60059af0 con 0x7f0b7006cb10 2024-04-13T03:13:25.078 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:25.079 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:52.017572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:13:25.079 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 30 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.079+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 msgr2=0x7f0b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b701e8f00 tx=0x7f0b6c008040 comp rx=0 tx=0).stop 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 msgr2=0x7f0b701542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c430 tx=0x7f0b6000c8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 shutdown_connections 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b700cdf50 0x7f0b701e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b7006e150 0x7f0b70157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 --2- 172.21.15.106:0/3982578893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b7006cb10 0x7f0b701542e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 >> 172.21.15.106:0/3982578893 conn(0x7f0b700b1f10 msgr2=0x7f0b7015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 shutdown_connections 2024-04-13T03:13:25.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.080+0000 7f0b77711640 1 -- 172.21.15.106:0/3982578893 wait complete. 2024-04-13T03:13:25.091 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:25.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.192+0000 7fa753be9640 1 Processor -- start 2024-04-13T03:13:25.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.192+0000 7fa753be9640 1 -- start start 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa753be9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa74c0aebe0 0x7fa74c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa753be9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa74c0aefc0 con 0x7fa74c0b5ab0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa753be9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa74c0ad180 con 0x7fa74c15ce90 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa753be9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa74c06e180 con 0x7fa74c0aebe0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa751985640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa74c0aebe0 0x7fa74c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa752186640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa74c0b5ab0 0x7fa74c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41320/0 (socket says 172.21.15.106:41320) 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa752186640 1 -- 172.21.15.106:0/2061883077 learned_addr learned my addr 172.21.15.106:0/2061883077 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa751985640 1 -- 172.21.15.106:0/2061883077 >> v1:172.21.15.114:6789/0 conn(0x7fa74c15ce90 legacy=0x7fa74c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa751985640 1 -- 172.21.15.106:0/2061883077 >> v1:172.21.15.106:6789/0 conn(0x7fa74c0b5ab0 legacy=0x7fa74c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa751985640 1 -- 172.21.15.106:0/2061883077 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c160ca0 con 0x7fa74c0aebe0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.193+0000 7fa750983640 1 -- 172.21.15.106:0/2061883077 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969312371 0 0) 0x7fa744001490 con 0x7fa74c0b5ab0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa751985640 1 --2- 172.21.15.106:0/2061883077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0aebe0 0x7fa74c1603a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0cdea0 tx=0x7fa74802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c069622b8dff6d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa750983640 1 -- 172.21.15.106:0/2061883077 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7480315a0 con 0x7fa74c0aebe0 2024-04-13T03:13:25.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa750983640 1 -- 172.21.15.106:0/2061883077 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa748037070 con 0x7fa74c0aebe0 2024-04-13T03:13:25.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa750983640 1 -- 172.21.15.106:0/2061883077 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748031960 con 0x7fa74c0aebe0 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa753be9640 1 -- 172.21.15.106:0/2061883077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0aebe0 msgr2=0x7fa74c1603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa753be9640 1 --2- 172.21.15.106:0/2061883077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0aebe0 0x7fa74c1603a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0cdea0 tx=0x7fa74802b840 comp rx=0 tx=0).stop 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa753be9640 1 -- 172.21.15.106:0/2061883077 shutdown_connections 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa753be9640 1 --2- 172.21.15.106:0/2061883077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0aebe0 0x7fa74c1603a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.194+0000 7fa753be9640 1 -- 172.21.15.106:0/2061883077 >> 172.21.15.106:0/2061883077 conn(0x7fa74c0b1f10 msgr2=0x7fa74c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.195+0000 7fa753be9640 1 -- 172.21.15.106:0/2061883077 shutdown_connections 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.195+0000 7fa753be9640 1 -- 172.21.15.106:0/2061883077 wait complete. 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 Processor -- start 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 -- start start 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 0x7fa74c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 0x7fa74c0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c06de10 con 0x7fa74c15ce90 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c161260 con 0x7fa74c0aebe0 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa753be9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c1608e0 con 0x7fa74c0b5ab0 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa752186640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa752186640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34436/0 (socket says 172.21.15.106:34436) 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.196+0000 7fa752186640 1 -- 172.21.15.106:0/3035281221 learned_addr learned my addr 172.21.15.106:0/3035281221 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa751985640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 0x7fa74c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa751184640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 0x7fa74c0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 -- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 msgr2=0x7fa74c0c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 0x7fa74c0c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 -- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 msgr2=0x7fa74c156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 0x7fa74c156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 -- 172.21.15.106:0/3035281221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c15b1e0 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa751985640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 0x7fa74c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa751184640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 0x7fa74c0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa752186640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa74400c7c0 tx=0x7fa74400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744018070 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74400a5e0 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400a9d0 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c15b4c0 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.197+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c0ca1e0 con 0x7fa74c15ce90 2024-04-13T03:13:25.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.198+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c15ff60 con 0x7fa74c15ce90 2024-04-13T03:13:25.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.198+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa74401b070 con 0x7fa74c15ce90 2024-04-13T03:13:25.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.199+0000 7fa7427fc640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 0x7fa73006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.199+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fa74408b2c0 con 0x7fa74c15ce90 2024-04-13T03:13:25.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.199+0000 7fa751985640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 0x7fa73006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.199+0000 7fa751985640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 0x7fa73006edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa748036000 tx=0x7fa74803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.203+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa74c15ff60 con 0x7fa74c15ce90 2024-04-13T03:13:25.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.379+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa74c068a10 con 0x7fa74c15ce90 2024-04-13T03:13:25.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.380+0000 7fa7427fc640 1 -- 172.21.15.106:0/3035281221 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17160 (secure 0 0 0) 0x7fa74c0aefc0 con 0x7fa74c15ce90 2024-04-13T03:13:25.381 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:25.382 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":66,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:07.122095+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 msgr2=0x7fa73006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 0x7fa73006edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa748036000 tx=0x7fa74803d040 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 msgr2=0x7fa74c0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa74400c7c0 tx=0x7fa74400cc80 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 shutdown_connections 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa73006c8f0 0x7fa73006edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.382+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa74c0b5ab0 0x7fa74c0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.383+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c15ce90 0x7fa74c0c7990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.383+0000 7fa753be9640 1 --2- 172.21.15.106:0/3035281221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa74c0aebe0 0x7fa74c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.383+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 >> 172.21.15.106:0/3035281221 conn(0x7fa74c0b1f10 msgr2=0x7fa74c0b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.383+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 shutdown_connections 2024-04-13T03:13:25.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.383+0000 7fa753be9640 1 -- 172.21.15.106:0/3035281221 wait complete. 2024-04-13T03:13:25.394 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:13:25.395 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:25.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.493+0000 7fd000e36640 1 Processor -- start 2024-04-13T03:13:25.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.494+0000 7fd000e36640 1 -- start start 2024-04-13T03:13:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.494+0000 7fd000e36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcffc06f180 0x7fcffc166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.494+0000 7fd000e36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc072de0 con 0x7fcffc06cb80 2024-04-13T03:13:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.494+0000 7fd000e36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc073090 con 0x7fcffc0cdf50 2024-04-13T03:13:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.494+0000 7fd000e36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc073370 con 0x7fcffc06f180 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff9d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcffc06cb80 0x7fcffc0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41322/0 (socket says 172.21.15.106:41322) 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff9d9b640 1 -- 172.21.15.106:0/3986384677 learned_addr learned my addr 172.21.15.106:0/3986384677 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcffad9d640 1 --2- 172.21.15.106:0/3986384677 >> v2:172.21.15.114:3300/0 conn(0x7fcffc06f180 0x7fcffc166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061174084 0 0) 0x7fcffc072de0 con 0x7fcffc06cb80 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfe4003520 con 0x7fcffc06cb80 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236694981 0 0) 0x7fcffc073090 con 0x7fcffc0cdf50 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcffc072de0 con 0x7fcffc0cdf50 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 667901164 0 0) 0x7fcfe4003520 con 0x7fcffc06cb80 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 >> v2:172.21.15.114:3300/0 conn(0x7fcffc06f180 msgr2=0x7fcffc166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 --2- 172.21.15.106:0/3986384677 >> v2:172.21.15.114:3300/0 conn(0x7fcffc06f180 0x7fcffc166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 >> v1:172.21.15.114:6789/0 conn(0x7fcffc0cdf50 legacy=0x7fcffc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc073650 con 0x7fcffc06cb80 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fcfec001670 con 0x7fcffc06cb80 2024-04-13T03:13:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 >> v1:172.21.15.106:6789/0 conn(0x7fcffc06cb80 legacy=0x7fcffc0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 0x7fcfe4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfe4004800 0x7fcfe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe40051c0 0x7fcfe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc073370 con 0x7fcfe4003b90 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe400c090 con 0x7fcfe4004800 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe400c290 con 0x7fcfe40051c0 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcffa59c640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 0x7fcfe4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcffad9d640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe40051c0 0x7fcfe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.495+0000 7fcff9d9b640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfe4004800 0x7fcfe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 -- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe40051c0 msgr2=0x7fcfe400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe40051c0 0x7fcfe400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 -- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfe4004800 msgr2=0x7fcfe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfe4004800 0x7fcfe4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 -- 172.21.15.106:0/3986384677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe4003520 con 0x7fcfe4003b90 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffad9d640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe40051c0 0x7fcfe400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffad9d640 1 -- 172.21.15.106:0/3986384677 reap_dead start 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcffa59c640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 0x7fcfe4003f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcff00014b0 tx=0x7fcfe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e5264bb6d76516 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00014f0 con 0x7fcfe4003b90 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff00052d0 con 0x7fcfe4003b90 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.496+0000 7fcff959a640 1 -- 172.21.15.106:0/3986384677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0003810 con 0x7fcfe4003b90 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 -- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 msgr2=0x7fcfe4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 0x7fcfe4003f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcff00014b0 tx=0x7fcfe4005830 comp rx=0 tx=0).stop 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 -- 172.21.15.106:0/3986384677 shutdown_connections 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 --2- 172.21.15.106:0/3986384677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe4003b90 0x7fcfe4003f90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 -- 172.21.15.106:0/3986384677 >> 172.21.15.106:0/3986384677 conn(0x7fcffc0b1f10 msgr2=0x7fcffc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 -- 172.21.15.106:0/3986384677 shutdown_connections 2024-04-13T03:13:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.497+0000 7fd000e36640 1 -- 172.21.15.106:0/3986384677 wait complete. 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.498+0000 7fd000e36640 1 Processor -- start 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.498+0000 7fd000e36640 1 -- start start 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 0x7fcffc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 0x7fcffc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc099930 con 0x7fcffc153030 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc0997b0 con 0x7fcffc15e6c0 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fd000e36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc0726d0 con 0x7fcfe4003b90 2024-04-13T03:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34456/0 (socket says 172.21.15.106:34456) 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 -- 172.21.15.106:0/2080248541 learned_addr learned my addr 172.21.15.106:0/2080248541 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcffa59c640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 0x7fcffc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcffad9d640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 0x7fcffc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 -- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 msgr2=0x7fcffc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 0x7fcffc0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 -- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 msgr2=0x7fcffc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 0x7fcffc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcff9d9b640 1 -- 172.21.15.106:0/2080248541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc158a00 con 0x7fcffc153030 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcffa59c640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 0x7fcffc0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.499+0000 7fcffad9d640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 0x7fcffc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fcff9d9b640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcfec007290 tx=0x7fcfec007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec001610 con 0x7fcffc153030 2024-04-13T03:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec018070 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc158ce0 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00fc00 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.500+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1e4730 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.501+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc1e4930 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.501+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfec01d460 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.502+0000 7fcfeaffd640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 0x7fcfdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.502+0000 7fcffa59c640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 0x7fcfdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.502+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fcfec08bce0 con 0x7fcffc153030 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.502+0000 7fcffa59c640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 0x7fcfdc06ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcff0036000 tx=0x7fcff0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.506+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcffc1e4930 con 0x7fcffc153030 2024-04-13T03:13:25.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.717+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcffc068a10 con 0x7fcffc153030 2024-04-13T03:13:25.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.717+0000 7fcfeaffd640 1 -- 172.21.15.106:0/2080248541 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5186 (secure 0 0 0) 0x7fcfec059ef0 con 0x7fcffc153030 2024-04-13T03:13:25.719 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:25.719 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:52.017572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:13:25.719 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 30 2024-04-13T03:13:25.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 msgr2=0x7fcfdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 0x7fcfdc06ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcff0036000 tx=0x7fcff0033000 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 msgr2=0x7fcffc098950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcfec007290 tx=0x7fcfec007750 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 shutdown_connections 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfdc06c920 0x7fcfdc06ede0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcffc15e6c0 0x7fcffc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcffc153030 0x7fcffc098950 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 --2- 172.21.15.106:0/2080248541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfe4003b90 0x7fcffc0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 >> 172.21.15.106:0/2080248541 conn(0x7fcffc0b1f10 msgr2=0x7fcffc0ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.719+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 shutdown_connections 2024-04-13T03:13:25.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.720+0000 7fd000e36640 1 -- 172.21.15.106:0/2080248541 wait complete. 2024-04-13T03:13:25.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:25.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 Processor -- start 2024-04-13T03:13:25.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 -- start start 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2e40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e40cfd20 con 0x7fe2e415e9b0 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e4162af0 con 0x7fe2e415ed90 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.831+0000 7fe2eaf15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e4162dd0 con 0x7fe2e40b3d60 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e8cb1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe2e415e9b0 0x7fe2e40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41330/0 (socket says 172.21.15.106:41330) 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e8cb1640 1 -- 172.21.15.106:0/1795234339 learned_addr learned my addr 172.21.15.106:0/1795234339 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:25.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2e40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817010911 0 0) 0x7fe2e40cfd20 con 0x7fe2e415e9b0 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2cc003520 con 0x7fe2e415e9b0 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236983069 0 0) 0x7fe2e4162af0 con 0x7fe2e415ed90 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e40cfd20 con 0x7fe2e415ed90 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659705326 0 0) 0x7fe2cc003520 con 0x7fe2e415e9b0 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 msgr2=0x7fe2e40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2e40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 >> v1:172.21.15.114:6789/0 conn(0x7fe2e415ed90 legacy=0x7fe2e40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e41630b0 con 0x7fe2e415e9b0 2024-04-13T03:13:25.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fe2d4001670 con 0x7fe2e415e9b0 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 >> v1:172.21.15.106:6789/0 conn(0x7fe2e415e9b0 legacy=0x7fe2e40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2cc003b70 0x7fe2cc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2cc004d40 0x7fe2cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e4162dd0 con 0x7fe2cc004d40 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc00ba70 con 0x7fe2cc003b70 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc00bc20 con 0x7fe2e40b3d60 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2cc004d40 0x7fe2cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e3fff640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.832+0000 7fe2e8cb1640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2cc003b70 0x7fe2cc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 -- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 msgr2=0x7fe2cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e40b3d60 0x7fe2cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 -- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2cc003b70 msgr2=0x7fe2cc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:25.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2cc003b70 0x7fe2cc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 -- 172.21.15.106:0/1795234339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc003520 con 0x7fe2cc004d40 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2cc004d40 0x7fe2cc0051c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fc0 tx=0x7fe2cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35919fe180e955ee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc039070 con 0x7fe2cc004d40 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc02cc20 con 0x7fe2cc004d40 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.833+0000 7fe2e37fe640 1 -- 172.21.15.106:0/1795234339 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc009c10 con 0x7fe2cc004d40 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 -- 172.21.15.106:0/1795234339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2cc004d40 msgr2=0x7fe2cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/1795234339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2cc004d40 0x7fe2cc0051c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fc0 tx=0x7fe2cc005830 comp rx=0 tx=0).stop 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2e94b2640 1 -- 172.21.15.106:0/1795234339 reap_dead start 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 -- 172.21.15.106:0/1795234339 shutdown_connections 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 -- 172.21.15.106:0/1795234339 >> 172.21.15.106:0/1795234339 conn(0x7fe2e408bf10 msgr2=0x7fe2e40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 -- 172.21.15.106:0/1795234339 shutdown_connections 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.834+0000 7fe2eaf15640 1 -- 172.21.15.106:0/1795234339 wait complete. 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.835+0000 7fe2eaf15640 1 Processor -- start 2024-04-13T03:13:25.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.835+0000 7fe2eaf15640 1 -- start start 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e41f5c80 0x7fe2e40c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e40c9070 0x7fe2e40b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e4162800 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e4162680 con 0x7fe2e40c9070 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2eaf15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e40d4d00 con 0x7fe2e41f5c80 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34482/0 (socket says 172.21.15.106:34482) 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 -- 172.21.15.106:0/214508408 learned_addr learned my addr 172.21.15.106:0/214508408 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e3fff640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e41f5c80 0x7fe2e40c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e94b2640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e40c9070 0x7fe2e40b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 -- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e41f5c80 msgr2=0x7fe2e40c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e41f5c80 0x7fe2e40c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 -- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e40c9070 msgr2=0x7fe2e40b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e40c9070 0x7fe2e40b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.836+0000 7fe2e8cb1640 1 -- 172.21.15.106:0/214508408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e40b4390 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2e8cb1640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40072f0 tx=0x7fe2d40076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400f960 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d4018070 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e40b4670 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400fd60 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.837+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e40b6be0 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.838+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e4068a10 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.838+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2d401d460 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.838+0000 7fe2e17fa640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 0x7fe2d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.838+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7fe2d408bd10 con 0x7fe2e41f1ad0 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.838+0000 7fe2e3fff640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 0x7fe2d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.839+0000 7fe2e3fff640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 0x7fe2d006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2e41f6060 tx=0x7fe2d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:25.843+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2d4059fb0 con 0x7fe2e41f1ad0 2024-04-13T03:13:26.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.055+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2e40d53a0 con 0x7fe2e41f1ad0 2024-04-13T03:13:26.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.055+0000 7fe2e17fa640 1 -- 172.21.15.106:0/214508408 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5186 (secure 0 0 0) 0x7fe2d40598b0 con 0x7fe2e41f1ad0 2024-04-13T03:13:26.056 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:26.057 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:12:11.338248+0000","modified":"2024-04-13T03:12:52.017572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6273":{"gid":6273,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6836/536113082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":536113082},{"type":"v1","addr":"172.21.15.106:6836","nonce":536113082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6273,"qdb_cluster":[6273]},"id":4}]} 2024-04-13T03:13:26.057 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 30 2024-04-13T03:13:26.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 msgr2=0x7fe2d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 0x7fe2d006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2e41f6060 tx=0x7fe2d8009040 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 msgr2=0x7fe2e41f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40072f0 tx=0x7fe2d40076f0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 shutdown_connections 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2d006c8f0 0x7fe2d006edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2e41f5c80 0x7fe2e40c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e40c9070 0x7fe2e40b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 --2- 172.21.15.106:0/214508408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2e41f1ad0 0x7fe2e41f5740 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 >> 172.21.15.106:0/214508408 conn(0x7fe2e408bf10 msgr2=0x7fe2e40b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 shutdown_connections 2024-04-13T03:13:26.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.057+0000 7fe2eaf15640 1 -- 172.21.15.106:0/214508408 wait complete. 2024-04-13T03:13:26.069 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.168+0000 7f9f8eddc640 1 Processor -- start 2024-04-13T03:13:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.168+0000 7f9f8eddc640 1 -- start start 2024-04-13T03:13:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8eddc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8815b6a0 0x7f9f880d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8eddc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f880734c0 0x7f9f880cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8eddc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f880d0880 con 0x7f9f88073e10 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8eddc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8806d280 con 0x7f9f8815b6a0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8eddc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9f880d0b60 con 0x7f9f880734c0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8815b6a0 0x7f9f880d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9f8815b6a0 0x7f9f880d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34496/0 (socket says 172.21.15.106:34496) 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 -- 172.21.15.106:0/3006132410 learned_addr learned my addr 172.21.15.106:0/3006132410 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8d379640 1 --2- 172.21.15.106:0/3006132410 >> v2:172.21.15.114:3300/0 conn(0x7f9f880734c0 0x7f9f880cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 -- 172.21.15.106:0/3006132410 >> v2:172.21.15.114:3300/0 conn(0x7f9f880734c0 msgr2=0x7f9f880cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 --2- 172.21.15.106:0/3006132410 >> v2:172.21.15.114:3300/0 conn(0x7f9f880734c0 0x7f9f880cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 -- 172.21.15.106:0/3006132410 >> v1:172.21.15.114:6789/0 conn(0x7f9f88073e10 legacy=0x7f9f88072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.169+0000 7f9f8cb78640 1 -- 172.21.15.106:0/3006132410 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f880d0e40 con 0x7f9f8815b6a0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f8cb78640 1 --2- 172.21.15.106:0/3006132410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8815b6a0 0x7f9f880d02a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9f88072d40 tx=0x7f9f7c02e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4393c4a626959af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f877fe640 1 -- 172.21.15.106:0/3006132410 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c03b070 con 0x7f9f8815b6a0 2024-04-13T03:13:26.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f877fe640 1 -- 172.21.15.106:0/3006132410 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c036440 con 0x7f9f8815b6a0 2024-04-13T03:13:26.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f877fe640 1 -- 172.21.15.106:0/3006132410 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c036810 con 0x7f9f8815b6a0 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f8eddc640 1 -- 172.21.15.106:0/3006132410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8815b6a0 msgr2=0x7f9f880d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.170+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/3006132410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8815b6a0 0x7f9f880d02a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9f88072d40 tx=0x7f9f7c02e170 comp rx=0 tx=0).stop 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 -- 172.21.15.106:0/3006132410 shutdown_connections 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/3006132410 >> v2:172.21.15.114:3300/0 conn(0x7f9f880734c0 0x7f9f880cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/3006132410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8815b6a0 0x7f9f880d02a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 -- 172.21.15.106:0/3006132410 >> 172.21.15.106:0/3006132410 conn(0x7f9f880cdef0 msgr2=0x7f9f880ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 -- 172.21.15.106:0/3006132410 shutdown_connections 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.171+0000 7f9f8eddc640 1 -- 172.21.15.106:0/3006132410 wait complete. 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.172+0000 7f9f8eddc640 1 Processor -- start 2024-04-13T03:13:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.172+0000 7f9f8eddc640 1 -- start start 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f880734c0 0x7f9f88155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8815b6a0 0x7f9f881f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880cd760 con 0x7f9f88073e10 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880cd5e0 con 0x7f9f880734c0 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f880cd8e0 con 0x7f9f8815b6a0 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34502/0 (socket says 172.21.15.106:34502) 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 -- 172.21.15.106:0/1377644619 learned_addr learned my addr 172.21.15.106:0/1377644619 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8d379640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8815b6a0 0x7f9f881f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8cb78640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f880734c0 0x7f9f88155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 -- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8815b6a0 msgr2=0x7f9f881f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8815b6a0 0x7f9f881f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 -- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f880734c0 msgr2=0x7f9f88155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f880734c0 0x7f9f88155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 -- 172.21.15.106:0/1377644619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8815ca90 con 0x7f9f88073e10 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f87fff640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c400 tx=0x7f9f7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740052d0 con 0x7f9f88073e10 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74005490 con 0x7f9f88073e10 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f880b4230 con 0x7f9f88073e10 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74007260 con 0x7f9f88073e10 2024-04-13T03:13:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.173+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8815cd00 con 0x7f9f88073e10 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.174+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f880b2ee0 con 0x7f9f88073e10 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.175+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f74007400 con 0x7f9f88073e10 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.175+0000 7f9f857fa640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.175+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f9f7408b250 con 0x7f9f88073e10 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.175+0000 7f9f8cb78640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.176+0000 7f9f8cb78640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c038000 tx=0x7f9f7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.180+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f880b2ee0 con 0x7f9f88073e10 2024-04-13T03:13:26.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.354+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f88069120 con 0x7f9f88073e10 2024-04-13T03:13:26.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.355+0000 7f9f857fa640 1 -- 172.21.15.106:0/1377644619 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17160 (secure 0 0 0) 0x7f9f74058e10 con 0x7f9f88073e10 2024-04-13T03:13:26.356 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:26.356 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":66,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:07.122095+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:12:09.560834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-13T03:12:10.285668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:26.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 msgr2=0x7f9f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c038000 tx=0x7f9f7c008040 comp rx=0 tx=0).stop 2024-04-13T03:13:26.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 msgr2=0x7f9f881f8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c400 tx=0x7f9f7400c8c0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 shutdown_connections 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9f8815b6a0 0x7f9f881f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f88073e10 0x7f9f881f8070 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 --2- 172.21.15.106:0/1377644619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f880734c0 0x7f9f88155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 >> 172.21.15.106:0/1377644619 conn(0x7f9f880cdef0 msgr2=0x7f9f88157890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 shutdown_connections 2024-04-13T03:13:26.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.357+0000 7f9f8eddc640 1 -- 172.21.15.106:0/1377644619 wait complete. 2024-04-13T03:13:26.369 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:13:26.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.468+0000 7f974cdbb640 1 Processor -- start 2024-04-13T03:13:26.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.469+0000 7f974cdbb640 1 -- start start 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.469+0000 7f974cdbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f974cdbb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97480d07c0 0x7f974806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f974cdbb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97480cef80 con 0x7f974815fb50 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f974cdbb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f97480cf260 con 0x7f97480d07c0 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f974cdbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f97480cf540 con 0x7f974806bff0 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9745d9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97480d07c0 0x7f974806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97480d07c0 0x7f974806ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:41744/0 (socket says 172.21.15.106:41744) 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9745d9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:54758/0 (socket says 172.21.15.106:54758) 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 -- 172.21.15.106:0/1608393498 learned_addr learned my addr 172.21.15.106:0/1608393498 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 -- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 msgr2=0x7f974806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 -- 172.21.15.106:0/1608393498 >> v1:172.21.15.114:6789/0 conn(0x7f974815fb50 legacy=0x7f974806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9746d9d640 1 -- 172.21.15.106:0/1608393498 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97480cf820 con 0x7f97480d07c0 2024-04-13T03:13:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.470+0000 7f9745d9b640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.471+0000 7f9746d9d640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97480d07c0 0x7f974806ebf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9740009e50 tx=0x7f974002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0eb580b92191f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.471+0000 7f974559a640 1 -- 172.21.15.106:0/1608393498 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974002ca70 con 0x7f97480d07c0 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.471+0000 7f974559a640 1 -- 172.21.15.106:0/1608393498 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974002cc10 con 0x7f97480d07c0 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.471+0000 7f974559a640 1 -- 172.21.15.106:0/1608393498 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400343f0 con 0x7f97480d07c0 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 -- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97480d07c0 msgr2=0x7f974806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97480d07c0 0x7f974806ebf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9740009e50 tx=0x7f974002bed0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 -- 172.21.15.106:0/1608393498 shutdown_connections 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97480d07c0 0x7f974806ebf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 --2- 172.21.15.106:0/1608393498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f974806bff0 0x7f974806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 -- 172.21.15.106:0/1608393498 >> 172.21.15.106:0/1608393498 conn(0x7f97480cd6a0 msgr2=0x7f97480cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 -- 172.21.15.106:0/1608393498 shutdown_connections 2024-04-13T03:13:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.472+0000 7f974cdbb640 1 -- 172.21.15.106:0/1608393498 wait complete. 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.473+0000 7f974cdbb640 1 Processor -- start 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.473+0000 7f974cdbb640 1 -- start start 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974806bff0 0x7f9748156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f97480d07c0 0x7f9748152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97480ced20 con 0x7f974815fb50 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97480ceba0 con 0x7f974806bff0 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974cdbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f974815ef10 con 0x7f97480d07c0 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34514/0 (socket says 172.21.15.106:34514) 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 -- 172.21.15.106:0/3456690096 learned_addr learned my addr 172.21.15.106:0/3456690096 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f974659c640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974806bff0 0x7f9748156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9745d9b640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f97480d07c0 0x7f9748152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 -- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f97480d07c0 msgr2=0x7f9748152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f97480d07c0 0x7f9748152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 -- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974806bff0 msgr2=0x7f9748156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974806bff0 0x7f9748156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 -- 172.21.15.106:0/3456690096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97481569d0 con 0x7f974815fb50 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.474+0000 7f9746d9d640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9740036040 tx=0x7f9740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740043020 con 0x7f974815fb50 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974003e450 con 0x7f974815fb50 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9748156c70 con 0x7f974815fb50 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740009510 con 0x7f974815fb50 2024-04-13T03:13:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9748157230 con 0x7f974815fb50 2024-04-13T03:13:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.475+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974814ee10 con 0x7f974815fb50 2024-04-13T03:13:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.476+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9740046020 con 0x7f974815fb50 2024-04-13T03:13:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.476+0000 7f9736ffd640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 0x7f972806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.477+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6246+0+0 (secure 0 0 0) 0x7f97400ae540 con 0x7f974815fb50 2024-04-13T03:13:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.477+0000 7f974659c640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 0x7f972806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.477+0000 7f974659c640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 0x7f972806edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9738007fd0 tx=0x7f9738009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.481+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f974814ee10 con 0x7f974815fb50 2024-04-13T03:13:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.693+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9748068a10 con 0x7f974815fb50 2024-04-13T03:13:26.694 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:13:26.693+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:13:26.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.695+0000 7f9736ffd640 1 -- 172.21.15.106:0/3456690096 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f974004d030 con 0x7f974815fb50 2024-04-13T03:13:26.695 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 msgr2=0x7f972806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 0x7f972806edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9738007fd0 tx=0x7f9738009040 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 msgr2=0x7f9748152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9740036040 tx=0x7f9740004a40 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 shutdown_connections 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f972806c8f0 0x7f972806edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f97480d07c0 0x7f9748152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f974815fb50 0x7f9748152f90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 --2- 172.21.15.106:0/3456690096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974806bff0 0x7f9748156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 >> 172.21.15.106:0/3456690096 conn(0x7f97480cd6a0 msgr2=0x7f97480b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 shutdown_connections 2024-04-13T03:13:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.698+0000 7f974cdbb640 1 -- 172.21.15.106:0/3456690096 wait complete. 2024-04-13T03:13:26.703 INFO:tasks.ceph.mds.c.smithi106.stderr: -6216> 2024-04-13T03:12:17.660+0000 7fbb993f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:13:26.704 INFO:tasks.ceph.mds.c.smithi106.stderr: -4841> 2024-04-13T03:12:18.832+0000 7fbb953e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6420=0-4194304@1} caps={6420=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6420 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55e633d34680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:13:26.706 INFO:tasks.ceph.mds.c.smithi106.stderr: -3049> 2024-04-13T03:12:50.122+0000 7fbb953e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6420=0-4194304@1} caps={6420=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6420 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55e633d34680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:13:26.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:13:26.732 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:13:26.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.812+0000 7f244851e640 1 Processor -- start 2024-04-13T03:13:26.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.812+0000 7f244851e640 1 -- start start 2024-04-13T03:13:26.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f244851e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244015ed90 0x7f24400d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f244851e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f24400b3d60 0x7f24400d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f244851e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400cfd20 con 0x7f244015e9b0 2024-04-13T03:13:26.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f244851e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2440162af0 con 0x7f24400b3d60 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f244851e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2440162dd0 con 0x7f244015ed90 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f2446abb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f24400b3d60 0x7f24400d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f24462ba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f244015e9b0 0x7f24400d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41336/0 (socket says 172.21.15.106:41336) 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.813+0000 7f24462ba640 1 -- 172.21.15.106:0/756525209 learned_addr learned my addr 172.21.15.106:0/756525209 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2445ab9640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244015ed90 0x7f24400d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2446abb640 1 -- 172.21.15.106:0/756525209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244015ed90 msgr2=0x7f24400d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2446abb640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244015ed90 0x7f24400d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2446abb640 1 -- 172.21.15.106:0/756525209 >> v1:172.21.15.106:6789/0 conn(0x7f244015e9b0 legacy=0x7f24400d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f24452b8640 1 -- 172.21.15.106:0/756525209 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971789668 0 0) 0x7f24400cfd20 con 0x7f244015e9b0 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2446abb640 1 -- 172.21.15.106:0/756525209 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401630b0 con 0x7f24400b3d60 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.814+0000 7f2446abb640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24400b3d60 0x7f24400d0200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f243802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18102a08ba2f1bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f24452b8640 1 -- 172.21.15.106:0/756525209 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438039070 con 0x7f24400b3d60 2024-04-13T03:13:26.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f24452b8640 1 -- 172.21.15.106:0/756525209 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243802cce0 con 0x7f24400b3d60 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f24452b8640 1 -- 172.21.15.106:0/756525209 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438034660 con 0x7f24400b3d60 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 -- 172.21.15.106:0/756525209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24400b3d60 msgr2=0x7f24400d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24400b3d60 0x7f24400d0200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f243802bf30 comp rx=0 tx=0).stop 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 -- 172.21.15.106:0/756525209 shutdown_connections 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24400b3d60 0x7f24400d0200 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 --2- 172.21.15.106:0/756525209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244015ed90 0x7f24400d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 -- 172.21.15.106:0/756525209 >> 172.21.15.106:0/756525209 conn(0x7f244008bf10 msgr2=0x7f24400b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 -- 172.21.15.106:0/756525209 shutdown_connections 2024-04-13T03:13:26.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.815+0000 7f244851e640 1 -- 172.21.15.106:0/756525209 wait complete. 2024-04-13T03:13:26.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.816+0000 7f244851e640 1 Processor -- start 2024-04-13T03:13:26.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.816+0000 7f244851e640 1 -- start start 2024-04-13T03:13:26.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 0x7f244014a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 0x7f24400b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2440162800 con 0x7f24401f8ae0 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2440162680 con 0x7f244015e9b0 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f244851e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24400d4d00 con 0x7f24400b3d60 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34546/0 (socket says 172.21.15.106:34546) 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 -- 172.21.15.106:0/1014129101 learned_addr learned my addr 172.21.15.106:0/1014129101 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f24462ba640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 0x7f244014a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2445ab9640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 0x7f24400b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 -- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 msgr2=0x7f244014a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 0x7f244014a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 -- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 msgr2=0x7f24400b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:26.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 0x7f24400b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2446abb640 1 -- 172.21.15.106:0/1014129101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400c9fc0 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f2445ab9640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 0x7f24400b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.817+0000 7f24462ba640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 0x7f244014a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f2446abb640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f2438004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438044020 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243803a040 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244014d930 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438039050 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.818+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401f82e0 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.819+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2440068a10 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.819+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2438003bb0 con 0x7f24401f8ae0 2024-04-13T03:13:26.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.820+0000 7f2436ffd640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:26.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.820+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6332+0+0 (secure 0 0 0) 0x7f24380ae3a0 con 0x7f24401f8ae0 2024-04-13T03:13:26.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.820+0000 7f24462ba640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:26.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.820+0000 7f24462ba640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2430007fd0 tx=0x7f2430009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:26.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:26.824+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f243804e060 con 0x7f24401f8ae0 2024-04-13T03:13:27.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.035+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f244015d640 con 0x7f24401f8ae0 2024-04-13T03:13:27.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.696+0000 7f2436ffd640 1 -- 172.21.15.106:0/1014129101 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f243807c570 con 0x7f24401f8ae0 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 msgr2=0x7f242406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2430007fd0 tx=0x7f2430009040 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 msgr2=0x7f24400b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f2438004a40 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 shutdown_connections 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24401f8ae0 0x7f24400b4840 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244015e9b0 0x7f24400b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 --2- 172.21.15.106:0/1014129101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24400b3d60 0x7f244014a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.699+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 >> 172.21.15.106:0/1014129101 conn(0x7f244008bf10 msgr2=0x7f244015c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:27.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.700+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 shutdown_connections 2024-04-13T03:13:27.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.700+0000 7f244851e640 1 -- 172.21.15.106:0/1014129101 wait complete. 2024-04-13T03:13:27.713 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:13:27.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.862+0000 7f2a1031e640 1 Processor -- start 2024-04-13T03:13:27.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.862+0000 7f2a1031e640 1 -- start start 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a1031e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a1031e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a0815fb20 0x7f2a080704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a1031e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0815ece0 con 0x7f2a080709e0 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a1031e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2a08160040 con 0x7f2a0815fb20 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a1031e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0806da90 con 0x7f2a0815e900 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0e0ba640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a0815fb20 0x7f2a080704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0e0ba640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:54798/0 (socket says 172.21.15.106:54798) 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0e0ba640 1 -- 172.21.15.106:0/1080661157 learned_addr learned my addr 172.21.15.106:0/1080661157 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a0815fb20 0x7f2a080704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:41782/0 (socket says 172.21.15.106:41782) 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 -- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 msgr2=0x7f2a0806d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 -- 172.21.15.106:0/1080661157 >> v1:172.21.15.114:6789/0 conn(0x7f2a080709e0 legacy=0x7f2a08162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0d8b9640 1 -- 172.21.15.106:0/1080661157 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0806dd70 con 0x7f2a0815fb20 2024-04-13T03:13:27.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.863+0000 7f2a0e0ba640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.864+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815fb20 0x7f2a080704a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29f80097a0 tx=0x7f29f802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2700b49ea2dec35 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.864+0000 7f2a0d0b8640 1 -- 172.21.15.106:0/1080661157 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8038070 con 0x7f2a0815fb20 2024-04-13T03:13:27.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.864+0000 7f2a0d0b8640 1 -- 172.21.15.106:0/1080661157 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8031700 con 0x7f2a0815fb20 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.864+0000 7f2a0d0b8640 1 -- 172.21.15.106:0/1080661157 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8031ad0 con 0x7f2a0815fb20 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 -- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815fb20 msgr2=0x7f2a080704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815fb20 0x7f2a080704a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29f80097a0 tx=0x7f29f802b7a0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 -- 172.21.15.106:0/1080661157 shutdown_connections 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815fb20 0x7f2a080704a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 --2- 172.21.15.106:0/1080661157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815e900 0x7f2a0806d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 -- 172.21.15.106:0/1080661157 >> 172.21.15.106:0/1080661157 conn(0x7f2a080717e0 msgr2=0x7f2a08071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 -- 172.21.15.106:0/1080661157 shutdown_connections 2024-04-13T03:13:27.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.865+0000 7f2a1031e640 1 -- 172.21.15.106:0/1080661157 wait complete. 2024-04-13T03:13:27.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.866+0000 7f2a1031e640 1 Processor -- start 2024-04-13T03:13:27.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.866+0000 7f2a1031e640 1 -- start start 2024-04-13T03:13:27.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 0x7f2a080cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815fb20 0x7f2a08152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0806d6d0 con 0x7f2a080709e0 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0806d550 con 0x7f2a0815e900 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a1031e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2a081617c0 con 0x7f2a0815fb20 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34562/0 (socket says 172.21.15.106:34562) 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 -- 172.21.15.106:0/2496641222 learned_addr learned my addr 172.21.15.106:0/2496641222 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 0x7f2a080cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e8bb640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815fb20 0x7f2a08152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 -- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815fb20 msgr2=0x7f2a08152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815fb20 0x7f2a08152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 -- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 msgr2=0x7f2a080cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 0x7f2a080cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:27.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0e0ba640 1 -- 172.21.15.106:0/2496641222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a08150910 con 0x7f2a080709e0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.867+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 0x7f2a080cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f2a0e0ba640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9d0 tx=0x7f2a0400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04017070 con 0x7f2a080709e0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0400ca60 con 0x7f2a080709e0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a080ce440 con 0x7f2a080709e0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400ce10 con 0x7f2a080709e0 2024-04-13T03:13:27.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.868+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a080cd680 con 0x7f2a080709e0 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.869+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a08068a10 con 0x7f2a080709e0 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.869+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a0401a070 con 0x7f2a080709e0 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.869+0000 7f29feffd640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.870+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6332+0+0 (secure 0 0 0) 0x7f2a0408bca0 con 0x7f2a080709e0 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.870+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:27.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.870+0000 7f2a0d8b9640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 0x7f29ec06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f29f8035000 tx=0x7f29f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:27.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:27.874+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a04059810 con 0x7f2a080709e0 2024-04-13T03:13:28.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.048+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a080712e0 con 0x7f2a080709e0 2024-04-13T03:13:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.697+0000 7f29feffd640 1 -- 172.21.15.106:0/2496641222 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) ==== 181+0+0 (secure 0 0 0) 0x7f2a0405db10 con 0x7f2a080709e0 2024-04-13T03:13:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 msgr2=0x7f29ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 0x7f29ec06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f29f8035000 tx=0x7f29f8008030 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 msgr2=0x7f2a081503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9d0 tx=0x7f2a0400ee90 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 shutdown_connections 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29ec06c8f0 0x7f29ec06edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a0815fb20 0x7f2a08152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0815e900 0x7f2a080cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 --2- 172.21.15.106:0/2496641222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a080709e0 0x7f2a081503d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 >> 172.21.15.106:0/2496641222 conn(0x7f2a080717e0 msgr2=0x7f2a080b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 shutdown_connections 2024-04-13T03:13:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.700+0000 7f2a1031e640 1 -- 172.21.15.106:0/2496641222 wait complete. 2024-04-13T03:13:28.715 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:13:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.820+0000 7fb68a18c640 1 Processor -- start 2024-04-13T03:13:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 -- start start 2024-04-13T03:13:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb68415fb50 0x7fb68406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 0x7fb68406be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 0x7fb684162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb6840cef80 con 0x7fb68415fb50 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb6840cf260 con 0x7fb68415ef10 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb68a18c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb6840cf540 con 0x7fb6840d07c0 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.821+0000 7fb6837fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb68415fb50 0x7fb68406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb68415fb50 0x7fb68406b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34572/0 (socket says 172.21.15.106:34572) 2024-04-13T03:13:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 -- 172.21.15.106:0/1816471217 learned_addr learned my addr 172.21.15.106:0/1816471217 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb683fff640 1 --2- 172.21.15.106:0/1816471217 >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 0x7fb684162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb682ffd640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 0x7fb68406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 -- 172.21.15.106:0/1816471217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 msgr2=0x7fb68406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 0x7fb68406be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 -- 172.21.15.106:0/1816471217 >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 msgr2=0x7fb684162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 --2- 172.21.15.106:0/1816471217 >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 0x7fb684162490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 -- 172.21.15.106:0/1816471217 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6840cf820 con 0x7fb68415fb50 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb683fff640 1 --2- 172.21.15.106:0/1816471217 >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 0x7fb684162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb682ffd640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 0x7fb68406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.822+0000 7fb6837fe640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb68415fb50 0x7fb68406b8e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb684163530 tx=0x7fb67402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48746c7b052b9d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb6827fc640 1 -- 172.21.15.106:0/1816471217 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402e070 con 0x7fb68415fb50 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb6827fc640 1 -- 172.21.15.106:0/1816471217 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674037070 con 0x7fb68415fb50 2024-04-13T03:13:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb6827fc640 1 -- 172.21.15.106:0/1816471217 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674031a00 con 0x7fb68415fb50 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 -- 172.21.15.106:0/1816471217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb68415fb50 msgr2=0x7fb68406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb68415fb50 0x7fb68406b8e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb684163530 tx=0x7fb67402b880 comp rx=0 tx=0).stop 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 -- 172.21.15.106:0/1816471217 shutdown_connections 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 --2- 172.21.15.106:0/1816471217 >> v2:172.21.15.114:3300/0 conn(0x7fb68415ef10 0x7fb684162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6840d07c0 0x7fb68406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 --2- 172.21.15.106:0/1816471217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb68415fb50 0x7fb68406b8e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 -- 172.21.15.106:0/1816471217 >> 172.21.15.106:0/1816471217 conn(0x7fb6840cd6a0 msgr2=0x7fb6840cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 -- 172.21.15.106:0/1816471217 shutdown_connections 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.823+0000 7fb68a18c640 1 -- 172.21.15.106:0/1816471217 wait complete. 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.824+0000 7fb68a18c640 1 Processor -- start 2024-04-13T03:13:28.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 -- start start 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68415ef10 0x7fb6841f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb68415fb50 0x7fb6840c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840ced20 con 0x7fb6840d07c0 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840ceba0 con 0x7fb68415ef10 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb68a18c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb6841629d0 con 0x7fb68415fb50 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb6837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb6837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34586/0 (socket says 172.21.15.106:34586) 2024-04-13T03:13:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb6837fe640 1 -- 172.21.15.106:0/4107004832 learned_addr learned my addr 172.21.15.106:0/4107004832 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb682ffd640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68415ef10 0x7fb6841f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.825+0000 7fb683fff640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb68415fb50 0x7fb6840c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 -- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb68415fb50 msgr2=0x7fb6840c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb68415fb50 0x7fb6840c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 -- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68415ef10 msgr2=0x7fb6841f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68415ef10 0x7fb6841f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 -- 172.21.15.106:0/4107004832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6840c6990 con 0x7fb6840d07c0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb6837fe640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb67402b8c0 tx=0x7fb6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674032880 con 0x7fb6840d07c0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674032a20 con 0x7fb6840d07c0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6840c6c70 con 0x7fb6840d07c0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674032e10 con 0x7fb6840d07c0 2024-04-13T03:13:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.826+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6840c7230 con 0x7fb6840d07c0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.827+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb684068a10 con 0x7fb6840d07c0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.827+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6740453f0 con 0x7fb6840d07c0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.828+0000 7fb66bfff640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.828+0000 7fb682ffd640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.828+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5865+0+0 (secure 0 0 0) 0x7fb6740adb90 con 0x7fb6840d07c0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.828+0000 7fb682ffd640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb684073460 tx=0x7fb678008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:28.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:28.832+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb674040070 con 0x7fb6840d07c0 2024-04-13T03:13:29.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.006+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6840b6a90 con 0x7fb6840d07c0 2024-04-13T03:13:29.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.702+0000 7fb66bfff640 1 -- 172.21.15.106:0/4107004832 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) ==== 169+0+0 (secure 0 0 0) 0x7fb67407bfb0 con 0x7fb6840d07c0 2024-04-13T03:13:29.704 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:13:29.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.705+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 msgr2=0x7fb67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.705+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb684073460 tx=0x7fb678008040 comp rx=0 tx=0).stop 2024-04-13T03:13:29.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.705+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 msgr2=0x7fb6840b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.705+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb67402b8c0 tx=0x7fb6740046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 shutdown_connections 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb68415fb50 0x7fb6840c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68415ef10 0x7fb6841f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 --2- 172.21.15.106:0/4107004832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6840d07c0 0x7fb6840b63a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 >> 172.21.15.106:0/4107004832 conn(0x7fb6840cd6a0 msgr2=0x7fb6840cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 shutdown_connections 2024-04-13T03:13:29.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.707+0000 7fb68a18c640 1 -- 172.21.15.106:0/4107004832 wait complete. 2024-04-13T03:13:29.725 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:13:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.828+0000 7f2120ed5640 1 Processor -- start 2024-04-13T03:13:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.828+0000 7f2120ed5640 1 -- start start 2024-04-13T03:13:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2120ed5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f211c15e8c0 0x7f211c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2120ed5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 0x7f211c072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2120ed5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211c0b5060 con 0x7f211c073350 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2120ed5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f211c073780 con 0x7f211c15e8c0 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2120ed5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f211c0b5340 con 0x7f211c15eca0 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f211c15e8c0 0x7f211c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f211c073350 0x7f211c0b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41342/0 (socket says 172.21.15.106:41342) 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211ad9d640 1 -- 172.21.15.106:0/3120919405 learned_addr learned my addr 172.21.15.106:0/3120919405 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2119d9b640 1 --2- 172.21.15.106:0/3120919405 >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 0x7f211c072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211a59c640 1 -- 172.21.15.106:0/3120919405 >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 msgr2=0x7f211c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211a59c640 1 --2- 172.21.15.106:0/3120919405 >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 0x7f211c072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211a59c640 1 -- 172.21.15.106:0/3120919405 >> v1:172.21.15.106:6789/0 conn(0x7f211c073350 legacy=0x7f211c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211a59c640 1 -- 172.21.15.106:0/3120919405 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c06c500 con 0x7f211c15e8c0 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f211959a640 1 -- 172.21.15.106:0/3120919405 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1333377160 0 0) 0x7f211c0b5060 con 0x7f211c073350 2024-04-13T03:13:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.829+0000 7f2119d9b640 1 --2- 172.21.15.106:0/3120919405 >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 0x7f211c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.830+0000 7f211a59c640 1 --2- 172.21.15.106:0/3120919405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c0ce250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f211c072ae0 tx=0x7f210c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0296e7f7f88e76d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.830+0000 7f211959a640 1 -- 172.21.15.106:0/3120919405 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c039070 con 0x7f211c15e8c0 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.830+0000 7f211959a640 1 -- 172.21.15.106:0/3120919405 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c02cd00 con 0x7f211c15e8c0 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.830+0000 7f211959a640 1 -- 172.21.15.106:0/3120919405 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c034660 con 0x7f211c15e8c0 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 -- 172.21.15.106:0/3120919405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 msgr2=0x7f211c0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 --2- 172.21.15.106:0/3120919405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c0ce250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f211c072ae0 tx=0x7f210c02bf50 comp rx=0 tx=0).stop 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 -- 172.21.15.106:0/3120919405 shutdown_connections 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 --2- 172.21.15.106:0/3120919405 >> v2:172.21.15.114:3300/0 conn(0x7f211c15eca0 0x7f211c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 --2- 172.21.15.106:0/3120919405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c0ce250 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 -- 172.21.15.106:0/3120919405 >> 172.21.15.106:0/3120919405 conn(0x7f211c06f0d0 msgr2=0x7f211c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 -- 172.21.15.106:0/3120919405 shutdown_connections 2024-04-13T03:13:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.831+0000 7f2120ed5640 1 -- 172.21.15.106:0/3120919405 wait complete. 2024-04-13T03:13:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.832+0000 7f2120ed5640 1 Processor -- start 2024-04-13T03:13:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.832+0000 7f2120ed5640 1 -- start start 2024-04-13T03:13:29.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c073350 0x7f211c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c15eca0 0x7f211c152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f211c0b4cc0 con 0x7f211c15e8c0 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f211c0b4b40 con 0x7f211c15eca0 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2120ed5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f211c0b4e40 con 0x7f211c073350 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34614/0 (socket says 172.21.15.106:34614) 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 -- 172.21.15.106:0/2776568430 learned_addr learned my addr 172.21.15.106:0/2776568430 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f211a59c640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c073350 0x7f211c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f211ad9d640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c15eca0 0x7f211c152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 -- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c073350 msgr2=0x7f211c1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c073350 0x7f211c1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 -- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c15eca0 msgr2=0x7f211c152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c15eca0 0x7f211c152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.833+0000 7f2119d9b640 1 -- 172.21.15.106:0/2776568430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0cd260 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f2119d9b640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f211000c7f0 tx=0x7f211000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110015070 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2110007820 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c0ccf80 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110007bd0 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.834+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c0cbef0 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.835+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c068a10 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.835+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2110018070 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.835+0000 7f210affd640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.836+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f211008baf0 con 0x7f211c15e8c0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.836+0000 7f211a59c640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.836+0000 7f211a59c640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f210c036000 tx=0x7f210c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:29.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:29.840+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f211005a090 con 0x7f211c15e8c0 2024-04-13T03:13:30.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.023+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f211c0cdbf0 con 0x7f211c15e8c0 2024-04-13T03:13:30.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.026+0000 7f210affd640 1 -- 172.21.15.106:0/2776568430 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f2110059990 con 0x7f211c15e8c0 2024-04-13T03:13:30.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 msgr2=0x7f20fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f210c036000 tx=0x7f210c030000 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 msgr2=0x7f211c153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f211000c7f0 tx=0x7f211000ccb0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 shutdown_connections 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c15eca0 0x7f211c152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15e8c0 0x7f211c153890 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 --2- 172.21.15.106:0/2776568430 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c073350 0x7f211c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.028+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 >> 172.21.15.106:0/2776568430 conn(0x7f211c06f0d0 msgr2=0x7f211c162a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.029+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 shutdown_connections 2024-04-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.029+0000 7f2120ed5640 1 -- 172.21.15.106:0/2776568430 wait complete. 2024-04-13T03:13:30.040 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-13T03:13:30.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.140+0000 7ff897295640 1 Processor -- start 2024-04-13T03:13:30.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.140+0000 7ff897295640 1 -- start start 2024-04-13T03:13:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff897295640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff890160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff897295640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 0x7ff890160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff897295640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8900d05b0 con 0x7ff890160ff0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff897295640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff8900ceb50 con 0x7ff8900b0390 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff897295640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff8900cf110 con 0x7ff890070090 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895832640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff890160ff0 0x7ff89006ccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41354/0 (socket says 172.21.15.106:41354) 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895832640 1 -- 172.21.15.106:0/1556260338 learned_addr learned my addr 172.21.15.106:0/1556260338 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff894830640 1 --2- 172.21.15.106:0/1556260338 >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 0x7ff890160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895031640 1 --2- 172.21.15.106:0/1556260338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff890160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609676256 0 0) 0x7ff8900d05b0 con 0x7ff890160ff0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff87c003520 con 0x7ff890160ff0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2166640616 0 0) 0x7ff87c003520 con 0x7ff890160ff0 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895031640 1 -- 172.21.15.106:0/1556260338 >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 msgr2=0x7ff890160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895031640 1 --2- 172.21.15.106:0/1556260338 >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 0x7ff890160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895031640 1 -- 172.21.15.106:0/1556260338 >> v1:172.21.15.106:6789/0 conn(0x7ff890160ff0 legacy=0x7ff89006ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff895031640 1 -- 172.21.15.106:0/1556260338 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff890070470 con 0x7ff890070090 2024-04-13T03:13:30.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.141+0000 7ff894830640 1 --2- 172.21.15.106:0/1556260338 >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 0x7ff890160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.142+0000 7ff895031640 1 --2- 172.21.15.106:0/1556260338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff890160570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0097a0 tx=0x7ff88c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516331dcc931daf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.142+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c037070 con 0x7ff890070090 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.142+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c030510 con 0x7ff890070090 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.142+0000 7ff887fff640 1 -- 172.21.15.106:0/1556260338 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880019f0 con 0x7ff890070090 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 -- 172.21.15.106:0/1556260338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 msgr2=0x7ff890160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 --2- 172.21.15.106:0/1556260338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff890160570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0097a0 tx=0x7ff88c02b710 comp rx=0 tx=0).stop 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 -- 172.21.15.106:0/1556260338 shutdown_connections 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 --2- 172.21.15.106:0/1556260338 >> v2:172.21.15.114:3300/0 conn(0x7ff8900b0390 0x7ff890160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 --2- 172.21.15.106:0/1556260338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff890160570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 -- 172.21.15.106:0/1556260338 >> 172.21.15.106:0/1556260338 conn(0x7ff8900cd6d0 msgr2=0x7ff8900cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:30.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 -- 172.21.15.106:0/1556260338 shutdown_connections 2024-04-13T03:13:30.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.143+0000 7ff897295640 1 -- 172.21.15.106:0/1556260338 wait complete. 2024-04-13T03:13:30.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.144+0000 7ff897295640 1 Processor -- start 2024-04-13T03:13:30.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.144+0000 7ff897295640 1 -- start start 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff89014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 0x7ff8901479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8900ceda0 con 0x7ff8900b0390 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff89015dcc0 con 0x7ff890160ff0 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff897295640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff89006e560 con 0x7ff890070090 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34624/0 (socket says 172.21.15.106:34624) 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 -- 172.21.15.106:0/1674525031 learned_addr learned my addr 172.21.15.106:0/1674525031 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:30.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff895031640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff89014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff895832640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 0x7ff8901479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 -- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 msgr2=0x7ff89014d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff89014d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 -- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 msgr2=0x7ff8901479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 0x7ff8901479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 -- 172.21.15.106:0/1674525031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff890148a30 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff895832640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 0x7ff8901479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff894830640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ebe0 tx=0x7ff88000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.145+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880017070 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.146+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88000cc30 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.146+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89014a360 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.146+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880013470 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.146+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89014adf0 con 0x7ff8900b0390 2024-04-13T03:13:30.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.147+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff890068a10 con 0x7ff8900b0390 2024-04-13T03:13:30.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.147+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff88001a070 con 0x7ff8900b0390 2024-04-13T03:13:30.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.147+0000 7ff885ffb640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.148+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7ff88008bc40 con 0x7ff8900b0390 2024-04-13T03:13:30.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.148+0000 7ff895031640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.148+0000 7ff895031640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff88c034000 tx=0x7ff88c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.152+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff880059ba0 con 0x7ff8900b0390 2024-04-13T03:13:30.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.394+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7ff8900d0840 con 0x7ff8900b0390 2024-04-13T03:13:30.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.704+0000 7ff885ffb640 1 -- 172.21.15.106:0/1674525031 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v219) ==== 169+0+0 (secure 0 0 0) 0x7ff88005dea0 con 0x7ff8900b0390 2024-04-13T03:13:30.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 msgr2=0x7ff87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff88c034000 tx=0x7ff88c008030 comp rx=0 tx=0).stop 2024-04-13T03:13:30.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 msgr2=0x7ff890149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ebe0 tx=0x7ff88000c520 comp rx=0 tx=0).stop 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 shutdown_connections 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff890160ff0 0x7ff8901479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8900b0390 0x7ff890149cd0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 --2- 172.21.15.106:0/1674525031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff890070090 0x7ff89014d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 >> 172.21.15.106:0/1674525031 conn(0x7ff8900cd6d0 msgr2=0x7ff8900b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 shutdown_connections 2024-04-13T03:13:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.706+0000 7ff897295640 1 -- 172.21.15.106:0/1674525031 wait complete. 2024-04-13T03:13:30.718 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-13T03:13:30.719 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-13T03:13:30.719 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:13:30.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:30.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.819+0000 7f827bf1e640 1 Processor -- start 2024-04-13T03:13:30.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.819+0000 7f827bf1e640 1 -- start start 2024-04-13T03:13:30.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f827bf1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f827406e8e0 0x7f827415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f827bf1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8274070c80 0x7f82740710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f827bf1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f827415f420 con 0x7f82740cd770 2024-04-13T03:13:30.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f827bf1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8274071630 con 0x7f827406e8e0 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f827bf1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f827406ecc0 con 0x7f8274070c80 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f8279cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f827406e8e0 0x7f827415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f8279cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f827406e8e0 0x7f827415ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34628/0 (socket says 172.21.15.106:34628) 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.820+0000 7f8279cba640 1 -- 172.21.15.106:0/985503750 learned_addr learned my addr 172.21.15.106:0/985503750 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f827a4bb640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8274070c80 0x7f82740710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f8279cba640 1 -- 172.21.15.106:0/985503750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8274070c80 msgr2=0x7f82740710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f8279cba640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8274070c80 0x7f82740710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f8279cba640 1 -- 172.21.15.106:0/985503750 >> v1:172.21.15.114:6789/0 conn(0x7f82740cd770 legacy=0x7f82740cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f8279cba640 1 -- 172.21.15.106:0/985503750 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827415f700 con 0x7f827406e8e0 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.821+0000 7f8279cba640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f827406e8e0 0x7f827415ed80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365cae4d23fc8451 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f8278cb8640 1 -- 172.21.15.106:0/985503750 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270034070 con 0x7f827406e8e0 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f8278cb8640 1 -- 172.21.15.106:0/985503750 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270030840 con 0x7f827406e8e0 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f8278cb8640 1 -- 172.21.15.106:0/985503750 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270030c10 con 0x7f827406e8e0 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 -- 172.21.15.106:0/985503750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f827406e8e0 msgr2=0x7f827415ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f827406e8e0 0x7f827415ed80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002b980 comp rx=0 tx=0).stop 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 -- 172.21.15.106:0/985503750 shutdown_connections 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8274070c80 0x7f82740710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 --2- 172.21.15.106:0/985503750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f827406e8e0 0x7f827415ed80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 -- 172.21.15.106:0/985503750 >> 172.21.15.106:0/985503750 conn(0x7f827406f180 msgr2=0x7f827406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 -- 172.21.15.106:0/985503750 shutdown_connections 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.822+0000 7f827bf1e640 1 -- 172.21.15.106:0/985503750 wait complete. 2024-04-13T03:13:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.823+0000 7f827bf1e640 1 Processor -- start 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.823+0000 7f827bf1e640 1 -- start start 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 0x7f8274157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82740cd770 0x7f8274159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f827415fcc0 con 0x7f8274070c80 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82740707f0 con 0x7f82740cd770 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827bf1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f82740cdbd0 con 0x7f827406e8e0 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f82794b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f82794b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34636/0 (socket says 172.21.15.106:34636) 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f82794b9640 1 -- 172.21.15.106:0/3694303863 learned_addr learned my addr 172.21.15.106:0/3694303863 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f8279cba640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 0x7f8274157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f827a4bb640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82740cd770 0x7f8274159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f82794b9640 1 -- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 msgr2=0x7f8274157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.824+0000 7f82794b9640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 0x7f8274157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f82794b9640 1 -- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82740cd770 msgr2=0x7f8274159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f82794b9640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82740cd770 0x7f8274159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f82794b9640 1 -- 172.21.15.106:0/3694303863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8274158920 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f8279cba640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 0x7f8274157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f82794b9640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f826400c400 tx=0x7f826400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264017070 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82640052d0 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8274158c00 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264003830 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.825+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741591c0 con 0x7f8274070c80 2024-04-13T03:13:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.826+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274068a10 con 0x7f8274070c80 2024-04-13T03:13:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.826+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826401a070 con 0x7f8274070c80 2024-04-13T03:13:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.827+0000 7f826a7fc640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.827+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f826408ad40 con 0x7f8274070c80 2024-04-13T03:13:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.827+0000 7f8279cba640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.827+0000 7f8279cba640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 0x7f825806edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8270036000 tx=0x7f827003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:30.831+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82640592e0 con 0x7f8274070c80 2024-04-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.008+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82740cb780 con 0x7f8274070c80 2024-04-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.009+0000 7f826a7fc640 1 -- 172.21.15.106:0/3694303863 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+13330 (secure 0 0 0) 0x7f8264058be0 con 0x7f8274070c80 2024-04-13T03:13:31.010 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:31.010 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":69,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:29.698947+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6836/536113082":"2024-04-14T03:13:26.694249+0000","172.21.15.106:6834/536113082":"2024-04-14T03:13:26.694249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 msgr2=0x7f825806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 0x7f825806edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8270036000 tx=0x7f827003b040 comp rx=0 tx=0).stop 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 msgr2=0x7f82740b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f826400c400 tx=0x7f826400c8c0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 shutdown_connections 2024-04-13T03:13:31.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82740cd770 0x7f8274159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8274070c80 0x7f82740b6b00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 --2- 172.21.15.106:0/3694303863 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f827406e8e0 0x7f8274157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 >> 172.21.15.106:0/3694303863 conn(0x7f827406f180 msgr2=0x7f827406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 shutdown_connections 2024-04-13T03:13:31.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.011+0000 7f827bf1e640 1 -- 172.21.15.106:0/3694303863 wait complete. 2024-04-13T03:13:31.023 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:31.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:31.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.125+0000 7f861071e640 1 Processor -- start 2024-04-13T03:13:31.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.125+0000 7f861071e640 1 -- start start 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f861071e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f860815e9b0 0x7f86080d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f861071e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f860815ed90 0x7f86080d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f861071e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86080cfd20 con 0x7f86080b3d60 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f861071e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8608162af0 con 0x7f860815ed90 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f861071e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8608162dd0 con 0x7f860815e9b0 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.126+0000 7f860dcb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f860815ed90 0x7f86080d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f860815ed90 0x7f86080d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34644/0 (socket says 172.21.15.106:34644) 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 -- 172.21.15.106:0/1578180949 learned_addr learned my addr 172.21.15.106:0/1578180949 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860e4ba640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f860815e9b0 0x7f86080d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 -- 172.21.15.106:0/1578180949 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f860815e9b0 msgr2=0x7f86080d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:31.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f860815e9b0 0x7f86080d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 -- 172.21.15.106:0/1578180949 >> v1:172.21.15.114:6789/0 conn(0x7f86080b3d60 legacy=0x7f86080d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.127+0000 7f860dcb9640 1 -- 172.21.15.106:0/1578180949 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081630b0 con 0x7f860815ed90 2024-04-13T03:13:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f860dcb9640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815ed90 0x7f86080d45f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009850 tx=0x7f85f802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996b3384345c1ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f860d4b8640 1 -- 172.21.15.106:0/1578180949 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8004230 con 0x7f860815ed90 2024-04-13T03:13:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f860d4b8640 1 -- 172.21.15.106:0/1578180949 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f80043d0 con 0x7f860815ed90 2024-04-13T03:13:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f860d4b8640 1 -- 172.21.15.106:0/1578180949 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8003810 con 0x7f860815ed90 2024-04-13T03:13:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 -- 172.21.15.106:0/1578180949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815ed90 msgr2=0x7f86080d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815ed90 0x7f86080d45f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009850 tx=0x7f85f802b9f0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 -- 172.21.15.106:0/1578180949 shutdown_connections 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815ed90 0x7f86080d45f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 --2- 172.21.15.106:0/1578180949 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f860815e9b0 0x7f86080d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 -- 172.21.15.106:0/1578180949 >> 172.21.15.106:0/1578180949 conn(0x7f860808bf10 msgr2=0x7f86080b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 -- 172.21.15.106:0/1578180949 shutdown_connections 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.128+0000 7f861071e640 1 -- 172.21.15.106:0/1578180949 wait complete. 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.129+0000 7f861071e640 1 Processor -- start 2024-04-13T03:13:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 -- start start 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86080b3d60 0x7f8608152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86080b5ab0 0x7f86080b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8608162800 con 0x7f860815e9b0 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8608162680 con 0x7f86080b5ab0 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f861071e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f860806eb20 con 0x7f86080b3d60 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f860dcb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f860dcb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34646/0 (socket says 172.21.15.106:34646) 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.130+0000 7f860dcb9640 1 -- 172.21.15.106:0/1922330906 learned_addr learned my addr 172.21.15.106:0/1922330906 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 -- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86080b3d60 msgr2=0x7f8608152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86080b3d60 0x7f8608152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 -- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86080b5ab0 msgr2=0x7f86080b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86080b5ab0 0x7f86080b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 -- 172.21.15.106:0/1922330906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86080b5fc0 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f860dcb9640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8047070 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f8005e10 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8003a40 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86080b62a0 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.131+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86080b4340 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.132+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608068a10 con 0x7f860815e9b0 2024-04-13T03:13:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.132+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85f804a070 con 0x7f860815e9b0 2024-04-13T03:13:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.133+0000 7f85feffd640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 0x7f85ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.133+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f85f80acff0 con 0x7f860815e9b0 2024-04-13T03:13:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.133+0000 7f860e4ba640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 0x7f85ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.133+0000 7f860e4ba640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 0x7f85ec06ede0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8604006a90 tx=0x7f8604006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.137+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85f8033070 con 0x7f860815e9b0 2024-04-13T03:13:31.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.311+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86080d53a0 con 0x7f860815e9b0 2024-04-13T03:13:31.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.311+0000 7f85feffd640 1 -- 172.21.15.106:0/1922330906 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f85f807b620 con 0x7f860815e9b0 2024-04-13T03:13:31.313 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 msgr2=0x7f85ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 0x7f85ec06ede0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8604006a90 tx=0x7f8604006a20 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 msgr2=0x7f86080b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f80046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 shutdown_connections 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85ec06c920 0x7f85ec06ede0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86080b5ab0 0x7f86080b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f860815e9b0 0x7f86080b5570 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 --2- 172.21.15.106:0/1922330906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f86080b3d60 0x7f8608152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 >> 172.21.15.106:0/1922330906 conn(0x7f860808bf10 msgr2=0x7f860815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 shutdown_connections 2024-04-13T03:13:31.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.313+0000 7f861071e640 1 -- 172.21.15.106:0/1922330906 wait complete. 2024-04-13T03:13:31.326 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-13T03:13:31.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.425+0000 7fd6309fa640 1 Processor -- start 2024-04-13T03:13:31.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.425+0000 7fd6309fa640 1 -- start start 2024-04-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd6309fa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd62c15e9c0 0x7fd62c0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd6309fa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd62c1624b0 0x7fd62c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd6309fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62c15eda0 con 0x7fd62c0b6700 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd6309fa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd62c0d27f0 con 0x7fd62c15e9c0 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd6309fa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd62c162c20 con 0x7fd62c1624b0 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd62c15e9c0 0x7fd62c0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd62c15e9c0 0x7fd62c0ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34652/0 (socket says 172.21.15.106:34652) 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd62c0b6700 0x7fd62c06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41364/0 (socket says 172.21.15.106:41364) 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 -- 172.21.15.106:0/3269216925 learned_addr learned my addr 172.21.15.106:0/3269216925 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd629d9b640 1 --2- 172.21.15.106:0/3269216925 >> v2:172.21.15.114:3300/0 conn(0x7fd62c1624b0 0x7fd62c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 -- 172.21.15.106:0/3269216925 >> v2:172.21.15.114:3300/0 conn(0x7fd62c1624b0 msgr2=0x7fd62c0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 --2- 172.21.15.106:0/3269216925 >> v2:172.21.15.114:3300/0 conn(0x7fd62c1624b0 0x7fd62c0b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.426+0000 7fd62a59c640 1 -- 172.21.15.106:0/3269216925 >> v1:172.21.15.106:6789/0 conn(0x7fd62c0b6700 legacy=0x7fd62c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62a59c640 1 -- 172.21.15.106:0/3269216925 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c162f00 con 0x7fd62c15e9c0 2024-04-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62959a640 1 -- 172.21.15.106:0/3269216925 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721940126 0 0) 0x7fd62c15eda0 con 0x7fd62c0b6700 2024-04-13T03:13:31.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62a59c640 1 --2- 172.21.15.106:0/3269216925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c0ba290 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0b7330 tx=0x7fd61c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806729f7e84e174b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62959a640 1 -- 172.21.15.106:0/3269216925 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c02cb20 con 0x7fd62c15e9c0 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62959a640 1 -- 172.21.15.106:0/3269216925 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c02ccc0 con 0x7fd62c15e9c0 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.427+0000 7fd62959a640 1 -- 172.21.15.106:0/3269216925 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c034610 con 0x7fd62c15e9c0 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 -- 172.21.15.106:0/3269216925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 msgr2=0x7fd62c0ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3269216925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c0ba290 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0b7330 tx=0x7fd61c02bf80 comp rx=0 tx=0).stop 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 -- 172.21.15.106:0/3269216925 shutdown_connections 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3269216925 >> v2:172.21.15.114:3300/0 conn(0x7fd62c1624b0 0x7fd62c0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3269216925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c0ba290 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 -- 172.21.15.106:0/3269216925 >> 172.21.15.106:0/3269216925 conn(0x7fd62c0d11c0 msgr2=0x7fd62c0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 -- 172.21.15.106:0/3269216925 shutdown_connections 2024-04-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.428+0000 7fd6309fa640 1 -- 172.21.15.106:0/3269216925 wait complete. 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.429+0000 7fd6309fa640 1 Processor -- start 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.429+0000 7fd6309fa640 1 -- start start 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd62c0b6700 0x7fd62c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 0x7fd62c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c162910 con 0x7fd62c15e9c0 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0d2a40 con 0x7fd62c1624b0 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd6309fa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0d3f70 con 0x7fd62c0b6700 2024-04-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34654/0 (socket says 172.21.15.106:34654) 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 -- 172.21.15.106:0/3593308825 learned_addr learned my addr 172.21.15.106:0/3593308825 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd62a59c640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd62c0b6700 0x7fd62c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd62ad9d640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 0x7fd62c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 -- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd62c0b6700 msgr2=0x7fd62c1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd62c0b6700 0x7fd62c1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 -- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 msgr2=0x7fd62c0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 0x7fd62c0cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd629d9b640 1 -- 172.21.15.106:0/3593308825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c15f850 con 0x7fd62c15e9c0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.430+0000 7fd62ad9d640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 0x7fd62c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd629d9b640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd62000d620 tx=0x7fd62000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62000b620 con 0x7fd62c15e9c0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620019070 con 0x7fd62c15e9c0 2024-04-13T03:13:31.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c15fb30 con 0x7fd62c15e9c0 2024-04-13T03:13:31.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62000ba00 con 0x7fd62c15e9c0 2024-04-13T03:13:31.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.431+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c1600f0 con 0x7fd62c15e9c0 2024-04-13T03:13:31.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.432+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62c0d0270 con 0x7fd62c15e9c0 2024-04-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.432+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd62000bba0 con 0x7fd62c15e9c0 2024-04-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.433+0000 7fd61affd640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 0x7fd60c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.433+0000 7fd62a59c640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 0x7fd60c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.433+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7fd62008bd50 con 0x7fd62c15e9c0 2024-04-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.433+0000 7fd62a59c640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 0x7fd60c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd61c035000 tx=0x7fd61c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.437+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd62c0d0270 con 0x7fd62c15e9c0 2024-04-13T03:13:31.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.683+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fd62c15de50 con 0x7fd62c15e9c0 2024-04-13T03:13:31.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.704+0000 7fd61affd640 1 -- 172.21.15.106:0/3593308825 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v220) ==== 169+0+0 (secure 0 0 0) 0x7fd62008f030 con 0x7fd62c15e9c0 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 msgr2=0x7fd60c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 0x7fd60c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd61c035000 tx=0x7fd61c030000 comp rx=0 tx=0).stop 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 msgr2=0x7fd62c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd62000d620 tx=0x7fd62000dae0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 shutdown_connections 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd60c06c8f0 0x7fd60c06edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd62c1624b0 0x7fd62c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd62c15e9c0 0x7fd62c15ef50 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 --2- 172.21.15.106:0/3593308825 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd62c0b6700 0x7fd62c1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 >> 172.21.15.106:0/3593308825 conn(0x7fd62c0d11c0 msgr2=0x7fd62c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 shutdown_connections 2024-04-13T03:13:31.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.706+0000 7fd6309fa640 1 -- 172.21.15.106:0/3593308825 wait complete. 2024-04-13T03:13:31.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.819+0000 7f2aaff1e640 1 Processor -- start 2024-04-13T03:13:31.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 -- start start 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2aa815b4e0 0x7f2aa815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2aa80b5390 0x7f2aa806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa80b3690 con 0x7f2aa80b4730 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa80b3970 con 0x7f2aa815b4e0 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.820+0000 7f2aaff1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa80b3c50 con 0x7f2aa80b5390 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2aa815b4e0 0x7f2aa815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2aa815b4e0 0x7f2aa815fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34662/0 (socket says 172.21.15.106:34662) 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 -- 172.21.15.106:0/472698951 learned_addr learned my addr 172.21.15.106:0/472698951 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aad4b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2aa80b4730 0x7f2aa80b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41378/0 (socket says 172.21.15.106:41378) 2024-04-13T03:13:31.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 -- 172.21.15.106:0/472698951 >> v2:172.21.15.114:3300/0 conn(0x7f2aa80b5390 msgr2=0x7f2aa806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 --2- 172.21.15.106:0/472698951 >> v2:172.21.15.114:3300/0 conn(0x7f2aa80b5390 0x7f2aa806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 -- 172.21.15.106:0/472698951 >> v1:172.21.15.106:6789/0 conn(0x7f2aa80b4730 legacy=0x7f2aa80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aaccb8640 1 -- 172.21.15.106:0/472698951 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529925960 0 0) 0x7f2aa80b3690 con 0x7f2aa80b4730 2024-04-13T03:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.821+0000 7f2aadcba640 1 -- 172.21.15.106:0/472698951 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa80b3f30 con 0x7f2aa815b4e0 2024-04-13T03:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aadcba640 1 --2- 172.21.15.106:0/472698951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa815b4e0 0x7f2aa815fb60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80cd9e0 tx=0x7f2aa002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d1f033c69dda58 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaccb8640 1 -- 172.21.15.106:0/472698951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa003b070 con 0x7f2aa815b4e0 2024-04-13T03:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaccb8640 1 -- 172.21.15.106:0/472698951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa0036440 con 0x7f2aa815b4e0 2024-04-13T03:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaccb8640 1 -- 172.21.15.106:0/472698951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0036810 con 0x7f2aa815b4e0 2024-04-13T03:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 -- 172.21.15.106:0/472698951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa815b4e0 msgr2=0x7f2aa815fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/472698951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa815b4e0 0x7f2aa815fb60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80cd9e0 tx=0x7f2aa002e170 comp rx=0 tx=0).stop 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 -- 172.21.15.106:0/472698951 shutdown_connections 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/472698951 >> v2:172.21.15.114:3300/0 conn(0x7f2aa80b5390 0x7f2aa806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/472698951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa815b4e0 0x7f2aa815fb60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 -- 172.21.15.106:0/472698951 >> 172.21.15.106:0/472698951 conn(0x7f2aa80b2600 msgr2=0x7f2aa80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 -- 172.21.15.106:0/472698951 shutdown_connections 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.822+0000 7f2aaff1e640 1 -- 172.21.15.106:0/472698951 wait complete. 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.823+0000 7f2aaff1e640 1 Processor -- start 2024-04-13T03:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 -- start start 2024-04-13T03:13:31.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2aa80b4730 0x7f2aa815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 0x7f2aa80cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa806d0c0 con 0x7f2aa80b5390 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa80b32d0 con 0x7f2aa815b4e0 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aaff1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa80ad730 con 0x7f2aa80b4730 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aad4b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aad4b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34674/0 (socket says 172.21.15.106:34674) 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.824+0000 7f2aad4b9640 1 -- 172.21.15.106:0/1370045814 learned_addr learned my addr 172.21.15.106:0/1370045814 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 -- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2aa80b4730 msgr2=0x7f2aa815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2aa80b4730 0x7f2aa815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 -- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 msgr2=0x7f2aa80cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aae4bb640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 0x7f2aa80cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 0x7f2aa80cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 -- 172.21.15.106:0/1370045814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81f4350 con 0x7f2aa80b5390 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aae4bb640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 0x7f2aa80cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aad4b9640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c4d0 tx=0x7f2a9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98018070 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a98007500 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98004a20 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81f4630 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.825+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa80ad960 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.826+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa8068a10 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.826+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a9801b070 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.827+0000 7f2a9e7fc640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.827+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f2a9808ac90 con 0x7f2aa80b5390 2024-04-13T03:13:31.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.827+0000 7f2aadcba640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:31.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.827+0000 7f2aadcba640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0038000 tx=0x7f2aa0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:31.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:31.831+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a980592c0 con 0x7f2aa80b5390 2024-04-13T03:13:32.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.010+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2aa80adb30 con 0x7f2aa80b5390 2024-04-13T03:13:32.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.015+0000 7f2a9e7fc640 1 -- 172.21.15.106:0/1370045814 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f2a98058bc0 con 0x7f2aa80b5390 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 msgr2=0x7f2a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0038000 tx=0x7f2aa0030000 comp rx=0 tx=0).stop 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 msgr2=0x7f2aa81f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c4d0 tx=0x7f2a9800c990 comp rx=0 tx=0).stop 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 shutdown_connections 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2aa815b4e0 0x7f2aa80cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2aa80b5390 0x7f2aa81f3c70 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 --2- 172.21.15.106:0/1370045814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2aa80b4730 0x7f2aa815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 >> 172.21.15.106:0/1370045814 conn(0x7f2aa80b2600 msgr2=0x7f2aa80b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:32.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 shutdown_connections 2024-04-13T03:13:32.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.018+0000 7f2aaff1e640 1 -- 172.21.15.106:0/1370045814 wait complete. 2024-04-13T03:13:32.030 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:13:32.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:13:32.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:13:32.113 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:13:32.113 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:32.113 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:32.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:13:32.129 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:13:32.129 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:32.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:13:32.147 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:32.147 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:32.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:13:32.183 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:13:32.183 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:32.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:13:32.204 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:13:32.204 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:32.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.322+0000 7ff94cb1e640 1 Processor -- start 2024-04-13T03:13:32.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.322+0000 7ff94cb1e640 1 -- start start 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff94cb1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9480d1c00 0x7ff9480d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff94cb1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff9480b8080 0x7ff9481688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff94cb1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9480d3530 con 0x7ff9481622d0 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff94cb1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff9481626b0 con 0x7ff9480d1c00 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff94cb1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff9480d0b60 con 0x7ff9480b8080 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9480d1c00 0x7ff9480d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9480d1c00 0x7ff9480d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34688/0 (socket says 172.21.15.106:34688) 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 -- 172.21.15.106:0/2912020481 learned_addr learned my addr 172.21.15.106:0/2912020481 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 -- 172.21.15.106:0/2912020481 >> v2:172.21.15.114:3300/0 conn(0x7ff9480b8080 msgr2=0x7ff9481688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 --2- 172.21.15.106:0/2912020481 >> v2:172.21.15.114:3300/0 conn(0x7ff9480b8080 0x7ff9481688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 -- 172.21.15.106:0/2912020481 >> v1:172.21.15.114:6789/0 conn(0x7ff9481622d0 legacy=0x7ff9480b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:32.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.323+0000 7ff945d9b640 1 -- 172.21.15.106:0/2912020481 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9480d0e40 con 0x7ff9480d1c00 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff945d9b640 1 --2- 172.21.15.106:0/2912020481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff9480d04d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ceef0 tx=0x7ff93c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce1cec646166cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94559a640 1 -- 172.21.15.106:0/2912020481 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c035020 con 0x7ff9480d1c00 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94559a640 1 -- 172.21.15.106:0/2912020481 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c031870 con 0x7ff9480d1c00 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94559a640 1 -- 172.21.15.106:0/2912020481 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c031c40 con 0x7ff9480d1c00 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 -- 172.21.15.106:0/2912020481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 msgr2=0x7ff9480d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/2912020481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff9480d04d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ceef0 tx=0x7ff93c02b910 comp rx=0 tx=0).stop 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 -- 172.21.15.106:0/2912020481 shutdown_connections 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/2912020481 >> v2:172.21.15.114:3300/0 conn(0x7ff9480b8080 0x7ff9481688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/2912020481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff9480d04d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 -- 172.21.15.106:0/2912020481 >> 172.21.15.106:0/2912020481 conn(0x7ff9480d25d0 msgr2=0x7ff9480d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:32.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 -- 172.21.15.106:0/2912020481 shutdown_connections 2024-04-13T03:13:32.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.324+0000 7ff94cb1e640 1 -- 172.21.15.106:0/2912020481 wait complete. 2024-04-13T03:13:32.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 Processor -- start 2024-04-13T03:13:32.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 -- start start 2024-04-13T03:13:32.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9480b8080 0x7ff9481489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9481622d0 0x7ff94815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480d16e0 con 0x7ff9480d1c00 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480d1400 con 0x7ff9481622d0 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.326+0000 7ff94cb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff9480cc6f0 con 0x7ff9480b8080 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34692/0 (socket says 172.21.15.106:34692) 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 -- 172.21.15.106:0/1063655908 learned_addr learned my addr 172.21.15.106:0/1063655908 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 -- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9480b8080 msgr2=0x7ff9481489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9480b8080 0x7ff9481489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 -- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9481622d0 msgr2=0x7ff94815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9481622d0 0x7ff94815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 -- 172.21.15.106:0/1063655908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94814bbb0 con 0x7ff9480d1c00 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff945d9b640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c03ad60 con 0x7ff9480d1c00 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c0443f0 con 0x7ff9480d1c00 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0447e0 con 0x7ff9480d1c00 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94814be90 con 0x7ff9480d1c00 2024-04-13T03:13:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.327+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9480ce7a0 con 0x7ff9480d1c00 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.328+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff948068a10 con 0x7ff9480d1c00 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.329+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff93c038020 con 0x7ff9480d1c00 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.329+0000 7ff936ffd640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 0x7ff92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.329+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7ff93c0adcc0 con 0x7ff9480d1c00 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.329+0000 7ff94659c640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 0x7ff92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.329+0000 7ff94659c640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 0x7ff92806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff93800abb0 tx=0x7ff938009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.333+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff93c03f070 con 0x7ff9480d1c00 2024-04-13T03:13:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.547+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff948149670 con 0x7ff9480d1c00 2024-04-13T03:13:32.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.547+0000 7ff936ffd640 1 -- 172.21.15.106:0/1063655908 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+3875 (secure 0 0 0) 0x7ff93c07c260 con 0x7ff9480d1c00 2024-04-13T03:13:32.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:32.550 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1309261874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1309261874},{"type":"v1","addr":"172.21.15.106:6837","nonce":1309261874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/318871725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":318871725},{"type":"v1","addr":"172.21.15.114:6834","nonce":318871725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/290921265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":290921265},{"type":"v1","addr":"172.21.15.114:6835","nonce":290921265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3619425563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3619425563},{"type":"v1","addr":"172.21.15.106:6836","nonce":3619425563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[]} 2024-04-13T03:13:32.550 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 32 2024-04-13T03:13:32.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 msgr2=0x7ff92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 0x7ff92806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff93800abb0 tx=0x7ff938009250 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 msgr2=0x7ff94814b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 shutdown_connections 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff92806c8f0 0x7ff92806edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9481622d0 0x7ff94815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9480d1c00 0x7ff94814b0a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 --2- 172.21.15.106:0/1063655908 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9480b8080 0x7ff9481489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 >> 172.21.15.106:0/1063655908 conn(0x7ff9480d25d0 msgr2=0x7ff9480b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 shutdown_connections 2024-04-13T03:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.549+0000 7ff94cb1e640 1 -- 172.21.15.106:0/1063655908 wait complete. 2024-04-13T03:13:32.562 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:13:32.562 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:13:32.563 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:13:32.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:13:32.563 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:13:32.563 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:13:32.563 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:13:32.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:13:32.563 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:13:32.564 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:13:32.564 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:13:32.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:13:32.564 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:13:32.564 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:13:32.564 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:13:32.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:13:32.572 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:13:32.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:13:32.573 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:13:32.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:13:32.577 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:13:32.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:13:32.577 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:13:32.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:13:32.615 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:13:32.615 DEBUG:teuthology.parallel:result is None 2024-04-13T03:13:32.616 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:13:32.616 DEBUG:teuthology.parallel:result is None 2024-04-13T03:13:32.620 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:13:32.620 DEBUG:teuthology.parallel:result is None 2024-04-13T03:13:32.620 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:13:32.621 DEBUG:teuthology.parallel:result is None 2024-04-13T03:13:32.621 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:13:32.691 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:13:32.691 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:13:32.706 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:13:32.706 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:13:32.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.728+0000 7f2db38b7640 1 Processor -- start 2024-04-13T03:13:32.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.728+0000 7f2db38b7640 1 -- start start 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db38b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2dac15ed90 0x7f2dac0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db38b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2dac0b3d60 0x7f2dac0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db38b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dac0cfd20 con 0x7f2dac15e9b0 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db38b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2dac162af0 con 0x7f2dac15ed90 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db38b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2dac162dd0 con 0x7f2dac0b3d60 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db0e52640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2dac15ed90 0x7f2dac0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.729+0000 7f2db1653640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2dac15e9b0 0x7f2dac0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48188/0 (socket says 172.21.15.106:48188) 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db0e52640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2dac15ed90 0x7f2dac0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59398/0 (socket says 172.21.15.106:59398) 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db1653640 1 -- 172.21.15.106:0/1639455299 learned_addr learned my addr 172.21.15.106:0/1639455299 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:32.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db1e54640 1 --2- 172.21.15.106:0/1639455299 >> v2:172.21.15.114:3300/0 conn(0x7f2dac0b3d60 0x7f2dac0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db0e52640 1 -- 172.21.15.106:0/1639455299 >> v2:172.21.15.114:3300/0 conn(0x7f2dac0b3d60 msgr2=0x7f2dac0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:32.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db0e52640 1 --2- 172.21.15.106:0/1639455299 >> v2:172.21.15.114:3300/0 conn(0x7f2dac0b3d60 0x7f2dac0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db0e52640 1 -- 172.21.15.106:0/1639455299 >> v1:172.21.15.106:6789/0 conn(0x7f2dac15e9b0 legacy=0x7f2dac0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.730+0000 7f2db0e52640 1 -- 172.21.15.106:0/1639455299 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1630b0 con 0x7f2dac15ed90 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.731+0000 7f2db0e52640 1 --2- 172.21.15.106:0/1639455299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac15ed90 0x7f2dac0d2300 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0cb7b0 tx=0x7f2d9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89726c7948c064a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.731+0000 7f2da3fff640 1 -- 172.21.15.106:0/1639455299 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c034070 con 0x7f2dac15ed90 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.731+0000 7f2da3fff640 1 -- 172.21.15.106:0/1639455299 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9c030610 con 0x7f2dac15ed90 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.731+0000 7f2da3fff640 1 -- 172.21.15.106:0/1639455299 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0309e0 con 0x7f2dac15ed90 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 -- 172.21.15.106:0/1639455299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac15ed90 msgr2=0x7f2dac0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 --2- 172.21.15.106:0/1639455299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac15ed90 0x7f2dac0d2300 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0cb7b0 tx=0x7f2d9c02b960 comp rx=0 tx=0).stop 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 -- 172.21.15.106:0/1639455299 shutdown_connections 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 --2- 172.21.15.106:0/1639455299 >> v2:172.21.15.114:3300/0 conn(0x7f2dac0b3d60 0x7f2dac0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 --2- 172.21.15.106:0/1639455299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac15ed90 0x7f2dac0d2300 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 -- 172.21.15.106:0/1639455299 >> 172.21.15.106:0/1639455299 conn(0x7f2dac08bf10 msgr2=0x7f2dac0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 -- 172.21.15.106:0/1639455299 shutdown_connections 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.732+0000 7f2db38b7640 1 -- 172.21.15.106:0/1639455299 wait complete. 2024-04-13T03:13:32.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.734+0000 7f2db38b7640 1 Processor -- start 2024-04-13T03:13:32.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.734+0000 7f2db38b7640 1 -- start start 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac162800 con 0x7f2dac06b8e0 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac162680 con 0x7f2dac15e9b0 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db38b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac0d4d00 con 0x7f2dac0b3d60 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1653640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db0e52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1653640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:44354/0 (socket says 172.21.15.106:44354) 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db0e52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:48156/0 (socket says 172.21.15.106:48156) 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1653640 1 -- 172.21.15.106:0/195785456 learned_addr learned my addr 172.21.15.106:0/195785456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59404/0 (socket says 172.21.15.106:59404) 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 -- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 msgr2=0x7f2dac1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 -- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 msgr2=0x7f2dac0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1e54640 1 -- 172.21.15.106:0/195785456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1e9460 con 0x7f2dac06b8e0 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db1653640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.735+0000 7f2db0e52640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2db1e54640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e7f0 tx=0x7f2da800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8017070 con 0x7f2dac06b8e0 2024-04-13T03:13:32.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da800c940 con 0x7f2dac06b8e0 2024-04-13T03:13:32.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac1e9740 con 0x7f2dac06b8e0 2024-04-13T03:13:32.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da800cd10 con 0x7f2dac06b8e0 2024-04-13T03:13:32.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac152800 con 0x7f2dac06b8e0 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.737+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac1e9f40 con 0x7f2dac06b8e0 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.737+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2da801a070 con 0x7f2dac06b8e0 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.738+0000 7f2da1ffb640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.738+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f2da808b8b0 con 0x7f2dac06b8e0 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.738+0000 7f2db1653640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:32.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.738+0000 7f2db1653640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 0x7f2d9006ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0d1cc0 tx=0x7f2da4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:32.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.742+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2da80597f0 con 0x7f2dac06b8e0 2024-04-13T03:13:32.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:32.927+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2dac068a10 con 0x7f2dac06b8e0 2024-04-13T03:13:33.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.733+0000 7f2da1ffb640 1 -- 172.21.15.106:0/195785456 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7f2da805daf0 con 0x7f2dac06b8e0 2024-04-13T03:13:33.735 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:13:33.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 msgr2=0x7f2d9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 0x7f2d9006ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0d1cc0 tx=0x7f2da4009040 comp rx=0 tx=0).stop 2024-04-13T03:13:33.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 msgr2=0x7f2dac0cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e7f0 tx=0x7f2da800ecb0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 shutdown_connections 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dac06b8e0 0x7f2dac0cdf10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dac15e9b0 0x7f2dac0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 --2- 172.21.15.106:0/195785456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2dac0b3d60 0x7f2dac1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 >> 172.21.15.106:0/195785456 conn(0x7f2dac08bf10 msgr2=0x7f2dac0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 shutdown_connections 2024-04-13T03:13:33.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.736+0000 7f2db38b7640 1 -- 172.21.15.106:0/195785456 wait complete. 2024-04-13T03:13:33.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.852+0000 7f7ed462d640 1 Processor -- start 2024-04-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.852+0000 7f7ed462d640 1 -- start start 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed462d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed462d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 0x7f7ecc0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed462d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc0d3530 con 0x7f7ecc0cc6f0 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed462d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc1626b0 con 0x7f7ecc1622d0 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed462d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc0d0b60 con 0x7f7ecc0d1c00 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed2bca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7ecc0cc6f0 0x7f7ecc0ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48198/0 (socket says 172.21.15.106:48198) 2024-04-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59410/0 (socket says 172.21.15.106:59410) 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed2bca640 1 -- 172.21.15.106:0/2071975710 learned_addr learned my addr 172.21.15.106:0/2071975710 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 -- 172.21.15.106:0/2071975710 >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 msgr2=0x7f7ecc0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed1bc8640 1 --2- 172.21.15.106:0/2071975710 >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 0x7f7ecc0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2071975710 >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 0x7f7ecc0b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed13c7640 1 -- 172.21.15.106:0/2071975710 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945652623 0 0) 0x7f7ecc0d3530 con 0x7f7ecc0cc6f0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 -- 172.21.15.106:0/2071975710 >> v1:172.21.15.106:6789/0 conn(0x7f7ecc0cc6f0 legacy=0x7f7ecc0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed23c9640 1 -- 172.21.15.106:0/2071975710 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc0d0e40 con 0x7f7ecc1622d0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.853+0000 7f7ed1bc8640 1 --2- 172.21.15.106:0/2071975710 >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 0x7f7ecc0b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.854+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2071975710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc073520 tx=0x7f7ec802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc9f851827532de server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.854+0000 7f7ed13c7640 1 -- 172.21.15.106:0/2071975710 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8035020 con 0x7f7ecc1622d0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.854+0000 7f7ed13c7640 1 -- 172.21.15.106:0/2071975710 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8031830 con 0x7f7ecc1622d0 2024-04-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.854+0000 7f7ed13c7640 1 -- 172.21.15.106:0/2071975710 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8031c00 con 0x7f7ecc1622d0 2024-04-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 -- 172.21.15.106:0/2071975710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 msgr2=0x7f7ecc0d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2071975710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc073520 tx=0x7f7ec802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 -- 172.21.15.106:0/2071975710 shutdown_connections 2024-04-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2071975710 >> v2:172.21.15.114:3300/0 conn(0x7f7ecc0d1c00 0x7f7ecc0b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2071975710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc0d04d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 -- 172.21.15.106:0/2071975710 >> 172.21.15.106:0/2071975710 conn(0x7f7ecc0d25d0 msgr2=0x7f7ecc0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 -- 172.21.15.106:0/2071975710 shutdown_connections 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.855+0000 7f7ed462d640 1 -- 172.21.15.106:0/2071975710 wait complete. 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.856+0000 7f7ed462d640 1 Processor -- start 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.856+0000 7f7ed462d640 1 -- start start 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 0x7f7ecc15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ecc0d1c00 0x7f7ecc06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d16e0 con 0x7f7ecc1622d0 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d1400 con 0x7f7ecc0cc6f0 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc0b7eb0 con 0x7f7ecc0d1c00 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59414/0 (socket says 172.21.15.106:59414) 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 -- 172.21.15.106:0/2960906456 learned_addr learned my addr 172.21.15.106:0/2960906456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 0x7f7ecc15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 -- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ecc0d1c00 msgr2=0x7f7ecc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed1bc8640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ecc0d1c00 0x7f7ecc06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ecc0d1c00 0x7f7ecc06d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 -- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 msgr2=0x7f7ecc15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 0x7f7ecc15c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 -- 172.21.15.106:0/2960906456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc06ff60 con 0x7f7ecc1622d0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 0x7f7ecc15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed2bca640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7a0 tx=0x7f7ec400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4018070 con 0x7f7ecc1622d0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec400a640 con 0x7f7ecc1622d0 2024-04-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc0704e0 con 0x7f7ecc1622d0 2024-04-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.857+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400aab0 con 0x7f7ecc1622d0 2024-04-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.858+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc070aa0 con 0x7f7ecc1622d0 2024-04-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.858+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc068a10 con 0x7f7ecc1622d0 2024-04-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.859+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec401b070 con 0x7f7ecc1622d0 2024-04-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.859+0000 7f7ec2ffd640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.859+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.859+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f7ec408af80 con 0x7f7ecc1622d0 2024-04-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.860+0000 7f7ed23c9640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 0x7f7eb006edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8036000 tx=0x7f7ec8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:33.864+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ec4059570 con 0x7f7ecc1622d0 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.107+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7ecc0b63a0 con 0x7f7ecc1622d0 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.107+0000 7f7ec2ffd640 1 -- 172.21.15.106:0/2960906456 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f7ec4058e70 con 0x7f7ecc1622d0 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.109 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.110 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:13:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:13:34.114 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.115 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:13:34.116 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.117 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:13:34.118 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:13:34.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.120 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:13:34.121 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:13:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 msgr2=0x7f7eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 0x7f7eb006edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8036000 tx=0x7f7ec8008030 comp rx=0 tx=0).stop 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 msgr2=0x7f7ecc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7a0 tx=0x7f7ec400cc60 comp rx=0 tx=0).stop 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 shutdown_connections 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ecc0d1c00 0x7f7ecc06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ecc1622d0 0x7f7ecc06d7c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 --2- 172.21.15.106:0/2960906456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ecc0cc6f0 0x7f7ecc15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 >> 172.21.15.106:0/2960906456 conn(0x7f7ecc0d25d0 msgr2=0x7f7ecc0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 shutdown_connections 2024-04-13T03:13:34.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.109+0000 7f7ed462d640 1 -- 172.21.15.106:0/2960906456 wait complete. 2024-04-13T03:13:34.125 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:34.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.223+0000 7f560c71e640 1 Processor -- start 2024-04-13T03:13:34.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.223+0000 7f560c71e640 1 -- start start 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560c71e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f560415c7d0 0x7f5604160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560c71e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f56040d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560c71e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56040d2310 con 0x7f56040ccba0 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560c71e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f56040b1f30 con 0x7f560415c7d0 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560c71e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f56040b2210 con 0x7f56040c60a0 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.224+0000 7f560acbb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f56040ccba0 0x7f56040b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48210/0 (socket says 172.21.15.106:48210) 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560acbb640 1 -- 172.21.15.106:0/3258090154 learned_addr learned my addr 172.21.15.106:0/3258090154 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534972215 0 0) 0x7f56040d2310 con 0x7f56040ccba0 2024-04-13T03:13:34.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f0003520 con 0x7f56040ccba0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> v2:172.21.15.114:3300/0 conn(0x7f560415c7d0 0x7f5604160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f5609cb9640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f56040d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 870263482 0 0) 0x7f55f0003520 con 0x7f56040ccba0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 msgr2=0x7f56040d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f56040d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 >> v2:172.21.15.114:3300/0 conn(0x7f560415c7d0 msgr2=0x7f5604160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 --2- 172.21.15.106:0/3258090154 >> v2:172.21.15.114:3300/0 conn(0x7f560415c7d0 0x7f5604160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56040b24f0 con 0x7f56040ccba0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> v2:172.21.15.114:3300/0 conn(0x7f560415c7d0 0x7f5604160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f55fc0019f0 con 0x7f56040ccba0 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 >> v1:172.21.15.106:6789/0 conn(0x7f56040ccba0 legacy=0x7f56040b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00039b0 0x7f55f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f55f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b1f30 con 0x7f55f00039b0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b2210 con 0x7f55f0004d40 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f55f000b9c0 con 0x7f56040c60a0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00039b0 0x7f55f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f5609cb9640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f55f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560acbb640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 -- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 msgr2=0x7f55f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f56040c60a0 0x7f55f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 -- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55f0004d40 msgr2=0x7f55f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.225+0000 7f560a4ba640 1 -- 172.21.15.106:0/3258090154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0003520 con 0x7f55f00039b0 2024-04-13T03:13:34.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.226+0000 7f560a4ba640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00039b0 0x7f55f0003dd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098d0 tx=0x7f55f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da28b819e5fe792 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.226+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400f040 con 0x7f55f00039b0 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.226+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4004230 con 0x7f55f00039b0 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.226+0000 7f56094b8640 1 -- 172.21.15.106:0/3258090154 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4003610 con 0x7f55f00039b0 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 -- 172.21.15.106:0/3258090154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00039b0 msgr2=0x7f55f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 --2- 172.21.15.106:0/3258090154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00039b0 0x7f55f0003dd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098d0 tx=0x7f55f0005830 comp rx=0 tx=0).stop 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560acbb640 1 -- 172.21.15.106:0/3258090154 reap_dead start 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 -- 172.21.15.106:0/3258090154 shutdown_connections 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 -- 172.21.15.106:0/3258090154 >> 172.21.15.106:0/3258090154 conn(0x7f56040d2550 msgr2=0x7f56040d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 -- 172.21.15.106:0/3258090154 shutdown_connections 2024-04-13T03:13:34.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.227+0000 7f560c71e640 1 -- 172.21.15.106:0/3258090154 wait complete. 2024-04-13T03:13:34.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.228+0000 7f560c71e640 1 Processor -- start 2024-04-13T03:13:34.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.228+0000 7f560c71e640 1 -- start start 2024-04-13T03:13:34.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 0x7f56041519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56040b2f90 0x7f56040b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f560415cbb0 con 0x7f560415de20 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56040d1c10 con 0x7f56040b2f90 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560c71e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f560415d060 con 0x7f55f00039b0 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59432/0 (socket says 172.21.15.106:59432) 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 -- 172.21.15.106:0/91106536 learned_addr learned my addr 172.21.15.106:0/91106536 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560acbb640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56040b2f90 0x7f56040b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560a4ba640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 0x7f56041519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 -- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 msgr2=0x7f56041519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 0x7f56041519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 -- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56040b2f90 msgr2=0x7f56040b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56040b2f90 0x7f56040b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 -- 172.21.15.106:0/91106536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560415d340 con 0x7f560415de20 2024-04-13T03:13:34.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f560a4ba640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 0x7f56041519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f5609cb9640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5600011630 tx=0x7f5600011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.229+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560000f920 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.230+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560000fac0 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.230+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560415d620 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.230+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560000fe70 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.230+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041f2710 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.231+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604069120 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.231+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5600015460 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.231+0000 7f55faffd640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 0x7f55e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.231+0000 7f560a4ba640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 0x7f55e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.231+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f560008c280 con 0x7f560415de20 2024-04-13T03:13:34.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.232+0000 7f560a4ba640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 0x7f55e806ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f55f4035000 tx=0x7f55f4039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.236+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5600019030 con 0x7f560415de20 2024-04-13T03:13:34.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.412+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56040d31b0 con 0x7f560415de20 2024-04-13T03:13:34.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.413+0000 7f55faffd640 1 -- 172.21.15.106:0/91106536 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13201 (secure 0 0 0) 0x7f560005a870 con 0x7f560415de20 2024-04-13T03:13:34.415 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:34.415 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":74,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:33.734073+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 msgr2=0x7f55e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 0x7f55e806ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f55f4035000 tx=0x7f55f4039040 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 msgr2=0x7f560415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5600011630 tx=0x7f5600011af0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 shutdown_connections 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f55e806c920 0x7f55e806ede0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56040b2f90 0x7f56040b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f560415de20 0x7f560415e2a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 --2- 172.21.15.106:0/91106536 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f55f00039b0 0x7f56041519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.415+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 >> 172.21.15.106:0/91106536 conn(0x7f56040d2550 msgr2=0x7f560415a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.416+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 shutdown_connections 2024-04-13T03:13:34.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.416+0000 7f560c71e640 1 -- 172.21.15.106:0/91106536 wait complete. 2024-04-13T03:13:34.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:34.429 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.531+0000 7f543fe36640 1 Processor -- start 2024-04-13T03:13:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.531+0000 7f543fe36640 1 -- start start 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543fe36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54380b47e0 0x7f543815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543fe36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f543806d220 con 0x7f5438145d20 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543fe36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54380b3740 con 0x7f54380b4c20 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543fe36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f54380b3a20 con 0x7f54380b47e0 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543d3d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54380b47e0 0x7f543815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543dbd2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5438145d20 0x7f543806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48226/0 (socket says 172.21.15.106:48226) 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543d3d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54380b47e0 0x7f543815cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59440/0 (socket says 172.21.15.106:59440) 2024-04-13T03:13:34.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543dbd2640 1 -- 172.21.15.106:0/620943074 learned_addr learned my addr 172.21.15.106:0/620943074 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543d3d1640 1 -- 172.21.15.106:0/620943074 >> v1:172.21.15.114:6789/0 conn(0x7f54380b4c20 legacy=0x7f54380cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543d3d1640 1 -- 172.21.15.106:0/620943074 >> v1:172.21.15.106:6789/0 conn(0x7f5438145d20 legacy=0x7f543806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543cbd0640 1 -- 172.21.15.106:0/620943074 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632569692 0 0) 0x7f543806d220 con 0x7f5438145d20 2024-04-13T03:13:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.532+0000 7f543d3d1640 1 -- 172.21.15.106:0/620943074 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54380b3d00 con 0x7f54380b47e0 2024-04-13T03:13:34.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543d3d1640 1 --2- 172.21.15.106:0/620943074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54380b47e0 0x7f543815cbc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f54280097a0 tx=0x7f542802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ebeaef376de7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543cbd0640 1 -- 172.21.15.106:0/620943074 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280315a0 con 0x7f54380b47e0 2024-04-13T03:13:34.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543cbd0640 1 -- 172.21.15.106:0/620943074 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428037070 con 0x7f54380b47e0 2024-04-13T03:13:34.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543cbd0640 1 -- 172.21.15.106:0/620943074 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428031960 con 0x7f54380b47e0 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543fe36640 1 -- 172.21.15.106:0/620943074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54380b47e0 msgr2=0x7f543815cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.533+0000 7f543fe36640 1 --2- 172.21.15.106:0/620943074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54380b47e0 0x7f543815cbc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f54280097a0 tx=0x7f542802b840 comp rx=0 tx=0).stop 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.534+0000 7f543fe36640 1 -- 172.21.15.106:0/620943074 shutdown_connections 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.534+0000 7f543fe36640 1 --2- 172.21.15.106:0/620943074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54380b47e0 0x7f543815cbc0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.534+0000 7f543fe36640 1 -- 172.21.15.106:0/620943074 >> 172.21.15.106:0/620943074 conn(0x7f54380b2600 msgr2=0x7f54380d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.534+0000 7f543fe36640 1 -- 172.21.15.106:0/620943074 shutdown_connections 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.534+0000 7f543fe36640 1 -- 172.21.15.106:0/620943074 wait complete. 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 Processor -- start 2024-04-13T03:13:34.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 -- start start 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f54380b47e0 0x7f543815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54380b4c20 0x7f5438143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5438146100 con 0x7f5438145d20 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f54380b42c0 con 0x7f54380b4c20 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543fe36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5438146280 con 0x7f54380b47e0 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.535+0000 7f543e3d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59442/0 (socket says 172.21.15.106:59442) 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 -- 172.21.15.106:0/4211027371 learned_addr learned my addr 172.21.15.106:0/4211027371 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 -- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f54380b47e0 msgr2=0x7f543815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543dbd2640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f54380b47e0 0x7f543815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543d3d1640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54380b4c20 0x7f5438143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f54380b47e0 0x7f543815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 -- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54380b4c20 msgr2=0x7f5438143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54380b4c20 0x7f5438143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 -- 172.21.15.106:0/4211027371 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543815ea80 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543e3d3640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f543000c780 tx=0x7f543000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430017070 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5430003da0 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543815ed60 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54300052b0 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.536+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543815f320 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.537+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5438068a10 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.538+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543001a070 con 0x7f5438145d20 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.538+0000 7f542e7fc640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 0x7f541c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.538+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f543008b000 con 0x7f5438145d20 2024-04-13T03:13:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.538+0000 7f543dbd2640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 0x7f541c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.538+0000 7f543dbd2640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 0x7f541c06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5434007fd0 tx=0x7f5434009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.542+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54300595f0 con 0x7f5438145d20 2024-04-13T03:13:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.714+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54380b02c0 con 0x7f5438145d20 2024-04-13T03:13:34.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.714+0000 7f542e7fc640 1 -- 172.21.15.106:0/4211027371 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f5430058ef0 con 0x7f5438145d20 2024-04-13T03:13:34.716 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 msgr2=0x7f541c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 0x7f541c06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5434007fd0 tx=0x7f5434009040 comp rx=0 tx=0).stop 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 msgr2=0x7f5438132600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f543000c780 tx=0x7f543000cc40 comp rx=0 tx=0).stop 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 shutdown_connections 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f541c06c8f0 0x7f541c06edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5438145d20 0x7f5438132600 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54380b4c20 0x7f5438143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 --2- 172.21.15.106:0/4211027371 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f54380b47e0 0x7f543815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 >> 172.21.15.106:0/4211027371 conn(0x7f54380b2600 msgr2=0x7f543806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 shutdown_connections 2024-04-13T03:13:34.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.716+0000 7f543fe36640 1 -- 172.21.15.106:0/4211027371 wait complete. 2024-04-13T03:13:34.729 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:13:34.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.827+0000 7effb1018640 1 Processor -- start 2024-04-13T03:13:34.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.827+0000 7effb1018640 1 -- start start 2024-04-13T03:13:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 0x7effac0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 0x7effac06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7effac0af6f0 0x7effac06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7effac06fac0 con 0x7effac0af6f0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7effac0cb6d0 con 0x7effac0cadb0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effb1018640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7effac0aff20 con 0x7effac0b65f0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effab59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7effac0af6f0 0x7effac06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effab59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7effac0af6f0 0x7effac06be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59450/0 (socket says 172.21.15.106:59450) 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effab59e640 1 -- 172.21.15.106:0/3738951780 learned_addr learned my addr 172.21.15.106:0/3738951780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effaa59c640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 0x7effac06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.828+0000 7effaad9d640 1 --2- 172.21.15.106:0/3738951780 >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 0x7effac0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 -- 172.21.15.106:0/3738951780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 msgr2=0x7effac06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 0x7effac06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 -- 172.21.15.106:0/3738951780 >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 msgr2=0x7effac0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 --2- 172.21.15.106:0/3738951780 >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 0x7effac0cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 -- 172.21.15.106:0/3738951780 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac0cbaa0 con 0x7effac0af6f0 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effaad9d640 1 --2- 172.21.15.106:0/3738951780 >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 0x7effac0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effaa59c640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 0x7effac06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effab59e640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac06be20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7effac06dfb0 tx=0x7effa402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c13716610c7eff server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.829+0000 7effa9d9b640 1 -- 172.21.15.106:0/3738951780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa402ca70 con 0x7effac0af6f0 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effa9d9b640 1 -- 172.21.15.106:0/3738951780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa402cc10 con 0x7effac0af6f0 2024-04-13T03:13:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effa9d9b640 1 -- 172.21.15.106:0/3738951780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa40343f0 con 0x7effac0af6f0 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 -- 172.21.15.106:0/3738951780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 msgr2=0x7effac06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac06be20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7effac06dfb0 tx=0x7effa402bed0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 -- 172.21.15.106:0/3738951780 shutdown_connections 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac06be20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 --2- 172.21.15.106:0/3738951780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0b65f0 0x7effac06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 --2- 172.21.15.106:0/3738951780 >> v2:172.21.15.114:3300/0 conn(0x7effac0cadb0 0x7effac0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 -- 172.21.15.106:0/3738951780 >> 172.21.15.106:0/3738951780 conn(0x7effac0cd1e0 msgr2=0x7effac0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 -- 172.21.15.106:0/3738951780 shutdown_connections 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.830+0000 7effb1018640 1 -- 172.21.15.106:0/3738951780 wait complete. 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.831+0000 7effb1018640 1 Processor -- start 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 -- start start 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 0x7effac15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 0x7effac148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effac06f470 con 0x7effac0af6f0 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7effac06f190 con 0x7effac0b65f0 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effb1018640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7effac06f750 con 0x7effac0cadb0 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effaad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effaad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59454/0 (socket says 172.21.15.106:59454) 2024-04-13T03:13:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.832+0000 7effaad9d640 1 -- 172.21.15.106:0/663374539 learned_addr learned my addr 172.21.15.106:0/663374539 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaa59c640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 0x7effac15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effab59e640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 0x7effac148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 -- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 msgr2=0x7effac148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 0x7effac148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 -- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 msgr2=0x7effac15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 0x7effac15f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 -- 172.21.15.106:0/663374539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac15fe00 con 0x7effac0af6f0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaa59c640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 0x7effac15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effab59e640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 0x7effac148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effaad9d640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7effa000e760 tx=0x7effa000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000f040 con 0x7effac0af6f0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0004230 con 0x7effac0af6f0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effac15f2f0 con 0x7effac0af6f0 2024-04-13T03:13:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000ee70 con 0x7effac0af6f0 2024-04-13T03:13:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.833+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effac15dfc0 con 0x7effac0af6f0 2024-04-13T03:13:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.834+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effac068a10 con 0x7effac0af6f0 2024-04-13T03:13:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.834+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effa000c6d0 con 0x7effac0af6f0 2024-04-13T03:13:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.835+0000 7eff937fe640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 0x7eff9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.835+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7effa008b610 con 0x7effac0af6f0 2024-04-13T03:13:34.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.835+0000 7effaa59c640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 0x7eff9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:34.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.835+0000 7effaa59c640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 0x7eff9806edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7effac15c430 tx=0x7eff9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:34.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:34.839+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effa0017020 con 0x7effac0af6f0 2024-04-13T03:13:35.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.017+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7effac0b5530 con 0x7effac0af6f0 2024-04-13T03:13:35.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.017+0000 7eff937fe640 1 -- 172.21.15.106:0/663374539 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7effa0059c90 con 0x7effac0af6f0 2024-04-13T03:13:35.019 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:35.019 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:13:35.020 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:13:35.020 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:13:35.020 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:13:35.020 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:35.020 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:13:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 msgr2=0x7eff9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 0x7eff9806edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7effac15c430 tx=0x7eff9c008040 comp rx=0 tx=0).stop 2024-04-13T03:13:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 msgr2=0x7effac151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7effa000e760 tx=0x7effa000ec20 comp rx=0 tx=0).stop 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 shutdown_connections 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff9806c8f0 0x7eff9806edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effac0cadb0 0x7effac148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0b65f0 0x7effac15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 --2- 172.21.15.106:0/663374539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effac0af6f0 0x7effac151290 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 >> 172.21.15.106:0/663374539 conn(0x7effac0cd1e0 msgr2=0x7effac0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 shutdown_connections 2024-04-13T03:13:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.019+0000 7effb1018640 1 -- 172.21.15.106:0/663374539 wait complete. 2024-04-13T03:13:35.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:13:35.032 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:13:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.135+0000 7fd00b188640 1 Processor -- start 2024-04-13T03:13:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.135+0000 7fd00b188640 1 -- start start 2024-04-13T03:13:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd00b188640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415b620 0x7fd00406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd00b188640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd00415c760 0x7fd00415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd00b188640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00406f030 con 0x7fd00415bf40 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd00b188640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd0040ce1c0 con 0x7fd00415c760 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd00b188640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd00406fbb0 con 0x7fd00415b620 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd00415c760 0x7fd00415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd00415c760 0x7fd00415ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59464/0 (socket says 172.21.15.106:59464) 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 -- 172.21.15.106:0/3504590080 learned_addr learned my addr 172.21.15.106:0/3504590080 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 -- 172.21.15.106:0/3504590080 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415b620 msgr2=0x7fd00406e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd008f24640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415b620 0x7fd00406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415b620 0x7fd00406e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 -- 172.21.15.106:0/3504590080 >> v1:172.21.15.114:6789/0 conn(0x7fd00415bf40 legacy=0x7fd00415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.136+0000 7fd003fff640 1 -- 172.21.15.106:0/3504590080 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00406f310 con 0x7fd00415c760 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd003fff640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415c760 0x7fd00415ba00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcff000bb10 tx=0x7fcff002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc9878a309e2454 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd0037fe640 1 -- 172.21.15.106:0/3504590080 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff003b070 con 0x7fd00415c760 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd0037fe640 1 -- 172.21.15.106:0/3504590080 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff0036440 con 0x7fd00415c760 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd0037fe640 1 -- 172.21.15.106:0/3504590080 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0036810 con 0x7fd00415c760 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 -- 172.21.15.106:0/3504590080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415c760 msgr2=0x7fd00415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415c760 0x7fd00415ba00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcff000bb10 tx=0x7fcff002e0c0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 -- 172.21.15.106:0/3504590080 shutdown_connections 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415c760 0x7fd00415ba00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 --2- 172.21.15.106:0/3504590080 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415b620 0x7fd00406e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.137+0000 7fd00b188640 1 -- 172.21.15.106:0/3504590080 >> 172.21.15.106:0/3504590080 conn(0x7fd0040d0370 msgr2=0x7fd0040d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.138+0000 7fd00b188640 1 -- 172.21.15.106:0/3504590080 shutdown_connections 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.138+0000 7fd00b188640 1 -- 172.21.15.106:0/3504590080 wait complete. 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 Processor -- start 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 -- start start 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 0x7fd0040b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415c760 0x7fd00415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd00406ed20 con 0x7fd00415b620 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.139+0000 7fd00b188640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd00406f8d0 con 0x7fd00415bf40 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd00b188640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd00415c3d0 con 0x7fd00415c760 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59480/0 (socket says 172.21.15.106:59480) 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 -- 172.21.15.106:0/1057965145 learned_addr learned my addr 172.21.15.106:0/1057965145 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd003fff640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 0x7fd0040b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 -- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415c760 msgr2=0x7fd00415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd009725640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415c760 0x7fd00415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415c760 0x7fd00415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 -- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 msgr2=0x7fd0040b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 0x7fd0040b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 -- 172.21.15.106:0/1057965145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0040b4480 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd003fff640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 0x7fd0040b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd008f24640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7f0 tx=0x7fcff800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8015070 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8004240 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8007620 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0041407b0 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.140+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0040b49d0 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.141+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd004068a10 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.142+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcff80079d0 con 0x7fd00415b620 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.142+0000 7fd0017fa640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 0x7fcfe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.142+0000 7fd003fff640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 0x7fcfe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.142+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7fcff808be30 con 0x7fd00415b620 2024-04-13T03:13:35.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.143+0000 7fd003fff640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 0x7fcfe806edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcff0038000 tx=0x7fcff0032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.147+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0041407b0 con 0x7fd00415b620 2024-04-13T03:13:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.323+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd004156060 con 0x7fd00415b620 2024-04-13T03:13:35.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.742+0000 7fd0017fa640 1 -- 172.21.15.106:0/1057965145 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7fcff805a390 con 0x7fd00415b620 2024-04-13T03:13:35.744 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:13:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.745+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 msgr2=0x7fcfe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.745+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 0x7fcfe806edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcff0038000 tx=0x7fcff0032000 comp rx=0 tx=0).stop 2024-04-13T03:13:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.745+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 msgr2=0x7fd004140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.745+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7f0 tx=0x7fcff800ccb0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 shutdown_connections 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcfe806c8f0 0x7fcfe806edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd00415c760 0x7fd00415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd00415bf40 0x7fd0040b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 --2- 172.21.15.106:0/1057965145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd00415b620 0x7fd004140270 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 >> 172.21.15.106:0/1057965145 conn(0x7fd0040d0370 msgr2=0x7fd004154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 shutdown_connections 2024-04-13T03:13:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.746+0000 7fd00b188640 1 -- 172.21.15.106:0/1057965145 wait complete. 2024-04-13T03:13:35.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:13:35.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.867+0000 7fc7adeb7640 1 Processor -- start 2024-04-13T03:13:35.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.867+0000 7fc7adeb7640 1 -- start start 2024-04-13T03:13:35.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7adeb7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7a806f200 0x7fc7a806f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7adeb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7a806b8e0 0x7fc7a806bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7adeb7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a80b5090 con 0x7fc7a80af800 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7adeb7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a80b3ff0 con 0x7fc7a806b8e0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7adeb7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a80b42d0 con 0x7fc7a806f200 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7a7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7a806b8e0 0x7fc7a806bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7a7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7a806b8e0 0x7fc7a806bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59496/0 (socket says 172.21.15.106:59496) 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.868+0000 7fc7a7fff640 1 -- 172.21.15.106:0/2889151894 learned_addr learned my addr 172.21.15.106:0/2889151894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a7fff640 1 -- 172.21.15.106:0/2889151894 >> v2:172.21.15.114:3300/0 conn(0x7fc7a806f200 msgr2=0x7fc7a806f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a7fff640 1 --2- 172.21.15.106:0/2889151894 >> v2:172.21.15.114:3300/0 conn(0x7fc7a806f200 0x7fc7a806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a7fff640 1 -- 172.21.15.106:0/2889151894 >> v1:172.21.15.114:6789/0 conn(0x7fc7a80af800 legacy=0x7fc7a815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a7fff640 1 -- 172.21.15.106:0/2889151894 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a80b45b0 con 0x7fc7a806b8e0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a7fff640 1 --2- 172.21.15.106:0/2889151894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a806bd50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc7a815c600 tx=0x7fc7a002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80a9dbac1cb6517 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a67fc640 1 -- 172.21.15.106:0/2889151894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a002ea90 con 0x7fc7a806b8e0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a67fc640 1 -- 172.21.15.106:0/2889151894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a002ec30 con 0x7fc7a806b8e0 2024-04-13T03:13:35.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.869+0000 7fc7a67fc640 1 -- 172.21.15.106:0/2889151894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0009600 con 0x7fc7a806b8e0 2024-04-13T03:13:35.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 -- 172.21.15.106:0/2889151894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 msgr2=0x7fc7a806bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/2889151894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a806bd50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc7a815c600 tx=0x7fc7a002e0f0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 -- 172.21.15.106:0/2889151894 shutdown_connections 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/2889151894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a806bd50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/2889151894 >> v2:172.21.15.114:3300/0 conn(0x7fc7a806f200 0x7fc7a806f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 -- 172.21.15.106:0/2889151894 >> 172.21.15.106:0/2889151894 conn(0x7fc7a80b2600 msgr2=0x7fc7a80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 -- 172.21.15.106:0/2889151894 shutdown_connections 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.870+0000 7fc7adeb7640 1 -- 172.21.15.106:0/2889151894 wait complete. 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.871+0000 7fc7adeb7640 1 Processor -- start 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.871+0000 7fc7adeb7640 1 -- start start 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a806f200 0x7fc7a81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a80af800 0x7fc7a81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a80b3cc0 con 0x7fc7a806b8e0 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a80b4b70 con 0x7fc7a806f200 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7adeb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a806fef0 con 0x7fc7a80af800 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59506/0 (socket says 172.21.15.106:59506) 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 -- 172.21.15.106:0/386992157 learned_addr learned my addr 172.21.15.106:0/386992157 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a7fff640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a80af800 0x7fc7a81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 -- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a80af800 msgr2=0x7fc7a81537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a80af800 0x7fc7a81537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 -- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a806f200 msgr2=0x7fc7a81532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a806f200 0x7fc7a81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:35.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.872+0000 7fc7a77fe640 1 -- 172.21.15.106:0/386992157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a8148f80 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc7a77fe640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c410 tx=0x7fc79800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798017070 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7980035f0 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a814aa70 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798003a20 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.873+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a8150160 con 0x7fc7a806b8e0 2024-04-13T03:13:35.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.874+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a80b5280 con 0x7fc7a806b8e0 2024-04-13T03:13:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.874+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc79801a070 con 0x7fc7a806b8e0 2024-04-13T03:13:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.875+0000 7fc78ffff640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 0x7fc79406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.875+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5619+0+0 (secure 0 0 0) 0x7fc79808af80 con 0x7fc7a806b8e0 2024-04-13T03:13:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.875+0000 7fc7a6ffd640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 0x7fc79406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:35.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.875+0000 7fc7a6ffd640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 0x7fc79406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7a814fb40 tx=0x7fc79c006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:35.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:35.879+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7a80b5280 con 0x7fc7a806b8e0 2024-04-13T03:13:36.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.054+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc7a8068a10 con 0x7fc7a806b8e0 2024-04-13T03:13:36.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.748+0000 7fc78ffff640 1 -- 172.21.15.106:0/386992157 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7fc798059490 con 0x7fc7a806b8e0 2024-04-13T03:13:36.750 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:13:36.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.750+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 msgr2=0x7fc79406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.750+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 0x7fc79406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7a814fb40 tx=0x7fc79c006c90 comp rx=0 tx=0).stop 2024-04-13T03:13:36.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.750+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 msgr2=0x7fc7a8148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.750+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c410 tx=0x7fc79800c8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 shutdown_connections 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79406c8f0 0x7fc79406edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a80af800 0x7fc7a81537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a806f200 0x7fc7a81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 --2- 172.21.15.106:0/386992157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a806b8e0 0x7fc7a8148a40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 >> 172.21.15.106:0/386992157 conn(0x7fc7a80b2600 msgr2=0x7fc7a8149660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.751+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 shutdown_connections 2024-04-13T03:13:36.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.752+0000 7fc7adeb7640 1 -- 172.21.15.106:0/386992157 wait complete. 2024-04-13T03:13:36.774 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:13:36.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.876+0000 7fb1beb1e640 1 Processor -- start 2024-04-13T03:13:36.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.876+0000 7fb1beb1e640 1 -- start start 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1beb1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1b80af800 0x7fb1b815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1beb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b806f690 0x7fb1b815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1beb1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b815c200 con 0x7fb1b806f200 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1beb1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b80b5090 con 0x7fb1b80af800 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1beb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b806fb00 con 0x7fb1b806f690 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1b80af800 0x7fb1b815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb1b806f200 0x7fb1b815e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48240/0 (socket says 172.21.15.106:48240) 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b7fff640 1 -- 172.21.15.106:0/3873825767 learned_addr learned my addr 172.21.15.106:0/3873825767 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 -- 172.21.15.106:0/3873825767 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b806f690 msgr2=0x7fb1b815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:36.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 --2- 172.21.15.106:0/3873825767 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b806f690 0x7fb1b815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 -- 172.21.15.106:0/3873825767 >> v1:172.21.15.106:6789/0 conn(0x7fb1b806f200 legacy=0x7fb1b815e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 -- 172.21.15.106:0/3873825767 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80b3ff0 con 0x7fb1b80af800 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b77fe640 1 -- 172.21.15.106:0/3873825767 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980505720 0 0) 0x7fb1ac001110 con 0x7fb1b806f200 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1bc8ba640 1 --2- 172.21.15.106:0/3873825767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b80af800 0x7fb1b815bcc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80097a0 tx=0x7fb1a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c519a57c991be627 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b77fe640 1 -- 172.21.15.106:0/3873825767 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8035020 con 0x7fb1b80af800 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b77fe640 1 -- 172.21.15.106:0/3873825767 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a8031830 con 0x7fb1b80af800 2024-04-13T03:13:36.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.878+0000 7fb1b77fe640 1 -- 172.21.15.106:0/3873825767 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8031c00 con 0x7fb1b80af800 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 -- 172.21.15.106:0/3873825767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b80af800 msgr2=0x7fb1b815bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/3873825767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b80af800 0x7fb1b815bcc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80097a0 tx=0x7fb1a802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 -- 172.21.15.106:0/3873825767 shutdown_connections 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/3873825767 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b806f690 0x7fb1b815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/3873825767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b80af800 0x7fb1b815bcc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 -- 172.21.15.106:0/3873825767 >> 172.21.15.106:0/3873825767 conn(0x7fb1b80b2600 msgr2=0x7fb1b80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 -- 172.21.15.106:0/3873825767 shutdown_connections 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.879+0000 7fb1beb1e640 1 -- 172.21.15.106:0/3873825767 wait complete. 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.880+0000 7fb1beb1e640 1 Processor -- start 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.880+0000 7fb1beb1e640 1 -- start start 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b806f200 0x7fb1b8154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b80af800 0x7fb1b80b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80b4890 con 0x7fb1b806f690 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80b45b0 con 0x7fb1b806f200 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80b4b70 con 0x7fb1b80af800 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59536/0 (socket says 172.21.15.106:59536) 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 -- 172.21.15.106:0/1148132279 learned_addr learned my addr 172.21.15.106:0/1148132279 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:36.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1bc8ba640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b806f200 0x7fb1b8154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1bd0bb640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b80af800 0x7fb1b80b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 -- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b80af800 msgr2=0x7fb1b80b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b80af800 0x7fb1b80b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 -- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b806f200 msgr2=0x7fb1b8154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b806f200 0x7fb1b8154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 -- 172.21.15.106:0/1148132279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b81553f0 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b7fff640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c400 tx=0x7fb1ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac017070 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ac005ce0 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b8159b50 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.881+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac007310 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.882+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b815a110 con 0x7fb1b806f690 2024-04-13T03:13:36.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.883+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8068a10 con 0x7fb1b806f690 2024-04-13T03:13:36.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.883+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1ac01a070 con 0x7fb1b806f690 2024-04-13T03:13:36.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.883+0000 7fb1b57fa640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 0x7fb1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:36.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.883+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6001+0+0 (secure 0 0 0) 0x7fb1ac08b020 con 0x7fb1b806f690 2024-04-13T03:13:36.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.883+0000 7fb1bc8ba640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 0x7fb1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:36.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.884+0000 7fb1bc8ba640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 0x7fb1a406edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8036000 tx=0x7fb1a803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:36.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:36.889+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1b815c200 con 0x7fb1b806f690 2024-04-13T03:13:37.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.101+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb1b80cc5e0 con 0x7fb1b806f690 2024-04-13T03:13:37.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.750+0000 7fb1b57fa640 1 -- 172.21.15.106:0/1148132279 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v34) ==== 174+0+0 (secure 0 0 0) 0x7fb1ac0593b0 con 0x7fb1b806f690 2024-04-13T03:13:37.752 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 11 and data pool 12 2024-04-13T03:13:37.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.753+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 msgr2=0x7fb1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.753+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 0x7fb1a406edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8036000 tx=0x7fb1a803c040 comp rx=0 tx=0).stop 2024-04-13T03:13:37.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.753+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 msgr2=0x7fb1b80b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.753+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c400 tx=0x7fb1ac00c8c0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 shutdown_connections 2024-04-13T03:13:37.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb1a406c8f0 0x7fb1a406edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b80af800 0x7fb1b80b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b806f690 0x7fb1b80b0a40 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 --2- 172.21.15.106:0/1148132279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b806f200 0x7fb1b8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.754+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 >> 172.21.15.106:0/1148132279 conn(0x7fb1b80b2600 msgr2=0x7fb1b80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.755+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 shutdown_connections 2024-04-13T03:13:37.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.755+0000 7fb1beb1e640 1 -- 172.21.15.106:0/1148132279 wait complete. 2024-04-13T03:13:37.780 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:37.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.882+0000 7f60ce636640 1 Processor -- start 2024-04-13T03:13:37.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.882+0000 7f60ce636640 1 -- start start 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60ce636640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60c815ed90 0x7f60c80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60ce636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c80b3d60 0x7f60c80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60ce636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80cfd20 con 0x7f60c815e9b0 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60ce636640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60c8162af0 con 0x7f60c815ed90 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60ce636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60c8162dd0 con 0x7f60c80b3d60 2024-04-13T03:13:37.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60c77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60c815ed90 0x7f60c80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60c7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f60c815e9b0 0x7f60c80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48246/0 (socket says 172.21.15.106:48246) 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.883+0000 7f60c7fff640 1 -- 172.21.15.106:0/3589434203 learned_addr learned my addr 172.21.15.106:0/3589434203 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60ccbd3640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c80b3d60 0x7f60c80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c77fe640 1 -- 172.21.15.106:0/3589434203 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c80b3d60 msgr2=0x7f60c80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c77fe640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c80b3d60 0x7f60c80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c77fe640 1 -- 172.21.15.106:0/3589434203 >> v1:172.21.15.106:6789/0 conn(0x7f60c815e9b0 legacy=0x7f60c80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c6ffd640 1 -- 172.21.15.106:0/3589434203 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852907558 0 0) 0x7f60c80cfd20 con 0x7f60c815e9b0 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c77fe640 1 -- 172.21.15.106:0/3589434203 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81630b0 con 0x7f60c815ed90 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.884+0000 7f60c77fe640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c815ed90 0x7f60c80d2300 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009790 tx=0x7f60bc02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876efe0d7218aede server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60c6ffd640 1 -- 172.21.15.106:0/3589434203 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc035020 con 0x7f60c815ed90 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60c6ffd640 1 -- 172.21.15.106:0/3589434203 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc031890 con 0x7f60c815ed90 2024-04-13T03:13:37.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60c6ffd640 1 -- 172.21.15.106:0/3589434203 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc031c60 con 0x7f60c815ed90 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 -- 172.21.15.106:0/3589434203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c815ed90 msgr2=0x7f60c80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c815ed90 0x7f60c80d2300 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009790 tx=0x7f60bc02b930 comp rx=0 tx=0).stop 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 -- 172.21.15.106:0/3589434203 shutdown_connections 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c80b3d60 0x7f60c80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 --2- 172.21.15.106:0/3589434203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c815ed90 0x7f60c80d2300 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 -- 172.21.15.106:0/3589434203 >> 172.21.15.106:0/3589434203 conn(0x7f60c808bf10 msgr2=0x7f60c80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 -- 172.21.15.106:0/3589434203 shutdown_connections 2024-04-13T03:13:37.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.885+0000 7f60ce636640 1 -- 172.21.15.106:0/3589434203 wait complete. 2024-04-13T03:13:37.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.886+0000 7f60ce636640 1 Processor -- start 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 -- start start 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c815e9b0 0x7f60c81e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c81e8d50 0x7f60c81e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8162800 con 0x7f60c80b3d60 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8162680 con 0x7f60c81e8d50 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ce636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80d4d00 con 0x7f60c815e9b0 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59564/0 (socket says 172.21.15.106:59564) 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60c7fff640 1 -- 172.21.15.106:0/458601251 learned_addr learned my addr 172.21.15.106:0/458601251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60ccbd3640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c81e8d50 0x7f60c81e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.887+0000 7f60c77fe640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c815e9b0 0x7f60c81e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 -- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c815e9b0 msgr2=0x7f60c81e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c815e9b0 0x7f60c81e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 -- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c81e8d50 msgr2=0x7f60c81e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c81e8d50 0x7f60c81e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 -- 172.21.15.106:0/458601251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81603d0 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c7fff640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c500 tx=0x7f60b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8018070 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b8007580 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81606b0 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8004a20 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.888+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c8160c70 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.889+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c8068a10 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.889+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60b801b070 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.890+0000 7f60c4ff9640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 0x7f60b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.890+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f60b808af00 con 0x7f60c80b3d60 2024-04-13T03:13:37.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.890+0000 7f60c77fe640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 0x7f60b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:37.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.890+0000 7f60c77fe640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 0x7f60b406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f60bc036000 tx=0x7f60bc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:37.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:37.894+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60b8058be0 con 0x7f60c80b3d60 2024-04-13T03:13:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.070+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60c80b36e0 con 0x7f60c80b3d60 2024-04-13T03:13:38.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.071+0000 7f60c4ff9640 1 -- 172.21.15.106:0/458601251 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16994 (secure 0 0 0) 0x7f60b805cee0 con 0x7f60c80b3d60 2024-04-13T03:13:38.072 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:38.072 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:37.103809+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 msgr2=0x7f60b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 0x7f60b406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f60bc036000 tx=0x7f60bc03c040 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 msgr2=0x7f60c815fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c500 tx=0x7f60b800c9c0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 shutdown_connections 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60b406c8f0 0x7f60b406edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60c815e9b0 0x7f60c81e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c81e8d50 0x7f60c81e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 --2- 172.21.15.106:0/458601251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60c80b3d60 0x7f60c815fdf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 >> 172.21.15.106:0/458601251 conn(0x7f60c808bf10 msgr2=0x7f60c80b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 shutdown_connections 2024-04-13T03:13:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.073+0000 7f60ce636640 1 -- 172.21.15.106:0/458601251 wait complete. 2024-04-13T03:13:38.085 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.185+0000 7f834a55e640 1 Processor -- start 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.185+0000 7f834a55e640 1 -- start start 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f834a55e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f834406e8e0 0x7f834415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f834a55e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83440cd770 0x7f83440cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f834a55e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834415f420 con 0x7f83440cf5c0 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f834a55e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f834406ecc0 con 0x7f83440cd770 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f834a55e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f83440707f0 con 0x7f834406e8e0 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83440cd770 0x7f83440cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f8348afb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f83440cf5c0 0x7f8344070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48250/0 (socket says 172.21.15.106:48250) 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83440cd770 0x7f83440cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59568/0 (socket says 172.21.15.106:59568) 2024-04-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f8348afb640 1 -- 172.21.15.106:0/3466009504 learned_addr learned my addr 172.21.15.106:0/3466009504 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f8343fff640 1 --2- 172.21.15.106:0/3466009504 >> v2:172.21.15.114:3300/0 conn(0x7f834406e8e0 0x7f834415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 -- 172.21.15.106:0/3466009504 >> v2:172.21.15.114:3300/0 conn(0x7f834406e8e0 msgr2=0x7f834415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 --2- 172.21.15.106:0/3466009504 >> v2:172.21.15.114:3300/0 conn(0x7f834406e8e0 0x7f834415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 -- 172.21.15.106:0/3466009504 >> v1:172.21.15.106:6789/0 conn(0x7f83440cf5c0 legacy=0x7f8344070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f83437fe640 1 -- 172.21.15.106:0/3466009504 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834415f700 con 0x7f83440cd770 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.186+0000 7f8342ffd640 1 -- 172.21.15.106:0/3466009504 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414134482 0 0) 0x7f834415f420 con 0x7f83440cf5c0 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f83437fe640 1 --2- 172.21.15.106:0/3466009504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cd770 0x7f83440cf080 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8344070970 tx=0x7f833802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189a4ab910d69f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f8342ffd640 1 -- 172.21.15.106:0/3466009504 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338035020 con 0x7f83440cd770 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f8342ffd640 1 -- 172.21.15.106:0/3466009504 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8338031830 con 0x7f83440cd770 2024-04-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f8342ffd640 1 -- 172.21.15.106:0/3466009504 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338031c00 con 0x7f83440cd770 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f834a55e640 1 -- 172.21.15.106:0/3466009504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cd770 msgr2=0x7f83440cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.187+0000 7f834a55e640 1 --2- 172.21.15.106:0/3466009504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cd770 0x7f83440cf080 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8344070970 tx=0x7f833802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 -- 172.21.15.106:0/3466009504 shutdown_connections 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 --2- 172.21.15.106:0/3466009504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cd770 0x7f83440cf080 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 --2- 172.21.15.106:0/3466009504 >> v2:172.21.15.114:3300/0 conn(0x7f834406e8e0 0x7f834415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 -- 172.21.15.106:0/3466009504 >> 172.21.15.106:0/3466009504 conn(0x7f834406f180 msgr2=0x7f834406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 -- 172.21.15.106:0/3466009504 shutdown_connections 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.188+0000 7f834a55e640 1 -- 172.21.15.106:0/3466009504 wait complete. 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.189+0000 7f834a55e640 1 Processor -- start 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.189+0000 7f834a55e640 1 -- start start 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f834406e8e0 0x7f83440a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 0x7f83440739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83440cdbd0 con 0x7f83440cf5c0 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f834415fcc0 con 0x7f83440cd770 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8344071390 con 0x7f834406e8e0 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59584/0 (socket says 172.21.15.106:59584) 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 -- 172.21.15.106:0/1051744427 learned_addr learned my addr 172.21.15.106:0/1051744427 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8343fff640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f834406e8e0 0x7f83440a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f83437fe640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 0x7f83440739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 -- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f834406e8e0 msgr2=0x7f83440a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f834406e8e0 0x7f83440a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 -- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 msgr2=0x7f83440739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 0x7f83440739f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 -- 172.21.15.106:0/1051744427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83440a9ae0 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f83437fe640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 0x7f83440739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8348afb640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c700 tx=0x7f833c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c018070 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c00ce10 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83440a9dc0 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c00a860 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.190+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83440aa380 con 0x7f83440cf5c0 2024-04-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.191+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834406fac0 con 0x7f83440cf5c0 2024-04-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.192+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f833c01b070 con 0x7f83440cf5c0 2024-04-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.192+0000 7f8340ff9640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.192+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f833c08b250 con 0x7f83440cf5c0 2024-04-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.192+0000 7f8343fff640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.193+0000 7f8343fff640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 0x7f833006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8334006fd0 tx=0x7f8334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.197+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f834406fac0 con 0x7f83440cf5c0 2024-04-13T03:13:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.372+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8344068a10 con 0x7f83440cf5c0 2024-04-13T03:13:38.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.373+0000 7f8340ff9640 1 -- 172.21.15.106:0/1051744427 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16994 (secure 0 0 0) 0x7f833c0594f0 con 0x7f83440cf5c0 2024-04-13T03:13:38.374 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:38.374 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:37.103809+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 msgr2=0x7f833006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 0x7f833006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8334006fd0 tx=0x7f8334008040 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 msgr2=0x7f834414cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c700 tx=0x7f833c00cbc0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 shutdown_connections 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83440cf5c0 0x7f834414cf20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83440cd770 0x7f83440739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 --2- 172.21.15.106:0/1051744427 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f834406e8e0 0x7f83440a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 >> 172.21.15.106:0/1051744427 conn(0x7f834406f180 msgr2=0x7f8344162350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 shutdown_connections 2024-04-13T03:13:38.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.375+0000 7f834a55e640 1 -- 172.21.15.106:0/1051744427 wait complete. 2024-04-13T03:13:38.387 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:13:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.489+0000 7fdaa391e640 1 Processor -- start 2024-04-13T03:13:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.490+0000 7fdaa391e640 1 -- start start 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.490+0000 7fdaa391e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 0x7fda9c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.490+0000 7fdaa391e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9c06daa0 con 0x7fda9c15e6b0 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.490+0000 7fdaa391e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9c15d2d0 con 0x7fda9c0735a0 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.490+0000 7fdaa391e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda9c15db60 con 0x7fda9c06d180 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa1ebb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fda9c15e6b0 0x7fda9c15eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48262/0 (socket says 172.21.15.106:48262) 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa1ebb640 1 -- 172.21.15.106:0/2709954562 learned_addr learned my addr 172.21.15.106:0/2709954562 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa16ba640 1 --2- 172.21.15.106:0/2709954562 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 0x7fda9c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823685900 0 0) 0x7fda9c06daa0 con 0x7fda9c15e6b0 2024-04-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda88003520 con 0x7fda9c15e6b0 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 776947190 0 0) 0x7fda9c15d2d0 con 0x7fda9c0735a0 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda9c06daa0 con 0x7fda9c0735a0 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa16ba640 1 -- 172.21.15.106:0/2709954562 >> v1:172.21.15.114:6789/0 conn(0x7fda9c0735a0 legacy=0x7fda9c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa16ba640 1 -- 172.21.15.106:0/2709954562 >> v1:172.21.15.106:6789/0 conn(0x7fda9c15e6b0 legacy=0x7fda9c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1348891860 0 0) 0x7fda88003520 con 0x7fda9c15e6b0 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.491+0000 7fdaa16ba640 1 -- 172.21.15.106:0/2709954562 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c15d5a0 con 0x7fda9c06d180 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fdaa16ba640 1 --2- 172.21.15.106:0/2709954562 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 0x7fda9c06d560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda98009dc0 tx=0x7fda9802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605db3fece32ee28 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98039070 con 0x7fda9c06d180 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9802c7f0 con 0x7fda9c06d180 2024-04-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fda93fff640 1 -- 172.21.15.106:0/2709954562 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94001670 con 0x7fda9c06d180 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fdaa391e640 1 -- 172.21.15.106:0/2709954562 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 msgr2=0x7fda9c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.492+0000 7fdaa391e640 1 --2- 172.21.15.106:0/2709954562 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 0x7fda9c06d560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda98009dc0 tx=0x7fda9802bcc0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.493+0000 7fdaa391e640 1 -- 172.21.15.106:0/2709954562 shutdown_connections 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.493+0000 7fdaa391e640 1 --2- 172.21.15.106:0/2709954562 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c06d180 0x7fda9c06d560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.493+0000 7fdaa391e640 1 -- 172.21.15.106:0/2709954562 >> 172.21.15.106:0/2709954562 conn(0x7fda9c0725c0 msgr2=0x7fda9c0729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.493+0000 7fdaa391e640 1 -- 172.21.15.106:0/2709954562 shutdown_connections 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.493+0000 7fdaa391e640 1 -- 172.21.15.106:0/2709954562 wait complete. 2024-04-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.494+0000 7fdaa391e640 1 Processor -- start 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.494+0000 7fdaa391e640 1 -- start start 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.494+0000 7fdaa391e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.494+0000 7fdaa391e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda9c0735a0 0x7fda9c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.494+0000 7fdaa391e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 0x7fda9c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa391e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c15e190 con 0x7fda9c06d180 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa391e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c15deb0 con 0x7fda9c0735a0 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa391e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fda9c0d0870 con 0x7fda9c15e6b0 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59596/0 (socket says 172.21.15.106:59596) 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 -- 172.21.15.106:0/1427374181 learned_addr learned my addr 172.21.15.106:0/1427374181 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa0eb9640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda9c0735a0 0x7fda9c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa1ebb640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 0x7fda9c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 -- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 msgr2=0x7fda9c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 0x7fda9c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 -- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda9c0735a0 msgr2=0x7fda9c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda9c0735a0 0x7fda9c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 -- 172.21.15.106:0/1427374181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c15c370 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa1ebb640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 0x7fda9c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa16ba640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fda98009ee0 tx=0x7fda98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98044030 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9802de30 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c0aef20 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98039040 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.495+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c140a50 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.496+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c069120 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.497+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda98047070 con 0x7fda9c06d180 2024-04-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.497+0000 7fda91ffb640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.497+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fda980ae2a0 con 0x7fda9c06d180 2024-04-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.497+0000 7fdaa0eb9640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.498+0000 7fdaa0eb9640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 0x7fda8006edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda9c152c60 tx=0x7fda8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.502+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda9807c540 con 0x7fda9c06d180 2024-04-13T03:13:38.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.715+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fda9c15b4e0 con 0x7fda9c06d180 2024-04-13T03:13:38.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.766+0000 7fda91ffb640 1 -- 172.21.15.106:0/1427374181 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7fda9807be40 con 0x7fda9c06d180 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.768+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 msgr2=0x7fda8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.768+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 0x7fda8006edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda9c152c60 tx=0x7fda8c009040 comp rx=0 tx=0).stop 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.768+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 msgr2=0x7fda9c140510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.768+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fda98009ee0 tx=0x7fda98004a40 comp rx=0 tx=0).stop 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 shutdown_connections 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fda9c15e6b0 0x7fda9c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda9c0735a0 0x7fda9c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 --2- 172.21.15.106:0/1427374181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda9c06d180 0x7fda9c140510 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 >> 172.21.15.106:0/1427374181 conn(0x7fda9c0725c0 msgr2=0x7fda9c0b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 shutdown_connections 2024-04-13T03:13:38.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.769+0000 7fdaa391e640 1 -- 172.21.15.106:0/1427374181 wait complete. 2024-04-13T03:13:38.781 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:13:38.782 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:13:38.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 Processor -- start 2024-04-13T03:13:38.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 -- start start 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f129c15ed90 0x7f129c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c0cfd20 con 0x7f129c15e9b0 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c162af0 con 0x7f129c0b3d60 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.884+0000 7f12a1888640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f129c162dd0 con 0x7f129c15ed90 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f129c15e9b0 0x7f129c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48264/0 (socket says 172.21.15.106:48264) 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129affd640 1 -- 172.21.15.106:0/3477057087 learned_addr learned my addr 172.21.15.106:0/3477057087 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:38.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129a7fc640 1 --2- 172.21.15.106:0/3477057087 >> v2:172.21.15.114:3300/0 conn(0x7f129c15ed90 0x7f129c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2772631769 0 0) 0x7f129c0cfd20 con 0x7f129c15e9b0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1284003520 con 0x7f129c15e9b0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4173088162 0 0) 0x7f1284003520 con 0x7f129c15e9b0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 >> v2:172.21.15.114:3300/0 conn(0x7f129c15ed90 msgr2=0x7f129c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 --2- 172.21.15.106:0/3477057087 >> v2:172.21.15.114:3300/0 conn(0x7f129c15ed90 0x7f129c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 >> v1:172.21.15.114:6789/0 conn(0x7f129c0b3d60 legacy=0x7f129c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c1630b0 con 0x7f129c15e9b0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f128c001670 con 0x7f129c15e9b0 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 >> v1:172.21.15.106:6789/0 conn(0x7f129c15e9b0 legacy=0x7f129c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f1284003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1284004800 0x7f1284004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12840051c0 0x7f128400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f129c162dd0 con 0x7f12840039b0 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f128400c070 con 0x7f1284004800 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f128400c270 con 0x7f12840051c0 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129affd640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f1284003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129a7fc640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1284004800 0x7f1284004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.885+0000 7f129b7fe640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12840051c0 0x7f128400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 -- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12840051c0 msgr2=0x7f128400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12840051c0 0x7f128400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 -- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1284004800 msgr2=0x7f1284004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1284004800 0x7f1284004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 -- 172.21.15.106:0/3477057087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1284003520 con 0x7f12840039b0 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129b7fe640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12840051c0 0x7f128400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129b7fe640 1 -- 172.21.15.106:0/3477057087 reap_dead start 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f129affd640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f1284003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c240 tx=0x7f1284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821c1bde7e653ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c007cf0 con 0x7f12840039b0 2024-04-13T03:13:38.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c030070 con 0x7f12840039b0 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.886+0000 7f1299ffb640 1 -- 172.21.15.106:0/3477057087 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c005d30 con 0x7f12840039b0 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 -- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 msgr2=0x7f1284003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f1284003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c240 tx=0x7f1284005830 comp rx=0 tx=0).stop 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 -- 172.21.15.106:0/3477057087 shutdown_connections 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 --2- 172.21.15.106:0/3477057087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f1284003dd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 -- 172.21.15.106:0/3477057087 >> 172.21.15.106:0/3477057087 conn(0x7f129c08bf10 msgr2=0x7f129c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 -- 172.21.15.106:0/3477057087 shutdown_connections 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.887+0000 7f12a1888640 1 -- 172.21.15.106:0/3477057087 wait complete. 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.888+0000 7f12a1888640 1 Processor -- start 2024-04-13T03:13:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.888+0000 7f12a1888640 1 -- start start 2024-04-13T03:13:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f129c1601d0 0x7f129c160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f129c0c86d0 0x7f129c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f129c162800 con 0x7f12840039b0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f129c162680 con 0x7f129c1601d0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f12a1888640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f129c06eb20 con 0x7f129c0c86d0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59610/0 (socket says 172.21.15.106:59610) 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 -- 172.21.15.106:0/432718258 learned_addr learned my addr 172.21.15.106:0/432718258 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129a7fc640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f129c1601d0 0x7f129c160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129b7fe640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f129c0c86d0 0x7f129c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 -- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f129c0c86d0 msgr2=0x7f129c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f129c0c86d0 0x7f129c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 -- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f129c1601d0 msgr2=0x7f129c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f129c1601d0 0x7f129c160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.889+0000 7f129affd640 1 -- 172.21.15.106:0/432718258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c0cade0 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f129affd640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c240 tx=0x7f129c162cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c001c30 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c004bf0 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c0c92b0 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c007070 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.890+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c0b5730 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.891+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c068a10 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.891+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f128c007210 con 0x7f12840039b0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.892+0000 7f1283fff640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.892+0000 7f129a7fc640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.892+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f128c0acf10 con 0x7f12840039b0 2024-04-13T03:13:38.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.892+0000 7f129a7fc640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 0x7f128806edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f129c15a380 tx=0x7f1290008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:38.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:38.896+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f128c07b1b0 con 0x7f12840039b0 2024-04-13T03:13:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.106+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f129c0d53a0 con 0x7f12840039b0 2024-04-13T03:13:39.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.776+0000 7f1283fff640 1 -- 172.21.15.106:0/432718258 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f128c07aab0 con 0x7f12840039b0 2024-04-13T03:13:39.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 msgr2=0x7f128806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:39.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 0x7f128806edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f129c15a380 tx=0x7f1290008040 comp rx=0 tx=0).stop 2024-04-13T03:13:39.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 msgr2=0x7f129c158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c240 tx=0x7f129c162cc0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 shutdown_connections 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f129c0c86d0 0x7f129c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f129c1601d0 0x7f129c160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 --2- 172.21.15.106:0/432718258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12840039b0 0x7f129c158d90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 >> 172.21.15.106:0/432718258 conn(0x7f129c08bf10 msgr2=0x7f129c15c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 shutdown_connections 2024-04-13T03:13:39.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.778+0000 7f12a1888640 1 -- 172.21.15.106:0/432718258 wait complete. 2024-04-13T03:13:39.792 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:13:39.792 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:39.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.895+0000 7fc48bede640 1 Processor -- start 2024-04-13T03:13:39.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.895+0000 7fc48bede640 1 -- start start 2024-04-13T03:13:39.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48bede640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc48415e9c0 0x7fc4840ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48bede640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 0x7fc4840d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48bede640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48415eda0 con 0x7fc4841624b0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48bede640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc4840d27f0 con 0x7fc48415e9c0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48bede640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc484162c20 con 0x7fc4840b68d0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc48415e9c0 0x7fc4840ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc48415e9c0 0x7fc4840ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59612/0 (socket says 172.21.15.106:59612) 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 -- 172.21.15.106:0/1653903235 learned_addr learned my addr 172.21.15.106:0/1653903235 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48a47b640 1 --2- 172.21.15.106:0/1653903235 >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 0x7fc4840d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 -- 172.21.15.106:0/1653903235 >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 msgr2=0x7fc4840d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 --2- 172.21.15.106:0/1653903235 >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 0x7fc4840d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 -- 172.21.15.106:0/1653903235 >> v1:172.21.15.114:6789/0 conn(0x7fc4841624b0 legacy=0x7fc4840b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc489c7a640 1 -- 172.21.15.106:0/1653903235 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc484162f00 con 0x7fc48415e9c0 2024-04-13T03:13:39.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.896+0000 7fc48a47b640 1 --2- 172.21.15.106:0/1653903235 >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 0x7fc4840d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.897+0000 7fc489c7a640 1 --2- 172.21.15.106:0/1653903235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc48415e9c0 0x7fc4840ba290 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc484073c10 tx=0x7fc48002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e15d0725c0d7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.897+0000 7fc488c78640 1 -- 172.21.15.106:0/1653903235 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480035020 con 0x7fc48415e9c0 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.897+0000 7fc488c78640 1 -- 172.21.15.106:0/1653903235 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480031870 con 0x7fc48415e9c0 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.897+0000 7fc488c78640 1 -- 172.21.15.106:0/1653903235 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480031c40 con 0x7fc48415e9c0 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 -- 172.21.15.106:0/1653903235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc48415e9c0 msgr2=0x7fc4840ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 --2- 172.21.15.106:0/1653903235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc48415e9c0 0x7fc4840ba290 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc484073c10 tx=0x7fc48002b910 comp rx=0 tx=0).stop 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 -- 172.21.15.106:0/1653903235 shutdown_connections 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 --2- 172.21.15.106:0/1653903235 >> v2:172.21.15.114:3300/0 conn(0x7fc4840b68d0 0x7fc4840d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 --2- 172.21.15.106:0/1653903235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc48415e9c0 0x7fc4840ba290 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 -- 172.21.15.106:0/1653903235 >> 172.21.15.106:0/1653903235 conn(0x7fc4840d11c0 msgr2=0x7fc4840d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 -- 172.21.15.106:0/1653903235 shutdown_connections 2024-04-13T03:13:39.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.898+0000 7fc48bede640 1 -- 172.21.15.106:0/1653903235 wait complete. 2024-04-13T03:13:39.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.899+0000 7fc48bede640 1 Processor -- start 2024-04-13T03:13:39.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.899+0000 7fc48bede640 1 -- start start 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc48415e9c0 0x7fc4841f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4841624b0 0x7fc484152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc484162910 con 0x7fc4840b68d0 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840d2a40 con 0x7fc48415e9c0 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc4840d34a0 con 0x7fc4841624b0 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59628/0 (socket says 172.21.15.106:59628) 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 -- 172.21.15.106:0/3065539188 learned_addr learned my addr 172.21.15.106:0/3065539188 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 -- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4841624b0 msgr2=0x7fc484152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4841624b0 0x7fc484152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 -- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc48415e9c0 msgr2=0x7fc4841f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:39.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc48415e9c0 0x7fc4841f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 -- 172.21.15.106:0/3065539188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4840cabc0 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc489c7a640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc480000c00 tx=0x7fc4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480040070 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48003a420 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480035040 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4840caea0 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.900+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4840c93d0 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.901+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4840c8f20 con 0x7fc4840b68d0 2024-04-13T03:13:39.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.902+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc48004b060 con 0x7fc4840b68d0 2024-04-13T03:13:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.902+0000 7fc47a7fc640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.902+0000 7fc489479640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.902+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fc4800adc40 con 0x7fc4840b68d0 2024-04-13T03:13:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.903+0000 7fc489479640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc474006fd0 tx=0x7fc474006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:39.907+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc48007bf70 con 0x7fc4840b68d0 2024-04-13T03:13:40.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.121+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc484068a10 con 0x7fc4840b68d0 2024-04-13T03:13:40.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.121+0000 7fc47a7fc640 1 -- 172.21.15.106:0/3065539188 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7fc48007b870 con 0x7fc4840b68d0 2024-04-13T03:13:40.122 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:40.122 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:39.107280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6834/390879687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":390879687},{"type":"v1","addr":"172.21.15.114:6834","nonce":390879687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":5}]} 2024-04-13T03:13:40.123 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 36 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 msgr2=0x7fc46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc474006fd0 tx=0x7fc474006bc0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 msgr2=0x7fc4840c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc480000c00 tx=0x7fc4800046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 shutdown_connections 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4841624b0 0x7fc484152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc48415e9c0 0x7fc4841f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 --2- 172.21.15.106:0/3065539188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4840b68d0 0x7fc4840c98f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.123+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 >> 172.21.15.106:0/3065539188 conn(0x7fc4840d11c0 msgr2=0x7fc48415ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.124+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 shutdown_connections 2024-04-13T03:13:40.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.124+0000 7fc48bede640 1 -- 172.21.15.106:0/3065539188 wait complete. 2024-04-13T03:13:40.135 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:40.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.234+0000 7f652d8c8640 1 Processor -- start 2024-04-13T03:13:40.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.234+0000 7f652d8c8640 1 -- start start 2024-04-13T03:13:40.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f65280d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f652815ed90 0x7f65280d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f65280b3d60 0x7f65280d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f65280d5070 con 0x7f652815ed90 2024-04-13T03:13:40.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f65280cfd20 con 0x7f65280b3d60 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f652d8c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6528162af0 con 0x7f652815e9b0 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f652815ed90 0x7f65280d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f652815ed90 0x7f65280d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59632/0 (socket says 172.21.15.106:59632) 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 -- 172.21.15.106:0/73557305 learned_addr learned my addr 172.21.15.106:0/73557305 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 -- 172.21.15.106:0/73557305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 msgr2=0x7f65280d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 --2- 172.21.15.106:0/73557305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f65280d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 -- 172.21.15.106:0/73557305 >> v2:172.21.15.114:3300/0 conn(0x7f65280b3d60 msgr2=0x7f65280d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f65277fe640 1 --2- 172.21.15.106:0/73557305 >> v2:172.21.15.114:3300/0 conn(0x7f65280b3d60 0x7f65280d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 --2- 172.21.15.106:0/73557305 >> v2:172.21.15.114:3300/0 conn(0x7f65280b3d60 0x7f65280d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.235+0000 7f651ffff640 1 -- 172.21.15.106:0/73557305 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6528162dd0 con 0x7f652815ed90 2024-04-13T03:13:40.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.236+0000 7f651ffff640 1 --2- 172.21.15.106:0/73557305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652815ed90 0x7f65280d45f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6528162d00 tx=0x7f651002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d24e1cc0c36b33a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.236+0000 7f65267fc640 1 -- 172.21.15.106:0/73557305 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510004230 con 0x7f652815ed90 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.236+0000 7f65267fc640 1 -- 172.21.15.106:0/73557305 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65100043d0 con 0x7f652815ed90 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.236+0000 7f65267fc640 1 -- 172.21.15.106:0/73557305 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510003810 con 0x7f652815ed90 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 -- 172.21.15.106:0/73557305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652815ed90 msgr2=0x7f65280d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 --2- 172.21.15.106:0/73557305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652815ed90 0x7f65280d45f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6528162d00 tx=0x7f651002b960 comp rx=0 tx=0).stop 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 -- 172.21.15.106:0/73557305 shutdown_connections 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 --2- 172.21.15.106:0/73557305 >> v2:172.21.15.114:3300/0 conn(0x7f65280b3d60 0x7f65280d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 --2- 172.21.15.106:0/73557305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652815ed90 0x7f65280d45f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 --2- 172.21.15.106:0/73557305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f65280d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 -- 172.21.15.106:0/73557305 >> 172.21.15.106:0/73557305 conn(0x7f652808bf10 msgr2=0x7f65280b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 -- 172.21.15.106:0/73557305 shutdown_connections 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.237+0000 7f652d8c8640 1 -- 172.21.15.106:0/73557305 wait complete. 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.238+0000 7f652d8c8640 1 Processor -- start 2024-04-13T03:13:40.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.238+0000 7f652d8c8640 1 -- start start 2024-04-13T03:13:40.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f652806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f652806be20 0x7f652806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6528162680 con 0x7f65280b3d60 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6528163390 con 0x7f652806be20 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6528162800 con 0x7f652815e9b0 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59634/0 (socket says 172.21.15.106:59634) 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 -- 172.21.15.106:0/3458933435 learned_addr learned my addr 172.21.15.106:0/3458933435 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f65277fe640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f652806be20 0x7f652806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f651ffff640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f652806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 -- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 msgr2=0x7f652806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f652806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 -- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f652806be20 msgr2=0x7f652806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f652806be20 0x7f652806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 -- 172.21.15.106:0/3458933435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652806c330 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6526ffd640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f651800b4a0 tx=0x7f651800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518014070 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518010610 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.239+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652806c610 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.240+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518010a00 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.240+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652806cbd0 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.240+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6528068a10 con 0x7f65280b3d60 2024-04-13T03:13:40.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.241+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6518017070 con 0x7f65280b3d60 2024-04-13T03:13:40.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.241+0000 7f6524ff9640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 0x7f650806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.241+0000 7f651ffff640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 0x7f650806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.241+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f651808b740 con 0x7f65280b3d60 2024-04-13T03:13:40.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.242+0000 7f651ffff640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 0x7f650806ede0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f6510008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.246+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6518059300 con 0x7f65280b3d60 2024-04-13T03:13:40.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.425+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65280d53a0 con 0x7f65280b3d60 2024-04-13T03:13:40.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.426+0000 7f6524ff9640 1 -- 172.21.15.106:0/3458933435 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16981 (secure 0 0 0) 0x7f651805d600 con 0x7f65280b3d60 2024-04-13T03:13:40.427 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:40.427 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":78,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:38.762950+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 msgr2=0x7f650806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 0x7f650806ede0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f6510008030 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 msgr2=0x7f65280b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f651800b4a0 tx=0x7f651800b960 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 shutdown_connections 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f650806c920 0x7f650806ede0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f652815e9b0 0x7f652806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f652806be20 0x7f652806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 --2- 172.21.15.106:0/3458933435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65280b3d60 0x7f65280b6800 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 >> 172.21.15.106:0/3458933435 conn(0x7f652808bf10 msgr2=0x7f65280b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 shutdown_connections 2024-04-13T03:13:40.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.428+0000 7f652d8c8640 1 -- 172.21.15.106:0/3458933435 wait complete. 2024-04-13T03:13:40.440 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:13:40.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:13:40.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.538+0000 7f647dc98640 1 Processor -- start 2024-04-13T03:13:40.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 -- start start 2024-04-13T03:13:40.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 0x7f6478162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f647815ed60 0x7f647806d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647815fe40 con 0x7f647815e980 2024-04-13T03:13:40.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f64781633f0 con 0x7f647815ed60 2024-04-13T03:13:40.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.539+0000 7f647dc98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f647815f2c0 con 0x7f64780cd550 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f647815ed60 0x7f647806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6476ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 0x7f6478162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f64777fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f647815e980 0x7f64780cce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:35318/0 (socket says 172.21.15.106:35318) 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6476ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 0x7f6478162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:44582/0 (socket says 172.21.15.106:44582) 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f64777fe640 1 -- 172.21.15.106:0/213159344 learned_addr learned my addr 172.21.15.106:0/213159344 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f647815ed60 0x7f647806d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:48338/0 (socket says 172.21.15.106:48338) 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 -- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 msgr2=0x7f6478162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:40.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 --2- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 0x7f6478162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 -- 172.21.15.106:0/213159344 >> v1:172.21.15.114:6789/0 conn(0x7f647815e980 legacy=0x7f64780cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.540+0000 7f6477fff640 1 -- 172.21.15.106:0/213159344 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647815f5a0 con 0x7f647815ed60 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.541+0000 7f6477fff640 1 --2- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f647806d370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647002bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2515831cbdf4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.541+0000 7f64767fc640 1 -- 172.21.15.106:0/213159344 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470039070 con 0x7f647815ed60 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.541+0000 7f64767fc640 1 -- 172.21.15.106:0/213159344 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f647002cce0 con 0x7f647815ed60 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.541+0000 7f64767fc640 1 -- 172.21.15.106:0/213159344 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470034660 con 0x7f647815ed60 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 -- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 msgr2=0x7f647806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 --2- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f647806d370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647002bf30 comp rx=0 tx=0).stop 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 -- 172.21.15.106:0/213159344 shutdown_connections 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 --2- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f647806d370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 --2- 172.21.15.106:0/213159344 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64780cd550 0x7f6478162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 -- 172.21.15.106:0/213159344 >> 172.21.15.106:0/213159344 conn(0x7f64780d2550 msgr2=0x7f64780d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 -- 172.21.15.106:0/213159344 shutdown_connections 2024-04-13T03:13:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.542+0000 7f647dc98640 1 -- 172.21.15.106:0/213159344 wait complete. 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.543+0000 7f647dc98640 1 Processor -- start 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.543+0000 7f647dc98640 1 -- start start 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 0x7f6478158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f64781592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f64780d1e20 con 0x7f64780cd550 2024-04-13T03:13:40.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f647815fb60 con 0x7f647815ed60 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f64780d1fa0 con 0x7f647815e980 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59642/0 (socket says 172.21.15.106:59642) 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 -- 172.21.15.106:0/534173520 learned_addr learned my addr 172.21.15.106:0/534173520 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f6477fff640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f64781592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f6476ffd640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 0x7f6478158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 -- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 msgr2=0x7f6478158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 0x7f6478158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 -- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 msgr2=0x7f64781592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f64781592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 -- 172.21.15.106:0/534173520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647814cff0 con 0x7f64780cd550 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f6476ffd640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 0x7f6478158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f64777fe640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f646800c430 tx=0x7f646800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468017070 con 0x7f64780cd550 2024-04-13T03:13:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468005ce0 con 0x7f64780cd550 2024-04-13T03:13:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.544+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64780cc420 con 0x7f64780cd550 2024-04-13T03:13:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.545+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64680072f0 con 0x7f64780cd550 2024-04-13T03:13:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.545+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64780cb660 con 0x7f64780cd550 2024-04-13T03:13:40.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.545+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6478069120 con 0x7f64780cd550 2024-04-13T03:13:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.546+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f646801a070 con 0x7f64780cd550 2024-04-13T03:13:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.546+0000 7f645ffff640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 0x7f646406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.546+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f646808b040 con 0x7f64780cd550 2024-04-13T03:13:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.546+0000 7f6476ffd640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 0x7f646406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.547+0000 7f6476ffd640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 0x7f646406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f647815a0a0 tx=0x7f646c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.551+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f647814cff0 con 0x7f64780cd550 2024-04-13T03:13:40.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.722+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f647806f250 con 0x7f646406c8f0 2024-04-13T03:13:40.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.727+0000 7f645ffff640 1 -- 172.21.15.106:0/534173520 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7f647806f250 con 0x7f646406c8f0 2024-04-13T03:13:40.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:40.732 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 msgr2=0x7f646406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 0x7f646406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f647815a0a0 tx=0x7f646c008040 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 msgr2=0x7f647814cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f646800c430 tx=0x7f646800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 shutdown_connections 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f646406c8f0 0x7f646406edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f647815e980 0x7f6478158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f647815ed60 0x7f64781592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 --2- 172.21.15.106:0/534173520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64780cd550 0x7f647814cab0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 >> 172.21.15.106:0/534173520 conn(0x7f64780d2550 msgr2=0x7f647806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 shutdown_connections 2024-04-13T03:13:40.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.730+0000 7f647dc98640 1 -- 172.21.15.106:0/534173520 wait complete. 2024-04-13T03:13:40.742 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-13T03:13:40.417060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228752,"kb_used_data":8412,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754745968,"statfs":{"total":773094113280,"available":772859871232,"internally_reserved":0,"allocated":8613888,"data_stored":4556565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672266"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760106+0000","last_change":"2024-04-13T03:13:36.760106+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760106+0000","last_undegraded":"2024-04-13T03:13:36.760106+0000","last_fullsized":"2024-04-13T03:13:36.760106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755567+0000","last_change":"2024-04-13T03:13:36.755567+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755567+0000","last_undegraded":"2024-04-13T03:13:36.755567+0000","last_fullsized":"2024-04-13T03:13:36.755567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756896+0000","last_change":"2024-04-13T03:13:36.756896+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756896+0000","last_undegraded":"2024-04-13T03:13:36.756896+0000","last_fullsized":"2024-04-13T03:13:36.756896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755051+0000","last_change":"2024-04-13T03:13:36.755051+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755051+0000","last_undegraded":"2024-04-13T03:13:36.755051+0000","last_fullsized":"2024-04-13T03:13:36.755051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758625+0000","last_change":"2024-04-13T03:13:36.758625+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758625+0000","last_undegraded":"2024-04-13T03:13:36.758625+0000","last_fullsized":"2024-04-13T03:13:36.758625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758991+0000","last_change":"2024-04-13T03:13:36.758991+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758991+0000","last_undegraded":"2024-04-13T03:13:36.758991+0000","last_fullsized":"2024-04-13T03:13:36.758991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760023+0000","last_change":"2024-04-13T03:13:36.760023+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760023+0000","last_undegraded":"2024-04-13T03:13:36.760023+0000","last_fullsized":"2024-04-13T03:13:36.760023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758589+0000","last_change":"2024-04-13T03:13:36.758589+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758589+0000","last_undegraded":"2024-04-13T03:13:36.758589+0000","last_fullsized":"2024-04-13T03:13:36.758589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757338+0000","last_change":"2024-04-13T03:13:36.757338+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757338+0000","last_undegraded":"2024-04-13T03:13:36.757338+0000","last_fullsized":"2024-04-13T03:13:36.757338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754743+0000","last_change":"2024-04-13T03:13:36.754743+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754743+0000","last_undegraded":"2024-04-13T03:13:36.754743+0000","last_fullsized":"2024-04-13T03:13:36.754743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753756+0000","last_change":"2024-04-13T03:13:36.753756+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753756+0000","last_undegraded":"2024-04-13T03:13:36.753756+0000","last_fullsized":"2024-04-13T03:13:36.753756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754566+0000","last_change":"2024-04-13T03:13:36.754566+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754566+0000","last_undegraded":"2024-04-13T03:13:36.754566+0000","last_fullsized":"2024-04-13T03:13:36.754566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758260+0000","last_change":"2024-04-13T03:13:36.758260+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758260+0000","last_undegraded":"2024-04-13T03:13:36.758260+0000","last_fullsized":"2024-04-13T03:13:36.758260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758584+0000","last_change":"2024-04-13T03:13:36.758584+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758584+0000","last_undegraded":"2024-04-13T03:13:36.758584+0000","last_fullsized":"2024-04-13T03:13:36.758584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753512+0000","last_change":"2024-04-13T03:13:36.753512+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753512+0000","last_undegraded":"2024-04-13T03:13:36.753512+0000","last_fullsized":"2024-04-13T03:13:36.753512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760411+0000","last_change":"2024-04-13T03:13:36.760411+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760411+0000","last_undegraded":"2024-04-13T03:13:36.760411+0000","last_fullsized":"2024-04-13T03:13:36.760411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759321+0000","last_change":"2024-04-13T03:13:36.759321+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759321+0000","last_undegraded":"2024-04-13T03:13:36.759321+0000","last_fullsized":"2024-04-13T03:13:36.759321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756859+0000","last_change":"2024-04-13T03:13:36.756859+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756859+0000","last_undegraded":"2024-04-13T03:13:36.756859+0000","last_fullsized":"2024-04-13T03:13:36.756859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760997+0000","last_change":"2024-04-13T03:13:36.760997+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760997+0000","last_undegraded":"2024-04-13T03:13:36.760997+0000","last_fullsized":"2024-04-13T03:13:36.760997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758123+0000","last_change":"2024-04-13T03:13:36.758123+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758123+0000","last_undegraded":"2024-04-13T03:13:36.758123+0000","last_fullsized":"2024-04-13T03:13:36.758123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759788+0000","last_change":"2024-04-13T03:13:36.759788+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759788+0000","last_undegraded":"2024-04-13T03:13:36.759788+0000","last_fullsized":"2024-04-13T03:13:36.759788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757376+0000","last_change":"2024-04-13T03:13:36.757376+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757376+0000","last_undegraded":"2024-04-13T03:13:36.757376+0000","last_fullsized":"2024-04-13T03:13:36.757376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758911+0000","last_change":"2024-04-13T03:13:36.758911+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758911+0000","last_undegraded":"2024-04-13T03:13:36.758911+0000","last_fullsized":"2024-04-13T03:13:36.758911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755750+0000","last_change":"2024-04-13T03:13:36.755750+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755750+0000","last_undegraded":"2024-04-13T03:13:36.755750+0000","last_fullsized":"2024-04-13T03:13:36.755750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.752998+0000","last_change":"2024-04-13T03:13:36.752998+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.752998+0000","last_undegraded":"2024-04-13T03:13:36.752998+0000","last_fullsized":"2024-04-13T03:13:36.752998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754262+0000","last_change":"2024-04-13T03:13:36.754262+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754262+0000","last_undegraded":"2024-04-13T03:13:36.754262+0000","last_fullsized":"2024-04-13T03:13:36.754262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.762340+0000","last_change":"2024-04-13T03:13:36.762340+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.762340+0000","last_undegraded":"2024-04-13T03:13:36.762340+0000","last_fullsized":"2024-04-13T03:13:36.762340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757784+0000","last_change":"2024-04-13T03:13:36.757784+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757784+0000","last_undegraded":"2024-04-13T03:13:36.757784+0000","last_fullsized":"2024-04-13T03:13:36.757784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755190+0000","last_change":"2024-04-13T03:13:36.755190+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755190+0000","last_undegraded":"2024-04-13T03:13:36.755190+0000","last_fullsized":"2024-04-13T03:13:36.755190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758905+0000","last_change":"2024-04-13T03:13:36.758905+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758905+0000","last_undegraded":"2024-04-13T03:13:36.758905+0000","last_fullsized":"2024-04-13T03:13:36.758905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757226+0000","last_change":"2024-04-13T03:13:36.757226+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757226+0000","last_undegraded":"2024-04-13T03:13:36.757226+0000","last_fullsized":"2024-04-13T03:13:36.757226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759924+0000","last_change":"2024-04-13T03:13:36.759924+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759924+0000","last_undegraded":"2024-04-13T03:13:36.759924+0000","last_fullsized":"2024-04-13T03:13:36.759924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756357+0000","last_change":"2024-04-13T03:13:36.756357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756357+0000","last_undegraded":"2024-04-13T03:13:36.756357+0000","last_fullsized":"2024-04-13T03:13:36.756357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770054+0000","last_change":"2024-04-13T03:13:36.767960+0000","last_active":"2024-04-13T03:13:36.770054+0000","last_peered":"2024-04-13T03:13:36.770054+0000","last_clean":"2024-04-13T03:13:36.770054+0000","last_became_active":"2024-04-13T03:13:36.767704+0000","last_became_peered":"2024-04-13T03:13:36.767704+0000","last_unstale":"2024-04-13T03:13:36.770054+0000","last_undegraded":"2024-04-13T03:13:36.770054+0000","last_fullsized":"2024-04-13T03:13:36.770054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:36:14.758876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753075+0000","last_change":"2024-04-13T03:13:36.753075+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753075+0000","last_undegraded":"2024-04-13T03:13:36.753075+0000","last_fullsized":"2024-04-13T03:13:36.753075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771269+0000","last_change":"2024-04-13T03:13:36.769252+0000","last_active":"2024-04-13T03:13:36.771269+0000","last_peered":"2024-04-13T03:13:36.771269+0000","last_clean":"2024-04-13T03:13:36.771269+0000","last_became_active":"2024-04-13T03:13:36.768970+0000","last_became_peered":"2024-04-13T03:13:36.768970+0000","last_unstale":"2024-04-13T03:13:36.771269+0000","last_undegraded":"2024-04-13T03:13:36.771269+0000","last_fullsized":"2024-04-13T03:13:36.771269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:49.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758502+0000","last_change":"2024-04-13T03:13:36.758502+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758502+0000","last_undegraded":"2024-04-13T03:13:36.758502+0000","last_fullsized":"2024-04-13T03:13:36.758502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770243+0000","last_change":"2024-04-13T03:13:36.767892+0000","last_active":"2024-04-13T03:13:36.770243+0000","last_peered":"2024-04-13T03:13:36.770243+0000","last_clean":"2024-04-13T03:13:36.770243+0000","last_became_active":"2024-04-13T03:13:36.767618+0000","last_became_peered":"2024-04-13T03:13:36.767618+0000","last_unstale":"2024-04-13T03:13:36.770243+0000","last_undegraded":"2024-04-13T03:13:36.770243+0000","last_fullsized":"2024-04-13T03:13:36.770243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:43.589502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758721+0000","last_change":"2024-04-13T03:13:36.758721+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758721+0000","last_undegraded":"2024-04-13T03:13:36.758721+0000","last_fullsized":"2024-04-13T03:13:36.758721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772188+0000","last_change":"2024-04-13T03:13:36.769804+0000","last_active":"2024-04-13T03:13:36.772188+0000","last_peered":"2024-04-13T03:13:36.772188+0000","last_clean":"2024-04-13T03:13:36.772188+0000","last_became_active":"2024-04-13T03:13:36.769493+0000","last_became_peered":"2024-04-13T03:13:36.769493+0000","last_unstale":"2024-04-13T03:13:36.772188+0000","last_undegraded":"2024-04-13T03:13:36.772188+0000","last_fullsized":"2024-04-13T03:13:36.772188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753617+0000","last_change":"2024-04-13T03:13:36.753617+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753617+0000","last_undegraded":"2024-04-13T03:13:36.753617+0000","last_fullsized":"2024-04-13T03:13:36.753617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771158+0000","last_change":"2024-04-13T03:13:36.768909+0000","last_active":"2024-04-13T03:13:36.771158+0000","last_peered":"2024-04-13T03:13:36.771158+0000","last_clean":"2024-04-13T03:13:36.771158+0000","last_became_active":"2024-04-13T03:13:36.768554+0000","last_became_peered":"2024-04-13T03:13:36.768554+0000","last_unstale":"2024-04-13T03:13:36.771158+0000","last_undegraded":"2024-04-13T03:13:36.771158+0000","last_fullsized":"2024-04-13T03:13:36.771158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:56.727538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753296+0000","last_change":"2024-04-13T03:13:36.753296+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753296+0000","last_undegraded":"2024-04-13T03:13:36.753296+0000","last_fullsized":"2024-04-13T03:13:36.753296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770360+0000","last_change":"2024-04-13T03:13:36.767672+0000","last_active":"2024-04-13T03:13:36.770360+0000","last_peered":"2024-04-13T03:13:36.770360+0000","last_clean":"2024-04-13T03:13:36.770360+0000","last_became_active":"2024-04-13T03:13:36.767312+0000","last_became_peered":"2024-04-13T03:13:36.767312+0000","last_unstale":"2024-04-13T03:13:36.770360+0000","last_undegraded":"2024-04-13T03:13:36.770360+0000","last_fullsized":"2024-04-13T03:13:36.770360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:51.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757834+0000","last_change":"2024-04-13T03:13:36.757834+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757834+0000","last_undegraded":"2024-04-13T03:13:36.757834+0000","last_fullsized":"2024-04-13T03:13:36.757834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769561+0000","last_change":"2024-04-13T03:13:36.765735+0000","last_active":"2024-04-13T03:13:36.769561+0000","last_peered":"2024-04-13T03:13:36.769561+0000","last_clean":"2024-04-13T03:13:36.769561+0000","last_became_active":"2024-04-13T03:13:36.765485+0000","last_became_peered":"2024-04-13T03:13:36.765485+0000","last_unstale":"2024-04-13T03:13:36.769561+0000","last_undegraded":"2024-04-13T03:13:36.769561+0000","last_fullsized":"2024-04-13T03:13:36.769561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:56.813106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758918+0000","last_change":"2024-04-13T03:13:36.758918+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758918+0000","last_undegraded":"2024-04-13T03:13:36.758918+0000","last_fullsized":"2024-04-13T03:13:36.758918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770068+0000","last_change":"2024-04-13T03:13:36.769186+0000","last_active":"2024-04-13T03:13:36.770068+0000","last_peered":"2024-04-13T03:13:36.770068+0000","last_clean":"2024-04-13T03:13:36.770068+0000","last_became_active":"2024-04-13T03:13:36.768951+0000","last_became_peered":"2024-04-13T03:13:36.768951+0000","last_unstale":"2024-04-13T03:13:36.770068+0000","last_undegraded":"2024-04-13T03:13:36.770068+0000","last_fullsized":"2024-04-13T03:13:36.770068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:16:46.808010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753081+0000","last_change":"2024-04-13T03:13:36.753081+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753081+0000","last_undegraded":"2024-04-13T03:13:36.753081+0000","last_fullsized":"2024-04-13T03:13:36.753081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772133+0000","last_change":"2024-04-13T03:13:36.769948+0000","last_active":"2024-04-13T03:13:36.772133+0000","last_peered":"2024-04-13T03:13:36.772133+0000","last_clean":"2024-04-13T03:13:36.772133+0000","last_became_active":"2024-04-13T03:13:36.769546+0000","last_became_peered":"2024-04-13T03:13:36.769546+0000","last_unstale":"2024-04-13T03:13:36.772133+0000","last_undegraded":"2024-04-13T03:13:36.772133+0000","last_fullsized":"2024-04-13T03:13:36.772133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:46.141793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754015+0000","last_change":"2024-04-13T03:13:36.754015+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754015+0000","last_undegraded":"2024-04-13T03:13:36.754015+0000","last_fullsized":"2024-04-13T03:13:36.754015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769696+0000","last_change":"2024-04-13T03:13:36.766240+0000","last_active":"2024-04-13T03:13:36.769696+0000","last_peered":"2024-04-13T03:13:36.769696+0000","last_clean":"2024-04-13T03:13:36.769696+0000","last_became_active":"2024-04-13T03:13:36.763111+0000","last_became_peered":"2024-04-13T03:13:36.763111+0000","last_unstale":"2024-04-13T03:13:36.769696+0000","last_undegraded":"2024-04-13T03:13:36.769696+0000","last_fullsized":"2024-04-13T03:13:36.769696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:28.008984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758157+0000","last_change":"2024-04-13T03:13:36.758157+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758157+0000","last_undegraded":"2024-04-13T03:13:36.758157+0000","last_fullsized":"2024-04-13T03:13:36.758157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770017+0000","last_change":"2024-04-13T03:13:36.767663+0000","last_active":"2024-04-13T03:13:36.770017+0000","last_peered":"2024-04-13T03:13:36.770017+0000","last_clean":"2024-04-13T03:13:36.770017+0000","last_became_active":"2024-04-13T03:13:36.767407+0000","last_became_peered":"2024-04-13T03:13:36.767407+0000","last_unstale":"2024-04-13T03:13:36.770017+0000","last_undegraded":"2024-04-13T03:13:36.770017+0000","last_fullsized":"2024-04-13T03:13:36.770017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:48.797655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755936+0000","last_change":"2024-04-13T03:13:36.755936+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755936+0000","last_undegraded":"2024-04-13T03:13:36.755936+0000","last_fullsized":"2024-04-13T03:13:36.755936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772167+0000","last_change":"2024-04-13T03:13:36.768069+0000","last_active":"2024-04-13T03:13:36.772167+0000","last_peered":"2024-04-13T03:13:36.772167+0000","last_clean":"2024-04-13T03:13:36.772167+0000","last_became_active":"2024-04-13T03:13:36.767790+0000","last_became_peered":"2024-04-13T03:13:36.767790+0000","last_unstale":"2024-04-13T03:13:36.772167+0000","last_undegraded":"2024-04-13T03:13:36.772167+0000","last_fullsized":"2024-04-13T03:13:36.772167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:32:59.935067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753985+0000","last_change":"2024-04-13T03:13:36.753985+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753985+0000","last_undegraded":"2024-04-13T03:13:36.753985+0000","last_fullsized":"2024-04-13T03:13:36.753985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769647+0000","last_change":"2024-04-13T03:13:36.766629+0000","last_active":"2024-04-13T03:13:36.769647+0000","last_peered":"2024-04-13T03:13:36.769647+0000","last_clean":"2024-04-13T03:13:36.769647+0000","last_became_active":"2024-04-13T03:13:36.766361+0000","last_became_peered":"2024-04-13T03:13:36.766361+0000","last_unstale":"2024-04-13T03:13:36.769647+0000","last_undegraded":"2024-04-13T03:13:36.769647+0000","last_fullsized":"2024-04-13T03:13:36.769647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:44:10.176246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758070+0000","last_change":"2024-04-13T03:13:36.758070+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758070+0000","last_undegraded":"2024-04-13T03:13:36.758070+0000","last_fullsized":"2024-04-13T03:13:36.758070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772234+0000","last_change":"2024-04-13T03:13:36.770589+0000","last_active":"2024-04-13T03:13:36.772234+0000","last_peered":"2024-04-13T03:13:36.772234+0000","last_clean":"2024-04-13T03:13:36.772234+0000","last_became_active":"2024-04-13T03:13:36.770281+0000","last_became_peered":"2024-04-13T03:13:36.770281+0000","last_unstale":"2024-04-13T03:13:36.772234+0000","last_undegraded":"2024-04-13T03:13:36.772234+0000","last_fullsized":"2024-04-13T03:13:36.772234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:53.535105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758508+0000","last_change":"2024-04-13T03:13:36.758508+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758508+0000","last_undegraded":"2024-04-13T03:13:36.758508+0000","last_fullsized":"2024-04-13T03:13:36.758508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770078+0000","last_change":"2024-04-13T03:13:36.768599+0000","last_active":"2024-04-13T03:13:36.770078+0000","last_peered":"2024-04-13T03:13:36.770078+0000","last_clean":"2024-04-13T03:13:36.770078+0000","last_became_active":"2024-04-13T03:13:36.768343+0000","last_became_peered":"2024-04-13T03:13:36.768343+0000","last_unstale":"2024-04-13T03:13:36.770078+0000","last_undegraded":"2024-04-13T03:13:36.770078+0000","last_fullsized":"2024-04-13T03:13:36.770078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:22.711857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758484+0000","last_change":"2024-04-13T03:13:36.758484+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758484+0000","last_undegraded":"2024-04-13T03:13:36.758484+0000","last_fullsized":"2024-04-13T03:13:36.758484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765921+0000","last_change":"2024-04-13T03:13:36.764197+0000","last_active":"2024-04-13T03:13:36.765921+0000","last_peered":"2024-04-13T03:13:36.765921+0000","last_clean":"2024-04-13T03:13:36.765921+0000","last_became_active":"2024-04-13T03:13:36.763920+0000","last_became_peered":"2024-04-13T03:13:36.763920+0000","last_unstale":"2024-04-13T03:13:36.765921+0000","last_undegraded":"2024-04-13T03:13:36.765921+0000","last_fullsized":"2024-04-13T03:13:36.765921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:49.280931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761836+0000","last_change":"2024-04-13T03:13:36.761836+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761836+0000","last_undegraded":"2024-04-13T03:13:36.761836+0000","last_fullsized":"2024-04-13T03:13:36.761836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772173+0000","last_change":"2024-04-13T03:13:36.770260+0000","last_active":"2024-04-13T03:13:36.772173+0000","last_peered":"2024-04-13T03:13:36.772173+0000","last_clean":"2024-04-13T03:13:36.772173+0000","last_became_active":"2024-04-13T03:13:36.769888+0000","last_became_peered":"2024-04-13T03:13:36.769888+0000","last_unstale":"2024-04-13T03:13:36.772173+0000","last_undegraded":"2024-04-13T03:13:36.772173+0000","last_fullsized":"2024-04-13T03:13:36.772173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:39.653567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760574+0000","last_change":"2024-04-13T03:13:36.760574+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760574+0000","last_undegraded":"2024-04-13T03:13:36.760574+0000","last_fullsized":"2024-04-13T03:13:36.760574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770965+0000","last_change":"2024-04-13T03:13:36.765493+0000","last_active":"2024-04-13T03:13:36.770965+0000","last_peered":"2024-04-13T03:13:36.770965+0000","last_clean":"2024-04-13T03:13:36.770965+0000","last_became_active":"2024-04-13T03:13:36.765174+0000","last_became_peered":"2024-04-13T03:13:36.765174+0000","last_unstale":"2024-04-13T03:13:36.770965+0000","last_undegraded":"2024-04-13T03:13:36.770965+0000","last_fullsized":"2024-04-13T03:13:36.770965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757837+0000","last_change":"2024-04-13T03:13:36.757837+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757837+0000","last_undegraded":"2024-04-13T03:13:36.757837+0000","last_fullsized":"2024-04-13T03:13:36.757837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770670+0000","last_change":"2024-04-13T03:13:36.769841+0000","last_active":"2024-04-13T03:13:36.770670+0000","last_peered":"2024-04-13T03:13:36.770670+0000","last_clean":"2024-04-13T03:13:36.770670+0000","last_became_active":"2024-04-13T03:13:36.769598+0000","last_became_peered":"2024-04-13T03:13:36.769598+0000","last_unstale":"2024-04-13T03:13:36.770670+0000","last_undegraded":"2024-04-13T03:13:36.770670+0000","last_fullsized":"2024-04-13T03:13:36.770670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:11:43.265755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759904+0000","last_change":"2024-04-13T03:13:36.759904+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759904+0000","last_undegraded":"2024-04-13T03:13:36.759904+0000","last_fullsized":"2024-04-13T03:13:36.759904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767378+0000","last_change":"2024-04-13T03:13:36.765265+0000","last_active":"2024-04-13T03:13:36.767378+0000","last_peered":"2024-04-13T03:13:36.767378+0000","last_clean":"2024-04-13T03:13:36.767378+0000","last_became_active":"2024-04-13T03:13:36.764966+0000","last_became_peered":"2024-04-13T03:13:36.764966+0000","last_unstale":"2024-04-13T03:13:36.767378+0000","last_undegraded":"2024-04-13T03:13:36.767378+0000","last_fullsized":"2024-04-13T03:13:36.767378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:36.244015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753237+0000","last_change":"2024-04-13T03:13:36.753237+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753237+0000","last_undegraded":"2024-04-13T03:13:36.753237+0000","last_fullsized":"2024-04-13T03:13:36.753237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767948+0000","last_change":"2024-04-13T03:13:36.765596+0000","last_active":"2024-04-13T03:13:36.767948+0000","last_peered":"2024-04-13T03:13:36.767948+0000","last_clean":"2024-04-13T03:13:36.767948+0000","last_became_active":"2024-04-13T03:13:36.765330+0000","last_became_peered":"2024-04-13T03:13:36.765330+0000","last_unstale":"2024-04-13T03:13:36.767948+0000","last_undegraded":"2024-04-13T03:13:36.767948+0000","last_fullsized":"2024-04-13T03:13:36.767948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:02.126210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753008+0000","last_change":"2024-04-13T03:13:36.753008+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753008+0000","last_undegraded":"2024-04-13T03:13:36.753008+0000","last_fullsized":"2024-04-13T03:13:36.753008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773139+0000","last_change":"2024-04-13T03:13:36.772152+0000","last_active":"2024-04-13T03:13:36.773139+0000","last_peered":"2024-04-13T03:13:36.773139+0000","last_clean":"2024-04-13T03:13:36.773139+0000","last_became_active":"2024-04-13T03:13:36.771816+0000","last_became_peered":"2024-04-13T03:13:36.771816+0000","last_unstale":"2024-04-13T03:13:36.773139+0000","last_undegraded":"2024-04-13T03:13:36.773139+0000","last_fullsized":"2024-04-13T03:13:36.773139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:51:05.902931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757953+0000","last_change":"2024-04-13T03:13:36.757953+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757953+0000","last_undegraded":"2024-04-13T03:13:36.757953+0000","last_fullsized":"2024-04-13T03:13:36.757953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773107+0000","last_change":"2024-04-13T03:13:36.772084+0000","last_active":"2024-04-13T03:13:36.773107+0000","last_peered":"2024-04-13T03:13:36.773107+0000","last_clean":"2024-04-13T03:13:36.773107+0000","last_became_active":"2024-04-13T03:13:36.771800+0000","last_became_peered":"2024-04-13T03:13:36.771800+0000","last_unstale":"2024-04-13T03:13:36.773107+0000","last_undegraded":"2024-04-13T03:13:36.773107+0000","last_fullsized":"2024-04-13T03:13:36.773107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:02.961934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757223+0000","last_change":"2024-04-13T03:13:36.757223+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757223+0000","last_undegraded":"2024-04-13T03:13:36.757223+0000","last_fullsized":"2024-04-13T03:13:36.757223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768589+0000","last_change":"2024-04-13T03:13:36.766089+0000","last_active":"2024-04-13T03:13:36.768589+0000","last_peered":"2024-04-13T03:13:36.768589+0000","last_clean":"2024-04-13T03:13:36.768589+0000","last_became_active":"2024-04-13T03:13:36.765748+0000","last_became_peered":"2024-04-13T03:13:36.765748+0000","last_unstale":"2024-04-13T03:13:36.768589+0000","last_undegraded":"2024-04-13T03:13:36.768589+0000","last_fullsized":"2024-04-13T03:13:36.768589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:32.855294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758105+0000","last_change":"2024-04-13T03:13:36.758105+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758105+0000","last_undegraded":"2024-04-13T03:13:36.758105+0000","last_fullsized":"2024-04-13T03:13:36.758105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765901+0000","last_change":"2024-04-13T03:13:36.764900+0000","last_active":"2024-04-13T03:13:36.765901+0000","last_peered":"2024-04-13T03:13:36.765901+0000","last_clean":"2024-04-13T03:13:36.765901+0000","last_became_active":"2024-04-13T03:13:36.764633+0000","last_became_peered":"2024-04-13T03:13:36.764633+0000","last_unstale":"2024-04-13T03:13:36.765901+0000","last_undegraded":"2024-04-13T03:13:36.765901+0000","last_fullsized":"2024-04-13T03:13:36.765901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:29.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757357+0000","last_change":"2024-04-13T03:13:36.757357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757357+0000","last_undegraded":"2024-04-13T03:13:36.757357+0000","last_fullsized":"2024-04-13T03:13:36.757357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772191+0000","last_change":"2024-04-13T03:13:36.770570+0000","last_active":"2024-04-13T03:13:36.772191+0000","last_peered":"2024-04-13T03:13:36.772191+0000","last_clean":"2024-04-13T03:13:36.772191+0000","last_became_active":"2024-04-13T03:13:36.769914+0000","last_became_peered":"2024-04-13T03:13:36.769914+0000","last_unstale":"2024-04-13T03:13:36.772191+0000","last_undegraded":"2024-04-13T03:13:36.772191+0000","last_fullsized":"2024-04-13T03:13:36.772191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:54.641101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753427+0000","last_change":"2024-04-13T03:13:36.753427+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753427+0000","last_undegraded":"2024-04-13T03:13:36.753427+0000","last_fullsized":"2024-04-13T03:13:36.753427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771317+0000","last_change":"2024-04-13T03:13:36.768818+0000","last_active":"2024-04-13T03:13:36.771317+0000","last_peered":"2024-04-13T03:13:36.771317+0000","last_clean":"2024-04-13T03:13:36.771317+0000","last_became_active":"2024-04-13T03:13:36.768492+0000","last_became_peered":"2024-04-13T03:13:36.768492+0000","last_unstale":"2024-04-13T03:13:36.771317+0000","last_undegraded":"2024-04-13T03:13:36.771317+0000","last_fullsized":"2024-04-13T03:13:36.771317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:49.380954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759786+0000","last_change":"2024-04-13T03:13:36.759786+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759786+0000","last_undegraded":"2024-04-13T03:13:36.759786+0000","last_fullsized":"2024-04-13T03:13:36.759786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770868+0000","last_change":"2024-04-13T03:13:36.766877+0000","last_active":"2024-04-13T03:13:36.770868+0000","last_peered":"2024-04-13T03:13:36.770868+0000","last_clean":"2024-04-13T03:13:36.770868+0000","last_became_active":"2024-04-13T03:13:36.765880+0000","last_became_peered":"2024-04-13T03:13:36.765880+0000","last_unstale":"2024-04-13T03:13:36.770868+0000","last_undegraded":"2024-04-13T03:13:36.770868+0000","last_fullsized":"2024-04-13T03:13:36.770868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:39:57.972949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757534+0000","last_change":"2024-04-13T03:13:36.757534+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757534+0000","last_undegraded":"2024-04-13T03:13:36.757534+0000","last_fullsized":"2024-04-13T03:13:36.757534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770312+0000","last_change":"2024-04-13T03:13:36.767371+0000","last_active":"2024-04-13T03:13:36.770312+0000","last_peered":"2024-04-13T03:13:36.770312+0000","last_clean":"2024-04-13T03:13:36.770312+0000","last_became_active":"2024-04-13T03:13:36.767116+0000","last_became_peered":"2024-04-13T03:13:36.767116+0000","last_unstale":"2024-04-13T03:13:36.770312+0000","last_undegraded":"2024-04-13T03:13:36.770312+0000","last_fullsized":"2024-04-13T03:13:36.770312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:05.559185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755043+0000","last_change":"2024-04-13T03:13:36.755043+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755043+0000","last_undegraded":"2024-04-13T03:13:36.755043+0000","last_fullsized":"2024-04-13T03:13:36.755043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770829+0000","last_change":"2024-04-13T03:13:36.767237+0000","last_active":"2024-04-13T03:13:36.770829+0000","last_peered":"2024-04-13T03:13:36.770829+0000","last_clean":"2024-04-13T03:13:36.770829+0000","last_became_active":"2024-04-13T03:13:36.766972+0000","last_became_peered":"2024-04-13T03:13:36.766972+0000","last_unstale":"2024-04-13T03:13:36.770829+0000","last_undegraded":"2024-04-13T03:13:36.770829+0000","last_fullsized":"2024-04-13T03:13:36.770829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:54:40.326165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755519+0000","last_change":"2024-04-13T03:13:36.755519+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755519+0000","last_undegraded":"2024-04-13T03:13:36.755519+0000","last_fullsized":"2024-04-13T03:13:36.755519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768808+0000","last_change":"2024-04-13T03:13:36.766340+0000","last_active":"2024-04-13T03:13:36.768808+0000","last_peered":"2024-04-13T03:13:36.768808+0000","last_clean":"2024-04-13T03:13:36.768808+0000","last_became_active":"2024-04-13T03:13:36.766071+0000","last_became_peered":"2024-04-13T03:13:36.766071+0000","last_unstale":"2024-04-13T03:13:36.768808+0000","last_undegraded":"2024-04-13T03:13:36.768808+0000","last_fullsized":"2024-04-13T03:13:36.768808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761893+0000","last_change":"2024-04-13T03:13:36.761893+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761893+0000","last_undegraded":"2024-04-13T03:13:36.761893+0000","last_fullsized":"2024-04-13T03:13:36.761893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773123+0000","last_change":"2024-04-13T03:13:36.772096+0000","last_active":"2024-04-13T03:13:36.773123+0000","last_peered":"2024-04-13T03:13:36.773123+0000","last_clean":"2024-04-13T03:13:36.773123+0000","last_became_active":"2024-04-13T03:13:36.771833+0000","last_became_peered":"2024-04-13T03:13:36.771833+0000","last_unstale":"2024-04-13T03:13:36.773123+0000","last_undegraded":"2024-04-13T03:13:36.773123+0000","last_fullsized":"2024-04-13T03:13:36.773123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:13:35.264150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.760793+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:13:36.760793+0000","last_peered":"2024-04-13T03:13:36.760793+0000","last_clean":"2024-04-13T03:13:36.760793+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:13:36.760793+0000","last_undegraded":"2024-04-13T03:13:36.760793+0000","last_fullsized":"2024-04-13T03:13:36.760793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":416145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:13:40.746 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:13:40.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.881+0000 7f6de0c60640 1 Processor -- start 2024-04-13T03:13:40.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.881+0000 7f6de0c60640 1 -- start start 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6de0c60640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6ddc15e9b0 0x7f6ddc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6de0c60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6de0c60640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0cfd20 con 0x7f6ddc0b3d60 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6de0c60640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc162af0 con 0x7f6ddc15ed90 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6de0c60640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc162dd0 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6dd9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6dd9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59658/0 (socket says 172.21.15.106:59658) 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6ddad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6ddc0b3d60 0x7f6ddc0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48270/0 (socket says 172.21.15.106:48270) 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.882+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/2012821066 learned_addr learned my addr 172.21.15.106:0/2012821066 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dda59c640 1 --2- 172.21.15.106:0/2012821066 >> v2:172.21.15.114:3300/0 conn(0x7f6ddc15e9b0 0x7f6ddc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/2012821066 >> v2:172.21.15.114:3300/0 conn(0x7f6ddc15e9b0 msgr2=0x7f6ddc0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd9d9b640 1 --2- 172.21.15.106:0/2012821066 >> v2:172.21.15.114:3300/0 conn(0x7f6ddc15e9b0 0x7f6ddc0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/2012821066 >> v1:172.21.15.106:6789/0 conn(0x7f6ddc0b3d60 legacy=0x7f6ddc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd959a640 1 -- 172.21.15.106:0/2012821066 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085686024 0 0) 0x7f6ddc0cfd20 con 0x7f6ddc0b3d60 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/2012821066 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc1630b0 con 0x7f6ddc15ed90 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd9d9b640 1 --2- 172.21.15.106:0/2012821066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc08e4e0 tx=0x7f6dd002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55bafe3d78651b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd959a640 1 -- 172.21.15.106:0/2012821066 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0004230 con 0x7f6ddc15ed90 2024-04-13T03:13:40.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.883+0000 7f6dd959a640 1 -- 172.21.15.106:0/2012821066 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd00043d0 con 0x7f6ddc15ed90 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6dd959a640 1 -- 172.21.15.106:0/2012821066 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd002f840 con 0x7f6ddc15ed90 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 -- 172.21.15.106:0/2012821066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15ed90 msgr2=0x7f6ddc0d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 --2- 172.21.15.106:0/2012821066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc08e4e0 tx=0x7f6dd002ba20 comp rx=0 tx=0).stop 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 -- 172.21.15.106:0/2012821066 shutdown_connections 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 --2- 172.21.15.106:0/2012821066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15ed90 0x7f6ddc0d45f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 --2- 172.21.15.106:0/2012821066 >> v2:172.21.15.114:3300/0 conn(0x7f6ddc15e9b0 0x7f6ddc0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 -- 172.21.15.106:0/2012821066 >> 172.21.15.106:0/2012821066 conn(0x7f6ddc08bf10 msgr2=0x7f6ddc0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 -- 172.21.15.106:0/2012821066 shutdown_connections 2024-04-13T03:13:40.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.884+0000 7f6de0c60640 1 -- 172.21.15.106:0/2012821066 wait complete. 2024-04-13T03:13:40.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.885+0000 7f6de0c60640 1 Processor -- start 2024-04-13T03:13:40.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 -- start start 2024-04-13T03:13:40.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ddc0b3d60 0x7f6ddc154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc155ab0 0x7f6ddc15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc162800 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc162680 con 0x7f6ddc155ab0 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6de0c60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc06eb20 con 0x7f6ddc0b3d60 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59664/0 (socket says 172.21.15.106:59664) 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.886+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/599354544 learned_addr learned my addr 172.21.15.106:0/599354544 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6ddad9d640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc155ab0 0x7f6ddc15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ddc0b3d60 msgr2=0x7f6ddc154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dda59c640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ddc0b3d60 0x7f6ddc154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ddc0b3d60 0x7f6ddc154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc155ab0 msgr2=0x7f6ddc15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:40.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc155ab0 0x7f6ddc15b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 -- 172.21.15.106:0/599354544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc154a10 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dd9d9b640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0000c00 tx=0x7f6dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0047070 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd002fc00 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc154cf0 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0003610 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.887+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1552b0 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.888+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc068a10 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.888+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dd004a070 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.889+0000 7f6dcaffd640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 0x7f6dbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.889+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f6dd00ad950 con 0x7f6ddc15e9b0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.889+0000 7f6dda59c640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 0x7f6dbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:40.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.889+0000 7f6dda59c640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 0x7f6dbc06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc069220 tx=0x7f6dcc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:40.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:40.893+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6dd0033070 con 0x7f6ddc15e9b0 2024-04-13T03:13:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.063+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ddc0d53a0 con 0x7f6dbc06c8f0 2024-04-13T03:13:41.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.069+0000 7f6dcaffd640 1 -- 172.21.15.106:0/599354544 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7f6ddc0d53a0 con 0x7f6dbc06c8f0 2024-04-13T03:13:41.073 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:41.074 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:13:41.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 msgr2=0x7f6dbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 0x7f6dbc06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc069220 tx=0x7f6dcc00b040 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 msgr2=0x7f6ddc152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0000c00 tx=0x7f6dd00046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 shutdown_connections 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6dbc06c8f0 0x7f6dbc06edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc155ab0 0x7f6ddc15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ddc15e9b0 0x7f6ddc152b20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 --2- 172.21.15.106:0/599354544 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ddc0b3d60 0x7f6ddc154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 >> 172.21.15.106:0/599354544 conn(0x7f6ddc08bf10 msgr2=0x7f6ddc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 shutdown_connections 2024-04-13T03:13:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.072+0000 7f6de0c60640 1 -- 172.21.15.106:0/599354544 wait complete. 2024-04-13T03:13:41.084 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-13T03:13:40.417060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228752,"kb_used_data":8412,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754745968,"statfs":{"total":773094113280,"available":772859871232,"internally_reserved":0,"allocated":8613888,"data_stored":4556565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672266"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760106+0000","last_change":"2024-04-13T03:13:36.760106+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760106+0000","last_undegraded":"2024-04-13T03:13:36.760106+0000","last_fullsized":"2024-04-13T03:13:36.760106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755567+0000","last_change":"2024-04-13T03:13:36.755567+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755567+0000","last_undegraded":"2024-04-13T03:13:36.755567+0000","last_fullsized":"2024-04-13T03:13:36.755567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756896+0000","last_change":"2024-04-13T03:13:36.756896+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756896+0000","last_undegraded":"2024-04-13T03:13:36.756896+0000","last_fullsized":"2024-04-13T03:13:36.756896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755051+0000","last_change":"2024-04-13T03:13:36.755051+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755051+0000","last_undegraded":"2024-04-13T03:13:36.755051+0000","last_fullsized":"2024-04-13T03:13:36.755051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758625+0000","last_change":"2024-04-13T03:13:36.758625+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758625+0000","last_undegraded":"2024-04-13T03:13:36.758625+0000","last_fullsized":"2024-04-13T03:13:36.758625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758991+0000","last_change":"2024-04-13T03:13:36.758991+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758991+0000","last_undegraded":"2024-04-13T03:13:36.758991+0000","last_fullsized":"2024-04-13T03:13:36.758991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760023+0000","last_change":"2024-04-13T03:13:36.760023+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760023+0000","last_undegraded":"2024-04-13T03:13:36.760023+0000","last_fullsized":"2024-04-13T03:13:36.760023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758589+0000","last_change":"2024-04-13T03:13:36.758589+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758589+0000","last_undegraded":"2024-04-13T03:13:36.758589+0000","last_fullsized":"2024-04-13T03:13:36.758589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757338+0000","last_change":"2024-04-13T03:13:36.757338+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757338+0000","last_undegraded":"2024-04-13T03:13:36.757338+0000","last_fullsized":"2024-04-13T03:13:36.757338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754743+0000","last_change":"2024-04-13T03:13:36.754743+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754743+0000","last_undegraded":"2024-04-13T03:13:36.754743+0000","last_fullsized":"2024-04-13T03:13:36.754743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753756+0000","last_change":"2024-04-13T03:13:36.753756+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753756+0000","last_undegraded":"2024-04-13T03:13:36.753756+0000","last_fullsized":"2024-04-13T03:13:36.753756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754566+0000","last_change":"2024-04-13T03:13:36.754566+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754566+0000","last_undegraded":"2024-04-13T03:13:36.754566+0000","last_fullsized":"2024-04-13T03:13:36.754566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758260+0000","last_change":"2024-04-13T03:13:36.758260+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758260+0000","last_undegraded":"2024-04-13T03:13:36.758260+0000","last_fullsized":"2024-04-13T03:13:36.758260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758584+0000","last_change":"2024-04-13T03:13:36.758584+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758584+0000","last_undegraded":"2024-04-13T03:13:36.758584+0000","last_fullsized":"2024-04-13T03:13:36.758584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753512+0000","last_change":"2024-04-13T03:13:36.753512+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753512+0000","last_undegraded":"2024-04-13T03:13:36.753512+0000","last_fullsized":"2024-04-13T03:13:36.753512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760411+0000","last_change":"2024-04-13T03:13:36.760411+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760411+0000","last_undegraded":"2024-04-13T03:13:36.760411+0000","last_fullsized":"2024-04-13T03:13:36.760411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759321+0000","last_change":"2024-04-13T03:13:36.759321+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759321+0000","last_undegraded":"2024-04-13T03:13:36.759321+0000","last_fullsized":"2024-04-13T03:13:36.759321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756859+0000","last_change":"2024-04-13T03:13:36.756859+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756859+0000","last_undegraded":"2024-04-13T03:13:36.756859+0000","last_fullsized":"2024-04-13T03:13:36.756859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760997+0000","last_change":"2024-04-13T03:13:36.760997+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760997+0000","last_undegraded":"2024-04-13T03:13:36.760997+0000","last_fullsized":"2024-04-13T03:13:36.760997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758123+0000","last_change":"2024-04-13T03:13:36.758123+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758123+0000","last_undegraded":"2024-04-13T03:13:36.758123+0000","last_fullsized":"2024-04-13T03:13:36.758123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759788+0000","last_change":"2024-04-13T03:13:36.759788+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759788+0000","last_undegraded":"2024-04-13T03:13:36.759788+0000","last_fullsized":"2024-04-13T03:13:36.759788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757376+0000","last_change":"2024-04-13T03:13:36.757376+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757376+0000","last_undegraded":"2024-04-13T03:13:36.757376+0000","last_fullsized":"2024-04-13T03:13:36.757376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758911+0000","last_change":"2024-04-13T03:13:36.758911+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758911+0000","last_undegraded":"2024-04-13T03:13:36.758911+0000","last_fullsized":"2024-04-13T03:13:36.758911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755750+0000","last_change":"2024-04-13T03:13:36.755750+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755750+0000","last_undegraded":"2024-04-13T03:13:36.755750+0000","last_fullsized":"2024-04-13T03:13:36.755750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.752998+0000","last_change":"2024-04-13T03:13:36.752998+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.752998+0000","last_undegraded":"2024-04-13T03:13:36.752998+0000","last_fullsized":"2024-04-13T03:13:36.752998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754262+0000","last_change":"2024-04-13T03:13:36.754262+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754262+0000","last_undegraded":"2024-04-13T03:13:36.754262+0000","last_fullsized":"2024-04-13T03:13:36.754262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.762340+0000","last_change":"2024-04-13T03:13:36.762340+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.762340+0000","last_undegraded":"2024-04-13T03:13:36.762340+0000","last_fullsized":"2024-04-13T03:13:36.762340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757784+0000","last_change":"2024-04-13T03:13:36.757784+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757784+0000","last_undegraded":"2024-04-13T03:13:36.757784+0000","last_fullsized":"2024-04-13T03:13:36.757784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755190+0000","last_change":"2024-04-13T03:13:36.755190+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755190+0000","last_undegraded":"2024-04-13T03:13:36.755190+0000","last_fullsized":"2024-04-13T03:13:36.755190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758905+0000","last_change":"2024-04-13T03:13:36.758905+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758905+0000","last_undegraded":"2024-04-13T03:13:36.758905+0000","last_fullsized":"2024-04-13T03:13:36.758905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757226+0000","last_change":"2024-04-13T03:13:36.757226+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757226+0000","last_undegraded":"2024-04-13T03:13:36.757226+0000","last_fullsized":"2024-04-13T03:13:36.757226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759924+0000","last_change":"2024-04-13T03:13:36.759924+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759924+0000","last_undegraded":"2024-04-13T03:13:36.759924+0000","last_fullsized":"2024-04-13T03:13:36.759924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756357+0000","last_change":"2024-04-13T03:13:36.756357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756357+0000","last_undegraded":"2024-04-13T03:13:36.756357+0000","last_fullsized":"2024-04-13T03:13:36.756357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770054+0000","last_change":"2024-04-13T03:13:36.767960+0000","last_active":"2024-04-13T03:13:36.770054+0000","last_peered":"2024-04-13T03:13:36.770054+0000","last_clean":"2024-04-13T03:13:36.770054+0000","last_became_active":"2024-04-13T03:13:36.767704+0000","last_became_peered":"2024-04-13T03:13:36.767704+0000","last_unstale":"2024-04-13T03:13:36.770054+0000","last_undegraded":"2024-04-13T03:13:36.770054+0000","last_fullsized":"2024-04-13T03:13:36.770054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:36:14.758876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753075+0000","last_change":"2024-04-13T03:13:36.753075+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753075+0000","last_undegraded":"2024-04-13T03:13:36.753075+0000","last_fullsized":"2024-04-13T03:13:36.753075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771269+0000","last_change":"2024-04-13T03:13:36.769252+0000","last_active":"2024-04-13T03:13:36.771269+0000","last_peered":"2024-04-13T03:13:36.771269+0000","last_clean":"2024-04-13T03:13:36.771269+0000","last_became_active":"2024-04-13T03:13:36.768970+0000","last_became_peered":"2024-04-13T03:13:36.768970+0000","last_unstale":"2024-04-13T03:13:36.771269+0000","last_undegraded":"2024-04-13T03:13:36.771269+0000","last_fullsized":"2024-04-13T03:13:36.771269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:49.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758502+0000","last_change":"2024-04-13T03:13:36.758502+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758502+0000","last_undegraded":"2024-04-13T03:13:36.758502+0000","last_fullsized":"2024-04-13T03:13:36.758502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770243+0000","last_change":"2024-04-13T03:13:36.767892+0000","last_active":"2024-04-13T03:13:36.770243+0000","last_peered":"2024-04-13T03:13:36.770243+0000","last_clean":"2024-04-13T03:13:36.770243+0000","last_became_active":"2024-04-13T03:13:36.767618+0000","last_became_peered":"2024-04-13T03:13:36.767618+0000","last_unstale":"2024-04-13T03:13:36.770243+0000","last_undegraded":"2024-04-13T03:13:36.770243+0000","last_fullsized":"2024-04-13T03:13:36.770243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:43.589502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758721+0000","last_change":"2024-04-13T03:13:36.758721+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758721+0000","last_undegraded":"2024-04-13T03:13:36.758721+0000","last_fullsized":"2024-04-13T03:13:36.758721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772188+0000","last_change":"2024-04-13T03:13:36.769804+0000","last_active":"2024-04-13T03:13:36.772188+0000","last_peered":"2024-04-13T03:13:36.772188+0000","last_clean":"2024-04-13T03:13:36.772188+0000","last_became_active":"2024-04-13T03:13:36.769493+0000","last_became_peered":"2024-04-13T03:13:36.769493+0000","last_unstale":"2024-04-13T03:13:36.772188+0000","last_undegraded":"2024-04-13T03:13:36.772188+0000","last_fullsized":"2024-04-13T03:13:36.772188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753617+0000","last_change":"2024-04-13T03:13:36.753617+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753617+0000","last_undegraded":"2024-04-13T03:13:36.753617+0000","last_fullsized":"2024-04-13T03:13:36.753617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771158+0000","last_change":"2024-04-13T03:13:36.768909+0000","last_active":"2024-04-13T03:13:36.771158+0000","last_peered":"2024-04-13T03:13:36.771158+0000","last_clean":"2024-04-13T03:13:36.771158+0000","last_became_active":"2024-04-13T03:13:36.768554+0000","last_became_peered":"2024-04-13T03:13:36.768554+0000","last_unstale":"2024-04-13T03:13:36.771158+0000","last_undegraded":"2024-04-13T03:13:36.771158+0000","last_fullsized":"2024-04-13T03:13:36.771158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:56.727538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753296+0000","last_change":"2024-04-13T03:13:36.753296+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753296+0000","last_undegraded":"2024-04-13T03:13:36.753296+0000","last_fullsized":"2024-04-13T03:13:36.753296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770360+0000","last_change":"2024-04-13T03:13:36.767672+0000","last_active":"2024-04-13T03:13:36.770360+0000","last_peered":"2024-04-13T03:13:36.770360+0000","last_clean":"2024-04-13T03:13:36.770360+0000","last_became_active":"2024-04-13T03:13:36.767312+0000","last_became_peered":"2024-04-13T03:13:36.767312+0000","last_unstale":"2024-04-13T03:13:36.770360+0000","last_undegraded":"2024-04-13T03:13:36.770360+0000","last_fullsized":"2024-04-13T03:13:36.770360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:51.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757834+0000","last_change":"2024-04-13T03:13:36.757834+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757834+0000","last_undegraded":"2024-04-13T03:13:36.757834+0000","last_fullsized":"2024-04-13T03:13:36.757834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769561+0000","last_change":"2024-04-13T03:13:36.765735+0000","last_active":"2024-04-13T03:13:36.769561+0000","last_peered":"2024-04-13T03:13:36.769561+0000","last_clean":"2024-04-13T03:13:36.769561+0000","last_became_active":"2024-04-13T03:13:36.765485+0000","last_became_peered":"2024-04-13T03:13:36.765485+0000","last_unstale":"2024-04-13T03:13:36.769561+0000","last_undegraded":"2024-04-13T03:13:36.769561+0000","last_fullsized":"2024-04-13T03:13:36.769561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:56.813106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758918+0000","last_change":"2024-04-13T03:13:36.758918+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758918+0000","last_undegraded":"2024-04-13T03:13:36.758918+0000","last_fullsized":"2024-04-13T03:13:36.758918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770068+0000","last_change":"2024-04-13T03:13:36.769186+0000","last_active":"2024-04-13T03:13:36.770068+0000","last_peered":"2024-04-13T03:13:36.770068+0000","last_clean":"2024-04-13T03:13:36.770068+0000","last_became_active":"2024-04-13T03:13:36.768951+0000","last_became_peered":"2024-04-13T03:13:36.768951+0000","last_unstale":"2024-04-13T03:13:36.770068+0000","last_undegraded":"2024-04-13T03:13:36.770068+0000","last_fullsized":"2024-04-13T03:13:36.770068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:16:46.808010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753081+0000","last_change":"2024-04-13T03:13:36.753081+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753081+0000","last_undegraded":"2024-04-13T03:13:36.753081+0000","last_fullsized":"2024-04-13T03:13:36.753081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772133+0000","last_change":"2024-04-13T03:13:36.769948+0000","last_active":"2024-04-13T03:13:36.772133+0000","last_peered":"2024-04-13T03:13:36.772133+0000","last_clean":"2024-04-13T03:13:36.772133+0000","last_became_active":"2024-04-13T03:13:36.769546+0000","last_became_peered":"2024-04-13T03:13:36.769546+0000","last_unstale":"2024-04-13T03:13:36.772133+0000","last_undegraded":"2024-04-13T03:13:36.772133+0000","last_fullsized":"2024-04-13T03:13:36.772133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:46.141793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754015+0000","last_change":"2024-04-13T03:13:36.754015+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754015+0000","last_undegraded":"2024-04-13T03:13:36.754015+0000","last_fullsized":"2024-04-13T03:13:36.754015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769696+0000","last_change":"2024-04-13T03:13:36.766240+0000","last_active":"2024-04-13T03:13:36.769696+0000","last_peered":"2024-04-13T03:13:36.769696+0000","last_clean":"2024-04-13T03:13:36.769696+0000","last_became_active":"2024-04-13T03:13:36.763111+0000","last_became_peered":"2024-04-13T03:13:36.763111+0000","last_unstale":"2024-04-13T03:13:36.769696+0000","last_undegraded":"2024-04-13T03:13:36.769696+0000","last_fullsized":"2024-04-13T03:13:36.769696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:28.008984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758157+0000","last_change":"2024-04-13T03:13:36.758157+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758157+0000","last_undegraded":"2024-04-13T03:13:36.758157+0000","last_fullsized":"2024-04-13T03:13:36.758157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770017+0000","last_change":"2024-04-13T03:13:36.767663+0000","last_active":"2024-04-13T03:13:36.770017+0000","last_peered":"2024-04-13T03:13:36.770017+0000","last_clean":"2024-04-13T03:13:36.770017+0000","last_became_active":"2024-04-13T03:13:36.767407+0000","last_became_peered":"2024-04-13T03:13:36.767407+0000","last_unstale":"2024-04-13T03:13:36.770017+0000","last_undegraded":"2024-04-13T03:13:36.770017+0000","last_fullsized":"2024-04-13T03:13:36.770017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:48.797655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755936+0000","last_change":"2024-04-13T03:13:36.755936+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755936+0000","last_undegraded":"2024-04-13T03:13:36.755936+0000","last_fullsized":"2024-04-13T03:13:36.755936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772167+0000","last_change":"2024-04-13T03:13:36.768069+0000","last_active":"2024-04-13T03:13:36.772167+0000","last_peered":"2024-04-13T03:13:36.772167+0000","last_clean":"2024-04-13T03:13:36.772167+0000","last_became_active":"2024-04-13T03:13:36.767790+0000","last_became_peered":"2024-04-13T03:13:36.767790+0000","last_unstale":"2024-04-13T03:13:36.772167+0000","last_undegraded":"2024-04-13T03:13:36.772167+0000","last_fullsized":"2024-04-13T03:13:36.772167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:32:59.935067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753985+0000","last_change":"2024-04-13T03:13:36.753985+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753985+0000","last_undegraded":"2024-04-13T03:13:36.753985+0000","last_fullsized":"2024-04-13T03:13:36.753985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769647+0000","last_change":"2024-04-13T03:13:36.766629+0000","last_active":"2024-04-13T03:13:36.769647+0000","last_peered":"2024-04-13T03:13:36.769647+0000","last_clean":"2024-04-13T03:13:36.769647+0000","last_became_active":"2024-04-13T03:13:36.766361+0000","last_became_peered":"2024-04-13T03:13:36.766361+0000","last_unstale":"2024-04-13T03:13:36.769647+0000","last_undegraded":"2024-04-13T03:13:36.769647+0000","last_fullsized":"2024-04-13T03:13:36.769647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:44:10.176246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758070+0000","last_change":"2024-04-13T03:13:36.758070+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758070+0000","last_undegraded":"2024-04-13T03:13:36.758070+0000","last_fullsized":"2024-04-13T03:13:36.758070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772234+0000","last_change":"2024-04-13T03:13:36.770589+0000","last_active":"2024-04-13T03:13:36.772234+0000","last_peered":"2024-04-13T03:13:36.772234+0000","last_clean":"2024-04-13T03:13:36.772234+0000","last_became_active":"2024-04-13T03:13:36.770281+0000","last_became_peered":"2024-04-13T03:13:36.770281+0000","last_unstale":"2024-04-13T03:13:36.772234+0000","last_undegraded":"2024-04-13T03:13:36.772234+0000","last_fullsized":"2024-04-13T03:13:36.772234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:53.535105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758508+0000","last_change":"2024-04-13T03:13:36.758508+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758508+0000","last_undegraded":"2024-04-13T03:13:36.758508+0000","last_fullsized":"2024-04-13T03:13:36.758508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770078+0000","last_change":"2024-04-13T03:13:36.768599+0000","last_active":"2024-04-13T03:13:36.770078+0000","last_peered":"2024-04-13T03:13:36.770078+0000","last_clean":"2024-04-13T03:13:36.770078+0000","last_became_active":"2024-04-13T03:13:36.768343+0000","last_became_peered":"2024-04-13T03:13:36.768343+0000","last_unstale":"2024-04-13T03:13:36.770078+0000","last_undegraded":"2024-04-13T03:13:36.770078+0000","last_fullsized":"2024-04-13T03:13:36.770078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:22.711857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758484+0000","last_change":"2024-04-13T03:13:36.758484+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758484+0000","last_undegraded":"2024-04-13T03:13:36.758484+0000","last_fullsized":"2024-04-13T03:13:36.758484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765921+0000","last_change":"2024-04-13T03:13:36.764197+0000","last_active":"2024-04-13T03:13:36.765921+0000","last_peered":"2024-04-13T03:13:36.765921+0000","last_clean":"2024-04-13T03:13:36.765921+0000","last_became_active":"2024-04-13T03:13:36.763920+0000","last_became_peered":"2024-04-13T03:13:36.763920+0000","last_unstale":"2024-04-13T03:13:36.765921+0000","last_undegraded":"2024-04-13T03:13:36.765921+0000","last_fullsized":"2024-04-13T03:13:36.765921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:49.280931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761836+0000","last_change":"2024-04-13T03:13:36.761836+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761836+0000","last_undegraded":"2024-04-13T03:13:36.761836+0000","last_fullsized":"2024-04-13T03:13:36.761836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772173+0000","last_change":"2024-04-13T03:13:36.770260+0000","last_active":"2024-04-13T03:13:36.772173+0000","last_peered":"2024-04-13T03:13:36.772173+0000","last_clean":"2024-04-13T03:13:36.772173+0000","last_became_active":"2024-04-13T03:13:36.769888+0000","last_became_peered":"2024-04-13T03:13:36.769888+0000","last_unstale":"2024-04-13T03:13:36.772173+0000","last_undegraded":"2024-04-13T03:13:36.772173+0000","last_fullsized":"2024-04-13T03:13:36.772173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:39.653567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760574+0000","last_change":"2024-04-13T03:13:36.760574+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760574+0000","last_undegraded":"2024-04-13T03:13:36.760574+0000","last_fullsized":"2024-04-13T03:13:36.760574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770965+0000","last_change":"2024-04-13T03:13:36.765493+0000","last_active":"2024-04-13T03:13:36.770965+0000","last_peered":"2024-04-13T03:13:36.770965+0000","last_clean":"2024-04-13T03:13:36.770965+0000","last_became_active":"2024-04-13T03:13:36.765174+0000","last_became_peered":"2024-04-13T03:13:36.765174+0000","last_unstale":"2024-04-13T03:13:36.770965+0000","last_undegraded":"2024-04-13T03:13:36.770965+0000","last_fullsized":"2024-04-13T03:13:36.770965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757837+0000","last_change":"2024-04-13T03:13:36.757837+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757837+0000","last_undegraded":"2024-04-13T03:13:36.757837+0000","last_fullsized":"2024-04-13T03:13:36.757837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770670+0000","last_change":"2024-04-13T03:13:36.769841+0000","last_active":"2024-04-13T03:13:36.770670+0000","last_peered":"2024-04-13T03:13:36.770670+0000","last_clean":"2024-04-13T03:13:36.770670+0000","last_became_active":"2024-04-13T03:13:36.769598+0000","last_became_peered":"2024-04-13T03:13:36.769598+0000","last_unstale":"2024-04-13T03:13:36.770670+0000","last_undegraded":"2024-04-13T03:13:36.770670+0000","last_fullsized":"2024-04-13T03:13:36.770670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:11:43.265755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759904+0000","last_change":"2024-04-13T03:13:36.759904+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759904+0000","last_undegraded":"2024-04-13T03:13:36.759904+0000","last_fullsized":"2024-04-13T03:13:36.759904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767378+0000","last_change":"2024-04-13T03:13:36.765265+0000","last_active":"2024-04-13T03:13:36.767378+0000","last_peered":"2024-04-13T03:13:36.767378+0000","last_clean":"2024-04-13T03:13:36.767378+0000","last_became_active":"2024-04-13T03:13:36.764966+0000","last_became_peered":"2024-04-13T03:13:36.764966+0000","last_unstale":"2024-04-13T03:13:36.767378+0000","last_undegraded":"2024-04-13T03:13:36.767378+0000","last_fullsized":"2024-04-13T03:13:36.767378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:36.244015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753237+0000","last_change":"2024-04-13T03:13:36.753237+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753237+0000","last_undegraded":"2024-04-13T03:13:36.753237+0000","last_fullsized":"2024-04-13T03:13:36.753237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767948+0000","last_change":"2024-04-13T03:13:36.765596+0000","last_active":"2024-04-13T03:13:36.767948+0000","last_peered":"2024-04-13T03:13:36.767948+0000","last_clean":"2024-04-13T03:13:36.767948+0000","last_became_active":"2024-04-13T03:13:36.765330+0000","last_became_peered":"2024-04-13T03:13:36.765330+0000","last_unstale":"2024-04-13T03:13:36.767948+0000","last_undegraded":"2024-04-13T03:13:36.767948+0000","last_fullsized":"2024-04-13T03:13:36.767948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:02.126210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753008+0000","last_change":"2024-04-13T03:13:36.753008+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753008+0000","last_undegraded":"2024-04-13T03:13:36.753008+0000","last_fullsized":"2024-04-13T03:13:36.753008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773139+0000","last_change":"2024-04-13T03:13:36.772152+0000","last_active":"2024-04-13T03:13:36.773139+0000","last_peered":"2024-04-13T03:13:36.773139+0000","last_clean":"2024-04-13T03:13:36.773139+0000","last_became_active":"2024-04-13T03:13:36.771816+0000","last_became_peered":"2024-04-13T03:13:36.771816+0000","last_unstale":"2024-04-13T03:13:36.773139+0000","last_undegraded":"2024-04-13T03:13:36.773139+0000","last_fullsized":"2024-04-13T03:13:36.773139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:51:05.902931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757953+0000","last_change":"2024-04-13T03:13:36.757953+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757953+0000","last_undegraded":"2024-04-13T03:13:36.757953+0000","last_fullsized":"2024-04-13T03:13:36.757953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773107+0000","last_change":"2024-04-13T03:13:36.772084+0000","last_active":"2024-04-13T03:13:36.773107+0000","last_peered":"2024-04-13T03:13:36.773107+0000","last_clean":"2024-04-13T03:13:36.773107+0000","last_became_active":"2024-04-13T03:13:36.771800+0000","last_became_peered":"2024-04-13T03:13:36.771800+0000","last_unstale":"2024-04-13T03:13:36.773107+0000","last_undegraded":"2024-04-13T03:13:36.773107+0000","last_fullsized":"2024-04-13T03:13:36.773107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:02.961934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757223+0000","last_change":"2024-04-13T03:13:36.757223+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757223+0000","last_undegraded":"2024-04-13T03:13:36.757223+0000","last_fullsized":"2024-04-13T03:13:36.757223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768589+0000","last_change":"2024-04-13T03:13:36.766089+0000","last_active":"2024-04-13T03:13:36.768589+0000","last_peered":"2024-04-13T03:13:36.768589+0000","last_clean":"2024-04-13T03:13:36.768589+0000","last_became_active":"2024-04-13T03:13:36.765748+0000","last_became_peered":"2024-04-13T03:13:36.765748+0000","last_unstale":"2024-04-13T03:13:36.768589+0000","last_undegraded":"2024-04-13T03:13:36.768589+0000","last_fullsized":"2024-04-13T03:13:36.768589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:32.855294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758105+0000","last_change":"2024-04-13T03:13:36.758105+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758105+0000","last_undegraded":"2024-04-13T03:13:36.758105+0000","last_fullsized":"2024-04-13T03:13:36.758105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765901+0000","last_change":"2024-04-13T03:13:36.764900+0000","last_active":"2024-04-13T03:13:36.765901+0000","last_peered":"2024-04-13T03:13:36.765901+0000","last_clean":"2024-04-13T03:13:36.765901+0000","last_became_active":"2024-04-13T03:13:36.764633+0000","last_became_peered":"2024-04-13T03:13:36.764633+0000","last_unstale":"2024-04-13T03:13:36.765901+0000","last_undegraded":"2024-04-13T03:13:36.765901+0000","last_fullsized":"2024-04-13T03:13:36.765901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:29.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757357+0000","last_change":"2024-04-13T03:13:36.757357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757357+0000","last_undegraded":"2024-04-13T03:13:36.757357+0000","last_fullsized":"2024-04-13T03:13:36.757357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772191+0000","last_change":"2024-04-13T03:13:36.770570+0000","last_active":"2024-04-13T03:13:36.772191+0000","last_peered":"2024-04-13T03:13:36.772191+0000","last_clean":"2024-04-13T03:13:36.772191+0000","last_became_active":"2024-04-13T03:13:36.769914+0000","last_became_peered":"2024-04-13T03:13:36.769914+0000","last_unstale":"2024-04-13T03:13:36.772191+0000","last_undegraded":"2024-04-13T03:13:36.772191+0000","last_fullsized":"2024-04-13T03:13:36.772191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:54.641101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753427+0000","last_change":"2024-04-13T03:13:36.753427+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753427+0000","last_undegraded":"2024-04-13T03:13:36.753427+0000","last_fullsized":"2024-04-13T03:13:36.753427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771317+0000","last_change":"2024-04-13T03:13:36.768818+0000","last_active":"2024-04-13T03:13:36.771317+0000","last_peered":"2024-04-13T03:13:36.771317+0000","last_clean":"2024-04-13T03:13:36.771317+0000","last_became_active":"2024-04-13T03:13:36.768492+0000","last_became_peered":"2024-04-13T03:13:36.768492+0000","last_unstale":"2024-04-13T03:13:36.771317+0000","last_undegraded":"2024-04-13T03:13:36.771317+0000","last_fullsized":"2024-04-13T03:13:36.771317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:49.380954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759786+0000","last_change":"2024-04-13T03:13:36.759786+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759786+0000","last_undegraded":"2024-04-13T03:13:36.759786+0000","last_fullsized":"2024-04-13T03:13:36.759786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770868+0000","last_change":"2024-04-13T03:13:36.766877+0000","last_active":"2024-04-13T03:13:36.770868+0000","last_peered":"2024-04-13T03:13:36.770868+0000","last_clean":"2024-04-13T03:13:36.770868+0000","last_became_active":"2024-04-13T03:13:36.765880+0000","last_became_peered":"2024-04-13T03:13:36.765880+0000","last_unstale":"2024-04-13T03:13:36.770868+0000","last_undegraded":"2024-04-13T03:13:36.770868+0000","last_fullsized":"2024-04-13T03:13:36.770868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:39:57.972949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757534+0000","last_change":"2024-04-13T03:13:36.757534+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757534+0000","last_undegraded":"2024-04-13T03:13:36.757534+0000","last_fullsized":"2024-04-13T03:13:36.757534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770312+0000","last_change":"2024-04-13T03:13:36.767371+0000","last_active":"2024-04-13T03:13:36.770312+0000","last_peered":"2024-04-13T03:13:36.770312+0000","last_clean":"2024-04-13T03:13:36.770312+0000","last_became_active":"2024-04-13T03:13:36.767116+0000","last_became_peered":"2024-04-13T03:13:36.767116+0000","last_unstale":"2024-04-13T03:13:36.770312+0000","last_undegraded":"2024-04-13T03:13:36.770312+0000","last_fullsized":"2024-04-13T03:13:36.770312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:05.559185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755043+0000","last_change":"2024-04-13T03:13:36.755043+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755043+0000","last_undegraded":"2024-04-13T03:13:36.755043+0000","last_fullsized":"2024-04-13T03:13:36.755043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770829+0000","last_change":"2024-04-13T03:13:36.767237+0000","last_active":"2024-04-13T03:13:36.770829+0000","last_peered":"2024-04-13T03:13:36.770829+0000","last_clean":"2024-04-13T03:13:36.770829+0000","last_became_active":"2024-04-13T03:13:36.766972+0000","last_became_peered":"2024-04-13T03:13:36.766972+0000","last_unstale":"2024-04-13T03:13:36.770829+0000","last_undegraded":"2024-04-13T03:13:36.770829+0000","last_fullsized":"2024-04-13T03:13:36.770829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:54:40.326165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755519+0000","last_change":"2024-04-13T03:13:36.755519+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755519+0000","last_undegraded":"2024-04-13T03:13:36.755519+0000","last_fullsized":"2024-04-13T03:13:36.755519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768808+0000","last_change":"2024-04-13T03:13:36.766340+0000","last_active":"2024-04-13T03:13:36.768808+0000","last_peered":"2024-04-13T03:13:36.768808+0000","last_clean":"2024-04-13T03:13:36.768808+0000","last_became_active":"2024-04-13T03:13:36.766071+0000","last_became_peered":"2024-04-13T03:13:36.766071+0000","last_unstale":"2024-04-13T03:13:36.768808+0000","last_undegraded":"2024-04-13T03:13:36.768808+0000","last_fullsized":"2024-04-13T03:13:36.768808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761893+0000","last_change":"2024-04-13T03:13:36.761893+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761893+0000","last_undegraded":"2024-04-13T03:13:36.761893+0000","last_fullsized":"2024-04-13T03:13:36.761893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773123+0000","last_change":"2024-04-13T03:13:36.772096+0000","last_active":"2024-04-13T03:13:36.773123+0000","last_peered":"2024-04-13T03:13:36.773123+0000","last_clean":"2024-04-13T03:13:36.773123+0000","last_became_active":"2024-04-13T03:13:36.771833+0000","last_became_peered":"2024-04-13T03:13:36.771833+0000","last_unstale":"2024-04-13T03:13:36.773123+0000","last_undegraded":"2024-04-13T03:13:36.773123+0000","last_fullsized":"2024-04-13T03:13:36.773123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:13:35.264150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.760793+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:13:36.760793+0000","last_peered":"2024-04-13T03:13:36.760793+0000","last_clean":"2024-04-13T03:13:36.760793+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:13:36.760793+0000","last_undegraded":"2024-04-13T03:13:36.760793+0000","last_fullsized":"2024-04-13T03:13:36.760793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":416145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:13:41.087 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:13:41.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.224+0000 7fb1b215e640 1 Processor -- start 2024-04-13T03:13:41.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.224+0000 7fb1b215e640 1 -- start start 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1b215e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 0x7fb1ac160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1b215e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1ac06e480 0x7fb1ac15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1b215e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1ac15d690 con 0x7fb1ac15cc10 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1b215e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ac15d970 con 0x7fb1ac06e480 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1b215e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac160410 con 0x7fb1ac156b50 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1aaffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb1ac15cc10 0x7fb1ac06dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48278/0 (socket says 172.21.15.106:48278) 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1aaffd640 1 -- 172.21.15.106:0/810100964 learned_addr learned my addr 172.21.15.106:0/810100964 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1abfff640 1 --2- 172.21.15.106:0/810100964 >> v2:172.21.15.106:3300/0 conn(0x7fb1ac06e480 0x7fb1ac15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1aa7fc640 1 -- 172.21.15.106:0/810100964 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492967038 0 0) 0x7fb1ac15d690 con 0x7fb1ac15cc10 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1aa7fc640 1 -- 172.21.15.106:0/810100964 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb194003520 con 0x7fb1ac15cc10 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 0x7fb1ac160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1abfff640 1 -- 172.21.15.106:0/810100964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 msgr2=0x7fb1ac160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1abfff640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 0x7fb1ac160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1abfff640 1 -- 172.21.15.106:0/810100964 >> v1:172.21.15.106:6789/0 conn(0x7fb1ac15cc10 legacy=0x7fb1ac06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1abfff640 1 -- 172.21.15.106:0/810100964 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac1606f0 con 0x7fb1ac06e480 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.225+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 0x7fb1ac160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.226+0000 7fb1abfff640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac06e480 0x7fb1ac15d050 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009d90 tx=0x7fb1a402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9f1986a40f9237 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.226+0000 7fb1aa7fc640 1 -- 172.21.15.106:0/810100964 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4039070 con 0x7fb1ac06e480 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.226+0000 7fb1aa7fc640 1 -- 172.21.15.106:0/810100964 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a402cc50 con 0x7fb1ac06e480 2024-04-13T03:13:41.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.226+0000 7fb1aa7fc640 1 -- 172.21.15.106:0/810100964 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4034440 con 0x7fb1ac06e480 2024-04-13T03:13:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 -- 172.21.15.106:0/810100964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac06e480 msgr2=0x7fb1ac15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac06e480 0x7fb1ac15d050 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009d90 tx=0x7fb1a402bea0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 -- 172.21.15.106:0/810100964 shutdown_connections 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac06e480 0x7fb1ac15d050 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 --2- 172.21.15.106:0/810100964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac156b50 0x7fb1ac160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 -- 172.21.15.106:0/810100964 >> 172.21.15.106:0/810100964 conn(0x7fb1ac0b1f10 msgr2=0x7fb1ac0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 -- 172.21.15.106:0/810100964 shutdown_connections 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.227+0000 7fb1b215e640 1 -- 172.21.15.106:0/810100964 wait complete. 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.228+0000 7fb1b215e640 1 Processor -- start 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.228+0000 7fb1b215e640 1 -- start start 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 0x7fb1ac0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 0x7fb1ac156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac0afc20 con 0x7fb1ac15cc10 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac0afaa0 con 0x7fb1ac156b50 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1b215e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac06bcb0 con 0x7fb1ac06e480 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59672/0 (socket says 172.21.15.106:59672) 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 -- 172.21.15.106:0/2915582271 learned_addr learned my addr 172.21.15.106:0/2915582271 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1aaffd640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 0x7fb1ac156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 0x7fb1ac0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 -- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 msgr2=0x7fb1ac0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 0x7fb1ac0c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 -- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 msgr2=0x7fb1ac156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 0x7fb1ac156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 -- 172.21.15.106:0/2915582271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac15b990 con 0x7fb1ac15cc10 2024-04-13T03:13:41.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1aaffd640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 0x7fb1ac156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 0x7fb1ac0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.229+0000 7fb1abfff640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4037040 tx=0x7fb1a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.230+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4042030 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.230+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a403a040 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.230+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4039070 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.230+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac15bc70 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.230+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac15c230 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.231+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac068a10 con 0x7fb1ac15cc10 2024-04-13T03:13:41.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.231+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1a4045070 con 0x7fb1ac15cc10 2024-04-13T03:13:41.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.231+0000 7fb193fff640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 0x7fb19806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.232+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fb1a40adaa0 con 0x7fb1ac15cc10 2024-04-13T03:13:41.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.232+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 0x7fb19806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.232+0000 7fb1ab7fe640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 0x7fb19806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00aa30 tx=0x7fb19c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.236+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1a404c070 con 0x7fb1ac15cc10 2024-04-13T03:13:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.405+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1ac06ba90 con 0x7fb19806c8f0 2024-04-13T03:13:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.411+0000 7fb193fff640 1 -- 172.21.15.106:0/2915582271 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7fb1ac06ba90 con 0x7fb19806c8f0 2024-04-13T03:13:41.414 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 msgr2=0x7fb19806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 0x7fb19806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00aa30 tx=0x7fb19c008040 comp rx=0 tx=0).stop 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 msgr2=0x7fb1ac0c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4037040 tx=0x7fb1a4004a40 comp rx=0 tx=0).stop 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 shutdown_connections 2024-04-13T03:13:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19806c8f0 0x7fb19806edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1ac15cc10 0x7fb1ac0c7450 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1ac156b50 0x7fb1ac156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 --2- 172.21.15.106:0/2915582271 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1ac06e480 0x7fb1ac0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 >> 172.21.15.106:0/2915582271 conn(0x7fb1ac0b1f10 msgr2=0x7fb1ac0afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 shutdown_connections 2024-04-13T03:13:41.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.414+0000 7fb1b215e640 1 -- 172.21.15.106:0/2915582271 wait complete. 2024-04-13T03:13:41.425 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-13T03:13:40.417060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228752,"kb_used_data":8412,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754745968,"statfs":{"total":773094113280,"available":772859871232,"internally_reserved":0,"allocated":8613888,"data_stored":4556565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672266"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760106+0000","last_change":"2024-04-13T03:13:36.760106+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760106+0000","last_undegraded":"2024-04-13T03:13:36.760106+0000","last_fullsized":"2024-04-13T03:13:36.760106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755567+0000","last_change":"2024-04-13T03:13:36.755567+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755567+0000","last_undegraded":"2024-04-13T03:13:36.755567+0000","last_fullsized":"2024-04-13T03:13:36.755567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756896+0000","last_change":"2024-04-13T03:13:36.756896+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756896+0000","last_undegraded":"2024-04-13T03:13:36.756896+0000","last_fullsized":"2024-04-13T03:13:36.756896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755051+0000","last_change":"2024-04-13T03:13:36.755051+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755051+0000","last_undegraded":"2024-04-13T03:13:36.755051+0000","last_fullsized":"2024-04-13T03:13:36.755051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758625+0000","last_change":"2024-04-13T03:13:36.758625+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758625+0000","last_undegraded":"2024-04-13T03:13:36.758625+0000","last_fullsized":"2024-04-13T03:13:36.758625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758991+0000","last_change":"2024-04-13T03:13:36.758991+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758991+0000","last_undegraded":"2024-04-13T03:13:36.758991+0000","last_fullsized":"2024-04-13T03:13:36.758991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760023+0000","last_change":"2024-04-13T03:13:36.760023+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760023+0000","last_undegraded":"2024-04-13T03:13:36.760023+0000","last_fullsized":"2024-04-13T03:13:36.760023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758589+0000","last_change":"2024-04-13T03:13:36.758589+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758589+0000","last_undegraded":"2024-04-13T03:13:36.758589+0000","last_fullsized":"2024-04-13T03:13:36.758589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757338+0000","last_change":"2024-04-13T03:13:36.757338+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757338+0000","last_undegraded":"2024-04-13T03:13:36.757338+0000","last_fullsized":"2024-04-13T03:13:36.757338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754743+0000","last_change":"2024-04-13T03:13:36.754743+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754743+0000","last_undegraded":"2024-04-13T03:13:36.754743+0000","last_fullsized":"2024-04-13T03:13:36.754743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753756+0000","last_change":"2024-04-13T03:13:36.753756+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753756+0000","last_undegraded":"2024-04-13T03:13:36.753756+0000","last_fullsized":"2024-04-13T03:13:36.753756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754566+0000","last_change":"2024-04-13T03:13:36.754566+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754566+0000","last_undegraded":"2024-04-13T03:13:36.754566+0000","last_fullsized":"2024-04-13T03:13:36.754566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758260+0000","last_change":"2024-04-13T03:13:36.758260+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758260+0000","last_undegraded":"2024-04-13T03:13:36.758260+0000","last_fullsized":"2024-04-13T03:13:36.758260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758584+0000","last_change":"2024-04-13T03:13:36.758584+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758584+0000","last_undegraded":"2024-04-13T03:13:36.758584+0000","last_fullsized":"2024-04-13T03:13:36.758584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753512+0000","last_change":"2024-04-13T03:13:36.753512+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753512+0000","last_undegraded":"2024-04-13T03:13:36.753512+0000","last_fullsized":"2024-04-13T03:13:36.753512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760411+0000","last_change":"2024-04-13T03:13:36.760411+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760411+0000","last_undegraded":"2024-04-13T03:13:36.760411+0000","last_fullsized":"2024-04-13T03:13:36.760411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759321+0000","last_change":"2024-04-13T03:13:36.759321+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759321+0000","last_undegraded":"2024-04-13T03:13:36.759321+0000","last_fullsized":"2024-04-13T03:13:36.759321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756859+0000","last_change":"2024-04-13T03:13:36.756859+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756859+0000","last_undegraded":"2024-04-13T03:13:36.756859+0000","last_fullsized":"2024-04-13T03:13:36.756859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760997+0000","last_change":"2024-04-13T03:13:36.760997+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760997+0000","last_undegraded":"2024-04-13T03:13:36.760997+0000","last_fullsized":"2024-04-13T03:13:36.760997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758123+0000","last_change":"2024-04-13T03:13:36.758123+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758123+0000","last_undegraded":"2024-04-13T03:13:36.758123+0000","last_fullsized":"2024-04-13T03:13:36.758123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759788+0000","last_change":"2024-04-13T03:13:36.759788+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759788+0000","last_undegraded":"2024-04-13T03:13:36.759788+0000","last_fullsized":"2024-04-13T03:13:36.759788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757376+0000","last_change":"2024-04-13T03:13:36.757376+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757376+0000","last_undegraded":"2024-04-13T03:13:36.757376+0000","last_fullsized":"2024-04-13T03:13:36.757376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758911+0000","last_change":"2024-04-13T03:13:36.758911+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758911+0000","last_undegraded":"2024-04-13T03:13:36.758911+0000","last_fullsized":"2024-04-13T03:13:36.758911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755750+0000","last_change":"2024-04-13T03:13:36.755750+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755750+0000","last_undegraded":"2024-04-13T03:13:36.755750+0000","last_fullsized":"2024-04-13T03:13:36.755750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.752998+0000","last_change":"2024-04-13T03:13:36.752998+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.752998+0000","last_undegraded":"2024-04-13T03:13:36.752998+0000","last_fullsized":"2024-04-13T03:13:36.752998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754262+0000","last_change":"2024-04-13T03:13:36.754262+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754262+0000","last_undegraded":"2024-04-13T03:13:36.754262+0000","last_fullsized":"2024-04-13T03:13:36.754262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.762340+0000","last_change":"2024-04-13T03:13:36.762340+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.762340+0000","last_undegraded":"2024-04-13T03:13:36.762340+0000","last_fullsized":"2024-04-13T03:13:36.762340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757784+0000","last_change":"2024-04-13T03:13:36.757784+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757784+0000","last_undegraded":"2024-04-13T03:13:36.757784+0000","last_fullsized":"2024-04-13T03:13:36.757784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755190+0000","last_change":"2024-04-13T03:13:36.755190+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755190+0000","last_undegraded":"2024-04-13T03:13:36.755190+0000","last_fullsized":"2024-04-13T03:13:36.755190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758905+0000","last_change":"2024-04-13T03:13:36.758905+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758905+0000","last_undegraded":"2024-04-13T03:13:36.758905+0000","last_fullsized":"2024-04-13T03:13:36.758905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757226+0000","last_change":"2024-04-13T03:13:36.757226+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757226+0000","last_undegraded":"2024-04-13T03:13:36.757226+0000","last_fullsized":"2024-04-13T03:13:36.757226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759924+0000","last_change":"2024-04-13T03:13:36.759924+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759924+0000","last_undegraded":"2024-04-13T03:13:36.759924+0000","last_fullsized":"2024-04-13T03:13:36.759924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756357+0000","last_change":"2024-04-13T03:13:36.756357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756357+0000","last_undegraded":"2024-04-13T03:13:36.756357+0000","last_fullsized":"2024-04-13T03:13:36.756357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770054+0000","last_change":"2024-04-13T03:13:36.767960+0000","last_active":"2024-04-13T03:13:36.770054+0000","last_peered":"2024-04-13T03:13:36.770054+0000","last_clean":"2024-04-13T03:13:36.770054+0000","last_became_active":"2024-04-13T03:13:36.767704+0000","last_became_peered":"2024-04-13T03:13:36.767704+0000","last_unstale":"2024-04-13T03:13:36.770054+0000","last_undegraded":"2024-04-13T03:13:36.770054+0000","last_fullsized":"2024-04-13T03:13:36.770054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:36:14.758876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753075+0000","last_change":"2024-04-13T03:13:36.753075+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753075+0000","last_undegraded":"2024-04-13T03:13:36.753075+0000","last_fullsized":"2024-04-13T03:13:36.753075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771269+0000","last_change":"2024-04-13T03:13:36.769252+0000","last_active":"2024-04-13T03:13:36.771269+0000","last_peered":"2024-04-13T03:13:36.771269+0000","last_clean":"2024-04-13T03:13:36.771269+0000","last_became_active":"2024-04-13T03:13:36.768970+0000","last_became_peered":"2024-04-13T03:13:36.768970+0000","last_unstale":"2024-04-13T03:13:36.771269+0000","last_undegraded":"2024-04-13T03:13:36.771269+0000","last_fullsized":"2024-04-13T03:13:36.771269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:49.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758502+0000","last_change":"2024-04-13T03:13:36.758502+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758502+0000","last_undegraded":"2024-04-13T03:13:36.758502+0000","last_fullsized":"2024-04-13T03:13:36.758502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770243+0000","last_change":"2024-04-13T03:13:36.767892+0000","last_active":"2024-04-13T03:13:36.770243+0000","last_peered":"2024-04-13T03:13:36.770243+0000","last_clean":"2024-04-13T03:13:36.770243+0000","last_became_active":"2024-04-13T03:13:36.767618+0000","last_became_peered":"2024-04-13T03:13:36.767618+0000","last_unstale":"2024-04-13T03:13:36.770243+0000","last_undegraded":"2024-04-13T03:13:36.770243+0000","last_fullsized":"2024-04-13T03:13:36.770243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:43.589502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758721+0000","last_change":"2024-04-13T03:13:36.758721+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758721+0000","last_undegraded":"2024-04-13T03:13:36.758721+0000","last_fullsized":"2024-04-13T03:13:36.758721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772188+0000","last_change":"2024-04-13T03:13:36.769804+0000","last_active":"2024-04-13T03:13:36.772188+0000","last_peered":"2024-04-13T03:13:36.772188+0000","last_clean":"2024-04-13T03:13:36.772188+0000","last_became_active":"2024-04-13T03:13:36.769493+0000","last_became_peered":"2024-04-13T03:13:36.769493+0000","last_unstale":"2024-04-13T03:13:36.772188+0000","last_undegraded":"2024-04-13T03:13:36.772188+0000","last_fullsized":"2024-04-13T03:13:36.772188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753617+0000","last_change":"2024-04-13T03:13:36.753617+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753617+0000","last_undegraded":"2024-04-13T03:13:36.753617+0000","last_fullsized":"2024-04-13T03:13:36.753617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771158+0000","last_change":"2024-04-13T03:13:36.768909+0000","last_active":"2024-04-13T03:13:36.771158+0000","last_peered":"2024-04-13T03:13:36.771158+0000","last_clean":"2024-04-13T03:13:36.771158+0000","last_became_active":"2024-04-13T03:13:36.768554+0000","last_became_peered":"2024-04-13T03:13:36.768554+0000","last_unstale":"2024-04-13T03:13:36.771158+0000","last_undegraded":"2024-04-13T03:13:36.771158+0000","last_fullsized":"2024-04-13T03:13:36.771158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:56.727538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753296+0000","last_change":"2024-04-13T03:13:36.753296+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753296+0000","last_undegraded":"2024-04-13T03:13:36.753296+0000","last_fullsized":"2024-04-13T03:13:36.753296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770360+0000","last_change":"2024-04-13T03:13:36.767672+0000","last_active":"2024-04-13T03:13:36.770360+0000","last_peered":"2024-04-13T03:13:36.770360+0000","last_clean":"2024-04-13T03:13:36.770360+0000","last_became_active":"2024-04-13T03:13:36.767312+0000","last_became_peered":"2024-04-13T03:13:36.767312+0000","last_unstale":"2024-04-13T03:13:36.770360+0000","last_undegraded":"2024-04-13T03:13:36.770360+0000","last_fullsized":"2024-04-13T03:13:36.770360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:51.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757834+0000","last_change":"2024-04-13T03:13:36.757834+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757834+0000","last_undegraded":"2024-04-13T03:13:36.757834+0000","last_fullsized":"2024-04-13T03:13:36.757834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769561+0000","last_change":"2024-04-13T03:13:36.765735+0000","last_active":"2024-04-13T03:13:36.769561+0000","last_peered":"2024-04-13T03:13:36.769561+0000","last_clean":"2024-04-13T03:13:36.769561+0000","last_became_active":"2024-04-13T03:13:36.765485+0000","last_became_peered":"2024-04-13T03:13:36.765485+0000","last_unstale":"2024-04-13T03:13:36.769561+0000","last_undegraded":"2024-04-13T03:13:36.769561+0000","last_fullsized":"2024-04-13T03:13:36.769561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:56.813106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758918+0000","last_change":"2024-04-13T03:13:36.758918+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758918+0000","last_undegraded":"2024-04-13T03:13:36.758918+0000","last_fullsized":"2024-04-13T03:13:36.758918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770068+0000","last_change":"2024-04-13T03:13:36.769186+0000","last_active":"2024-04-13T03:13:36.770068+0000","last_peered":"2024-04-13T03:13:36.770068+0000","last_clean":"2024-04-13T03:13:36.770068+0000","last_became_active":"2024-04-13T03:13:36.768951+0000","last_became_peered":"2024-04-13T03:13:36.768951+0000","last_unstale":"2024-04-13T03:13:36.770068+0000","last_undegraded":"2024-04-13T03:13:36.770068+0000","last_fullsized":"2024-04-13T03:13:36.770068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:16:46.808010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753081+0000","last_change":"2024-04-13T03:13:36.753081+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753081+0000","last_undegraded":"2024-04-13T03:13:36.753081+0000","last_fullsized":"2024-04-13T03:13:36.753081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772133+0000","last_change":"2024-04-13T03:13:36.769948+0000","last_active":"2024-04-13T03:13:36.772133+0000","last_peered":"2024-04-13T03:13:36.772133+0000","last_clean":"2024-04-13T03:13:36.772133+0000","last_became_active":"2024-04-13T03:13:36.769546+0000","last_became_peered":"2024-04-13T03:13:36.769546+0000","last_unstale":"2024-04-13T03:13:36.772133+0000","last_undegraded":"2024-04-13T03:13:36.772133+0000","last_fullsized":"2024-04-13T03:13:36.772133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:46.141793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754015+0000","last_change":"2024-04-13T03:13:36.754015+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754015+0000","last_undegraded":"2024-04-13T03:13:36.754015+0000","last_fullsized":"2024-04-13T03:13:36.754015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769696+0000","last_change":"2024-04-13T03:13:36.766240+0000","last_active":"2024-04-13T03:13:36.769696+0000","last_peered":"2024-04-13T03:13:36.769696+0000","last_clean":"2024-04-13T03:13:36.769696+0000","last_became_active":"2024-04-13T03:13:36.763111+0000","last_became_peered":"2024-04-13T03:13:36.763111+0000","last_unstale":"2024-04-13T03:13:36.769696+0000","last_undegraded":"2024-04-13T03:13:36.769696+0000","last_fullsized":"2024-04-13T03:13:36.769696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:28.008984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758157+0000","last_change":"2024-04-13T03:13:36.758157+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758157+0000","last_undegraded":"2024-04-13T03:13:36.758157+0000","last_fullsized":"2024-04-13T03:13:36.758157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770017+0000","last_change":"2024-04-13T03:13:36.767663+0000","last_active":"2024-04-13T03:13:36.770017+0000","last_peered":"2024-04-13T03:13:36.770017+0000","last_clean":"2024-04-13T03:13:36.770017+0000","last_became_active":"2024-04-13T03:13:36.767407+0000","last_became_peered":"2024-04-13T03:13:36.767407+0000","last_unstale":"2024-04-13T03:13:36.770017+0000","last_undegraded":"2024-04-13T03:13:36.770017+0000","last_fullsized":"2024-04-13T03:13:36.770017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:48.797655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755936+0000","last_change":"2024-04-13T03:13:36.755936+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755936+0000","last_undegraded":"2024-04-13T03:13:36.755936+0000","last_fullsized":"2024-04-13T03:13:36.755936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772167+0000","last_change":"2024-04-13T03:13:36.768069+0000","last_active":"2024-04-13T03:13:36.772167+0000","last_peered":"2024-04-13T03:13:36.772167+0000","last_clean":"2024-04-13T03:13:36.772167+0000","last_became_active":"2024-04-13T03:13:36.767790+0000","last_became_peered":"2024-04-13T03:13:36.767790+0000","last_unstale":"2024-04-13T03:13:36.772167+0000","last_undegraded":"2024-04-13T03:13:36.772167+0000","last_fullsized":"2024-04-13T03:13:36.772167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:32:59.935067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753985+0000","last_change":"2024-04-13T03:13:36.753985+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753985+0000","last_undegraded":"2024-04-13T03:13:36.753985+0000","last_fullsized":"2024-04-13T03:13:36.753985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769647+0000","last_change":"2024-04-13T03:13:36.766629+0000","last_active":"2024-04-13T03:13:36.769647+0000","last_peered":"2024-04-13T03:13:36.769647+0000","last_clean":"2024-04-13T03:13:36.769647+0000","last_became_active":"2024-04-13T03:13:36.766361+0000","last_became_peered":"2024-04-13T03:13:36.766361+0000","last_unstale":"2024-04-13T03:13:36.769647+0000","last_undegraded":"2024-04-13T03:13:36.769647+0000","last_fullsized":"2024-04-13T03:13:36.769647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:44:10.176246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758070+0000","last_change":"2024-04-13T03:13:36.758070+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758070+0000","last_undegraded":"2024-04-13T03:13:36.758070+0000","last_fullsized":"2024-04-13T03:13:36.758070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772234+0000","last_change":"2024-04-13T03:13:36.770589+0000","last_active":"2024-04-13T03:13:36.772234+0000","last_peered":"2024-04-13T03:13:36.772234+0000","last_clean":"2024-04-13T03:13:36.772234+0000","last_became_active":"2024-04-13T03:13:36.770281+0000","last_became_peered":"2024-04-13T03:13:36.770281+0000","last_unstale":"2024-04-13T03:13:36.772234+0000","last_undegraded":"2024-04-13T03:13:36.772234+0000","last_fullsized":"2024-04-13T03:13:36.772234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:53.535105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758508+0000","last_change":"2024-04-13T03:13:36.758508+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758508+0000","last_undegraded":"2024-04-13T03:13:36.758508+0000","last_fullsized":"2024-04-13T03:13:36.758508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770078+0000","last_change":"2024-04-13T03:13:36.768599+0000","last_active":"2024-04-13T03:13:36.770078+0000","last_peered":"2024-04-13T03:13:36.770078+0000","last_clean":"2024-04-13T03:13:36.770078+0000","last_became_active":"2024-04-13T03:13:36.768343+0000","last_became_peered":"2024-04-13T03:13:36.768343+0000","last_unstale":"2024-04-13T03:13:36.770078+0000","last_undegraded":"2024-04-13T03:13:36.770078+0000","last_fullsized":"2024-04-13T03:13:36.770078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:22.711857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758484+0000","last_change":"2024-04-13T03:13:36.758484+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758484+0000","last_undegraded":"2024-04-13T03:13:36.758484+0000","last_fullsized":"2024-04-13T03:13:36.758484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765921+0000","last_change":"2024-04-13T03:13:36.764197+0000","last_active":"2024-04-13T03:13:36.765921+0000","last_peered":"2024-04-13T03:13:36.765921+0000","last_clean":"2024-04-13T03:13:36.765921+0000","last_became_active":"2024-04-13T03:13:36.763920+0000","last_became_peered":"2024-04-13T03:13:36.763920+0000","last_unstale":"2024-04-13T03:13:36.765921+0000","last_undegraded":"2024-04-13T03:13:36.765921+0000","last_fullsized":"2024-04-13T03:13:36.765921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:49.280931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761836+0000","last_change":"2024-04-13T03:13:36.761836+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761836+0000","last_undegraded":"2024-04-13T03:13:36.761836+0000","last_fullsized":"2024-04-13T03:13:36.761836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772173+0000","last_change":"2024-04-13T03:13:36.770260+0000","last_active":"2024-04-13T03:13:36.772173+0000","last_peered":"2024-04-13T03:13:36.772173+0000","last_clean":"2024-04-13T03:13:36.772173+0000","last_became_active":"2024-04-13T03:13:36.769888+0000","last_became_peered":"2024-04-13T03:13:36.769888+0000","last_unstale":"2024-04-13T03:13:36.772173+0000","last_undegraded":"2024-04-13T03:13:36.772173+0000","last_fullsized":"2024-04-13T03:13:36.772173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:39.653567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760574+0000","last_change":"2024-04-13T03:13:36.760574+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760574+0000","last_undegraded":"2024-04-13T03:13:36.760574+0000","last_fullsized":"2024-04-13T03:13:36.760574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770965+0000","last_change":"2024-04-13T03:13:36.765493+0000","last_active":"2024-04-13T03:13:36.770965+0000","last_peered":"2024-04-13T03:13:36.770965+0000","last_clean":"2024-04-13T03:13:36.770965+0000","last_became_active":"2024-04-13T03:13:36.765174+0000","last_became_peered":"2024-04-13T03:13:36.765174+0000","last_unstale":"2024-04-13T03:13:36.770965+0000","last_undegraded":"2024-04-13T03:13:36.770965+0000","last_fullsized":"2024-04-13T03:13:36.770965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757837+0000","last_change":"2024-04-13T03:13:36.757837+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757837+0000","last_undegraded":"2024-04-13T03:13:36.757837+0000","last_fullsized":"2024-04-13T03:13:36.757837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770670+0000","last_change":"2024-04-13T03:13:36.769841+0000","last_active":"2024-04-13T03:13:36.770670+0000","last_peered":"2024-04-13T03:13:36.770670+0000","last_clean":"2024-04-13T03:13:36.770670+0000","last_became_active":"2024-04-13T03:13:36.769598+0000","last_became_peered":"2024-04-13T03:13:36.769598+0000","last_unstale":"2024-04-13T03:13:36.770670+0000","last_undegraded":"2024-04-13T03:13:36.770670+0000","last_fullsized":"2024-04-13T03:13:36.770670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:11:43.265755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759904+0000","last_change":"2024-04-13T03:13:36.759904+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759904+0000","last_undegraded":"2024-04-13T03:13:36.759904+0000","last_fullsized":"2024-04-13T03:13:36.759904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767378+0000","last_change":"2024-04-13T03:13:36.765265+0000","last_active":"2024-04-13T03:13:36.767378+0000","last_peered":"2024-04-13T03:13:36.767378+0000","last_clean":"2024-04-13T03:13:36.767378+0000","last_became_active":"2024-04-13T03:13:36.764966+0000","last_became_peered":"2024-04-13T03:13:36.764966+0000","last_unstale":"2024-04-13T03:13:36.767378+0000","last_undegraded":"2024-04-13T03:13:36.767378+0000","last_fullsized":"2024-04-13T03:13:36.767378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:36.244015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753237+0000","last_change":"2024-04-13T03:13:36.753237+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753237+0000","last_undegraded":"2024-04-13T03:13:36.753237+0000","last_fullsized":"2024-04-13T03:13:36.753237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767948+0000","last_change":"2024-04-13T03:13:36.765596+0000","last_active":"2024-04-13T03:13:36.767948+0000","last_peered":"2024-04-13T03:13:36.767948+0000","last_clean":"2024-04-13T03:13:36.767948+0000","last_became_active":"2024-04-13T03:13:36.765330+0000","last_became_peered":"2024-04-13T03:13:36.765330+0000","last_unstale":"2024-04-13T03:13:36.767948+0000","last_undegraded":"2024-04-13T03:13:36.767948+0000","last_fullsized":"2024-04-13T03:13:36.767948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:02.126210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753008+0000","last_change":"2024-04-13T03:13:36.753008+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753008+0000","last_undegraded":"2024-04-13T03:13:36.753008+0000","last_fullsized":"2024-04-13T03:13:36.753008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773139+0000","last_change":"2024-04-13T03:13:36.772152+0000","last_active":"2024-04-13T03:13:36.773139+0000","last_peered":"2024-04-13T03:13:36.773139+0000","last_clean":"2024-04-13T03:13:36.773139+0000","last_became_active":"2024-04-13T03:13:36.771816+0000","last_became_peered":"2024-04-13T03:13:36.771816+0000","last_unstale":"2024-04-13T03:13:36.773139+0000","last_undegraded":"2024-04-13T03:13:36.773139+0000","last_fullsized":"2024-04-13T03:13:36.773139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:51:05.902931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757953+0000","last_change":"2024-04-13T03:13:36.757953+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757953+0000","last_undegraded":"2024-04-13T03:13:36.757953+0000","last_fullsized":"2024-04-13T03:13:36.757953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773107+0000","last_change":"2024-04-13T03:13:36.772084+0000","last_active":"2024-04-13T03:13:36.773107+0000","last_peered":"2024-04-13T03:13:36.773107+0000","last_clean":"2024-04-13T03:13:36.773107+0000","last_became_active":"2024-04-13T03:13:36.771800+0000","last_became_peered":"2024-04-13T03:13:36.771800+0000","last_unstale":"2024-04-13T03:13:36.773107+0000","last_undegraded":"2024-04-13T03:13:36.773107+0000","last_fullsized":"2024-04-13T03:13:36.773107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:02.961934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757223+0000","last_change":"2024-04-13T03:13:36.757223+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757223+0000","last_undegraded":"2024-04-13T03:13:36.757223+0000","last_fullsized":"2024-04-13T03:13:36.757223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768589+0000","last_change":"2024-04-13T03:13:36.766089+0000","last_active":"2024-04-13T03:13:36.768589+0000","last_peered":"2024-04-13T03:13:36.768589+0000","last_clean":"2024-04-13T03:13:36.768589+0000","last_became_active":"2024-04-13T03:13:36.765748+0000","last_became_peered":"2024-04-13T03:13:36.765748+0000","last_unstale":"2024-04-13T03:13:36.768589+0000","last_undegraded":"2024-04-13T03:13:36.768589+0000","last_fullsized":"2024-04-13T03:13:36.768589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:32.855294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758105+0000","last_change":"2024-04-13T03:13:36.758105+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758105+0000","last_undegraded":"2024-04-13T03:13:36.758105+0000","last_fullsized":"2024-04-13T03:13:36.758105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765901+0000","last_change":"2024-04-13T03:13:36.764900+0000","last_active":"2024-04-13T03:13:36.765901+0000","last_peered":"2024-04-13T03:13:36.765901+0000","last_clean":"2024-04-13T03:13:36.765901+0000","last_became_active":"2024-04-13T03:13:36.764633+0000","last_became_peered":"2024-04-13T03:13:36.764633+0000","last_unstale":"2024-04-13T03:13:36.765901+0000","last_undegraded":"2024-04-13T03:13:36.765901+0000","last_fullsized":"2024-04-13T03:13:36.765901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:29.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757357+0000","last_change":"2024-04-13T03:13:36.757357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757357+0000","last_undegraded":"2024-04-13T03:13:36.757357+0000","last_fullsized":"2024-04-13T03:13:36.757357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772191+0000","last_change":"2024-04-13T03:13:36.770570+0000","last_active":"2024-04-13T03:13:36.772191+0000","last_peered":"2024-04-13T03:13:36.772191+0000","last_clean":"2024-04-13T03:13:36.772191+0000","last_became_active":"2024-04-13T03:13:36.769914+0000","last_became_peered":"2024-04-13T03:13:36.769914+0000","last_unstale":"2024-04-13T03:13:36.772191+0000","last_undegraded":"2024-04-13T03:13:36.772191+0000","last_fullsized":"2024-04-13T03:13:36.772191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:54.641101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753427+0000","last_change":"2024-04-13T03:13:36.753427+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753427+0000","last_undegraded":"2024-04-13T03:13:36.753427+0000","last_fullsized":"2024-04-13T03:13:36.753427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771317+0000","last_change":"2024-04-13T03:13:36.768818+0000","last_active":"2024-04-13T03:13:36.771317+0000","last_peered":"2024-04-13T03:13:36.771317+0000","last_clean":"2024-04-13T03:13:36.771317+0000","last_became_active":"2024-04-13T03:13:36.768492+0000","last_became_peered":"2024-04-13T03:13:36.768492+0000","last_unstale":"2024-04-13T03:13:36.771317+0000","last_undegraded":"2024-04-13T03:13:36.771317+0000","last_fullsized":"2024-04-13T03:13:36.771317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:49.380954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759786+0000","last_change":"2024-04-13T03:13:36.759786+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759786+0000","last_undegraded":"2024-04-13T03:13:36.759786+0000","last_fullsized":"2024-04-13T03:13:36.759786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770868+0000","last_change":"2024-04-13T03:13:36.766877+0000","last_active":"2024-04-13T03:13:36.770868+0000","last_peered":"2024-04-13T03:13:36.770868+0000","last_clean":"2024-04-13T03:13:36.770868+0000","last_became_active":"2024-04-13T03:13:36.765880+0000","last_became_peered":"2024-04-13T03:13:36.765880+0000","last_unstale":"2024-04-13T03:13:36.770868+0000","last_undegraded":"2024-04-13T03:13:36.770868+0000","last_fullsized":"2024-04-13T03:13:36.770868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:39:57.972949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757534+0000","last_change":"2024-04-13T03:13:36.757534+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757534+0000","last_undegraded":"2024-04-13T03:13:36.757534+0000","last_fullsized":"2024-04-13T03:13:36.757534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770312+0000","last_change":"2024-04-13T03:13:36.767371+0000","last_active":"2024-04-13T03:13:36.770312+0000","last_peered":"2024-04-13T03:13:36.770312+0000","last_clean":"2024-04-13T03:13:36.770312+0000","last_became_active":"2024-04-13T03:13:36.767116+0000","last_became_peered":"2024-04-13T03:13:36.767116+0000","last_unstale":"2024-04-13T03:13:36.770312+0000","last_undegraded":"2024-04-13T03:13:36.770312+0000","last_fullsized":"2024-04-13T03:13:36.770312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:05.559185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755043+0000","last_change":"2024-04-13T03:13:36.755043+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755043+0000","last_undegraded":"2024-04-13T03:13:36.755043+0000","last_fullsized":"2024-04-13T03:13:36.755043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770829+0000","last_change":"2024-04-13T03:13:36.767237+0000","last_active":"2024-04-13T03:13:36.770829+0000","last_peered":"2024-04-13T03:13:36.770829+0000","last_clean":"2024-04-13T03:13:36.770829+0000","last_became_active":"2024-04-13T03:13:36.766972+0000","last_became_peered":"2024-04-13T03:13:36.766972+0000","last_unstale":"2024-04-13T03:13:36.770829+0000","last_undegraded":"2024-04-13T03:13:36.770829+0000","last_fullsized":"2024-04-13T03:13:36.770829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:54:40.326165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755519+0000","last_change":"2024-04-13T03:13:36.755519+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755519+0000","last_undegraded":"2024-04-13T03:13:36.755519+0000","last_fullsized":"2024-04-13T03:13:36.755519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768808+0000","last_change":"2024-04-13T03:13:36.766340+0000","last_active":"2024-04-13T03:13:36.768808+0000","last_peered":"2024-04-13T03:13:36.768808+0000","last_clean":"2024-04-13T03:13:36.768808+0000","last_became_active":"2024-04-13T03:13:36.766071+0000","last_became_peered":"2024-04-13T03:13:36.766071+0000","last_unstale":"2024-04-13T03:13:36.768808+0000","last_undegraded":"2024-04-13T03:13:36.768808+0000","last_fullsized":"2024-04-13T03:13:36.768808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761893+0000","last_change":"2024-04-13T03:13:36.761893+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761893+0000","last_undegraded":"2024-04-13T03:13:36.761893+0000","last_fullsized":"2024-04-13T03:13:36.761893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773123+0000","last_change":"2024-04-13T03:13:36.772096+0000","last_active":"2024-04-13T03:13:36.773123+0000","last_peered":"2024-04-13T03:13:36.773123+0000","last_clean":"2024-04-13T03:13:36.773123+0000","last_became_active":"2024-04-13T03:13:36.771833+0000","last_became_peered":"2024-04-13T03:13:36.771833+0000","last_unstale":"2024-04-13T03:13:36.773123+0000","last_undegraded":"2024-04-13T03:13:36.773123+0000","last_fullsized":"2024-04-13T03:13:36.773123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:13:35.264150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.760793+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:13:36.760793+0000","last_peered":"2024-04-13T03:13:36.760793+0000","last_clean":"2024-04-13T03:13:36.760793+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:13:36.760793+0000","last_undegraded":"2024-04-13T03:13:36.760793+0000","last_fullsized":"2024-04-13T03:13:36.760793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":416145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:13:41.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-04-13T03:13:41.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.760106+0000', 'last_change': '2024-04-13T03:13:36.760106+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.760106+0000', 'last_undegraded': '2024-04-13T03:13:36.760106+0000', 'last_fullsized': '2024-04-13T03:13:36.760106+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2024-04-13T03:13:41.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755567+0000', 'last_change': '2024-04-13T03:13:36.755567+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755567+0000', 'last_undegraded': '2024-04-13T03:13:36.755567+0000', 'last_fullsized': '2024-04-13T03:13:36.755567+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.756896+0000', 'last_change': '2024-04-13T03:13:36.756896+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.756896+0000', 'last_undegraded': '2024-04-13T03:13:36.756896+0000', 'last_fullsized': '2024-04-13T03:13:36.756896+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755051+0000', 'last_change': '2024-04-13T03:13:36.755051+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755051+0000', 'last_undegraded': '2024-04-13T03:13:36.755051+0000', 'last_fullsized': '2024-04-13T03:13:36.755051+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758625+0000', 'last_change': '2024-04-13T03:13:36.758625+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758625+0000', 'last_undegraded': '2024-04-13T03:13:36.758625+0000', 'last_fullsized': '2024-04-13T03:13:36.758625+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758991+0000', 'last_change': '2024-04-13T03:13:36.758991+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758991+0000', 'last_undegraded': '2024-04-13T03:13:36.758991+0000', 'last_fullsized': '2024-04-13T03:13:36.758991+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.760023+0000', 'last_change': '2024-04-13T03:13:36.760023+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.760023+0000', 'last_undegraded': '2024-04-13T03:13:36.760023+0000', 'last_fullsized': '2024-04-13T03:13:36.760023+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758589+0000', 'last_change': '2024-04-13T03:13:36.758589+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758589+0000', 'last_undegraded': '2024-04-13T03:13:36.758589+0000', 'last_fullsized': '2024-04-13T03:13:36.758589+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757338+0000', 'last_change': '2024-04-13T03:13:36.757338+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757338+0000', 'last_undegraded': '2024-04-13T03:13:36.757338+0000', 'last_fullsized': '2024-04-13T03:13:36.757338+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.754743+0000', 'last_change': '2024-04-13T03:13:36.754743+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.754743+0000', 'last_undegraded': '2024-04-13T03:13:36.754743+0000', 'last_fullsized': '2024-04-13T03:13:36.754743+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753756+0000', 'last_change': '2024-04-13T03:13:36.753756+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753756+0000', 'last_undegraded': '2024-04-13T03:13:36.753756+0000', 'last_fullsized': '2024-04-13T03:13:36.753756+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.754566+0000', 'last_change': '2024-04-13T03:13:36.754566+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.754566+0000', 'last_undegraded': '2024-04-13T03:13:36.754566+0000', 'last_fullsized': '2024-04-13T03:13:36.754566+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2024-04-13T03:13:41.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758260+0000', 'last_change': '2024-04-13T03:13:36.758260+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758260+0000', 'last_undegraded': '2024-04-13T03:13:36.758260+0000', 'last_fullsized': '2024-04-13T03:13:36.758260+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758584+0000', 'last_change': '2024-04-13T03:13:36.758584+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758584+0000', 'last_undegraded': '2024-04-13T03:13:36.758584+0000', 'last_fullsized': '2024-04-13T03:13:36.758584+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753512+0000', 'last_change': '2024-04-13T03:13:36.753512+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753512+0000', 'last_undegraded': '2024-04-13T03:13:36.753512+0000', 'last_fullsized': '2024-04-13T03:13:36.753512+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.760411+0000', 'last_change': '2024-04-13T03:13:36.760411+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.760411+0000', 'last_undegraded': '2024-04-13T03:13:36.760411+0000', 'last_fullsized': '2024-04-13T03:13:36.760411+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.759321+0000', 'last_change': '2024-04-13T03:13:36.759321+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.759321+0000', 'last_undegraded': '2024-04-13T03:13:36.759321+0000', 'last_fullsized': '2024-04-13T03:13:36.759321+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.756859+0000', 'last_change': '2024-04-13T03:13:36.756859+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.756859+0000', 'last_undegraded': '2024-04-13T03:13:36.756859+0000', 'last_fullsized': '2024-04-13T03:13:36.756859+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2024-04-13T03:13:41.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.760997+0000', 'last_change': '2024-04-13T03:13:36.760997+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.760997+0000', 'last_undegraded': '2024-04-13T03:13:36.760997+0000', 'last_fullsized': '2024-04-13T03:13:36.760997+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758123+0000', 'last_change': '2024-04-13T03:13:36.758123+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758123+0000', 'last_undegraded': '2024-04-13T03:13:36.758123+0000', 'last_fullsized': '2024-04-13T03:13:36.758123+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.759788+0000', 'last_change': '2024-04-13T03:13:36.759788+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.759788+0000', 'last_undegraded': '2024-04-13T03:13:36.759788+0000', 'last_fullsized': '2024-04-13T03:13:36.759788+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757376+0000', 'last_change': '2024-04-13T03:13:36.757376+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757376+0000', 'last_undegraded': '2024-04-13T03:13:36.757376+0000', 'last_fullsized': '2024-04-13T03:13:36.757376+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758911+0000', 'last_change': '2024-04-13T03:13:36.758911+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758911+0000', 'last_undegraded': '2024-04-13T03:13:36.758911+0000', 'last_fullsized': '2024-04-13T03:13:36.758911+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755750+0000', 'last_change': '2024-04-13T03:13:36.755750+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755750+0000', 'last_undegraded': '2024-04-13T03:13:36.755750+0000', 'last_fullsized': '2024-04-13T03:13:36.755750+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2024-04-13T03:13:41.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.752998+0000', 'last_change': '2024-04-13T03:13:36.752998+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.752998+0000', 'last_undegraded': '2024-04-13T03:13:36.752998+0000', 'last_fullsized': '2024-04-13T03:13:36.752998+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.754262+0000', 'last_change': '2024-04-13T03:13:36.754262+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.754262+0000', 'last_undegraded': '2024-04-13T03:13:36.754262+0000', 'last_fullsized': '2024-04-13T03:13:36.754262+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.762340+0000', 'last_change': '2024-04-13T03:13:36.762340+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.762340+0000', 'last_undegraded': '2024-04-13T03:13:36.762340+0000', 'last_fullsized': '2024-04-13T03:13:36.762340+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757784+0000', 'last_change': '2024-04-13T03:13:36.757784+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757784+0000', 'last_undegraded': '2024-04-13T03:13:36.757784+0000', 'last_fullsized': '2024-04-13T03:13:36.757784+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755190+0000', 'last_change': '2024-04-13T03:13:36.755190+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755190+0000', 'last_undegraded': '2024-04-13T03:13:36.755190+0000', 'last_fullsized': '2024-04-13T03:13:36.755190+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-04-13T03:13:41.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758905+0000', 'last_change': '2024-04-13T03:13:36.758905+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758905+0000', 'last_undegraded': '2024-04-13T03:13:36.758905+0000', 'last_fullsized': '2024-04-13T03:13:36.758905+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757226+0000', 'last_change': '2024-04-13T03:13:36.757226+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757226+0000', 'last_undegraded': '2024-04-13T03:13:36.757226+0000', 'last_fullsized': '2024-04-13T03:13:36.757226+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.759924+0000', 'last_change': '2024-04-13T03:13:36.759924+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.759924+0000', 'last_undegraded': '2024-04-13T03:13:36.759924+0000', 'last_fullsized': '2024-04-13T03:13:36.759924+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.756357+0000', 'last_change': '2024-04-13T03:13:36.756357+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.756357+0000', 'last_undegraded': '2024-04-13T03:13:36.756357+0000', 'last_fullsized': '2024-04-13T03:13:36.756357+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753075+0000', 'last_change': '2024-04-13T03:13:36.753075+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753075+0000', 'last_undegraded': '2024-04-13T03:13:36.753075+0000', 'last_fullsized': '2024-04-13T03:13:36.753075+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758502+0000', 'last_change': '2024-04-13T03:13:36.758502+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758502+0000', 'last_undegraded': '2024-04-13T03:13:36.758502+0000', 'last_fullsized': '2024-04-13T03:13:36.758502+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758721+0000', 'last_change': '2024-04-13T03:13:36.758721+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758721+0000', 'last_undegraded': '2024-04-13T03:13:36.758721+0000', 'last_fullsized': '2024-04-13T03:13:36.758721+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753617+0000', 'last_change': '2024-04-13T03:13:36.753617+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753617+0000', 'last_undegraded': '2024-04-13T03:13:36.753617+0000', 'last_fullsized': '2024-04-13T03:13:36.753617+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753296+0000', 'last_change': '2024-04-13T03:13:36.753296+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753296+0000', 'last_undegraded': '2024-04-13T03:13:36.753296+0000', 'last_fullsized': '2024-04-13T03:13:36.753296+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757834+0000', 'last_change': '2024-04-13T03:13:36.757834+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757834+0000', 'last_undegraded': '2024-04-13T03:13:36.757834+0000', 'last_fullsized': '2024-04-13T03:13:36.757834+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758918+0000', 'last_change': '2024-04-13T03:13:36.758918+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758918+0000', 'last_undegraded': '2024-04-13T03:13:36.758918+0000', 'last_fullsized': '2024-04-13T03:13:36.758918+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753081+0000', 'last_change': '2024-04-13T03:13:36.753081+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753081+0000', 'last_undegraded': '2024-04-13T03:13:36.753081+0000', 'last_fullsized': '2024-04-13T03:13:36.753081+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.754015+0000', 'last_change': '2024-04-13T03:13:36.754015+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.754015+0000', 'last_undegraded': '2024-04-13T03:13:36.754015+0000', 'last_fullsized': '2024-04-13T03:13:36.754015+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758157+0000', 'last_change': '2024-04-13T03:13:36.758157+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758157+0000', 'last_undegraded': '2024-04-13T03:13:36.758157+0000', 'last_fullsized': '2024-04-13T03:13:36.758157+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755936+0000', 'last_change': '2024-04-13T03:13:36.755936+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755936+0000', 'last_undegraded': '2024-04-13T03:13:36.755936+0000', 'last_fullsized': '2024-04-13T03:13:36.755936+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753985+0000', 'last_change': '2024-04-13T03:13:36.753985+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753985+0000', 'last_undegraded': '2024-04-13T03:13:36.753985+0000', 'last_fullsized': '2024-04-13T03:13:36.753985+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758070+0000', 'last_change': '2024-04-13T03:13:36.758070+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758070+0000', 'last_undegraded': '2024-04-13T03:13:36.758070+0000', 'last_fullsized': '2024-04-13T03:13:36.758070+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758508+0000', 'last_change': '2024-04-13T03:13:36.758508+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758508+0000', 'last_undegraded': '2024-04-13T03:13:36.758508+0000', 'last_fullsized': '2024-04-13T03:13:36.758508+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758484+0000', 'last_change': '2024-04-13T03:13:36.758484+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758484+0000', 'last_undegraded': '2024-04-13T03:13:36.758484+0000', 'last_fullsized': '2024-04-13T03:13:36.758484+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2024-04-13T03:13:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.761836+0000', 'last_change': '2024-04-13T03:13:36.761836+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.761836+0000', 'last_undegraded': '2024-04-13T03:13:36.761836+0000', 'last_fullsized': '2024-04-13T03:13:36.761836+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.760574+0000', 'last_change': '2024-04-13T03:13:36.760574+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.760574+0000', 'last_undegraded': '2024-04-13T03:13:36.760574+0000', 'last_fullsized': '2024-04-13T03:13:36.760574+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757837+0000', 'last_change': '2024-04-13T03:13:36.757837+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757837+0000', 'last_undegraded': '2024-04-13T03:13:36.757837+0000', 'last_fullsized': '2024-04-13T03:13:36.757837+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.759904+0000', 'last_change': '2024-04-13T03:13:36.759904+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.759904+0000', 'last_undegraded': '2024-04-13T03:13:36.759904+0000', 'last_fullsized': '2024-04-13T03:13:36.759904+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753237+0000', 'last_change': '2024-04-13T03:13:36.753237+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753237+0000', 'last_undegraded': '2024-04-13T03:13:36.753237+0000', 'last_fullsized': '2024-04-13T03:13:36.753237+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753008+0000', 'last_change': '2024-04-13T03:13:36.753008+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753008+0000', 'last_undegraded': '2024-04-13T03:13:36.753008+0000', 'last_fullsized': '2024-04-13T03:13:36.753008+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757953+0000', 'last_change': '2024-04-13T03:13:36.757953+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757953+0000', 'last_undegraded': '2024-04-13T03:13:36.757953+0000', 'last_fullsized': '2024-04-13T03:13:36.757953+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757223+0000', 'last_change': '2024-04-13T03:13:36.757223+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757223+0000', 'last_undegraded': '2024-04-13T03:13:36.757223+0000', 'last_fullsized': '2024-04-13T03:13:36.757223+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.758105+0000', 'last_change': '2024-04-13T03:13:36.758105+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.758105+0000', 'last_undegraded': '2024-04-13T03:13:36.758105+0000', 'last_fullsized': '2024-04-13T03:13:36.758105+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757357+0000', 'last_change': '2024-04-13T03:13:36.757357+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757357+0000', 'last_undegraded': '2024-04-13T03:13:36.757357+0000', 'last_fullsized': '2024-04-13T03:13:36.757357+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.753427+0000', 'last_change': '2024-04-13T03:13:36.753427+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.753427+0000', 'last_undegraded': '2024-04-13T03:13:36.753427+0000', 'last_fullsized': '2024-04-13T03:13:36.753427+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.759786+0000', 'last_change': '2024-04-13T03:13:36.759786+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.759786+0000', 'last_undegraded': '2024-04-13T03:13:36.759786+0000', 'last_fullsized': '2024-04-13T03:13:36.759786+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.757534+0000', 'last_change': '2024-04-13T03:13:36.757534+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.757534+0000', 'last_undegraded': '2024-04-13T03:13:36.757534+0000', 'last_fullsized': '2024-04-13T03:13:36.757534+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755043+0000', 'last_change': '2024-04-13T03:13:36.755043+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755043+0000', 'last_undegraded': '2024-04-13T03:13:36.755043+0000', 'last_fullsized': '2024-04-13T03:13:36.755043+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.755519+0000', 'last_change': '2024-04-13T03:13:36.755519+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.755519+0000', 'last_undegraded': '2024-04-13T03:13:36.755519+0000', 'last_fullsized': '2024-04-13T03:13:36.755519+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-04-13T03:13:41.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 76, 'state': 'creating+peering', 'last_fresh': '2024-04-13T03:13:36.761893+0000', 'last_change': '2024-04-13T03:13:36.761893+0000', 'last_active': '2024-04-13T03:13:36.744747+0000', 'last_peered': '2024-04-13T03:13:36.744747+0000', 'last_clean': '2024-04-13T03:13:36.744747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-13T03:13:36.761893+0000', 'last_undegraded': '2024-04-13T03:13:36.761893+0000', 'last_fullsized': '2024-04-13T03:13:36.761893+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'last_clean_scrub_stamp': '2024-04-13T03:13:36.744747+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-13T03:13:41.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-13T03:13:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.564+0000 7f0644276640 1 Processor -- start 2024-04-13T03:13:41.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.564+0000 7f0644276640 1 -- start start 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0644276640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f063c160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0644276640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c0d2310 con 0x7f063c0c60a0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0644276640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c0b1f30 con 0x7f063c0cd2b0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0644276640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0b2210 con 0x7f063c15c7d0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0641811640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f063c0c60a0 0x7f063c0ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48288/0 (socket says 172.21.15.106:48288) 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0641811640 1 -- 172.21.15.106:0/2463195818 learned_addr learned my addr 172.21.15.106:0/2463195818 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.565+0000 7f0642012640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f063c160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518484548 0 0) 0x7f063c0d2310 con 0x7f063c0c60a0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0628003520 con 0x7f063c0c60a0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751495805 0 0) 0x7f063c0b1f30 con 0x7f063c0cd2b0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f063c0d2310 con 0x7f063c0cd2b0 2024-04-13T03:13:41.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408518151 0 0) 0x7f0628003520 con 0x7f063c0c60a0 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 msgr2=0x7f063c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f063c160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 >> v1:172.21.15.114:6789/0 conn(0x7f063c0cd2b0 legacy=0x7f063c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0b24f0 con 0x7f063c0c60a0 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f062c001670 con 0x7f063c0c60a0 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0642012640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f063c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 >> v1:172.21.15.106:6789/0 conn(0x7f063c0c60a0 legacy=0x7f063c0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f0628003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06280040b0 0x7f0628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0628004d40 0x7f06280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0b2210 con 0x7f06280040b0 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f062800ba70 con 0x7f0628004d40 2024-04-13T03:13:41.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f062800bc70 con 0x7f063c15c7d0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06280040b0 0x7f0628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0642012640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f0628003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0642813640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0628004d40 0x7f06280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 -- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 msgr2=0x7f0628003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15c7d0 0x7f0628003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 -- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0628004d40 msgr2=0x7f06280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0628004d40 0x7f06280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.566+0000 7f0641811640 1 -- 172.21.15.106:0/2463195818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628003520 con 0x7f06280040b0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.567+0000 7f0641811640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06280040b0 0x7f0628004800 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f062c007fd0 tx=0x7f0628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c339aa5cf0e6709 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.567+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c007ca0 con 0x7f06280040b0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.567+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c030070 con 0x7f06280040b0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.567+0000 7f0641010640 1 -- 172.21.15.106:0/2463195818 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c005d30 con 0x7f06280040b0 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 -- 172.21.15.106:0/2463195818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06280040b0 msgr2=0x7f0628004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 --2- 172.21.15.106:0/2463195818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06280040b0 0x7f0628004800 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f062c007fd0 tx=0x7f0628005830 comp rx=0 tx=0).stop 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0642813640 1 -- 172.21.15.106:0/2463195818 reap_dead start 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 -- 172.21.15.106:0/2463195818 shutdown_connections 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 -- 172.21.15.106:0/2463195818 >> 172.21.15.106:0/2463195818 conn(0x7f063c0d2550 msgr2=0x7f063c0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 -- 172.21.15.106:0/2463195818 shutdown_connections 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.568+0000 7f0644276640 1 -- 172.21.15.106:0/2463195818 wait complete. 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.569+0000 7f0644276640 1 Processor -- start 2024-04-13T03:13:41.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.569+0000 7f0644276640 1 -- start start 2024-04-13T03:13:41.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06280040b0 0x7f063c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15ec00 0x7f063c15f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f063c15cbb0 con 0x7f063c1525a0 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f063c160dd0 con 0x7f06280040b0 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f063c15e770 con 0x7f063c15ec00 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59680/0 (socket says 172.21.15.106:59680) 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 -- 172.21.15.106:0/762933494 learned_addr learned my addr 172.21.15.106:0/762933494 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 -- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15ec00 msgr2=0x7f063c15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15ec00 0x7f063c15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 -- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06280040b0 msgr2=0x7f063c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06280040b0 0x7f063c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 -- 172.21.15.106:0/762933494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c15db60 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0641811640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f062c0018f0 tx=0x7f063c0b2100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c001bf0 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c004bf0 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c15de40 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c007c80 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.570+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c15e400 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.571+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c0b3e50 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.572+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062c047020 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.572+0000 7f0632ffd640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.572+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f062c0ace70 con 0x7f063c1525a0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.572+0000 7f0642012640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.573+0000 7f0642012640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0638009ef0 tx=0x7f0638009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.577+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f062c07b110 con 0x7f063c1525a0 2024-04-13T03:13:41.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.825+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f063c069120 con 0x7f063c1525a0 2024-04-13T03:13:41.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.825+0000 7f0632ffd640 1 -- 172.21.15.106:0/762933494 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f062c07aa10 con 0x7f063c1525a0 2024-04-13T03:13:41.826 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:41.826 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"d7ff0368-2781-4238-b477-50485b08a223","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":281,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":78,"num_osds":8,"num_up_osds":8,"osd_up_since":1712977751,"num_in_osds":8,"osd_in_since":1712977742,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":234242048,"bytes_avail":772859871232,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":36,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6825}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-13T03:13:24.414259+0000","services":{}},"progress_events":{}} 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 msgr2=0x7f062006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0638009ef0 tx=0x7f0638009750 comp rx=0 tx=0).stop 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 msgr2=0x7f063c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f062c0018f0 tx=0x7f063c0b2100 comp rx=0 tx=0).stop 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 shutdown_connections 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f063c15ec00 0x7f063c15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f063c1525a0 0x7f063c0c9620 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 --2- 172.21.15.106:0/762933494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06280040b0 0x7f063c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 >> 172.21.15.106:0/762933494 conn(0x7f063c0d2550 msgr2=0x7f063c070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 shutdown_connections 2024-04-13T03:13:41.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.827+0000 7f0644276640 1 -- 172.21.15.106:0/762933494 wait complete. 2024-04-13T03:13:41.839 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-13T03:13:41.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:13:41.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.938+0000 7ff0a89fe640 1 Processor -- start 2024-04-13T03:13:41.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.938+0000 7ff0a89fe640 1 -- start start 2024-04-13T03:13:41.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a89fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0a415ebf0 0x7ff0a406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a89fe640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a406f380 con 0x7ff0a40ce1f0 2024-04-13T03:13:41.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a89fe640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a40ce6f0 con 0x7ff0a40b5bf0 2024-04-13T03:13:41.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a89fe640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a415fa70 con 0x7ff0a415ebf0 2024-04-13T03:13:41.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0a415ebf0 0x7ff0a406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0a415ebf0 0x7ff0a406f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59682/0 (socket says 172.21.15.106:59682) 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff099d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff0a40ce1f0 0x7ff0a406e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48304/0 (socket says 172.21.15.106:48304) 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 -- 172.21.15.106:0/1616384763 learned_addr learned my addr 172.21.15.106:0/1616384763 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 -- 172.21.15.106:0/1616384763 >> v1:172.21.15.114:6789/0 conn(0x7ff0a40b5bf0 legacy=0x7ff0a406ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 -- 172.21.15.106:0/1616384763 >> v1:172.21.15.106:6789/0 conn(0x7ff0a40ce1f0 legacy=0x7ff0a406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a1d9b640 1 -- 172.21.15.106:0/1616384763 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956572861 0 0) 0x7ff0a406f380 con 0x7ff0a40ce1f0 2024-04-13T03:13:41.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.939+0000 7ff0a259c640 1 -- 172.21.15.106:0/1616384763 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a415f420 con 0x7ff0a415ebf0 2024-04-13T03:13:41.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.940+0000 7ff0a259c640 1 --2- 172.21.15.106:0/1616384763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a406f780 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff094009870 tx=0x7ff09402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d502d16003f810bc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.940+0000 7ff0a1d9b640 1 -- 172.21.15.106:0/1616384763 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094034070 con 0x7ff0a415ebf0 2024-04-13T03:13:41.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.940+0000 7ff0a1d9b640 1 -- 172.21.15.106:0/1616384763 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff094030860 con 0x7ff0a415ebf0 2024-04-13T03:13:41.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.940+0000 7ff0a1d9b640 1 -- 172.21.15.106:0/1616384763 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094030c30 con 0x7ff0a415ebf0 2024-04-13T03:13:41.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 -- 172.21.15.106:0/1616384763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 msgr2=0x7ff0a406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/1616384763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a406f780 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff094009870 tx=0x7ff09402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 -- 172.21.15.106:0/1616384763 shutdown_connections 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/1616384763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a406f780 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 -- 172.21.15.106:0/1616384763 >> 172.21.15.106:0/1616384763 conn(0x7ff0a40cd6d0 msgr2=0x7ff0a40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 -- 172.21.15.106:0/1616384763 shutdown_connections 2024-04-13T03:13:41.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.941+0000 7ff0a89fe640 1 -- 172.21.15.106:0/1616384763 wait complete. 2024-04-13T03:13:41.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.942+0000 7ff0a89fe640 1 Processor -- start 2024-04-13T03:13:41.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.942+0000 7ff0a89fe640 1 -- start start 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 0x7ff0a4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 0x7ff0a40af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a40b04d0 con 0x7ff0a415ebf0 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a415ff00 con 0x7ff0a40b5bf0 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a89fe640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a40b0650 con 0x7ff0a40ce1f0 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59688/0 (socket says 172.21.15.106:59688) 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 -- 172.21.15.106:0/127309094 learned_addr learned my addr 172.21.15.106:0/127309094 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:41.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a259c640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 0x7ff0a4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff099d9b640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 0x7ff0a40af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 -- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 msgr2=0x7ff0a40af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 0x7ff0a40af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 -- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 msgr2=0x7ff0a4154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 0x7ff0a4154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 -- 172.21.15.106:0/127309094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a40afba0 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a259c640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 0x7ff0a4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff099d9b640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 0x7ff0a40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.943+0000 7ff0a2d9d640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c6b0 tx=0x7ff09c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.944+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c003b70 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.944+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c003d30 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.944+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c005310 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.944+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a41602d0 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.944+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a4160890 con 0x7ff0a415ebf0 2024-04-13T03:13:41.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.945+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a4068a10 con 0x7ff0a415ebf0 2024-04-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.945+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff09c0054b0 con 0x7ff0a415ebf0 2024-04-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.945+0000 7ff09b7fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.946+0000 7ff0a259c640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.946+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7ff09c08bab0 con 0x7ff0a415ebf0 2024-04-13T03:13:41.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.946+0000 7ff0a259c640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 0x7ff08406edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff094036000 tx=0x7ff09403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:41.950+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0a40afba0 con 0x7ff0a415ebf0 2024-04-13T03:13:42.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.121+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0a40cec80 con 0x7ff08406c8f0 2024-04-13T03:13:42.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.127+0000 7ff09b7fe640 1 -- 172.21.15.106:0/127309094 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246756 (secure 0 0 0) 0x7ff0a40cec80 con 0x7ff08406c8f0 2024-04-13T03:13:42.130 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:42.131 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.129+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 msgr2=0x7ff08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.129+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 0x7ff08406edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff094036000 tx=0x7ff09403b040 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.129+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 msgr2=0x7ff0a40b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.129+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00c6b0 tx=0x7ff09c00cb70 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 shutdown_connections 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0a40ce1f0 0x7ff0a40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0a415ebf0 0x7ff0a40b1ac0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 --2- 172.21.15.106:0/127309094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0a40b5bf0 0x7ff0a4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:42.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 >> 172.21.15.106:0/127309094 conn(0x7ff0a40cd6d0 msgr2=0x7ff0a40ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:42.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 shutdown_connections 2024-04-13T03:13:42.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:42.130+0000 7ff0a89fe640 1 -- 172.21.15.106:0/127309094 wait complete. 2024-04-13T03:13:42.141 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-13T03:13:40.417060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228752,"kb_used_data":8412,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754745968,"statfs":{"total":773094113280,"available":772859871232,"internally_reserved":0,"allocated":8613888,"data_stored":4556565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672266"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760106+0000","last_change":"2024-04-13T03:13:36.760106+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760106+0000","last_undegraded":"2024-04-13T03:13:36.760106+0000","last_fullsized":"2024-04-13T03:13:36.760106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755567+0000","last_change":"2024-04-13T03:13:36.755567+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755567+0000","last_undegraded":"2024-04-13T03:13:36.755567+0000","last_fullsized":"2024-04-13T03:13:36.755567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756896+0000","last_change":"2024-04-13T03:13:36.756896+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756896+0000","last_undegraded":"2024-04-13T03:13:36.756896+0000","last_fullsized":"2024-04-13T03:13:36.756896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755051+0000","last_change":"2024-04-13T03:13:36.755051+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755051+0000","last_undegraded":"2024-04-13T03:13:36.755051+0000","last_fullsized":"2024-04-13T03:13:36.755051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758625+0000","last_change":"2024-04-13T03:13:36.758625+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758625+0000","last_undegraded":"2024-04-13T03:13:36.758625+0000","last_fullsized":"2024-04-13T03:13:36.758625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758991+0000","last_change":"2024-04-13T03:13:36.758991+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758991+0000","last_undegraded":"2024-04-13T03:13:36.758991+0000","last_fullsized":"2024-04-13T03:13:36.758991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760023+0000","last_change":"2024-04-13T03:13:36.760023+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760023+0000","last_undegraded":"2024-04-13T03:13:36.760023+0000","last_fullsized":"2024-04-13T03:13:36.760023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758589+0000","last_change":"2024-04-13T03:13:36.758589+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758589+0000","last_undegraded":"2024-04-13T03:13:36.758589+0000","last_fullsized":"2024-04-13T03:13:36.758589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757338+0000","last_change":"2024-04-13T03:13:36.757338+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757338+0000","last_undegraded":"2024-04-13T03:13:36.757338+0000","last_fullsized":"2024-04-13T03:13:36.757338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754743+0000","last_change":"2024-04-13T03:13:36.754743+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754743+0000","last_undegraded":"2024-04-13T03:13:36.754743+0000","last_fullsized":"2024-04-13T03:13:36.754743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753756+0000","last_change":"2024-04-13T03:13:36.753756+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753756+0000","last_undegraded":"2024-04-13T03:13:36.753756+0000","last_fullsized":"2024-04-13T03:13:36.753756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754566+0000","last_change":"2024-04-13T03:13:36.754566+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754566+0000","last_undegraded":"2024-04-13T03:13:36.754566+0000","last_fullsized":"2024-04-13T03:13:36.754566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758260+0000","last_change":"2024-04-13T03:13:36.758260+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758260+0000","last_undegraded":"2024-04-13T03:13:36.758260+0000","last_fullsized":"2024-04-13T03:13:36.758260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758584+0000","last_change":"2024-04-13T03:13:36.758584+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758584+0000","last_undegraded":"2024-04-13T03:13:36.758584+0000","last_fullsized":"2024-04-13T03:13:36.758584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753512+0000","last_change":"2024-04-13T03:13:36.753512+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753512+0000","last_undegraded":"2024-04-13T03:13:36.753512+0000","last_fullsized":"2024-04-13T03:13:36.753512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760411+0000","last_change":"2024-04-13T03:13:36.760411+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760411+0000","last_undegraded":"2024-04-13T03:13:36.760411+0000","last_fullsized":"2024-04-13T03:13:36.760411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759321+0000","last_change":"2024-04-13T03:13:36.759321+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759321+0000","last_undegraded":"2024-04-13T03:13:36.759321+0000","last_fullsized":"2024-04-13T03:13:36.759321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756859+0000","last_change":"2024-04-13T03:13:36.756859+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756859+0000","last_undegraded":"2024-04-13T03:13:36.756859+0000","last_fullsized":"2024-04-13T03:13:36.756859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760997+0000","last_change":"2024-04-13T03:13:36.760997+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760997+0000","last_undegraded":"2024-04-13T03:13:36.760997+0000","last_fullsized":"2024-04-13T03:13:36.760997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758123+0000","last_change":"2024-04-13T03:13:36.758123+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758123+0000","last_undegraded":"2024-04-13T03:13:36.758123+0000","last_fullsized":"2024-04-13T03:13:36.758123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759788+0000","last_change":"2024-04-13T03:13:36.759788+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759788+0000","last_undegraded":"2024-04-13T03:13:36.759788+0000","last_fullsized":"2024-04-13T03:13:36.759788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757376+0000","last_change":"2024-04-13T03:13:36.757376+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757376+0000","last_undegraded":"2024-04-13T03:13:36.757376+0000","last_fullsized":"2024-04-13T03:13:36.757376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758911+0000","last_change":"2024-04-13T03:13:36.758911+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758911+0000","last_undegraded":"2024-04-13T03:13:36.758911+0000","last_fullsized":"2024-04-13T03:13:36.758911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755750+0000","last_change":"2024-04-13T03:13:36.755750+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755750+0000","last_undegraded":"2024-04-13T03:13:36.755750+0000","last_fullsized":"2024-04-13T03:13:36.755750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.752998+0000","last_change":"2024-04-13T03:13:36.752998+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.752998+0000","last_undegraded":"2024-04-13T03:13:36.752998+0000","last_fullsized":"2024-04-13T03:13:36.752998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754262+0000","last_change":"2024-04-13T03:13:36.754262+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754262+0000","last_undegraded":"2024-04-13T03:13:36.754262+0000","last_fullsized":"2024-04-13T03:13:36.754262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.762340+0000","last_change":"2024-04-13T03:13:36.762340+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.762340+0000","last_undegraded":"2024-04-13T03:13:36.762340+0000","last_fullsized":"2024-04-13T03:13:36.762340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757784+0000","last_change":"2024-04-13T03:13:36.757784+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757784+0000","last_undegraded":"2024-04-13T03:13:36.757784+0000","last_fullsized":"2024-04-13T03:13:36.757784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755190+0000","last_change":"2024-04-13T03:13:36.755190+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755190+0000","last_undegraded":"2024-04-13T03:13:36.755190+0000","last_fullsized":"2024-04-13T03:13:36.755190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758905+0000","last_change":"2024-04-13T03:13:36.758905+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758905+0000","last_undegraded":"2024-04-13T03:13:36.758905+0000","last_fullsized":"2024-04-13T03:13:36.758905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757226+0000","last_change":"2024-04-13T03:13:36.757226+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757226+0000","last_undegraded":"2024-04-13T03:13:36.757226+0000","last_fullsized":"2024-04-13T03:13:36.757226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759924+0000","last_change":"2024-04-13T03:13:36.759924+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759924+0000","last_undegraded":"2024-04-13T03:13:36.759924+0000","last_fullsized":"2024-04-13T03:13:36.759924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.756357+0000","last_change":"2024-04-13T03:13:36.756357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.756357+0000","last_undegraded":"2024-04-13T03:13:36.756357+0000","last_fullsized":"2024-04-13T03:13:36.756357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770054+0000","last_change":"2024-04-13T03:13:36.767960+0000","last_active":"2024-04-13T03:13:36.770054+0000","last_peered":"2024-04-13T03:13:36.770054+0000","last_clean":"2024-04-13T03:13:36.770054+0000","last_became_active":"2024-04-13T03:13:36.767704+0000","last_became_peered":"2024-04-13T03:13:36.767704+0000","last_unstale":"2024-04-13T03:13:36.770054+0000","last_undegraded":"2024-04-13T03:13:36.770054+0000","last_fullsized":"2024-04-13T03:13:36.770054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:36:14.758876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753075+0000","last_change":"2024-04-13T03:13:36.753075+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753075+0000","last_undegraded":"2024-04-13T03:13:36.753075+0000","last_fullsized":"2024-04-13T03:13:36.753075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771269+0000","last_change":"2024-04-13T03:13:36.769252+0000","last_active":"2024-04-13T03:13:36.771269+0000","last_peered":"2024-04-13T03:13:36.771269+0000","last_clean":"2024-04-13T03:13:36.771269+0000","last_became_active":"2024-04-13T03:13:36.768970+0000","last_became_peered":"2024-04-13T03:13:36.768970+0000","last_unstale":"2024-04-13T03:13:36.771269+0000","last_undegraded":"2024-04-13T03:13:36.771269+0000","last_fullsized":"2024-04-13T03:13:36.771269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:49.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758502+0000","last_change":"2024-04-13T03:13:36.758502+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758502+0000","last_undegraded":"2024-04-13T03:13:36.758502+0000","last_fullsized":"2024-04-13T03:13:36.758502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770243+0000","last_change":"2024-04-13T03:13:36.767892+0000","last_active":"2024-04-13T03:13:36.770243+0000","last_peered":"2024-04-13T03:13:36.770243+0000","last_clean":"2024-04-13T03:13:36.770243+0000","last_became_active":"2024-04-13T03:13:36.767618+0000","last_became_peered":"2024-04-13T03:13:36.767618+0000","last_unstale":"2024-04-13T03:13:36.770243+0000","last_undegraded":"2024-04-13T03:13:36.770243+0000","last_fullsized":"2024-04-13T03:13:36.770243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:43.589502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758721+0000","last_change":"2024-04-13T03:13:36.758721+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758721+0000","last_undegraded":"2024-04-13T03:13:36.758721+0000","last_fullsized":"2024-04-13T03:13:36.758721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772188+0000","last_change":"2024-04-13T03:13:36.769804+0000","last_active":"2024-04-13T03:13:36.772188+0000","last_peered":"2024-04-13T03:13:36.772188+0000","last_clean":"2024-04-13T03:13:36.772188+0000","last_became_active":"2024-04-13T03:13:36.769493+0000","last_became_peered":"2024-04-13T03:13:36.769493+0000","last_unstale":"2024-04-13T03:13:36.772188+0000","last_undegraded":"2024-04-13T03:13:36.772188+0000","last_fullsized":"2024-04-13T03:13:36.772188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753617+0000","last_change":"2024-04-13T03:13:36.753617+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753617+0000","last_undegraded":"2024-04-13T03:13:36.753617+0000","last_fullsized":"2024-04-13T03:13:36.753617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771158+0000","last_change":"2024-04-13T03:13:36.768909+0000","last_active":"2024-04-13T03:13:36.771158+0000","last_peered":"2024-04-13T03:13:36.771158+0000","last_clean":"2024-04-13T03:13:36.771158+0000","last_became_active":"2024-04-13T03:13:36.768554+0000","last_became_peered":"2024-04-13T03:13:36.768554+0000","last_unstale":"2024-04-13T03:13:36.771158+0000","last_undegraded":"2024-04-13T03:13:36.771158+0000","last_fullsized":"2024-04-13T03:13:36.771158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:56.727538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753296+0000","last_change":"2024-04-13T03:13:36.753296+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753296+0000","last_undegraded":"2024-04-13T03:13:36.753296+0000","last_fullsized":"2024-04-13T03:13:36.753296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770360+0000","last_change":"2024-04-13T03:13:36.767672+0000","last_active":"2024-04-13T03:13:36.770360+0000","last_peered":"2024-04-13T03:13:36.770360+0000","last_clean":"2024-04-13T03:13:36.770360+0000","last_became_active":"2024-04-13T03:13:36.767312+0000","last_became_peered":"2024-04-13T03:13:36.767312+0000","last_unstale":"2024-04-13T03:13:36.770360+0000","last_undegraded":"2024-04-13T03:13:36.770360+0000","last_fullsized":"2024-04-13T03:13:36.770360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:51.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757834+0000","last_change":"2024-04-13T03:13:36.757834+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757834+0000","last_undegraded":"2024-04-13T03:13:36.757834+0000","last_fullsized":"2024-04-13T03:13:36.757834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769561+0000","last_change":"2024-04-13T03:13:36.765735+0000","last_active":"2024-04-13T03:13:36.769561+0000","last_peered":"2024-04-13T03:13:36.769561+0000","last_clean":"2024-04-13T03:13:36.769561+0000","last_became_active":"2024-04-13T03:13:36.765485+0000","last_became_peered":"2024-04-13T03:13:36.765485+0000","last_unstale":"2024-04-13T03:13:36.769561+0000","last_undegraded":"2024-04-13T03:13:36.769561+0000","last_fullsized":"2024-04-13T03:13:36.769561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:56.813106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758918+0000","last_change":"2024-04-13T03:13:36.758918+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758918+0000","last_undegraded":"2024-04-13T03:13:36.758918+0000","last_fullsized":"2024-04-13T03:13:36.758918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770068+0000","last_change":"2024-04-13T03:13:36.769186+0000","last_active":"2024-04-13T03:13:36.770068+0000","last_peered":"2024-04-13T03:13:36.770068+0000","last_clean":"2024-04-13T03:13:36.770068+0000","last_became_active":"2024-04-13T03:13:36.768951+0000","last_became_peered":"2024-04-13T03:13:36.768951+0000","last_unstale":"2024-04-13T03:13:36.770068+0000","last_undegraded":"2024-04-13T03:13:36.770068+0000","last_fullsized":"2024-04-13T03:13:36.770068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:16:46.808010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753081+0000","last_change":"2024-04-13T03:13:36.753081+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753081+0000","last_undegraded":"2024-04-13T03:13:36.753081+0000","last_fullsized":"2024-04-13T03:13:36.753081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772133+0000","last_change":"2024-04-13T03:13:36.769948+0000","last_active":"2024-04-13T03:13:36.772133+0000","last_peered":"2024-04-13T03:13:36.772133+0000","last_clean":"2024-04-13T03:13:36.772133+0000","last_became_active":"2024-04-13T03:13:36.769546+0000","last_became_peered":"2024-04-13T03:13:36.769546+0000","last_unstale":"2024-04-13T03:13:36.772133+0000","last_undegraded":"2024-04-13T03:13:36.772133+0000","last_fullsized":"2024-04-13T03:13:36.772133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:46.141793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.754015+0000","last_change":"2024-04-13T03:13:36.754015+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.754015+0000","last_undegraded":"2024-04-13T03:13:36.754015+0000","last_fullsized":"2024-04-13T03:13:36.754015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769696+0000","last_change":"2024-04-13T03:13:36.766240+0000","last_active":"2024-04-13T03:13:36.769696+0000","last_peered":"2024-04-13T03:13:36.769696+0000","last_clean":"2024-04-13T03:13:36.769696+0000","last_became_active":"2024-04-13T03:13:36.763111+0000","last_became_peered":"2024-04-13T03:13:36.763111+0000","last_unstale":"2024-04-13T03:13:36.769696+0000","last_undegraded":"2024-04-13T03:13:36.769696+0000","last_fullsized":"2024-04-13T03:13:36.769696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:28.008984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758157+0000","last_change":"2024-04-13T03:13:36.758157+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758157+0000","last_undegraded":"2024-04-13T03:13:36.758157+0000","last_fullsized":"2024-04-13T03:13:36.758157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770017+0000","last_change":"2024-04-13T03:13:36.767663+0000","last_active":"2024-04-13T03:13:36.770017+0000","last_peered":"2024-04-13T03:13:36.770017+0000","last_clean":"2024-04-13T03:13:36.770017+0000","last_became_active":"2024-04-13T03:13:36.767407+0000","last_became_peered":"2024-04-13T03:13:36.767407+0000","last_unstale":"2024-04-13T03:13:36.770017+0000","last_undegraded":"2024-04-13T03:13:36.770017+0000","last_fullsized":"2024-04-13T03:13:36.770017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:55:48.797655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755936+0000","last_change":"2024-04-13T03:13:36.755936+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755936+0000","last_undegraded":"2024-04-13T03:13:36.755936+0000","last_fullsized":"2024-04-13T03:13:36.755936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772167+0000","last_change":"2024-04-13T03:13:36.768069+0000","last_active":"2024-04-13T03:13:36.772167+0000","last_peered":"2024-04-13T03:13:36.772167+0000","last_clean":"2024-04-13T03:13:36.772167+0000","last_became_active":"2024-04-13T03:13:36.767790+0000","last_became_peered":"2024-04-13T03:13:36.767790+0000","last_unstale":"2024-04-13T03:13:36.772167+0000","last_undegraded":"2024-04-13T03:13:36.772167+0000","last_fullsized":"2024-04-13T03:13:36.772167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:32:59.935067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753985+0000","last_change":"2024-04-13T03:13:36.753985+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753985+0000","last_undegraded":"2024-04-13T03:13:36.753985+0000","last_fullsized":"2024-04-13T03:13:36.753985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.769647+0000","last_change":"2024-04-13T03:13:36.766629+0000","last_active":"2024-04-13T03:13:36.769647+0000","last_peered":"2024-04-13T03:13:36.769647+0000","last_clean":"2024-04-13T03:13:36.769647+0000","last_became_active":"2024-04-13T03:13:36.766361+0000","last_became_peered":"2024-04-13T03:13:36.766361+0000","last_unstale":"2024-04-13T03:13:36.769647+0000","last_undegraded":"2024-04-13T03:13:36.769647+0000","last_fullsized":"2024-04-13T03:13:36.769647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:44:10.176246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758070+0000","last_change":"2024-04-13T03:13:36.758070+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758070+0000","last_undegraded":"2024-04-13T03:13:36.758070+0000","last_fullsized":"2024-04-13T03:13:36.758070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772234+0000","last_change":"2024-04-13T03:13:36.770589+0000","last_active":"2024-04-13T03:13:36.772234+0000","last_peered":"2024-04-13T03:13:36.772234+0000","last_clean":"2024-04-13T03:13:36.772234+0000","last_became_active":"2024-04-13T03:13:36.770281+0000","last_became_peered":"2024-04-13T03:13:36.770281+0000","last_unstale":"2024-04-13T03:13:36.772234+0000","last_undegraded":"2024-04-13T03:13:36.772234+0000","last_fullsized":"2024-04-13T03:13:36.772234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:53.535105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758508+0000","last_change":"2024-04-13T03:13:36.758508+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758508+0000","last_undegraded":"2024-04-13T03:13:36.758508+0000","last_fullsized":"2024-04-13T03:13:36.758508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770078+0000","last_change":"2024-04-13T03:13:36.768599+0000","last_active":"2024-04-13T03:13:36.770078+0000","last_peered":"2024-04-13T03:13:36.770078+0000","last_clean":"2024-04-13T03:13:36.770078+0000","last_became_active":"2024-04-13T03:13:36.768343+0000","last_became_peered":"2024-04-13T03:13:36.768343+0000","last_unstale":"2024-04-13T03:13:36.770078+0000","last_undegraded":"2024-04-13T03:13:36.770078+0000","last_fullsized":"2024-04-13T03:13:36.770078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:22.711857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758484+0000","last_change":"2024-04-13T03:13:36.758484+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758484+0000","last_undegraded":"2024-04-13T03:13:36.758484+0000","last_fullsized":"2024-04-13T03:13:36.758484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765921+0000","last_change":"2024-04-13T03:13:36.764197+0000","last_active":"2024-04-13T03:13:36.765921+0000","last_peered":"2024-04-13T03:13:36.765921+0000","last_clean":"2024-04-13T03:13:36.765921+0000","last_became_active":"2024-04-13T03:13:36.763920+0000","last_became_peered":"2024-04-13T03:13:36.763920+0000","last_unstale":"2024-04-13T03:13:36.765921+0000","last_undegraded":"2024-04-13T03:13:36.765921+0000","last_fullsized":"2024-04-13T03:13:36.765921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:49.280931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761836+0000","last_change":"2024-04-13T03:13:36.761836+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761836+0000","last_undegraded":"2024-04-13T03:13:36.761836+0000","last_fullsized":"2024-04-13T03:13:36.761836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772173+0000","last_change":"2024-04-13T03:13:36.770260+0000","last_active":"2024-04-13T03:13:36.772173+0000","last_peered":"2024-04-13T03:13:36.772173+0000","last_clean":"2024-04-13T03:13:36.772173+0000","last_became_active":"2024-04-13T03:13:36.769888+0000","last_became_peered":"2024-04-13T03:13:36.769888+0000","last_unstale":"2024-04-13T03:13:36.772173+0000","last_undegraded":"2024-04-13T03:13:36.772173+0000","last_fullsized":"2024-04-13T03:13:36.772173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:39.653567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.760574+0000","last_change":"2024-04-13T03:13:36.760574+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.760574+0000","last_undegraded":"2024-04-13T03:13:36.760574+0000","last_fullsized":"2024-04-13T03:13:36.760574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770965+0000","last_change":"2024-04-13T03:13:36.765493+0000","last_active":"2024-04-13T03:13:36.770965+0000","last_peered":"2024-04-13T03:13:36.770965+0000","last_clean":"2024-04-13T03:13:36.770965+0000","last_became_active":"2024-04-13T03:13:36.765174+0000","last_became_peered":"2024-04-13T03:13:36.765174+0000","last_unstale":"2024-04-13T03:13:36.770965+0000","last_undegraded":"2024-04-13T03:13:36.770965+0000","last_fullsized":"2024-04-13T03:13:36.770965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:15.463748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757837+0000","last_change":"2024-04-13T03:13:36.757837+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757837+0000","last_undegraded":"2024-04-13T03:13:36.757837+0000","last_fullsized":"2024-04-13T03:13:36.757837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770670+0000","last_change":"2024-04-13T03:13:36.769841+0000","last_active":"2024-04-13T03:13:36.770670+0000","last_peered":"2024-04-13T03:13:36.770670+0000","last_clean":"2024-04-13T03:13:36.770670+0000","last_became_active":"2024-04-13T03:13:36.769598+0000","last_became_peered":"2024-04-13T03:13:36.769598+0000","last_unstale":"2024-04-13T03:13:36.770670+0000","last_undegraded":"2024-04-13T03:13:36.770670+0000","last_fullsized":"2024-04-13T03:13:36.770670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:11:43.265755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759904+0000","last_change":"2024-04-13T03:13:36.759904+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759904+0000","last_undegraded":"2024-04-13T03:13:36.759904+0000","last_fullsized":"2024-04-13T03:13:36.759904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767378+0000","last_change":"2024-04-13T03:13:36.765265+0000","last_active":"2024-04-13T03:13:36.767378+0000","last_peered":"2024-04-13T03:13:36.767378+0000","last_clean":"2024-04-13T03:13:36.767378+0000","last_became_active":"2024-04-13T03:13:36.764966+0000","last_became_peered":"2024-04-13T03:13:36.764966+0000","last_unstale":"2024-04-13T03:13:36.767378+0000","last_undegraded":"2024-04-13T03:13:36.767378+0000","last_fullsized":"2024-04-13T03:13:36.767378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:36.244015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753237+0000","last_change":"2024-04-13T03:13:36.753237+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753237+0000","last_undegraded":"2024-04-13T03:13:36.753237+0000","last_fullsized":"2024-04-13T03:13:36.753237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.767948+0000","last_change":"2024-04-13T03:13:36.765596+0000","last_active":"2024-04-13T03:13:36.767948+0000","last_peered":"2024-04-13T03:13:36.767948+0000","last_clean":"2024-04-13T03:13:36.767948+0000","last_became_active":"2024-04-13T03:13:36.765330+0000","last_became_peered":"2024-04-13T03:13:36.765330+0000","last_unstale":"2024-04-13T03:13:36.767948+0000","last_undegraded":"2024-04-13T03:13:36.767948+0000","last_fullsized":"2024-04-13T03:13:36.767948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:02.126210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753008+0000","last_change":"2024-04-13T03:13:36.753008+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753008+0000","last_undegraded":"2024-04-13T03:13:36.753008+0000","last_fullsized":"2024-04-13T03:13:36.753008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773139+0000","last_change":"2024-04-13T03:13:36.772152+0000","last_active":"2024-04-13T03:13:36.773139+0000","last_peered":"2024-04-13T03:13:36.773139+0000","last_clean":"2024-04-13T03:13:36.773139+0000","last_became_active":"2024-04-13T03:13:36.771816+0000","last_became_peered":"2024-04-13T03:13:36.771816+0000","last_unstale":"2024-04-13T03:13:36.773139+0000","last_undegraded":"2024-04-13T03:13:36.773139+0000","last_fullsized":"2024-04-13T03:13:36.773139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:51:05.902931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757953+0000","last_change":"2024-04-13T03:13:36.757953+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757953+0000","last_undegraded":"2024-04-13T03:13:36.757953+0000","last_fullsized":"2024-04-13T03:13:36.757953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773107+0000","last_change":"2024-04-13T03:13:36.772084+0000","last_active":"2024-04-13T03:13:36.773107+0000","last_peered":"2024-04-13T03:13:36.773107+0000","last_clean":"2024-04-13T03:13:36.773107+0000","last_became_active":"2024-04-13T03:13:36.771800+0000","last_became_peered":"2024-04-13T03:13:36.771800+0000","last_unstale":"2024-04-13T03:13:36.773107+0000","last_undegraded":"2024-04-13T03:13:36.773107+0000","last_fullsized":"2024-04-13T03:13:36.773107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:02.961934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757223+0000","last_change":"2024-04-13T03:13:36.757223+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757223+0000","last_undegraded":"2024-04-13T03:13:36.757223+0000","last_fullsized":"2024-04-13T03:13:36.757223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768589+0000","last_change":"2024-04-13T03:13:36.766089+0000","last_active":"2024-04-13T03:13:36.768589+0000","last_peered":"2024-04-13T03:13:36.768589+0000","last_clean":"2024-04-13T03:13:36.768589+0000","last_became_active":"2024-04-13T03:13:36.765748+0000","last_became_peered":"2024-04-13T03:13:36.765748+0000","last_unstale":"2024-04-13T03:13:36.768589+0000","last_undegraded":"2024-04-13T03:13:36.768589+0000","last_fullsized":"2024-04-13T03:13:36.768589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:32.855294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.758105+0000","last_change":"2024-04-13T03:13:36.758105+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.758105+0000","last_undegraded":"2024-04-13T03:13:36.758105+0000","last_fullsized":"2024-04-13T03:13:36.758105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.765901+0000","last_change":"2024-04-13T03:13:36.764900+0000","last_active":"2024-04-13T03:13:36.765901+0000","last_peered":"2024-04-13T03:13:36.765901+0000","last_clean":"2024-04-13T03:13:36.765901+0000","last_became_active":"2024-04-13T03:13:36.764633+0000","last_became_peered":"2024-04-13T03:13:36.764633+0000","last_unstale":"2024-04-13T03:13:36.765901+0000","last_undegraded":"2024-04-13T03:13:36.765901+0000","last_fullsized":"2024-04-13T03:13:36.765901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:29.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757357+0000","last_change":"2024-04-13T03:13:36.757357+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757357+0000","last_undegraded":"2024-04-13T03:13:36.757357+0000","last_fullsized":"2024-04-13T03:13:36.757357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.772191+0000","last_change":"2024-04-13T03:13:36.770570+0000","last_active":"2024-04-13T03:13:36.772191+0000","last_peered":"2024-04-13T03:13:36.772191+0000","last_clean":"2024-04-13T03:13:36.772191+0000","last_became_active":"2024-04-13T03:13:36.769914+0000","last_became_peered":"2024-04-13T03:13:36.769914+0000","last_unstale":"2024-04-13T03:13:36.772191+0000","last_undegraded":"2024-04-13T03:13:36.772191+0000","last_fullsized":"2024-04-13T03:13:36.772191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:54.641101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.753427+0000","last_change":"2024-04-13T03:13:36.753427+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.753427+0000","last_undegraded":"2024-04-13T03:13:36.753427+0000","last_fullsized":"2024-04-13T03:13:36.753427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.771317+0000","last_change":"2024-04-13T03:13:36.768818+0000","last_active":"2024-04-13T03:13:36.771317+0000","last_peered":"2024-04-13T03:13:36.771317+0000","last_clean":"2024-04-13T03:13:36.771317+0000","last_became_active":"2024-04-13T03:13:36.768492+0000","last_became_peered":"2024-04-13T03:13:36.768492+0000","last_unstale":"2024-04-13T03:13:36.771317+0000","last_undegraded":"2024-04-13T03:13:36.771317+0000","last_fullsized":"2024-04-13T03:13:36.771317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:49.380954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.759786+0000","last_change":"2024-04-13T03:13:36.759786+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.759786+0000","last_undegraded":"2024-04-13T03:13:36.759786+0000","last_fullsized":"2024-04-13T03:13:36.759786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770868+0000","last_change":"2024-04-13T03:13:36.766877+0000","last_active":"2024-04-13T03:13:36.770868+0000","last_peered":"2024-04-13T03:13:36.770868+0000","last_clean":"2024-04-13T03:13:36.770868+0000","last_became_active":"2024-04-13T03:13:36.765880+0000","last_became_peered":"2024-04-13T03:13:36.765880+0000","last_unstale":"2024-04-13T03:13:36.770868+0000","last_undegraded":"2024-04-13T03:13:36.770868+0000","last_fullsized":"2024-04-13T03:13:36.770868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:39:57.972949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.757534+0000","last_change":"2024-04-13T03:13:36.757534+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.757534+0000","last_undegraded":"2024-04-13T03:13:36.757534+0000","last_fullsized":"2024-04-13T03:13:36.757534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770312+0000","last_change":"2024-04-13T03:13:36.767371+0000","last_active":"2024-04-13T03:13:36.770312+0000","last_peered":"2024-04-13T03:13:36.770312+0000","last_clean":"2024-04-13T03:13:36.770312+0000","last_became_active":"2024-04-13T03:13:36.767116+0000","last_became_peered":"2024-04-13T03:13:36.767116+0000","last_unstale":"2024-04-13T03:13:36.770312+0000","last_undegraded":"2024-04-13T03:13:36.770312+0000","last_fullsized":"2024-04-13T03:13:36.770312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:05.559185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755043+0000","last_change":"2024-04-13T03:13:36.755043+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755043+0000","last_undegraded":"2024-04-13T03:13:36.755043+0000","last_fullsized":"2024-04-13T03:13:36.755043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.770829+0000","last_change":"2024-04-13T03:13:36.767237+0000","last_active":"2024-04-13T03:13:36.770829+0000","last_peered":"2024-04-13T03:13:36.770829+0000","last_clean":"2024-04-13T03:13:36.770829+0000","last_became_active":"2024-04-13T03:13:36.766972+0000","last_became_peered":"2024-04-13T03:13:36.766972+0000","last_unstale":"2024-04-13T03:13:36.770829+0000","last_undegraded":"2024-04-13T03:13:36.770829+0000","last_fullsized":"2024-04-13T03:13:36.770829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:54:40.326165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.755519+0000","last_change":"2024-04-13T03:13:36.755519+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.755519+0000","last_undegraded":"2024-04-13T03:13:36.755519+0000","last_fullsized":"2024-04-13T03:13:36.755519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.768808+0000","last_change":"2024-04-13T03:13:36.766340+0000","last_active":"2024-04-13T03:13:36.768808+0000","last_peered":"2024-04-13T03:13:36.768808+0000","last_clean":"2024-04-13T03:13:36.768808+0000","last_became_active":"2024-04-13T03:13:36.766071+0000","last_became_peered":"2024-04-13T03:13:36.766071+0000","last_unstale":"2024-04-13T03:13:36.768808+0000","last_undegraded":"2024-04-13T03:13:36.768808+0000","last_fullsized":"2024-04-13T03:13:36.768808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-04-13T03:13:36.761893+0000","last_change":"2024-04-13T03:13:36.761893+0000","last_active":"2024-04-13T03:13:36.744747+0000","last_peered":"2024-04-13T03:13:36.744747+0000","last_clean":"2024-04-13T03:13:36.744747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:13:36.761893+0000","last_undegraded":"2024-04-13T03:13:36.761893+0000","last_fullsized":"2024-04-13T03:13:36.761893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.773123+0000","last_change":"2024-04-13T03:13:36.772096+0000","last_active":"2024-04-13T03:13:36.773123+0000","last_peered":"2024-04-13T03:13:36.773123+0000","last_clean":"2024-04-13T03:13:36.773123+0000","last_became_active":"2024-04-13T03:13:36.771833+0000","last_became_peered":"2024-04-13T03:13:36.771833+0000","last_unstale":"2024-04-13T03:13:36.773123+0000","last_undegraded":"2024-04-13T03:13:36.773123+0000","last_fullsized":"2024-04-13T03:13:36.773123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:13:35.264150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-13T03:13:36.760793+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:13:36.760793+0000","last_peered":"2024-04-13T03:13:36.760793+0000","last_clean":"2024-04-13T03:13:36.760793+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:13:36.760793+0000","last_undegraded":"2024-04-13T03:13:36.760793+0000","last_fullsized":"2024-04-13T03:13:36.760793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":416145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343332,"statfs":{"total":96636764160,"available":96607571968,"internally_reserved":0,"allocated":1523712,"data_stored":1013180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:13:43.026 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:13:43.025+0000 7f56249e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:13:45.146 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:13:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.267+0000 7fe57b31e640 1 Processor -- start 2024-04-13T03:13:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.267+0000 7fe57b31e640 1 -- start start 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe57b31e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe57415ebf0 0x7fe57406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe57b31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 0x7fe57406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe57b31e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5740ce6f0 con 0x7fe5740b5bf0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe57b31e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe57415fa70 con 0x7fe57415ebf0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe57b31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe57415f420 con 0x7fe5740ce1f0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe5798bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe5740b5bf0 0x7fe57406e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44444/0 (socket says 172.21.15.106:44444) 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.268+0000 7fe5798bb640 1 -- 172.21.15.106:0/1594946591 learned_addr learned my addr 172.21.15.106:0/1594946591 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1594946591 >> v2:172.21.15.114:3300/0 conn(0x7fe57415ebf0 0x7fe57406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5788b9640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 0x7fe57406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe56bfff640 1 -- 172.21.15.106:0/1594946591 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272299925 0 0) 0x7fe5740ce6f0 con 0x7fe5740b5bf0 2024-04-13T03:13:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe56bfff640 1 -- 172.21.15.106:0/1594946591 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe560003520 con 0x7fe5740b5bf0 2024-04-13T03:13:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5790ba640 1 -- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 msgr2=0x7fe57406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 0x7fe57406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5790ba640 1 -- 172.21.15.106:0/1594946591 >> v1:172.21.15.106:6789/0 conn(0x7fe5740b5bf0 legacy=0x7fe57406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5790ba640 1 -- 172.21.15.106:0/1594946591 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57415f700 con 0x7fe57415ebf0 2024-04-13T03:13:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.269+0000 7fe5788b9640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 0x7fe57406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.270+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57406f780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5700097a0 tx=0x7fe57002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88686de15e6c7f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.270+0000 7fe56bfff640 1 -- 172.21.15.106:0/1594946591 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700315a0 con 0x7fe57415ebf0 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.270+0000 7fe56bfff640 1 -- 172.21.15.106:0/1594946591 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe570037070 con 0x7fe57415ebf0 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.270+0000 7fe56bfff640 1 -- 172.21.15.106:0/1594946591 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570031960 con 0x7fe57415ebf0 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 -- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 msgr2=0x7fe57406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57406f780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5700097a0 tx=0x7fe57002b840 comp rx=0 tx=0).stop 2024-04-13T03:13:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 -- 172.21.15.106:0/1594946591 shutdown_connections 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740ce1f0 0x7fe57406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1594946591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57406f780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 -- 172.21.15.106:0/1594946591 >> 172.21.15.106:0/1594946591 conn(0x7fe5740cd6d0 msgr2=0x7fe5740cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 -- 172.21.15.106:0/1594946591 shutdown_connections 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.271+0000 7fe57b31e640 1 -- 172.21.15.106:0/1594946591 wait complete. 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.272+0000 7fe57b31e640 1 Processor -- start 2024-04-13T03:13:45.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.272+0000 7fe57b31e640 1 -- start start 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740b5bf0 0x7fe57414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740b0650 con 0x7fe5740ce1f0 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740b04d0 con 0x7fe57415ebf0 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe57b31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe57415efd0 con 0x7fe5740b5bf0 2024-04-13T03:13:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe5788b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe5788b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55734/0 (socket says 172.21.15.106:55734) 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe5788b9640 1 -- 172.21.15.106:0/1094376829 learned_addr learned my addr 172.21.15.106:0/1094376829 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe5798bb640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.273+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740b5bf0 0x7fe57414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 -- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740b5bf0 msgr2=0x7fe57414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740b5bf0 0x7fe57414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 -- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 msgr2=0x7fe57415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 -- 172.21.15.106:0/1094376829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574160360 con 0x7fe5740ce1f0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe5788b9640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe56400ebe0 tx=0x7fe56400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564017070 con 0x7fe5740ce1f0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56400cc30 con 0x7fe5740ce1f0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564013470 con 0x7fe5740ce1f0 2024-04-13T03:13:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.274+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe574160640 con 0x7fe5740ce1f0 2024-04-13T03:13:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.275+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe574160c00 con 0x7fe5740ce1f0 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.276+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe574068a10 con 0x7fe5740ce1f0 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.276+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe56401a070 con 0x7fe5740ce1f0 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.277+0000 7fe569ffb640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 0x7fe55806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.277+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fe56408bf60 con 0x7fe5740ce1f0 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.277+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 0x7fe55806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.277+0000 7fe5790ba640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 0x7fe55806edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe570036000 tx=0x7fe57003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.282+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe564059b30 con 0x7fe5740ce1f0 2024-04-13T03:13:45.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.458+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe574160110 con 0x7fe55806c8f0 2024-04-13T03:13:45.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.464+0000 7fe569ffb640 1 -- 172.21.15.106:0/1094376829 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7fe574160110 con 0x7fe55806c8f0 2024-04-13T03:13:45.467 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:45.469 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:13:45.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 msgr2=0x7fe55806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 0x7fe55806edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe570036000 tx=0x7fe57003d040 comp rx=0 tx=0).stop 2024-04-13T03:13:45.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 msgr2=0x7fe57406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe56400ebe0 tx=0x7fe56400c520 comp rx=0 tx=0).stop 2024-04-13T03:13:45.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 shutdown_connections 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55806c8f0 0x7fe55806edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57415ebf0 0x7fe57415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5740ce1f0 0x7fe57406d550 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 --2- 172.21.15.106:0/1094376829 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5740b5bf0 0x7fe57414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 >> 172.21.15.106:0/1094376829 conn(0x7fe5740cd6d0 msgr2=0x7fe5740cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 shutdown_connections 2024-04-13T03:13:45.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.467+0000 7fe57b31e640 1 -- 172.21.15.106:0/1094376829 wait complete. 2024-04-13T03:13:45.479 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-13T03:13:44.417990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233200,"kb_used_data":8680,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754741520,"statfs":{"total":773094113280,"available":772855316480,"internally_reserved":0,"allocated":8888320,"data_stored":4678924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.667248"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182240+0000","last_change":"2024-04-13T03:13:37.778226+0000","last_active":"2024-04-13T03:13:39.182240+0000","last_peered":"2024-04-13T03:13:39.182240+0000","last_clean":"2024-04-13T03:13:39.182240+0000","last_became_active":"2024-04-13T03:13:37.777960+0000","last_became_peered":"2024-04-13T03:13:37.777960+0000","last_unstale":"2024-04-13T03:13:39.182240+0000","last_undegraded":"2024-04-13T03:13:39.182240+0000","last_fullsized":"2024-04-13T03:13:39.182240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:55.173779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771553+0000","last_change":"2024-04-13T03:13:37.773362+0000","last_active":"2024-04-13T03:13:38.771553+0000","last_peered":"2024-04-13T03:13:38.771553+0000","last_clean":"2024-04-13T03:13:38.771553+0000","last_became_active":"2024-04-13T03:13:37.773095+0000","last_became_peered":"2024-04-13T03:13:37.773095+0000","last_unstale":"2024-04-13T03:13:38.771553+0000","last_undegraded":"2024-04-13T03:13:38.771553+0000","last_fullsized":"2024-04-13T03:13:38.771553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:18:58.339707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770679+0000","last_change":"2024-04-13T03:13:37.771973+0000","last_active":"2024-04-13T03:13:38.770679+0000","last_peered":"2024-04-13T03:13:38.770679+0000","last_clean":"2024-04-13T03:13:38.770679+0000","last_became_active":"2024-04-13T03:13:37.771676+0000","last_became_peered":"2024-04-13T03:13:37.771676+0000","last_unstale":"2024-04-13T03:13:38.770679+0000","last_undegraded":"2024-04-13T03:13:38.770679+0000","last_fullsized":"2024-04-13T03:13:38.770679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:50.360463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164780+0000","last_change":"2024-04-13T03:13:37.765107+0000","last_active":"2024-04-13T03:13:39.164780+0000","last_peered":"2024-04-13T03:13:39.164780+0000","last_clean":"2024-04-13T03:13:39.164780+0000","last_became_active":"2024-04-13T03:13:37.764734+0000","last_became_peered":"2024-04-13T03:13:37.764734+0000","last_unstale":"2024-04-13T03:13:39.164780+0000","last_undegraded":"2024-04-13T03:13:39.164780+0000","last_fullsized":"2024-04-13T03:13:39.164780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:23:11.569920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181212+0000","last_change":"2024-04-13T03:13:37.771920+0000","last_active":"2024-04-13T03:13:39.181212+0000","last_peered":"2024-04-13T03:13:39.181212+0000","last_clean":"2024-04-13T03:13:39.181212+0000","last_became_active":"2024-04-13T03:13:37.771558+0000","last_became_peered":"2024-04-13T03:13:37.771558+0000","last_unstale":"2024-04-13T03:13:39.181212+0000","last_undegraded":"2024-04-13T03:13:39.181212+0000","last_fullsized":"2024-04-13T03:13:39.181212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:08:01.412728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181684+0000","last_change":"2024-04-13T03:13:37.773276+0000","last_active":"2024-04-13T03:13:39.181684+0000","last_peered":"2024-04-13T03:13:39.181684+0000","last_clean":"2024-04-13T03:13:39.181684+0000","last_became_active":"2024-04-13T03:13:37.772702+0000","last_became_peered":"2024-04-13T03:13:37.772702+0000","last_unstale":"2024-04-13T03:13:39.181684+0000","last_undegraded":"2024-04-13T03:13:39.181684+0000","last_fullsized":"2024-04-13T03:13:39.181684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:26:35.631651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770211+0000","last_change":"2024-04-13T03:13:37.782549+0000","last_active":"2024-04-13T03:13:38.770211+0000","last_peered":"2024-04-13T03:13:38.770211+0000","last_clean":"2024-04-13T03:13:38.770211+0000","last_became_active":"2024-04-13T03:13:37.782058+0000","last_became_peered":"2024-04-13T03:13:37.782058+0000","last_unstale":"2024-04-13T03:13:38.770211+0000","last_undegraded":"2024-04-13T03:13:38.770211+0000","last_fullsized":"2024-04-13T03:13:38.770211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:47.146435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.180972+0000","last_change":"2024-04-13T03:13:37.771170+0000","last_active":"2024-04-13T03:13:39.180972+0000","last_peered":"2024-04-13T03:13:39.180972+0000","last_clean":"2024-04-13T03:13:39.180972+0000","last_became_active":"2024-04-13T03:13:37.770872+0000","last_became_peered":"2024-04-13T03:13:37.770872+0000","last_unstale":"2024-04-13T03:13:39.180972+0000","last_undegraded":"2024-04-13T03:13:39.180972+0000","last_fullsized":"2024-04-13T03:13:39.180972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:16:25.255858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182557+0000","last_change":"2024-04-13T03:13:37.783428+0000","last_active":"2024-04-13T03:13:39.182557+0000","last_peered":"2024-04-13T03:13:39.182557+0000","last_clean":"2024-04-13T03:13:39.182557+0000","last_became_active":"2024-04-13T03:13:37.783139+0000","last_became_peered":"2024-04-13T03:13:37.783139+0000","last_unstale":"2024-04-13T03:13:39.182557+0000","last_undegraded":"2024-04-13T03:13:39.182557+0000","last_fullsized":"2024-04-13T03:13:39.182557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:46.856331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164806+0000","last_change":"2024-04-13T03:13:37.780058+0000","last_active":"2024-04-13T03:13:39.164806+0000","last_peered":"2024-04-13T03:13:39.164806+0000","last_clean":"2024-04-13T03:13:39.164806+0000","last_became_active":"2024-04-13T03:13:37.779785+0000","last_became_peered":"2024-04-13T03:13:37.779785+0000","last_unstale":"2024-04-13T03:13:39.164806+0000","last_undegraded":"2024-04-13T03:13:39.164806+0000","last_fullsized":"2024-04-13T03:13:39.164806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:23.407230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771548+0000","last_change":"2024-04-13T03:13:37.776356+0000","last_active":"2024-04-13T03:13:38.771548+0000","last_peered":"2024-04-13T03:13:38.771548+0000","last_clean":"2024-04-13T03:13:38.771548+0000","last_became_active":"2024-04-13T03:13:37.776076+0000","last_became_peered":"2024-04-13T03:13:37.776076+0000","last_unstale":"2024-04-13T03:13:38.771548+0000","last_undegraded":"2024-04-13T03:13:38.771548+0000","last_fullsized":"2024-04-13T03:13:38.771548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:23.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164931+0000","last_change":"2024-04-13T03:13:37.773746+0000","last_active":"2024-04-13T03:13:39.164931+0000","last_peered":"2024-04-13T03:13:39.164931+0000","last_clean":"2024-04-13T03:13:39.164931+0000","last_became_active":"2024-04-13T03:13:37.773485+0000","last_became_peered":"2024-04-13T03:13:37.773485+0000","last_unstale":"2024-04-13T03:13:39.164931+0000","last_undegraded":"2024-04-13T03:13:39.164931+0000","last_fullsized":"2024-04-13T03:13:39.164931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:42:35.764882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181358+0000","last_change":"2024-04-13T03:13:37.764397+0000","last_active":"2024-04-13T03:13:39.181358+0000","last_peered":"2024-04-13T03:13:39.181358+0000","last_clean":"2024-04-13T03:13:39.181358+0000","last_became_active":"2024-04-13T03:13:37.764130+0000","last_became_peered":"2024-04-13T03:13:37.764130+0000","last_unstale":"2024-04-13T03:13:39.181358+0000","last_undegraded":"2024-04-13T03:13:39.181358+0000","last_fullsized":"2024-04-13T03:13:39.181358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:30.854601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181822+0000","last_change":"2024-04-13T03:13:37.773497+0000","last_active":"2024-04-13T03:13:39.181822+0000","last_peered":"2024-04-13T03:13:39.181822+0000","last_clean":"2024-04-13T03:13:39.181822+0000","last_became_active":"2024-04-13T03:13:37.773041+0000","last_became_peered":"2024-04-13T03:13:37.773041+0000","last_unstale":"2024-04-13T03:13:39.181822+0000","last_undegraded":"2024-04-13T03:13:39.181822+0000","last_fullsized":"2024-04-13T03:13:39.181822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:18:58.339707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770994+0000","last_change":"2024-04-13T03:13:37.772067+0000","last_active":"2024-04-13T03:13:38.770994+0000","last_peered":"2024-04-13T03:13:38.770994+0000","last_clean":"2024-04-13T03:13:38.770994+0000","last_became_active":"2024-04-13T03:13:37.770942+0000","last_became_peered":"2024-04-13T03:13:37.770942+0000","last_unstale":"2024-04-13T03:13:38.770994+0000","last_undegraded":"2024-04-13T03:13:38.770994+0000","last_fullsized":"2024-04-13T03:13:38.770994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:50:18.077339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181618+0000","last_change":"2024-04-13T03:13:37.769382+0000","last_active":"2024-04-13T03:13:39.181618+0000","last_peered":"2024-04-13T03:13:39.181618+0000","last_clean":"2024-04-13T03:13:39.181618+0000","last_became_active":"2024-04-13T03:13:37.769105+0000","last_became_peered":"2024-04-13T03:13:37.769105+0000","last_unstale":"2024-04-13T03:13:39.181618+0000","last_undegraded":"2024-04-13T03:13:39.181618+0000","last_fullsized":"2024-04-13T03:13:39.181618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:04:13.465101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181215+0000","last_change":"2024-04-13T03:13:37.776685+0000","last_active":"2024-04-13T03:13:39.181215+0000","last_peered":"2024-04-13T03:13:39.181215+0000","last_clean":"2024-04-13T03:13:39.181215+0000","last_became_active":"2024-04-13T03:13:37.776431+0000","last_became_peered":"2024-04-13T03:13:37.776431+0000","last_unstale":"2024-04-13T03:13:39.181215+0000","last_undegraded":"2024-04-13T03:13:39.181215+0000","last_fullsized":"2024-04-13T03:13:39.181215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:26.632860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770936+0000","last_change":"2024-04-13T03:13:37.772760+0000","last_active":"2024-04-13T03:13:38.770936+0000","last_peered":"2024-04-13T03:13:38.770936+0000","last_clean":"2024-04-13T03:13:38.770936+0000","last_became_active":"2024-04-13T03:13:37.772448+0000","last_became_peered":"2024-04-13T03:13:37.772448+0000","last_unstale":"2024-04-13T03:13:38.770936+0000","last_undegraded":"2024-04-13T03:13:38.770936+0000","last_fullsized":"2024-04-13T03:13:38.770936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:55.645743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181596+0000","last_change":"2024-04-13T03:13:37.769599+0000","last_active":"2024-04-13T03:13:39.181596+0000","last_peered":"2024-04-13T03:13:39.181596+0000","last_clean":"2024-04-13T03:13:39.181596+0000","last_became_active":"2024-04-13T03:13:37.769313+0000","last_became_peered":"2024-04-13T03:13:37.769313+0000","last_unstale":"2024-04-13T03:13:39.181596+0000","last_undegraded":"2024-04-13T03:13:39.181596+0000","last_fullsized":"2024-04-13T03:13:39.181596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:21:05.693722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181558+0000","last_change":"2024-04-13T03:13:37.763705+0000","last_active":"2024-04-13T03:13:39.181558+0000","last_peered":"2024-04-13T03:13:39.181558+0000","last_clean":"2024-04-13T03:13:39.181558+0000","last_became_active":"2024-04-13T03:13:37.763428+0000","last_became_peered":"2024-04-13T03:13:37.763428+0000","last_unstale":"2024-04-13T03:13:39.181558+0000","last_undegraded":"2024-04-13T03:13:39.181558+0000","last_fullsized":"2024-04-13T03:13:39.181558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:34.473298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182008+0000","last_change":"2024-04-13T03:13:37.773974+0000","last_active":"2024-04-13T03:13:39.182008+0000","last_peered":"2024-04-13T03:13:39.182008+0000","last_clean":"2024-04-13T03:13:39.182008+0000","last_became_active":"2024-04-13T03:13:37.773645+0000","last_became_peered":"2024-04-13T03:13:37.773645+0000","last_unstale":"2024-04-13T03:13:39.182008+0000","last_undegraded":"2024-04-13T03:13:39.182008+0000","last_fullsized":"2024-04-13T03:13:39.182008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:47.146435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181370+0000","last_change":"2024-04-13T03:13:37.781331+0000","last_active":"2024-04-13T03:13:39.181370+0000","last_peered":"2024-04-13T03:13:39.181370+0000","last_clean":"2024-04-13T03:13:39.181370+0000","last_became_active":"2024-04-13T03:13:37.781038+0000","last_became_peered":"2024-04-13T03:13:37.781038+0000","last_unstale":"2024-04-13T03:13:39.181370+0000","last_undegraded":"2024-04-13T03:13:39.181370+0000","last_fullsized":"2024-04-13T03:13:39.181370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:29:16.589029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182283+0000","last_change":"2024-04-13T03:13:37.774564+0000","last_active":"2024-04-13T03:13:39.182283+0000","last_peered":"2024-04-13T03:13:39.182283+0000","last_clean":"2024-04-13T03:13:39.182283+0000","last_became_active":"2024-04-13T03:13:37.773946+0000","last_became_peered":"2024-04-13T03:13:37.773946+0000","last_unstale":"2024-04-13T03:13:39.182283+0000","last_undegraded":"2024-04-13T03:13:39.182283+0000","last_fullsized":"2024-04-13T03:13:39.182283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:55.645743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770924+0000","last_change":"2024-04-13T03:13:37.782314+0000","last_active":"2024-04-13T03:13:38.770924+0000","last_peered":"2024-04-13T03:13:38.770924+0000","last_clean":"2024-04-13T03:13:38.770924+0000","last_became_active":"2024-04-13T03:13:37.782080+0000","last_became_peered":"2024-04-13T03:13:37.782080+0000","last_unstale":"2024-04-13T03:13:38.770924+0000","last_undegraded":"2024-04-13T03:13:38.770924+0000","last_fullsized":"2024-04-13T03:13:38.770924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:47.442657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771582+0000","last_change":"2024-04-13T03:13:37.781408+0000","last_active":"2024-04-13T03:13:38.771582+0000","last_peered":"2024-04-13T03:13:38.771582+0000","last_clean":"2024-04-13T03:13:38.771582+0000","last_became_active":"2024-04-13T03:13:37.781123+0000","last_became_peered":"2024-04-13T03:13:37.781123+0000","last_unstale":"2024-04-13T03:13:38.771582+0000","last_undegraded":"2024-04-13T03:13:38.771582+0000","last_fullsized":"2024-04-13T03:13:38.771582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:44:45.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164846+0000","last_change":"2024-04-13T03:13:37.771572+0000","last_active":"2024-04-13T03:13:39.164846+0000","last_peered":"2024-04-13T03:13:39.164846+0000","last_clean":"2024-04-13T03:13:39.164846+0000","last_became_active":"2024-04-13T03:13:37.771276+0000","last_became_peered":"2024-04-13T03:13:37.771276+0000","last_unstale":"2024-04-13T03:13:39.164846+0000","last_undegraded":"2024-04-13T03:13:39.164846+0000","last_fullsized":"2024-04-13T03:13:39.164846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:56:42.979143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182121+0000","last_change":"2024-04-13T03:13:37.775002+0000","last_active":"2024-04-13T03:13:39.182121+0000","last_peered":"2024-04-13T03:13:39.182121+0000","last_clean":"2024-04-13T03:13:39.182121+0000","last_became_active":"2024-04-13T03:13:37.774718+0000","last_became_peered":"2024-04-13T03:13:37.774718+0000","last_unstale":"2024-04-13T03:13:39.182121+0000","last_undegraded":"2024-04-13T03:13:39.182121+0000","last_fullsized":"2024-04-13T03:13:39.182121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:12.851738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181591+0000","last_change":"2024-04-13T03:13:37.768546+0000","last_active":"2024-04-13T03:13:39.181591+0000","last_peered":"2024-04-13T03:13:39.181591+0000","last_clean":"2024-04-13T03:13:39.181591+0000","last_became_active":"2024-04-13T03:13:37.767725+0000","last_became_peered":"2024-04-13T03:13:37.767725+0000","last_unstale":"2024-04-13T03:13:39.181591+0000","last_undegraded":"2024-04-13T03:13:39.181591+0000","last_fullsized":"2024-04-13T03:13:39.181591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:13:22.506185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771444+0000","last_change":"2024-04-13T03:13:37.776496+0000","last_active":"2024-04-13T03:13:38.771444+0000","last_peered":"2024-04-13T03:13:38.771444+0000","last_clean":"2024-04-13T03:13:38.771444+0000","last_became_active":"2024-04-13T03:13:37.776247+0000","last_became_peered":"2024-04-13T03:13:37.776247+0000","last_unstale":"2024-04-13T03:13:38.771444+0000","last_undegraded":"2024-04-13T03:13:38.771444+0000","last_fullsized":"2024-04-13T03:13:38.771444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:21:05.693722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181831+0000","last_change":"2024-04-13T03:13:37.773653+0000","last_active":"2024-04-13T03:13:39.181831+0000","last_peered":"2024-04-13T03:13:39.181831+0000","last_clean":"2024-04-13T03:13:39.181831+0000","last_became_active":"2024-04-13T03:13:37.773370+0000","last_became_peered":"2024-04-13T03:13:37.773370+0000","last_unstale":"2024-04-13T03:13:39.181831+0000","last_undegraded":"2024-04-13T03:13:39.181831+0000","last_fullsized":"2024-04-13T03:13:39.181831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:35.352254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770401+0000","last_change":"2024-04-13T03:13:37.778453+0000","last_active":"2024-04-13T03:13:38.770401+0000","last_peered":"2024-04-13T03:13:38.770401+0000","last_clean":"2024-04-13T03:13:38.770401+0000","last_became_active":"2024-04-13T03:13:37.777560+0000","last_became_peered":"2024-04-13T03:13:37.777560+0000","last_unstale":"2024-04-13T03:13:38.770401+0000","last_undegraded":"2024-04-13T03:13:38.770401+0000","last_fullsized":"2024-04-13T03:13:38.770401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:44.594144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181783+0000","last_change":"2024-04-13T03:13:37.778837+0000","last_active":"2024-04-13T03:13:39.181783+0000","last_peered":"2024-04-13T03:13:39.181783+0000","last_clean":"2024-04-13T03:13:39.181783+0000","last_became_active":"2024-04-13T03:13:37.778559+0000","last_became_peered":"2024-04-13T03:13:37.778559+0000","last_unstale":"2024-04-13T03:13:39.181783+0000","last_undegraded":"2024-04-13T03:13:39.181783+0000","last_fullsized":"2024-04-13T03:13:39.181783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:48.666358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770362+0000","last_change":"2024-04-13T03:13:37.764769+0000","last_active":"2024-04-13T03:13:38.770362+0000","last_peered":"2024-04-13T03:13:38.770362+0000","last_clean":"2024-04-13T03:13:38.770362+0000","last_became_active":"2024-04-13T03:13:37.764531+0000","last_became_peered":"2024-04-13T03:13:37.764531+0000","last_unstale":"2024-04-13T03:13:38.770362+0000","last_undegraded":"2024-04-13T03:13:38.770362+0000","last_fullsized":"2024-04-13T03:13:38.770362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:52.600814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181314+0000","last_change":"2024-04-13T03:13:36.767960+0000","last_active":"2024-04-13T03:13:39.181314+0000","last_peered":"2024-04-13T03:13:39.181314+0000","last_clean":"2024-04-13T03:13:39.181314+0000","last_became_active":"2024-04-13T03:13:36.767704+0000","last_became_peered":"2024-04-13T03:13:36.767704+0000","last_unstale":"2024-04-13T03:13:39.181314+0000","last_undegraded":"2024-04-13T03:13:39.181314+0000","last_fullsized":"2024-04-13T03:13:39.181314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:11:04.668603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771139+0000","last_change":"2024-04-13T03:13:37.778836+0000","last_active":"2024-04-13T03:13:38.771139+0000","last_peered":"2024-04-13T03:13:38.771139+0000","last_clean":"2024-04-13T03:13:38.771139+0000","last_became_active":"2024-04-13T03:13:37.777555+0000","last_became_peered":"2024-04-13T03:13:37.777555+0000","last_unstale":"2024-04-13T03:13:38.771139+0000","last_undegraded":"2024-04-13T03:13:38.771139+0000","last_fullsized":"2024-04-13T03:13:38.771139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:11:31.718189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182138+0000","last_change":"2024-04-13T03:13:36.769252+0000","last_active":"2024-04-13T03:13:39.182138+0000","last_peered":"2024-04-13T03:13:39.182138+0000","last_clean":"2024-04-13T03:13:39.182138+0000","last_became_active":"2024-04-13T03:13:36.768970+0000","last_became_peered":"2024-04-13T03:13:36.768970+0000","last_unstale":"2024-04-13T03:13:39.182138+0000","last_undegraded":"2024-04-13T03:13:39.182138+0000","last_fullsized":"2024-04-13T03:13:39.182138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:11:00.169744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181820+0000","last_change":"2024-04-13T03:13:37.778888+0000","last_active":"2024-04-13T03:13:39.181820+0000","last_peered":"2024-04-13T03:13:39.181820+0000","last_clean":"2024-04-13T03:13:39.181820+0000","last_became_active":"2024-04-13T03:13:37.778620+0000","last_became_peered":"2024-04-13T03:13:37.778620+0000","last_unstale":"2024-04-13T03:13:39.181820+0000","last_undegraded":"2024-04-13T03:13:39.181820+0000","last_fullsized":"2024-04-13T03:13:39.181820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:50.385596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182142+0000","last_change":"2024-04-13T03:13:36.767892+0000","last_active":"2024-04-13T03:13:39.182142+0000","last_peered":"2024-04-13T03:13:39.182142+0000","last_clean":"2024-04-13T03:13:39.182142+0000","last_became_active":"2024-04-13T03:13:36.767618+0000","last_became_peered":"2024-04-13T03:13:36.767618+0000","last_unstale":"2024-04-13T03:13:39.182142+0000","last_undegraded":"2024-04-13T03:13:39.182142+0000","last_fullsized":"2024-04-13T03:13:39.182142+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:00:29.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182169+0000","last_change":"2024-04-13T03:13:37.773740+0000","last_active":"2024-04-13T03:13:39.182169+0000","last_peered":"2024-04-13T03:13:39.182169+0000","last_clean":"2024-04-13T03:13:39.182169+0000","last_became_active":"2024-04-13T03:13:37.773294+0000","last_became_peered":"2024-04-13T03:13:37.773294+0000","last_unstale":"2024-04-13T03:13:39.182169+0000","last_undegraded":"2024-04-13T03:13:39.182169+0000","last_fullsized":"2024-04-13T03:13:39.182169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:40.658209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770353+0000","last_change":"2024-04-13T03:13:36.769804+0000","last_active":"2024-04-13T03:13:38.770353+0000","last_peered":"2024-04-13T03:13:38.770353+0000","last_clean":"2024-04-13T03:13:38.770353+0000","last_became_active":"2024-04-13T03:13:36.769493+0000","last_became_peered":"2024-04-13T03:13:36.769493+0000","last_unstale":"2024-04-13T03:13:38.770353+0000","last_undegraded":"2024-04-13T03:13:38.770353+0000","last_fullsized":"2024-04-13T03:13:38.770353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:15:56.727538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771115+0000","last_change":"2024-04-13T03:13:37.771070+0000","last_active":"2024-04-13T03:13:38.771115+0000","last_peered":"2024-04-13T03:13:38.771115+0000","last_clean":"2024-04-13T03:13:38.771115+0000","last_became_active":"2024-04-13T03:13:37.770170+0000","last_became_peered":"2024-04-13T03:13:37.770170+0000","last_unstale":"2024-04-13T03:13:38.771115+0000","last_undegraded":"2024-04-13T03:13:38.771115+0000","last_fullsized":"2024-04-13T03:13:38.771115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:15:55.292034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182461+0000","last_change":"2024-04-13T03:13:36.768909+0000","last_active":"2024-04-13T03:13:39.182461+0000","last_peered":"2024-04-13T03:13:39.182461+0000","last_clean":"2024-04-13T03:13:39.182461+0000","last_became_active":"2024-04-13T03:13:36.768554+0000","last_became_peered":"2024-04-13T03:13:36.768554+0000","last_unstale":"2024-04-13T03:13:39.182461+0000","last_undegraded":"2024-04-13T03:13:39.182461+0000","last_fullsized":"2024-04-13T03:13:39.182461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:18:45.238629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771694+0000","last_change":"2024-04-13T03:13:37.773974+0000","last_active":"2024-04-13T03:13:38.771694+0000","last_peered":"2024-04-13T03:13:38.771694+0000","last_clean":"2024-04-13T03:13:38.771694+0000","last_became_active":"2024-04-13T03:13:37.773713+0000","last_became_peered":"2024-04-13T03:13:37.773713+0000","last_unstale":"2024-04-13T03:13:38.771694+0000","last_undegraded":"2024-04-13T03:13:38.771694+0000","last_fullsized":"2024-04-13T03:13:38.771694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:35:17.605572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164564+0000","last_change":"2024-04-13T03:13:36.767672+0000","last_active":"2024-04-13T03:13:39.164564+0000","last_peered":"2024-04-13T03:13:39.164564+0000","last_clean":"2024-04-13T03:13:39.164564+0000","last_became_active":"2024-04-13T03:13:36.767312+0000","last_became_peered":"2024-04-13T03:13:36.767312+0000","last_unstale":"2024-04-13T03:13:39.164564+0000","last_undegraded":"2024-04-13T03:13:39.164564+0000","last_fullsized":"2024-04-13T03:13:39.164564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:43.589502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182536+0000","last_change":"2024-04-13T03:13:37.766258+0000","last_active":"2024-04-13T03:13:39.182536+0000","last_peered":"2024-04-13T03:13:39.182536+0000","last_clean":"2024-04-13T03:13:39.182536+0000","last_became_active":"2024-04-13T03:13:37.765399+0000","last_became_peered":"2024-04-13T03:13:37.765399+0000","last_unstale":"2024-04-13T03:13:39.182536+0000","last_undegraded":"2024-04-13T03:13:39.182536+0000","last_fullsized":"2024-04-13T03:13:39.182536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:54.539747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771224+0000","last_change":"2024-04-13T03:13:36.765735+0000","last_active":"2024-04-13T03:13:38.771224+0000","last_peered":"2024-04-13T03:13:38.771224+0000","last_clean":"2024-04-13T03:13:38.771224+0000","last_became_active":"2024-04-13T03:13:36.765485+0000","last_became_peered":"2024-04-13T03:13:36.765485+0000","last_unstale":"2024-04-13T03:13:38.771224+0000","last_undegraded":"2024-04-13T03:13:38.771224+0000","last_fullsized":"2024-04-13T03:13:38.771224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:23:10.565278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770514+0000","last_change":"2024-04-13T03:13:37.782820+0000","last_active":"2024-04-13T03:13:38.770514+0000","last_peered":"2024-04-13T03:13:38.770514+0000","last_clean":"2024-04-13T03:13:38.770514+0000","last_became_active":"2024-04-13T03:13:37.782016+0000","last_became_peered":"2024-04-13T03:13:37.782016+0000","last_unstale":"2024-04-13T03:13:38.770514+0000","last_undegraded":"2024-04-13T03:13:38.770514+0000","last_fullsized":"2024-04-13T03:13:38.770514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:50.360463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.180983+0000","last_change":"2024-04-13T03:13:36.769186+0000","last_active":"2024-04-13T03:13:39.180983+0000","last_peered":"2024-04-13T03:13:39.180983+0000","last_clean":"2024-04-13T03:13:39.180983+0000","last_became_active":"2024-04-13T03:13:36.768951+0000","last_became_peered":"2024-04-13T03:13:36.768951+0000","last_unstale":"2024-04-13T03:13:39.180983+0000","last_undegraded":"2024-04-13T03:13:39.180983+0000","last_fullsized":"2024-04-13T03:13:39.180983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:51.989067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771089+0000","last_change":"2024-04-13T03:13:37.774237+0000","last_active":"2024-04-13T03:13:38.771089+0000","last_peered":"2024-04-13T03:13:38.771089+0000","last_clean":"2024-04-13T03:13:38.771089+0000","last_became_active":"2024-04-13T03:13:37.773621+0000","last_became_peered":"2024-04-13T03:13:37.773621+0000","last_unstale":"2024-04-13T03:13:38.771089+0000","last_undegraded":"2024-04-13T03:13:38.771089+0000","last_fullsized":"2024-04-13T03:13:38.771089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:40:25.822994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182163+0000","last_change":"2024-04-13T03:13:36.769948+0000","last_active":"2024-04-13T03:13:39.182163+0000","last_peered":"2024-04-13T03:13:39.182163+0000","last_clean":"2024-04-13T03:13:39.182163+0000","last_became_active":"2024-04-13T03:13:36.769546+0000","last_became_peered":"2024-04-13T03:13:36.769546+0000","last_unstale":"2024-04-13T03:13:39.182163+0000","last_undegraded":"2024-04-13T03:13:39.182163+0000","last_fullsized":"2024-04-13T03:13:39.182163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:37.388461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164695+0000","last_change":"2024-04-13T03:13:37.764404+0000","last_active":"2024-04-13T03:13:39.164695+0000","last_peered":"2024-04-13T03:13:39.164695+0000","last_clean":"2024-04-13T03:13:39.164695+0000","last_became_active":"2024-04-13T03:13:37.764132+0000","last_became_peered":"2024-04-13T03:13:37.764132+0000","last_unstale":"2024-04-13T03:13:39.164695+0000","last_undegraded":"2024-04-13T03:13:39.164695+0000","last_fullsized":"2024-04-13T03:13:39.164695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:38.393103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164945+0000","last_change":"2024-04-13T03:13:36.766240+0000","last_active":"2024-04-13T03:13:39.164945+0000","last_peered":"2024-04-13T03:13:39.164945+0000","last_clean":"2024-04-13T03:13:39.164945+0000","last_became_active":"2024-04-13T03:13:36.763111+0000","last_became_peered":"2024-04-13T03:13:36.763111+0000","last_unstale":"2024-04-13T03:13:39.164945+0000","last_undegraded":"2024-04-13T03:13:39.164945+0000","last_fullsized":"2024-04-13T03:13:39.164945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:49.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181866+0000","last_change":"2024-04-13T03:13:37.769001+0000","last_active":"2024-04-13T03:13:39.181866+0000","last_peered":"2024-04-13T03:13:39.181866+0000","last_clean":"2024-04-13T03:13:39.181866+0000","last_became_active":"2024-04-13T03:13:37.768463+0000","last_became_peered":"2024-04-13T03:13:37.768463+0000","last_unstale":"2024-04-13T03:13:39.181866+0000","last_undegraded":"2024-04-13T03:13:39.181866+0000","last_fullsized":"2024-04-13T03:13:39.181866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:44:45.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181467+0000","last_change":"2024-04-13T03:13:36.767663+0000","last_active":"2024-04-13T03:13:39.181467+0000","last_peered":"2024-04-13T03:13:39.181467+0000","last_clean":"2024-04-13T03:13:39.181467+0000","last_became_active":"2024-04-13T03:13:36.767407+0000","last_became_peered":"2024-04-13T03:13:36.767407+0000","last_unstale":"2024-04-13T03:13:39.181467+0000","last_undegraded":"2024-04-13T03:13:39.181467+0000","last_fullsized":"2024-04-13T03:13:39.181467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:16:40.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770660+0000","last_change":"2024-04-13T03:13:37.771803+0000","last_active":"2024-04-13T03:13:38.770660+0000","last_peered":"2024-04-13T03:13:38.770660+0000","last_clean":"2024-04-13T03:13:38.770660+0000","last_became_active":"2024-04-13T03:13:37.771530+0000","last_became_peered":"2024-04-13T03:13:37.771530+0000","last_unstale":"2024-04-13T03:13:38.770660+0000","last_undegraded":"2024-04-13T03:13:38.770660+0000","last_fullsized":"2024-04-13T03:13:38.770660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:50.385596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770458+0000","last_change":"2024-04-13T03:13:36.768069+0000","last_active":"2024-04-13T03:13:38.770458+0000","last_peered":"2024-04-13T03:13:38.770458+0000","last_clean":"2024-04-13T03:13:38.770458+0000","last_became_active":"2024-04-13T03:13:36.767790+0000","last_became_peered":"2024-04-13T03:13:36.767790+0000","last_unstale":"2024-04-13T03:13:38.770458+0000","last_undegraded":"2024-04-13T03:13:38.770458+0000","last_fullsized":"2024-04-13T03:13:38.770458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:44:36.244015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164962+0000","last_change":"2024-04-13T03:13:37.762685+0000","last_active":"2024-04-13T03:13:39.164962+0000","last_peered":"2024-04-13T03:13:39.164962+0000","last_clean":"2024-04-13T03:13:39.164962+0000","last_became_active":"2024-04-13T03:13:37.762388+0000","last_became_peered":"2024-04-13T03:13:37.762388+0000","last_unstale":"2024-04-13T03:13:39.164962+0000","last_undegraded":"2024-04-13T03:13:39.164962+0000","last_fullsized":"2024-04-13T03:13:39.164962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:03.130852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164593+0000","last_change":"2024-04-13T03:13:36.766629+0000","last_active":"2024-04-13T03:13:39.164593+0000","last_peered":"2024-04-13T03:13:39.164593+0000","last_clean":"2024-04-13T03:13:39.164593+0000","last_became_active":"2024-04-13T03:13:36.766361+0000","last_became_peered":"2024-04-13T03:13:36.766361+0000","last_unstale":"2024-04-13T03:13:39.164593+0000","last_undegraded":"2024-04-13T03:13:39.164593+0000","last_fullsized":"2024-04-13T03:13:39.164593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:49.380954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770213+0000","last_change":"2024-04-13T03:13:37.775890+0000","last_active":"2024-04-13T03:13:38.770213+0000","last_peered":"2024-04-13T03:13:38.770213+0000","last_clean":"2024-04-13T03:13:38.770213+0000","last_became_active":"2024-04-13T03:13:37.775391+0000","last_became_peered":"2024-04-13T03:13:37.775391+0000","last_unstale":"2024-04-13T03:13:38.770213+0000","last_undegraded":"2024-04-13T03:13:38.770213+0000","last_fullsized":"2024-04-13T03:13:38.770213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:44.594144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181925+0000","last_change":"2024-04-13T03:13:36.770589+0000","last_active":"2024-04-13T03:13:39.181925+0000","last_peered":"2024-04-13T03:13:39.181925+0000","last_clean":"2024-04-13T03:13:39.181925+0000","last_became_active":"2024-04-13T03:13:36.770281+0000","last_became_peered":"2024-04-13T03:13:36.770281+0000","last_unstale":"2024-04-13T03:13:39.181925+0000","last_undegraded":"2024-04-13T03:13:39.181925+0000","last_fullsized":"2024-04-13T03:13:39.181925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:49.280931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770763+0000","last_change":"2024-04-13T03:13:37.780477+0000","last_active":"2024-04-13T03:13:38.770763+0000","last_peered":"2024-04-13T03:13:38.770763+0000","last_clean":"2024-04-13T03:13:38.770763+0000","last_became_active":"2024-04-13T03:13:37.780182+0000","last_became_peered":"2024-04-13T03:13:37.780182+0000","last_unstale":"2024-04-13T03:13:38.770763+0000","last_undegraded":"2024-04-13T03:13:38.770763+0000","last_fullsized":"2024-04-13T03:13:38.770763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:43:50.285573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181078+0000","last_change":"2024-04-13T03:13:36.768599+0000","last_active":"2024-04-13T03:13:39.181078+0000","last_peered":"2024-04-13T03:13:39.181078+0000","last_clean":"2024-04-13T03:13:39.181078+0000","last_became_active":"2024-04-13T03:13:36.768343+0000","last_became_peered":"2024-04-13T03:13:36.768343+0000","last_unstale":"2024-04-13T03:13:39.181078+0000","last_undegraded":"2024-04-13T03:13:39.181078+0000","last_fullsized":"2024-04-13T03:13:39.181078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:30:40.503400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770637+0000","last_change":"2024-04-13T03:13:37.782198+0000","last_active":"2024-04-13T03:13:38.770637+0000","last_peered":"2024-04-13T03:13:38.770637+0000","last_clean":"2024-04-13T03:13:38.770637+0000","last_became_active":"2024-04-13T03:13:37.781782+0000","last_became_peered":"2024-04-13T03:13:37.781782+0000","last_unstale":"2024-04-13T03:13:38.770637+0000","last_undegraded":"2024-04-13T03:13:38.770637+0000","last_fullsized":"2024-04-13T03:13:38.770637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:02:55.645743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771182+0000","last_change":"2024-04-13T03:13:36.764197+0000","last_active":"2024-04-13T03:13:38.771182+0000","last_peered":"2024-04-13T03:13:38.771182+0000","last_clean":"2024-04-13T03:13:38.771182+0000","last_became_active":"2024-04-13T03:13:36.763920+0000","last_became_peered":"2024-04-13T03:13:36.763920+0000","last_unstale":"2024-04-13T03:13:38.771182+0000","last_undegraded":"2024-04-13T03:13:38.771182+0000","last_fullsized":"2024-04-13T03:13:38.771182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:37.388461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.180963+0000","last_change":"2024-04-13T03:13:37.768165+0000","last_active":"2024-04-13T03:13:39.180963+0000","last_peered":"2024-04-13T03:13:39.180963+0000","last_clean":"2024-04-13T03:13:39.180963+0000","last_became_active":"2024-04-13T03:13:37.767838+0000","last_became_peered":"2024-04-13T03:13:37.767838+0000","last_unstale":"2024-04-13T03:13:39.180963+0000","last_undegraded":"2024-04-13T03:13:39.180963+0000","last_fullsized":"2024-04-13T03:13:39.180963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:39.438711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182334+0000","last_change":"2024-04-13T03:13:36.770260+0000","last_active":"2024-04-13T03:13:39.182334+0000","last_peered":"2024-04-13T03:13:39.182334+0000","last_clean":"2024-04-13T03:13:39.182334+0000","last_became_active":"2024-04-13T03:13:36.769888+0000","last_became_peered":"2024-04-13T03:13:36.769888+0000","last_unstale":"2024-04-13T03:13:39.182334+0000","last_undegraded":"2024-04-13T03:13:39.182334+0000","last_fullsized":"2024-04-13T03:13:39.182334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:23:10.565278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181068+0000","last_change":"2024-04-13T03:13:37.781245+0000","last_active":"2024-04-13T03:13:39.181068+0000","last_peered":"2024-04-13T03:13:39.181068+0000","last_clean":"2024-04-13T03:13:39.181068+0000","last_became_active":"2024-04-13T03:13:37.780980+0000","last_became_peered":"2024-04-13T03:13:37.780980+0000","last_unstale":"2024-04-13T03:13:39.181068+0000","last_undegraded":"2024-04-13T03:13:39.181068+0000","last_fullsized":"2024-04-13T03:13:39.181068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:48.638422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182536+0000","last_change":"2024-04-13T03:13:36.765493+0000","last_active":"2024-04-13T03:13:39.182536+0000","last_peered":"2024-04-13T03:13:39.182536+0000","last_clean":"2024-04-13T03:13:39.182536+0000","last_became_active":"2024-04-13T03:13:36.765174+0000","last_became_peered":"2024-04-13T03:13:36.765174+0000","last_unstale":"2024-04-13T03:13:39.182536+0000","last_undegraded":"2024-04-13T03:13:39.182536+0000","last_fullsized":"2024-04-13T03:13:39.182536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181670+0000","last_change":"2024-04-13T03:13:37.774183+0000","last_active":"2024-04-13T03:13:39.181670+0000","last_peered":"2024-04-13T03:13:39.181670+0000","last_clean":"2024-04-13T03:13:39.181670+0000","last_became_active":"2024-04-13T03:13:37.773922+0000","last_became_peered":"2024-04-13T03:13:37.773922+0000","last_unstale":"2024-04-13T03:13:39.181670+0000","last_undegraded":"2024-04-13T03:13:39.181670+0000","last_fullsized":"2024-04-13T03:13:39.181670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:02.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181568+0000","last_change":"2024-04-13T03:13:36.769841+0000","last_active":"2024-04-13T03:13:39.181568+0000","last_peered":"2024-04-13T03:13:39.181568+0000","last_clean":"2024-04-13T03:13:39.181568+0000","last_became_active":"2024-04-13T03:13:36.769598+0000","last_became_peered":"2024-04-13T03:13:36.769598+0000","last_unstale":"2024-04-13T03:13:39.181568+0000","last_undegraded":"2024-04-13T03:13:39.181568+0000","last_fullsized":"2024-04-13T03:13:39.181568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:22:51.665614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181852+0000","last_change":"2024-04-13T03:13:37.768296+0000","last_active":"2024-04-13T03:13:39.181852+0000","last_peered":"2024-04-13T03:13:39.181852+0000","last_clean":"2024-04-13T03:13:39.181852+0000","last_became_active":"2024-04-13T03:13:37.767705+0000","last_became_peered":"2024-04-13T03:13:37.767705+0000","last_unstale":"2024-04-13T03:13:39.181852+0000","last_undegraded":"2024-04-13T03:13:39.181852+0000","last_fullsized":"2024-04-13T03:13:39.181852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:48:23.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770634+0000","last_change":"2024-04-13T03:13:36.765265+0000","last_active":"2024-04-13T03:13:38.770634+0000","last_peered":"2024-04-13T03:13:38.770634+0000","last_clean":"2024-04-13T03:13:38.770634+0000","last_became_active":"2024-04-13T03:13:36.764966+0000","last_became_peered":"2024-04-13T03:13:36.764966+0000","last_unstale":"2024-04-13T03:13:38.770634+0000","last_undegraded":"2024-04-13T03:13:38.770634+0000","last_fullsized":"2024-04-13T03:13:38.770634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:48.137691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771263+0000","last_change":"2024-04-13T03:13:37.770524+0000","last_active":"2024-04-13T03:13:38.771263+0000","last_peered":"2024-04-13T03:13:38.771263+0000","last_clean":"2024-04-13T03:13:38.771263+0000","last_became_active":"2024-04-13T03:13:37.769887+0000","last_became_peered":"2024-04-13T03:13:37.769887+0000","last_unstale":"2024-04-13T03:13:38.771263+0000","last_undegraded":"2024-04-13T03:13:38.771263+0000","last_fullsized":"2024-04-13T03:13:38.771263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:39.438711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181886+0000","last_change":"2024-04-13T03:13:36.765596+0000","last_active":"2024-04-13T03:13:39.181886+0000","last_peered":"2024-04-13T03:13:39.181886+0000","last_clean":"2024-04-13T03:13:39.181886+0000","last_became_active":"2024-04-13T03:13:36.765330+0000","last_became_peered":"2024-04-13T03:13:36.765330+0000","last_unstale":"2024-04-13T03:13:39.181886+0000","last_undegraded":"2024-04-13T03:13:39.181886+0000","last_fullsized":"2024-04-13T03:13:39.181886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:11.847096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771598+0000","last_change":"2024-04-13T03:13:37.777271+0000","last_active":"2024-04-13T03:13:38.771598+0000","last_peered":"2024-04-13T03:13:38.771598+0000","last_clean":"2024-04-13T03:13:38.771598+0000","last_became_active":"2024-04-13T03:13:37.777023+0000","last_became_peered":"2024-04-13T03:13:37.777023+0000","last_unstale":"2024-04-13T03:13:38.771598+0000","last_undegraded":"2024-04-13T03:13:38.771598+0000","last_fullsized":"2024-04-13T03:13:38.771598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:49:11.880533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181260+0000","last_change":"2024-04-13T03:13:36.772152+0000","last_active":"2024-04-13T03:13:39.181260+0000","last_peered":"2024-04-13T03:13:39.181260+0000","last_clean":"2024-04-13T03:13:39.181260+0000","last_became_active":"2024-04-13T03:13:36.771816+0000","last_became_peered":"2024-04-13T03:13:36.771816+0000","last_unstale":"2024-04-13T03:13:39.181260+0000","last_undegraded":"2024-04-13T03:13:39.181260+0000","last_fullsized":"2024-04-13T03:13:39.181260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:47.661716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770676+0000","last_change":"2024-04-13T03:13:37.777412+0000","last_active":"2024-04-13T03:13:38.770676+0000","last_peered":"2024-04-13T03:13:38.770676+0000","last_clean":"2024-04-13T03:13:38.770676+0000","last_became_active":"2024-04-13T03:13:37.776532+0000","last_became_peered":"2024-04-13T03:13:37.776532+0000","last_unstale":"2024-04-13T03:13:38.770676+0000","last_undegraded":"2024-04-13T03:13:38.770676+0000","last_fullsized":"2024-04-13T03:13:38.770676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:54.539747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181024+0000","last_change":"2024-04-13T03:13:36.772084+0000","last_active":"2024-04-13T03:13:39.181024+0000","last_peered":"2024-04-13T03:13:39.181024+0000","last_clean":"2024-04-13T03:13:39.181024+0000","last_became_active":"2024-04-13T03:13:36.771800+0000","last_became_peered":"2024-04-13T03:13:36.771800+0000","last_unstale":"2024-04-13T03:13:39.181024+0000","last_undegraded":"2024-04-13T03:13:39.181024+0000","last_fullsized":"2024-04-13T03:13:39.181024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:34.347612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181705+0000","last_change":"2024-04-13T03:13:37.777817+0000","last_active":"2024-04-13T03:13:39.181705+0000","last_peered":"2024-04-13T03:13:39.181705+0000","last_clean":"2024-04-13T03:13:39.181705+0000","last_became_active":"2024-04-13T03:13:37.777547+0000","last_became_peered":"2024-04-13T03:13:37.777547+0000","last_unstale":"2024-04-13T03:13:39.181705+0000","last_undegraded":"2024-04-13T03:13:39.181705+0000","last_fullsized":"2024-04-13T03:13:39.181705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:59:44.594144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.164862+0000","last_change":"2024-04-13T03:13:36.766089+0000","last_active":"2024-04-13T03:13:39.164862+0000","last_peered":"2024-04-13T03:13:39.164862+0000","last_clean":"2024-04-13T03:13:39.164862+0000","last_became_active":"2024-04-13T03:13:36.765748+0000","last_became_peered":"2024-04-13T03:13:36.765748+0000","last_unstale":"2024-04-13T03:13:39.164862+0000","last_undegraded":"2024-04-13T03:13:39.164862+0000","last_fullsized":"2024-04-13T03:13:39.164862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:46.141793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770542+0000","last_change":"2024-04-13T03:13:37.781540+0000","last_active":"2024-04-13T03:13:38.770542+0000","last_peered":"2024-04-13T03:13:38.770542+0000","last_clean":"2024-04-13T03:13:38.770542+0000","last_became_active":"2024-04-13T03:13:37.781268+0000","last_became_peered":"2024-04-13T03:13:37.781268+0000","last_unstale":"2024-04-13T03:13:38.770542+0000","last_undegraded":"2024-04-13T03:13:38.770542+0000","last_fullsized":"2024-04-13T03:13:38.770542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:40.658209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771685+0000","last_change":"2024-04-13T03:13:36.764900+0000","last_active":"2024-04-13T03:13:38.771685+0000","last_peered":"2024-04-13T03:13:38.771685+0000","last_clean":"2024-04-13T03:13:38.771685+0000","last_became_active":"2024-04-13T03:13:36.764633+0000","last_became_peered":"2024-04-13T03:13:36.764633+0000","last_unstale":"2024-04-13T03:13:38.771685+0000","last_undegraded":"2024-04-13T03:13:38.771685+0000","last_fullsized":"2024-04-13T03:13:38.771685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:22.402588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181936+0000","last_change":"2024-04-13T03:13:37.778807+0000","last_active":"2024-04-13T03:13:39.181936+0000","last_peered":"2024-04-13T03:13:39.181936+0000","last_clean":"2024-04-13T03:13:39.181936+0000","last_became_active":"2024-04-13T03:13:37.778541+0000","last_became_peered":"2024-04-13T03:13:37.778541+0000","last_unstale":"2024-04-13T03:13:39.181936+0000","last_undegraded":"2024-04-13T03:13:39.181936+0000","last_fullsized":"2024-04-13T03:13:39.181936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:02:50.360463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182309+0000","last_change":"2024-04-13T03:13:36.770570+0000","last_active":"2024-04-13T03:13:39.182309+0000","last_peered":"2024-04-13T03:13:39.182309+0000","last_clean":"2024-04-13T03:13:39.182309+0000","last_became_active":"2024-04-13T03:13:36.769914+0000","last_became_peered":"2024-04-13T03:13:36.769914+0000","last_unstale":"2024-04-13T03:13:39.182309+0000","last_undegraded":"2024-04-13T03:13:39.182309+0000","last_fullsized":"2024-04-13T03:13:39.182309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:56.813106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.771510+0000","last_change":"2024-04-13T03:13:37.773246+0000","last_active":"2024-04-13T03:13:38.771510+0000","last_peered":"2024-04-13T03:13:38.771510+0000","last_clean":"2024-04-13T03:13:38.771510+0000","last_became_active":"2024-04-13T03:13:37.772986+0000","last_became_peered":"2024-04-13T03:13:37.772986+0000","last_unstale":"2024-04-13T03:13:38.771510+0000","last_undegraded":"2024-04-13T03:13:38.771510+0000","last_fullsized":"2024-04-13T03:13:38.771510+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:26:35.631651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182013+0000","last_change":"2024-04-13T03:13:36.768818+0000","last_active":"2024-04-13T03:13:39.182013+0000","last_peered":"2024-04-13T03:13:39.182013+0000","last_clean":"2024-04-13T03:13:39.182013+0000","last_became_active":"2024-04-13T03:13:36.768492+0000","last_became_peered":"2024-04-13T03:13:36.768492+0000","last_unstale":"2024-04-13T03:13:39.182013+0000","last_undegraded":"2024-04-13T03:13:39.182013+0000","last_fullsized":"2024-04-13T03:13:39.182013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:45.851689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770364+0000","last_change":"2024-04-13T03:13:37.782296+0000","last_active":"2024-04-13T03:13:38.770364+0000","last_peered":"2024-04-13T03:13:38.770364+0000","last_clean":"2024-04-13T03:13:38.770364+0000","last_became_active":"2024-04-13T03:13:37.782008+0000","last_became_peered":"2024-04-13T03:13:37.782008+0000","last_unstale":"2024-04-13T03:13:38.770364+0000","last_undegraded":"2024-04-13T03:13:38.770364+0000","last_fullsized":"2024-04-13T03:13:38.770364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:50.385596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770287+0000","last_change":"2024-04-13T03:13:36.766877+0000","last_active":"2024-04-13T03:13:38.770287+0000","last_peered":"2024-04-13T03:13:38.770287+0000","last_clean":"2024-04-13T03:13:38.770287+0000","last_became_active":"2024-04-13T03:13:36.765880+0000","last_became_peered":"2024-04-13T03:13:36.765880+0000","last_unstale":"2024-04-13T03:13:38.770287+0000","last_undegraded":"2024-04-13T03:13:38.770287+0000","last_fullsized":"2024-04-13T03:13:38.770287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:25:39.516825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770693+0000","last_change":"2024-04-13T03:13:37.775647+0000","last_active":"2024-04-13T03:13:38.770693+0000","last_peered":"2024-04-13T03:13:38.770693+0000","last_clean":"2024-04-13T03:13:38.770693+0000","last_became_active":"2024-04-13T03:13:37.775081+0000","last_became_peered":"2024-04-13T03:13:37.775081+0000","last_unstale":"2024-04-13T03:13:38.770693+0000","last_undegraded":"2024-04-13T03:13:38.770693+0000","last_fullsized":"2024-04-13T03:13:38.770693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:46.856331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181446+0000","last_change":"2024-04-13T03:13:36.767371+0000","last_active":"2024-04-13T03:13:39.181446+0000","last_peered":"2024-04-13T03:13:39.181446+0000","last_clean":"2024-04-13T03:13:39.181446+0000","last_became_active":"2024-04-13T03:13:36.767116+0000","last_became_peered":"2024-04-13T03:13:36.767116+0000","last_unstale":"2024-04-13T03:13:39.181446+0000","last_undegraded":"2024-04-13T03:13:39.181446+0000","last_fullsized":"2024-04-13T03:13:39.181446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:54:25.094676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770762+0000","last_change":"2024-04-13T03:13:37.770994+0000","last_active":"2024-04-13T03:13:38.770762+0000","last_peered":"2024-04-13T03:13:38.770762+0000","last_clean":"2024-04-13T03:13:38.770762+0000","last_became_active":"2024-04-13T03:13:37.770741+0000","last_became_peered":"2024-04-13T03:13:37.770741+0000","last_unstale":"2024-04-13T03:13:38.770762+0000","last_undegraded":"2024-04-13T03:13:38.770762+0000","last_fullsized":"2024-04-13T03:13:38.770762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:28:40.658209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770657+0000","last_change":"2024-04-13T03:13:36.767237+0000","last_active":"2024-04-13T03:13:38.770657+0000","last_peered":"2024-04-13T03:13:38.770657+0000","last_clean":"2024-04-13T03:13:38.770657+0000","last_became_active":"2024-04-13T03:13:36.766972+0000","last_became_peered":"2024-04-13T03:13:36.766972+0000","last_unstale":"2024-04-13T03:13:38.770657+0000","last_undegraded":"2024-04-13T03:13:38.770657+0000","last_fullsized":"2024-04-13T03:13:38.770657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:18:45.238629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770735+0000","last_change":"2024-04-13T03:13:37.770278+0000","last_active":"2024-04-13T03:13:38.770735+0000","last_peered":"2024-04-13T03:13:38.770735+0000","last_clean":"2024-04-13T03:13:38.770735+0000","last_became_active":"2024-04-13T03:13:37.770029+0000","last_became_peered":"2024-04-13T03:13:37.770029+0000","last_unstale":"2024-04-13T03:13:38.770735+0000","last_undegraded":"2024-04-13T03:13:38.770735+0000","last_fullsized":"2024-04-13T03:13:38.770735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:18:47.146435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:38.770291+0000","last_change":"2024-04-13T03:13:36.766340+0000","last_active":"2024-04-13T03:13:38.770291+0000","last_peered":"2024-04-13T03:13:38.770291+0000","last_clean":"2024-04-13T03:13:38.770291+0000","last_became_active":"2024-04-13T03:13:36.766071+0000","last_became_peered":"2024-04-13T03:13:36.766071+0000","last_unstale":"2024-04-13T03:13:38.770291+0000","last_undegraded":"2024-04-13T03:13:38.770291+0000","last_fullsized":"2024-04-13T03:13:38.770291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:18:45.238629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.182334+0000","last_change":"2024-04-13T03:13:37.775853+0000","last_active":"2024-04-13T03:13:39.182334+0000","last_peered":"2024-04-13T03:13:39.182334+0000","last_clean":"2024-04-13T03:13:39.182334+0000","last_became_active":"2024-04-13T03:13:37.775572+0000","last_became_peered":"2024-04-13T03:13:37.775572+0000","last_unstale":"2024-04-13T03:13:39.182334+0000","last_undegraded":"2024-04-13T03:13:39.182334+0000","last_fullsized":"2024-04-13T03:13:39.182334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:36.744747+0000","last_clean_scrub_stamp":"2024-04-13T03:13:36.744747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:19:43.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181986+0000","last_change":"2024-04-13T03:13:36.772096+0000","last_active":"2024-04-13T03:13:39.181986+0000","last_peered":"2024-04-13T03:13:39.181986+0000","last_clean":"2024-04-13T03:13:39.181986+0000","last_became_active":"2024-04-13T03:13:36.771833+0000","last_became_peered":"2024-04-13T03:13:36.771833+0000","last_unstale":"2024-04-13T03:13:39.181986+0000","last_undegraded":"2024-04-13T03:13:39.181986+0000","last_fullsized":"2024-04-13T03:13:39.181986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:13:35.740105+0000","last_clean_scrub_stamp":"2024-04-13T03:13:35.740105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:33:54.169137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":276,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-13T03:13:39.181888+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:13:39.181888+0000","last_peered":"2024-04-13T03:13:39.181888+0000","last_clean":"2024-04-13T03:13:39.181888+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:13:39.181888+0000","last_undegraded":"2024-04-13T03:13:39.181888+0000","last_fullsized":"2024-04-13T03:13:39.181888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":1027061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":438283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":436705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":438387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":437253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":437413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":436783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1548288,"data_stored":1027039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:13:45.483 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:13:45.483 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:13:45.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.626+0000 7fc764c88640 1 Processor -- start 2024-04-13T03:13:45.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.626+0000 7fc764c88640 1 -- start start 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc764c88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc76015b4e0 0x7fc76015fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc764c88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7600b3690 con 0x7fc7600b5390 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc764c88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7600b3970 con 0x7fc7600b4730 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc764c88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc7600b3c50 con 0x7fc76015b4e0 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc76015b4e0 0x7fc76015fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc7600b5390 0x7fc76006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44454/0 (socket says 172.21.15.106:44454) 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75ed9d640 1 -- 172.21.15.106:0/3678250326 learned_addr learned my addr 172.21.15.106:0/3678250326 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:45.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc76015b4e0 0x7fc76015fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55740/0 (socket says 172.21.15.106:55740) 2024-04-13T03:13:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75e59c640 1 -- 172.21.15.106:0/3678250326 >> v1:172.21.15.114:6789/0 conn(0x7fc7600b4730 legacy=0x7fc7600b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75e59c640 1 -- 172.21.15.106:0/3678250326 >> v1:172.21.15.106:6789/0 conn(0x7fc7600b5390 legacy=0x7fc76006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.627+0000 7fc75e59c640 1 -- 172.21.15.106:0/3678250326 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7600b3f30 con 0x7fc76015b4e0 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.628+0000 7fc75e59c640 1 --2- 172.21.15.106:0/3678250326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76015fb60 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc750009d90 tx=0x7fc75002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acd9f6a180401d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.628+0000 7fc75d59a640 1 -- 172.21.15.106:0/3678250326 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750039070 con 0x7fc76015b4e0 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc75d59a640 1 -- 172.21.15.106:0/3678250326 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75002cc60 con 0x7fc76015b4e0 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc75d59a640 1 -- 172.21.15.106:0/3678250326 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750034660 con 0x7fc76015b4e0 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 -- 172.21.15.106:0/3678250326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 msgr2=0x7fc76015fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 --2- 172.21.15.106:0/3678250326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76015fb60 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc750009d90 tx=0x7fc75002beb0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 -- 172.21.15.106:0/3678250326 shutdown_connections 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 --2- 172.21.15.106:0/3678250326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76015fb60 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 -- 172.21.15.106:0/3678250326 >> 172.21.15.106:0/3678250326 conn(0x7fc7600b2600 msgr2=0x7fc7600d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 -- 172.21.15.106:0/3678250326 shutdown_connections 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.629+0000 7fc764c88640 1 -- 172.21.15.106:0/3678250326 wait complete. 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.630+0000 7fc764c88640 1 Processor -- start 2024-04-13T03:13:45.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 -- start start 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7600b4730 0x7fc7601f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7600b5390 0x7fc76014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006d0c0 con 0x7fc76015b4e0 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600b32d0 con 0x7fc7600b4730 2024-04-13T03:13:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc764c88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc760157ee0 con 0x7fc7600b5390 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc75ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc75ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55750/0 (socket says 172.21.15.106:55750) 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.631+0000 7fc75ed9d640 1 -- 172.21.15.106:0/2792209744 learned_addr learned my addr 172.21.15.106:0/2792209744 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75e59c640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7600b4730 0x7fc7601f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75dd9b640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7600b5390 0x7fc76014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 -- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7600b5390 msgr2=0x7fc76014fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7600b5390 0x7fc76014fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 -- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7600b4730 msgr2=0x7fc7601f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7600b4730 0x7fc7601f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 -- 172.21.15.106:0/2792209744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7601581c0 con 0x7fc76015b4e0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc75ed9d640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c430 tx=0x7fc75800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758017070 con 0x7fc76015b4e0 2024-04-13T03:13:45.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758005ce0 con 0x7fc76015b4e0 2024-04-13T03:13:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7601584a0 con 0x7fc76015b4e0 2024-04-13T03:13:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7580072d0 con 0x7fc76015b4e0 2024-04-13T03:13:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.632+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc760158a60 con 0x7fc76015b4e0 2024-04-13T03:13:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.633+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc760068a10 con 0x7fc76015b4e0 2024-04-13T03:13:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.634+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc75801a070 con 0x7fc76015b4e0 2024-04-13T03:13:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.634+0000 7fc74effd640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 0x7fc74006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.634+0000 7fc75e59c640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 0x7fc74006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.634+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fc75808aee0 con 0x7fc76015b4e0 2024-04-13T03:13:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.634+0000 7fc75e59c640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 0x7fc74006ede0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc750036000 tx=0x7fc750030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:45.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.638+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc758058bc0 con 0x7fc76015b4e0 2024-04-13T03:13:45.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.886+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc7600af9c0 con 0x7fc76015b4e0 2024-04-13T03:13:45.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.890+0000 7fc74effd640 1 -- 172.21.15.106:0/2792209744 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fc75805cec0 con 0x7fc76015b4e0 2024-04-13T03:13:45.891 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:13:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:13:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:13:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:13:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:13:45.892 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.892+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 msgr2=0x7fc74006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.892+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 0x7fc74006ede0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc750036000 tx=0x7fc750030000 comp rx=0 tx=0).stop 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.892+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 msgr2=0x7fc76006b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.892+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c430 tx=0x7fc75800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 shutdown_connections 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc74006c920 0x7fc74006ede0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7600b5390 0x7fc76014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc76015b4e0 0x7fc76006b8e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 --2- 172.21.15.106:0/2792209744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7600b4730 0x7fc7601f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:45.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 >> 172.21.15.106:0/2792209744 conn(0x7fc7600b2600 msgr2=0x7fc7600b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:45.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 shutdown_connections 2024-04-13T03:13:45.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:45.893+0000 7fc764c88640 1 -- 172.21.15.106:0/2792209744 wait complete. 2024-04-13T03:13:45.904 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:46.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.004+0000 7f3bc33f7640 1 Processor -- start 2024-04-13T03:13:46.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.004+0000 7f3bc33f7640 1 -- start start 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.004+0000 7f3bc33f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3bbc0cdf50 0x7f3bbc0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc33f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bbc0ce870 con 0x7f3bbc06e150 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc33f7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bbc0998d0 con 0x7f3bbc06cb10 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc33f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3bbc073020 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1994640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3bbc06e150 0x7f3bbc06e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44466/0 (socket says 172.21.15.106:44466) 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1994640 1 -- 172.21.15.106:0/80884789 learned_addr learned my addr 172.21.15.106:0/80884789 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:46.006 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1193640 1 --2- 172.21.15.106:0/80884789 >> v2:172.21.15.114:3300/0 conn(0x7f3bbc0cdf50 0x7f3bbc0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407661596 0 0) 0x7f3bbc0ce870 con 0x7f3bbc06e150 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba8003520 con 0x7f3bbc06e150 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1641385722 0 0) 0x7f3ba8003520 con 0x7f3bbc06e150 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1193640 1 -- 172.21.15.106:0/80884789 >> v1:172.21.15.114:6789/0 conn(0x7f3bbc06cb10 legacy=0x7f3bbc072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1193640 1 -- 172.21.15.106:0/80884789 >> v1:172.21.15.106:6789/0 conn(0x7f3bbc06e150 legacy=0x7f3bbc06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.005+0000 7f3bc1193640 1 -- 172.21.15.106:0/80884789 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc073300 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.006+0000 7f3bc1193640 1 --2- 172.21.15.106:0/80884789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc0ce330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80097a0 tx=0x7f3bb802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b745837d8c6403f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.006+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8038070 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.006+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb8031500 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.006+0000 7f3bb3fff640 1 -- 172.21.15.106:0/80884789 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40019f0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 -- 172.21.15.106:0/80884789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc0cdf50 msgr2=0x7f3bbc0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/80884789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc0ce330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80097a0 tx=0x7f3bb802b820 comp rx=0 tx=0).stop 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 -- 172.21.15.106:0/80884789 shutdown_connections 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/80884789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc0ce330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 -- 172.21.15.106:0/80884789 >> 172.21.15.106:0/80884789 conn(0x7f3bbc0b1f10 msgr2=0x7f3bbc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 -- 172.21.15.106:0/80884789 shutdown_connections 2024-04-13T03:13:46.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.007+0000 7f3bc33f7640 1 -- 172.21.15.106:0/80884789 wait complete. 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.008+0000 7f3bc33f7640 1 Processor -- start 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.008+0000 7f3bc33f7640 1 -- start start 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3bbc06cb10 0x7f3bbc15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 0x7f3bbc0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc06ed10 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc0738c0 con 0x7f3bbc06e150 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc06ee90 con 0x7f3bbc06cb10 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55758/0 (socket says 172.21.15.106:55758) 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 -- 172.21.15.106:0/3166567230 learned_addr learned my addr 172.21.15.106:0/3166567230 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc0992640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 0x7f3bbc0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1193640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3bbc06cb10 0x7f3bbc15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 -- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3bbc06cb10 msgr2=0x7f3bbc15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3bbc06cb10 0x7f3bbc15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 -- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 msgr2=0x7f3bbc0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 0x7f3bbc0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 -- 172.21.15.106:0/3166567230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc0cd4f0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc0992640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 0x7f3bbc0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:46.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc1994640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40018c0 tx=0x7f3bb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4012400 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb40125c0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc0cd7d0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.009+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40129b0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.010+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc0c8540 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.011+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bbc068a10 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.011+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bb4012b50 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.011+0000 7f3bb1ffb640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.012+0000 7f3bc1193640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.012+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f3bb408c510 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.012+0000 7f3bc1193640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 0x7f3ba006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8035000 tx=0x7f3bb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.016+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bb40182b0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.226+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bbc0c89e0 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.227+0000 7f3bb1ffb640 1 -- 172.21.15.106:0/3166567230 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f3bb405a900 con 0x7f3bbc0cdf50 2024-04-13T03:13:46.228 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:46.228 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:39.107280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6834/390879687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":390879687},{"type":"v1","addr":"172.21.15.114:6834","nonce":390879687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":5}]} 2024-04-13T03:13:46.228 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 36 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 msgr2=0x7f3ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 0x7f3ba006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8035000 tx=0x7f3bb8008030 comp rx=0 tx=0).stop 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 msgr2=0x7f3bbc1f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40018c0 tx=0x7f3bb400a430 comp rx=0 tx=0).stop 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 shutdown_connections 2024-04-13T03:13:46.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bbc0cdf50 0x7f3bbc1f46b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3bbc06e150 0x7f3bbc0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 --2- 172.21.15.106:0/3166567230 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3bbc06cb10 0x7f3bbc15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 >> 172.21.15.106:0/3166567230 conn(0x7f3bbc0b1f10 msgr2=0x7f3bbc06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 shutdown_connections 2024-04-13T03:13:46.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.229+0000 7f3bc33f7640 1 -- 172.21.15.106:0/3166567230 wait complete. 2024-04-13T03:13:46.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:13:37.103773+0000', 'modified': '2024-04-13T03:13:39.107280+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6825}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6825': {'gid': 6825, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6834/390879687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 390879687}, {'type': 'v1', 'addr': '172.21.15.114:6834', 'nonce': 390879687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6825, 'qdb_cluster': [6825]} 2024-04-13T03:13:46.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:13:46.643 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:13:46.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.745+0000 7f1b5bb1e640 1 Processor -- start 2024-04-13T03:13:46.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.745+0000 7f1b5bb1e640 1 -- start start 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5bb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b540b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5bb1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5406e280 con 0x7f1b5406ed70 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5bb1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5406e560 con 0x7f1b5415e930 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5bb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b540cccf0 con 0x7f1b540d09f0 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5a0bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1b5406ed70 0x7f1b540cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44482/0 (socket says 172.21.15.106:44482) 2024-04-13T03:13:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b5a0bb640 1 -- 172.21.15.106:0/184979130 learned_addr learned my addr 172.21.15.106:0/184979130 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b590b9640 1 --2- 172.21.15.106:0/184979130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b540b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003146774 0 0) 0x7f1b5406e280 con 0x7f1b5406ed70 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003520 con 0x7f1b5406ed70 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4024889358 0 0) 0x7f1b40003520 con 0x7f1b5406ed70 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b590b9640 1 -- 172.21.15.106:0/184979130 >> v1:172.21.15.114:6789/0 conn(0x7f1b5415e930 legacy=0x7f1b540d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b590b9640 1 -- 172.21.15.106:0/184979130 >> v1:172.21.15.106:6789/0 conn(0x7f1b5406ed70 legacy=0x7f1b540cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.746+0000 7f1b590b9640 1 -- 172.21.15.106:0/184979130 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b540ccfd0 con 0x7f1b540d09f0 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.747+0000 7f1b590b9640 1 --2- 172.21.15.106:0/184979130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b540b5350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b440097d0 tx=0x7f1b4402b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c12b5daf0b1d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.747+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44037070 con 0x7f1b540d09f0 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.747+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b44030510 con 0x7f1b540d09f0 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.747+0000 7f1b588b8640 1 -- 172.21.15.106:0/184979130 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4402be60 con 0x7f1b540d09f0 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/184979130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 msgr2=0x7f1b540b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/184979130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b540b5350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b440097d0 tx=0x7f1b4402b660 comp rx=0 tx=0).stop 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/184979130 shutdown_connections 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/184979130 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b540b5350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/184979130 >> 172.21.15.106:0/184979130 conn(0x7f1b540cef80 msgr2=0x7f1b540cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/184979130 shutdown_connections 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.748+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/184979130 wait complete. 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.749+0000 7f1b5bb1e640 1 Processor -- start 2024-04-13T03:13:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.749+0000 7f1b5bb1e640 1 -- start start 2024-04-13T03:13:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b541615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b5415e930 0x7f1b5414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540ce830 con 0x7f1b5406ed70 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5406f1c0 con 0x7f1b5415e930 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b540ce9b0 con 0x7f1b540d09f0 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55766/0 (socket says 172.21.15.106:55766) 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 -- 172.21.15.106:0/3365007486 learned_addr learned my addr 172.21.15.106:0/3365007486 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 -- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 msgr2=0x7f1b541615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b541615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 -- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b5415e930 msgr2=0x7f1b5414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b5415e930 0x7f1b5414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 -- 172.21.15.106:0/3365007486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54160540 con 0x7f1b5406ed70 2024-04-13T03:13:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c550 tx=0x7f1b5000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b4a7fc640 1 -- 172.21.15.106:0/3365007486 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50017070 con 0x7f1b5406ed70 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b4a7fc640 1 -- 172.21.15.106:0/3365007486 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50005480 con 0x7f1b5406ed70 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b4a7fc640 1 -- 172.21.15.106:0/3365007486 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50003810 con 0x7f1b5406ed70 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b54160820 con 0x7f1b5406ed70 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.750+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b54160de0 con 0x7f1b5406ed70 2024-04-13T03:13:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.752+0000 7f1b4a7fc640 1 -- 172.21.15.106:0/3365007486 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b5001a070 con 0x7f1b5406ed70 2024-04-13T03:13:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.752+0000 7f1b4a7fc640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 0x7f1b3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.752+0000 7f1b4a7fc640 1 -- 172.21.15.106:0/3365007486 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f1b5008b180 con 0x7f1b5406ed70 2024-04-13T03:13:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.752+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 0x7f1b3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.753+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 0x7f1b3806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1b44034000 tx=0x7f1b4402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.753+0000 7f1b5e9fc740 1 Processor -- start 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.753+0000 7f1b5e9fc740 1 -- start start 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548ce2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d12b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564548c4d990 con 0x564548ce32c0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564548bf4620 con 0x564548d101c0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564548c49950 con 0x564548d0fde0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55768/0 (socket says 172.21.15.106:55768) 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 -- 172.21.15.106:0/278906986 learned_addr learned my addr 172.21.15.106:0/278906986 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b598ba640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548ce2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b590b9640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d12b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 -- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 msgr2=0x564548ce2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548ce2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 -- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 msgr2=0x564548d12b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d12b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 -- 172.21.15.106:0/278906986 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645489602d0 con 0x564548ce32c0 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b590b9640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d12b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.754+0000 7f1b5a0bb640 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0021c0 tx=0x7f1b4c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dabfa56d4dced4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b377fe640 1 -- 172.21.15.106:0/278906986 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c004d60 con 0x564548ce32c0 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b377fe640 1 -- 172.21.15.106:0/278906986 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c00ce50 con 0x564548ce32c0 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b377fe640 1 -- 172.21.15.106:0/278906986 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c007310 con 0x564548ce32c0 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/278906986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 msgr2=0x564548d130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0021c0 tx=0x7f1b4c0021f0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/278906986 shutdown_connections 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548ce32c0 0x564548d130b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d12b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/278906986 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548ce2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/278906986 >> 172.21.15.106:0/278906986 conn(0x564548ce4ff0 msgr2=0x564548d2db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/278906986 shutdown_connections 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.755+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/278906986 wait complete. 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.756+0000 7f1b5e9fc740 1 -- start start 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.756+0000 7f1b5e9fc740 1 Processor -- start 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548cfce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d12710 0x564548d0af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564548ce7590 con 0x564548d101c0 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564548c079e0 con 0x564548d12710 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564548ce6980 con 0x564548d0fde0 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5e9fc740 10 client.0 _ll_register_callbacks cb 0x564548c43220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55782/0 (socket says 172.21.15.106:55782) 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 -- 172.21.15.106:0/2664917718 learned_addr learned my addr 172.21.15.106:0/2664917718 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b5a0bb640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d12710 0x564548d0af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b598ba640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548cfce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 -- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 msgr2=0x564548cfce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548cfce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 -- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d12710 msgr2=0x564548d0af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d12710 0x564548d0af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 -- 172.21.15.106:0/2664917718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564548a2a6b0 con 0x564548d101c0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b590b9640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1b4402b6a0 tx=0x7f1b44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305bd9bf7afaa4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.757+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4404c070 con 0x564548d101c0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f1b44030710 con 0x564548d101c0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4c00ce50 con 0x564548d101c0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1b44057050 con 0x564548d101c0 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 10 client.6996 fetch_fsmap learned FSMap version 36 2024-04-13T03:13:46.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564548d11c30 con 0x564548d101c0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c00ce50 con 0x564548d101c0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f1b4405a020 con 0x564548d101c0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 10 client.6996 fetch_fsmap finished waiting for FSMap version 36 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 10 client.6996 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 20 client.6996 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d419d0 0x564548d41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 4 client.6996 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 --> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564548c5b3b0 con 0x564548d419d0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.758+0000 7f1b598ba640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d419d0 0x564548d41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b34ff9640 10 client.6996 ms_handle_connect on v2:172.21.15.114:6832/390879687 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b598ba640 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d419d0 0x564548d41db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69021ddbefe5e98d server_cookie=2611a79209dad08d in_seq=0 out_seq=0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b34ff9640 1 -- 172.21.15.106:0/2664917718 <== mds.0 v2:172.21.15.114:6832/390879687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564548c5b3b0 con 0x564548d419d0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b34ff9640 10 client.6996 handle_command_reply: tid=0 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 client.6996 shutdown 2024-04-13T03:13:46.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 msgr2=0x564548d08930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1b4402b6a0 tx=0x7f1b44004730 comp rx=0 tx=0).stop 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 shutdown_connections 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d419d0 0x564548d41db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d12710 0x564548d0af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d08930 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/2664917718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d0fde0 0x564548cfce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b34ff9640 0 client.6996 ms_handle_reset on v2:172.21.15.114:6832/390879687 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.759+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 >> 172.21.15.106:0/2664917718 conn(0x564548ce32c0 msgr2=0x564548d31080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.760+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 shutdown_connections 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.760+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/2664917718 wait complete. 2024-04-13T03:13:46.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.760+0000 7f1b5e9fc740 20 client.6996 trim_cache size 0 max 16384 2024-04-13T03:13:46.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.777+0000 7f1b5e9fc740 1 Processor -- start 2024-04-13T03:13:46.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 -- start start 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d0fde0 0x564548da0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548ce6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54200530 con 0x564548d101c0 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540ca990 con 0x564548d0fde0 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b540cafe0 con 0x564548d12710 2024-04-13T03:13:46.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b590b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b590b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55790/0 (socket says 172.21.15.106:55790) 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.778+0000 7f1b590b9640 1 -- 172.21.15.106:0/3481036979 learned_addr learned my addr 172.21.15.106:0/3481036979 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 -- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 msgr2=0x564548ce6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548ce6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 -- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d0fde0 msgr2=0x564548da0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d0fde0 0x564548da0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 -- 172.21.15.106:0/3481036979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564548d0b280 con 0x564548d101c0 2024-04-13T03:13:46.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009560 tx=0x7f1b44047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b249b9f6c37bea78 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b36ffd640 1 -- 172.21.15.106:0/3481036979 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44060070 con 0x564548d101c0 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b36ffd640 1 -- 172.21.15.106:0/3481036979 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b44042070 con 0x564548d101c0 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b36ffd640 1 -- 172.21.15.106:0/3481036979 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4404c070 con 0x564548d101c0 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3481036979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 msgr2=0x564548d31080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009560 tx=0x7f1b44047000 comp rx=0 tx=0).stop 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3481036979 shutdown_connections 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548ce6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d101c0 0x564548d31080 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3481036979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d0fde0 0x564548da0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3481036979 >> 172.21.15.106:0/3481036979 conn(0x564548ce32c0 msgr2=0x564548ce8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3481036979 shutdown_connections 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.779+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3481036979 wait complete. 2024-04-13T03:13:46.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.780+0000 7f1b5e9fc740 1 -- start start 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.780+0000 7f1b5e9fc740 1 Processor -- start 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d0e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548d2d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564548d15200 con 0x564548d0fde0 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564548d11f70 con 0x564548d101c0 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564548d14b80 con 0x564548d12710 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5e9fc740 10 client.0 _ll_register_callbacks cb 0x7f1b44009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55800/0 (socket says 172.21.15.106:55800) 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 -- 172.21.15.106:0/3771067798 learned_addr learned my addr 172.21.15.106:0/3771067798 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b5a0bb640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548d2d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d0e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 -- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 msgr2=0x564548d2d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548d2d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 -- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 msgr2=0x564548d0e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d0e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 -- 172.21.15.106:0/3771067798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564548ce2d10 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b598ba640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c520 tx=0x7f1b5000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74be4ec5217fb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.781+0000 7f1b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50089030 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f1b50072d40 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b44042070 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1b50078300 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 10 client.7002 fetch_fsmap learned FSMap version 36 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564548d15f70 con 0x564548d0fde0 2024-04-13T03:13:46.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b44042070 con 0x564548d0fde0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f1b50055390 con 0x564548d0fde0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 10 client.7002 fetch_fsmap finished waiting for FSMap version 36 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 10 client.7002 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 20 client.7002 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d2dcf0 0x564548d50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 4 client.7002 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 --> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] -- command(tid 0: {"prefix": "status"}) -- 0x564548c55220 con 0x564548d2dcf0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.782+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d2dcf0 0x564548d50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b590b9640 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d2dcf0 0x564548d50770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2976b693bbc0ba9 server_cookie=a866adea38afbd91 in_seq=0 out_seq=0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b34ff9640 10 client.7002 ms_handle_connect on v2:172.21.15.114:6832/390879687 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b34ff9640 1 -- 172.21.15.106:0/3771067798 <== mds.0 v2:172.21.15.114:6832/390879687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564548c55220 con 0x564548d2dcf0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b34ff9640 10 client.7002 handle_command_reply: tid=0 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 client.7002 shutdown 2024-04-13T03:13:46.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 msgr2=0x564548d0de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c520 tx=0x7f1b5000f000 comp rx=0 tx=0).stop 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 shutdown_connections 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:6832/390879687,v1:172.21.15.114:6834/390879687] conn(0x564548d2dcf0 0x564548d50770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564548d12710 0x564548d2d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564548d101c0 0x564548d0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 --2- 172.21.15.106:0/3771067798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564548d0fde0 0x564548d0de70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b34ff9640 0 client.7002 ms_handle_reset on v2:172.21.15.114:6832/390879687 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 >> 172.21.15.106:0/3771067798 conn(0x564548ce32c0 msgr2=0x564548ce6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 shutdown_connections 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 1 -- 172.21.15.106:0/3771067798 wait complete. 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5e9fc740 20 client.7002 trim_cache size 0 max 16384 2024-04-13T03:13:46.791 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 6825, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 9.0317650090000008, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 36, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 78, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 78, 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 14.075675089000001 2024-04-13T03:13:46.792 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:46.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 msgr2=0x7f1b3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.783+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 0x7f1b3806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1b44034000 tx=0x7f1b4402f000 comp rx=0 tx=0).stop 2024-04-13T03:13:46.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 msgr2=0x7f1b5414db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c550 tx=0x7f1b5000ca10 comp rx=0 tx=0).stop 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 shutdown_connections 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b3806c8f0 0x7f1b3806edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b540d09f0 0x7f1b541615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b5415e930 0x7f1b5414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 --2- 172.21.15.106:0/3365007486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5406ed70 0x7f1b5414db50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 >> 172.21.15.106:0/3365007486 conn(0x7f1b540cef80 msgr2=0x7f1b540b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 shutdown_connections 2024-04-13T03:13:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.784+0000 7f1b5bb1e640 1 -- 172.21.15.106:0/3365007486 wait complete. 2024-04-13T03:13:46.795 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:13:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.893+0000 7f0e5ff88640 1 Processor -- start 2024-04-13T03:13:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.893+0000 7f0e5ff88640 1 -- start start 2024-04-13T03:13:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0e5815ea80 0x7f0e580d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e580b68f0 0x7f0e580d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0e580d3f70 con 0x7f0e580b68f0 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0e580b7100 con 0x7f0e5815ea80 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5ff88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0e581635b0 con 0x7f0e580d3b10 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5e525640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e580b68f0 0x7f0e580d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5e525640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e580b68f0 0x7f0e580d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55816/0 (socket says 172.21.15.106:55816) 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5e525640 1 -- 172.21.15.106:0/2917724169 learned_addr learned my addr 172.21.15.106:0/2917724169 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5dd24640 1 --2- 172.21.15.106:0/2917724169 >> v2:172.21.15.114:3300/0 conn(0x7f0e5815ea80 0x7f0e580d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.894+0000 7f0e5d523640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 -- 172.21.15.106:0/2917724169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 msgr2=0x7f0e580b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 -- 172.21.15.106:0/2917724169 >> v2:172.21.15.114:3300/0 conn(0x7f0e5815ea80 msgr2=0x7f0e580d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 --2- 172.21.15.106:0/2917724169 >> v2:172.21.15.114:3300/0 conn(0x7f0e5815ea80 0x7f0e580d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 -- 172.21.15.106:0/2917724169 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581627f0 con 0x7f0e580b68f0 2024-04-13T03:13:46.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5e525640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e580d23b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0e580746c0 tx=0x7f0e5002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a0be41d7d64015 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.895+0000 7f0e5cd22640 1 -- 172.21.15.106:0/2917724169 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5002e070 con 0x7f0e580b68f0 2024-04-13T03:13:46.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5cd22640 1 -- 172.21.15.106:0/2917724169 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50037070 con 0x7f0e580b68f0 2024-04-13T03:13:46.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5cd22640 1 -- 172.21.15.106:0/2917724169 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031a00 con 0x7f0e580b68f0 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 -- 172.21.15.106:0/2917724169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 msgr2=0x7f0e580d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e580d23b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0e580746c0 tx=0x7f0e5002b880 comp rx=0 tx=0).stop 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 -- 172.21.15.106:0/2917724169 shutdown_connections 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e580d23b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/2917724169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/2917724169 >> v2:172.21.15.114:3300/0 conn(0x7f0e5815ea80 0x7f0e580d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 -- 172.21.15.106:0/2917724169 >> 172.21.15.106:0/2917724169 conn(0x7f0e580b97a0 msgr2=0x7f0e580b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 -- 172.21.15.106:0/2917724169 shutdown_connections 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.896+0000 7f0e5ff88640 1 -- 172.21.15.106:0/2917724169 wait complete. 2024-04-13T03:13:46.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.897+0000 7f0e5ff88640 1 Processor -- start 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.897+0000 7f0e5ff88640 1 -- start start 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5815ea80 0x7f0e580b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58163090 con 0x7f0e580b68f0 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58162db0 con 0x7f0e5815ea80 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5ff88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580b33e0 con 0x7f0e580d3b10 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55830/0 (socket says 172.21.15.106:55830) 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 -- 172.21.15.106:0/606170077 learned_addr learned my addr 172.21.15.106:0/606170077 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5e525640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5815ea80 0x7f0e580b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5d523640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 -- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 msgr2=0x7f0e580cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 -- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5815ea80 msgr2=0x7f0e580b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5815ea80 0x7f0e580b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.898+0000 7f0e5dd24640 1 -- 172.21.15.106:0/606170077 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581f4240 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e5dd24640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400efc0 tx=0x7f0e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54018020 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e54014480 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e580cdb50 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54014850 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.899+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e580ccb30 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.900+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58068a10 con 0x7f0e580b68f0 2024-04-13T03:13:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.900+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e54005d10 con 0x7f0e580b68f0 2024-04-13T03:13:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.901+0000 7f0e4e7fc640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 0x7f0e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.901+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f0e5408c810 con 0x7f0e580b68f0 2024-04-13T03:13:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.901+0000 7f0e5d523640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 0x7f0e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.901+0000 7f0e5d523640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 0x7f0e3c06edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006fd0 tx=0x7f0e48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:46.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:46.905+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e54022070 con 0x7f0e580b68f0 2024-04-13T03:13:47.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.120+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0e580d54c0 con 0x7f0e580b68f0 2024-04-13T03:13:47.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.120+0000 7f0e4e7fc640 1 -- 172.21.15.106:0/606170077 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f0e54010040 con 0x7f0e580b68f0 2024-04-13T03:13:47.121 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:13:47.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 msgr2=0x7f0e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:47.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 0x7f0e3c06edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006fd0 tx=0x7f0e48008040 comp rx=0 tx=0).stop 2024-04-13T03:13:47.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 msgr2=0x7f0e5815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:47.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400efc0 tx=0x7f0e5400c490 comp rx=0 tx=0).stop 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 shutdown_connections 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e3c06c8f0 0x7f0e3c06edb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e580d3b10 0x7f0e580cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5815ea80 0x7f0e580b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 --2- 172.21.15.106:0/606170077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e580b68f0 0x7f0e5815d690 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 >> 172.21.15.106:0/606170077 conn(0x7f0e580b97a0 msgr2=0x7f0e580d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 shutdown_connections 2024-04-13T03:13:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47.122+0000 7f0e5ff88640 1 -- 172.21.15.106:0/606170077 wait complete. 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:47.134 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:13:47.135 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:13:47.135 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:13:47.135 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:47.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.135 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42322sec preferred_lft 42322sec 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:47.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.152 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:13:47.218 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:13:47.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.220 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:13:47.276 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:13:47.276 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:13:47.276 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:13:47.276 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:13:47.276 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:13:47.277 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:13:47.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:13:47.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:13:47.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.412 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:13:47.426 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:13:47.426 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:13:47.426 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:47.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:13:47.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:13:47.493 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:47.548 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:47.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:13:47.548 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:47.603 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:13:47.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:47.687 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:47.687 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:13:47.707 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:13:47.708 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:13:47.710 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:13:47.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:47.737 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:13:47.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:13:47.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:13:47.838 INFO:teuthology.orchestra.run.smithi106.stdout:client7011 2024-04-13T03:13:47.838 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:47.839 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:47.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.854 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-13T03:13:47.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.914 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-13T03:13:47.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:47.971 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-13T03:13:48.029 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:13:48.132 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:13:48.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:13:48.141 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:48.141 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:48.196 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:48.196 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:48.252 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:48.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.390+0000 7f7a6f836640 1 Processor -- start 2024-04-13T03:13:48.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 -- start start 2024-04-13T03:13:48.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 0x7f7a680ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7a6806cb80 0x7f7a680726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a680ce870 con 0x7f7a68072c10 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7a68073090 con 0x7f7a6806cb80 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.391+0000 7f7a6f836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7a68073370 con 0x7f7a680cdf50 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6d5d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 0x7f7a680ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7a6806cb80 0x7f7a680726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7a6806cb80 0x7f7a680726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:37366/0 (socket says 172.21.15.106:37366) 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6d5d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 0x7f7a680ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49754/0 (socket says 172.21.15.106:49754) 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 -- 172.21.15.106:0/4118660456 learned_addr learned my addr 172.21.15.106:0/4118660456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:48.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6ddd3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7a68072c10 0x7f7a6806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:37874/0 (socket says 172.21.15.106:37874) 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 -- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 msgr2=0x7f7a680ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 --2- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 0x7f7a680ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 -- 172.21.15.106:0/4118660456 >> v1:172.21.15.114:6789/0 conn(0x7f7a68072c10 legacy=0x7f7a6806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.392+0000 7f7a6cdd1640 1 -- 172.21.15.106:0/4118660456 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a68073650 con 0x7f7a6806cb80 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a6cdd1640 1 --2- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a6806cb80 0x7f7a680726d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009870 tx=0x7f7a5802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69dbf730a914fa59 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a5ffff640 1 -- 172.21.15.106:0/4118660456 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58034070 con 0x7f7a6806cb80 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a5ffff640 1 -- 172.21.15.106:0/4118660456 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58030840 con 0x7f7a6806cb80 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a5ffff640 1 -- 172.21.15.106:0/4118660456 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58030c10 con 0x7f7a6806cb80 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a6f836640 1 -- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a6806cb80 msgr2=0x7f7a680726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.393+0000 7f7a6f836640 1 --2- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a6806cb80 0x7f7a680726d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009870 tx=0x7f7a5802b980 comp rx=0 tx=0).stop 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 -- 172.21.15.106:0/4118660456 shutdown_connections 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 --2- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a6806cb80 0x7f7a680726d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 --2- 172.21.15.106:0/4118660456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a680cdf50 0x7f7a680ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 -- 172.21.15.106:0/4118660456 >> 172.21.15.106:0/4118660456 conn(0x7f7a680b1f10 msgr2=0x7f7a680b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 -- 172.21.15.106:0/4118660456 shutdown_connections 2024-04-13T03:13:48.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.394+0000 7f7a6f836640 1 -- 172.21.15.106:0/4118660456 wait complete. 2024-04-13T03:13:48.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 Processor -- start 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 -- start start 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a6806cb80 0x7f7a6815bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a68072c10 0x7f7a681e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68099930 con 0x7f7a680cdf50 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a680997b0 con 0x7f7a68072c10 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.395+0000 7f7a6f836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6806f180 con 0x7f7a6806cb80 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55842/0 (socket says 172.21.15.106:55842) 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 -- 172.21.15.106:0/2971179842 learned_addr learned my addr 172.21.15.106:0/2971179842 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6d5d2640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a6806cb80 0x7f7a6815bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6cdd1640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a68072c10 0x7f7a681e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 -- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a6806cb80 msgr2=0x7f7a6815bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a6806cb80 0x7f7a6815bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 -- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a68072c10 msgr2=0x7f7a681e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a68072c10 0x7f7a681e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 -- 172.21.15.106:0/2971179842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a681e6730 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6ddd3640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000c3f0 tx=0x7f7a6000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600074e0 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60007680 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a680c4c10 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60004a00 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.396+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6815c040 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.397+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a68068a10 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.398+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a6001a020 con 0x7f7a680cdf50 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.398+0000 7f7a5dffb640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 0x7f7a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.398+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f7a6008b4c0 con 0x7f7a680cdf50 2024-04-13T03:13:48.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.398+0000 7f7a6d5d2640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 0x7f7a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.398+0000 7f7a6d5d2640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 0x7f7a4c06edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a681f38f0 tx=0x7f7a6400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.402+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a60059760 con 0x7f7a680cdf50 2024-04-13T03:13:48.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.578+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a680d0110 con 0x7f7a680cdf50 2024-04-13T03:13:48.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.579+0000 7f7a5dffb640 1 -- 172.21.15.106:0/2971179842 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16981 (secure 0 0 0) 0x7f7a60059060 con 0x7f7a680cdf50 2024-04-13T03:13:48.580 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:48.581 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":78,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:38.762950+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:48.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 msgr2=0x7f7a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 0x7f7a4c06edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a681f38f0 tx=0x7f7a6400b040 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 msgr2=0x7f7a68158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000c3f0 tx=0x7f7a6000c8b0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 shutdown_connections 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.581+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a4c06c8f0 0x7f7a4c06edb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a680cdf50 0x7f7a68158320 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a68072c10 0x7f7a681e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 --2- 172.21.15.106:0/2971179842 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a6806cb80 0x7f7a6815bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 >> 172.21.15.106:0/2971179842 conn(0x7f7a680b1f10 msgr2=0x7f7a680b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 shutdown_connections 2024-04-13T03:13:48.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.582+0000 7f7a6f836640 1 -- 172.21.15.106:0/2971179842 wait complete. 2024-04-13T03:13:48.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:13:35.329204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:13:36.059991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:48.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:48.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.696+0000 7fe7e535e640 1 Processor -- start 2024-04-13T03:13:48.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.696+0000 7fe7e535e640 1 -- start start 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7e535e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7e015ed90 0x7fe7e00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7e535e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e00b3d60 0x7fe7e00d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7e535e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e00cfd20 con 0x7fe7e015e9b0 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7e535e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e0162af0 con 0x7fe7e015ed90 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7e535e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e0162dd0 con 0x7fe7e00b3d60 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7e015ed90 0x7fe7e00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe7e015ed90 0x7fe7e00d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55850/0 (socket says 172.21.15.106:55850) 2024-04-13T03:13:48.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 -- 172.21.15.106:0/3443764634 learned_addr learned my addr 172.21.15.106:0/3443764634 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:48.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 -- 172.21.15.106:0/3443764634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e00b3d60 msgr2=0x7fe7e00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:48.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 --2- 172.21.15.106:0/3443764634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e00b3d60 0x7fe7e00d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 -- 172.21.15.106:0/3443764634 >> v1:172.21.15.114:6789/0 conn(0x7fe7e015e9b0 legacy=0x7fe7e00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:48.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.697+0000 7fe7de7fc640 1 -- 172.21.15.106:0/3443764634 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01630b0 con 0x7fe7e015ed90 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7de7fc640 1 --2- 172.21.15.106:0/3443764634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e015ed90 0x7fe7e00d2300 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009850 tx=0x7fe7d402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d8781b340a8ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7ddffb640 1 -- 172.21.15.106:0/3443764634 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4004230 con 0x7fe7e015ed90 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7ddffb640 1 -- 172.21.15.106:0/3443764634 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d40043d0 con 0x7fe7e015ed90 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7ddffb640 1 -- 172.21.15.106:0/3443764634 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4003810 con 0x7fe7e015ed90 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7e535e640 1 -- 172.21.15.106:0/3443764634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e015ed90 msgr2=0x7fe7e00d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.698+0000 7fe7e535e640 1 --2- 172.21.15.106:0/3443764634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e015ed90 0x7fe7e00d2300 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009850 tx=0x7fe7d402b960 comp rx=0 tx=0).stop 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 -- 172.21.15.106:0/3443764634 shutdown_connections 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 --2- 172.21.15.106:0/3443764634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e00b3d60 0x7fe7e00d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 --2- 172.21.15.106:0/3443764634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e015ed90 0x7fe7e00d2300 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 -- 172.21.15.106:0/3443764634 >> 172.21.15.106:0/3443764634 conn(0x7fe7e008bf10 msgr2=0x7fe7e00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:48.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 -- 172.21.15.106:0/3443764634 shutdown_connections 2024-04-13T03:13:48.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.699+0000 7fe7e535e640 1 -- 172.21.15.106:0/3443764634 wait complete. 2024-04-13T03:13:48.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.700+0000 7fe7e535e640 1 Processor -- start 2024-04-13T03:13:48.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.700+0000 7fe7e535e640 1 -- start start 2024-04-13T03:13:48.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.700+0000 7fe7e535e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 0x7fe7e0159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e015e9b0 0x7fe7e00ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0162800 con 0x7fe7e00ca970 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0162680 con 0x7fe7e00b3d60 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e00d4d00 con 0x7fe7e015e9b0 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55852/0 (socket says 172.21.15.106:55852) 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 -- 172.21.15.106:0/556104136 learned_addr learned my addr 172.21.15.106:0/556104136 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7de7fc640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e015e9b0 0x7fe7e00ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7deffd640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 0x7fe7e0159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 -- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e015e9b0 msgr2=0x7fe7e00ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e015e9b0 0x7fe7e00ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 -- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 msgr2=0x7fe7e0159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 0x7fe7e0159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 -- 172.21.15.106:0/556104136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e015a440 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7deffd640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 0x7fe7e0159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7df7fe640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b9e0 tx=0x7fe7d800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8019070 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d800cb40 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e00b60b0 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8014440 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.701+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00c84b0 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.702+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0068a10 con 0x7fe7e00ca970 2024-04-13T03:13:48.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.703+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7d801c070 con 0x7fe7e00ca970 2024-04-13T03:13:48.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.703+0000 7fe7c77fe640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:48.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.703+0000 7fe7deffd640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:48.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.703+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fe7d808b880 con 0x7fe7e00ca970 2024-04-13T03:13:48.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.704+0000 7fe7deffd640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ca1b0 tx=0x7fe7d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:48.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.708+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7d8059560 con 0x7fe7e00ca970 2024-04-13T03:13:48.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.882+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7e00b2fd0 con 0x7fe7e00ca970 2024-04-13T03:13:48.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.882+0000 7fe7c77fe640 1 -- 172.21.15.106:0/556104136 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fe7d805d860 con 0x7fe7e00ca970 2024-04-13T03:13:48.883 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:48.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 msgr2=0x7fe7cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ca1b0 tx=0x7fe7d0009040 comp rx=0 tx=0).stop 2024-04-13T03:13:48.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 msgr2=0x7fe7e00b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b9e0 tx=0x7fe7d800bea0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 shutdown_connections 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe7cc06c8f0 0x7fe7cc06edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe7e015e9b0 0x7fe7e00ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7e00ca970 0x7fe7e00b5380 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 --2- 172.21.15.106:0/556104136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7e00b3d60 0x7fe7e0159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 >> 172.21.15.106:0/556104136 conn(0x7fe7e008bf10 msgr2=0x7fe7e015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 shutdown_connections 2024-04-13T03:13:48.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.884+0000 7fe7e535e640 1 -- 172.21.15.106:0/556104136 wait complete. 2024-04-13T03:13:48.896 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:13:49.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.999+0000 7f7d4e62d640 1 Processor -- start 2024-04-13T03:13:49.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:48.999+0000 7f7d4e62d640 1 -- start start 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d4e62d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d480cfea0 0x7f7d4815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d4e62d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480704a0 0x7f7d480d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d4e62d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d4815ddd0 con 0x7f7d480d1a60 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d4e62d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4815e100 con 0x7f7d480cfea0 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d4e62d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4815d4d0 con 0x7f7d480704a0 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d480cfea0 0x7f7d4815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d480cfea0 0x7f7d4815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55854/0 (socket says 172.21.15.106:55854) 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 -- 172.21.15.106:0/3015112035 learned_addr learned my addr 172.21.15.106:0/3015112035 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 -- 172.21.15.106:0/3015112035 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480704a0 msgr2=0x7f7d480d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 --2- 172.21.15.106:0/3015112035 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480704a0 0x7f7d480d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 -- 172.21.15.106:0/3015112035 >> v1:172.21.15.114:6789/0 conn(0x7f7d480d1a60 legacy=0x7f7d48160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.000+0000 7f7d47fff640 1 -- 172.21.15.106:0/3015112035 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4815d7b0 con 0x7f7d480cfea0 2024-04-13T03:13:49.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d47fff640 1 --2- 172.21.15.106:0/3015112035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d4815cc10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64addb8c119bc9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d46ffd640 1 -- 172.21.15.106:0/3015112035 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38035020 con 0x7f7d480cfea0 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d46ffd640 1 -- 172.21.15.106:0/3015112035 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d38031640 con 0x7f7d480cfea0 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d46ffd640 1 -- 172.21.15.106:0/3015112035 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802d040 con 0x7f7d480cfea0 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 -- 172.21.15.106:0/3015112035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 msgr2=0x7f7d4815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/3015112035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d4815cc10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 -- 172.21.15.106:0/3015112035 shutdown_connections 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/3015112035 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480704a0 0x7f7d480d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/3015112035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d4815cc10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.001+0000 7f7d4e62d640 1 -- 172.21.15.106:0/3015112035 >> 172.21.15.106:0/3015112035 conn(0x7f7d48071870 msgr2=0x7f7d48071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.002+0000 7f7d4e62d640 1 -- 172.21.15.106:0/3015112035 shutdown_connections 2024-04-13T03:13:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.002+0000 7f7d4e62d640 1 -- 172.21.15.106:0/3015112035 wait complete. 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.003+0000 7f7d4e62d640 1 Processor -- start 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.003+0000 7f7d4e62d640 1 -- start start 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d480704a0 0x7f7d481f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480d1a60 0x7f7d480b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4815d150 con 0x7f7d480cfea0 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d480d1ef0 con 0x7f7d480704a0 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4815f5a0 con 0x7f7d480d1a60 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55868/0 (socket says 172.21.15.106:55868) 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 -- 172.21.15.106:0/4119193118 learned_addr learned my addr 172.21.15.106:0/4119193118 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 -- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480d1a60 msgr2=0x7f7d480b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480d1a60 0x7f7d480b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 -- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d480704a0 msgr2=0x7f7d481f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d480704a0 0x7f7d481f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 -- 172.21.15.106:0/4119193118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d481412e0 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d477fe640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b4a0 tx=0x7f7d3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c014070 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3c010610 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d481f3030 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c010a00 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.004+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d481583c0 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.005+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d481585c0 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.006+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d3c017070 con 0x7f7d480cfea0 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.006+0000 7f7d44ff9640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 0x7f7d3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.006+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f7d3c08b6d0 con 0x7f7d480cfea0 2024-04-13T03:13:49.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.006+0000 7f7d47fff640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 0x7f7d3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.007+0000 7f7d47fff640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 0x7f7d3406ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7d38036000 tx=0x7f7d38030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.010+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d481585c0 con 0x7f7d480cfea0 2024-04-13T03:13:49.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.183+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d48068a10 con 0x7f7d480cfea0 2024-04-13T03:13:49.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.183+0000 7f7d44ff9640 1 -- 172.21.15.106:0/4119193118 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f7d3c059a00 con 0x7f7d480cfea0 2024-04-13T03:13:49.184 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:13:49.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 msgr2=0x7f7d3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 0x7f7d3406ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7d38036000 tx=0x7f7d38030000 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 msgr2=0x7f7d48140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b4a0 tx=0x7f7d3c00b960 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 shutdown_connections 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d3406c920 0x7f7d3406ede0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d480d1a60 0x7f7d480b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d480cfea0 0x7f7d48140da0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 --2- 172.21.15.106:0/4119193118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d480704a0 0x7f7d481f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 >> 172.21.15.106:0/4119193118 conn(0x7f7d48071870 msgr2=0x7f7d480b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 shutdown_connections 2024-04-13T03:13:49.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.185+0000 7f7d4e62d640 1 -- 172.21.15.106:0/4119193118 wait complete. 2024-04-13T03:13:49.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:13:49.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.300+0000 7f5f866de640 1 Processor -- start 2024-04-13T03:13:49.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.300+0000 7f5f866de640 1 -- start start 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f866de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f800704a0 0x7f5f800d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f866de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f800d1a60 0x7f5f8015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f866de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8015ddd0 con 0x7f5f800cfea0 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f866de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8015e100 con 0x7f5f800704a0 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f866de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8015d4d0 con 0x7f5f800d1a60 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f800704a0 0x7f5f800d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f800704a0 0x7f5f800d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55870/0 (socket says 172.21.15.106:55870) 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4124523981 learned_addr learned my addr 172.21.15.106:0/4124523981 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4124523981 >> v2:172.21.15.114:3300/0 conn(0x7f5f800d1a60 msgr2=0x7f5f8015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f84c7b640 1 --2- 172.21.15.106:0/4124523981 >> v2:172.21.15.114:3300/0 conn(0x7f5f800d1a60 0x7f5f8015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/4124523981 >> v2:172.21.15.114:3300/0 conn(0x7f5f800d1a60 0x7f5f8015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4124523981 >> v1:172.21.15.114:6789/0 conn(0x7f5f800cfea0 legacy=0x7f5f8015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:49.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.301+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4124523981 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8015d7b0 con 0x7f5f800704a0 2024-04-13T03:13:49.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/4124523981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800704a0 0x7f5f800d1520 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5f80073f70 tx=0x7f5f7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf638313defad4c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f7effd640 1 -- 172.21.15.106:0/4124523981 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74035020 con 0x7f5f800704a0 2024-04-13T03:13:49.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f7effd640 1 -- 172.21.15.106:0/4124523981 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f74031870 con 0x7f5f800704a0 2024-04-13T03:13:49.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f7effd640 1 -- 172.21.15.106:0/4124523981 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74031c40 con 0x7f5f800704a0 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f866de640 1 -- 172.21.15.106:0/4124523981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800704a0 msgr2=0x7f5f800d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.302+0000 7f5f866de640 1 --2- 172.21.15.106:0/4124523981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800704a0 0x7f5f800d1520 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5f80073f70 tx=0x7f5f7402b910 comp rx=0 tx=0).stop 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 -- 172.21.15.106:0/4124523981 shutdown_connections 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 --2- 172.21.15.106:0/4124523981 >> v2:172.21.15.114:3300/0 conn(0x7f5f800d1a60 0x7f5f8015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 --2- 172.21.15.106:0/4124523981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800704a0 0x7f5f800d1520 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 -- 172.21.15.106:0/4124523981 >> 172.21.15.106:0/4124523981 conn(0x7f5f80071870 msgr2=0x7f5f80071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 -- 172.21.15.106:0/4124523981 shutdown_connections 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.303+0000 7f5f866de640 1 -- 172.21.15.106:0/4124523981 wait complete. 2024-04-13T03:13:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.304+0000 7f5f866de640 1 Processor -- start 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.304+0000 7f5f866de640 1 -- start start 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f800704a0 0x7f5f80150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 0x7f5f8013ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8015ead0 con 0x7f5f800cfea0 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5f800d1ef0 con 0x7f5f800d1a60 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8015ec50 con 0x7f5f800704a0 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55878/0 (socket says 172.21.15.106:55878) 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/1019970877 learned_addr learned my addr 172.21.15.106:0/1019970877 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7ffff640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f800704a0 0x7f5f80150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f84c7b640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 0x7f5f8013ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f800704a0 msgr2=0x7f5f80150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f800704a0 0x7f5f80150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 msgr2=0x7f5f8013ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 0x7f5f8013ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/1019970877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8014e9a0 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f84c7b640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 0x7f5f8013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5f7402b950 tx=0x7f5f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f740443f0 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f74044590 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f80140520 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74044980 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.305+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f80150310 con 0x7f5f800cfea0 2024-04-13T03:13:49.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.306+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f80068a10 con 0x7f5f800cfea0 2024-04-13T03:13:49.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.307+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f74038020 con 0x7f5f800cfea0 2024-04-13T03:13:49.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.307+0000 7f5f7cff9640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 0x7f5f6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.307+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f5f740adf80 con 0x7f5f800cfea0 2024-04-13T03:13:49.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.307+0000 7f5f7ffff640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 0x7f5f6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.308+0000 7f5f7ffff640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 0x7f5f6c06ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f70003d90 tx=0x7f5f70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.312+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f7403f070 con 0x7f5f800cfea0 2024-04-13T03:13:49.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.485+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5f80072360 con 0x7f5f800cfea0 2024-04-13T03:13:49.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.486+0000 7f5f7cff9640 1 -- 172.21.15.106:0/1019970877 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f5f7407c2b0 con 0x7f5f800cfea0 2024-04-13T03:13:49.487 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 msgr2=0x7f5f6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 0x7f5f6c06ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f70003d90 tx=0x7f5f70009040 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 msgr2=0x7f5f8013cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5f7402b950 tx=0x7f5f740046c0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 shutdown_connections 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5f6c06c920 0x7f5f6c06ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f800d1a60 0x7f5f8013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f800cfea0 0x7f5f8013cdd0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 --2- 172.21.15.106:0/1019970877 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5f800704a0 0x7f5f80150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 >> 172.21.15.106:0/1019970877 conn(0x7f5f80071870 msgr2=0x7f5f80154210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 shutdown_connections 2024-04-13T03:13:49.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.488+0000 7f5f866de640 1 -- 172.21.15.106:0/1019970877 wait complete. 2024-04-13T03:13:49.500 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:13:49.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.599+0000 7f9ad0d1e640 1 Processor -- start 2024-04-13T03:13:49.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 -- start start 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 0x7f9acc06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9acc0d07c0 0x7f9acc06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9acc0cef80 con 0x7f9acc15fb50 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc0cf260 con 0x7f9acc0d07c0 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.600+0000 7f9ad0d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0cf540 con 0x7f9acc06bff0 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9aca59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9acc15fb50 0x7f9acc06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44494/0 (socket says 172.21.15.106:44494) 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9aca59c640 1 -- 172.21.15.106:0/1895142943 learned_addr learned my addr 172.21.15.106:0/1895142943 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9acad9d640 1 --2- 172.21.15.106:0/1895142943 >> v2:172.21.15.114:3300/0 conn(0x7f9acc0d07c0 0x7f9acc06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9ac9d9b640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 0x7f9acc06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433133529 0 0) 0x7f9acc0cef80 con 0x7f9acc15fb50 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ab4003520 con 0x7f9acc15fb50 2024-04-13T03:13:49.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9acad9d640 1 -- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 msgr2=0x7f9acc06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3546077032 0 0) 0x7f9ab4003520 con 0x7f9acc15fb50 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9acad9d640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 0x7f9acc06e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9acad9d640 1 -- 172.21.15.106:0/1895142943 >> v1:172.21.15.106:6789/0 conn(0x7f9acc15fb50 legacy=0x7f9acc06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9acad9d640 1 -- 172.21.15.106:0/1895142943 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc0cf820 con 0x7f9acc0d07c0 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.601+0000 7f9ac9d9b640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 0x7f9acc06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.602+0000 7f9acad9d640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc06ebf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40097a0 tx=0x7f9ac402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a1812d8fc9e81b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.602+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4037070 con 0x7f9acc0d07c0 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.602+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac4030510 con 0x7f9acc0d07c0 2024-04-13T03:13:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.602+0000 7f9ac959a640 1 -- 172.21.15.106:0/1895142943 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0019f0 con 0x7f9acc0d07c0 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 msgr2=0x7f9acc06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc06ebf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40097a0 tx=0x7f9ac402b710 comp rx=0 tx=0).stop 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/1895142943 shutdown_connections 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc06ebf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/1895142943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc06bff0 0x7f9acc06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/1895142943 >> 172.21.15.106:0/1895142943 conn(0x7f9acc0cd6a0 msgr2=0x7f9acc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/1895142943 shutdown_connections 2024-04-13T03:13:49.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.603+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/1895142943 wait complete. 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.604+0000 7f9ad0d1e640 1 Processor -- start 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.604+0000 7f9ad0d1e640 1 -- start start 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc15fb50 0x7f9acc072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0ced20 con 0x7f9acc06bff0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0ceba0 con 0x7f9acc0d07c0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc15ef10 con 0x7f9acc15fb50 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55890/0 (socket says 172.21.15.106:55890) 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 -- 172.21.15.106:0/3783697054 learned_addr learned my addr 172.21.15.106:0/3783697054 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ac9d9b640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9acad9d640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc15fb50 0x7f9acc072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 -- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc15fb50 msgr2=0x7f9acc072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc15fb50 0x7f9acc072ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 -- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 msgr2=0x7f9acc072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 -- 172.21.15.106:0/3783697054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc156210 con 0x7f9acc06bff0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9aca59c640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00cc10 tx=0x7f9abc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc017070 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc00ce20 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1564f0 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc012620 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.605+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc156ab0 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.606+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc15f420 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.607+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9abc01a070 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.607+0000 7f9abaffd640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 0x7f9aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.607+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f9abc08bc70 con 0x7f9acc06bff0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.607+0000 7f9ac9d9b640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 0x7f9aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:49.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.608+0000 7f9ac9d9b640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 0x7f9aac06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009560 tx=0x7f9ac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:49.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.612+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9acc15f420 con 0x7f9acc06bff0 2024-04-13T03:13:49.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.797+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9acc06fc90 con 0x7f9acc06bff0 2024-04-13T03:13:49.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.797+0000 7f9abaffd640 1 -- 172.21.15.106:0/3783697054 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f9abc059f20 con 0x7f9acc06bff0 2024-04-13T03:13:49.798 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 msgr2=0x7f9aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 0x7f9aac06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009560 tx=0x7f9ac0008040 comp rx=0 tx=0).stop 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 msgr2=0x7f9acc1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00cc10 tx=0x7f9abc00a430 comp rx=0 tx=0).stop 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 shutdown_connections 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9aac06c8f0 0x7f9aac06edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9acc15fb50 0x7f9acc072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9acc0d07c0 0x7f9acc072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 --2- 172.21.15.106:0/3783697054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9acc06bff0 0x7f9acc1536b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:49.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 >> 172.21.15.106:0/3783697054 conn(0x7f9acc0cd6a0 msgr2=0x7f9acc155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:49.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 shutdown_connections 2024-04-13T03:13:49.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:49.799+0000 7f9ad0d1e640 1 -- 172.21.15.106:0/3783697054 wait complete. 2024-04-13T03:13:49.811 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:13:49.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:49.811 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:49.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:13:49.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:49.870 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:13:49.890 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.036+0000 7f755d698640 1 Processor -- start 2024-04-13T03:13:50.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.036+0000 7f755d698640 1 -- start start 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.037+0000 7f755d698640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7558161f10 0x7f75580cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.037+0000 7f755d698640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75581622f0 0x7f75580cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.037+0000 7f755d698640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75580d0010 con 0x7f75580b7a60 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.037+0000 7f755d698640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f75580d36b0 con 0x7f75581622f0 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.037+0000 7f755d698640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f75580d3990 con 0x7f7558161f10 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75581622f0 0x7f75580cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f7556ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7558161f10 0x7f75580cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f75581622f0 0x7f75580cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55892/0 (socket says 172.21.15.106:55892) 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 -- 172.21.15.106:0/3097145249 learned_addr learned my addr 172.21.15.106:0/3097145249 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 -- 172.21.15.106:0/3097145249 >> v2:172.21.15.114:3300/0 conn(0x7f7558161f10 msgr2=0x7f75580cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:50.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 --2- 172.21.15.106:0/3097145249 >> v2:172.21.15.114:3300/0 conn(0x7f7558161f10 0x7f75580cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 -- 172.21.15.106:0/3097145249 >> v1:172.21.15.114:6789/0 conn(0x7f75580b7a60 legacy=0x7f75580b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.038+0000 7f75567fc640 1 -- 172.21.15.106:0/3097145249 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580d3c70 con 0x7f75581622f0 2024-04-13T03:13:50.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f75567fc640 1 --2- 172.21.15.106:0/3097145249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580cf6b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7558074980 tx=0x7f754802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a96f60eef8f3773 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f7555ffb640 1 -- 172.21.15.106:0/3097145249 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754802f850 con 0x7f75581622f0 2024-04-13T03:13:50.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f7555ffb640 1 -- 172.21.15.106:0/3097145249 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754802f9f0 con 0x7f75581622f0 2024-04-13T03:13:50.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f7555ffb640 1 -- 172.21.15.106:0/3097145249 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754802fdc0 con 0x7f75581622f0 2024-04-13T03:13:50.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 -- 172.21.15.106:0/3097145249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 msgr2=0x7f75580cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 --2- 172.21.15.106:0/3097145249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580cf6b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7558074980 tx=0x7f754802ba30 comp rx=0 tx=0).stop 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 -- 172.21.15.106:0/3097145249 shutdown_connections 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 --2- 172.21.15.106:0/3097145249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580cf6b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 --2- 172.21.15.106:0/3097145249 >> v2:172.21.15.114:3300/0 conn(0x7f7558161f10 0x7f75580cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.039+0000 7f755d698640 1 -- 172.21.15.106:0/3097145249 >> 172.21.15.106:0/3097145249 conn(0x7f75580d4780 msgr2=0x7f75580d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.040+0000 7f755d698640 1 -- 172.21.15.106:0/3097145249 shutdown_connections 2024-04-13T03:13:50.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.040+0000 7f755d698640 1 -- 172.21.15.106:0/3097145249 wait complete. 2024-04-13T03:13:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.041+0000 7f755d698640 1 Processor -- start 2024-04-13T03:13:50.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.041+0000 7f755d698640 1 -- start start 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75580b7a60 0x7f755815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7558161f10 0x7f75581f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b8590 con 0x7f75581622f0 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f75580d4230 con 0x7f7558161f10 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f755d698640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f75580b8710 con 0x7f75580b7a60 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55902/0 (socket says 172.21.15.106:55902) 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 -- 172.21.15.106:0/1817378017 learned_addr learned my addr 172.21.15.106:0/1817378017 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75567fc640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7558161f10 0x7f75581f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f7556ffd640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75580b7a60 0x7f755815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 -- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75580b7a60 msgr2=0x7f755815f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75580b7a60 0x7f755815f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 -- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7558161f10 msgr2=0x7f75581f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7558161f10 0x7f75581f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.042+0000 7f75577fe640 1 -- 172.21.15.106:0/1817378017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580ab190 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f75577fe640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f755000c7a0 tx=0x7f755000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550003b70 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7550003d30 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75581f5480 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755000aa90 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.043+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75581f5a40 con 0x7f75581622f0 2024-04-13T03:13:50.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.044+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75580ab360 con 0x7f75581622f0 2024-04-13T03:13:50.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.044+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f755000ac30 con 0x7f75581622f0 2024-04-13T03:13:50.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.045+0000 7f754f7fe640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.045+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f755008ba60 con 0x7f75581622f0 2024-04-13T03:13:50.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.045+0000 7f7556ffd640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.045+0000 7f7556ffd640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 0x7f753806edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7540009b40 tx=0x7f7540009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.049+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7550059740 con 0x7f75581622f0 2024-04-13T03:13:50.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.269+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7558069120 con 0x7f75581622f0 2024-04-13T03:13:50.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.269+0000 7f754f7fe640 1 -- 172.21.15.106:0/1817378017 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f755005da40 con 0x7f75581622f0 2024-04-13T03:13:50.271 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:50.271 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:39.107280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6834/390879687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":390879687},{"type":"v1","addr":"172.21.15.114:6834","nonce":390879687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":5}]} 2024-04-13T03:13:50.271 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 36 2024-04-13T03:13:50.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 msgr2=0x7f753806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 0x7f753806edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7540009b40 tx=0x7f7540009190 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 msgr2=0x7f75580acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f755000c7a0 tx=0x7f755000cc60 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 shutdown_connections 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581622f0 0x7f75580acc80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7558161f10 0x7f75581f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 --2- 172.21.15.106:0/1817378017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f75580b7a60 0x7f755815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 >> 172.21.15.106:0/1817378017 conn(0x7f75580d4780 msgr2=0x7f75580b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 shutdown_connections 2024-04-13T03:13:50.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.271+0000 7f755d698640 1 -- 172.21.15.106:0/1817378017 wait complete. 2024-04-13T03:13:50.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-13T03:13:50.426 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:13:50.426 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:13:50.426 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:13:50.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.527+0000 7f121041c640 1 Processor -- start 2024-04-13T03:13:50.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.528+0000 7f121041c640 1 -- start start 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.528+0000 7f121041c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f120806e740 0x7f120815c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.528+0000 7f121041c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120806d180 con 0x7f12081592b0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.528+0000 7f121041c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12080ce180 con 0x7f120815c7a0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.528+0000 7f121041c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f120806ec90 con 0x7f120806e740 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120e1b8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f12081592b0 0x7f120815bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44498/0 (socket says 172.21.15.106:44498) 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120e1b8640 1 -- 172.21.15.106:0/2636254856 learned_addr learned my addr 172.21.15.106:0/2636254856 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d9b7640 1 --2- 172.21.15.106:0/2636254856 >> v2:172.21.15.114:3300/0 conn(0x7f120806e740 0x7f120815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580237201 0 0) 0x7f120806d180 con 0x7f12081592b0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11f4005730 con 0x7f12081592b0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346980932 0 0) 0x7f12080ce180 con 0x7f120815c7a0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f120806d180 con 0x7f120815c7a0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2849277708 0 0) 0x7f11f4005730 con 0x7f12081592b0 2024-04-13T03:13:50.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d9b7640 1 -- 172.21.15.106:0/2636254856 >> v1:172.21.15.114:6789/0 conn(0x7f120815c7a0 legacy=0x7f12080ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d9b7640 1 -- 172.21.15.106:0/2636254856 >> v1:172.21.15.106:6789/0 conn(0x7f12081592b0 legacy=0x7f120815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.529+0000 7f120d9b7640 1 -- 172.21.15.106:0/2636254856 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120806ef70 con 0x7f120806e740 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f120d9b7640 1 --2- 172.21.15.106:0/2636254856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f120815c260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e152cda5012e99f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204038070 con 0x7f120806e740 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204031500 con 0x7f120806e740 2024-04-13T03:13:50.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f120d1b6640 1 -- 172.21.15.106:0/2636254856 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8001670 con 0x7f120806e740 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f121041c640 1 -- 172.21.15.106:0/2636254856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 msgr2=0x7f120815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.530+0000 7f121041c640 1 --2- 172.21.15.106:0/2636254856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f120815c260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402b820 comp rx=0 tx=0).stop 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.531+0000 7f121041c640 1 -- 172.21.15.106:0/2636254856 shutdown_connections 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.531+0000 7f121041c640 1 --2- 172.21.15.106:0/2636254856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f120815c260 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.531+0000 7f121041c640 1 -- 172.21.15.106:0/2636254856 >> 172.21.15.106:0/2636254856 conn(0x7f12080b2600 msgr2=0x7f12080d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.531+0000 7f121041c640 1 -- 172.21.15.106:0/2636254856 shutdown_connections 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.531+0000 7f121041c640 1 -- 172.21.15.106:0/2636254856 wait complete. 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.532+0000 7f121041c640 1 Processor -- start 2024-04-13T03:13:50.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.532+0000 7f121041c640 1 -- start start 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.532+0000 7f121041c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f12081426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.532+0000 7f121041c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12081592b0 0x7f1208155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f12080ced30 con 0x7f120815c7a0 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f120806f530 con 0x7f120806e740 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f12080ceeb0 con 0x7f12081592b0 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55914/0 (socket says 172.21.15.106:55914) 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 -- 172.21.15.106:0/3164637627 learned_addr learned my addr 172.21.15.106:0/3164637627 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120d9b7640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12081592b0 0x7f1208155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e1b8640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f12081426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 -- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12081592b0 msgr2=0x7f1208155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12081592b0 0x7f1208155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 -- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 msgr2=0x7f12081426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f12081426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 -- 172.21.15.106:0/3164637627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1208142f80 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e1b8640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f12081426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f120e9b9640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f120000c6b0 tx=0x7f120000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200003b70 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1200003d10 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200005310 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1208143260 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.533+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1208143820 con 0x7f120815c7a0 2024-04-13T03:13:50.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.534+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1208068a10 con 0x7f120815c7a0 2024-04-13T03:13:50.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.535+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12000054b0 con 0x7f120815c7a0 2024-04-13T03:13:50.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.535+0000 7f11feffd640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 0x7f11ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.535+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f120008bc30 con 0x7f120815c7a0 2024-04-13T03:13:50.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.535+0000 7f120e1b8640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 0x7f11ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.536+0000 7f120e1b8640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 0x7f11ec06edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11f80017e0 tx=0x7f11f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.540+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1200059e40 con 0x7f120815c7a0 2024-04-13T03:13:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.749+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1208070050 con 0x7f120815c7a0 2024-04-13T03:13:50.750 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:13:50.750+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:13:50.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.751+0000 7f11feffd640 1 -- 172.21.15.106:0/3164637627 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f1200059740 con 0x7f120815c7a0 2024-04-13T03:13:50.752 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.753+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 msgr2=0x7f11ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.753+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 0x7f11ec06edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11f80017e0 tx=0x7f11f8009040 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.753+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 msgr2=0x7f12080a93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.753+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f120000c6b0 tx=0x7f120000cb70 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 shutdown_connections 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f11ec06c8f0 0x7f11ec06edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f12081592b0 0x7f1208155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f120815c7a0 0x7f12080a93e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 --2- 172.21.15.106:0/3164637627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f120806e740 0x7f12081426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 >> 172.21.15.106:0/3164637627 conn(0x7f12080b2600 msgr2=0x7f1208159690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 shutdown_connections 2024-04-13T03:13:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.754+0000 7f121041c640 1 -- 172.21.15.106:0/3164637627 wait complete. 2024-04-13T03:13:50.757 INFO:tasks.ceph.mds.d.smithi114.stderr: -4624> 2024-04-13T03:13:43.025+0000 7f56249e5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:13:50.765 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-13T03:13:50.786 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:13:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.808+0000 7f4eeec65f40 1 Processor -- start 2024-04-13T03:13:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.808+0000 7f4eeec65f40 1 -- start start 2024-04-13T03:13:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe2926c0 0x55ddfe24a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe292aa0 0x55ddfe2485a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ddfe1f0d00 con 0x55ddfe305e70 2024-04-13T03:13:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ddfe1f4d40 con 0x55ddfe2926c0 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ddfe31dc30 con 0x55ddfe292aa0 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55920/0 (socket says 172.21.15.106:55920) 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 -- 172.21.15.106:0/2859595538 learned_addr learned my addr 172.21.15.106:0/2859595538 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 -- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe292aa0 msgr2=0x55ddfe2485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe292aa0 0x55ddfe2485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 -- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe2926c0 msgr2=0x55ddfe24a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe2926c0 0x55ddfe24a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 -- 172.21.15.106:0/2859595538 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfe305410 con 0x55ddfe305e70 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eecd50640 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001e10 tx=0x7f4ee400d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ca20f13d4b2a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eead4c640 1 -- 172.21.15.106:0/2859595538 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400e910 con 0x55ddfe305e70 2024-04-13T03:13:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eead4c640 1 -- 172.21.15.106:0/2859595538 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee400eaf0 con 0x55ddfe305e70 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.809+0000 7f4eead4c640 1 -- 172.21.15.106:0/2859595538 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400b430 con 0x55ddfe305e70 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- 172.21.15.106:0/2859595538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 msgr2=0x55ddfe31c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001e10 tx=0x7f4ee400d860 comp rx=0 tx=0).stop 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- 172.21.15.106:0/2859595538 shutdown_connections 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe292aa0 0x55ddfe2485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31c5d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/2859595538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe2926c0 0x55ddfe24a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- 172.21.15.106:0/2859595538 >> 172.21.15.106:0/2859595538 conn(0x55ddfe273ae0 msgr2=0x55ddfe247100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- 172.21.15.106:0/2859595538 shutdown_connections 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- 172.21.15.106:0/2859595538 wait complete. 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 Processor -- start 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.810+0000 7f4eeec65f40 1 -- start start 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe2926c0 0x55ddfe26f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe292aa0 0x55ddfe3020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ddfe247c30 con 0x55ddfe305e70 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ddfe247990 con 0x55ddfe292aa0 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eeec65f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ddfe247ed0 con 0x55ddfe2926c0 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55936/0 (socket says 172.21.15.106:55936) 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 -- 172.21.15.106:0/557261002 learned_addr learned my addr 172.21.15.106:0/557261002 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eebd4e640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe292aa0 0x55ddfe3020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eec54f640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe2926c0 0x55ddfe26f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 -- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe2926c0 msgr2=0x55ddfe26f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ddfe2926c0 0x55ddfe26f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 -- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe292aa0 msgr2=0x55ddfe3020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfe292aa0 0x55ddfe3020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.811+0000 7f4eecd50640 1 -- 172.21.15.106:0/557261002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfe248760 con 0x55ddfe305e70 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4eecd50640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31b9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d8a0 tx=0x7f4ee4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4ee9d4a640 1 -- 172.21.15.106:0/557261002 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400e910 con 0x55ddfe305e70 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4ee9d4a640 1 -- 172.21.15.106:0/557261002 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4008070 con 0x55ddfe305e70 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddfe248a40 con 0x55ddfe305e70 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4ee9d4a640 1 -- 172.21.15.106:0/557261002 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4022680 con 0x55ddfe305e70 2024-04-13T03:13:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.812+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfe26f730 con 0x55ddfe305e70 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.813+0000 7f4ee9d4a640 1 -- 172.21.15.106:0/557261002 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ee4022820 con 0x55ddfe305e70 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.813+0000 7f4ee9d4a640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4ed806b300 0x7f4ed806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eec54f640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4ed806b300 0x7f4ed806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4ee9d4a640 1 -- 172.21.15.106:0/557261002 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f4ee408c770 con 0x55ddfe305e70 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55ddfe26eb80 0x55ddfe2717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) -- 0x55ddfe271cf0 con 0x55ddfe26eb80 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eebd4e640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55ddfe26eb80 0x55ddfe2717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eec54f640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4ed806b300 0x7f4ed806d7c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4edc056000 tx=0x7f4edc051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.814+0000 7f4eebd4e640 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55ddfe26eb80 0x55ddfe2717b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.815+0000 7f4eebd4e640 1 -- 172.21.15.106:0/557261002 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ee0009040 con 0x55ddfe26eb80 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55ddfe26eb80 msgr2=0x55ddfe2717b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55ddfe26eb80 0x55ddfe2717b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4ed806b300 msgr2=0x7f4ed806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4ed806b300 0x7f4ed806d7c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4edc056000 tx=0x7f4edc051000 comp rx=0 tx=0).stop 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 msgr2=0x55ddfe31b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 --2- 172.21.15.106:0/557261002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ddfe305e70 0x55ddfe31b9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d8a0 tx=0x7f4ee4000ed0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eecd50640 1 -- 172.21.15.106:0/557261002 reap_dead start 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 shutdown_connections 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 >> 172.21.15.106:0/557261002 conn(0x55ddfe273ae0 msgr2=0x55ddfe247100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 shutdown_connections 2024-04-13T03:13:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.816+0000 7f4eeec65f40 1 -- 172.21.15.106:0/557261002 wait complete. 2024-04-13T03:13:50.822 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:13:50.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.965+0000 7f06478c4640 1 Processor -- start 2024-04-13T03:13:50.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.965+0000 7f06478c4640 1 -- start start 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.966+0000 7f06478c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.966+0000 7f06478c4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06400b53a0 0x7f06400d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.966+0000 7f06478c4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064006d080 con 0x7f064006eb10 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.966+0000 7f06478c4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f06400cdc10 con 0x7f06400b53a0 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.966+0000 7f06478c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06400cd650 con 0x7f064015e8f0 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0645660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0644e5f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06400b53a0 0x7f06400d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0645660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:49864/0 (socket says 172.21.15.106:49864) 2024-04-13T03:13:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0645660640 1 -- 172.21.15.106:0/2307907506 learned_addr learned my addr 172.21.15.106:0/2307907506 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0645e61640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f064006eb10 0x7f064006db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:37922/0 (socket says 172.21.15.106:37922) 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0644e5f640 1 -- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 msgr2=0x7f06400d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0644e5f640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0644e5f640 1 -- 172.21.15.106:0/2307907506 >> v1:172.21.15.114:6789/0 conn(0x7f064006eb10 legacy=0x7f064006db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0637fff640 1 -- 172.21.15.106:0/2307907506 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705760675 0 0) 0x7f063c001490 con 0x7f064006eb10 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0644e5f640 1 -- 172.21.15.106:0/2307907506 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06400cd930 con 0x7f06400b53a0 2024-04-13T03:13:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.967+0000 7f0645660640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f0644e5f640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b53a0 0x7f06400d07c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f06300097a0 tx=0x7f063002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21950ca9fa914ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f0637fff640 1 -- 172.21.15.106:0/2307907506 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630035020 con 0x7f06400b53a0 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f0637fff640 1 -- 172.21.15.106:0/2307907506 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630031830 con 0x7f06400b53a0 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f0637fff640 1 -- 172.21.15.106:0/2307907506 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630031c00 con 0x7f06400b53a0 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 -- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b53a0 msgr2=0x7f06400d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b53a0 0x7f06400d07c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f06300097a0 tx=0x7f063002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 -- 172.21.15.106:0/2307907506 shutdown_connections 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b53a0 0x7f06400d07c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 --2- 172.21.15.106:0/2307907506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f064015e8f0 0x7f06400d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 -- 172.21.15.106:0/2307907506 >> 172.21.15.106:0/2307907506 conn(0x7f06400d2550 msgr2=0x7f06400d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 -- 172.21.15.106:0/2307907506 shutdown_connections 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.968+0000 7f06478c4640 1 -- 172.21.15.106:0/2307907506 wait complete. 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.969+0000 7f06478c4640 1 Processor -- start 2024-04-13T03:13:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 -- start start 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06400b53a0 0x7f064015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015e8f0 0x7f0640159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f064015ecd0 con 0x7f064006eb10 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f064006d5e0 con 0x7f064015e8f0 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f06478c4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f064015ee50 con 0x7f06400b53a0 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f0645660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f0645660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55948/0 (socket says 172.21.15.106:55948) 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f0645660640 1 -- 172.21.15.106:0/860388509 learned_addr learned my addr 172.21.15.106:0/860388509 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:50.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f0644e5f640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06400b53a0 0x7f064015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.970+0000 7f0645e61640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015e8f0 0x7f0640159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 -- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06400b53a0 msgr2=0x7f064015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06400b53a0 0x7f064015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 -- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015e8f0 msgr2=0x7f0640159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015e8f0 0x7f0640159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 -- 172.21.15.106:0/860388509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064015f760 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0645660640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f063800e9d0 tx=0x7f063800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638017070 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063800ca60 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064015fa40 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063800ce10 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.971+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06400c9b50 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.972+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0640069120 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.972+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f063801a070 con 0x7f064006eb10 2024-04-13T03:13:50.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.973+0000 7f0635ffb640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 0x7f062406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:50.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.973+0000 7f0644e5f640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 0x7f062406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:50.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.973+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f063808bf30 con 0x7f064006eb10 2024-04-13T03:13:50.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.973+0000 7f0644e5f640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 0x7f062406edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0630036000 tx=0x7f063003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:50.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:50.978+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0638059bc0 con 0x7f064006eb10 2024-04-13T03:13:51.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.190+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f06400cac10 con 0x7f064006eb10 2024-04-13T03:13:51.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.753+0000 7f0635ffb640 1 -- 172.21.15.106:0/860388509 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v38) ==== 162+0+0 (secure 0 0 0) 0x7f063805dec0 con 0x7f064006eb10 2024-04-13T03:13:51.754 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 msgr2=0x7f062406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 0x7f062406edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0630036000 tx=0x7f063003c040 comp rx=0 tx=0).stop 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 msgr2=0x7f06400cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f063800e9d0 tx=0x7f063800ee90 comp rx=0 tx=0).stop 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 shutdown_connections 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062406c8f0 0x7f062406edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06400b53a0 0x7f064015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015e8f0 0x7f0640159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 --2- 172.21.15.106:0/860388509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f064006eb10 0x7f06400cba60 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 >> 172.21.15.106:0/860388509 conn(0x7f06400d2550 msgr2=0x7f06400b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:51.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 shutdown_connections 2024-04-13T03:13:51.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.756+0000 7f06478c4640 1 -- 172.21.15.106:0/860388509 wait complete. 2024-04-13T03:13:51.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.872+0000 7f11481cc640 1 Processor -- start 2024-04-13T03:13:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.872+0000 7f11481cc640 1 -- start start 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f11481cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f1140167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f11481cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1140161390 con 0x7f1140070090 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f11481cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11400d05b0 con 0x7f11400b0390 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f11481cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f11400ceb50 con 0x7f11401658f0 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1145f68640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1140070090 0x7f1140160570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44504/0 (socket says 172.21.15.106:44504) 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1145f68640 1 -- 172.21.15.106:0/3504854550 learned_addr learned my addr 172.21.15.106:0/3504854550 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309807034 0 0) 0x7f1140161390 con 0x7f1140070090 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1146769640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f1140167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f112c003520 con 0x7f1140070090 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3762307135 0 0) 0x7f112c003520 con 0x7f1140070090 2024-04-13T03:13:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 msgr2=0x7f1140167ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f1140167ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 >> v1:172.21.15.114:6789/0 conn(0x7f11400b0390 legacy=0x7f1140160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1140070470 con 0x7f1140070090 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f113c001670 con 0x7f1140070090 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.873+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 >> v1:172.21.15.106:6789/0 conn(0x7f1140070090 legacy=0x7f1140160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f112c0039e0 0x7f112c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f112c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f112c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11400ceb50 con 0x7f112c004d40 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f112c00bb50 con 0x7f112c0039e0 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f112c00bd50 con 0x7f11401658f0 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f112c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1145f68640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f112c0039e0 0x7f112c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1145767640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f112c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 -- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 msgr2=0x7f112c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11401658f0 0x7f112c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 -- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f112c0039e0 msgr2=0x7f112c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f112c0039e0 0x7f112c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.874+0000 7f1146769640 1 -- 172.21.15.106:0/3504854550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c003520 con 0x7f112c004d40 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f1146769640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f112c0051c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f114006e0a0 tx=0x7f112c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecddbdb5f05a072 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138039070 con 0x7f112c004d40 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113802cb90 con 0x7f112c004d40 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f1144f66640 1 -- 172.21.15.106:0/3504854550 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138009c10 con 0x7f112c004d40 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 -- 172.21.15.106:0/3504854550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 msgr2=0x7f112c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 --2- 172.21.15.106:0/3504854550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f112c0051c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f114006e0a0 tx=0x7f112c005830 comp rx=0 tx=0).stop 2024-04-13T03:13:51.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f1146769640 1 -- 172.21.15.106:0/3504854550 reap_dead start 2024-04-13T03:13:51.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 -- 172.21.15.106:0/3504854550 shutdown_connections 2024-04-13T03:13:51.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 -- 172.21.15.106:0/3504854550 >> 172.21.15.106:0/3504854550 conn(0x7f11400cd6d0 msgr2=0x7f11400cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:51.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 -- 172.21.15.106:0/3504854550 shutdown_connections 2024-04-13T03:13:51.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.875+0000 7f11481cc640 1 -- 172.21.15.106:0/3504854550 wait complete. 2024-04-13T03:13:51.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 Processor -- start 2024-04-13T03:13:51.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 -- start start 2024-04-13T03:13:51.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 0x7f1140152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 0x7f1140156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f114015dcc0 con 0x7f112c004d40 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f11400cf110 con 0x7f1140140ef0 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f11481cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f11400ceda0 con 0x7f1140152e50 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f1145f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f1145f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55972/0 (socket says 172.21.15.106:55972) 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.877+0000 7f1145f68640 1 -- 172.21.15.106:0/4173473357 learned_addr learned my addr 172.21.15.106:0/4173473357 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1146769640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 0x7f1140156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145767640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 0x7f1140152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 -- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 msgr2=0x7f1140156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 0x7f1140156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 -- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 msgr2=0x7f1140152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 0x7f1140152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 -- 172.21.15.106:0/4173473357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401f4830 con 0x7f112c004d40 2024-04-13T03:13:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1146769640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 0x7f1140156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145767640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 0x7f1140152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f1145f68640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f113c007290 tx=0x7f113c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00f960 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c018070 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401f4ad0 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00fd20 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.878+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401f5090 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.879+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1140153260 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.880+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f113c01d460 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.880+0000 7f11367fc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 0x7f112406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.880+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f113c08bd30 con 0x7f112c004d40 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.880+0000 7f1145767640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 0x7f112406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:51.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.880+0000 7f1145767640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 0x7f112406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11400b2380 tx=0x7f1130009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:51.884+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1140153260 con 0x7f112c004d40 2024-04-13T03:13:52.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.097+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11400d0840 con 0x7f112c004d40 2024-04-13T03:13:52.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.097+0000 7f11367fc640 1 -- 172.21.15.106:0/4173473357 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5183 (secure 0 0 0) 0x7f113c05a010 con 0x7f112c004d40 2024-04-13T03:13:52.098 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:52.099 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:51.756894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-13T03:13:52.099 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 39 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 msgr2=0x7f112406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 0x7f112406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11400b2380 tx=0x7f1130009040 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 msgr2=0x7f114009e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f113c007290 tx=0x7f113c007750 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 shutdown_connections 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f112406c920 0x7f112406ede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1140152e50 0x7f1140156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1140140ef0 0x7f1140152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 --2- 172.21.15.106:0/4173473357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f112c004d40 0x7f114009e000 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.099+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 >> 172.21.15.106:0/4173473357 conn(0x7f11400cd6d0 msgr2=0x7f11400cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.100+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 shutdown_connections 2024-04-13T03:13:52.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:52.100+0000 7f11481cc640 1 -- 172.21.15.106:0/4173473357 wait complete. 2024-04-13T03:13:52.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:13:37.103773+0000', 'modified': '2024-04-13T03:13:51.756894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6816': {'gid': 6816, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/2254289128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2254289128}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2254289128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:13:52.514 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6816:up:replay 2024-04-13T03:13:53.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:53.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.623+0000 7f53a17bb640 1 Processor -- start 2024-04-13T03:13:53.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.623+0000 7f53a17bb640 1 -- start start 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f53a17bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f539c0cd550 0x7f539c162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f53a17bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f539c15ed60 0x7f539c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f53a17bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c15fe40 con 0x7f539c15e980 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f53a17bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f539c1633f0 con 0x7f539c15ed60 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f53a17bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f539c15f2c0 con 0x7f539c0cd550 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f539b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f539c15ed60 0x7f539c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f539b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f539c15ed60 0x7f539c06d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51442/0 (socket says 172.21.15.106:51442) 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.624+0000 7f539b7fe640 1 -- 172.21.15.106:0/1860900254 learned_addr learned my addr 172.21.15.106:0/1860900254 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539a7fc640 1 --2- 172.21.15.106:0/1860900254 >> v2:172.21.15.114:3300/0 conn(0x7f539c0cd550 0x7f539c162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539b7fe640 1 -- 172.21.15.106:0/1860900254 >> v2:172.21.15.114:3300/0 conn(0x7f539c0cd550 msgr2=0x7f539c162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:53.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539b7fe640 1 --2- 172.21.15.106:0/1860900254 >> v2:172.21.15.114:3300/0 conn(0x7f539c0cd550 0x7f539c162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539b7fe640 1 -- 172.21.15.106:0/1860900254 >> v1:172.21.15.114:6789/0 conn(0x7f539c15e980 legacy=0x7f539c0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539b7fe640 1 -- 172.21.15.106:0/1860900254 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c15f5a0 con 0x7f539c15ed60 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.625+0000 7f539b7fe640 1 --2- 172.21.15.106:0/1860900254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c15ed60 0x7f539c06d370 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f539c0c6030 tx=0x7f539402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebeb8c2bbb0859 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f5399ffb640 1 -- 172.21.15.106:0/1860900254 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539402f850 con 0x7f539c15ed60 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f5399ffb640 1 -- 172.21.15.106:0/1860900254 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539402f9f0 con 0x7f539c15ed60 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f5399ffb640 1 -- 172.21.15.106:0/1860900254 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539402fdc0 con 0x7f539c15ed60 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 -- 172.21.15.106:0/1860900254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c15ed60 msgr2=0x7f539c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1860900254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c15ed60 0x7f539c06d370 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f539c0c6030 tx=0x7f539402ba30 comp rx=0 tx=0).stop 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 -- 172.21.15.106:0/1860900254 shutdown_connections 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1860900254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c15ed60 0x7f539c06d370 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1860900254 >> v2:172.21.15.114:3300/0 conn(0x7f539c0cd550 0x7f539c162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 -- 172.21.15.106:0/1860900254 >> 172.21.15.106:0/1860900254 conn(0x7f539c0d2550 msgr2=0x7f539c0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:53.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 -- 172.21.15.106:0/1860900254 shutdown_connections 2024-04-13T03:13:53.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.626+0000 7f53a17bb640 1 -- 172.21.15.106:0/1860900254 wait complete. 2024-04-13T03:13:53.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 Processor -- start 2024-04-13T03:13:53.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 -- start start 2024-04-13T03:13:53.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f539c15e980 0x7f539c0c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f539c15ed60 0x7f539c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0d1e20 con 0x7f539c0cd550 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f539c15fb60 con 0x7f539c15ed60 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f53a17bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0d1fa0 con 0x7f539c15e980 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f539affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.628+0000 7f539affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51450/0 (socket says 172.21.15.106:51450) 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 -- 172.21.15.106:0/1518811931 learned_addr learned my addr 172.21.15.106:0/1518811931 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539a7fc640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f539c15e980 0x7f539c0c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539b7fe640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f539c15ed60 0x7f539c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 -- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f539c15e980 msgr2=0x7f539c0c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f539c15e980 0x7f539c0c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 -- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f539c15ed60 msgr2=0x7f539c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f539c15ed60 0x7f539c0cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 -- 172.21.15.106:0/1518811931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c1f4d20 con 0x7f539c0cd550 2024-04-13T03:13:53.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f539affd640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c790 tx=0x7f538c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c016020 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c006d50 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c0caac0 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c003bb0 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.629+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c0cb080 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.630+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c069120 con 0x7f539c0cd550 2024-04-13T03:13:53.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.631+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f538c00a850 con 0x7f539c0cd550 2024-04-13T03:13:53.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.631+0000 7f53837fe640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 0x7f538806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:53.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.631+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f538c08b9f0 con 0x7f539c0cd550 2024-04-13T03:13:53.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.631+0000 7f539a7fc640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 0x7f538806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:53.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.631+0000 7f539a7fc640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 0x7f538806edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f539c06e340 tx=0x7f5390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:53.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.635+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f538c020020 con 0x7f539c0cd550 2024-04-13T03:13:53.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.852+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f539c0b84d0 con 0x7f539c0cd550 2024-04-13T03:13:53.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.852+0000 7f53837fe640 1 -- 172.21.15.106:0/1518811931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5183 (secure 0 0 0) 0x7f539c1f4d20 con 0x7f539c0cd550 2024-04-13T03:13:53.853 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:53.853 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:52.764388+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-13T03:13:53.854 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 41 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 msgr2=0x7f538806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 0x7f538806edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f539c06e340 tx=0x7f5390008040 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 msgr2=0x7f539c1f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c790 tx=0x7f538c00cc50 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 shutdown_connections 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f538806c8f0 0x7f538806edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f539c15e980 0x7f539c0c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f539c15ed60 0x7f539c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 --2- 172.21.15.106:0/1518811931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f539c0cd550 0x7f539c1f47e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:53.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 >> 172.21.15.106:0/1518811931 conn(0x7f539c0d2550 msgr2=0x7f539c1623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:53.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 shutdown_connections 2024-04-13T03:13:53.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:53.855+0000 7f53a17bb640 1 -- 172.21.15.106:0/1518811931 wait complete. 2024-04-13T03:13:54.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:13:37.103773+0000', 'modified': '2024-04-13T03:13:52.764388+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6816': {'gid': 6816, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.106:6837/2254289128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2254289128}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2254289128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:13:54.269 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6816:up:rejoin 2024-04-13T03:13:55.133 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:13:55.136 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:13:55.270 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:13:55.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.377+0000 7f8706595640 1 Processor -- start 2024-04-13T03:13:55.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.377+0000 7f8706595640 1 -- start start 2024-04-13T03:13:55.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8706595640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f870015e9b0 0x7f87000d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8706595640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f87000b3d60 0x7f87000d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8706595640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87000cfd20 con 0x7f870015ed90 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8706595640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8700162af0 con 0x7f870015e9b0 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8706595640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8700162dd0 con 0x7f87000b3d60 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86fffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f870015e9b0 0x7f87000d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86ff7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f870015ed90 0x7f87000d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40758/0 (socket says 172.21.15.106:40758) 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86ff7fe640 1 -- 172.21.15.106:0/2849740888 learned_addr learned my addr 172.21.15.106:0/2849740888 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86fffff640 1 -- 172.21.15.106:0/2849740888 >> v2:172.21.15.114:3300/0 conn(0x7f87000b3d60 msgr2=0x7f87000d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f8704b32640 1 --2- 172.21.15.106:0/2849740888 >> v2:172.21.15.114:3300/0 conn(0x7f87000b3d60 0x7f87000d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86fffff640 1 --2- 172.21.15.106:0/2849740888 >> v2:172.21.15.114:3300/0 conn(0x7f87000b3d60 0x7f87000d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86fffff640 1 -- 172.21.15.106:0/2849740888 >> v1:172.21.15.106:6789/0 conn(0x7f870015ed90 legacy=0x7f87000d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86feffd640 1 -- 172.21.15.106:0/2849740888 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111569263 0 0) 0x7f87000cfd20 con 0x7f870015ed90 2024-04-13T03:13:55.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.378+0000 7f86fffff640 1 -- 172.21.15.106:0/2849740888 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87001630b0 con 0x7f870015e9b0 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.379+0000 7f86fffff640 1 --2- 172.21.15.106:0/2849740888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870015e9b0 0x7f87000d2300 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f87000cb7b0 tx=0x7f86f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a12b9841e5ad77 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.379+0000 7f86feffd640 1 -- 172.21.15.106:0/2849740888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4034070 con 0x7f870015e9b0 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.379+0000 7f86feffd640 1 -- 172.21.15.106:0/2849740888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f4030610 con 0x7f870015e9b0 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.379+0000 7f86feffd640 1 -- 172.21.15.106:0/2849740888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40309e0 con 0x7f870015e9b0 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 -- 172.21.15.106:0/2849740888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870015e9b0 msgr2=0x7f87000d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 --2- 172.21.15.106:0/2849740888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870015e9b0 0x7f87000d2300 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f87000cb7b0 tx=0x7f86f402b960 comp rx=0 tx=0).stop 2024-04-13T03:13:55.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 -- 172.21.15.106:0/2849740888 shutdown_connections 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 --2- 172.21.15.106:0/2849740888 >> v2:172.21.15.114:3300/0 conn(0x7f87000b3d60 0x7f87000d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 --2- 172.21.15.106:0/2849740888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870015e9b0 0x7f87000d2300 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 -- 172.21.15.106:0/2849740888 >> 172.21.15.106:0/2849740888 conn(0x7f870008bf10 msgr2=0x7f87000b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 -- 172.21.15.106:0/2849740888 shutdown_connections 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.380+0000 7f8706595640 1 -- 172.21.15.106:0/2849740888 wait complete. 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.381+0000 7f8706595640 1 Processor -- start 2024-04-13T03:13:55.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.381+0000 7f8706595640 1 -- start start 2024-04-13T03:13:55.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 0x7f870014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 0x7f87000ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8700162800 con 0x7f87000cc160 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8700162680 con 0x7f870015e9b0 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8706595640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f87000d4d00 con 0x7f87000b3d60 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8704b32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8704b32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51472/0 (socket says 172.21.15.106:51472) 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f8704b32640 1 -- 172.21.15.106:0/3254554997 learned_addr learned my addr 172.21.15.106:0/3254554997 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f86fffff640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 0x7f870014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.382+0000 7f86ff7fe640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 0x7f87000ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 -- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 msgr2=0x7f870014c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 0x7f870014c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 -- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 msgr2=0x7f87000ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 0x7f87000ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 -- 172.21.15.106:0/3254554997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87000b5210 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f86fffff640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 0x7f870014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f86ff7fe640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 0x7f87000ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8704b32640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f86f800cda0 tx=0x7f86f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8003b90 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f8003d30 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004240 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87000b54f0 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.383+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87000b5ab0 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.384+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8700075180 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.384+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86f80043e0 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.385+0000 7f86fcff9640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 0x7f86ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.385+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f86f808c7d0 con 0x7f87000cc160 2024-04-13T03:13:55.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.385+0000 7f86fffff640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 0x7f86ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:55.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.385+0000 7f86fffff640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 0x7f86ec06ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f86f4036000 tx=0x7f86f403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:55.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.389+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8700075180 con 0x7f87000cc160 2024-04-13T03:13:55.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.611+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87000d1600 con 0x7f87000cc160 2024-04-13T03:13:55.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.612+0000 7f86fcff9640 1 -- 172.21.15.106:0/3254554997 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5190 (secure 0 0 0) 0x7f86f805aa20 con 0x7f87000cc160 2024-04-13T03:13:55.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:55.613 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:54.763270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:13:55.614 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 42 2024-04-13T03:13:55.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 msgr2=0x7f86ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 0x7f86ec06ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f86f4036000 tx=0x7f86f403b040 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 msgr2=0x7f87000cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f86f800cda0 tx=0x7f86f800a660 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 shutdown_connections 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f86ec06c920 0x7f86ec06ede0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87000cc160 0x7f87000cc610 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f870015e9b0 0x7f87000ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 --2- 172.21.15.106:0/3254554997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87000b3d60 0x7f870014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 >> 172.21.15.106:0/3254554997 conn(0x7f870008bf10 msgr2=0x7f8700161190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 shutdown_connections 2024-04-13T03:13:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:55.614+0000 7f8706595640 1 -- 172.21.15.106:0/3254554997 wait complete. 2024-04-13T03:13:56.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:13:37.103773+0000', 'modified': '2024-04-13T03:13:54.763270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6816': {'gid': 6816, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.106:6837/2254289128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2254289128}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2254289128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6816, 'qdb_cluster': [6816]} 2024-04-13T03:13:56.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:13:56.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-13T03:13:56.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.131+0000 7eff9d15e640 1 Processor -- start 2024-04-13T03:13:56.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.131+0000 7eff9d15e640 1 -- start start 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9d15e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff9815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9d15e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff980b3690 con 0x7eff980b4730 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9d15e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff980b3970 con 0x7eff980b5390 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9d15e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7eff980b3c50 con 0x7eff9815b4e0 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7eff980b4730 0x7eff980b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40764/0 (socket says 172.21.15.106:40764) 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff9659c640 1 -- 172.21.15.106:0/4198112069 learned_addr learned my addr 172.21.15.106:0/4198112069 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.132+0000 7eff96d9d640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff9815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 790928914 0 0) 0x7eff980b3690 con 0x7eff980b4730 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff80003520 con 0x7eff980b4730 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1282095595 0 0) 0x7eff80003520 con 0x7eff980b4730 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 msgr2=0x7eff9815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff9815fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 >> v1:172.21.15.114:6789/0 conn(0x7eff980b5390 legacy=0x7eff9806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff980b3f30 con 0x7eff980b4730 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7eff8c001670 con 0x7eff980b4730 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 >> v1:172.21.15.106:6789/0 conn(0x7eff980b4730 legacy=0x7eff980b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff80004390 0x7eff80004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff800050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff980b3c50 con 0x7eff800039e0 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7eff8000b9c0 con 0x7eff80004390 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7eff8000bbc0 con 0x7eff9815b4e0 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff9659c640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff80004390 0x7eff80004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff9759e640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff800050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 -- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 msgr2=0x7eff800050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9815b4e0 0x7eff800050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 -- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff80004390 msgr2=0x7eff80004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff80004390 0x7eff80004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.133+0000 7eff96d9d640 1 -- 172.21.15.106:0/4198112069 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff80003520 con 0x7eff800039e0 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.134+0000 7eff96d9d640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff80003dc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7eff88009ac0 tx=0x7eff80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2403018bf2e2acb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.134+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8803b070 con 0x7eff800039e0 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.134+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff88036440 con 0x7eff800039e0 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.134+0000 7eff95d9b640 1 -- 172.21.15.106:0/4198112069 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88009690 con 0x7eff800039e0 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 -- 172.21.15.106:0/4198112069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 msgr2=0x7eff80003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 --2- 172.21.15.106:0/4198112069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff80003dc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7eff88009ac0 tx=0x7eff80005830 comp rx=0 tx=0).stop 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9759e640 1 -- 172.21.15.106:0/4198112069 reap_dead start 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 -- 172.21.15.106:0/4198112069 shutdown_connections 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 -- 172.21.15.106:0/4198112069 >> 172.21.15.106:0/4198112069 conn(0x7eff980b2600 msgr2=0x7eff980d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 -- 172.21.15.106:0/4198112069 shutdown_connections 2024-04-13T03:13:56.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.135+0000 7eff9d15e640 1 -- 172.21.15.106:0/4198112069 wait complete. 2024-04-13T03:13:56.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.136+0000 7eff9d15e640 1 Processor -- start 2024-04-13T03:13:56.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.136+0000 7eff9d15e640 1 -- start start 2024-04-13T03:13:56.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9813c880 0x7eff98153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff9815f940 0x7eff9815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff9806d0c0 con 0x7eff800039e0 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7eff980b32d0 con 0x7eff9815f940 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9d15e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7eff98154680 con 0x7eff9813c880 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51486/0 (socket says 172.21.15.106:51486) 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 -- 172.21.15.106:0/3044954460 learned_addr learned my addr 172.21.15.106:0/3044954460 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9659c640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9813c880 0x7eff98153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff9759e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff9815f940 0x7eff9815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 -- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9813c880 msgr2=0x7eff98153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9813c880 0x7eff98153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 -- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff9815f940 msgr2=0x7eff9815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff9815f940 0x7eff9815cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.137+0000 7eff96d9d640 1 -- 172.21.15.106:0/3044954460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff98154960 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff7f7fe640 1 -- 172.21.15.106:0/3044954460 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88044020 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff7f7fe640 1 -- 172.21.15.106:0/3044954460 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8803c040 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff98154c40 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff7f7fe640 1 -- 172.21.15.106:0/3044954460 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8803b070 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.138+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9815fdc0 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.140+0000 7eff7f7fe640 1 -- 172.21.15.106:0/3044954460 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff88047020 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.140+0000 7eff7f7fe640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 0x7eff8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.140+0000 7eff7f7fe640 1 -- 172.21.15.106:0/3044954460 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7eff880afcd0 con 0x7eff800039e0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.140+0000 7eff9659c640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 0x7eff8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.141+0000 7eff9659c640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 0x7eff8406ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7eff8c000fb0 tx=0x7eff8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.141+0000 7effa002b740 1 Processor -- start 2024-04-13T03:13:56.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.141+0000 7effa002b740 1 -- start start 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a34d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55898699f990 con 0x558986a352c0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558986946620 con 0x558986a61de0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55898699b950 con 0x558986a621c0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51496/0 (socket says 172.21.15.106:51496) 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 -- 172.21.15.106:0/1616343679 learned_addr learned my addr 172.21.15.106:0/1616343679 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9659c640 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff96d9d640 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a34d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 -- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 msgr2=0x558986a64b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a64b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 -- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 msgr2=0x558986a34d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a34d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 -- 172.21.15.106:0/1616343679 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589866b22d0 con 0x558986a352c0 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff9759e640 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7eff980b4b70 tx=0x7eff900017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70bae4983935a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff7cff9640 1 -- 172.21.15.106:0/1616343679 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900055b0 con 0x558986a352c0 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff7cff9640 1 -- 172.21.15.106:0/1616343679 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90005750 con 0x558986a352c0 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.142+0000 7eff7cff9640 1 -- 172.21.15.106:0/1616343679 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90015470 con 0x558986a352c0 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 -- 172.21.15.106:0/1616343679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 msgr2=0x558986a650b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7eff980b4b70 tx=0x7eff900017c0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 -- 172.21.15.106:0/1616343679 shutdown_connections 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a352c0 0x558986a650b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 --2- 172.21.15.106:0/1616343679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a34d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 -- 172.21.15.106:0/1616343679 >> 172.21.15.106:0/1616343679 conn(0x558986a36ff0 msgr2=0x558986a7fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 -- 172.21.15.106:0/1616343679 shutdown_connections 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.143+0000 7effa002b740 1 -- 172.21.15.106:0/1616343679 wait complete. 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.144+0000 7effa002b740 1 -- start start 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.144+0000 7effa002b740 1 Processor -- start 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558986a39590 con 0x558986a79cf0 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5589869599e0 con 0x558986a61de0 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558986a38980 con 0x558986a621c0 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 10 client.0 _ll_register_callbacks cb 0x558986995220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51504/0 (socket says 172.21.15.106:51504) 2024-04-13T03:13:56.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 -- 172.21.15.106:0/1798495487 learned_addr learned my addr 172.21.15.106:0/1798495487 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9659c640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff96d9d640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 -- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 msgr2=0x558986a797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 -- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 msgr2=0x558986a4faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a4faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 -- 172.21.15.106:0/1798495487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898677c6b0 con 0x558986a79cf0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff96d9d640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a4faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff9759e640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7eff90017040 tx=0x7eff900016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f12914f1bbb2803 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900055b0 con 0x558986a79cf0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7eff9001a030 con 0x558986a79cf0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff90005750 con 0x558986a79cf0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.145+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7eff9002e050 con 0x558986a79cf0 2024-04-13T03:13:56.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 10 client.7101 fetch_fsmap learned FSMap version 42 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x558986a50dc0 con 0x558986a79cf0 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90005750 con 0x558986a79cf0 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 42) ==== 3494+0+0 (secure 0 0 0) 0x7eff90031020 con 0x558986a79cf0 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 10 client.7101 fetch_fsmap finished waiting for FSMap version 42 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 10 client.7101 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 20 client.7101 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a51790 0x558986a67330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 4 client.7101 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589869ad3b0 con 0x558986a51790 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7eff96d9d640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a51790 0x558986a67330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7eff75ffb640 10 client.7101 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.146+0000 7eff96d9d640 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a51790 0x558986a67330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55d4ba9cab2f66e server_cookie=7225b495ca50b2b1 in_seq=0 out_seq=0 2024-04-13T03:13:56.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7eff75ffb640 1 -- 172.21.15.106:0/1798495487 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589869ad3b0 con 0x558986a51790 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7eff75ffb640 10 client.7101 handle_command_reply: tid=0 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 client.7101 shutdown 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 msgr2=0x558986a52240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7eff90017040 tx=0x7eff900016a0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 shutdown_connections 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a51790 0x558986a67330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a52240 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7eff75ffb640 0 client.7101 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 --2- 172.21.15.106:0/1798495487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986a4faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 >> 172.21.15.106:0/1798495487 conn(0x558986a352c0 msgr2=0x558986a64b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 shutdown_connections 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 1 -- 172.21.15.106:0/1798495487 wait complete. 2024-04-13T03:13:56.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.147+0000 7effa002b740 20 client.7101 trim_cache size 0 max 16384 2024-04-13T03:13:56.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.175+0000 7effa002b740 1 Processor -- start 2024-04-13T03:13:56.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.175+0000 7effa002b740 1 -- start start 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a79cf0 0x558986a387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff981fc870 con 0x558986a61de0 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7eff9813fc70 con 0x558986a79cf0 2024-04-13T03:13:56.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7eff981fc1a0 con 0x558986a621c0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51514/0 (socket says 172.21.15.106:51514) 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 -- 172.21.15.106:0/3698649264 learned_addr learned my addr 172.21.15.106:0/3698649264 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 -- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 msgr2=0x558986a65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 -- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a79cf0 msgr2=0x558986a387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a79cf0 0x558986a387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 -- 172.21.15.106:0/3698649264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558986a76cc0 con 0x558986a61de0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff96d9d640 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7eff8802ffd0 tx=0x7eff88042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24e90c93ecfbad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff77fff640 1 -- 172.21.15.106:0/3698649264 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8807d820 con 0x558986a61de0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff77fff640 1 -- 172.21.15.106:0/3698649264 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff88081bd0 con 0x558986a61de0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.176+0000 7eff77fff640 1 -- 172.21.15.106:0/3698649264 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88086400 con 0x558986a61de0 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- 172.21.15.106:0/3698649264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 msgr2=0x558986a92810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7eff8802ffd0 tx=0x7eff88042040 comp rx=0 tx=0).stop 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- 172.21.15.106:0/3698649264 shutdown_connections 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a79cf0 0x558986a387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 --2- 172.21.15.106:0/3698649264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a61de0 0x558986a92810 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- 172.21.15.106:0/3698649264 >> 172.21.15.106:0/3698649264 conn(0x558986a352c0 msgr2=0x558986a83080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- 172.21.15.106:0/3698649264 shutdown_connections 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- 172.21.15.106:0/3698649264 wait complete. 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 -- start start 2024-04-13T03:13:56.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.177+0000 7effa002b740 1 Processor -- start 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986b01000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558986a51610 con 0x558986a79cf0 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558986a510a0 con 0x558986a61de0 2024-04-13T03:13:56.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558986a51c60 con 0x558986a621c0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7effa002b740 10 client.0 _ll_register_callbacks cb 0x7eff90003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7eff9759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51518/0 (socket says 172.21.15.106:51518) 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7eff9759e640 1 -- 172.21.15.106:0/2739534275 learned_addr learned my addr 172.21.15.106:0/2739534275 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7eff9659c640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.178+0000 7eff96d9d640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986b01000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 -- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 msgr2=0x558986a387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 -- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 msgr2=0x558986b01000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986b01000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 -- 172.21.15.106:0/2739534275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558986a60990 con 0x558986a79cf0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff96d9d640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986b01000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff9759e640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7eff90017040 tx=0x7eff90001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807bf129903e41d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900056e0 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7eff9001a030 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff88081bd0 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7eff9003e050 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 10 client.7107 fetch_fsmap learned FSMap version 42 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x558986a99380 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff88081bd0 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 42) ==== 3494+0+0 (secure 0 0 0) 0x7eff90041020 con 0x558986a79cf0 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 10 client.7107 fetch_fsmap finished waiting for FSMap version 42 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 10 client.7107 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 20 client.7107 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a5f970 0x558986a39a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 4 client.7107 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:13:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "status"}) -- 0x5589869a7220 con 0x558986a5f970 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.179+0000 7eff96d9d640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a5f970 0x558986a39a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7eff75ffb640 10 client.7107 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7eff96d9d640 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a5f970 0x558986a39a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878419e34570fb2a server_cookie=76901ec085672701 in_seq=0 out_seq=0 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7eff75ffb640 1 -- 172.21.15.106:0/2739534275 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5589869a7220 con 0x558986a5f970 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7eff75ffb640 10 client.7107 handle_command_reply: tid=0 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 client.7107 shutdown 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 msgr2=0x558986a38cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7eff90017040 tx=0x7eff90001630 comp rx=0 tx=0).stop 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 shutdown_connections 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x558986a5f970 0x558986a39a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558986a621c0 0x558986a387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558986a79cf0 0x558986a38cf0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7eff75ffb640 0 client.7107 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 --2- 172.21.15.106:0/2739534275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558986a61de0 0x558986b01000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 >> 172.21.15.106:0/2739534275 conn(0x558986a352c0 msgr2=0x558986a83080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 shutdown_connections 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 1 -- 172.21.15.106:0/2739534275 wait complete. 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.180+0000 7effa002b740 20 client.7107 trim_cache size 0 max 16384 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 6816, 2024-04-13T03:13:56.188 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.4205325630000001, 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 42, 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 79, 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 79, 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 23.488291587999999 2024-04-13T03:13:56.189 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 msgr2=0x7eff8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 0x7eff8406ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7eff8c000fb0 tx=0x7eff8c009040 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 msgr2=0x7eff98155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88004730 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 shutdown_connections 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7eff8406c920 0x7eff8406ede0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7eff9813c880 0x7eff98153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff9815f940 0x7eff9815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 --2- 172.21.15.106:0/3044954460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff800039e0 0x7eff98155720 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 >> 172.21.15.106:0/3044954460 conn(0x7eff980b2600 msgr2=0x7eff980b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 shutdown_connections 2024-04-13T03:13:56.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.181+0000 7eff9d15e640 1 -- 172.21.15.106:0/3044954460 wait complete. 2024-04-13T03:13:56.192 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:13:56.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.292+0000 7f98c85f7640 1 Processor -- start 2024-04-13T03:13:56.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.292+0000 7f98c85f7640 1 -- start start 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c85f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98c00cd2b0 0x7f98c0160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c85f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00c60a0 0x7f98c00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c85f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c00d2310 con 0x7f98c015c7d0 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c85f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f98c00b1f30 con 0x7f98c00cd2b0 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c85f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00b2210 con 0x7f98c00c60a0 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c5b92640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98c00cd2b0 0x7f98c0160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c6393640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f98c015c7d0 0x7f98c00ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40780/0 (socket says 172.21.15.106:40780) 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.293+0000 7f98c6393640 1 -- 172.21.15.106:0/3846534331 learned_addr learned my addr 172.21.15.106:0/3846534331 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c6b94640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00c60a0 0x7f98c00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c5b92640 1 -- 172.21.15.106:0/3846534331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00c60a0 msgr2=0x7f98c00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c5b92640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00c60a0 0x7f98c00d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c5b92640 1 -- 172.21.15.106:0/3846534331 >> v1:172.21.15.106:6789/0 conn(0x7f98c015c7d0 legacy=0x7f98c00ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c5b92640 1 -- 172.21.15.106:0/3846534331 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c00b24f0 con 0x7f98c00cd2b0 2024-04-13T03:13:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.294+0000 7f98c5391640 1 -- 172.21.15.106:0/3846534331 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413978526 0 0) 0x7f98c00d2310 con 0x7f98c015c7d0 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c5b92640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00cd2b0 0x7f98c0160dd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f98b00097a0 tx=0x7f98b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d217f419008a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c5391640 1 -- 172.21.15.106:0/3846534331 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0035020 con 0x7f98c00cd2b0 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c5391640 1 -- 172.21.15.106:0/3846534331 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b0031830 con 0x7f98c00cd2b0 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c5391640 1 -- 172.21.15.106:0/3846534331 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0031c00 con 0x7f98c00cd2b0 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 -- 172.21.15.106:0/3846534331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00cd2b0 msgr2=0x7f98c0160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00cd2b0 0x7f98c0160dd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f98b00097a0 tx=0x7f98b002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 -- 172.21.15.106:0/3846534331 shutdown_connections 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00c60a0 0x7f98c00d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 --2- 172.21.15.106:0/3846534331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00cd2b0 0x7f98c0160dd0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 -- 172.21.15.106:0/3846534331 >> 172.21.15.106:0/3846534331 conn(0x7f98c00d2550 msgr2=0x7f98c00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 -- 172.21.15.106:0/3846534331 shutdown_connections 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.295+0000 7f98c85f7640 1 -- 172.21.15.106:0/3846534331 wait complete. 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 Processor -- start 2024-04-13T03:13:56.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 -- start start 2024-04-13T03:13:56.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00cd2b0 0x7f98c015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c015c7d0 0x7f98c01486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98c015cbb0 con 0x7f98c00c60a0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00b2cf0 con 0x7f98c015c7d0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c85f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98c015ecd0 con 0x7f98c00cd2b0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c6393640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c6393640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51546/0 (socket says 172.21.15.106:51546) 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c6393640 1 -- 172.21.15.106:0/901570406 learned_addr learned my addr 172.21.15.106:0/901570406 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.297+0000 7f98c5b92640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00cd2b0 0x7f98c015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6b94640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c015c7d0 0x7f98c01486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 -- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00cd2b0 msgr2=0x7f98c015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00cd2b0 0x7f98c015fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 -- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c015c7d0 msgr2=0x7f98c01486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c015c7d0 0x7f98c01486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 -- 172.21.15.106:0/901570406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c015efb0 con 0x7f98c00c60a0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c6393640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c430 tx=0x7f98bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0074e0 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98bc0047e0 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c015f290 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc004bd0 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.298+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c015d0f0 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.299+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c0069120 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.300+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98bc011070 con 0x7f98c00c60a0 2024-04-13T03:13:56.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.300+0000 7f98b6ffd640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 0x7f98a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.300+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f98bc08b870 con 0x7f98c00c60a0 2024-04-13T03:13:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.300+0000 7f98c5b92640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 0x7f98a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.301+0000 7f98c5b92640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 0x7f98a406edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98b0036000 tx=0x7f98b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:56.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.305+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98bc059470 con 0x7f98c00c60a0 2024-04-13T03:13:56.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.537+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98c00d31b0 con 0x7f98c00c60a0 2024-04-13T03:13:56.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.537+0000 7f98b6ffd640 1 -- 172.21.15.106:0/901570406 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) ==== 53+0+73 (secure 0 0 0) 0x7f98bc05d770 con 0x7f98c00c60a0 2024-04-13T03:13:56.538 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 msgr2=0x7f98a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 0x7f98a406edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98b0036000 tx=0x7f98b0008030 comp rx=0 tx=0).stop 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 msgr2=0x7f98c00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c430 tx=0x7f98bc00c8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 shutdown_connections 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a406c8f0 0x7f98a406edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98c00cd2b0 0x7f98c015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c015c7d0 0x7f98c01486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 --2- 172.21.15.106:0/901570406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98c00c60a0 0x7f98c00cbc20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:56.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 >> 172.21.15.106:0/901570406 conn(0x7f98c00d2550 msgr2=0x7f98c00b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:56.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 shutdown_connections 2024-04-13T03:13:56.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56.539+0000 7f98c85f7640 1 -- 172.21.15.106:0/901570406 wait complete. 2024-04-13T03:13:56.550 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:13:56.550 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:13:56.550 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:13:56.550 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:13:56.551 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:56.551 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:13:56.551 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:13:56.551 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:13:56.551 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:56.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:56.551 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:56.567 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42313sec preferred_lft 42313sec 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:13:56.568 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:13:56.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:56.569 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:13:56.632 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:13:56.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:56.634 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:13:56.690 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:13:56.690 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:13:56.690 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:13:56.690 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:13:56.690 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:13:56.691 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:13:56.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:13:56.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:13:56.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:56.826 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:13:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:13:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:13:56.841 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:56.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:56.841 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:13:56.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:13:56.909 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:56.963 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:13:56.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:13:56.964 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:57.018 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:13:57.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:57.018 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:57.102 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:57.102 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:13:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:13:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:13:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:13:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:13:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:13:57.110 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:13:57.110 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:13:57.110 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:13:57.110 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:13:57.110 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:13:57.112 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:13:57.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:57.112 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:57.180 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:13:57.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:13:57.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:13:57.276 INFO:teuthology.orchestra.run.smithi106.stdout:client5834 2024-04-13T03:13:57.276 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:57.276 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:57.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:57.292 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-13T03:13:57.349 INFO:teuthology.orchestra.run.smithi106.stdout:file_undamaged 2024-04-13T03:13:57.349 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:13:57.349 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:13:57.349 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:13:57.349 DEBUG:teuthology.orchestra.run.smithi106:> import json 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> import sys 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> try: 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> except OSError as e: 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> sys.exit(e.errno) 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> print(json.dumps( 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> indent=2)) 2024-04-13T03:13:57.350 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:13:57.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-13T03:13:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-13T03:13:57.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:57.464 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-13T03:13:57.519 INFO:teuthology.orchestra.run.smithi106.stdout:2 2024-04-13T03:13:57.519 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:57.519 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:57.574 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:13:57.575 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:13:57.630 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:13:57.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.769+0000 7f0d4dbb4640 1 Processor -- start 2024-04-13T03:13:57.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.769+0000 7f0d4dbb4640 1 -- start start 2024-04-13T03:13:57.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.769+0000 7f0d4dbb4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d48160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.769+0000 7f0d4dbb4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d480d2310 con 0x7f0d4815c7d0 2024-04-13T03:13:57.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d4dbb4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d480b1f30 con 0x7f0d480cd2b0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d4dbb4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0d480b2210 con 0x7f0d480c60a0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d477fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0d4815c7d0 0x7f0d480ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40782/0 (socket says 172.21.15.106:40782) 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d477fe640 1 -- 172.21.15.106:0/3119712191 learned_addr learned my addr 172.21.15.106:0/3119712191 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640254387 0 0) 0x7f0d480d2310 con 0x7f0d4815c7d0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d47fff640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d48160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d30003520 con 0x7f0d4815c7d0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 85257570 0 0) 0x7f0d30003520 con 0x7f0d4815c7d0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 msgr2=0x7f0d48160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d48160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 >> v1:172.21.15.114:6789/0 conn(0x7f0d480cd2b0 legacy=0x7f0d480b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d480b24f0 con 0x7f0d4815c7d0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f0d38001670 con 0x7f0d4815c7d0 2024-04-13T03:13:57.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 >> v1:172.21.15.106:6789/0 conn(0x7f0d4815c7d0 legacy=0x7f0d480ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d30004d40 0x7f0d300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d480b2210 con 0x7f0d300039e0 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3000ba70 con 0x7f0d30004d40 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3000bc70 con 0x7f0d480c60a0 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d477fe640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d46ffd640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.770+0000 7f0d47fff640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d30004d40 0x7f0d300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 -- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 msgr2=0x7f0d30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 -- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d30004d40 msgr2=0x7f0d300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d30004d40 0x7f0d300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 -- 172.21.15.106:0/3119712191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d30003520 con 0x7f0d300039e0 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d46ffd640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d480c60a0 0x7f0d30004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d477fe640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d30003e00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001eb0 tx=0x7f0d30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a99546526ab374 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007d30 con 0x7f0d300039e0 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38030070 con 0x7f0d300039e0 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.771+0000 7f0d467fc640 1 -- 172.21.15.106:0/3119712191 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38005d30 con 0x7f0d300039e0 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/3119712191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 msgr2=0x7f0d30003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/3119712191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d30003e00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001eb0 tx=0x7f0d30005830 comp rx=0 tx=0).stop 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d47fff640 1 -- 172.21.15.106:0/3119712191 reap_dead start 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/3119712191 shutdown_connections 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/3119712191 >> 172.21.15.106:0/3119712191 conn(0x7f0d480d2550 msgr2=0x7f0d480d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/3119712191 shutdown_connections 2024-04-13T03:13:57.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.772+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/3119712191 wait complete. 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.773+0000 7f0d4dbb4640 1 Processor -- start 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.773+0000 7f0d4dbb4640 1 -- start start 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d48155b60 0x7f0d4806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 0x7f0d4815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4815cbb0 con 0x7f0d300039e0 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48160dd0 con 0x7f0d4815e3f0 2024-04-13T03:13:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d4dbb4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0d480b6380 con 0x7f0d48155b60 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51568/0 (socket says 172.21.15.106:51568) 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 -- 172.21.15.106:0/1460879058 learned_addr learned my addr 172.21.15.106:0/1460879058 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d47fff640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 0x7f0d4815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d46ffd640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d48155b60 0x7f0d4806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 -- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d48155b60 msgr2=0x7f0d4806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d48155b60 0x7f0d4806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 -- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 msgr2=0x7f0d4815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 0x7f0d4815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 -- 172.21.15.106:0/1460879058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481551b0 con 0x7f0d300039e0 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d47fff640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 0x7f0d4815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.774+0000 7f0d477fe640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c300 tx=0x7f0d38001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:57.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.775+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38004bf0 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.775+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38004d90 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.775+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d48155490 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.775+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007270 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.775+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4815e100 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.776+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d48069120 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.776+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d38047020 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.777+0000 7f0d2ffff640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 0x7f0d3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.777+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f0d380ace30 con 0x7f0d300039e0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.777+0000 7f0d46ffd640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 0x7f0d3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.777+0000 7f0d46ffd640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 0x7f0d3406ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d481f7f40 tx=0x7f0d3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:57.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.781+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d3807b110 con 0x7f0d300039e0 2024-04-13T03:13:57.928 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:13:57.927+0000 7f707b3f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-13T03:13:57.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.959+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d480cd860 con 0x7f0d300039e0 2024-04-13T03:13:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.960+0000 7f0d2ffff640 1 -- 172.21.15.106:0/1460879058 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17110 (secure 0 0 0) 0x7f0d3807aa10 con 0x7f0d300039e0 2024-04-13T03:13:57.961 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:13:57.961 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:50.750679+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:13:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 msgr2=0x7f0d3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 0x7f0d3406ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d481f7f40 tx=0x7f0d3c009040 comp rx=0 tx=0).stop 2024-04-13T03:13:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 msgr2=0x7f0d48154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c300 tx=0x7f0d38001a40 comp rx=0 tx=0).stop 2024-04-13T03:13:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 shutdown_connections 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0d3406c920 0x7f0d3406ede0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0d48155b60 0x7f0d4806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d4815e3f0 0x7f0d4815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 --2- 172.21.15.106:0/1460879058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d300039e0 0x7f0d48154070 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 >> 172.21.15.106:0/1460879058 conn(0x7f0d480d2550 msgr2=0x7f0d48070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 shutdown_connections 2024-04-13T03:13:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:57.962+0000 7f0d4dbb4640 1 -- 172.21.15.106:0/1460879058 wait complete. 2024-04-13T03:13:57.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:13:35.329204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:13:36.059991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:13:57.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:13:58.026 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:13:58.025+0000 7f7606df1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:13:58.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.077+0000 7f3a4bec8640 1 Processor -- start 2024-04-13T03:13:58.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.077+0000 7f3a4bec8640 1 -- start start 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a4bec8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a440d09f0 0x7f3a440b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a4bec8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a4406e280 con 0x7f3a4415e930 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a4bec8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a4406e560 con 0x7f3a4406ed70 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a4bec8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3a440cccf0 con 0x7f3a440d09f0 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49463640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3a440d09f0 0x7f3a440b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49c64640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3a4415e930 0x7f3a440d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40792/0 (socket says 172.21.15.106:40792) 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49c64640 1 -- 172.21.15.106:0/637358649 learned_addr learned my addr 172.21.15.106:0/637358649 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49463640 1 -- 172.21.15.106:0/637358649 >> v1:172.21.15.114:6789/0 conn(0x7f3a4406ed70 legacy=0x7f3a440cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:58.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49463640 1 -- 172.21.15.106:0/637358649 >> v1:172.21.15.106:6789/0 conn(0x7f3a4415e930 legacy=0x7f3a440d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a48c62640 1 -- 172.21.15.106:0/637358649 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100117262 0 0) 0x7f3a4406e280 con 0x7f3a4415e930 2024-04-13T03:13:58.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.078+0000 7f3a49463640 1 -- 172.21.15.106:0/637358649 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a440ccfd0 con 0x7f3a440d09f0 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a49463640 1 --2- 172.21.15.106:0/637358649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a440d09f0 0x7f3a440b5350 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3a340097a0 tx=0x7f3a3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b461afa72122017d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a48c62640 1 -- 172.21.15.106:0/637358649 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a340315a0 con 0x7f3a440d09f0 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a48c62640 1 -- 172.21.15.106:0/637358649 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a34037070 con 0x7f3a440d09f0 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a48c62640 1 -- 172.21.15.106:0/637358649 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34031960 con 0x7f3a440d09f0 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a4bec8640 1 -- 172.21.15.106:0/637358649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a440d09f0 msgr2=0x7f3a440b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/637358649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a440d09f0 0x7f3a440b5350 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3a340097a0 tx=0x7f3a3402b840 comp rx=0 tx=0).stop 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a4bec8640 1 -- 172.21.15.106:0/637358649 shutdown_connections 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/637358649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a440d09f0 0x7f3a440b5350 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.079+0000 7f3a4bec8640 1 -- 172.21.15.106:0/637358649 >> 172.21.15.106:0/637358649 conn(0x7f3a440cef80 msgr2=0x7f3a440cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.080+0000 7f3a4bec8640 1 -- 172.21.15.106:0/637358649 shutdown_connections 2024-04-13T03:13:58.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.080+0000 7f3a4bec8640 1 -- 172.21.15.106:0/637358649 wait complete. 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 Processor -- start 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 -- start start 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 0x7f3a4415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 0x7f3a440cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440ce830 con 0x7f3a4415e930 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406f1a0 con 0x7f3a440d09f0 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4bec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a440ce9b0 con 0x7f3a4406ed70 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.081+0000 7f3a4a465640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51590/0 (socket says 172.21.15.106:51590) 2024-04-13T03:13:58.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 -- 172.21.15.106:0/1616972685 learned_addr learned my addr 172.21.15.106:0/1616972685 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a49463640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 0x7f3a440cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a49c64640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 0x7f3a4415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 -- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 msgr2=0x7f3a4415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 0x7f3a4415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 -- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 msgr2=0x7f3a440cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 0x7f3a440cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 -- 172.21.15.106:0/1616972685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a440cb3d0 con 0x7f3a4415e930 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a49463640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 0x7f3a440cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a49c64640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 0x7f3a4415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4a465640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c7b0 tx=0x7f3a3c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004b60 con 0x7f3a4415e930 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3c004d00 con 0x7f3a4415e930 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4415f2f0 con 0x7f3a4415e930 2024-04-13T03:13:58.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00aa90 con 0x7f3a4415e930 2024-04-13T03:13:58.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.082+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4415f8b0 con 0x7f3a4415e930 2024-04-13T03:13:58.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.083+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a44068a10 con 0x7f3a4415e930 2024-04-13T03:13:58.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.084+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a3c00ac30 con 0x7f3a4415e930 2024-04-13T03:13:58.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.084+0000 7f3a3a7fc640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 0x7f3a2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.084+0000 7f3a49c64640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 0x7f3a2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.084+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3a3c08bf20 con 0x7f3a4415e930 2024-04-13T03:13:58.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.085+0000 7f3a49c64640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 0x7f3a2806edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f60 tx=0x7f3a40007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.089+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a3c020070 con 0x7f3a4415e930 2024-04-13T03:13:58.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.264+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a44161c40 con 0x7f3a4415e930 2024-04-13T03:13:58.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.264+0000 7f3a3a7fc640 1 -- 172.21.15.106:0/1616972685 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f3a3c05a200 con 0x7f3a4415e930 2024-04-13T03:13:58.265 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 msgr2=0x7f3a2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 0x7f3a2806edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f60 tx=0x7f3a40007ca0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 msgr2=0x7f3a441f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c7b0 tx=0x7f3a3c00cc70 comp rx=0 tx=0).stop 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 shutdown_connections 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a2806c8f0 0x7f3a2806edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a4415e930 0x7f3a441f4d90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a440d09f0 0x7f3a440cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 --2- 172.21.15.106:0/1616972685 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a4406ed70 0x7f3a4415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 >> 172.21.15.106:0/1616972685 conn(0x7f3a440cef80 msgr2=0x7f3a4406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 shutdown_connections 2024-04-13T03:13:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.266+0000 7f3a4bec8640 1 -- 172.21.15.106:0/1616972685 wait complete. 2024-04-13T03:13:58.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:13:58.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.379+0000 7feff6c20640 1 Processor -- start 2024-04-13T03:13:58.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.379+0000 7feff6c20640 1 -- start start 2024-04-13T03:13:58.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff6c20640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feff00b47e0 0x7feff015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff6c20640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff006d220 con 0x7feff00b4c20 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff6c20640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff00b3740 con 0x7feff0145d20 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff6c20640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feff00b3a20 con 0x7feff00b47e0 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7fefeffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feff00b47e0 0x7feff015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff51bd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feff00b4c20 0x7feff00cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40804/0 (socket says 172.21.15.106:40804) 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.380+0000 7feff51bd640 1 -- 172.21.15.106:0/715466609 learned_addr learned my addr 172.21.15.106:0/715466609 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefeffff640 1 -- 172.21.15.106:0/715466609 >> v1:172.21.15.114:6789/0 conn(0x7feff0145d20 legacy=0x7feff006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefeffff640 1 -- 172.21.15.106:0/715466609 >> v1:172.21.15.106:6789/0 conn(0x7feff00b4c20 legacy=0x7feff00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefeffff640 1 -- 172.21.15.106:0/715466609 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00b3d00 con 0x7feff00b47e0 2024-04-13T03:13:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefeffff640 1 --2- 172.21.15.106:0/715466609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff015cbc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e21b3cdf4c0c41b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefef7fe640 1 -- 172.21.15.106:0/715466609 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40315a0 con 0x7feff00b47e0 2024-04-13T03:13:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefef7fe640 1 -- 172.21.15.106:0/715466609 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe4037070 con 0x7feff00b47e0 2024-04-13T03:13:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.381+0000 7fefef7fe640 1 -- 172.21.15.106:0/715466609 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4031960 con 0x7feff00b47e0 2024-04-13T03:13:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 -- 172.21.15.106:0/715466609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 msgr2=0x7feff015cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 --2- 172.21.15.106:0/715466609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff015cbc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b840 comp rx=0 tx=0).stop 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 -- 172.21.15.106:0/715466609 shutdown_connections 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 --2- 172.21.15.106:0/715466609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff015cbc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 -- 172.21.15.106:0/715466609 >> 172.21.15.106:0/715466609 conn(0x7feff00b2600 msgr2=0x7feff00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 -- 172.21.15.106:0/715466609 shutdown_connections 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.382+0000 7feff6c20640 1 -- 172.21.15.106:0/715466609 wait complete. 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.383+0000 7feff6c20640 1 Processor -- start 2024-04-13T03:13:58.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.383+0000 7feff6c20640 1 -- start start 2024-04-13T03:13:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feff00b4c20 0x7feff015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 0x7feff015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feff0146100 con 0x7feff00b47e0 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feff00b42c0 con 0x7feff00b4c20 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff6c20640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feff0146280 con 0x7feff0145d20 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51622/0 (socket says 172.21.15.106:51622) 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 -- 172.21.15.106:0/3421643021 learned_addr learned my addr 172.21.15.106:0/3421643021 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7fefeffff640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feff00b4c20 0x7feff015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff51bd640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 0x7feff015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 -- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 msgr2=0x7feff015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 0x7feff015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 -- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feff00b4c20 msgr2=0x7feff015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feff00b4c20 0x7feff015ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff49bc640 1 -- 172.21.15.106:0/3421643021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff013c270 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.384+0000 7feff51bd640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 0x7feff015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7feff49bc640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c430 tx=0x7fefe000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011790 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe0017070 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff0141d30 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011bb0 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.385+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01422f0 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.386+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0068a10 con 0x7feff00b47e0 2024-04-13T03:13:58.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.386+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefe001c460 con 0x7feff00b47e0 2024-04-13T03:13:58.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.386+0000 7fefed7fa640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 0x7fefdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.387+0000 7fefeffff640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 0x7fefdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.387+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fefe008bb80 con 0x7feff00b47e0 2024-04-13T03:13:58.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.387+0000 7fefeffff640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 0x7fefdc06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fefe4036000 tx=0x7fefe403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.391+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefe0059d40 con 0x7feff00b47e0 2024-04-13T03:13:58.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.567+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feff00b0f50 con 0x7feff00b47e0 2024-04-13T03:13:58.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.568+0000 7fefed7fa640 1 -- 172.21.15.106:0/3421643021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fefe0059640 con 0x7feff00b47e0 2024-04-13T03:13:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 msgr2=0x7fefdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 0x7fefdc06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fefe4036000 tx=0x7fefe403d040 comp rx=0 tx=0).stop 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 msgr2=0x7feff013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c430 tx=0x7fefe000c8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 shutdown_connections 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fefdc06c920 0x7fefdc06ede0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feff0145d20 0x7feff015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feff00b4c20 0x7feff015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 --2- 172.21.15.106:0/3421643021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feff00b47e0 0x7feff013bd30 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 >> 172.21.15.106:0/3421643021 conn(0x7feff00b2600 msgr2=0x7feff006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 shutdown_connections 2024-04-13T03:13:58.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.570+0000 7feff6c20640 1 -- 172.21.15.106:0/3421643021 wait complete. 2024-04-13T03:13:58.581 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:13:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 Processor -- start 2024-04-13T03:13:58.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 -- start start 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0c00b3d60 0x7ff0c00d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00cfd20 con 0x7ff0c015ed90 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c0162af0 con 0x7ff0c00b3d60 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.684+0000 7ff0c54d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c0162dd0 con 0x7ff0c015e9b0 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0be7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff0c015ed90 0x7ff0c00d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40812/0 (socket says 172.21.15.106:40812) 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0be7fc640 1 -- 172.21.15.106:0/4235910855 learned_addr learned my addr 172.21.15.106:0/4235910855 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/4235910855 >> v2:172.21.15.106:3300/0 conn(0x7ff0c00b3d60 0x7ff0c00d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0beffd640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/4235910855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 msgr2=0x7ff0c00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/4235910855 >> v1:172.21.15.106:6789/0 conn(0x7ff0c015ed90 legacy=0x7ff0c00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bdffb640 1 -- 172.21.15.106:0/4235910855 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3921073604 0 0) 0x7ff0c00cfd20 con 0x7ff0c015ed90 2024-04-13T03:13:58.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.685+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/4235910855 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c01630b0 con 0x7ff0c00b3d60 2024-04-13T03:13:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00b3d60 0x7ff0c00d0200 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d90 tx=0x7ff0b002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda8f85309edf67f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0bdffb640 1 -- 172.21.15.106:0/4235910855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0039070 con 0x7ff0c00b3d60 2024-04-13T03:13:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0bdffb640 1 -- 172.21.15.106:0/4235910855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b002cce0 con 0x7ff0c00b3d60 2024-04-13T03:13:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0bdffb640 1 -- 172.21.15.106:0/4235910855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0034660 con 0x7ff0c00b3d60 2024-04-13T03:13:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 -- 172.21.15.106:0/4235910855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00b3d60 msgr2=0x7ff0c00d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00b3d60 0x7ff0c00d0200 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d90 tx=0x7ff0b002bf30 comp rx=0 tx=0).stop 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 -- 172.21.15.106:0/4235910855 shutdown_connections 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00b3d60 0x7ff0c00d0200 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/4235910855 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 -- 172.21.15.106:0/4235910855 >> 172.21.15.106:0/4235910855 conn(0x7ff0c008bf10 msgr2=0x7ff0c00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 -- 172.21.15.106:0/4235910855 shutdown_connections 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.686+0000 7ff0c54d5640 1 -- 172.21.15.106:0/4235910855 wait complete. 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 Processor -- start 2024-04-13T03:13:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 -- start start 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c00b3d60 0x7ff0c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0162800 con 0x7ff0c00c8220 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0162680 con 0x7ff0c00b3d60 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0c54d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c00d4d00 con 0x7ff0c015e9b0 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51628/0 (socket says 172.21.15.106:51628) 2024-04-13T03:13:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.688+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/102292900 learned_addr learned my addr 172.21.15.106:0/102292900 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0beffd640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c00b3d60 0x7ff0c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0be7fc640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 msgr2=0x7ff0c00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c00b3d60 msgr2=0x7ff0c006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c00b3d60 0x7ff0c006d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 -- 172.21.15.106:0/102292900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c00c86b0 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0bf7fe640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d90 tx=0x7ff0b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0039050 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b0045070 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c00c8990 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b003e830 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.689+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c00c8f50 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.690+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0068a10 con 0x7ff0c00c8220 2024-04-13T03:13:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.690+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0b003e9d0 con 0x7ff0c00c8220 2024-04-13T03:13:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.691+0000 7ff0b77fe640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 0x7ff0a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.691+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7ff0b00aeae0 con 0x7ff0c00c8220 2024-04-13T03:13:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.691+0000 7ff0beffd640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 0x7ff0a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.691+0000 7ff0beffd640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 0x7ff0a006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8006fd0 tx=0x7ff0b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.695+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0b0032e20 con 0x7ff0c00c8220 2024-04-13T03:13:58.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.868+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff0c006ef70 con 0x7ff0c00c8220 2024-04-13T03:13:58.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.868+0000 7ff0b77fe640 1 -- 172.21.15.106:0/102292900 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7ff0b007ce50 con 0x7ff0c00c8220 2024-04-13T03:13:58.869 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:13:58.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 msgr2=0x7ff0a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 0x7ff0a006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8006fd0 tx=0x7ff0b8008040 comp rx=0 tx=0).stop 2024-04-13T03:13:58.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 msgr2=0x7ff0c00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d90 tx=0x7ff0b0004a40 comp rx=0 tx=0).stop 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 shutdown_connections 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff0a006c8f0 0x7ff0a006edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff0c015e9b0 0x7ff0c00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c00c8220 0x7ff0c00d22e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 --2- 172.21.15.106:0/102292900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c00b3d60 0x7ff0c006d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 >> 172.21.15.106:0/102292900 conn(0x7ff0c008bf10 msgr2=0x7ff0c00d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 shutdown_connections 2024-04-13T03:13:58.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.870+0000 7ff0c54d5640 1 -- 172.21.15.106:0/102292900 wait complete. 2024-04-13T03:13:58.882 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:13:58.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.985+0000 7fdc261c8640 1 Processor -- start 2024-04-13T03:13:58.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.985+0000 7fdc261c8640 1 -- start start 2024-04-13T03:13:58.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc261c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc2015ea80 0x7fdc201624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc261c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc261c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc200b3fc0 con 0x7fdc2006ee50 2024-04-13T03:13:58.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc261c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdc200b42a0 con 0x7fdc2015ea80 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc261c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdc200b4580 con 0x7fdc201629f0 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1effd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc2015ea80 0x7fdc201624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc2015ea80 0x7fdc201624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:53502/0 (socket says 172.21.15.106:53502) 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1effd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:33894/0 (socket says 172.21.15.106:33894) 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 -- 172.21.15.106:0/3813755524 learned_addr learned my addr 172.21.15.106:0/3813755524 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1ffff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdc2006ee50 0x7fdc2006f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:45316/0 (socket says 172.21.15.106:45316) 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 -- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 msgr2=0x7fdc20162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 -- 172.21.15.106:0/3813755524 >> v1:172.21.15.114:6789/0 conn(0x7fdc2006ee50 legacy=0x7fdc2006f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1f7fe640 1 -- 172.21.15.106:0/3813755524 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2006e930 con 0x7fdc2015ea80 2024-04-13T03:13:58.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.986+0000 7fdc1effd640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.987+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2015ea80 0x7fdc201624b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009870 tx=0x7fdc1002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91facb5988b87a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.987+0000 7fdc1e7fc640 1 -- 172.21.15.106:0/3813755524 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10034070 con 0x7fdc2015ea80 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.987+0000 7fdc1e7fc640 1 -- 172.21.15.106:0/3813755524 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10030840 con 0x7fdc2015ea80 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.987+0000 7fdc1e7fc640 1 -- 172.21.15.106:0/3813755524 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10030c10 con 0x7fdc2015ea80 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 -- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2015ea80 msgr2=0x7fdc201624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2015ea80 0x7fdc201624b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009870 tx=0x7fdc1002b980 comp rx=0 tx=0).stop 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 -- 172.21.15.106:0/3813755524 shutdown_connections 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc20162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 --2- 172.21.15.106:0/3813755524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2015ea80 0x7fdc201624b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 -- 172.21.15.106:0/3813755524 >> 172.21.15.106:0/3813755524 conn(0x7fdc200b2600 msgr2=0x7fdc200d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 -- 172.21.15.106:0/3813755524 shutdown_connections 2024-04-13T03:13:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.988+0000 7fdc261c8640 1 -- 172.21.15.106:0/3813755524 wait complete. 2024-04-13T03:13:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.989+0000 7fdc261c8640 1 Processor -- start 2024-04-13T03:13:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.989+0000 7fdc261c8640 1 -- start start 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 0x7fdc2014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc200cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200aec60 con 0x7fdc2015ea80 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200aeae0 con 0x7fdc2006ee50 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc261c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdc200aede0 con 0x7fdc201629f0 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51640/0 (socket says 172.21.15.106:51640) 2024-04-13T03:13:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 -- 172.21.15.106:0/468778101 learned_addr learned my addr 172.21.15.106:0/468778101 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 0x7fdc2014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1ffff640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc200cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 -- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 msgr2=0x7fdc200cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc200cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 -- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 msgr2=0x7fdc2014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 0x7fdc2014c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 -- 172.21.15.106:0/468778101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc200ccc90 con 0x7fdc2015ea80 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 0x7fdc2014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1ffff640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc200cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc1effd640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ec10 tx=0x7fdc1400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14017070 con 0x7fdc2015ea80 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.990+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1400cc30 con 0x7fdc2015ea80 2024-04-13T03:13:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.991+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc200ccf10 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.991+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14013470 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.991+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc200cd4d0 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.992+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc20068a10 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.992+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc1401a070 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.992+0000 7fdc07fff640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 0x7fdc0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.993+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fdc1408bc70 con 0x7fdc2015ea80 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.993+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 0x7fdc0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:58.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.993+0000 7fdc1f7fe640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 0x7fdc0c06ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc10036000 tx=0x7fdc1003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:58.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:58.997+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc200ccc90 con 0x7fdc2015ea80 2024-04-13T03:13:59.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.182+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc2015e1f0 con 0x7fdc2015ea80 2024-04-13T03:13:59.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.182+0000 7fdc07fff640 1 -- 172.21.15.106:0/468778101 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) ==== 80+0+122 (secure 0 0 0) 0x7fdc14059ec0 con 0x7fdc2015ea80 2024-04-13T03:13:59.184 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.114:6834/390879687 2024-04-14T03:13:50.750656+0000 2024-04-13T03:13:59.184 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.114:6832/390879687 2024-04-14T03:13:50.750656+0000 2024-04-13T03:13:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:listed 2 entries 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 msgr2=0x7fdc0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 0x7fdc0c06ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc10036000 tx=0x7fdc1003b040 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 msgr2=0x7fdc201f8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ec10 tx=0x7fdc1400c520 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 shutdown_connections 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdc0c06c920 0x7fdc0c06ede0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdc201629f0 0x7fdc200cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc2015ea80 0x7fdc201f8a20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 --2- 172.21.15.106:0/468778101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2006ee50 0x7fdc2014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 >> 172.21.15.106:0/468778101 conn(0x7fdc200b2600 msgr2=0x7fdc20162e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.184+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 shutdown_connections 2024-04-13T03:13:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.185+0000 7fdc261c8640 1 -- 172.21.15.106:0/468778101 wait complete. 2024-04-13T03:13:59.196 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:13:59.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:59.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:13:59.227 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:13:59.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:59.227 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:13:59.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:59.282 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-13T03:13:59.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.425+0000 7fab5b208640 1 Processor -- start 2024-04-13T03:13:59.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.425+0000 7fab5b208640 1 -- start start 2024-04-13T03:13:59.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab5b208640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5415b620 0x7fab5406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab5b208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415c760 0x7fab5415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab5b208640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5406f030 con 0x7fab5415bf40 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab5b208640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fab540ce1c0 con 0x7fab5415b620 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab5b208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fab5406fbb0 con 0x7fab5415c760 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5415b620 0x7fab5406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5415b620 0x7fab5406e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51654/0 (socket says 172.21.15.106:51654) 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 -- 172.21.15.106:0/3835732154 learned_addr learned my addr 172.21.15.106:0/3835732154 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 -- 172.21.15.106:0/3835732154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415c760 msgr2=0x7fab5415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 --2- 172.21.15.106:0/3835732154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415c760 0x7fab5415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 -- 172.21.15.106:0/3835732154 >> v1:172.21.15.114:6789/0 conn(0x7fab5415bf40 legacy=0x7fab5415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.426+0000 7fab58fa4640 1 -- 172.21.15.106:0/3835732154 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5406f310 con 0x7fab5415b620 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab58fa4640 1 --2- 172.21.15.106:0/3835732154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415b620 0x7fab5406e7e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fab48009870 tx=0x7fab4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7418c0b7b9c79c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab537fe640 1 -- 172.21.15.106:0/3835732154 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48035020 con 0x7fab5415b620 2024-04-13T03:13:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab537fe640 1 -- 172.21.15.106:0/3835732154 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab48031640 con 0x7fab5415b620 2024-04-13T03:13:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab537fe640 1 -- 172.21.15.106:0/3835732154 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4802d040 con 0x7fab5415b620 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 -- 172.21.15.106:0/3835732154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415b620 msgr2=0x7fab5406e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 --2- 172.21.15.106:0/3835732154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415b620 0x7fab5406e7e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fab48009870 tx=0x7fab4802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 -- 172.21.15.106:0/3835732154 shutdown_connections 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 --2- 172.21.15.106:0/3835732154 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415c760 0x7fab5415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 --2- 172.21.15.106:0/3835732154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415b620 0x7fab5406e7e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 -- 172.21.15.106:0/3835732154 >> 172.21.15.106:0/3835732154 conn(0x7fab540d0370 msgr2=0x7fab540d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 -- 172.21.15.106:0/3835732154 shutdown_connections 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.427+0000 7fab5b208640 1 -- 172.21.15.106:0/3835732154 wait complete. 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.428+0000 7fab5b208640 1 Processor -- start 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 -- start start 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415b620 0x7fab5414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5415bf40 0x7fab54149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab5406ed20 con 0x7fab5415c760 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab5406f8d0 con 0x7fab5415bf40 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab5b208640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fab5415c3d0 con 0x7fab5415b620 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.429+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51670/0 (socket says 172.21.15.106:51670) 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 -- 172.21.15.106:0/1643598666 learned_addr learned my addr 172.21.15.106:0/1643598666 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab58fa4640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415b620 0x7fab5414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 -- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415b620 msgr2=0x7fab5414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab53fff640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5415bf40 0x7fab54149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415b620 0x7fab5414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 -- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5415bf40 msgr2=0x7fab54149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5415bf40 0x7fab54149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 -- 172.21.15.106:0/1643598666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab54150750 con 0x7fab5415c760 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab597a5640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c780 tx=0x7fab4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab517fa640 1 -- 172.21.15.106:0/1643598666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c017070 con 0x7fab5415c760 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab517fa640 1 -- 172.21.15.106:0/1643598666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c003da0 con 0x7fab5415c760 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab517fa640 1 -- 172.21.15.106:0/1643598666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0052b0 con 0x7fab5415c760 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab54150ca0 con 0x7fab5415c760 2024-04-13T03:13:59.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.430+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5415adb0 con 0x7fab5415c760 2024-04-13T03:13:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.432+0000 7fab517fa640 1 -- 172.21.15.106:0/1643598666 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab4c01a070 con 0x7fab5415c760 2024-04-13T03:13:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.432+0000 7fab517fa640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 0x7fab3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.432+0000 7fab517fa640 1 -- 172.21.15.106:0/1643598666 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fab4c08b360 con 0x7fab5415c760 2024-04-13T03:13:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.432+0000 7fab58fa4640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 0x7fab3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.433+0000 7fab58fa4640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 0x7fab3806ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab4803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.433+0000 7fab5dff5740 1 Processor -- start 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.433+0000 7fab5dff5740 1 -- start start 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae949ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94ca2b0 0x55cae94ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cae94073b0 con 0x55cae949d3b0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cae93a3a80 con 0x55cae94c9ed0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cae9403370 con 0x55cae94ca2b0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51680/0 (socket says 172.21.15.106:51680) 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 -- 172.21.15.106:0/1143884615 learned_addr learned my addr 172.21.15.106:0/1143884615 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab53fff640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94ca2b0 0x55cae94ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab58fa4640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae949ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 -- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94ca2b0 msgr2=0x55cae94ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94ca2b0 0x55cae94ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 -- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 msgr2=0x55cae949ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae949ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 -- 172.21.15.106:0/1143884615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae91e5a20 con 0x55cae949d3b0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab58fa4640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae949ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.434+0000 7fab597a5640 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fab4c061560 tx=0x7fab4c058f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29a6f0387769c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab3e7fc640 1 -- 172.21.15.106:0/1143884615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c056f50 con 0x55cae949d3b0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab3e7fc640 1 -- 172.21.15.106:0/1143884615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c0571b0 con 0x55cae949d3b0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab3e7fc640 1 -- 172.21.15.106:0/1143884615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c055520 con 0x55cae949d3b0 2024-04-13T03:13:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 -- 172.21.15.106:0/1143884615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 msgr2=0x55cae94cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fab4c061560 tx=0x7fab4c058f70 comp rx=0 tx=0).stop 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 -- 172.21.15.106:0/1143884615 shutdown_connections 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94ca2b0 0x55cae94ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae949d3b0 0x55cae94cd1a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 --2- 172.21.15.106:0/1143884615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae949ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 -- 172.21.15.106:0/1143884615 >> 172.21.15.106:0/1143884615 conn(0x55cae949f0e0 msgr2=0x55cae94e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 -- 172.21.15.106:0/1143884615 shutdown_connections 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.435+0000 7fab5dff5740 1 -- 172.21.15.106:0/1143884615 wait complete. 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.436+0000 7fab5dff5740 1 -- start start 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.436+0000 7fab5dff5740 1 Processor -- start 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae94e8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae94cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cae94a1780 con 0x55cae94cba20 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cae93a1410 con 0x55cae94ca2b0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cae94a3fc0 con 0x55cae94c9ed0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 10 client.0 _ll_register_callbacks cb 0x55cae938d230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab58fa4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae94e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51688/0 (socket says 172.21.15.106:51688) 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 -- 172.21.15.106:0/629191087 learned_addr learned my addr 172.21.15.106:0/629191087 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab53fff640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae94cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 -- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 msgr2=0x55cae94e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae94e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 -- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 msgr2=0x55cae94cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae94cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 -- 172.21.15.106:0/629191087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae90d9310 con 0x55cae94cba20 2024-04-13T03:13:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab597a5640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fab4c058e40 tx=0x7fab4c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530853ddc35433f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab337fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c075020 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fab4c078310 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.437+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4c0571b0 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fab4c07f050 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 10 client.7164 fetch_fsmap learned FSMap version 43 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55cae92461b0 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c0571b0 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fab4c061ce0 con 0x55cae94cba20 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 10 client.7164 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 10 client.7164 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 20 client.7164 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94cd1c0 0x55cae94b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 4 client.7164 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae9414dd0 con 0x55cae94cd1c0 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab58fa4640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94cd1c0 0x55cae94b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab337fe640 10 client.7164 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.438+0000 7fab58fa4640 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94cd1c0 0x55cae94b2ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfaa65e799829d2 server_cookie=caff5c375a046f42 in_seq=0 out_seq=0 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab337fe640 1 -- 172.21.15.106:0/629191087 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cae9414dd0 con 0x55cae94cd1c0 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab337fe640 10 client.7164 handle_command_reply: tid=0 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 client.7164 shutdown 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 msgr2=0x55cae94c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fab4c058e40 tx=0x7fab4c003f90 comp rx=0 tx=0).stop 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 shutdown_connections 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94cd1c0 0x55cae94b2ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c6d80 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae94cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab337fe640 0 client.7164 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 --2- 172.21.15.106:0/629191087 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae94e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 >> 172.21.15.106:0/629191087 conn(0x55cae949d3b0 msgr2=0x55cae94a0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 shutdown_connections 2024-04-13T03:13:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 1 -- 172.21.15.106:0/629191087 wait complete. 2024-04-13T03:13:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.439+0000 7fab5dff5740 20 client.7164 trim_cache size 0 max 16384 2024-04-13T03:13:59.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 Processor -- start 2024-04-13T03:13:59.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 -- start start 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae949bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae949c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ff610 con 0x55cae94cba20 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab541fe670 con 0x55cae94ca2b0 2024-04-13T03:13:59.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fab54155aa0 con 0x55cae94c9ed0 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab597a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51694/0 (socket says 172.21.15.106:51694) 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.457+0000 7fab597a5640 1 -- 172.21.15.106:0/985319102 learned_addr learned my addr 172.21.15.106:0/985319102 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab58fa4640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae949bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab53fff640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae949c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 -- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 msgr2=0x55cae949bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae949bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 -- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 msgr2=0x55cae949c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae949c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 -- 172.21.15.106:0/985319102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae90b2490 con 0x55cae94cba20 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab58fa4640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae949bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab597a5640 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fab4c089700 tx=0x7fab4c084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7afde33378db10f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab3dffb640 1 -- 172.21.15.106:0/985319102 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c075040 con 0x55cae94cba20 2024-04-13T03:13:59.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab3dffb640 1 -- 172.21.15.106:0/985319102 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c0571b0 con 0x55cae94cba20 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab3dffb640 1 -- 172.21.15.106:0/985319102 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0589f0 con 0x55cae94cba20 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab5dff5740 1 -- 172.21.15.106:0/985319102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 msgr2=0x55cae94c1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.458+0000 7fab5dff5740 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fab4c089700 tx=0x7fab4c084000 comp rx=0 tx=0).stop 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 -- 172.21.15.106:0/985319102 shutdown_connections 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94cba20 0x55cae94c1d00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94ca2b0 0x55cae949c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 --2- 172.21.15.106:0/985319102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94c9ed0 0x55cae949bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 -- 172.21.15.106:0/985319102 >> 172.21.15.106:0/985319102 conn(0x55cae949d3b0 msgr2=0x55cae94ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 -- 172.21.15.106:0/985319102 shutdown_connections 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 -- 172.21.15.106:0/985319102 wait complete. 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 -- start start 2024-04-13T03:13:59.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.459+0000 7fab5dff5740 1 Processor -- start 2024-04-13T03:13:59.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae94b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94cba20 0x55cae94c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cae94a33c0 con 0x55cae94ca2b0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab541fe020 con 0x55cae94c9ed0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cae9561de0 con 0x55cae94cba20 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab5dff5740 10 client.0 _ll_register_callbacks cb 0x7fab4c059640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51704/0 (socket says 172.21.15.106:51704) 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 -- 172.21.15.106:0/2885525833 learned_addr learned my addr 172.21.15.106:0/2885525833 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab58fa4640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae94b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab597a5640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94cba20 0x55cae94c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 -- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94cba20 msgr2=0x55cae94c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94cba20 0x55cae94c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 -- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 msgr2=0x55cae94b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae94b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.460+0000 7fab53fff640 1 -- 172.21.15.106:0/2885525833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae94b3060 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab53fff640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fab54154070 tx=0x7fab40013760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04f6d897c0741d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40015630 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fab400157d0 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4c0571b0 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fab40027050 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 10 client.7170 fetch_fsmap learned FSMap version 43 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55cae9561b80 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c0571b0 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fab4002a020 con 0x55cae94ca2b0 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 10 client.7170 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 10 client.7170 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 20 client.7170 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94c8d10 0x55cae94c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 4 client.7170 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55cae940ec40 con 0x55cae94c8d10 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab58fa4640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94c8d10 0x55cae94c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.461+0000 7fab337fe640 10 client.7170 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab58fa4640 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94c8d10 0x55cae94c9120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676b3279bfd275f2 server_cookie=dd1025a529f20596 in_seq=0 out_seq=0 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab337fe640 1 -- 172.21.15.106:0/2885525833 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cae940ec40 con 0x55cae94c8d10 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab337fe640 10 client.7170 handle_command_reply: tid=0 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 client.7170 shutdown 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 msgr2=0x55cae94c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fab54154070 tx=0x7fab40013760 comp rx=0 tx=0).stop 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 shutdown_connections 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55cae94c8d10 0x55cae94c9120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cae94cba20 0x55cae94c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cae94ca2b0 0x55cae94c8240 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 --2- 172.21.15.106:0/2885525833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cae94c9ed0 0x55cae94b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 >> 172.21.15.106:0/2885525833 conn(0x55cae949d3b0 msgr2=0x55cae94eae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.462+0000 7fab337fe640 0 client.7170 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 shutdown_connections 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5dff5740 1 -- 172.21.15.106:0/2885525833 wait complete. 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5dff5740 20 client.7170 trim_cache size 0 max 16384 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "29ec8672-cfa3-4bca-a8be-49434c27ffed", 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:13:59.470 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 msgr2=0x7fab3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 0x7fab3806ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab4803c040 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 msgr2=0x7fab540b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c780 tx=0x7fab4c00cc40 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 shutdown_connections 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fab3806c920 0x7fab3806ede0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5415c760 0x7fab540b3050 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5415bf40 0x7fab54149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 --2- 172.21.15.106:0/1643598666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fab5415b620 0x7fab5414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 >> 172.21.15.106:0/1643598666 conn(0x7fab540d0370 msgr2=0x7fab5406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 shutdown_connections 2024-04-13T03:13:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.463+0000 7fab5b208640 1 -- 172.21.15.106:0/1643598666 wait complete. 2024-04-13T03:13:59.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:13:59.475 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-13T03:13:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.576+0000 7f3b33ede640 1 Processor -- start 2024-04-13T03:13:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.576+0000 7f3b33ede640 1 -- start start 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b33ede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b2c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b33ede640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3b2c0b54a0 0x7f3b2c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b33ede640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0b5cf0 con 0x7f3b2c0709e0 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b33ede640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2c0b5fd0 con 0x7f3b2c0b54a0 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b33ede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0b62b0 con 0x7f3b2c0b1c30 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b3247b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3b2c0709e0 0x7f3b2c162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40824/0 (socket says 172.21.15.106:40824) 2024-04-13T03:13:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b3247b640 1 -- 172.21.15.106:0/1764254006 learned_addr learned my addr 172.21.15.106:0/1764254006 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762967834 0 0) 0x7f3b2c0b5cf0 con 0x7f3b2c0709e0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b2c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b18003520 con 0x7f3b2c0709e0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b31479640 1 --2- 172.21.15.106:0/1764254006 >> v2:172.21.15.114:3300/0 conn(0x7f3b2c0b54a0 0x7f3b2c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1590289320 0 0) 0x7f3b18003520 con 0x7f3b2c0709e0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 msgr2=0x7f3b2c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b2c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 >> v2:172.21.15.114:3300/0 conn(0x7f3b2c0b54a0 msgr2=0x7f3b2c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 --2- 172.21.15.106:0/1764254006 >> v2:172.21.15.114:3300/0 conn(0x7f3b2c0b54a0 0x7f3b2c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0b6590 con 0x7f3b2c0709e0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f3b240019f0 con 0x7f3b2c0709e0 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 >> v1:172.21.15.106:6789/0 conn(0x7f3b2c0709e0 legacy=0x7f3b2c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.577+0000 7f3b30c78640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b18003f30 0x7f3b18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b30c78640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b18004d40 0x7f3b180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0b5fd0 con 0x7f3b18003f30 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0b62b0 con 0x7f3b18004d40 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1800b9c0 con 0x7f3b2c0b1c30 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b18003f30 0x7f3b18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b3247b640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b18004d40 0x7f3b180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 -- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 msgr2=0x7f3b180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b2c0b1c30 0x7f3b180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 -- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b18004d40 msgr2=0x7f3b180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b18004d40 0x7f3b180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.578+0000 7f3b31479640 1 -- 172.21.15.106:0/1764254006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b18003520 con 0x7f3b18003f30 2024-04-13T03:13:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b31479640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b18003f30 0x7f3b18004800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71b91526957f1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c039070 con 0x7f3b18003f30 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c02c6d0 con 0x7f3b18003f30 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b30c78640 1 -- 172.21.15.106:0/1764254006 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c009690 con 0x7f3b18003f30 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 -- 172.21.15.106:0/1764254006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b18003f30 msgr2=0x7f3b18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 --2- 172.21.15.106:0/1764254006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b18003f30 0x7f3b18004800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b18005830 comp rx=0 tx=0).stop 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b3247b640 1 -- 172.21.15.106:0/1764254006 reap_dead start 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 -- 172.21.15.106:0/1764254006 shutdown_connections 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 -- 172.21.15.106:0/1764254006 >> 172.21.15.106:0/1764254006 conn(0x7f3b2c071870 msgr2=0x7f3b2c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 -- 172.21.15.106:0/1764254006 shutdown_connections 2024-04-13T03:13:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.579+0000 7f3b33ede640 1 -- 172.21.15.106:0/1764254006 wait complete. 2024-04-13T03:13:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 Processor -- start 2024-04-13T03:13:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 -- start start 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 0x7f3b2c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b2c1438f0 0x7f3b2c15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c06e7f0 con 0x7f3b2c146140 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c06e670 con 0x7f3b2c1438f0 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b33ede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c06e970 con 0x7f3b18003f30 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51716/0 (socket says 172.21.15.106:51716) 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b3247b640 1 -- 172.21.15.106:0/2847240209 learned_addr learned my addr 172.21.15.106:0/2847240209 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.581+0000 7f3b31479640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b2c1438f0 0x7f3b2c15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 0x7f3b2c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 -- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 msgr2=0x7f3b2c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 0x7f3b2c0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 -- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b2c1438f0 msgr2=0x7f3b2c15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b2c1438f0 0x7f3b2c15fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 -- 172.21.15.106:0/2847240209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c160270 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 0x7f3b2c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001560 tx=0x7f3b24007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b227fc640 1 -- 172.21.15.106:0/2847240209 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24017020 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b227fc640 1 -- 172.21.15.106:0/2847240209 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2400ab20 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c162650 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b227fc640 1 -- 172.21.15.106:0/2847240209 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240133f0 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.582+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c162c10 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.583+0000 7f3b227fc640 1 -- 172.21.15.106:0/2847240209 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b2401a020 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.584+0000 7f3b227fc640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.584+0000 7f3b227fc640 1 -- 172.21.15.106:0/2847240209 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3b2408c340 con 0x7f3b2c146140 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.584+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.584+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 0x7f3b1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009ae0 tx=0x7f3b28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.585+0000 7f3b36dbb740 1 Processor -- start 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.585+0000 7f3b36dbb740 1 -- start start 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x564604675f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046764d0 0x5646046a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5646045e1170 con 0x5646046a33d0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5646045877f0 con 0x5646046a2ff0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5646045dd130 con 0x5646046764d0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51730/0 (socket says 172.21.15.106:51730) 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 -- 172.21.15.106:0/990708757 learned_addr learned my addr 172.21.15.106:0/990708757 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x564604675f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 -- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046764d0 msgr2=0x5646046a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046764d0 0x5646046a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 -- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 msgr2=0x564604675f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x564604675f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 -- 172.21.15.106:0/990708757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646043bd7d0 con 0x5646046a33d0 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x564604675f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b31479640 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b1c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aeb4eeed190c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b0f7fe640 1 -- 172.21.15.106:0/990708757 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c039030 con 0x5646046a33d0 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b0f7fe640 1 -- 172.21.15.106:0/990708757 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c03a040 con 0x5646046a33d0 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.586+0000 7f3b0f7fe640 1 -- 172.21.15.106:0/990708757 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004440 con 0x5646046a33d0 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 -- 172.21.15.106:0/990708757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 msgr2=0x5646046a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009ac0 tx=0x7f3b1c02e000 comp rx=0 tx=0).stop 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 -- 172.21.15.106:0/990708757 shutdown_connections 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046764d0 0x5646046a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046a5d80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/990708757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x564604675f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 -- 172.21.15.106:0/990708757 >> 172.21.15.106:0/990708757 conn(0x564604678200 msgr2=0x5646046c0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 -- 172.21.15.106:0/990708757 shutdown_connections 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.587+0000 7f3b36dbb740 1 -- 172.21.15.106:0/990708757 wait complete. 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 -- start start 2024-04-13T03:13:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 Processor -- start 2024-04-13T03:13:59.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a2ff0 0x5646046a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046d1a10 0x5646046a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.588+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56460467a8a0 con 0x5646046a33d0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56460459ad00 con 0x5646046d1a10 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56460467d0e0 con 0x5646046a2ff0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 10 client.0 _ll_register_callbacks cb 0x56460459a6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51742/0 (socket says 172.21.15.106:51742) 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 -- 172.21.15.106:0/2101039286 learned_addr learned my addr 172.21.15.106:0/2101039286 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a2ff0 0x5646046a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046d1a10 0x5646046a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 -- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a2ff0 msgr2=0x5646046a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a2ff0 0x5646046a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 -- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046d1a10 msgr2=0x5646046a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046d1a10 0x5646046a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 -- 172.21.15.106:0/2101039286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646042f33f0 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b31479640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009be0 tx=0x7f3b1c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01109eadd0384d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005b90 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3b1c005d30 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1c03a040 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c059050 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 10 client.7182 fetch_fsmap learned FSMap version 43 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.589+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5646042b30f0 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c03a040 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f3b1c05c020 con 0x5646046a33d0 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 10 client.7182 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:13:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 10 client.7182 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 20 client.7182 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5646046a95d0 0x5646046d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 4 client.7182 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646045eeb90 con 0x5646046a95d0 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5646046a95d0 0x5646046d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5646046a95d0 0x5646046d4580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfe00e5c5e57918 server_cookie=3fe94f605d101840 in_seq=0 out_seq=0 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.590+0000 7f3b0cff9640 10 client.7182 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b0cff9640 1 -- 172.21.15.106:0/2101039286 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646045eeb90 con 0x5646046a95d0 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b0cff9640 10 client.7182 handle_command_reply: tid=0 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 client.7182 shutdown 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 msgr2=0x5646046bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009be0 tx=0x7f3b1c02e070 comp rx=0 tx=0).stop 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 shutdown_connections 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5646046a95d0 0x5646046d4580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046d1a10 0x5646046a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046a33d0 0x5646046bfd70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2101039286 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a2ff0 0x5646046a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b0cff9640 0 client.7182 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 >> 172.21.15.106:0/2101039286 conn(0x5646046764d0 msgr2=0x5646046c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 shutdown_connections 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2101039286 wait complete. 2024-04-13T03:13:59.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.591+0000 7f3b36dbb740 20 client.7182 trim_cache size 0 max 16384 2024-04-13T03:13:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.608+0000 7f3b36dbb740 1 Processor -- start 2024-04-13T03:13:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 -- start start 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c201c80 con 0x5646046d1a10 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c200cc0 con 0x5646046a2ff0 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c200020 con 0x5646046a33d0 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51752/0 (socket says 172.21.15.106:51752) 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.609+0000 7f3b3247b640 1 -- 172.21.15.106:0/2429323526 learned_addr learned my addr 172.21.15.106:0/2429323526 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b31479640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 -- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 msgr2=0x5646046a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 -- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 msgr2=0x5646046d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 -- 172.21.15.106:0/2429323526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564604613ac0 con 0x5646046d1a10 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b31479640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b3247b640 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3b240625d0 tx=0x7f3b24059d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eedb6d15545d780 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b0effd640 1 -- 172.21.15.106:0/2429323526 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24057fc0 con 0x5646046d1a10 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b0effd640 1 -- 172.21.15.106:0/2429323526 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24087070 con 0x5646046d1a10 2024-04-13T03:13:59.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b0effd640 1 -- 172.21.15.106:0/2429323526 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2408c0c0 con 0x5646046d1a10 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2429323526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 msgr2=0x5646046a8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3b240625d0 tx=0x7f3b24059d70 comp rx=0 tx=0).stop 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2429323526 shutdown_connections 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a8900 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/2429323526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2429323526 >> 172.21.15.106:0/2429323526 conn(0x5646046764d0 msgr2=0x5646046c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2429323526 shutdown_connections 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.610+0000 7f3b36dbb740 1 -- 172.21.15.106:0/2429323526 wait complete. 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.611+0000 7f3b36dbb740 1 -- start start 2024-04-13T03:13:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.611+0000 7f3b36dbb740 1 Processor -- start 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5646046c4c20 con 0x5646046d1a10 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c200670 con 0x5646046a2ff0 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5646046a70e0 con 0x5646046a33d0 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b36dbb740 10 client.0 _ll_register_callbacks cb 0x7f3b1c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:13:59.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51760/0 (socket says 172.21.15.106:51760) 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 -- 172.21.15.106:0/146015018 learned_addr learned my addr 172.21.15.106:0/146015018 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b31479640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 -- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 msgr2=0x5646046a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 -- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 msgr2=0x5646046c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 -- 172.21.15.106:0/146015018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646046cf3e0 con 0x5646046d1a10 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b3247b640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3b24059e60 tx=0x7f3b24000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed352dffc7eccfce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.612+0000 7f3b0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2408a370 con 0x5646046d1a10 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3b240721e0 con 0x5646046d1a10 2024-04-13T03:13:59.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b24087070 con 0x5646046d1a10 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3b24083050 con 0x5646046d1a10 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 10 client.7188 fetch_fsmap learned FSMap version 43 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x56460473a7c0 con 0x5646046d1a10 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24087070 con 0x5646046d1a10 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f3b24059370 con 0x5646046d1a10 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 10 client.7188 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 20 client.7188 populate_metadata read hostname 'smithi106' 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x56460467a9b0 0x56460467adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 4 client.7188 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646045e8a00 con 0x56460467a9b0 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x56460467a9b0 0x56460467adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b0cff9640 10 client.7188 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.613+0000 7f3b31c7a640 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x56460467a9b0 0x56460467adc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bdae6387150bed server_cookie=2d16fd0acc2277bb in_seq=0 out_seq=0 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b0cff9640 1 -- 172.21.15.106:0/146015018 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646045e8a00 con 0x56460467a9b0 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b0cff9640 10 client.7188 handle_command_reply: tid=0 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 client.7188 shutdown 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 msgr2=0x5646046a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3b24059e60 tx=0x7f3b24000f40 comp rx=0 tx=0).stop 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 shutdown_connections 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x56460467a9b0 0x56460467adc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5646046a33d0 0x5646046a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5646046d1a10 0x5646046a87c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b0cff9640 0 client.7188 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 --2- 172.21.15.106:0/146015018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646046a2ff0 0x5646046c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 >> 172.21.15.106:0/146015018 conn(0x5646046764d0 msgr2=0x5646046e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 shutdown_connections 2024-04-13T03:13:59.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 1 -- 172.21.15.106:0/146015018 wait complete. 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b36dbb740 20 client.7188 trim_cache size 0 max 16384 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "29ec8672-cfa3-4bca-a8be-49434c27ffed": { 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/subdir", 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "29ec8672-cfa3-4bca-a8be-49434c27ffed", 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive" 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:59.622 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:13:59.623 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:13:59.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 msgr2=0x7f3b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 0x7f3b1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009ae0 tx=0x7f3b28008040 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 msgr2=0x7f3b2c1465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001560 tx=0x7f3b24007a90 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 shutdown_connections 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b2c146140 0x7f3b2c1465c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b2c1438f0 0x7f3b2c15fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 --2- 172.21.15.106:0/2847240209 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3b18003f30 0x7f3b2c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.614+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 >> 172.21.15.106:0/2847240209 conn(0x7f3b2c071870 msgr2=0x7f3b2c0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.615+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 shutdown_connections 2024-04-13T03:13:59.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:13:59.615+0000 7f3b33ede640 1 -- 172.21.15.106:0/2847240209 wait complete. 2024-04-13T03:13:59.626 INFO:tasks.cephfs.filesystem:scrub status for tag:29ec8672-cfa3-4bca-a8be-49434c27ffed - {'path': '/subdir', 'tag': '29ec8672-cfa3-4bca-a8be-49434c27ffed', 'options': 'recursive'} 2024-04-13T03:14:25.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:14:25.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:14:29.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:14:29.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-13T03:14:29.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.741+0000 7f9bbb629640 1 Processor -- start 2024-04-13T03:14:29.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.741+0000 7f9bbb629640 1 -- start start 2024-04-13T03:14:29.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bbb629640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9bb40af800 0x7f9bb415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bbb629640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bb406f690 0x7f9bb415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bbb629640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb415c200 con 0x7f9bb406f200 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bbb629640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb40b5090 con 0x7f9bb406f690 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bbb629640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb406fb00 con 0x7f9bb40af800 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bb406f690 0x7f9bb415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bb406f690 0x7f9bb415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50020/0 (socket says 172.21.15.106:50020) 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/78929993 learned_addr learned my addr 172.21.15.106:0/78929993 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/78929993 >> v2:172.21.15.114:3300/0 conn(0x7f9bb40af800 0x7f9bb415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/78929993 >> v2:172.21.15.114:3300/0 conn(0x7f9bb40af800 msgr2=0x7f9bb415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/78929993 >> v2:172.21.15.114:3300/0 conn(0x7f9bb40af800 0x7f9bb415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/78929993 >> v1:172.21.15.114:6789/0 conn(0x7f9bb406f200 legacy=0x7f9bb415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.742+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/78929993 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb40b3ff0 con 0x7f9bb406f690 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.743+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/78929993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b2e50 tx=0x7f9bac02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce68c6d9482aa29 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.743+0000 7f9babfff640 1 -- 172.21.15.106:0/78929993 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac02ea90 con 0x7f9bb406f690 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.743+0000 7f9babfff640 1 -- 172.21.15.106:0/78929993 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bac02ec30 con 0x7f9bb406f690 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.743+0000 7f9babfff640 1 -- 172.21.15.106:0/78929993 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac009600 con 0x7f9bb406f690 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 -- 172.21.15.106:0/78929993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 msgr2=0x7f9bb415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 --2- 172.21.15.106:0/78929993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b2e50 tx=0x7f9bac02e0f0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 -- 172.21.15.106:0/78929993 shutdown_connections 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 --2- 172.21.15.106:0/78929993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 --2- 172.21.15.106:0/78929993 >> v2:172.21.15.114:3300/0 conn(0x7f9bb40af800 0x7f9bb415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 -- 172.21.15.106:0/78929993 >> 172.21.15.106:0/78929993 conn(0x7f9bb40b2600 msgr2=0x7f9bb40d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 -- 172.21.15.106:0/78929993 shutdown_connections 2024-04-13T03:14:29.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.744+0000 7f9bbb629640 1 -- 172.21.15.106:0/78929993 wait complete. 2024-04-13T03:14:29.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.745+0000 7f9bbb629640 1 Processor -- start 2024-04-13T03:14:29.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.745+0000 7f9bbb629640 1 -- start start 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bb406f200 0x7f9bb41404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bb40af800 0x7f9bb414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb40b4890 con 0x7f9bb406f690 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb40b45b0 con 0x7f9bb40af800 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bbb629640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb40b4b70 con 0x7f9bb406f200 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50036/0 (socket says 172.21.15.106:50036) 2024-04-13T03:14:29.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/2080369490 learned_addr learned my addr 172.21.15.106:0/2080369490 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bb406f200 0x7f9bb41404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bb40af800 0x7f9bb414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bb406f200 msgr2=0x7f9bb41404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bb406f200 0x7f9bb41404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bb40af800 msgr2=0x7f9bb414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bb40af800 0x7f9bb414df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.746+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/2080369490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb41f57a0 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400b3e0 tx=0x7f9ba400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9ba9ffb640 1 -- 172.21.15.106:0/2080369490 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4015020 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9ba9ffb640 1 -- 172.21.15.106:0/2080369490 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4011850 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb415e140 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9ba9ffb640 1 -- 172.21.15.106:0/2080369490 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4011c40 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.747+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb415e700 con 0x7f9bb406f690 2024-04-13T03:14:29.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.748+0000 7f9ba9ffb640 1 -- 172.21.15.106:0/2080369490 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ba4018020 con 0x7f9bb406f690 2024-04-13T03:14:29.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.749+0000 7f9ba9ffb640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 0x7f9b9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.749+0000 7f9ba9ffb640 1 -- 172.21.15.106:0/2080369490 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9ba408bb60 con 0x7f9bb406f690 2024-04-13T03:14:29.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.749+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 0x7f9b9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.749+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 0x7f9b9806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000aac0 tx=0x7f9bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.750+0000 7f9bbe40b740 1 Processor -- start 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.750+0000 7f9bbe40b740 1 -- start start 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad20c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20c44d0 0x561ad20f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561ad202f170 con 0x561ad20f13d0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ad1fd57f0 con 0x561ad20f0ff0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561ad202b130 con 0x561ad20c44d0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50048/0 (socket says 172.21.15.106:50048) 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/1593061839 learned_addr learned my addr 172.21.15.106:0/1593061839 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20c44d0 0x561ad20f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad20c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20c44d0 msgr2=0x561ad20f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20c44d0 0x561ad20f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 msgr2=0x561ad20c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad20c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/1593061839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad1e0b7d0 con 0x561ad20f13d0 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4089ee0 tx=0x7f9ba405dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4643ac24f0dfe7c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9b9effd640 1 -- 172.21.15.106:0/1593061839 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4088070 con 0x561ad20f13d0 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9b9effd640 1 -- 172.21.15.106:0/1593061839 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4055e60 con 0x561ad20f13d0 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.751+0000 7f9b9effd640 1 -- 172.21.15.106:0/1593061839 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4058e60 con 0x561ad20f13d0 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1593061839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 msgr2=0x561ad20f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4089ee0 tx=0x7f9ba405dc70 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1593061839 shutdown_connections 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20c44d0 0x561ad20f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad20f3d80 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1593061839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad20c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1593061839 >> 172.21.15.106:0/1593061839 conn(0x561ad20c6200 msgr2=0x561ad210ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1593061839 shutdown_connections 2024-04-13T03:14:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.752+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1593061839 wait complete. 2024-04-13T03:14:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.753+0000 7f9bbe40b740 1 -- start start 2024-04-13T03:14:29.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.753+0000 7f9bbe40b740 1 Processor -- start 2024-04-13T03:14:29.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f13d0 0x561ad20f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f6710 0x561ad2122910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561ad202f170 con 0x561ad20f0ff0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ad1fe8d00 con 0x561ad20f6710 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561ad20c88a0 con 0x561ad20f13d0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 10 client.0 _ll_register_callbacks cb 0x561ad1fe86f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50062/0 (socket says 172.21.15.106:50062) 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 -- 172.21.15.106:0/1289725457 learned_addr learned my addr 172.21.15.106:0/1289725457 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f13d0 0x561ad20f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f6710 0x561ad2122910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 -- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f13d0 msgr2=0x561ad20f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f13d0 0x561ad20f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 -- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f6710 msgr2=0x561ad2122910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f6710 0x561ad2122910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 -- 172.21.15.106:0/1289725457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad1d413f0 con 0x561ad20f0ff0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0015a00 tx=0x7f9bb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f477cd0d7892fb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9b93fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:29.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0024030 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9bb0013bb0 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ba4055e60 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9bb0030050 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.754+0000 7f9bbe40b740 10 client.7200 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x561ad1d010f0 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4055e60 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f9bb0033020 con 0x561ad20f0ff0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 10 client.7200 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 10 client.7200 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 20 client.7200 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad210dee0 0x561ad210e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 4 client.7200 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ad203cb90 con 0x561ad210dee0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad210dee0 0x561ad210e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9b93fff640 10 client.7200 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:29.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.755+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad210dee0 0x561ad210e310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0af6c72174e29e server_cookie=cbe1cdf17bab7dee in_seq=0 out_seq=0 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9b93fff640 1 -- 172.21.15.106:0/1289725457 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ad203cb90 con 0x561ad210dee0 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9b93fff640 10 client.7200 handle_command_reply: tid=0 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 client.7200 shutdown 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 msgr2=0x561ad20f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0015a00 tx=0x7f9bb00046c0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 shutdown_connections 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad210dee0 0x561ad210e310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f13d0 0x561ad20f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9b93fff640 0 client.7200 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f6710 0x561ad2122910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1289725457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f0ff0 0x561ad20f9730 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 >> 172.21.15.106:0/1289725457 conn(0x561ad20c44d0 msgr2=0x561ad20c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 shutdown_connections 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1289725457 wait complete. 2024-04-13T03:14:29.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.756+0000 7f9bbe40b740 20 client.7200 trim_cache size 0 max 16384 2024-04-13T03:14:29.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 Processor -- start 2024-04-13T03:14:29.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 -- start start 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad210d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 0x561ad2110480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4201730 con 0x561ad20f13d0 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4200710 con 0x561ad20f0ff0 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.774+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb415d110 con 0x561ad20f6710 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50076/0 (socket says 172.21.15.106:50076) 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/3702296504 learned_addr learned my addr 172.21.15.106:0/3702296504 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad210d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 msgr2=0x561ad2110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 0x561ad2110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 0x561ad2110480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 msgr2=0x561ad210d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad210d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 -- 172.21.15.106:0/3702296504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad20f7fb0 con 0x561ad20f13d0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad210d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 0x561ad2110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400bda0 tx=0x7f9ba405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f151a1458722f842 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9b9e7fc640 1 -- 172.21.15.106:0/3702296504 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4088070 con 0x561ad20f13d0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9b9e7fc640 1 -- 172.21.15.106:0/3702296504 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4089040 con 0x561ad20f13d0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.775+0000 7f9b9e7fc640 1 -- 172.21.15.106:0/3702296504 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4078d70 con 0x561ad20f13d0 2024-04-13T03:14:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- 172.21.15.106:0/3702296504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 msgr2=0x561ad210ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400bda0 tx=0x7f9ba405c000 comp rx=0 tx=0).stop 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- 172.21.15.106:0/3702296504 shutdown_connections 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f6710 0x561ad2110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f13d0 0x561ad210ff40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/3702296504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f0ff0 0x561ad210d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- 172.21.15.106:0/3702296504 >> 172.21.15.106:0/3702296504 conn(0x561ad20c44d0 msgr2=0x561ad2111af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- 172.21.15.106:0/3702296504 shutdown_connections 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- 172.21.15.106:0/3702296504 wait complete. 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 -- start start 2024-04-13T03:14:29.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.776+0000 7f9bbe40b740 1 Processor -- start 2024-04-13T03:14:29.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f0ff0 0x561ad2131e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f13d0 0x561ad20f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb42000d0 con 0x561ad20f6710 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ad21895d0 con 0x561ad20f13d0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561ad2132ed0 con 0x561ad20f0ff0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bbe40b740 10 client.0 _ll_register_callbacks cb 0x7f9bb0015680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bb9bc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bb9bc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50078/0 (socket says 172.21.15.106:50078) 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/1128080800 learned_addr learned my addr 172.21.15.106:0/1128080800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bb8bc4640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f13d0 0x561ad20f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.777+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f0ff0 0x561ad2131e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f0ff0 msgr2=0x561ad2131e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f0ff0 0x561ad2131e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f13d0 msgr2=0x561ad20f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f13d0 0x561ad20f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 -- 172.21.15.106:0/1128080800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad2132660 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bb9bc6640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9bac02e130 tx=0x7f9bac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d519f963e580f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac040980 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9bac04ac80 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ba4089040 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9bac04aea0 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 10 client.7206 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x561ad210d700 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4089040 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f9bac048070 con 0x561ad20f6710 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 10 client.7206 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 20 client.7206 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad20f42a0 0x561ad218b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 4 client.7206 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.778+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ad2036a00 con 0x561ad20f42a0 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad20f42a0 0x561ad218b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9b93fff640 10 client.7206 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bb93c5640 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad20f42a0 0x561ad218b740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a33b129869ad59e server_cookie=e0c9e9184ca9a9f in_seq=0 out_seq=0 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9b93fff640 1 -- 172.21.15.106:0/1128080800 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ad2036a00 con 0x561ad20f42a0 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9b93fff640 10 client.7206 handle_command_reply: tid=0 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 client.7206 shutdown 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 msgr2=0x561ad20f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9bac02e130 tx=0x7f9bac004ab0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 shutdown_connections 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x561ad20f42a0 0x561ad218b740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ad20f6710 0x561ad20f3d60 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad20f13d0 0x561ad20f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9b93fff640 0 client.7206 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 --2- 172.21.15.106:0/1128080800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561ad20f0ff0 0x561ad2131e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 >> 172.21.15.106:0/1128080800 conn(0x561ad20c44d0 msgr2=0x561ad211e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 shutdown_connections 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 1 -- 172.21.15.106:0/1128080800 wait complete. 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.779+0000 7f9bbe40b740 20 client.7206 trim_cache size 0 max 16384 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:14:29.786 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 msgr2=0x7f9b9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 0x7f9b9806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000aac0 tx=0x7f9bb0008040 comp rx=0 tx=0).stop 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 msgr2=0x7f9bb415ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400b3e0 tx=0x7f9ba400b8a0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 shutdown_connections 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b9806c8f0 0x7f9b9806edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bb40af800 0x7f9bb414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bb406f690 0x7f9bb415ef00 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 --2- 172.21.15.106:0/2080369490 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9bb406f200 0x7f9bb41404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 >> 172.21.15.106:0/2080369490 conn(0x7f9bb40b2600 msgr2=0x7f9bb4153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 shutdown_connections 2024-04-13T03:14:29.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.780+0000 7f9bbb629640 1 -- 172.21.15.106:0/2080369490 wait complete. 2024-04-13T03:14:29.791 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:14:29.792 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:14:29.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.891+0000 7f3a72fcf640 1 Processor -- start 2024-04-13T03:14:29.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.891+0000 7f3a72fcf640 1 -- start start 2024-04-13T03:14:29.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.891+0000 7f3a72fcf640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a6c06cb10 0x7f3a6c072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a72fcf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c0998d0 con 0x7f3a6c06e150 2024-04-13T03:14:29.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a72fcf640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c073020 con 0x7f3a6c0cdf50 2024-04-13T03:14:29.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a72fcf640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c073300 con 0x7f3a6c06cb10 2024-04-13T03:14:29.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a7156c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3a6c06e150 0x7f3a6c06e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55128/0 (socket says 172.21.15.106:55128) 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a7156c640 1 -- 172.21.15.106:0/2621775990 learned_addr learned my addr 172.21.15.106:0/2621775990 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6bfff640 1 --2- 172.21.15.106:0/2621775990 >> v2:172.21.15.114:3300/0 conn(0x7f3a6c06cb10 0x7f3a6c072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3128046915 0 0) 0x7f3a6c0998d0 con 0x7f3a6c06e150 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a58003520 con 0x7f3a6c06e150 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183099383 0 0) 0x7f3a6c073020 con 0x7f3a6c0cdf50 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a6c0998d0 con 0x7f3a6c0cdf50 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3012249063 0 0) 0x7f3a58003520 con 0x7f3a6c06e150 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2621775990 >> v1:172.21.15.114:6789/0 conn(0x7f3a6c0cdf50 legacy=0x7f3a6c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2621775990 >> v1:172.21.15.106:6789/0 conn(0x7f3a6c06e150 legacy=0x7f3a6c06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.892+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2621775990 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c0735e0 con 0x7f3a6c06cb10 2024-04-13T03:14:29.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.893+0000 7f3a6bfff640 1 --2- 172.21.15.106:0/2621775990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c072720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3a540097a0 tx=0x7f3a5402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e92859d00ba3276c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.893+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54038070 con 0x7f3a6c06cb10 2024-04-13T03:14:29.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.893+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54031500 con 0x7f3a6c06cb10 2024-04-13T03:14:29.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.893+0000 7f3a6b7fe640 1 -- 172.21.15.106:0/2621775990 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c001670 con 0x7f3a6c06cb10 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2621775990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 msgr2=0x7f3a6c072720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2621775990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c072720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3a540097a0 tx=0x7f3a5402b820 comp rx=0 tx=0).stop 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2621775990 shutdown_connections 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2621775990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c072720 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2621775990 >> 172.21.15.106:0/2621775990 conn(0x7f3a6c0b1f10 msgr2=0x7f3a6c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2621775990 shutdown_connections 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.894+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2621775990 wait complete. 2024-04-13T03:14:29.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.895+0000 7f3a72fcf640 1 Processor -- start 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.895+0000 7f3a72fcf640 1 -- start start 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 0x7f3a6c1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c072de0 con 0x7f3a6c06e150 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c072c60 con 0x7f3a6c06cb10 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a72fcf640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c06ed10 con 0x7f3a6c0cdf50 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50088/0 (socket says 172.21.15.106:50088) 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2964249518 learned_addr learned my addr 172.21.15.106:0/2964249518 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:29.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a7156c640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 0x7f3a6c1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a70d6b640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 msgr2=0x7f3a6c1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 0x7f3a6c1f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 msgr2=0x7f3a6c0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 -- 172.21.15.106:0/2964249518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c1e4750 con 0x7f3a6c06e150 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a7156c640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 0x7f3a6c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a70d6b640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a6bfff640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402b860 tx=0x7f3a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.896+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54038050 con 0x7f3a6c06e150 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.897+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54044070 con 0x7f3a6c06e150 2024-04-13T03:14:29.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.897+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c1e4a30 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.897+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003a50 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.897+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c1e4ff0 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.898+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c068a10 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.898+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a54003bf0 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.898+0000 7f3a697fa640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.898+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3a540adbf0 con 0x7f3a6c06e150 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.898+0000 7f3a70d6b640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:29.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.899+0000 7f3a70d6b640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001560 tx=0x7f3a64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:29.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:29.903+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a54030070 con 0x7f3a6c06e150 2024-04-13T03:14:30.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.113+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a6c0008d0 con 0x7f3a6c06e150 2024-04-13T03:14:30.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.113+0000 7f3a697fa640 1 -- 172.21.15.106:0/2964249518 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5190 (secure 0 0 0) 0x7f3a5407bdc0 con 0x7f3a6c06e150 2024-04-13T03:14:30.115 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:14:30.115 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:13:57.763295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:14:30.115 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 43 2024-04-13T03:14:30.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 msgr2=0x7f3a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001560 tx=0x7f3a64009040 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 msgr2=0x7f3a6c1e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402b860 tx=0x7f3a540046c0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 shutdown_connections 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a6c0cdf50 0x7f3a6c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a6c06e150 0x7f3a6c1e5510 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 --2- 172.21.15.106:0/2964249518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a6c06cb10 0x7f3a6c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 >> 172.21.15.106:0/2964249518 conn(0x7f3a6c0b1f10 msgr2=0x7f3a6c067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 shutdown_connections 2024-04-13T03:14:30.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.115+0000 7f3a72fcf640 1 -- 172.21.15.106:0/2964249518 wait complete. 2024-04-13T03:14:30.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-13T03:14:30.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 Processor -- start 2024-04-13T03:14:30.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 -- start start 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a440d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a440cfd20 con 0x7f9a440b3d60 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a44162af0 con 0x7f9a4415ed90 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a4971e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a44162dd0 con 0x7f9a4415e9b0 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a437fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9a440b3d60 0x7f9a440d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55132/0 (socket says 172.21.15.106:55132) 2024-04-13T03:14:30.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.229+0000 7f9a437fe640 1 -- 172.21.15.106:0/3096736392 learned_addr learned my addr 172.21.15.106:0/3096736392 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a440d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942191728 0 0) 0x7f9a440cfd20 con 0x7f9a440b3d60 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a30003520 con 0x7f9a440b3d60 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1738349177 0 0) 0x7f9a30003520 con 0x7f9a440b3d60 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 msgr2=0x7f9a440d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a440d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 >> v1:172.21.15.114:6789/0 conn(0x7f9a4415ed90 legacy=0x7f9a440d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a441630b0 con 0x7f9a440b3d60 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9a3c001a10 con 0x7f9a440b3d60 2024-04-13T03:14:30.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 >> v1:172.21.15.106:6789/0 conn(0x7f9a440b3d60 legacy=0x7f9a440d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a30003f20 0x7f9a30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44162dd0 con 0x7f9a30004d40 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3000ba20 con 0x7f9a30003f20 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3000bc20 con 0x7f9a4415e9b0 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a427fc640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a30003f20 0x7f9a30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 -- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 msgr2=0x7f9a300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a4415e9b0 0x7f9a300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 -- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a30003f20 msgr2=0x7f9a30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a30003f20 0x7f9a30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a437fe640 1 -- 172.21.15.106:0/3096736392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a30003520 con 0x7f9a30004d40 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.230+0000 7f9a427fc640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a30003f20 0x7f9a30004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.231+0000 7f9a437fe640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a300051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9a440cfef0 tx=0x7f9a30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa66ba19d2ef93d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.231+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00a4a0 con 0x7f9a30004d40 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.231+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c034630 con 0x7f9a30004d40 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.231+0000 7f9a41ffb640 1 -- 172.21.15.106:0/3096736392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c034a20 con 0x7f9a30004d40 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 -- 172.21.15.106:0/3096736392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 msgr2=0x7f9a300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 --2- 172.21.15.106:0/3096736392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a300051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9a440cfef0 tx=0x7f9a30005830 comp rx=0 tx=0).stop 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a437fe640 1 -- 172.21.15.106:0/3096736392 reap_dead start 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 -- 172.21.15.106:0/3096736392 shutdown_connections 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 -- 172.21.15.106:0/3096736392 >> 172.21.15.106:0/3096736392 conn(0x7f9a4408bf10 msgr2=0x7f9a440b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 -- 172.21.15.106:0/3096736392 shutdown_connections 2024-04-13T03:14:30.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.232+0000 7f9a4971e640 1 -- 172.21.15.106:0/3096736392 wait complete. 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.233+0000 7f9a4971e640 1 Processor -- start 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.233+0000 7f9a4971e640 1 -- start start 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a440b4140 0x7f9a440b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a440ca7d0 0x7f9a440cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44162800 con 0x7f9a30004d40 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44162680 con 0x7f9a440b4140 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a4971e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a440d4d00 con 0x7f9a440ca7d0 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50110/0 (socket says 172.21.15.106:50110) 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 -- 172.21.15.106:0/349998713 learned_addr learned my addr 172.21.15.106:0/349998713 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a427fc640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a440b4140 0x7f9a440b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a437fe640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a440ca7d0 0x7f9a440cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 -- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a440ca7d0 msgr2=0x7f9a440cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a440ca7d0 0x7f9a440cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 -- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a440b4140 msgr2=0x7f9a440b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a440b4140 0x7f9a440b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 -- 172.21.15.106:0/349998713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440c9a10 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.234+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0119a0 tx=0x7f9a2c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.235+0000 7f9a3b7fe640 1 -- 172.21.15.106:0/349998713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00faf0 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.235+0000 7f9a3b7fe640 1 -- 172.21.15.106:0/349998713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00fc90 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.235+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a440c9cf0 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.235+0000 7f9a3b7fe640 1 -- 172.21.15.106:0/349998713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c014440 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.235+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a440ca2b0 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.236+0000 7f9a3b7fe640 1 -- 172.21.15.106:0/349998713 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a2c012040 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.237+0000 7f9a3b7fe640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.237+0000 7f9a3b7fe640 1 -- 172.21.15.106:0/349998713 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a2c08c4f0 con 0x7f9a30004d40 2024-04-13T03:14:30.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.237+0000 7f9a427fc640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.237+0000 7f9a427fc640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 0x7f9a2406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9a34007b90 tx=0x7f9a34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 Processor -- start 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 -- start start 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601e03e3fe0 con 0x5601e04a96d0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601e0380860 con 0x5601e04a6af0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.238+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5601e03dffa0 con 0x5601e047e800 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50122/0 (socket says 172.21.15.106:50122) 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 -- 172.21.15.106:0/3236148173 learned_addr learned my addr 172.21.15.106:0/3236148173 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a427fc640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a437fe640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 -- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 msgr2=0x5601e04a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 -- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 msgr2=0x5601e04a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 -- 172.21.15.106:0/3236148173 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e03af1a0 con 0x5601e04a96d0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a427fc640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0626f0 tx=0x7f9a2c059c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a799f222b96a5e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a38ff9640 1 -- 172.21.15.106:0/3236148173 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0582b0 con 0x5601e04a96d0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a38ff9640 1 -- 172.21.15.106:0/3236148173 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c056620 con 0x5601e04a96d0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.239+0000 7f9a38ff9640 1 -- 172.21.15.106:0/3236148173 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c05a450 con 0x5601e04a96d0 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- 172.21.15.106:0/3236148173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 msgr2=0x5601e0480560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0626f0 tx=0x7f9a2c059c20 comp rx=0 tx=0).stop 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- 172.21.15.106:0/3236148173 shutdown_connections 2024-04-13T03:14:30.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 --2- 172.21.15.106:0/3236148173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e0480560 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- 172.21.15.106:0/3236148173 >> 172.21.15.106:0/3236148173 conn(0x5601e0412dc0 msgr2=0x5601e04c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- 172.21.15.106:0/3236148173 shutdown_connections 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- 172.21.15.106:0/3236148173 wait complete. 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.240+0000 7f9a4c602740 1 -- start start 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 Processor -- start 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601e0479b00 con 0x5601e04a96d0 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601e037e1d0 con 0x5601e04a6af0 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5601e04c4c90 con 0x5601e047e800 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a4c602740 10 client.0 _ll_register_callbacks cb 0x5601e03a1010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50134/0 (socket says 172.21.15.106:50134) 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.241+0000 7f9a437fe640 1 -- 172.21.15.106:0/1280873698 learned_addr learned my addr 172.21.15.106:0/1280873698 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a427fc640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 -- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 msgr2=0x5601e04d8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04d8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 -- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 msgr2=0x5601e04d4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04d4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 -- 172.21.15.106:0/1280873698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e01c27c0 con 0x5601e04a96d0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a437fe640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c001d10 tx=0x7f9a44162cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878920e3d97a42d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c001810 con 0x5601e04a96d0 2024-04-13T03:14:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a3c0019b0 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2c056620 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9a3c04d050 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a4c602740 10 client.7224 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5601e0222f50 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c056620 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f9a3c050020 con 0x5601e04a96d0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.242+0000 7f9a4c602740 10 client.7224 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a4c602740 10 client.7224 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a4c602740 20 client.7224 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04ab440 0x5601e04e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a4c602740 4 client.7224 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601e03834a0 con 0x5601e04ab440 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04ab440 0x5601e04e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a21ffb640 10 client.7224 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04ab440 0x5601e04e24f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50139db9af62dcdb server_cookie=83821b299785f2e in_seq=0 out_seq=0 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.243+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1280873698 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601e03834a0 con 0x5601e04ab440 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a21ffb640 10 client.7224 handle_command_reply: tid=0 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 client.7224 shutdown 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 msgr2=0x5601e04aa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c001d10 tx=0x7f9a44162cc0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 shutdown_connections 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04ab440 0x5601e04e24f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a96d0 0x5601e04aa810 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a6af0 0x5601e04d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a21ffb640 0 client.7224 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1280873698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 >> 172.21.15.106:0/1280873698 conn(0x5601e0412dc0 msgr2=0x5601e0478e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 shutdown_connections 2024-04-13T03:14:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 1 -- 172.21.15.106:0/1280873698 wait complete. 2024-04-13T03:14:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.244+0000 7f9a4c602740 20 client.7224 trim_cache size 0 max 16384 2024-04-13T03:14:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.257+0000 7f9a4c602740 1 Processor -- start 2024-04-13T03:14:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.257+0000 7f9a4c602740 1 -- start start 2024-04-13T03:14:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e04a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44200c10 con 0x5601e04a6af0 2024-04-13T03:14:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441fffd0 con 0x5601e04a96d0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a442005c0 con 0x5601e047e800 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50146/0 (socket says 172.21.15.106:50146) 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 -- 172.21.15.106:0/52832189 learned_addr learned my addr 172.21.15.106:0/52832189 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a437fe640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e04a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 -- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 msgr2=0x5601e04c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 -- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 msgr2=0x5601e04a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e04a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 -- 172.21.15.106:0/52832189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e00b60b0 con 0x5601e04a6af0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a437fe640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e04a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.258+0000 7f9a427fc640 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001810 tx=0x7f9a3400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab68e8023b14fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a23fff640 1 -- 172.21.15.106:0/52832189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34028030 con 0x5601e04a6af0 2024-04-13T03:14:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a23fff640 1 -- 172.21.15.106:0/52832189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34001c10 con 0x5601e04a6af0 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a23fff640 1 -- 172.21.15.106:0/52832189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34014b00 con 0x5601e04a6af0 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 -- 172.21.15.106:0/52832189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 msgr2=0x5601e0538580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001810 tx=0x7f9a3400c000 comp rx=0 tx=0).stop 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 -- 172.21.15.106:0/52832189 shutdown_connections 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e04a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e0538580 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 --2- 172.21.15.106:0/52832189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e04c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 -- 172.21.15.106:0/52832189 >> 172.21.15.106:0/52832189 conn(0x5601e0412dc0 msgr2=0x5601e04ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 -- 172.21.15.106:0/52832189 shutdown_connections 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.259+0000 7f9a4c602740 1 -- 172.21.15.106:0/52832189 wait complete. 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.260+0000 7f9a4c602740 1 -- start start 2024-04-13T03:14:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.260+0000 7f9a4c602740 1 Processor -- start 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.260+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e0535490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.260+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e0478b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601e04ab2c0 con 0x5601e04a6af0 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601e04ac9a0 con 0x5601e04a96d0 2024-04-13T03:14:30.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5601e049fde0 con 0x5601e047e800 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 10 client.0 _ll_register_callbacks cb 0x7f9a2c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50148/0 (socket says 172.21.15.106:50148) 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 -- 172.21.15.106:0/1413005496 learned_addr learned my addr 172.21.15.106:0/1413005496 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a437fe640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e0478b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e0535490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 -- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 msgr2=0x5601e0535490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e0535490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 -- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 msgr2=0x5601e0478b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e0478b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 -- 172.21.15.106:0/1413005496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e0479190 con 0x5601e04a6af0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a427fc640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x5601e047bd30 tx=0x7f9a34001280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbed40dffb04c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34006cf0 con 0x5601e04a6af0 2024-04-13T03:14:30.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a34007050 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a34001c10 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9a3403d050 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 10 client.7230 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5601e04d8ca0 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.261+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34001c10 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f9a34040020 con 0x5601e04a6af0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 10 client.7230 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 10 client.7230 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 20 client.7230 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04d84c0 0x5601e047ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 4 client.7230 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5601e03f1a00 con 0x5601e04d84c0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04d84c0 0x5601e047ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a21ffb640 10 client.7230 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a42ffd640 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04d84c0 0x5601e047ebe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb778553d9ca5a5c server_cookie=6b63b95792ad1031 in_seq=0 out_seq=0 2024-04-13T03:14:30.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a21ffb640 1 -- 172.21.15.106:0/1413005496 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x5601e03f1a00 con 0x5601e04d84c0 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a21ffb640 10 client.7230 handle_command_reply: tid=0 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 client.7230 shutdown 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 msgr2=0x5601e04785c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x5601e047bd30 tx=0x7f9a34001280 comp rx=0 tx=0).stop 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 shutdown_connections 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x5601e04d84c0 0x5601e047ebe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601e04a96d0 0x5601e0478b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601e04a6af0 0x5601e04785c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a21ffb640 0 client.7230 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 --2- 172.21.15.106:0/1413005496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5601e047e800 0x5601e0535490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.262+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 >> 172.21.15.106:0/1413005496 conn(0x5601e0412dc0 msgr2=0x5601e04a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 shutdown_connections 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4c602740 1 -- 172.21.15.106:0/1413005496 wait complete. 2024-04-13T03:14:30.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4c602740 20 client.7230 trim_cache size 0 max 16384 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "dentry", 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 2161698395, 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627776, 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "frag": "*", 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "dname": "file_to_be_damaged", 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "snap_id": "head", 2024-04-13T03:14:30.270 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-13T03:14:30.271 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:14:30.271 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:14:30.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 msgr2=0x7f9a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 0x7f9a2406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9a34007b90 tx=0x7f9a34009040 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 msgr2=0x7f9a440b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0119a0 tx=0x7f9a2c011e60 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 shutdown_connections 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a440ca7d0 0x7f9a440cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a440b4140 0x7f9a440b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 --2- 172.21.15.106:0/349998713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a30004d40 0x7f9a440b2860 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 >> 172.21.15.106:0/349998713 conn(0x7f9a4408bf10 msgr2=0x7f9a4415dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 shutdown_connections 2024-04-13T03:14:30.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.263+0000 7f9a4971e640 1 -- 172.21.15.106:0/349998713 wait complete. 2024-04-13T03:14:30.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:14:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.373+0000 7f3629b1e640 1 Processor -- start 2024-04-13T03:14:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.373+0000 7f3629b1e640 1 -- start start 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3629b1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f36240cd2e0 0x7f3624160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3629b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362406ff30 0x7f3624070380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3629b1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f362406f490 con 0x7f362415c800 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3629b1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3624161240 con 0x7f36240cd2e0 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3629b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f362415cbe0 con 0x7f362406ff30 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f36240cd2e0 0x7f3624160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f36240cd2e0 0x7f3624160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50152/0 (socket says 172.21.15.106:50152) 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 -- 172.21.15.106:0/748344704 learned_addr learned my addr 172.21.15.106:0/748344704 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3623fff640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362406ff30 0x7f3624070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 -- 172.21.15.106:0/748344704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362406ff30 msgr2=0x7f3624070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362406ff30 0x7f3624070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 -- 172.21.15.106:0/748344704 >> v1:172.21.15.114:6789/0 conn(0x7f362415c800 legacy=0x7f36240ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.374+0000 7f3622ffd640 1 -- 172.21.15.106:0/748344704 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36240d1ef0 con 0x7f36240cd2e0 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.375+0000 7f3622ffd640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f3624160d00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3618009870 tx=0x7f361802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41860bd888d07fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.375+0000 7f36227fc640 1 -- 172.21.15.106:0/748344704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618035020 con 0x7f36240cd2e0 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.375+0000 7f36227fc640 1 -- 172.21.15.106:0/748344704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3618031640 con 0x7f36240cd2e0 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.375+0000 7f36227fc640 1 -- 172.21.15.106:0/748344704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361802d040 con 0x7f36240cd2e0 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 -- 172.21.15.106:0/748344704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 msgr2=0x7f3624160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f3624160d00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3618009870 tx=0x7f361802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 -- 172.21.15.106:0/748344704 shutdown_connections 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362406ff30 0x7f3624070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 --2- 172.21.15.106:0/748344704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f3624160d00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 -- 172.21.15.106:0/748344704 >> 172.21.15.106:0/748344704 conn(0x7f36240d2580 msgr2=0x7f36240d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 -- 172.21.15.106:0/748344704 shutdown_connections 2024-04-13T03:14:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.376+0000 7f3629b1e640 1 -- 172.21.15.106:0/748344704 wait complete. 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.377+0000 7f3629b1e640 1 Processor -- start 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.377+0000 7f3629b1e640 1 -- start start 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 0x7f362406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 0x7f36240ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f362406fa10 con 0x7f36240cd2e0 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f362406f730 con 0x7f362406ff30 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3629b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f36240d0310 con 0x7f362415c800 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50158/0 (socket says 172.21.15.106:50158) 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 -- 172.21.15.106:0/3581264274 learned_addr learned my addr 172.21.15.106:0/3581264274 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3623fff640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 0x7f36240ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f36237fe640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 0x7f362406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 -- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 msgr2=0x7f36240ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 0x7f36240ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 -- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 msgr2=0x7f362406eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 0x7f362406eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 -- 172.21.15.106:0/3581264274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36240ca670 con 0x7f36240cd2e0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3623fff640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 0x7f36240ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f36237fe640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 0x7f362406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f3622ffd640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3618000c00 tx=0x7f36180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.378+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618040070 con 0x7f36240cd2e0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.379+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36180399f0 con 0x7f36240cd2e0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.379+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36240ca950 con 0x7f36240cd2e0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.379+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618035040 con 0x7f36240cd2e0 2024-04-13T03:14:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.379+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36240caf10 con 0x7f36240cd2e0 2024-04-13T03:14:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.380+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3624068a10 con 0x7f36240cd2e0 2024-04-13T03:14:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.380+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f361804b060 con 0x7f36240cd2e0 2024-04-13T03:14:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.381+0000 7f360bfff640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 0x7f361006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.381+0000 7f36237fe640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 0x7f361006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.381+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f36180ad5e0 con 0x7f36240cd2e0 2024-04-13T03:14:30.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.381+0000 7f36237fe640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 0x7f361006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3614001700 tx=0x7f3614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.385+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f361807b7b0 con 0x7f36240cd2e0 2024-04-13T03:14:30.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.598+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3624067d90 con 0x7f36240cd2e0 2024-04-13T03:14:30.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.599+0000 7f360bfff640 1 -- 172.21.15.106:0/3581264274 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f361807b0b0 con 0x7f36240cd2e0 2024-04-13T03:14:30.600 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:14:30.600 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 msgr2=0x7f361006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 0x7f361006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3614001700 tx=0x7f3614009040 comp rx=0 tx=0).stop 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 msgr2=0x7f36240c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3618000c00 tx=0x7f36180046c0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 shutdown_connections 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f361006c8f0 0x7f361006edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f362415c800 0x7f36240ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36240cd2e0 0x7f36240c9af0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 --2- 172.21.15.106:0/3581264274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f362406ff30 0x7f362406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 >> 172.21.15.106:0/3581264274 conn(0x7f36240d2580 msgr2=0x7f36240b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 shutdown_connections 2024-04-13T03:14:30.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.601+0000 7f3629b1e640 1 -- 172.21.15.106:0/3581264274 wait complete. 2024-04-13T03:14:30.612 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:14:30.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:14:30.613 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-13T03:14:30.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.716+0000 7f3d3eca2640 1 Processor -- start 2024-04-13T03:14:30.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.716+0000 7f3d3eca2640 1 -- start start 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3eca2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d2370 0x7f3d380b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3eca2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3d380b3920 0x7f3d3815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3eca2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d3815c870 con 0x7f3d380d3490 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3eca2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3d380d5420 con 0x7f3d380b3920 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3eca2640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3815ee70 con 0x7f3d380d2370 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3d23f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3d380b3920 0x7f3d3815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3ca3e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3d380d3490 0x7f3d380d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55148/0 (socket says 172.21.15.106:55148) 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.717+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1319077787 learned_addr learned my addr 172.21.15.106:0/1319077787 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d3d23f640 1 -- 172.21.15.106:0/1319077787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d2370 msgr2=0x7f3d380b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d37fff640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d2370 0x7f3d380b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d2370 0x7f3d380b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d3d23f640 1 -- 172.21.15.106:0/1319077787 >> v1:172.21.15.106:6789/0 conn(0x7f3d380d3490 legacy=0x7f3d380d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d3d23f640 1 -- 172.21.15.106:0/1319077787 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380d2800 con 0x7f3d380b3920 2024-04-13T03:14:30.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d377fe640 1 -- 172.21.15.106:0/1319077787 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230717045 0 0) 0x7f3d3815c870 con 0x7f3d380d3490 2024-04-13T03:14:30.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.718+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d3815f360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009d90 tx=0x7f3d3002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae3f156ea63a562 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d377fe640 1 -- 172.21.15.106:0/1319077787 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30039070 con 0x7f3d380b3920 2024-04-13T03:14:30.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d377fe640 1 -- 172.21.15.106:0/1319077787 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3002cbc0 con 0x7f3d380b3920 2024-04-13T03:14:30.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d377fe640 1 -- 172.21.15.106:0/1319077787 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30034440 con 0x7f3d380b3920 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 -- 172.21.15.106:0/1319077787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 msgr2=0x7f3d3815f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d3815f360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009d90 tx=0x7f3d3002be10 comp rx=0 tx=0).stop 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 -- 172.21.15.106:0/1319077787 shutdown_connections 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d3815f360 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/1319077787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d2370 0x7f3d380b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 -- 172.21.15.106:0/1319077787 >> 172.21.15.106:0/1319077787 conn(0x7f3d380b9760 msgr2=0x7f3d380b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 -- 172.21.15.106:0/1319077787 shutdown_connections 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.719+0000 7f3d3eca2640 1 -- 172.21.15.106:0/1319077787 wait complete. 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.720+0000 7f3d3eca2640 1 Processor -- start 2024-04-13T03:14:30.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.720+0000 7f3d3eca2640 1 -- start start 2024-04-13T03:14:30.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d3490 0x7f3d381f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d381f5960 0x7f3d3806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380d30a0 con 0x7f3d380b3920 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380d2dc0 con 0x7f3d381f5960 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3eca2640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d380b1930 con 0x7f3d380d3490 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50176/0 (socket says 172.21.15.106:50176) 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/395298944 learned_addr learned my addr 172.21.15.106:0/395298944 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d37fff640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d3490 0x7f3d381f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.721+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d381f5960 0x7f3d3806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d3490 msgr2=0x7f3d381f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d3490 0x7f3d381f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d381f5960 msgr2=0x7f3d3806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d381f5960 0x7f3d3806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/395298944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3806b8e0 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c430 tx=0x7f3d2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d357fa640 1 -- 172.21.15.106:0/395298944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280074e0 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d357fa640 1 -- 172.21.15.106:0/395298944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d280047e0 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d357fa640 1 -- 172.21.15.106:0/395298944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28004bd0 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3806bbc0 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.722+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3806c180 con 0x7f3d380b3920 2024-04-13T03:14:30.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.723+0000 7f3d357fa640 1 -- 172.21.15.106:0/395298944 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d28011070 con 0x7f3d380b3920 2024-04-13T03:14:30.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.724+0000 7f3d357fa640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 0x7f3d2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.724+0000 7f3d357fa640 1 -- 172.21.15.106:0/395298944 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3d2808b770 con 0x7f3d380b3920 2024-04-13T03:14:30.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.724+0000 7f3d37fff640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 0x7f3d2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.724+0000 7f3d37fff640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 0x7f3d2406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.725+0000 7f3d41b63740 1 Processor -- start 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.725+0000 7f3d41b63740 1 -- start start 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.725+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb254528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25452e30 0x55fb25482c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d41b63740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb253bcfa0 con 0x55fb2547fd30 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fb253639c0 con 0x55fb25452e30 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fb253b8f60 con 0x55fb2547f950 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50190/0 (socket says 172.21.15.106:50190) 2024-04-13T03:14:30.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 -- 172.21.15.106:0/2837506210 learned_addr learned my addr 172.21.15.106:0/2837506210 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 -- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 msgr2=0x55fb254528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb254528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 -- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25452e30 msgr2=0x55fb25482c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25452e30 0x55fb25482c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25452e30 0x55fb25482c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 -- 172.21.15.106:0/2837506210 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb25197a50 con 0x55fb2547fd30 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d37fff640 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c004550 tx=0x7f3d2c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d8d6d3549dad2a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d1e7fc640 1 -- 172.21.15.106:0/2837506210 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c019070 con 0x55fb2547fd30 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d1e7fc640 1 -- 172.21.15.106:0/2837506210 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c025070 con 0x55fb2547fd30 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.726+0000 7f3d1e7fc640 1 -- 172.21.15.106:0/2837506210 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c020470 con 0x55fb2547fd30 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- 172.21.15.106:0/2837506210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 msgr2=0x55fb254826e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c004550 tx=0x7f3d2c012000 comp rx=0 tx=0).stop 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- 172.21.15.106:0/2837506210 shutdown_connections 2024-04-13T03:14:30.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25452e30 0x55fb25482c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547fd30 0x55fb254826e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 --2- 172.21.15.106:0/2837506210 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb254528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- 172.21.15.106:0/2837506210 >> 172.21.15.106:0/2837506210 conn(0x55fb25454b60 msgr2=0x55fb2549d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- 172.21.15.106:0/2837506210 shutdown_connections 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- 172.21.15.106:0/2837506210 wait complete. 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.727+0000 7f3d41b63740 1 -- start start 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 Processor -- start 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547fd30 0x55fb254693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25469930 0x55fb25478760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb25457200 con 0x55fb2547f950 2024-04-13T03:14:30.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fb25376d50 con 0x55fb25469930 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.728+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fb25459a40 con 0x55fb2547fd30 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d41b63740 10 client.0 _ll_register_callbacks cb 0x55fb25376740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50196/0 (socket says 172.21.15.106:50196) 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/3657975930 learned_addr learned my addr 172.21.15.106:0/3657975930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d37fff640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547fd30 0x55fb254693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25469930 0x55fb25478760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547fd30 msgr2=0x55fb254693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547fd30 0x55fb254693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25469930 msgr2=0x55fb25478760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25469930 0x55fb25478760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/3657975930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2508f3e0 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3d28089ac0 tx=0x7f3d280594c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95af2e4e6957f346 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28058850 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3d280589f0 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2c025070 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3d28055150 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d41b63740 10 client.7251 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55fb251f81e0 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c025070 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.729+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f3d280552f0 con 0x55fb2547f950 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 10 client.7251 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 10 client.7251 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 20 client.7251 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb25480d00 0x55fb25481150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 4 client.7251 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb253d0f50 con 0x55fb25480d00 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d37fff640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb25480d00 0x55fb25481150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d177fe640 10 client.7251 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d37fff640 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb25480d00 0x55fb25481150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6b247a68a4a362 server_cookie=588b79e277b18465 in_seq=0 out_seq=0 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d177fe640 1 -- 172.21.15.106:0/3657975930 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb253d0f50 con 0x55fb25480d00 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d177fe640 10 client.7251 handle_command_reply: tid=0 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.730+0000 7f3d41b63740 1 client.7251 shutdown 2024-04-13T03:14:30.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 msgr2=0x55fb254669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3d28089ac0 tx=0x7f3d280594c0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 shutdown_connections 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb25480d00 0x55fb25481150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547fd30 0x55fb254693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb25469930 0x55fb25478760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d177fe640 0 client.7251 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 --2- 172.21.15.106:0/3657975930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb2547f950 0x55fb254669b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 >> 172.21.15.106:0/3657975930 conn(0x55fb25452e30 msgr2=0x55fb25456800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 shutdown_connections 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 1 -- 172.21.15.106:0/3657975930 wait complete. 2024-04-13T03:14:30.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.731+0000 7f3d41b63740 20 client.7251 trim_cache size 0 max 16384 2024-04-13T03:14:30.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.748+0000 7f3d41b63740 1 Processor -- start 2024-04-13T03:14:30.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.748+0000 7f3d41b63740 1 -- start start 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb2549f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d382015a0 con 0x55fb25469930 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38200600 con 0x55fb2547fd30 2024-04-13T03:14:30.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3d381f85e0 con 0x55fb2547f950 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50212/0 (socket says 172.21.15.106:50212) 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1413292642 learned_addr learned my addr 172.21.15.106:0/1413292642 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d37fff640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb2549f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.749+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 msgr2=0x55fb2549f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb2549f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 msgr2=0x55fb25458580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1413292642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb250684a0 con 0x55fb25469930 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3d28059390 tx=0x7f3d2805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd005a39f0492589 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d1dffb640 1 -- 172.21.15.106:0/1413292642 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2805d450 con 0x55fb25469930 2024-04-13T03:14:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d1dffb640 1 -- 172.21.15.106:0/1413292642 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2805d170 con 0x55fb25469930 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d1dffb640 1 -- 172.21.15.106:0/1413292642 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28058850 con 0x55fb25469930 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 -- 172.21.15.106:0/1413292642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 msgr2=0x55fb2549d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3d28059390 tx=0x7f3d2805b000 comp rx=0 tx=0).stop 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 -- 172.21.15.106:0/1413292642 shutdown_connections 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb2549f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1413292642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb2549d0c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 -- 172.21.15.106:0/1413292642 >> 172.21.15.106:0/1413292642 conn(0x55fb25452e30 msgr2=0x55fb2549e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 -- 172.21.15.106:0/1413292642 shutdown_connections 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.750+0000 7f3d41b63740 1 -- 172.21.15.106:0/1413292642 wait complete. 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.751+0000 7f3d41b63740 1 -- start start 2024-04-13T03:14:30.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.751+0000 7f3d41b63740 1 Processor -- start 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb25458580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d381fffb0 con 0x55fb25469930 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fb25480930 con 0x55fb2547fd30 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55fb255187f0 con 0x55fb2547f950 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d41b63740 10 client.0 _ll_register_callbacks cb 0x7f3d28059010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50228/0 (socket says 172.21.15.106:50228) 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1723615918 learned_addr learned my addr 172.21.15.106:0/1723615918 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3d23f640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d37fff640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb25458580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 msgr2=0x55fb25458580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb25458580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 msgr2=0x55fb25458ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 -- 172.21.15.106:0/1723615918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb25479290 con 0x55fb25469930 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d3ca3e640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3d28007fd0 tx=0x7f3d28000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237d355e54812edf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.752+0000 7f3d177fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28058a50 con 0x55fb25469930 2024-04-13T03:14:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3d280582c0 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2805d170 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3d28004e30 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 10 client.7257 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55fb25479290 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2805d170 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f3d280584c0 con 0x55fb25469930 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 10 client.7257 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 10 client.7257 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 20 client.7257 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb2551a510 0x55fb254856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 4 client.7257 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55fb253ca9c0 con 0x55fb2551a510 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d37fff640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb2551a510 0x55fb254856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d177fe640 10 client.7257 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.753+0000 7f3d37fff640 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb2551a510 0x55fb254856f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabc2d35858ae2d2 server_cookie=d95f030f740b4070 in_seq=0 out_seq=0 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d177fe640 1 -- 172.21.15.106:0/1723615918 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb253ca9c0 con 0x55fb2551a510 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d177fe640 10 client.7257 handle_command_reply: tid=0 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 client.7257 shutdown 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 msgr2=0x55fb254c0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3d28007fd0 tx=0x7f3d28000fb0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 shutdown_connections 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55fb2551a510 0x55fb254856f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55fb2547f950 0x55fb25458580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2547fd30 0x55fb25458ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 --2- 172.21.15.106:0/1723615918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb25469930 0x55fb254c0190 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d177fe640 0 client.7257 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 >> 172.21.15.106:0/1723615918 conn(0x55fb25452e30 msgr2=0x55fb2549e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 shutdown_connections 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 1 -- 172.21.15.106:0/1723615918 wait complete. 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.754+0000 7f3d41b63740 20 client.7257 trim_cache size 0 max 16384 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "d1a573c8-de1b-4f16-9351-35d75623f92f", 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:14:30.763 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:14:30.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 msgr2=0x7f3d2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 0x7f3d2406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).stop 2024-04-13T03:14:30.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 msgr2=0x7f3d38156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c430 tx=0x7f3d2800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 shutdown_connections 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d2406c920 0x7f3d2406ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3d380d3490 0x7f3d381f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d381f5960 0x7f3d3806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 --2- 172.21.15.106:0/395298944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d380b3920 0x7f3d38156df0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 >> 172.21.15.106:0/395298944 conn(0x7f3d380b9760 msgr2=0x7f3d380b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 shutdown_connections 2024-04-13T03:14:30.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.755+0000 7f3d3eca2640 1 -- 172.21.15.106:0/395298944 wait complete. 2024-04-13T03:14:30.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:14:30.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-13T03:14:30.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.869+0000 7fa81ca88640 1 Processor -- start 2024-04-13T03:14:30.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.869+0000 7fa81ca88640 1 -- start start 2024-04-13T03:14:30.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa81815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8180b54a0 0x7fa8180704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa8180709e0 0x7fa818162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa8180b5cf0 con 0x7fa8180b54a0 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa8180b5fd0 con 0x7fa8180709e0 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81ca88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa8180b62b0 con 0x7fa8180b1c30 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8180b54a0 0x7fa8180704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8180b54a0 0x7fa8180704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50236/0 (socket says 172.21.15.106:50236) 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 -- 172.21.15.106:0/368026313 learned_addr learned my addr 172.21.15.106:0/368026313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa816d9d640 1 --2- 172.21.15.106:0/368026313 >> v2:172.21.15.114:3300/0 conn(0x7fa8180709e0 0x7fa818162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa81659c640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa81815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 -- 172.21.15.106:0/368026313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 msgr2=0x7fa81815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa81815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 -- 172.21.15.106:0/368026313 >> v2:172.21.15.114:3300/0 conn(0x7fa8180709e0 msgr2=0x7fa818162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 --2- 172.21.15.106:0/368026313 >> v2:172.21.15.114:3300/0 conn(0x7fa8180709e0 0x7fa818162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.870+0000 7fa80ffff640 1 -- 172.21.15.106:0/368026313 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8180b6590 con 0x7fa8180b54a0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa80ffff640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180b54a0 0x7fa8180704a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa800009870 tx=0x7fa80002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cfbb9cfd115d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa815d9b640 1 -- 172.21.15.106:0/368026313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800035020 con 0x7fa8180b54a0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa815d9b640 1 -- 172.21.15.106:0/368026313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800031640 con 0x7fa8180b54a0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa815d9b640 1 -- 172.21.15.106:0/368026313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80002d040 con 0x7fa8180b54a0 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa81ca88640 1 -- 172.21.15.106:0/368026313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180b54a0 msgr2=0x7fa8180704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa81ca88640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180b54a0 0x7fa8180704a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa800009870 tx=0x7fa80002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.871+0000 7fa81ca88640 1 -- 172.21.15.106:0/368026313 shutdown_connections 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 --2- 172.21.15.106:0/368026313 >> v2:172.21.15.114:3300/0 conn(0x7fa8180709e0 0x7fa818162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180b54a0 0x7fa8180704a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 --2- 172.21.15.106:0/368026313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa81815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 -- 172.21.15.106:0/368026313 >> 172.21.15.106:0/368026313 conn(0x7fa818071870 msgr2=0x7fa818071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 -- 172.21.15.106:0/368026313 shutdown_connections 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.872+0000 7fa81ca88640 1 -- 172.21.15.106:0/368026313 wait complete. 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.873+0000 7fa81ca88640 1 Processor -- start 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.873+0000 7fa81ca88640 1 -- start start 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.873+0000 7fa81ca88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa818148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8180b54a0 0x7fa8180cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa818162af0 con 0x7fa8180709e0 2024-04-13T03:14:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa818162970 con 0x7fa8180b54a0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa818162c70 con 0x7fa8180b1c30 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50238/0 (socket says 172.21.15.106:50238) 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 -- 172.21.15.106:0/2742300569 learned_addr learned my addr 172.21.15.106:0/2742300569 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa816d9d640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8180b54a0 0x7fa8180cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa80ffff640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa818148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 -- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 msgr2=0x7fa818148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa818148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 -- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8180b54a0 msgr2=0x7fa8180cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8180b54a0 0x7fa8180cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 -- 172.21.15.106:0/2742300569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8180cccc0 con 0x7fa8180709e0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa80ffff640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa818148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81659c640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9b0 tx=0x7fa80800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa80effd640 1 -- 172.21.15.106:0/2742300569 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808017070 con 0x7fa8180709e0 2024-04-13T03:14:30.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa80effd640 1 -- 172.21.15.106:0/2742300569 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80800c9a0 con 0x7fa8180709e0 2024-04-13T03:14:30.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8180ccfa0 con 0x7fa8180709e0 2024-04-13T03:14:30.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa80effd640 1 -- 172.21.15.106:0/2742300569 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800cd90 con 0x7fa8180709e0 2024-04-13T03:14:30.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.874+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8180cd560 con 0x7fa8180709e0 2024-04-13T03:14:30.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.876+0000 7fa80effd640 1 -- 172.21.15.106:0/2742300569 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa80801a070 con 0x7fa8180709e0 2024-04-13T03:14:30.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.876+0000 7fa80effd640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.876+0000 7fa80effd640 1 -- 172.21.15.106:0/2742300569 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fa80808c050 con 0x7fa8180709e0 2024-04-13T03:14:30.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.876+0000 7fa80ffff640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.877+0000 7fa80ffff640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 0x7fa7f806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa800036000 tx=0x7fa80003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.877+0000 7fa81f874740 1 Processor -- start 2024-04-13T03:14:30.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.877+0000 7fa81f874740 1 -- start start 2024-04-13T03:14:30.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 0x563019a2df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a2e4d0 0x563019a5e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563019999170 con 0x563019a5b3d0 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56301993f7f0 con 0x563019a5aff0 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563019995130 con 0x563019a2e4d0 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50242/0 (socket says 172.21.15.106:50242) 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 -- 172.21.15.106:0/3826834571 learned_addr learned my addr 172.21.15.106:0/3826834571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa816d9d640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a2e4d0 0x563019a5e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81659c640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 0x563019a2df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 -- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a2e4d0 msgr2=0x563019a5e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a2e4d0 0x563019a5e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 -- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 msgr2=0x563019a2df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 0x563019a2df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 -- 172.21.15.106:0/3826834571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630197757d0 con 0x563019a5b3d0 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa81659c640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 0x563019a2df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.878+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa8000090b0 tx=0x7fa800030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90c69bc57b4d0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa7f7fff640 1 -- 172.21.15.106:0/3826834571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80004a070 con 0x563019a5b3d0 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa7f7fff640 1 -- 172.21.15.106:0/3826834571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80003a4e0 con 0x563019a5b3d0 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa7f7fff640 1 -- 172.21.15.106:0/3826834571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800035040 con 0x563019a5b3d0 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 -- 172.21.15.106:0/3826834571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 msgr2=0x563019a5dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa8000090b0 tx=0x7fa800030000 comp rx=0 tx=0).stop 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 -- 172.21.15.106:0/3826834571 shutdown_connections 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a2e4d0 0x563019a5e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5b3d0 0x563019a5dd80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 --2- 172.21.15.106:0/3826834571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5aff0 0x563019a2df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 -- 172.21.15.106:0/3826834571 >> 172.21.15.106:0/3826834571 conn(0x563019a30200 msgr2=0x563019a78d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 -- 172.21.15.106:0/3826834571 shutdown_connections 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.879+0000 7fa81f874740 1 -- 172.21.15.106:0/3826834571 wait complete. 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.880+0000 7fa81f874740 1 -- start start 2024-04-13T03:14:30.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.880+0000 7fa81f874740 1 Processor -- start 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019a78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563019a328a0 con 0x563019a5aff0 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563019952d00 con 0x563019a5b3d0 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563019a350e0 con 0x563019a5f170 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81f874740 10 client.0 _ll_register_callbacks cb 0x5630199526f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50258/0 (socket says 172.21.15.106:50258) 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 -- 172.21.15.106:0/3593742859 learned_addr learned my addr 172.21.15.106:0/3593742859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa816d9d640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019a78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 -- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 msgr2=0x563019a78ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019a78ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 -- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 msgr2=0x563019a4e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a4e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.881+0000 7fa81659c640 1 -- 172.21.15.106:0/3593742859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630196ab3f0 con 0x563019a5aff0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81659c640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa808059950 tx=0x7fa808059d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478911e5d1c8ecd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808072020 con 0x563019a5aff0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fa808060150 con 0x563019a5aff0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80003a4e0 con 0x563019a5aff0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa808021040 con 0x563019a5aff0 2024-04-13T03:14:30.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 10 client.7269 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x56301966b0f0 con 0x563019a5aff0 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80003a4e0 con 0x563019a5aff0 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fa8080559a0 con 0x563019a5aff0 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 10 client.7269 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 10 client.7269 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 20 client.7269 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a8c970 0x563019a8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 4 client.7269 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630199a6b90 con 0x563019a8c970 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a8c970 0x563019a8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa80ffff640 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a8c970 0x563019a8ce00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c82b60f0119132 server_cookie=9f82100aa9d0b357 in_seq=0 out_seq=0 2024-04-13T03:14:30.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.882+0000 7fa7f57fa640 10 client.7269 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa7f57fa640 1 -- 172.21.15.106:0/3593742859 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630199a6b90 con 0x563019a8c970 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa7f57fa640 10 client.7269 handle_command_reply: tid=0 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 client.7269 shutdown 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 msgr2=0x563019a60090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa808059950 tx=0x7fa808059d90 comp rx=0 tx=0).stop 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 shutdown_connections 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a8c970 0x563019a8ce00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019a78ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 --2- 172.21.15.106:0/3593742859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a60090 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa7f57fa640 0 client.7269 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 >> 172.21.15.106:0/3593742859 conn(0x563019a2e4d0 msgr2=0x563019a31ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 shutdown_connections 2024-04-13T03:14:30.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 1 -- 172.21.15.106:0/3593742859 wait complete. 2024-04-13T03:14:30.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.883+0000 7fa81f874740 20 client.7269 trim_cache size 0 max 16384 2024-04-13T03:14:30.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.901+0000 7fa81f874740 1 Processor -- start 2024-04-13T03:14:30.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.901+0000 7fa81f874740 1 -- start start 2024-04-13T03:14:30.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 0x563019a78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5f170 0x563019ae91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa818201770 con 0x563019a5aff0 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa818200790 con 0x563019a5f170 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa8180cc320 con 0x563019a5b3d0 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50264/0 (socket says 172.21.15.106:50264) 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 -- 172.21.15.106:0/1275559974 learned_addr learned my addr 172.21.15.106:0/1275559974 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 0x563019a78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 -- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 msgr2=0x563019a78d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 0x563019a78d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 -- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5f170 msgr2=0x563019ae91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5f170 0x563019ae91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 -- 172.21.15.106:0/1275559974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563019a28b20 con 0x563019a5aff0 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 0x563019a78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:14:30.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa81659c640 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8080565d0 tx=0x7fa80801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96bf073b54156dc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa7f77fe640 1 -- 172.21.15.106:0/1275559974 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808077070 con 0x563019a5aff0 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa7f77fe640 1 -- 172.21.15.106:0/1275559974 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa808058010 con 0x563019a5aff0 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.902+0000 7fa7f77fe640 1 -- 172.21.15.106:0/1275559974 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808072040 con 0x563019a5aff0 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- 172.21.15.106:0/1275559974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 msgr2=0x563019a297e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8080565d0 tx=0x7fa80801e000 comp rx=0 tx=0).stop 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- 172.21.15.106:0/1275559974 shutdown_connections 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5b3d0 0x563019a78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5f170 0x563019ae91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 --2- 172.21.15.106:0/1275559974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a297e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- 172.21.15.106:0/1275559974 >> 172.21.15.106:0/1275559974 conn(0x563019a2e4d0 msgr2=0x563019a4e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- 172.21.15.106:0/1275559974 shutdown_connections 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- 172.21.15.106:0/1275559974 wait complete. 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 -- start start 2024-04-13T03:14:30.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.903+0000 7fa81f874740 1 Processor -- start 2024-04-13T03:14:30.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019af2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563019a792b0 con 0x563019a5aff0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa818200140 con 0x563019a5b3d0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563019af3670 con 0x563019a5f170 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81f874740 10 client.0 _ll_register_callbacks cb 0x7fa8080595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50278/0 (socket says 172.21.15.106:50278) 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa81659c640 1 -- 172.21.15.106:0/1336051729 learned_addr learned my addr 172.21.15.106:0/1336051729 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa816d9d640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019af2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.904+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 -- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 msgr2=0x563019af2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019af2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 -- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 msgr2=0x563019a761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:14:30.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 -- 172.21.15.106:0/1336051729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563019a5fb10 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81659c640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa80800cfd0 tx=0x7fa80808a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff4f972f88f263 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80807e060 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fa808058b10 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630199a0a00 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa808077040 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 10 client.7275 fetch_fsmap learned FSMap version 43 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x563019ad8eb0 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5630199a0a00 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fa808058d10 con 0x563019a5aff0 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 10 client.7275 fetch_fsmap finished waiting for FSMap version 43 2024-04-13T03:14:30.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 10 client.7275 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 20 client.7275 populate_metadata read hostname 'smithi106' 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a9ba70 0x563019af2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 4 client.7275 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56301991c530 con 0x563019a9ba70 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.905+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a9ba70 0x563019af2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa7f57fa640 10 client.7275 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa80ffff640 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a9ba70 0x563019af2d60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176dd383f93b62 server_cookie=3aed49acf54b92c0 in_seq=0 out_seq=0 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa7f57fa640 1 -- 172.21.15.106:0/1336051729 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56301991c530 con 0x563019a9ba70 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa7f57fa640 10 client.7275 handle_command_reply: tid=0 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 client.7275 shutdown 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 msgr2=0x563019a75ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa80800cfd0 tx=0x7fa80808a270 comp rx=0 tx=0).stop 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 shutdown_connections 2024-04-13T03:14:30.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x563019a9ba70 0x563019af2d60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563019a5f170 0x563019af2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563019a5b3d0 0x563019a761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa7f57fa640 0 client.7275 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 --2- 172.21.15.106:0/1336051729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563019a5aff0 0x563019a75ca0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 >> 172.21.15.106:0/1336051729 conn(0x563019a2e4d0 msgr2=0x563019a78930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 shutdown_connections 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 1 -- 172.21.15.106:0/1336051729 wait complete. 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.906+0000 7fa81f874740 20 client.7275 trim_cache size 0 max 16384 2024-04-13T03:14:30.914 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "d1a573c8-de1b-4f16-9351-35d75623f92f": { 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/subdir", 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "d1a573c8-de1b-4f16-9351-35d75623f92f", 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:14:30.915 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:14:30.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 msgr2=0x7fa7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 0x7fa7f806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa800036000 tx=0x7fa80003c040 comp rx=0 tx=0).stop 2024-04-13T03:14:30.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 msgr2=0x7fa8180d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9b0 tx=0x7fa80800ee70 comp rx=0 tx=0).stop 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 shutdown_connections 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa8180b1c30 0x7fa818148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8180b54a0 0x7fa8180cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 --2- 172.21.15.106:0/2742300569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8180709e0 0x7fa8180d0a40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:14:30.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 >> 172.21.15.106:0/2742300569 conn(0x7fa818071870 msgr2=0x7fa8180b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:14:30.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 shutdown_connections 2024-04-13T03:14:30.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:14:30.907+0000 7fa81ca88640 1 -- 172.21.15.106:0/2742300569 wait complete. 2024-04-13T03:14:30.920 INFO:tasks.cephfs.filesystem:scrub status for tag:d1a573c8-de1b-4f16-9351-35d75623f92f - {'path': '/subdir', 'tag': 'd1a573c8-de1b-4f16-9351-35d75623f92f', 'options': 'recursive,repair'} 2024-04-13T03:14:55.490 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:14:55.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:15:00.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:00.921 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-13T03:15:01.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.033+0000 7f0be5115640 1 Processor -- start 2024-04-13T03:15:01.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.033+0000 7f0be5115640 1 -- start start 2024-04-13T03:15:01.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0be5115640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0be00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0be5115640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0be015eca0 0x7f0be0072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0be5115640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be00b5060 con 0x7f0be0073350 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0be5115640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0be00b5340 con 0x7f0be015eca0 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0be5115640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0be006c500 con 0x7f0be015e8c0 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bdf59e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0be0073350 0x7f0be00b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37862/0 (socket says 172.21.15.106:37862) 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bdf59e640 1 -- 172.21.15.106:0/3170826887 learned_addr learned my addr 172.21.15.106:0/3170826887 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593136009 0 0) 0x7f0be00b5060 con 0x7f0be0073350 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bded9d640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0be00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/3170826887 >> v2:172.21.15.114:3300/0 conn(0x7f0be015eca0 0x7f0be0072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bcc003520 con 0x7f0be0073350 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 150824876 0 0) 0x7f0bcc003520 con 0x7f0be0073350 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 msgr2=0x7f0be00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0be00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 >> v2:172.21.15.114:3300/0 conn(0x7f0be015eca0 msgr2=0x7f0be0072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 --2- 172.21.15.106:0/3170826887 >> v2:172.21.15.114:3300/0 conn(0x7f0be015eca0 0x7f0be0072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be006c7e0 con 0x7f0be0073350 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.034+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f0bd80019f0 con 0x7f0be0073350 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 >> v1:172.21.15.106:6789/0 conn(0x7f0be0073350 legacy=0x7f0be00b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0bcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b5340 con 0x7f0bcc0039b0 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0be006c500 con 0x7f0bcc004800 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc00b9c0 con 0x7f0be015e8c0 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0bcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 -- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 msgr2=0x7f0bcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0bcc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 -- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bcc004800 msgr2=0x7f0bcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bded9d640 1 -- 172.21.15.106:0/3170826887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc003520 con 0x7f0bcc0039b0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.035+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0be015e8c0 0x7f0bcc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0bded9d640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009ac0 tx=0x7f0bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be25dbd65687620 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0039070 con 0x7f0bcc0039b0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd002c6b0 con 0x7f0bcc0039b0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0bde59c640 1 -- 172.21.15.106:0/3170826887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0009690 con 0x7f0bcc0039b0 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 -- 172.21.15.106:0/3170826887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bcc0039b0 msgr2=0x7f0bcc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 --2- 172.21.15.106:0/3170826887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009ac0 tx=0x7f0bcc005830 comp rx=0 tx=0).stop 2024-04-13T03:15:01.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0bdf59e640 1 -- 172.21.15.106:0/3170826887 reap_dead start 2024-04-13T03:15:01.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 -- 172.21.15.106:0/3170826887 shutdown_connections 2024-04-13T03:15:01.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 -- 172.21.15.106:0/3170826887 >> 172.21.15.106:0/3170826887 conn(0x7f0be006f0d0 msgr2=0x7f0be006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 -- 172.21.15.106:0/3170826887 shutdown_connections 2024-04-13T03:15:01.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.036+0000 7f0be5115640 1 -- 172.21.15.106:0/3170826887 wait complete. 2024-04-13T03:15:01.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.038+0000 7f0be5115640 1 Processor -- start 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.038+0000 7f0be5115640 1 -- start start 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.038+0000 7f0be5115640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 0x7f0be00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.038+0000 7f0be5115640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0be00cfeb0 0x7f0be014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.038+0000 7f0be5115640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0be5115640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00ce250 con 0x7f0be014c440 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0be5115640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b4d50 con 0x7f0be00cfeb0 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0be5115640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00ce3d0 con 0x7f0bcc0039b0 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40936/0 (socket says 172.21.15.106:40936) 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 -- 172.21.15.106:0/520183650 learned_addr learned my addr 172.21.15.106:0/520183650 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0be00cfeb0 0x7f0be014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bded9d640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 0x7f0be00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 -- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 msgr2=0x7f0be00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 0x7f0be00cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 -- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0be00cfeb0 msgr2=0x7f0be014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0be00cfeb0 0x7f0be014bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 -- 172.21.15.106:0/520183650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be0149dc0 con 0x7f0be014c440 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bded9d640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 0x7f0be00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0be0073da0 tx=0x7f0bd800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bd77fe640 1 -- 172.21.15.106:0/520183650 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800eda0 con 0x7f0be014c440 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bd77fe640 1 -- 172.21.15.106:0/520183650 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd8014480 con 0x7f0be014c440 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be014a0a0 con 0x7f0be014c440 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0bd77fe640 1 -- 172.21.15.106:0/520183650 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8014830 con 0x7f0be014c440 2024-04-13T03:15:01.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.039+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be014a660 con 0x7f0be014c440 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.041+0000 7f0bd77fe640 1 -- 172.21.15.106:0/520183650 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bd80149d0 con 0x7f0be014c440 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.042+0000 7f0bd77fe640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 0x7f0bc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.042+0000 7f0bd77fe640 1 -- 172.21.15.106:0/520183650 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f0bd808bbb0 con 0x7f0be014c440 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.042+0000 7f0bded9d640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 0x7f0bc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.042+0000 7f0bded9d640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 0x7f0bc006edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0036000 tx=0x7f0bd0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.042+0000 7f0be7fd8740 1 Processor -- start 2024-04-13T03:15:01.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 -- start start 2024-04-13T03:15:01.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bd3d0 0x556a209bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209904d0 0x556a209c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556a208fb170 con 0x556a209bcff0 2024-04-13T03:15:01.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556a208a17f0 con 0x556a209bd3d0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.043+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556a208f7130 con 0x556a209904d0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40952/0 (socket says 172.21.15.106:40952) 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 -- 172.21.15.106:0/724734844 learned_addr learned my addr 172.21.15.106:0/724734844 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bd3d0 0x556a209bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209904d0 0x556a209c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 -- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209904d0 msgr2=0x556a209c02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209904d0 0x556a209c02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 -- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bd3d0 msgr2=0x556a209bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bd3d0 0x556a209bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 -- 172.21.15.106:0/724734844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a206d77d0 con 0x556a209bcff0 2024-04-13T03:15:01.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bded9d640 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009bb0 tx=0x7f0bd002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92c09f9900b203 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bd4ff9640 1 -- 172.21.15.106:0/724734844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0039050 con 0x556a209bcff0 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bd4ff9640 1 -- 172.21.15.106:0/724734844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd003a040 con 0x556a209bcff0 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.044+0000 7f0bd4ff9640 1 -- 172.21.15.106:0/724734844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0005c70 con 0x556a209bcff0 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 -- 172.21.15.106:0/724734844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 msgr2=0x556a2098ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009bb0 tx=0x7f0bd002e000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 -- 172.21.15.106:0/724734844 shutdown_connections 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209904d0 0x556a209c02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bd3d0 0x556a209bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/724734844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bcff0 0x556a2098ff90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 -- 172.21.15.106:0/724734844 >> 172.21.15.106:0/724734844 conn(0x556a20992200 msgr2=0x556a209dad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 -- 172.21.15.106:0/724734844 shutdown_connections 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.045+0000 7f0be7fd8740 1 -- 172.21.15.106:0/724734844 wait complete. 2024-04-13T03:15:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 -- start start 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 Processor -- start 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209d82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a209c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556a209948a0 con 0x556a209bd3d0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556a208b4d00 con 0x556a209bcff0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.046+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556a209970e0 con 0x556a209c3390 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0be7fd8740 10 client.0 _ll_register_callbacks cb 0x556a208b46f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40968/0 (socket says 172.21.15.106:40968) 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 -- 172.21.15.106:0/4284076559 learned_addr learned my addr 172.21.15.106:0/4284076559 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a209c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bded9d640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209d82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 -- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 msgr2=0x556a209c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a209c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 -- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 msgr2=0x556a209d82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209d82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 -- 172.21.15.106:0/4284076559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2060d3f0 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009da0 tx=0x7f0bc80167a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3fd283a744c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017640 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f0bc8021cc0 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd003a040 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0bc8016e50 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0be7fd8740 10 client.7287 fetch_fsmap learned FSMap version 44 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x556a205cd0f0 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.047+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd003a040 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f0bc8028050 con 0x556a209bd3d0 2024-04-13T03:15:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 10 client.7287 fetch_fsmap finished waiting for FSMap version 44 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 10 client.7287 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 20 client.7287 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209daca0 0x556a209db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 4 client.7287 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a20908b90 con 0x556a209daca0 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0bded9d640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209daca0 0x556a209db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0bded9d640 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209daca0 0x556a209db0f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5200cd511d4d2720 server_cookie=fdddbeca5abe9c93 in_seq=0 out_seq=0 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.048+0000 7f0bbdffb640 10 client.7287 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0bbdffb640 1 -- 172.21.15.106:0/4284076559 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a20908b90 con 0x556a209daca0 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0bbdffb640 10 client.7287 handle_command_reply: tid=0 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 client.7287 shutdown 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 msgr2=0x556a209bf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009da0 tx=0x7f0bc80167a0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 shutdown_connections 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209daca0 0x556a209db0f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a209c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0bbdffb640 0 client.7287 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209bf450 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/4284076559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209d82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 >> 172.21.15.106:0/4284076559 conn(0x556a209904d0 msgr2=0x556a20993f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 shutdown_connections 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 1 -- 172.21.15.106:0/4284076559 wait complete. 2024-04-13T03:15:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.049+0000 7f0be7fd8740 20 client.7287 trim_cache size 0 max 16384 2024-04-13T03:15:01.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 Processor -- start 2024-04-13T03:15:01.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 -- start start 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a20993af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0201b20 con 0x556a209bd3d0 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0200b80 con 0x556a209bcff0 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0be014bb20 con 0x556a209c3390 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40978/0 (socket says 172.21.15.106:40978) 2024-04-13T03:15:01.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.067+0000 7f0bd7fff640 1 -- 172.21.15.106:0/2871839424 learned_addr learned my addr 172.21.15.106:0/2871839424 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a20993af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 -- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 msgr2=0x556a20993af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a20993af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 -- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 msgr2=0x556a209be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 -- 172.21.15.106:0/2871839424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a209c0300 con 0x556a209bd3d0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009190 tx=0x7f0bc8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f4212151882be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bbffff640 1 -- 172.21.15.106:0/2871839424 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017d10 con 0x556a209bd3d0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bbffff640 1 -- 172.21.15.106:0/2871839424 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc8004030 con 0x556a209bd3d0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0bbffff640 1 -- 172.21.15.106:0/2871839424 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80043e0 con 0x556a209bd3d0 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 -- 172.21.15.106:0/2871839424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 msgr2=0x556a209c1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009190 tx=0x7f0bc8019000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 -- 172.21.15.106:0/2871839424 shutdown_connections 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a20993af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a209c1f70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/2871839424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a209be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.068+0000 7f0be7fd8740 1 -- 172.21.15.106:0/2871839424 >> 172.21.15.106:0/2871839424 conn(0x556a209904d0 msgr2=0x556a209ea4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.069+0000 7f0be7fd8740 1 -- 172.21.15.106:0/2871839424 shutdown_connections 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.069+0000 7f0be7fd8740 1 -- 172.21.15.106:0/2871839424 wait complete. 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.069+0000 7f0be7fd8740 1 -- start start 2024-04-13T03:15:01.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.069+0000 7f0be7fd8740 1 Processor -- start 2024-04-13T03:15:01.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a20996ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a2098f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0200530 con 0x556a209bd3d0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556a209db990 con 0x556a209bcff0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556a209bb450 con 0x556a209c3390 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 10 client.0 _ll_register_callbacks cb 0x7f0bc8009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40992/0 (socket says 172.21.15.106:40992) 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 -- 172.21.15.106:0/518611624 learned_addr learned my addr 172.21.15.106:0/518611624 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bded9d640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a20996ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a2098f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 -- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 msgr2=0x556a2098f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a2098f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 -- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 msgr2=0x556a20996ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a20996ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 -- 172.21.15.106:0/518611624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a209baf80 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bdf59e640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a2098f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bded9d640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a20996ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bd7fff640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80095e0 tx=0x7f0bc8019070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b795d450bdc339 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8039070 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f0bc8017440 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.070+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc8004030 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0bc8047050 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 10 client.7293 fetch_fsmap learned FSMap version 44 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x556a209deae0 con 0x556a209bd3d0 2024-04-13T03:15:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc8004030 con 0x556a209bd3d0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f0bc804a020 con 0x556a209bd3d0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 10 client.7293 fetch_fsmap finished waiting for FSMap version 44 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 10 client.7293 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 20 client.7293 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209feae0 0x556a2098f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 4 client.7293 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a20902a00 con 0x556a209feae0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bded9d640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209feae0 0x556a2098f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 10 client.7293 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bded9d640 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209feae0 0x556a2098f600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc1e013fbb7cf90 server_cookie=93ad173fc2f0db86 in_seq=0 out_seq=0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 1 -- 172.21.15.106:0/518611624 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a20902a00 con 0x556a209feae0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 10 client.7293 handle_command_reply: tid=0 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 client.7293 shutdown 2024-04-13T03:15:01.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 msgr2=0x556a20997010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80095e0 tx=0x7f0bc8019070 comp rx=0 tx=0).stop 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 shutdown_connections 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x556a209feae0 0x556a2098f600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556a209c3390 0x556a2098f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a209bd3d0 0x556a20997010 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0bbdffb640 0 client.7293 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 --2- 172.21.15.106:0/518611624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a209bcff0 0x556a20996ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.071+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 >> 172.21.15.106:0/518611624 conn(0x556a209904d0 msgr2=0x556a209d8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 shutdown_connections 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be7fd8740 1 -- 172.21.15.106:0/518611624 wait complete. 2024-04-13T03:15:01.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be7fd8740 20 client.7293 trim_cache size 0 max 16384 2024-04-13T03:15:01.081 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:01.081 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:15:01.081 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:15:01.081 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 msgr2=0x7f0bc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 0x7f0bc006edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0036000 tx=0x7f0bd0030000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 msgr2=0x7f0be015b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f0be0073da0 tx=0x7f0bd800e530 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 shutdown_connections 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0bc006c8f0 0x7f0bc006edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0be014c440 0x7f0be015b780 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0be00cfeb0 0x7f0be014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 --2- 172.21.15.106:0/520183650 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0bcc0039b0 0x7f0be00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 >> 172.21.15.106:0/520183650 conn(0x7f0be006f0d0 msgr2=0x7f0be00cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 shutdown_connections 2024-04-13T03:15:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.072+0000 7f0be5115640 1 -- 172.21.15.106:0/520183650 wait complete. 2024-04-13T03:15:01.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:15:01.085 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:01.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.183+0000 7f79bb236640 1 Processor -- start 2024-04-13T03:15:01.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.183+0000 7f79bb236640 1 -- start start 2024-04-13T03:15:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79bb236640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79b406e8d0 0x7f79b4162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79bb236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40cf6c0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79bb236640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40cf9a0 con 0x7f79b415ea40 2024-04-13T03:15:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79bb236640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f79b40cfc80 con 0x7f79b406e8d0 2024-04-13T03:15:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f79b40ca1c0 0x7f79b406b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37874/0 (socket says 172.21.15.106:37874) 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b3fff640 1 -- 172.21.15.106:0/4017390902 learned_addr learned my addr 172.21.15.106:0/4017390902 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b97d3640 1 --2- 172.21.15.106:0/4017390902 >> v2:172.21.15.114:3300/0 conn(0x7f79b406e8d0 0x7f79b4162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252041619 0 0) 0x7f79b40cf6c0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79a4003520 con 0x7f79b40ca1c0 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1743854964 0 0) 0x7f79a4003520 con 0x7f79b40ca1c0 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b97d3640 1 -- 172.21.15.106:0/4017390902 >> v1:172.21.15.114:6789/0 conn(0x7f79b415ea40 legacy=0x7f79b40c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b97d3640 1 -- 172.21.15.106:0/4017390902 >> v1:172.21.15.106:6789/0 conn(0x7f79b40ca1c0 legacy=0x7f79b406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b97d3640 1 -- 172.21.15.106:0/4017390902 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b40cff60 con 0x7f79b406e8d0 2024-04-13T03:15:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b97d3640 1 --2- 172.21.15.106:0/4017390902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b406e8d0 0x7f79b4162eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009d90 tx=0x7f79ac02bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4674c8c623174e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0001670 con 0x7f79b406e8d0 2024-04-13T03:15:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac02d720 con 0x7f79b406e8d0 2024-04-13T03:15:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.185+0000 7f79b37fe640 1 -- 172.21.15.106:0/4017390902 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac02daf0 con 0x7f79b406e8d0 2024-04-13T03:15:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 -- 172.21.15.106:0/4017390902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b406e8d0 msgr2=0x7f79b4162eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 --2- 172.21.15.106:0/4017390902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b406e8d0 0x7f79b4162eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009d90 tx=0x7f79ac02bbe0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 -- 172.21.15.106:0/4017390902 shutdown_connections 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 --2- 172.21.15.106:0/4017390902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b406e8d0 0x7f79b4162eb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 -- 172.21.15.106:0/4017390902 >> 172.21.15.106:0/4017390902 conn(0x7f79b40cd1e0 msgr2=0x7f79b40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 -- 172.21.15.106:0/4017390902 shutdown_connections 2024-04-13T03:15:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.186+0000 7f79bb236640 1 -- 172.21.15.106:0/4017390902 wait complete. 2024-04-13T03:15:01.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.187+0000 7f79bb236640 1 Processor -- start 2024-04-13T03:15:01.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.187+0000 7f79bb236640 1 -- start start 2024-04-13T03:15:01.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79b406e8d0 0x7f79b41542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b415ea40 0x7f79b40b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40d0480 con 0x7f79b40ca1c0 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40cf370 con 0x7f79b415ea40 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79bb236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f79b406bff0 con 0x7f79b406e8d0 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40994/0 (socket says 172.21.15.106:40994) 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 -- 172.21.15.106:0/3435690161 learned_addr learned my addr 172.21.15.106:0/3435690161 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b8fd2640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79b406e8d0 0x7f79b41542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b97d3640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b415ea40 0x7f79b40b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 -- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79b406e8d0 msgr2=0x7f79b41542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79b406e8d0 0x7f79b41542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 -- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b415ea40 msgr2=0x7f79b40b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b415ea40 0x7f79b40b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.188+0000 7f79b3fff640 1 -- 172.21.15.106:0/3435690161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41f84d0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79b3fff640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7b0 tx=0x7f79a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0018070 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a00072d0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41f87b0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00047e0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.189+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b40b39a0 con 0x7f79b40ca1c0 2024-04-13T03:15:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.190+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b4068a10 con 0x7f79b40ca1c0 2024-04-13T03:15:01.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.190+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79a001b070 con 0x7f79b40ca1c0 2024-04-13T03:15:01.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.191+0000 7f79b17fa640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 0x7f799806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.191+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f79a008b150 con 0x7f79b40ca1c0 2024-04-13T03:15:01.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.191+0000 7f79b8fd2640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 0x7f799806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.191+0000 7f79b8fd2640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 0x7f799806edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000f60 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.196+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79a0059430 con 0x7f79b40ca1c0 2024-04-13T03:15:01.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.407+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79b40af670 con 0x7f79b40ca1c0 2024-04-13T03:15:01.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.408+0000 7f79b17fa640 1 -- 172.21.15.106:0/3435690161 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5191 (secure 0 0 0) 0x7f79a0058d30 con 0x7f79b40ca1c0 2024-04-13T03:15:01.409 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:01.409 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:14:33.765336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:15:01.409 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 msgr2=0x7f799806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 0x7f799806edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000f60 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 msgr2=0x7f79b41f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7b0 tx=0x7f79a000cc70 comp rx=0 tx=0).stop 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 shutdown_connections 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f799806c8f0 0x7f799806edb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79b415ea40 0x7f79b40b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b40ca1c0 0x7f79b41f7af0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 --2- 172.21.15.106:0/3435690161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79b406e8d0 0x7f79b41542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 >> 172.21.15.106:0/3435690161 conn(0x7f79b40cd1e0 msgr2=0x7f79b406f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 shutdown_connections 2024-04-13T03:15:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.410+0000 7f79bb236640 1 -- 172.21.15.106:0/3435690161 wait complete. 2024-04-13T03:15:01.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-13T03:15:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.523+0000 7fcb48c80640 1 Processor -- start 2024-04-13T03:15:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.523+0000 7fcb48c80640 1 -- start start 2024-04-13T03:15:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb48c80640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb4415e8c0 0x7fcb440ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb48c80640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb44073520 0x7fcb440b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb48c80640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb440b5060 con 0x7fcb4415eca0 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb48c80640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcb440b5340 con 0x7fcb4415e8c0 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb48c80640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4406c500 con 0x7fcb44073520 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb4415e8c0 0x7fcb440ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb4415e8c0 0x7fcb440ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41006/0 (socket says 172.21.15.106:41006) 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 -- 172.21.15.106:0/2059374357 learned_addr learned my addr 172.21.15.106:0/2059374357 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb44073520 0x7fcb440b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 -- 172.21.15.106:0/2059374357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb44073520 msgr2=0x7fcb440b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb44073520 0x7fcb440b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 -- 172.21.15.106:0/2059374357 >> v1:172.21.15.114:6789/0 conn(0x7fcb4415eca0 legacy=0x7fcb44072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.524+0000 7fcb4259c640 1 -- 172.21.15.106:0/2059374357 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4406c7e0 con 0x7fcb4415e8c0 2024-04-13T03:15:01.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.525+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415e8c0 0x7fcb440ce250 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009e50 tx=0x7fcb3402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cebd5794a6aaa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.525+0000 7fcb4159a640 1 -- 172.21.15.106:0/2059374357 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3402cb00 con 0x7fcb4415e8c0 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.525+0000 7fcb4159a640 1 -- 172.21.15.106:0/2059374357 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3402cca0 con 0x7fcb4415e8c0 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.525+0000 7fcb4159a640 1 -- 172.21.15.106:0/2059374357 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34034610 con 0x7fcb4415e8c0 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 -- 172.21.15.106:0/2059374357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415e8c0 msgr2=0x7fcb440ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415e8c0 0x7fcb440ce250 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009e50 tx=0x7fcb3402bf60 comp rx=0 tx=0).stop 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 -- 172.21.15.106:0/2059374357 shutdown_connections 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb44073520 0x7fcb440b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2059374357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415e8c0 0x7fcb440ce250 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 -- 172.21.15.106:0/2059374357 >> 172.21.15.106:0/2059374357 conn(0x7fcb4406f0d0 msgr2=0x7fcb4406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 -- 172.21.15.106:0/2059374357 shutdown_connections 2024-04-13T03:15:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.526+0000 7fcb48c80640 1 -- 172.21.15.106:0/2059374357 wait complete. 2024-04-13T03:15:01.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.527+0000 7fcb48c80640 1 Processor -- start 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.527+0000 7fcb48c80640 1 -- start start 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 0x7fcb441591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb4415e8c0 0x7fcb4415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440b4d80 con 0x7fcb4415eca0 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440b4c00 con 0x7fcb44073520 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb48c80640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4406cd00 con 0x7fcb4415e8c0 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41022/0 (socket says 172.21.15.106:41022) 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 -- 172.21.15.106:0/2067382235 learned_addr learned my addr 172.21.15.106:0/2067382235 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb4415e8c0 0x7fcb4415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 0x7fcb441591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 -- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb4415e8c0 msgr2=0x7fcb4415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb4415e8c0 0x7fcb4415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 -- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 msgr2=0x7fcb441591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 0x7fcb441591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 -- 172.21.15.106:0/2067382235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4415b2b0 con 0x7fcb4415eca0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 0x7fcb441591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.528+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c5c0 tx=0x7fcb3c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.529+0000 7fcb3affd640 1 -- 172.21.15.106:0/2067382235 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c015070 con 0x7fcb4415eca0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.529+0000 7fcb3affd640 1 -- 172.21.15.106:0/2067382235 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c004240 con 0x7fcb4415eca0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.529+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4415b590 con 0x7fcb4415eca0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.529+0000 7fcb3affd640 1 -- 172.21.15.106:0/2067382235 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c003610 con 0x7fcb4415eca0 2024-04-13T03:15:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.529+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4415bb50 con 0x7fcb4415eca0 2024-04-13T03:15:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.530+0000 7fcb3affd640 1 -- 172.21.15.106:0/2067382235 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb3c0037b0 con 0x7fcb4415eca0 2024-04-13T03:15:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.531+0000 7fcb3affd640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.531+0000 7fcb3affd640 1 -- 172.21.15.106:0/2067382235 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fcb3c08bea0 con 0x7fcb4415eca0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.531+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.531+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 0x7fcb2406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcb34035000 tx=0x7fcb34030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.531+0000 7fcb4ba6e740 1 Processor -- start 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 -- start start 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb13df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb07bfe0 con 0x55f2fb1416d0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb018860 con 0x55f2fb13eaf0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2fb077fa0 con 0x55f2fb116800 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41038/0 (socket says 172.21.15.106:41038) 2024-04-13T03:15:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.532+0000 7fcb4259c640 1 -- 172.21.15.106:0/3743932023 learned_addr learned my addr 172.21.15.106:0/3743932023 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 -- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 msgr2=0x55f2fb13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb13df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 -- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 msgr2=0x55f2fb141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 -- 172.21.15.106:0/3743932023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fb0471a0 con 0x55f2fb1416d0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb4259c640 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009c90 tx=0x7fcb3402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ca823403425ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb23fff640 1 -- 172.21.15.106:0/3743932023 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3404e070 con 0x55f2fb1416d0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb23fff640 1 -- 172.21.15.106:0/3743932023 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3402ce50 con 0x55f2fb1416d0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.533+0000 7fcb23fff640 1 -- 172.21.15.106:0/3743932023 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3403ede0 con 0x55f2fb1416d0 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3743932023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 msgr2=0x55f2fb118560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009c90 tx=0x7fcb3402e000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3743932023 shutdown_connections 2024-04-13T03:15:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb13df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3743932023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb118560 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3743932023 >> 172.21.15.106:0/3743932023 conn(0x55f2fb0aadc0 msgr2=0x55f2fb15c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3743932023 shutdown_connections 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3743932023 wait complete. 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 -- start start 2024-04-13T03:15:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.534+0000 7fcb4ba6e740 1 Processor -- start 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb1416d0 0x55f2fb13acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb111b00 con 0x55f2fb116800 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb0161d0 con 0x55f2fb13eaf0 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2fb15cc90 con 0x55f2fb1416d0 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4ba6e740 10 client.0 _ll_register_callbacks cb 0x55f2fb039010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41040/0 (socket says 172.21.15.106:41040) 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.535+0000 7fcb4259c640 1 -- 172.21.15.106:0/2680922001 learned_addr learned my addr 172.21.15.106:0/2680922001 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 -- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb1416d0 msgr2=0x55f2fb13acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb1416d0 0x55f2fb13acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 -- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 msgr2=0x55f2fb138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 -- 172.21.15.106:0/2680922001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fae5a7c0 con 0x55f2fb116800 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4259c640 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000f80 tx=0x7fcb34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babd92c47ce9c70a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34059060 con 0x55f2fb116800 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fcb34034890 con 0x55f2fb116800 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3402ce50 con 0x55f2fb116800 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb3405f050 con 0x55f2fb116800 2024-04-13T03:15:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 10 client.7311 fetch_fsmap learned FSMap version 44 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55f2faebaf50 con 0x55f2fb116800 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3402ce50 con 0x55f2fb116800 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fcb34062020 con 0x55f2fb116800 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 10 client.7311 fetch_fsmap finished waiting for FSMap version 44 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 10 client.7311 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 20 client.7311 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb177d00 0x55f2fb17a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 4 client.7311 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.536+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2fb01b4a0 con 0x55f2fb177d00 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb177d00 0x55f2fb17a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb217fa640 10 client.7311 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb177d00 0x55f2fb17a0f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86489721f67cadf server_cookie=6a1fe11a7d695643 in_seq=0 out_seq=0 2024-04-13T03:15:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb217fa640 1 -- 172.21.15.106:0/2680922001 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2fb01b4a0 con 0x55f2fb177d00 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb217fa640 10 client.7311 handle_command_reply: tid=0 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb4ba6e740 1 client.7311 shutdown 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 msgr2=0x55f2fb142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.537+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000f80 tx=0x7fcb34004ab0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 shutdown_connections 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb177d00 0x55f2fb17a0f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb1416d0 0x55f2fb13acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb217fa640 0 client.7311 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/2680922001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb142fb0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 >> 172.21.15.106:0/2680922001 conn(0x55f2fb0aadc0 msgr2=0x55f2fb110e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 shutdown_connections 2024-04-13T03:15:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/2680922001 wait complete. 2024-04-13T03:15:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.538+0000 7fcb4ba6e740 20 client.7311 trim_cache size 0 max 16384 2024-04-13T03:15:01.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.551+0000 7fcb4ba6e740 1 Processor -- start 2024-04-13T03:15:01.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.551+0000 7fcb4ba6e740 1 -- start start 2024-04-13T03:15:01.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb17fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb115150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb442017a0 con 0x55f2fb1416d0 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440cb9d0 con 0x55f2fb13eaf0 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcb441f5e50 con 0x55f2fb116800 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41056/0 (socket says 172.21.15.106:41056) 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 -- 172.21.15.106:0/3794353827 learned_addr learned my addr 172.21.15.106:0/3794353827 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb4259c640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb17fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 -- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 msgr2=0x55f2fb17fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb17fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 -- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 msgr2=0x55f2fb115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb115150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 -- 172.21.15.106:0/3794353827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fad4e0b0 con 0x55f2fb1416d0 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c004630 tx=0x7fcb3c059ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903c32707f388583 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb237fe640 1 -- 172.21.15.106:0/3794353827 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c01e030 con 0x55f2fb1416d0 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb237fe640 1 -- 172.21.15.106:0/3794353827 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c057d10 con 0x55f2fb1416d0 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.552+0000 7fcb237fe640 1 -- 172.21.15.106:0/3794353827 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0600b0 con 0x55f2fb1416d0 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3794353827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 msgr2=0x55f2fb115690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c004630 tx=0x7fcb3c059ad0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3794353827 shutdown_connections 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb1416d0 0x55f2fb115690 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb13eaf0 0x55f2fb115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/3794353827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb116800 0x55f2fb17fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3794353827 >> 172.21.15.106:0/3794353827 conn(0x55f2fb0aadc0 msgr2=0x55f2fb142e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3794353827 shutdown_connections 2024-04-13T03:15:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/3794353827 wait complete. 2024-04-13T03:15:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 -- start start 2024-04-13T03:15:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.553+0000 7fcb4ba6e740 1 Processor -- start 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 0x55f2fb1d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb1416d0 0x55f2fb1d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb13b910 con 0x55f2fb116800 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440cb380 con 0x55f2fb1416d0 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f2fb1372b0 con 0x55f2fb13eaf0 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4ba6e740 10 client.0 _ll_register_callbacks cb 0x7fcb3403fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41072/0 (socket says 172.21.15.106:41072) 2024-04-13T03:15:01.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.554+0000 7fcb4259c640 1 -- 172.21.15.106:0/1895384563 learned_addr learned my addr 172.21.15.106:0/1895384563 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 0x55f2fb1d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb42d9d640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb1416d0 0x55f2fb1d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 -- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 msgr2=0x55f2fb1d9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 0x55f2fb1d9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 -- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb1416d0 msgr2=0x55f2fb1d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb1416d0 0x55f2fb1d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 -- 172.21.15.106:0/1895384563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fb1d9180 con 0x55f2fb116800 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 0x55f2fb1d9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4259c640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcb3402c570 tx=0x7fcb34009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aafe1ceb7659675 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34068060 con 0x55f2fb116800 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fcb34054c70 con 0x55f2fb116800 2024-04-13T03:15:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c057d10 con 0x55f2fb116800 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb3406e050 con 0x55f2fb116800 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 10 client.7317 fetch_fsmap learned FSMap version 44 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55f2fb110fe0 con 0x55f2fb116800 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c057d10 con 0x55f2fb116800 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fcb34071020 con 0x55f2fb116800 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 10 client.7317 fetch_fsmap finished waiting for FSMap version 44 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 10 client.7317 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 20 client.7317 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb15cf80 0x55f2fb132240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 4 client.7317 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.555+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 --> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f2fb089a00 con 0x55f2fb15cf80 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb15cf80 0x55f2fb132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb217fa640 10 client.7317 ms_handle_connect on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb41d9b640 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb15cf80 0x55f2fb132240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2ad739f08846ae server_cookie=fc307f463729b2b3 in_seq=0 out_seq=0 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb217fa640 1 -- 172.21.15.106:0/1895384563 <== mds.0 v2:172.21.15.106:6835/2254289128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2fb089a00 con 0x55f2fb15cf80 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb217fa640 10 client.7317 handle_command_reply: tid=0 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 client.7317 shutdown 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 msgr2=0x55f2fb16e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcb3402c570 tx=0x7fcb34009000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 shutdown_connections 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:6835/2254289128,v1:172.21.15.106:6837/2254289128] conn(0x55f2fb15cf80 0x55f2fb132240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f2fb13eaf0 0x55f2fb1d9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2fb1416d0 0x55f2fb1d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 --2- 172.21.15.106:0/1895384563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2fb116800 0x55f2fb16e480 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb217fa640 0 client.7317 ms_handle_reset on v2:172.21.15.106:6835/2254289128 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 >> 172.21.15.106:0/1895384563 conn(0x55f2fb0aadc0 msgr2=0x55f2fb13bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 shutdown_connections 2024-04-13T03:15:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 1 -- 172.21.15.106:0/1895384563 wait complete. 2024-04-13T03:15:01.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.556+0000 7fcb4ba6e740 20 client.7317 trim_cache size 0 max 16384 2024-04-13T03:15:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:01.565 INFO:teuthology.orchestra.run.smithi106.stdout:[] 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 msgr2=0x7fcb2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 0x7fcb2406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcb34035000 tx=0x7fcb34030000 comp rx=0 tx=0).stop 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 msgr2=0x7fcb4415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c5c0 tx=0x7fcb3c00ca80 comp rx=0 tx=0).stop 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 shutdown_connections 2024-04-13T03:15:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcb4415e8c0 0x7fcb4415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb4415eca0 0x7fcb4415d370 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 --2- 172.21.15.106:0/2067382235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb44073520 0x7fcb441591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 >> 172.21.15.106:0/2067382235 conn(0x7fcb4406f0d0 msgr2=0x7fcb4415a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 shutdown_connections 2024-04-13T03:15:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.557+0000 7fcb48c80640 1 -- 172.21.15.106:0/2067382235 wait complete. 2024-04-13T03:15:01.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:15:01.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.666+0000 7fe30a029640 1 Processor -- start 2024-04-13T03:15:01.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.666+0000 7fe30a029640 1 -- start start 2024-04-13T03:15:01.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe30415e9b0 0x7fe30406e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 0x7fe30406d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30406d6c0 0x7fe3040cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe3040cb370 con 0x7fe30406d6c0 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe3040cb650 con 0x7fe3040ce770 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe30a029640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe3040cb930 con 0x7fe30415e9b0 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30406d6c0 0x7fe3040cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30406d6c0 0x7fe3040cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41086/0 (socket says 172.21.15.106:41086) 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 -- 172.21.15.106:0/483966997 learned_addr learned my addr 172.21.15.106:0/483966997 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe3037fe640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe30415e9b0 0x7fe30406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe302ffd640 1 --2- 172.21.15.106:0/483966997 >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 0x7fe30406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 -- 172.21.15.106:0/483966997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe30415e9b0 msgr2=0x7fe30406e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe30415e9b0 0x7fe30406e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 -- 172.21.15.106:0/483966997 >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 msgr2=0x7fe30406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 --2- 172.21.15.106:0/483966997 >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 0x7fe30406d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe303fff640 1 -- 172.21.15.106:0/483966997 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3040cbc10 con 0x7fe30406d6c0 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.667+0000 7fe302ffd640 1 --2- 172.21.15.106:0/483966997 >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 0x7fe30406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.668+0000 7fe303fff640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30406d6c0 0x7fe3040cc410 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe30406e140 tx=0x7fe2fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad451c22737eff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.668+0000 7fe3027fc640 1 -- 172.21.15.106:0/483966997 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc02ca70 con 0x7fe30406d6c0 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.668+0000 7fe3027fc640 1 -- 172.21.15.106:0/483966997 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc02cc10 con 0x7fe30406d6c0 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.668+0000 7fe3027fc640 1 -- 172.21.15.106:0/483966997 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0343f0 con 0x7fe30406d6c0 2024-04-13T03:15:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 -- 172.21.15.106:0/483966997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30406d6c0 msgr2=0x7fe3040cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30406d6c0 0x7fe3040cc410 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe30406e140 tx=0x7fe2fc02bed0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 -- 172.21.15.106:0/483966997 shutdown_connections 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30406d6c0 0x7fe3040cc410 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 --2- 172.21.15.106:0/483966997 >> v2:172.21.15.114:3300/0 conn(0x7fe3040ce770 0x7fe30406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 --2- 172.21.15.106:0/483966997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe30415e9b0 0x7fe30406e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 -- 172.21.15.106:0/483966997 >> 172.21.15.106:0/483966997 conn(0x7fe304072600 msgr2=0x7fe304072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 -- 172.21.15.106:0/483966997 shutdown_connections 2024-04-13T03:15:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.669+0000 7fe30a029640 1 -- 172.21.15.106:0/483966997 wait complete. 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.670+0000 7fe30a029640 1 Processor -- start 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.670+0000 7fe30a029640 1 -- start start 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe30406d6c0 0x7fe3041536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 0x7fe3040b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe304073480 con 0x7fe30415e9b0 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe30415ed90 con 0x7fe30406d6c0 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe304073600 con 0x7fe3040ce770 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41088/0 (socket says 172.21.15.106:41088) 2024-04-13T03:15:01.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 -- 172.21.15.106:0/4127294081 learned_addr learned my addr 172.21.15.106:0/4127294081 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe302ffd640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 0x7fe3040b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe3037fe640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe30406d6c0 0x7fe3041536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 -- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 msgr2=0x7fe3040b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 0x7fe3040b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 -- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe30406d6c0 msgr2=0x7fe3041536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe30406d6c0 0x7fe3041536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 -- 172.21.15.106:0/4127294081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3040b5800 con 0x7fe30415e9b0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe302ffd640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 0x7fe3040b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe303fff640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc036040 tx=0x7fe2fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc043020 con 0x7fe30415e9b0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc03e450 con 0x7fe30415e9b0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc009510 con 0x7fe30415e9b0 2024-04-13T03:15:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.671+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3040b5ae0 con 0x7fe30415e9b0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.672+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3040b60a0 con 0x7fe30415e9b0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.672+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe304068a10 con 0x7fe30415e9b0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.673+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2fc046020 con 0x7fe30415e9b0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.673+0000 7fe2ebfff640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.673+0000 7fe3037fe640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.673+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fe2fc0ae480 con 0x7fe30415e9b0 2024-04-13T03:15:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.674+0000 7fe3037fe640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b03f0 tx=0x7fe2f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.678+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2fc04d030 con 0x7fe30415e9b0 2024-04-13T03:15:01.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.891+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe3040700a0 con 0x7fe30415e9b0 2024-04-13T03:15:01.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.891+0000 7fe2ebfff640 1 -- 172.21.15.106:0/4127294081 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe2fc07c760 con 0x7fe30415e9b0 2024-04-13T03:15:01.892 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:01.892 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 msgr2=0x7fe2f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b03f0 tx=0x7fe2f4008040 comp rx=0 tx=0).stop 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 msgr2=0x7fe30415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc036040 tx=0x7fe2fc004a40 comp rx=0 tx=0).stop 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 shutdown_connections 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe3040ce770 0x7fe3040b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30415e9b0 0x7fe30415d270 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 --2- 172.21.15.106:0/4127294081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe30406d6c0 0x7fe3041536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:01.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 >> 172.21.15.106:0/4127294081 conn(0x7fe304072600 msgr2=0x7fe30406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:01.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 shutdown_connections 2024-04-13T03:15:01.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:01.893+0000 7fe30a029640 1 -- 172.21.15.106:0/4127294081 wait complete. 2024-04-13T03:15:01.904 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:15:01.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:15:02.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.004+0000 7fbaa1ac8640 1 Processor -- start 2024-04-13T03:15:02.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.004+0000 7fbaa1ac8640 1 -- start start 2024-04-13T03:15:02.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fbaa1ac8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba9c1591a0 0x7fba9c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fbaa1ac8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 0x7fba9c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fbaa1ac8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9c159580 con 0x7fba9c0b32d0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fbaa1ac8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fba9c0b3ab0 con 0x7fba9c1591a0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fbaa1ac8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c0b3d90 con 0x7fba9c0b5320 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba9c1591a0 0x7fba9c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba9c1591a0 0x7fba9c15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41098/0 (socket says 172.21.15.106:41098) 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 -- 172.21.15.106:0/2341438217 learned_addr learned my addr 172.21.15.106:0/2341438217 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9bfff640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 0x7fba9c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 -- 172.21.15.106:0/2341438217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 msgr2=0x7fba9c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 0x7fba9c06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 -- 172.21.15.106:0/2341438217 >> v1:172.21.15.114:6789/0 conn(0x7fba9c0b32d0 legacy=0x7fba9c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9b7fe640 1 -- 172.21.15.106:0/2341438217 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0b4070 con 0x7fba9c1591a0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.005+0000 7fba9bfff640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 0x7fba9c06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.006+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c15bbf0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0b3090 tx=0x7fba8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163d4573c840c7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.006+0000 7fba9a7fc640 1 -- 172.21.15.106:0/2341438217 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c035020 con 0x7fba9c1591a0 2024-04-13T03:15:02.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.006+0000 7fba9a7fc640 1 -- 172.21.15.106:0/2341438217 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c031830 con 0x7fba9c1591a0 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.006+0000 7fba9a7fc640 1 -- 172.21.15.106:0/2341438217 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c031c00 con 0x7fba9c1591a0 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/2341438217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 msgr2=0x7fba9c15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c15bbf0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0b3090 tx=0x7fba8c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/2341438217 shutdown_connections 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b5320 0x7fba9c06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/2341438217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c15bbf0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/2341438217 >> 172.21.15.106:0/2341438217 conn(0x7fba9c0d0340 msgr2=0x7fba9c0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/2341438217 shutdown_connections 2024-04-13T03:15:02.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.007+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/2341438217 wait complete. 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.008+0000 7fbaa1ac8640 1 Processor -- start 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.008+0000 7fbaa1ac8640 1 -- start start 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b32d0 0x7fba9c156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 0x7fba9c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c0b3760 con 0x7fba9c1591a0 2024-04-13T03:15:02.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c0b4630 con 0x7fba9c0b5320 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c06e310 con 0x7fba9c0b32d0 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41112/0 (socket says 172.21.15.106:41112) 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 -- 172.21.15.106:0/1814276830 learned_addr learned my addr 172.21.15.106:0/1814276830 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b32d0 0x7fba9c156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9affd640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 0x7fba9c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 -- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b32d0 msgr2=0x7fba9c156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b32d0 0x7fba9c156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 -- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 msgr2=0x7fba9c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 0x7fba9c0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 -- 172.21.15.106:0/1814276830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0cec20 con 0x7fba9c1591a0 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9affd640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 0x7fba9c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba9bfff640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fba9400cf20 tx=0x7fba9400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9401a070 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9400d840 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9400dc30 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c0cf1b0 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.009+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c0cf770 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.010+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c150710 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.011+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba9401d070 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.011+0000 7fba83fff640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 0x7fba8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.011+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 0x7fba8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.011+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fba9408c5d0 con 0x7fba9c1591a0 2024-04-13T03:15:02.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.012+0000 7fba9b7fe640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 0x7fba8806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fba8c036000 tx=0x7fba8c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.016+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba9405a820 con 0x7fba9c1591a0 2024-04-13T03:15:02.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.231+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fba9c068a10 con 0x7fba9c1591a0 2024-04-13T03:15:02.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.232+0000 7fba83fff640 1 -- 172.21.15.106:0/1814276830 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7fba9405a120 con 0x7fba9c1591a0 2024-04-13T03:15:02.233 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 msgr2=0x7fba8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 0x7fba8806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fba8c036000 tx=0x7fba8c03c040 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 msgr2=0x7fba9c1f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fba9400cf20 tx=0x7fba9400a7d0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 shutdown_connections 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba8806c8f0 0x7fba8806edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9c1591a0 0x7fba9c1f27c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c0b5320 0x7fba9c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.233+0000 7fbaa1ac8640 1 --2- 172.21.15.106:0/1814276830 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba9c0b32d0 0x7fba9c156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:02.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.234+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 >> 172.21.15.106:0/1814276830 conn(0x7fba9c0d0340 msgr2=0x7fba9c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:02.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.234+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 shutdown_connections 2024-04-13T03:15:02.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02.234+0000 7fbaa1ac8640 1 -- 172.21.15.106:0/1814276830 wait complete. 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:02.245 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:15:02.246 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:15:02.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:15:02.246 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:02.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.246 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:15:02.261 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42247sec preferred_lft 42247sec 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:15:02.262 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:15:02.263 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:02.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.264 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:15:02.327 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:15:02.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.329 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:15:02.384 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:15:02.384 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:15:02.384 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:15:02.384 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:15:02.384 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:15:02.385 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:15:02.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:15:02.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:02.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.521 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:15:02.535 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:15:02.535 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:15:02.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:02.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.536 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:15:02.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:02.602 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:02.657 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:02.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:02.658 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:02.712 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:15:02.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.713 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:02.797 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:02.797 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:02.804 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:02.804 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:02.804 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:15:02.804 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:15:02.804 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:15:02.805 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:15:02.805 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:15:02.805 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:15:02.805 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:15:02.805 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:15:02.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:15:02.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:02.833 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:15:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:15:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:02.929 INFO:teuthology.orchestra.run.smithi106.stdout:client7332 2024-04-13T03:15:02.930 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:02.930 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:02.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:02.946 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-13T03:15:03.001 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:15:03.001 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-13T03:15:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.083+0000 7fb9f9240f40 1 Processor -- start 2024-04-13T03:15:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.083+0000 7fb9f9240f40 1 -- start start 2024-04-13T03:15:03.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 0x55d3c6b57a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6b5b170 0x55d3c6ab2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d3c6a2dd00 con 0x55d3c6ab43e0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d3c6a31d40 con 0x55d3c6a85340 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d3c6acedb0 con 0x55d3c6b5b170 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36666/0 (socket says 172.21.15.106:36666) 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 -- 172.21.15.106:0/917639872 learned_addr learned my addr 172.21.15.106:0/917639872 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f6ba4640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6b5b170 0x55d3c6ab2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f63a3640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 0x55d3c6b57a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 -- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6b5b170 msgr2=0x55d3c6ab2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6b5b170 0x55d3c6ab2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 -- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 msgr2=0x55d3c6b57a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 0x55d3c6b57a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 -- 172.21.15.106:0/917639872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c6b43aa0 con 0x55d3c6ab43e0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9f63a3640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 0x55d3c6b57a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.084+0000 7fb9effff640 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000b320 tx=0x7fb9e000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae081ba27484ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f53a1640 1 -- 172.21.15.106:0/917639872 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0008070 con 0x55d3c6ab43e0 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f53a1640 1 -- 172.21.15.106:0/917639872 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e000d500 con 0x55d3c6ab43e0 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f53a1640 1 -- 172.21.15.106:0/917639872 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000d8f0 con 0x55d3c6ab43e0 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 -- 172.21.15.106:0/917639872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 msgr2=0x55d3c6b57fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000b320 tx=0x7fb9e000b7e0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 -- 172.21.15.106:0/917639872 shutdown_connections 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6b5b170 0x55d3c6ab2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b57fa0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/917639872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6a85340 0x55d3c6b57a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 -- 172.21.15.106:0/917639872 >> 172.21.15.106:0/917639872 conn(0x55d3c6a85f30 msgr2=0x55d3c6b58630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 -- 172.21.15.106:0/917639872 shutdown_connections 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.085+0000 7fb9f9240f40 1 -- 172.21.15.106:0/917639872 wait complete. 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 Processor -- start 2024-04-13T03:15:03.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 -- start start 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6a85340 0x55d3c6ab2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6b5b170 0x55d3c6b41f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d3c6ab12c0 con 0x55d3c6ab43e0 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d3c6ab0ab0 con 0x55d3c6b5b170 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9f9240f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d3c6b50dc0 con 0x55d3c6a85340 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.086+0000 7fb9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36678/0 (socket says 172.21.15.106:36678) 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 -- 172.21.15.106:0/1257385675 learned_addr learned my addr 172.21.15.106:0/1257385675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9f6ba4640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6b5b170 0x55d3c6b41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9f63a3640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6a85340 0x55d3c6ab2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 -- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6a85340 msgr2=0x55d3c6ab2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d3c6a85340 0x55d3c6ab2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 -- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6b5b170 msgr2=0x55d3c6b41f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6b5b170 0x55d3c6b41f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 -- 172.21.15.106:0/1257385675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c6b4cd30 con 0x55d3c6ab43e0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9f6ba4640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3c6b5b170 0x55d3c6b41f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9effff640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b419d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000f60 tx=0x7fb9e0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9ef7fe640 1 -- 172.21.15.106:0/1257385675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e001c850 con 0x55d3c6ab43e0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9ef7fe640 1 -- 172.21.15.106:0/1257385675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e0020030 con 0x55d3c6ab43e0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3c6b4d010 con 0x55d3c6ab43e0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9ef7fe640 1 -- 172.21.15.106:0/1257385675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e001cc50 con 0x55d3c6ab43e0 2024-04-13T03:15:03.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.087+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c6b4d5d0 con 0x55d3c6ab43e0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.088+0000 7fb9ef7fe640 1 -- 172.21.15.106:0/1257385675 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9e0017040 con 0x55d3c6ab43e0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9ef7fe640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb9d806b2d0 0x7fb9d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9ef7fe640 1 -- 172.21.15.106:0/1257385675 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fb9e008c800 con 0x55d3c6ab43e0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f63a3640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb9d806b2d0 0x7fb9d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55d3c6ab2a60 0x55d3c6b464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) -- 0x55d3c6ab0e00 con 0x55d3c6ab2a60 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f6ba4640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55d3c6ab2a60 0x55d3c6b464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f63a3640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb9d806b2d0 0x7fb9d806d790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8000c00 tx=0x7fb9e80017e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.089+0000 7fb9f6ba4640 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55d3c6ab2a60 0x55d3c6b464b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f6ba4640 1 -- 172.21.15.106:0/1257385675 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb9f0052980 con 0x55d3c6ab2a60 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55d3c6ab2a60 msgr2=0x55d3c6b464b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55d3c6ab2a60 0x55d3c6b464b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb9d806b2d0 msgr2=0x7fb9d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb9d806b2d0 0x7fb9d806d790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8000c00 tx=0x7fb9e80017e0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 msgr2=0x55d3c6b419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 --2- 172.21.15.106:0/1257385675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d3c6ab43e0 0x55d3c6b419d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000f60 tx=0x7fb9e0000ed0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f6ba4640 1 -- 172.21.15.106:0/1257385675 reap_dead start 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 shutdown_connections 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 >> 172.21.15.106:0/1257385675 conn(0x55d3c6a85f30 msgr2=0x55d3c6b58570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 shutdown_connections 2024-04-13T03:15:03.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.091+0000 7fb9f9240f40 1 -- 172.21.15.106:0/1257385675 wait complete. 2024-04-13T03:15:03.096 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:15:03.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:15:03.184 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:15:03.237 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:15:03.237 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:15:03.237 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:03.237 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:03.252 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:03.252 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:03.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:03.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.446+0000 7fa2db020640 1 Processor -- start 2024-04-13T03:15:03.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.446+0000 7fa2db020640 1 -- start start 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2d4156b50 0x7fa2d4160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 0x7fa2d415d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d415d690 con 0x7fa2d4156b50 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d415d970 con 0x7fa2d415cc10 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2db020640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d4160410 con 0x7fa2d406dd70 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2d4156b50 0x7fa2d4160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa2d4156b50 0x7fa2d4160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36688/0 (socket says 172.21.15.106:36688) 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 -- 172.21.15.106:0/4113219016 learned_addr learned my addr 172.21.15.106:0/4113219016 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d95bd640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 -- 172.21.15.106:0/4113219016 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 msgr2=0x7fa2d406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d3fff640 1 --2- 172.21.15.106:0/4113219016 >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 0x7fa2d415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d406e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 -- 172.21.15.106:0/4113219016 >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 msgr2=0x7fa2d415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/4113219016 >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 0x7fa2d415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d8dbc640 1 -- 172.21.15.106:0/4113219016 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41606f0 con 0x7fa2d4156b50 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d95bd640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.447+0000 7fa2d3fff640 1 --2- 172.21.15.106:0/4113219016 >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 0x7fa2d415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.448+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d4156b50 0x7fa2d4160c10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406d700 tx=0x7fa2c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6a04de372718de server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.448+0000 7fa2d37fe640 1 -- 172.21.15.106:0/4113219016 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4035020 con 0x7fa2d4156b50 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.448+0000 7fa2d37fe640 1 -- 172.21.15.106:0/4113219016 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4031870 con 0x7fa2d4156b50 2024-04-13T03:15:03.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.448+0000 7fa2d37fe640 1 -- 172.21.15.106:0/4113219016 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4031c40 con 0x7fa2d4156b50 2024-04-13T03:15:03.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 -- 172.21.15.106:0/4113219016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d4156b50 msgr2=0x7fa2d4160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d4156b50 0x7fa2d4160c10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406d700 tx=0x7fa2c402b910 comp rx=0 tx=0).stop 2024-04-13T03:15:03.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 -- 172.21.15.106:0/4113219016 shutdown_connections 2024-04-13T03:15:03.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 --2- 172.21.15.106:0/4113219016 >> v2:172.21.15.114:3300/0 conn(0x7fa2d415cc10 0x7fa2d415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 --2- 172.21.15.106:0/4113219016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d4156b50 0x7fa2d4160c10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 -- 172.21.15.106:0/4113219016 >> 172.21.15.106:0/4113219016 conn(0x7fa2d40b1f10 msgr2=0x7fa2d40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 -- 172.21.15.106:0/4113219016 shutdown_connections 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.449+0000 7fa2db020640 1 -- 172.21.15.106:0/4113219016 wait complete. 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.450+0000 7fa2db020640 1 Processor -- start 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.450+0000 7fa2db020640 1 -- start start 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d41f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4156b50 0x7fa2d4156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40afc20 con 0x7fa2d415cc10 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40afaa0 con 0x7fa2d4156b50 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2db020640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d406bcb0 con 0x7fa2d406dd70 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36700/0 (socket says 172.21.15.106:36700) 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 -- 172.21.15.106:0/3910559658 learned_addr learned my addr 172.21.15.106:0/3910559658 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d41f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d3fff640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4156b50 0x7fa2d4156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 -- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 msgr2=0x7fa2d41f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d41f2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 -- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4156b50 msgr2=0x7fa2d4156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4156b50 0x7fa2d4156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 -- 172.21.15.106:0/3910559658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41567b0 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.451+0000 7fa2d95bd640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00cd40 tx=0x7fa2cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.452+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f680 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.452+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00f840 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.452+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d40ae140 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.452+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00fc30 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.452+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d40c3b60 con 0x7fa2d415cc10 2024-04-13T03:15:03.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.453+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d4068a10 con 0x7fa2d415cc10 2024-04-13T03:15:03.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.453+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2cc0213f0 con 0x7fa2d415cc10 2024-04-13T03:15:03.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.453+0000 7fa2d17fa640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.454+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fa2cc08c860 con 0x7fa2d415cc10 2024-04-13T03:15:03.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.454+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.454+0000 7fa2d8dbc640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 0x7fa2c006edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4036000 tx=0x7fa2c403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.458+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2d41567b0 con 0x7fa2d415cc10 2024-04-13T03:15:03.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.632+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2d406ba90 con 0x7fa2d415cc10 2024-04-13T03:15:03.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.633+0000 7fa2d17fa640 1 -- 172.21.15.106:0/3910559658 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17110 (secure 0 0 0) 0x7fa2cc014030 con 0x7fa2d415cc10 2024-04-13T03:15:03.634 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:03.635 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:50.750679+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:03.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 msgr2=0x7fa2c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 0x7fa2c006edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4036000 tx=0x7fa2c403c040 comp rx=0 tx=0).stop 2024-04-13T03:15:03.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 msgr2=0x7fa2d415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00cd40 tx=0x7fa2cc00a650 comp rx=0 tx=0).stop 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 shutdown_connections 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2d415cc10 0x7fa2d415ec00 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4156b50 0x7fa2d4156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 --2- 172.21.15.106:0/3910559658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa2d406dd70 0x7fa2d41f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 >> 172.21.15.106:0/3910559658 conn(0x7fa2d40b1f10 msgr2=0x7fa2d40b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 shutdown_connections 2024-04-13T03:15:03.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.635+0000 7fa2db020640 1 -- 172.21.15.106:0/3910559658 wait complete. 2024-04-13T03:15:03.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:13:35.329204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:13:36.059991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:15:03.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:15:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.749+0000 7f805318c640 1 Processor -- start 2024-04-13T03:15:03.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.749+0000 7f805318c640 1 -- start start 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f805318c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f804c145d20 0x7f804c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f805318c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f805318c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0b3740 con 0x7f804c0b47e0 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f805318c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f804c0b3a20 con 0x7f804c145d20 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f805318c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0b3d00 con 0x7f804c0b4c20 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f804c145d20 0x7f804c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f804c145d20 0x7f804c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36704/0 (socket says 172.21.15.106:36704) 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 -- 172.21.15.106:0/3140309930 learned_addr learned my addr 172.21.15.106:0/3140309930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8051729640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 -- 172.21.15.106:0/3140309930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 msgr2=0x7f804c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c0cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 -- 172.21.15.106:0/3140309930 >> v1:172.21.15.114:6789/0 conn(0x7f804c0b47e0 legacy=0x7f804c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.750+0000 7f8050f28640 1 -- 172.21.15.106:0/3140309930 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0b3fe0 con 0x7f804c145d20 2024-04-13T03:15:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f8050f28640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c145d20 0x7f804c06cb10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdf5948a143b0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f804b7fe640 1 -- 172.21.15.106:0/3140309930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c035020 con 0x7f804c145d20 2024-04-13T03:15:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f804b7fe640 1 -- 172.21.15.106:0/3140309930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c031640 con 0x7f804c145d20 2024-04-13T03:15:03.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f804b7fe640 1 -- 172.21.15.106:0/3140309930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c02d040 con 0x7f804c145d20 2024-04-13T03:15:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 -- 172.21.15.106:0/3140309930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c145d20 msgr2=0x7f804c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c145d20 0x7f804c06cb10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 -- 172.21.15.106:0/3140309930 shutdown_connections 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c0cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 --2- 172.21.15.106:0/3140309930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c145d20 0x7f804c06cb10 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.751+0000 7f805318c640 1 -- 172.21.15.106:0/3140309930 >> 172.21.15.106:0/3140309930 conn(0x7f804c0b2600 msgr2=0x7f804c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.752+0000 7f805318c640 1 -- 172.21.15.106:0/3140309930 shutdown_connections 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.752+0000 7f805318c640 1 -- 172.21.15.106:0/3140309930 wait complete. 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 Processor -- start 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 -- start start 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c1e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f804c145d20 0x7f804c148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f804c146280 con 0x7f804c0b47e0 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f804c146100 con 0x7f804c145d20 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.753+0000 7f805318c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f804c06d050 con 0x7f804c0b4c20 2024-04-13T03:15:03.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36710/0 (socket says 172.21.15.106:36710) 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 -- 172.21.15.106:0/351463392 learned_addr learned my addr 172.21.15.106:0/351463392 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 -- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 msgr2=0x7f804c1e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c1e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 -- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f804c145d20 msgr2=0x7f804c148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f804c145d20 0x7f804c148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 -- 172.21.15.106:0/351463392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c1e6540 con 0x7f804c0b47e0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f8050f28640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f803c02b930 tx=0x7f803c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c040070 con 0x7f804c0b47e0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c039bf0 con 0x7f804c0b47e0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c035040 con 0x7f804c0b47e0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c1e6820 con 0x7f804c0b47e0 2024-04-13T03:15:03.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.754+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1e6de0 con 0x7f804c0b47e0 2024-04-13T03:15:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.755+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c068a10 con 0x7f804c0b47e0 2024-04-13T03:15:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.756+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f803c04b060 con 0x7f804c0b47e0 2024-04-13T03:15:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.756+0000 7f80497fa640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 0x7f803806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:03.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.756+0000 7f804bfff640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 0x7f803806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.756+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f803c0ad830 con 0x7f804c0b47e0 2024-04-13T03:15:03.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.757+0000 7f804bfff640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 0x7f803806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f804c15f950 tx=0x7f8040007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:03.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.761+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f803c02f300 con 0x7f804c0b47e0 2024-04-13T03:15:03.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.937+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f804c0cdc60 con 0x7f804c0b47e0 2024-04-13T03:15:03.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.937+0000 7f80497fa640 1 -- 172.21.15.106:0/351463392 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f803c07ba80 con 0x7f804c0b47e0 2024-04-13T03:15:03.938 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:15:03.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 msgr2=0x7f803806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 0x7f803806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f804c15f950 tx=0x7f8040007990 comp rx=0 tx=0).stop 2024-04-13T03:15:03.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 msgr2=0x7f804c15f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:03.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f803c02b930 tx=0x7f803c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 shutdown_connections 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f803806c8f0 0x7f803806edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f804c0b4c20 0x7f804c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f804c145d20 0x7f804c148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 --2- 172.21.15.106:0/351463392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f804c0b47e0 0x7f804c15f0d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 >> 172.21.15.106:0/351463392 conn(0x7f804c0b2600 msgr2=0x7f804c1439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 shutdown_connections 2024-04-13T03:15:03.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:03.939+0000 7f805318c640 1 -- 172.21.15.106:0/351463392 wait complete. 2024-04-13T03:15:03.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:15:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.053+0000 7f28fbfaa640 1 Processor -- start 2024-04-13T03:15:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.053+0000 7f28fbfaa640 1 -- start start 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fbfaa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f28f406e750 0x7f28f40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fbfaa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28f4073520 0x7f28f415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fbfaa640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f415d8e0 con 0x7f28f40ceeb0 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fbfaa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40cde90 con 0x7f28f4073520 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fbfaa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40ce170 con 0x7f28f406e750 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fa547640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28f4073520 0x7f28f415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fa547640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28f4073520 0x7f28f415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36718/0 (socket says 172.21.15.106:36718) 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28fa547640 1 -- 172.21.15.106:0/3935683110 learned_addr learned my addr 172.21.15.106:0/3935683110 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.054+0000 7f28f9d46640 1 --2- 172.21.15.106:0/3935683110 >> v2:172.21.15.114:3300/0 conn(0x7f28f406e750 0x7f28f40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.055+0000 7f28fa547640 1 -- 172.21.15.106:0/3935683110 >> v2:172.21.15.114:3300/0 conn(0x7f28f406e750 msgr2=0x7f28f40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.055+0000 7f28fa547640 1 --2- 172.21.15.106:0/3935683110 >> v2:172.21.15.114:3300/0 conn(0x7f28f406e750 0x7f28f40ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.055+0000 7f28fa547640 1 -- 172.21.15.106:0/3935683110 >> v1:172.21.15.114:6789/0 conn(0x7f28f40ceeb0 legacy=0x7f28f4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.055+0000 7f28fa547640 1 -- 172.21.15.106:0/3935683110 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f406eb30 con 0x7f28f4073520 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.055+0000 7f28fa547640 1 --2- 172.21.15.106:0/3935683110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f4073520 0x7f28f415cc30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f28f40702d0 tx=0x7f28ec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c378350903e64898 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28f8d44640 1 -- 172.21.15.106:0/3935683110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec02c9f0 con 0x7f28f4073520 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28f8d44640 1 -- 172.21.15.106:0/3935683110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec02cb90 con 0x7f28f4073520 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28f8d44640 1 -- 172.21.15.106:0/3935683110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0343f0 con 0x7f28f4073520 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 -- 172.21.15.106:0/3935683110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f4073520 msgr2=0x7f28f415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/3935683110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f4073520 0x7f28f415cc30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f28f40702d0 tx=0x7f28ec02be50 comp rx=0 tx=0).stop 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 -- 172.21.15.106:0/3935683110 shutdown_connections 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/3935683110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f4073520 0x7f28f415cc30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/3935683110 >> v2:172.21.15.114:3300/0 conn(0x7f28f406e750 0x7f28f40ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 -- 172.21.15.106:0/3935683110 >> 172.21.15.106:0/3935683110 conn(0x7f28f406f180 msgr2=0x7f28f406f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 -- 172.21.15.106:0/3935683110 shutdown_connections 2024-04-13T03:15:04.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.056+0000 7f28fbfaa640 1 -- 172.21.15.106:0/3935683110 wait complete. 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.057+0000 7f28fbfaa640 1 Processor -- start 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.057+0000 7f28fbfaa640 1 -- start start 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28f4073520 0x7f28f41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28f40ceeb0 0x7f28f413ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28f415d540 con 0x7f28f406e750 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28f415d3c0 con 0x7f28f40ceeb0 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fbfaa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28f415d6c0 con 0x7f28f4073520 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36722/0 (socket says 172.21.15.106:36722) 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 -- 172.21.15.106:0/739288049 learned_addr learned my addr 172.21.15.106:0/739288049 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9545640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28f4073520 0x7f28f41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28fa547640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28f40ceeb0 0x7f28f413ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 -- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28f4073520 msgr2=0x7f28f41549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28f4073520 0x7f28f41549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 -- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28f40ceeb0 msgr2=0x7f28f413ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28f40ceeb0 0x7f28f413ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.058+0000 7f28f9d46640 1 -- 172.21.15.106:0/739288049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f4154ef0 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28f9d46640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f28f000d620 tx=0x7f28f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000b620 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f0019070 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f4155190 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000ba00 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.059+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f4140ba0 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.060+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f4069120 con 0x7f28f406e750 2024-04-13T03:15:04.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.060+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28f000bba0 con 0x7f28f406e750 2024-04-13T03:15:04.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.060+0000 7f28ea7fc640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.061+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f28f008c160 con 0x7f28f406e750 2024-04-13T03:15:04.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.061+0000 7f28f9545640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.061+0000 7f28f9545640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 0x7f28d806edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28f40d0870 tx=0x7f28e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.065+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28f005a3b0 con 0x7f28f406e750 2024-04-13T03:15:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.238+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28f415c6f0 con 0x7f28f406e750 2024-04-13T03:15:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.239+0000 7f28ea7fc640 1 -- 172.21.15.106:0/739288049 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f28f0059cb0 con 0x7f28f406e750 2024-04-13T03:15:04.240 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:15:04.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.240+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 msgr2=0x7f28d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.240+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 0x7f28d806edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28f40d0870 tx=0x7f28e4009040 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.240+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 msgr2=0x7f28f4156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.240+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f28f000d620 tx=0x7f28f000dae0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 shutdown_connections 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28f4073520 0x7f28f41549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28f40ceeb0 0x7f28f413ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 --2- 172.21.15.106:0/739288049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28f406e750 0x7f28f4156c90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 >> 172.21.15.106:0/739288049 conn(0x7f28f406f180 msgr2=0x7f28f4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 shutdown_connections 2024-04-13T03:15:04.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.241+0000 7f28fbfaa640 1 -- 172.21.15.106:0/739288049 wait complete. 2024-04-13T03:15:04.252 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:15:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.354+0000 7fbafbf1e640 1 Processor -- start 2024-04-13T03:15:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.354+0000 7fbafbf1e640 1 -- start start 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafbf1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbaf40cfea0 0x7fbaf415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafbf1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbaf40d1c30 0x7fbaf415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafbf1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf415ddd0 con 0x7fbaf40704a0 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafbf1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf415e100 con 0x7fbaf40cfea0 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafbf1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf415d4d0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbaf40cfea0 0x7fbaf415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbaf40cfea0 0x7fbaf415cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36726/0 (socket says 172.21.15.106:36726) 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 -- 172.21.15.106:0/2648043063 learned_addr learned my addr 172.21.15.106:0/2648043063 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 -- 172.21.15.106:0/2648043063 >> v2:172.21.15.114:3300/0 conn(0x7fbaf40d1c30 msgr2=0x7fbaf415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbafa4bb640 1 --2- 172.21.15.106:0/2648043063 >> v2:172.21.15.114:3300/0 conn(0x7fbaf40d1c30 0x7fbaf415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/2648043063 >> v2:172.21.15.114:3300/0 conn(0x7fbaf40d1c30 0x7fbaf415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 -- 172.21.15.106:0/2648043063 >> v1:172.21.15.114:6789/0 conn(0x7fbaf40704a0 legacy=0x7fbaf40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.355+0000 7fbaf9cba640 1 -- 172.21.15.106:0/2648043063 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf415d7b0 con 0x7fbaf40cfea0 2024-04-13T03:15:04.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/2648043063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40cfea0 0x7fbaf415cc10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4073f70 tx=0x7fbaf002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70474de8f1b62eb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbaf8cb8640 1 -- 172.21.15.106:0/2648043063 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf002e070 con 0x7fbaf40cfea0 2024-04-13T03:15:04.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbaf8cb8640 1 -- 172.21.15.106:0/2648043063 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0037070 con 0x7fbaf40cfea0 2024-04-13T03:15:04.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbaf8cb8640 1 -- 172.21.15.106:0/2648043063 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0031a00 con 0x7fbaf40cfea0 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbafbf1e640 1 -- 172.21.15.106:0/2648043063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40cfea0 msgr2=0x7fbaf415cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.356+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/2648043063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40cfea0 0x7fbaf415cc10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4073f70 tx=0x7fbaf002b880 comp rx=0 tx=0).stop 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 -- 172.21.15.106:0/2648043063 shutdown_connections 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/2648043063 >> v2:172.21.15.114:3300/0 conn(0x7fbaf40d1c30 0x7fbaf415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/2648043063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40cfea0 0x7fbaf415cc10 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 -- 172.21.15.106:0/2648043063 >> 172.21.15.106:0/2648043063 conn(0x7fbaf4071870 msgr2=0x7fbaf4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 -- 172.21.15.106:0/2648043063 shutdown_connections 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.357+0000 7fbafbf1e640 1 -- 172.21.15.106:0/2648043063 wait complete. 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 Processor -- start 2024-04-13T03:15:04.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 -- start start 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 0x7fbaf415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 0x7fbaf4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf415ead0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf415d150 con 0x7fbaf41520d0 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.358+0000 7fbafbf1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf415ec50 con 0x7fbaf40704a0 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36736/0 (socket says 172.21.15.106:36736) 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 -- 172.21.15.106:0/778525454 learned_addr learned my addr 172.21.15.106:0/778525454 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 0x7fbaf415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 -- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 msgr2=0x7fbaf415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 0x7fbaf415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbafa4bb640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 0x7fbaf4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 -- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 msgr2=0x7fbaf4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 0x7fbaf4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 -- 172.21.15.106:0/778525454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf4157a90 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 0x7fbaf415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbafa4bb640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 0x7fbaf4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaf94b9640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbae400c440 tx=0x7fbae400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4017070 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae40035f0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf4157d70 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40039e0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.359+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf4158330 con 0x7fbaf40d1c30 2024-04-13T03:15:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.360+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf4068a10 con 0x7fbaf40d1c30 2024-04-13T03:15:04.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.360+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbae401a070 con 0x7fbaf40d1c30 2024-04-13T03:15:04.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.361+0000 7fbaea7fc640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.361+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fbae408b0e0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.361+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.361+0000 7fbaf9cba640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 0x7fbad806edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0036000 tx=0x7fbaf0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.365+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbae4059310 con 0x7fbaf40d1c30 2024-04-13T03:15:04.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.545+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbaf4071df0 con 0x7fbaf40d1c30 2024-04-13T03:15:04.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.545+0000 7fbaea7fc640 1 -- 172.21.15.106:0/778525454 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fbae4058c10 con 0x7fbaf40d1c30 2024-04-13T03:15:04.546 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 msgr2=0x7fbad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 0x7fbad806edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0036000 tx=0x7fbaf0008030 comp rx=0 tx=0).stop 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 msgr2=0x7fbaf4151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbae400c440 tx=0x7fbae400c900 comp rx=0 tx=0).stop 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 shutdown_connections 2024-04-13T03:15:04.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad806c8f0 0x7fbad806edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaf41520d0 0x7fbaf4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf40d1c30 0x7fbaf4151b90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 --2- 172.21.15.106:0/778525454 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf40704a0 0x7fbaf415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 >> 172.21.15.106:0/778525454 conn(0x7fbaf4071870 msgr2=0x7fbaf415bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 shutdown_connections 2024-04-13T03:15:04.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.547+0000 7fbafbf1e640 1 -- 172.21.15.106:0/778525454 wait complete. 2024-04-13T03:15:04.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:15:04.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.659+0000 7fe4aec18640 1 Processor -- start 2024-04-13T03:15:04.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.659+0000 7fe4aec18640 1 -- start start 2024-04-13T03:15:04.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4aec18640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe4a80cf480 0x7fe4a815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4aec18640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4a80d3220 0x7fe4a80d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4aec18640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a815f380 con 0x7fe4a80ba170 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4aec18640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a80d2d90 con 0x7fe4a80d3220 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4aec18640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a815f600 con 0x7fe4a80cf480 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4ad1b5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4a80d3220 0x7fe4a80d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4ad1b5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4a80d3220 0x7fe4a80d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36746/0 (socket says 172.21.15.106:36746) 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4ad1b5640 1 -- 172.21.15.106:0/480520380 learned_addr learned my addr 172.21.15.106:0/480520380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.660+0000 7fe4ac9b4640 1 --2- 172.21.15.106:0/480520380 >> v2:172.21.15.114:3300/0 conn(0x7fe4a80cf480 0x7fe4a815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4ad1b5640 1 -- 172.21.15.106:0/480520380 >> v2:172.21.15.114:3300/0 conn(0x7fe4a80cf480 msgr2=0x7fe4a815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4ad1b5640 1 --2- 172.21.15.106:0/480520380 >> v2:172.21.15.114:3300/0 conn(0x7fe4a80cf480 0x7fe4a815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4ad1b5640 1 -- 172.21.15.106:0/480520380 >> v1:172.21.15.114:6789/0 conn(0x7fe4a80ba170 legacy=0x7fe4a80cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4ad1b5640 1 -- 172.21.15.106:0/480520380 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a815f8e0 con 0x7fe4a80d3220 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4ad1b5640 1 --2- 172.21.15.106:0/480520380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80d3220 0x7fe4a80d36d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806e580 tx=0x7fe4a002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9deeba4ad12487 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4a77fe640 1 -- 172.21.15.106:0/480520380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a002cb20 con 0x7fe4a80d3220 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4a77fe640 1 -- 172.21.15.106:0/480520380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a002ccc0 con 0x7fe4a80d3220 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.661+0000 7fe4a77fe640 1 -- 172.21.15.106:0/480520380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0034610 con 0x7fe4a80d3220 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 -- 172.21.15.106:0/480520380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80d3220 msgr2=0x7fe4a80d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 --2- 172.21.15.106:0/480520380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80d3220 0x7fe4a80d36d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806e580 tx=0x7fe4a002bf80 comp rx=0 tx=0).stop 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 -- 172.21.15.106:0/480520380 shutdown_connections 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 --2- 172.21.15.106:0/480520380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80d3220 0x7fe4a80d36d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 --2- 172.21.15.106:0/480520380 >> v2:172.21.15.114:3300/0 conn(0x7fe4a80cf480 0x7fe4a815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 -- 172.21.15.106:0/480520380 >> 172.21.15.106:0/480520380 conn(0x7fe4a80d47d0 msgr2=0x7fe4a80d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 -- 172.21.15.106:0/480520380 shutdown_connections 2024-04-13T03:15:04.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.662+0000 7fe4aec18640 1 -- 172.21.15.106:0/480520380 wait complete. 2024-04-13T03:15:04.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.663+0000 7fe4aec18640 1 Processor -- start 2024-04-13T03:15:04.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.663+0000 7fe4aec18640 1 -- start start 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4a80cf480 0x7fe4a80ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4a80d3220 0x7fe4a80ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81592f0 con 0x7fe4a80ba170 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8159170 con 0x7fe4a80cf480 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4aec18640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a80d3c10 con 0x7fe4a80d3220 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36760/0 (socket says 172.21.15.106:36760) 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 -- 172.21.15.106:0/3807817489 learned_addr learned my addr 172.21.15.106:0/3807817489 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ad1b5640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4a80d3220 0x7fe4a80ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4a80cf480 0x7fe4a80ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 -- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4a80d3220 msgr2=0x7fe4a80ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4a80d3220 0x7fe4a80ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 -- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4a80cf480 msgr2=0x7fe4a80ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4a80cf480 0x7fe4a80ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.664+0000 7fe4ac9b4640 1 -- 172.21.15.106:0/3807817489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a81e7800 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4ac9b4640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d830 tx=0x7fe49000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000b6c0 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490019070 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a81e7ae0 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000ba80 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.665+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81e80a0 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.666+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a8068a10 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.666+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe49001e460 con 0x7fe4a80ba170 2024-04-13T03:15:04.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.666+0000 7fe4a57fa640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:04.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.666+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fe49008c3a0 con 0x7fe4a80ba170 2024-04-13T03:15:04.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.666+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:04.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.667+0000 7fe4a7fff640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 0x7fe48806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80ceb40 tx=0x7fe498009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:04.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.671+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe490059f10 con 0x7fe4a80ba170 2024-04-13T03:15:04.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.859+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4a8156e50 con 0x7fe4a80ba170 2024-04-13T03:15:04.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.859+0000 7fe4a57fa640 1 -- 172.21.15.106:0/3807817489 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) ==== 80+0+122 (secure 0 0 0) 0x7fe49005e210 con 0x7fe4a80ba170 2024-04-13T03:15:04.860 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.114:6834/390879687 2024-04-14T03:13:50.750656+0000 2024-04-13T03:15:04.860 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.114:6832/390879687 2024-04-14T03:13:50.750656+0000 2024-04-13T03:15:04.860 INFO:teuthology.orchestra.run.smithi106.stderr:listed 2 entries 2024-04-13T03:15:04.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 msgr2=0x7fe48806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 0x7fe48806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80ceb40 tx=0x7fe498009040 comp rx=0 tx=0).stop 2024-04-13T03:15:04.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 msgr2=0x7fe4a81e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d830 tx=0x7fe49000dcf0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 shutdown_connections 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4a80d3220 0x7fe4a80ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4a80cf480 0x7fe4a80ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 --2- 172.21.15.106:0/3807817489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a80ba170 0x7fe4a81e88a0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 >> 172.21.15.106:0/3807817489 conn(0x7fe4a80d47d0 msgr2=0x7fe4a81617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 shutdown_connections 2024-04-13T03:15:04.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:04.861+0000 7fe4aec18640 1 -- 172.21.15.106:0/3807817489 wait complete. 2024-04-13T03:15:04.872 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:15:04.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:04.873 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:04.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:15:04.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:04.902 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:15:04.958 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:15:04.958 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:15:04.958 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:04.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:15:04.977 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:15:04.977 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:04.977 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:05.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:15:05.013 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:15:05.013 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:15:05.013 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:05.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:15:05.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:05.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.152+0000 7f7842c36640 1 Processor -- start 2024-04-13T03:15:05.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.152+0000 7f7842c36640 1 -- start start 2024-04-13T03:15:05.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f7842c36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f783c06cfb0 0x7f783c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f7842c36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f783c0cc7c0 0x7f783c06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f7842c36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c15bb00 con 0x7f783c0ccd90 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f7842c36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f783c0ca9d0 con 0x7f783c06cfb0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f7842c36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f783c160050 con 0x7f783c0cc7c0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f783c06cfb0 0x7f783c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f783c06cfb0 0x7f783c06b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36766/0 (socket says 172.21.15.106:36766) 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 -- 172.21.15.106:0/111549266 learned_addr learned my addr 172.21.15.106:0/111549266 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f78411d3640 1 --2- 172.21.15.106:0/111549266 >> v2:172.21.15.114:3300/0 conn(0x7f783c0cc7c0 0x7f783c06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 -- 172.21.15.106:0/111549266 >> v2:172.21.15.114:3300/0 conn(0x7f783c0cc7c0 msgr2=0x7f783c06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 --2- 172.21.15.106:0/111549266 >> v2:172.21.15.114:3300/0 conn(0x7f783c0cc7c0 0x7f783c06be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 -- 172.21.15.106:0/111549266 >> v1:172.21.15.114:6789/0 conn(0x7f783c0ccd90 legacy=0x7f783c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.153+0000 7f783bfff640 1 -- 172.21.15.106:0/111549266 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c15f290 con 0x7f783c06cfb0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.154+0000 7f783bfff640 1 --2- 172.21.15.106:0/111549266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c06cfb0 0x7f783c06b8e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f783c0d1460 tx=0x7f782802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af904def979ad701 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.154+0000 7f783b7fe640 1 -- 172.21.15.106:0/111549266 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828034070 con 0x7f783c06cfb0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.154+0000 7f783b7fe640 1 -- 172.21.15.106:0/111549266 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828030860 con 0x7f783c06cfb0 2024-04-13T03:15:05.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.154+0000 7f783b7fe640 1 -- 172.21.15.106:0/111549266 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828030c30 con 0x7f783c06cfb0 2024-04-13T03:15:05.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 -- 172.21.15.106:0/111549266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c06cfb0 msgr2=0x7f783c06b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 --2- 172.21.15.106:0/111549266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c06cfb0 0x7f783c06b8e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f783c0d1460 tx=0x7f782802b9a0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 -- 172.21.15.106:0/111549266 shutdown_connections 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 --2- 172.21.15.106:0/111549266 >> v2:172.21.15.114:3300/0 conn(0x7f783c0cc7c0 0x7f783c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 --2- 172.21.15.106:0/111549266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c06cfb0 0x7f783c06b8e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 -- 172.21.15.106:0/111549266 >> 172.21.15.106:0/111549266 conn(0x7f783c0b1f10 msgr2=0x7f783c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 -- 172.21.15.106:0/111549266 shutdown_connections 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.155+0000 7f7842c36640 1 -- 172.21.15.106:0/111549266 wait complete. 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.156+0000 7f7842c36640 1 Processor -- start 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.156+0000 7f7842c36640 1 -- start start 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 0x7f783c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f783c0cc7c0 0x7f783c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f783c15fb30 con 0x7f783c0ccd90 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f783c15f850 con 0x7f783c06cfb0 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f7842c36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f783c06fb80 con 0x7f783c0cc7c0 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36774/0 (socket says 172.21.15.106:36774) 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 -- 172.21.15.106:0/592923604 learned_addr learned my addr 172.21.15.106:0/592923604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f783bfff640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f783c0cc7c0 0x7f783c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78409d2640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 0x7f783c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 -- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f783c0cc7c0 msgr2=0x7f783c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f783c0cc7c0 0x7f783c0b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 -- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 msgr2=0x7f783c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 0x7f783c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 -- 172.21.15.106:0/592923604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0c9e20 con 0x7f783c0ccd90 2024-04-13T03:15:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78409d2640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 0x7f783c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78411d3640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7834007e30 tx=0x7f783400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.157+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834014620 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.158+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834019070 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.158+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c15d460 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.158+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834014a00 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.158+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c15da20 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.159+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c068a10 con 0x7f783c0ccd90 2024-04-13T03:15:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.159+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7834014ba0 con 0x7f783c0ccd90 2024-04-13T03:15:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.159+0000 7f78397fa640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 0x7f782006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.159+0000 7f78409d2640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 0x7f782006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.159+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f783408c5c0 con 0x7f783c0ccd90 2024-04-13T03:15:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.160+0000 7f78409d2640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 0x7f782006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7830001660 tx=0x7f7830009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.164+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f783405a8a0 con 0x7f783c0ccd90 2024-04-13T03:15:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.374+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f783c073c30 con 0x7f783c0ccd90 2024-04-13T03:15:05.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.374+0000 7f78397fa640 1 -- 172.21.15.106:0/592923604 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5191 (secure 0 0 0) 0x7f783405a1a0 con 0x7f783c0ccd90 2024-04-13T03:15:05.375 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:05.376 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:14:33.765336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:15:05.376 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2024-04-13T03:15:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 msgr2=0x7f782006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 0x7f782006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7830001660 tx=0x7f7830009040 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 msgr2=0x7f783c15c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7834007e30 tx=0x7f783400e4b0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 shutdown_connections 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f782006c8f0 0x7f782006edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f783c0cc7c0 0x7f783c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f783c0ccd90 0x7f783c15c8f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 --2- 172.21.15.106:0/592923604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f783c06cfb0 0x7f783c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.376+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 >> 172.21.15.106:0/592923604 conn(0x7f783c0b1f10 msgr2=0x7f783c0caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.377+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 shutdown_connections 2024-04-13T03:15:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.377+0000 7f7842c36640 1 -- 172.21.15.106:0/592923604 wait complete. 2024-04-13T03:15:05.388 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:05.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.486+0000 7f9a244de640 1 Processor -- start 2024-04-13T03:15:05.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.486+0000 7f9a244de640 1 -- start start 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9a1c144f10 0x7f9a1c1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c0cad60 0x7f9a1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c0cd4d0 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c0cb190 con 0x7f9a1c144f10 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.487+0000 7f9a244de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0d1050 con 0x7f9a1c0cad60 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36778/0 (socket says 172.21.15.106:36778) 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 -- 172.21.15.106:0/1742488677 learned_addr learned my addr 172.21.15.106:0/1742488677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a22a7b640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c0cad60 0x7f9a1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a2227a640 1 --2- 172.21.15.106:0/1742488677 >> v2:172.21.15.114:3300/0 conn(0x7f9a1c144f10 0x7f9a1c1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 -- 172.21.15.106:0/1742488677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c0cad60 msgr2=0x7f9a1c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c0cad60 0x7f9a1c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 -- 172.21.15.106:0/1742488677 >> v2:172.21.15.114:3300/0 conn(0x7f9a1c144f10 msgr2=0x7f9a1c1630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1742488677 >> v2:172.21.15.114:3300/0 conn(0x7f9a1c144f10 0x7f9a1c1630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.488+0000 7f9a21a79640 1 -- 172.21.15.106:0/1742488677 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c0cd700 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c072b00 tx=0x7f9a1802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbb5b960c06df35 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a21278640 1 -- 172.21.15.106:0/1742488677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18034070 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a21278640 1 -- 172.21.15.106:0/1742488677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a18030840 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a21278640 1 -- 172.21.15.106:0/1742488677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18030c10 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 -- 172.21.15.106:0/1742488677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 msgr2=0x7f9a1c0ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c072b00 tx=0x7f9a1802b980 comp rx=0 tx=0).stop 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 -- 172.21.15.106:0/1742488677 shutdown_connections 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c0cad60 0x7f9a1c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 --2- 172.21.15.106:0/1742488677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c0ca820 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 --2- 172.21.15.106:0/1742488677 >> v2:172.21.15.114:3300/0 conn(0x7f9a1c144f10 0x7f9a1c1630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 -- 172.21.15.106:0/1742488677 >> 172.21.15.106:0/1742488677 conn(0x7f9a1c06f180 msgr2=0x7f9a1c06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 -- 172.21.15.106:0/1742488677 shutdown_connections 2024-04-13T03:15:05.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.489+0000 7f9a244de640 1 -- 172.21.15.106:0/1742488677 wait complete. 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 Processor -- start 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 -- start start 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 0x7f9a1c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c144f10 0x7f9a1c142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c0cf120 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c0cdcc0 con 0x7f9a1c0cad60 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a244de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c06c150 con 0x7f9a1c144f10 2024-04-13T03:15:05.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a21a79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a21a79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36784/0 (socket says 172.21.15.106:36784) 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.491+0000 7f9a21a79640 1 -- 172.21.15.106:0/1347132973 learned_addr learned my addr 172.21.15.106:0/1347132973 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a22a7b640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c144f10 0x7f9a1c142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a2227a640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 0x7f9a1c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 -- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c144f10 msgr2=0x7f9a1c142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c144f10 0x7f9a1c142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 -- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 msgr2=0x7f9a1c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 0x7f9a1c150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 -- 172.21.15.106:0/1347132973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c13b4d0 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a2227a640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 0x7f9a1c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a21a79640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9a1802b9c0 tx=0x7f9a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18048070 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a18030640 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c13b7b0 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18034040 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.492+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c13ca30 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.493+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c068a10 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.493+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1804b070 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.494+0000 7f9a12ffd640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.494+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a180add00 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.494+0000 7f9a2227a640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.494+0000 7f9a2227a640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 0x7f9a0006edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006fd0 tx=0x7f9a0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.498+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a18030810 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.674+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a1c0cb720 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.676+0000 7f9a12ffd640 1 -- 172.21.15.106:0/1347132973 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17110 (secure 0 0 0) 0x7f9a1807ba20 con 0x7f9a1c0d0bc0 2024-04-13T03:15:05.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:05.677 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:50.750679+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.677+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 msgr2=0x7f9a0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.677+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 0x7f9a0006edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006fd0 tx=0x7f9a0c008040 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.677+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 msgr2=0x7f9a1c1613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.677+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9a1802b9c0 tx=0x7f9a18004730 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 shutdown_connections 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9a1c144f10 0x7f9a1c142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a1c0d0bc0 0x7f9a1c1613e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 --2- 172.21.15.106:0/1347132973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a1c0cad60 0x7f9a1c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 >> 172.21.15.106:0/1347132973 conn(0x7f9a1c06f180 msgr2=0x7f9a1c06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 shutdown_connections 2024-04-13T03:15:05.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.678+0000 7f9a244de640 1 -- 172.21.15.106:0/1347132973 wait complete. 2024-04-13T03:15:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:13:35.329204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:13:36.059991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:15:05.690 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:15:05.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 Processor -- start 2024-04-13T03:15:05.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 -- start start 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe8240b4c20 0x7fe82415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe82406d220 con 0x7fe824145d20 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe82415d640 con 0x7fe8240b4c20 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.795+0000 7fe829cd5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe8240b3740 con 0x7fe8240b47e0 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:39790/0 (socket says 172.21.15.106:39790) 2024-04-13T03:15:05.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 -- 172.21.15.106:0/802200602 learned_addr learned my addr 172.21.15.106:0/802200602 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe823fff640 1 --2- 172.21.15.106:0/802200602 >> v2:172.21.15.114:3300/0 conn(0x7fe8240b4c20 0x7fe82415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 -- 172.21.15.106:0/802200602 >> v2:172.21.15.114:3300/0 conn(0x7fe8240b4c20 msgr2=0x7fe82415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 --2- 172.21.15.106:0/802200602 >> v2:172.21.15.114:3300/0 conn(0x7fe8240b4c20 0x7fe82415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 -- 172.21.15.106:0/802200602 >> v1:172.21.15.114:6789/0 conn(0x7fe824145d20 legacy=0x7fe82406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.796+0000 7fe822ffd640 1 -- 172.21.15.106:0/802200602 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8240b3a20 con 0x7fe8240b47e0 2024-04-13T03:15:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.797+0000 7fe822ffd640 1 --2- 172.21.15.106:0/802200602 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe818009870 tx=0x7fe81802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed904e2383d1b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.797+0000 7fe8227fc640 1 -- 172.21.15.106:0/802200602 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818034070 con 0x7fe8240b47e0 2024-04-13T03:15:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.797+0000 7fe8227fc640 1 -- 172.21.15.106:0/802200602 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe818030840 con 0x7fe8240b47e0 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.797+0000 7fe8227fc640 1 -- 172.21.15.106:0/802200602 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818030c10 con 0x7fe8240b47e0 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 -- 172.21.15.106:0/802200602 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 msgr2=0x7fe82415cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 --2- 172.21.15.106:0/802200602 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe818009870 tx=0x7fe81802b980 comp rx=0 tx=0).stop 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 -- 172.21.15.106:0/802200602 shutdown_connections 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 --2- 172.21.15.106:0/802200602 >> v2:172.21.15.114:3300/0 conn(0x7fe8240b4c20 0x7fe82415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 --2- 172.21.15.106:0/802200602 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b47e0 0x7fe82415cbc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 -- 172.21.15.106:0/802200602 >> 172.21.15.106:0/802200602 conn(0x7fe8240b2600 msgr2=0x7fe8240d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 -- 172.21.15.106:0/802200602 shutdown_connections 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.798+0000 7fe829cd5640 1 -- 172.21.15.106:0/802200602 wait complete. 2024-04-13T03:15:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.799+0000 7fe829cd5640 1 Processor -- start 2024-04-13T03:15:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.799+0000 7fe829cd5640 1 -- start start 2024-04-13T03:15:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8240b47e0 0x7fe82413b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b4c20 0x7fe82415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe8240b42c0 con 0x7fe824145d20 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe8240b3fe0 con 0x7fe8240b47e0 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe829cd5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe824146100 con 0x7fe8240b4c20 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36790/0 (socket says 172.21.15.106:36790) 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 -- 172.21.15.106:0/1778221640 learned_addr learned my addr 172.21.15.106:0/1778221640 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe822ffd640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b4c20 0x7fe82415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe8237fe640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8240b47e0 0x7fe82413b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 -- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b4c20 msgr2=0x7fe82415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b4c20 0x7fe82415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 -- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8240b47e0 msgr2=0x7fe82413b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8240b47e0 0x7fe82413b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 -- 172.21.15.106:0/1778221640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe824142210 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe823fff640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00efc0 tx=0x7fe81c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c018020 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.800+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c014480 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.801+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8241424f0 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.801+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c014870 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.801+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe824142ab0 con 0x7fe824145d20 2024-04-13T03:15:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.802+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe824068a10 con 0x7fe824145d20 2024-04-13T03:15:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.802+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe81c005d10 con 0x7fe824145d20 2024-04-13T03:15:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.802+0000 7fe80bfff640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.803+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fe81c08c7a0 con 0x7fe824145d20 2024-04-13T03:15:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.803+0000 7fe8237fe640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.803+0000 7fe8237fe640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 0x7fe81006edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe814001500 tx=0x7fe814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:05.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.807+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe81c022070 con 0x7fe824145d20 2024-04-13T03:15:05.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.980+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe82415e3f0 con 0x7fe824145d20 2024-04-13T03:15:05.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.981+0000 7fe80bfff640 1 -- 172.21.15.106:0/1778221640 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fe81c010040 con 0x7fe824145d20 2024-04-13T03:15:05.982 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 msgr2=0x7fe81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 0x7fe81006edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe814001500 tx=0x7fe814009040 comp rx=0 tx=0).stop 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 msgr2=0x7fe82415f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00efc0 tx=0x7fe81c00c490 comp rx=0 tx=0).stop 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 shutdown_connections 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe81006c8f0 0x7fe81006edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe8240b4c20 0x7fe82415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe824145d20 0x7fe82415f2a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 --2- 172.21.15.106:0/1778221640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8240b47e0 0x7fe82413b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:05.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 >> 172.21.15.106:0/1778221640 conn(0x7fe8240b2600 msgr2=0x7fe82406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:05.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 shutdown_connections 2024-04-13T03:15:05.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:05.983+0000 7fe829cd5640 1 -- 172.21.15.106:0/1778221640 wait complete. 2024-04-13T03:15:05.995 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:15:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.098+0000 7f4b58e80640 1 Processor -- start 2024-04-13T03:15:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.098+0000 7f4b58e80640 1 -- start start 2024-04-13T03:15:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b58e80640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b5406db90 0x7f4b54072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b58e80640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5406c880 con 0x7f4b54145f60 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b58e80640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5406caf0 con 0x7f4b5406e020 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b58e80640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5406cdd0 con 0x7f4b5406db90 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4b54145f60 0x7f4b5406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41888/0 (socket says 172.21.15.106:41888) 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5259c640 1 -- 172.21.15.106:0/2355695304 learned_addr learned my addr 172.21.15.106:0/2355695304 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b51d9b640 1 --2- 172.21.15.106:0/2355695304 >> v2:172.21.15.114:3300/0 conn(0x7f4b5406db90 0x7f4b54072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181071950 0 0) 0x7f4b5406c880 con 0x7f4b54145f60 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b3c003520 con 0x7f4b54145f60 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2411171626 0 0) 0x7f4b3c003520 con 0x7f4b54145f60 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 >> v2:172.21.15.114:3300/0 conn(0x7f4b5406db90 msgr2=0x7f4b54072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 --2- 172.21.15.106:0/2355695304 >> v2:172.21.15.114:3300/0 conn(0x7f4b5406db90 0x7f4b54072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.099+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 >> v1:172.21.15.114:6789/0 conn(0x7f4b5406e020 legacy=0x7f4b5415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5406d0b0 con 0x7f4b54145f60 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4b440019f0 con 0x7f4b54145f60 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 >> v1:172.21.15.106:6789/0 conn(0x7f4b54145f60 legacy=0x7f4b5406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b3c0039b0 0x7f4b3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5406cdd0 con 0x7f4b3c0051c0 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00c060 con 0x7f4b3c004800 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c00c260 con 0x7f4b3c0039b0 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b5259c640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b3c0039b0 0x7f4b3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b51d9b640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 -- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b3c0039b0 msgr2=0x7f4b3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b3c0039b0 0x7f4b3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 -- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c004800 msgr2=0x7f4b3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.100+0000 7f4b52d9d640 1 -- 172.21.15.106:0/2355695304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c003520 con 0x7f4b3c0051c0 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b52d9d640 1 -- 172.21.15.106:0/2355695304 reap_dead start 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b52d9d640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c001720 tx=0x7f4b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612fef649f5d4c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c001370 con 0x7f4b3c0051c0 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0052d0 con 0x7f4b3c0051c0 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b5159a640 1 -- 172.21.15.106:0/2355695304 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c003830 con 0x7f4b3c0051c0 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b58e80640 1 -- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 msgr2=0x7f4b3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b58e80640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c001720 tx=0x7f4b3c005830 comp rx=0 tx=0).stop 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b58e80640 1 -- 172.21.15.106:0/2355695304 shutdown_connections 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b58e80640 1 --2- 172.21.15.106:0/2355695304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.101+0000 7f4b58e80640 1 -- 172.21.15.106:0/2355695304 >> 172.21.15.106:0/2355695304 conn(0x7f4b5406f0d0 msgr2=0x7f4b5406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.102+0000 7f4b58e80640 1 -- 172.21.15.106:0/2355695304 shutdown_connections 2024-04-13T03:15:06.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.102+0000 7f4b58e80640 1 -- 172.21.15.106:0/2355695304 wait complete. 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.103+0000 7f4b58e80640 1 Processor -- start 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.103+0000 7f4b58e80640 1 -- start start 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.103+0000 7f4b58e80640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c0051c0 0x7f4b5415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.103+0000 7f4b58e80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b541f5da0 0x7f4b541f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540cdaa0 con 0x7f4b5415b860 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b54146340 con 0x7f4b3c0051c0 2024-04-13T03:15:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540cdc20 con 0x7f4b541f5da0 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36816/0 (socket says 172.21.15.106:36816) 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 -- 172.21.15.106:0/3470749166 learned_addr learned my addr 172.21.15.106:0/3470749166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b5259c640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c0051c0 0x7f4b5415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b52d9d640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b541f5da0 0x7f4b541f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 -- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b541f5da0 msgr2=0x7f4b541f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b541f5da0 0x7f4b541f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 -- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c0051c0 msgr2=0x7f4b5415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c0051c0 0x7f4b5415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 -- 172.21.15.106:0/3470749166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5413fe00 con 0x7f4b5415b860 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b51d9b640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4b480118e0 tx=0x7f4b48011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800faf0 con 0x7f4b5415b860 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4800fc90 con 0x7f4b5415b860 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b541400e0 con 0x7f4b5415b860 2024-04-13T03:15:06.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48014440 con 0x7f4b5415b860 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.104+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541f6770 con 0x7f4b5415b860 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.105+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5413d710 con 0x7f4b5415b860 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.106+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b48012040 con 0x7f4b5415b860 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.106+0000 7f4b42ffd640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 0x7f4b3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.106+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f4b4808c220 con 0x7f4b5415b860 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.106+0000 7f4b5259c640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 0x7f4b3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.107+0000 7f4b5259c640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 0x7f4b3406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004d00 tx=0x7f4b44006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.111+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b48019030 con 0x7f4b5415b860 2024-04-13T03:15:06.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.284+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b540b0e80 con 0x7f4b5415b860 2024-04-13T03:15:06.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.284+0000 7f4b42ffd640 1 -- 172.21.15.106:0/3470749166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f4b48059eb0 con 0x7f4b5415b860 2024-04-13T03:15:06.286 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 msgr2=0x7f4b3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 0x7f4b3406edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004d00 tx=0x7f4b44006040 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 msgr2=0x7f4b5413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4b480118e0 tx=0x7f4b48011da0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 shutdown_connections 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4b3406c8f0 0x7f4b3406edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b541f5da0 0x7f4b541f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b5415b860 0x7f4b5413d1d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 --2- 172.21.15.106:0/3470749166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b3c0051c0 0x7f4b5415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.286+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 >> 172.21.15.106:0/3470749166 conn(0x7f4b5406f0d0 msgr2=0x7f4b54145290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.287+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 shutdown_connections 2024-04-13T03:15:06.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.287+0000 7f4b58e80640 1 -- 172.21.15.106:0/3470749166 wait complete. 2024-04-13T03:15:06.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:15:06.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.399+0000 7fc73057b640 1 Processor -- start 2024-04-13T03:15:06.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.399+0000 7fc73057b640 1 -- start start 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.400+0000 7fc73057b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.400+0000 7fc73057b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc72806f800 0x7fc72806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.400+0000 7fc73057b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72815c400 con 0x7fc72806e8a0 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.400+0000 7fc73057b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7280cf700 con 0x7fc72806f800 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.400+0000 7fc73057b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc72815c6e0 con 0x7fc728159120 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:39838/0 (socket says 172.21.15.106:39838) 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 -- 172.21.15.106:0/785930557 learned_addr learned my addr 172.21.15.106:0/785930557 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72eb18640 1 --2- 172.21.15.106:0/785930557 >> v2:172.21.15.114:3300/0 conn(0x7fc72806f800 0x7fc72806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 -- 172.21.15.106:0/785930557 >> v2:172.21.15.114:3300/0 conn(0x7fc72806f800 msgr2=0x7fc72806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 --2- 172.21.15.106:0/785930557 >> v2:172.21.15.114:3300/0 conn(0x7fc72806f800 0x7fc72806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 -- 172.21.15.106:0/785930557 >> v1:172.21.15.114:6789/0 conn(0x7fc72806e8a0 legacy=0x7fc72815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.401+0000 7fc72e317640 1 -- 172.21.15.106:0/785930557 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72815c9c0 con 0x7fc728159120 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.402+0000 7fc72e317640 1 --2- 172.21.15.106:0/785930557 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc724009e50 tx=0x7fc72402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4a428b97855094 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.402+0000 7fc72d315640 1 -- 172.21.15.106:0/785930557 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72402cb00 con 0x7fc728159120 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.402+0000 7fc72d315640 1 -- 172.21.15.106:0/785930557 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72402cca0 con 0x7fc728159120 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.402+0000 7fc72d315640 1 -- 172.21.15.106:0/785930557 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724034610 con 0x7fc728159120 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 -- 172.21.15.106:0/785930557 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 msgr2=0x7fc7280b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 --2- 172.21.15.106:0/785930557 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc724009e50 tx=0x7fc72402bf60 comp rx=0 tx=0).stop 2024-04-13T03:15:06.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 -- 172.21.15.106:0/785930557 shutdown_connections 2024-04-13T03:15:06.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 --2- 172.21.15.106:0/785930557 >> v2:172.21.15.114:3300/0 conn(0x7fc72806f800 0x7fc72806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 --2- 172.21.15.106:0/785930557 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc728159120 0x7fc7280b4430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 -- 172.21.15.106:0/785930557 >> 172.21.15.106:0/785930557 conn(0x7fc7280d1b60 msgr2=0x7fc72806f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 -- 172.21.15.106:0/785930557 shutdown_connections 2024-04-13T03:15:06.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.403+0000 7fc73057b640 1 -- 172.21.15.106:0/785930557 wait complete. 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 Processor -- start 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 -- start start 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc72806e8a0 0x7fc72806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 0x7fc7281e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.404+0000 7fc73057b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7280cf320 con 0x7fc72806f800 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc73057b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc728159500 con 0x7fc728159120 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc73057b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc7280cfb90 con 0x7fc72806e8a0 2024-04-13T03:15:06.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36832/0 (socket says 172.21.15.106:36832) 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 -- 172.21.15.106:0/2601193251 learned_addr learned my addr 172.21.15.106:0/2601193251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72e317640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc72806e8a0 0x7fc72806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72eb18640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 0x7fc7281e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 -- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc72806e8a0 msgr2=0x7fc72806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc72806e8a0 0x7fc72806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 -- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 msgr2=0x7fc7281e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 0x7fc7281e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 -- 172.21.15.106:0/2601193251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7281e5ba0 con 0x7fc72806f800 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72eb18640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 0x7fc7281e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc72db16640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c430 tx=0x7fc71800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7180052d0 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc718005490 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281e5e80 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718007260 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.405+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72806c5f0 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.406+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc728069120 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.407+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc718007400 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.407+0000 7fc71effd640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.407+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fc71808b1c0 con 0x7fc72806f800 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.407+0000 7fc72e317640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.408+0000 7fc72e317640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc724035000 tx=0x7fc724030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.411+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc718058dc0 con 0x7fc72806f800 2024-04-13T03:15:06.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.586+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc72806ff30 con 0x7fc72806f800 2024-04-13T03:15:06.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.587+0000 7fc71effd640 1 -- 172.21.15.106:0/2601193251 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fc71805d0c0 con 0x7fc72806f800 2024-04-13T03:15:06.590 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 msgr2=0x7fc70c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc724035000 tx=0x7fc724030000 comp rx=0 tx=0).stop 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 msgr2=0x7fc728146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c430 tx=0x7fc71800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 shutdown_connections 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc728159120 0x7fc7281e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc72806f800 0x7fc728146e90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 --2- 172.21.15.106:0/2601193251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc72806e8a0 0x7fc72806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 >> 172.21.15.106:0/2601193251 conn(0x7fc7280d1b60 msgr2=0x7fc7280cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 shutdown_connections 2024-04-13T03:15:06.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.589+0000 7fc73057b640 1 -- 172.21.15.106:0/2601193251 wait complete. 2024-04-13T03:15:06.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.700+0000 7f3db6ec8640 1 Processor -- start 2024-04-13T03:15:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.700+0000 7f3db6ec8640 1 -- start start 2024-04-13T03:15:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3db0156b50 0x7f3db0160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 0x7f3db015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 0x7f3db006e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3db015d690 con 0x7f3db0156b50 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3db015d970 con 0x7f3db006dd70 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db6ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3db0160410 con 0x7f3db015cc10 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db4c64640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3db0156b50 0x7f3db0160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db4c64640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3db0156b50 0x7f3db0160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36838/0 (socket says 172.21.15.106:36838) 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db4c64640 1 -- 172.21.15.106:0/3442976732 learned_addr learned my addr 172.21.15.106:0/3442976732 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3daffff640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 0x7f3db015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.701+0000 7f3db5465640 1 --2- 172.21.15.106:0/3442976732 >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 0x7f3db006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 -- 172.21.15.106:0/3442976732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 msgr2=0x7f3db015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 0x7f3db015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 -- 172.21.15.106:0/3442976732 >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 msgr2=0x7f3db006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3442976732 >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 0x7f3db006e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 -- 172.21.15.106:0/3442976732 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db01606f0 con 0x7f3db0156b50 2024-04-13T03:15:06.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3daffff640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 0x7f3db015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:06.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db5465640 1 --2- 172.21.15.106:0/3442976732 >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 0x7f3db006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:06.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db0156b50 0x7f3db0160c10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3db015a750 tx=0x7f3da002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c2ce18cfc7cc2e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3daf7fe640 1 -- 172.21.15.106:0/3442976732 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0035020 con 0x7f3db0156b50 2024-04-13T03:15:06.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.702+0000 7f3daf7fe640 1 -- 172.21.15.106:0/3442976732 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da0031870 con 0x7f3db0156b50 2024-04-13T03:15:06.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3daf7fe640 1 -- 172.21.15.106:0/3442976732 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0031c40 con 0x7f3db0156b50 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3442976732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db0156b50 msgr2=0x7f3db0160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db0156b50 0x7f3db0160c10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3db015a750 tx=0x7f3da002b910 comp rx=0 tx=0).stop 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3442976732 shutdown_connections 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3442976732 >> v2:172.21.15.114:3300/0 conn(0x7f3db006dd70 0x7f3db006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db015cc10 0x7f3db015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3442976732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db0156b50 0x7f3db0160c10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3442976732 >> 172.21.15.106:0/3442976732 conn(0x7f3db00b1f10 msgr2=0x7f3db00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3442976732 shutdown_connections 2024-04-13T03:15:06.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.703+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3442976732 wait complete. 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.704+0000 7f3db6ec8640 1 Processor -- start 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.704+0000 7f3db6ec8640 1 -- start start 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 0x7f3db00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 0x7f3db01f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00afc20 con 0x7f3db015cc10 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00afaa0 con 0x7f3db006dd70 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db6ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3db006bcb0 con 0x7f3db0156b50 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db5465640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db5465640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36842/0 (socket says 172.21.15.106:36842) 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db5465640 1 -- 172.21.15.106:0/3443759135 learned_addr learned my addr 172.21.15.106:0/3443759135 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 0x7f3db00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.705+0000 7f3daffff640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 0x7f3db01f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 -- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 msgr2=0x7f3db01f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 0x7f3db01f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 -- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 msgr2=0x7f3db00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 0x7f3db00cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 -- 172.21.15.106:0/3443759135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00cc140 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 0x7f3db00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3daffff640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 0x7f3db01f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db5465640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007db0 tx=0x7f3da800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da801a070 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da8003990 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8003d60 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db00cc420 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.706+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db00cc9e0 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.707+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db015a5a0 con 0x7f3db015cc10 2024-04-13T03:15:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.707+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3da801d070 con 0x7f3db015cc10 2024-04-13T03:15:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.708+0000 7f3dad7fa640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 0x7f3d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.708+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3da808bf50 con 0x7f3db015cc10 2024-04-13T03:15:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.708+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 0x7f3d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.708+0000 7f3db4c64640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 0x7f3d9c06edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3da0036000 tx=0x7f3da003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.712+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3db015a5a0 con 0x7f3db015cc10 2024-04-13T03:15:06.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.927+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3db01610c0 con 0x7f3db015cc10 2024-04-13T03:15:06.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.927+0000 7f3dad7fa640 1 -- 172.21.15.106:0/3443759135 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5191 (secure 0 0 0) 0x7f3db01610c0 con 0x7f3db015cc10 2024-04-13T03:15:06.928 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:06.928 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:14:33.765336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:15:06.929 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2024-04-13T03:15:06.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 msgr2=0x7f3d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 0x7f3d9c06edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3da0036000 tx=0x7f3da003c040 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 msgr2=0x7f3db01f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007db0 tx=0x7f3da800d4a0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 shutdown_connections 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3d9c06c8f0 0x7f3d9c06edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3db0156b50 0x7f3db01f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db015cc10 0x7f3db01f8ad0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 --2- 172.21.15.106:0/3443759135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3db006dd70 0x7f3db00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 >> 172.21.15.106:0/3443759135 conn(0x7f3db00b1f10 msgr2=0x7f3db00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 shutdown_connections 2024-04-13T03:15:06.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:06.929+0000 7f3db6ec8640 1 -- 172.21.15.106:0/3443759135 wait complete. 2024-04-13T03:15:06.941 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:07.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.040+0000 7fcda131e640 1 Processor -- start 2024-04-13T03:15:07.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.040+0000 7fcda131e640 1 -- start start 2024-04-13T03:15:07.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 0x7fcd9c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 0x7fcd9c0d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c0cfda0 con 0x7fcd9c0b3980 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c0cf900 con 0x7fcd9c0cefe0 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcda131e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c06bfe0 con 0x7fcd9c0d2880 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36846/0 (socket says 172.21.15.106:36846) 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 -- 172.21.15.106:0/2574687457 learned_addr learned my addr 172.21.15.106:0/2574687457 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd9b7fe640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 0x7fcd9c0d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd9affd640 1 --2- 172.21.15.106:0/2574687457 >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 0x7fcd9c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 -- 172.21.15.106:0/2574687457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 msgr2=0x7fcd9c0d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 0x7fcd9c0d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 -- 172.21.15.106:0/2574687457 >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 msgr2=0x7fcd9c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2574687457 >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 0x7fcd9c06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd927fc640 1 -- 172.21.15.106:0/2574687457 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c06c2c0 con 0x7fcd9c0b3980 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd9b7fe640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 0x7fcd9c0d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.041+0000 7fcd9affd640 1 --2- 172.21.15.106:0/2574687457 >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 0x7fcd9c06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06e8f0 tx=0x7fcd8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d950c7a5279b180 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcd9a7fc640 1 -- 172.21.15.106:0/2574687457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84035020 con 0x7fcd9c0b3980 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcd9a7fc640 1 -- 172.21.15.106:0/2574687457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84031870 con 0x7fcd9c0b3980 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcd9a7fc640 1 -- 172.21.15.106:0/2574687457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84031c40 con 0x7fcd9c0b3980 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcda131e640 1 -- 172.21.15.106:0/2574687457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 msgr2=0x7fcd9c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcda131e640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06e8f0 tx=0x7fcd8402b910 comp rx=0 tx=0).stop 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcda131e640 1 -- 172.21.15.106:0/2574687457 shutdown_connections 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.042+0000 7fcda131e640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0d2880 0x7fcd9c0d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.043+0000 7fcda131e640 1 --2- 172.21.15.106:0/2574687457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c0cf3c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.043+0000 7fcda131e640 1 --2- 172.21.15.106:0/2574687457 >> v2:172.21.15.114:3300/0 conn(0x7fcd9c0cefe0 0x7fcd9c06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.043+0000 7fcda131e640 1 -- 172.21.15.106:0/2574687457 >> 172.21.15.106:0/2574687457 conn(0x7fcd9c0d4780 msgr2=0x7fcd9c0d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.043+0000 7fcda131e640 1 -- 172.21.15.106:0/2574687457 shutdown_connections 2024-04-13T03:15:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.043+0000 7fcda131e640 1 -- 172.21.15.106:0/2574687457 wait complete. 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 Processor -- start 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 -- start start 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 0x7fcd9c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd9c0d2880 0x7fcd9c1600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c06cb60 con 0x7fcd9c0b3980 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c06c880 con 0x7fcd9c0d2880 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.044+0000 7fcda131e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c06d080 con 0x7fcd9c0cefe0 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36858/0 (socket says 172.21.15.106:36858) 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 -- 172.21.15.106:0/2643938475 learned_addr learned my addr 172.21.15.106:0/2643938475 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 -- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 msgr2=0x7fcd9c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 0x7fcd9c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 0x7fcd9c0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9b7fe640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd9c0d2880 0x7fcd9c1600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 -- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd9c0d2880 msgr2=0x7fcd9c1600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd9c0d2880 0x7fcd9c1600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 -- 172.21.15.106:0/2643938475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c0ca430 con 0x7fcd9c0b3980 2024-04-13T03:15:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 0x7fcd9c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd9affd640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c7f0 tx=0x7fcd8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c015070 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c007820 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c0ca710 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c007bd0 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.045+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c0cacd0 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.046+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c068a10 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.047+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd8c018070 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.047+0000 7fcd93fff640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.047+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fcd8c08bb90 con 0x7fcd9c0b3980 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.047+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.048+0000 7fcd927fc640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd84036000 tx=0x7fcd84008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.051+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd8c059e70 con 0x7fcd9c0b3980 2024-04-13T03:15:07.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.229+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd9c0d35a0 con 0x7fcd9c0b3980 2024-04-13T03:15:07.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.230+0000 7fcd93fff640 1 -- 172.21.15.106:0/2643938475 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17110 (secure 0 0 0) 0x7fcd8c059770 con 0x7fcd9c0b3980 2024-04-13T03:15:07.231 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:07.232 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:50.750679+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:07.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 msgr2=0x7fcd7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd84036000 tx=0x7fcd84008030 comp rx=0 tx=0).stop 2024-04-13T03:15:07.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 msgr2=0x7fcd9c14a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c7f0 tx=0x7fcd8c00ccb0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 shutdown_connections 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd9c0cefe0 0x7fcd9c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd9c0d2880 0x7fcd9c1600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 --2- 172.21.15.106:0/2643938475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd9c0b3980 0x7fcd9c14a4e0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 >> 172.21.15.106:0/2643938475 conn(0x7fcd9c0d4780 msgr2=0x7fcd9c0b9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 shutdown_connections 2024-04-13T03:15:07.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.232+0000 7fcda131e640 1 -- 172.21.15.106:0/2643938475 wait complete. 2024-04-13T03:15:07.244 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:15:07.244 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.343+0000 7ff466dcc640 1 Processor -- start 2024-04-13T03:15:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.343+0000 7ff466dcc640 1 -- start start 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff466dcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff460167900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff466dcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600aff20 con 0x7ff46006c8f0 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff466dcc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600cbaa0 con 0x7ff4600cadb0 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff466dcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff46006fac0 con 0x7ff4600b65f0 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff45ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff46006c8f0 0x7ff4601641d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41890/0 (socket says 172.21.15.106:41890) 2024-04-13T03:15:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff45ffff640 1 -- 172.21.15.106:0/3814252939 learned_addr learned my addr 172.21.15.106:0/3814252939 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff465369640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff460167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.344+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026967082 0 0) 0x7ff4600aff20 con 0x7ff46006c8f0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff448003520 con 0x7ff46006c8f0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3330008845 0 0) 0x7ff448003520 con 0x7ff46006c8f0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 msgr2=0x7ff460167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff460167900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 >> v1:172.21.15.114:6789/0 conn(0x7ff4600cadb0 legacy=0x7ff4600cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600afc40 con 0x7ff46006c8f0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7ff454001670 con 0x7ff46006c8f0 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 >> v1:172.21.15.106:6789/0 conn(0x7ff46006c8f0 legacy=0x7ff4601641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff448004390 0x7ff448004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff4480050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff46006fac0 con 0x7ff4480039e0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff44800b9f0 con 0x7ff448004390 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff44800bbf0 con 0x7ff4600b65f0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff448004390 0x7ff448004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff465369640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff4480050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 -- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 msgr2=0x7ff4480050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff4600b65f0 0x7ff4480050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 -- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff448004390 msgr2=0x7ff448004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff448004390 0x7ff448004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff464b68640 1 -- 172.21.15.106:0/3814252939 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff448003520 con 0x7ff4480039e0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.345+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff448004390 0x7ff448004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.346+0000 7ff464b68640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff450001860 tx=0x7ff448005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f5d8eed313baaf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.346+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45002f050 con 0x7ff4480039e0 2024-04-13T03:15:07.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.346+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff450033650 con 0x7ff4480039e0 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.346+0000 7ff45f7fe640 1 -- 172.21.15.106:0/3814252939 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450001430 con 0x7ff4480039e0 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 -- 172.21.15.106:0/3814252939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 msgr2=0x7ff448003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3814252939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff450001860 tx=0x7ff448005830 comp rx=0 tx=0).stop 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff465369640 1 -- 172.21.15.106:0/3814252939 reap_dead start 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 -- 172.21.15.106:0/3814252939 shutdown_connections 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 -- 172.21.15.106:0/3814252939 >> 172.21.15.106:0/3814252939 conn(0x7ff4600cd1e0 msgr2=0x7ff4600cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 -- 172.21.15.106:0/3814252939 shutdown_connections 2024-04-13T03:15:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.347+0000 7ff466dcc640 1 -- 172.21.15.106:0/3814252939 wait complete. 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.348+0000 7ff466dcc640 1 Processor -- start 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.348+0000 7ff466dcc640 1 -- start start 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 0x7ff4601573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff460157920 0x7ff460158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff46006f750 con 0x7ff4480039e0 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff46006f470 con 0x7ff4601f6cc0 2024-04-13T03:15:07.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff4600af6f0 con 0x7ff460157920 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36876/0 (socket says 172.21.15.106:36876) 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 -- 172.21.15.106:0/3521081733 learned_addr learned my addr 172.21.15.106:0/3521081733 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff465369640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff460157920 0x7ff460158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 0x7ff4601573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 -- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff460157920 msgr2=0x7ff460158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff460157920 0x7ff460158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 -- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 msgr2=0x7ff4601573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 0x7ff4601573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 -- 172.21.15.106:0/3521081733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4601f6220 con 0x7ff4480039e0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 0x7ff4601573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff464b68640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff450001980 tx=0x7ff450001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450041070 con 0x7ff4480039e0 2024-04-13T03:15:07.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff450007050 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46014e7a0 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450007400 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.349+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4601f64c0 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.350+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff460068a10 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.351+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff450044070 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.351+0000 7ff45d7fa640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 0x7ff44c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.351+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7ff4500ad850 con 0x7ff4480039e0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.351+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 0x7ff44c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.352+0000 7ff45ffff640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 0x7ff44c06ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454001180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.356+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff45004b030 con 0x7ff4480039e0 2024-04-13T03:15:07.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.570+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff460070050 con 0x7ff4480039e0 2024-04-13T03:15:07.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.571+0000 7ff45d7fa640 1 -- 172.21.15.106:0/3521081733 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5191 (secure 0 0 0) 0x7ff450044360 con 0x7ff4480039e0 2024-04-13T03:15:07.572 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:07.572 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:14:33.765336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:15:07.572 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.572+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 msgr2=0x7ff44c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.572+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 0x7ff44c06ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454001180 comp rx=0 tx=0).stop 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.572+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 msgr2=0x7ff46014e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.572+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff450001980 tx=0x7ff450001230 comp rx=0 tx=0).stop 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 shutdown_connections 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff44c06c920 0x7ff44c06ede0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff460157920 0x7ff460158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4601f6cc0 0x7ff4601573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 --2- 172.21.15.106:0/3521081733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4480039e0 0x7ff46014e260 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 >> 172.21.15.106:0/3521081733 conn(0x7ff4600cd1e0 msgr2=0x7ff46006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 shutdown_connections 2024-04-13T03:15:07.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.573+0000 7ff466dcc640 1 -- 172.21.15.106:0/3521081733 wait complete. 2024-04-13T03:15:07.584 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:07.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.683+0000 7f6e527bb640 1 Processor -- start 2024-04-13T03:15:07.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.683+0000 7f6e527bb640 1 -- start start 2024-04-13T03:15:07.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e527bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e4c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e527bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6e4c0b8080 0x7f6e4c1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e527bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4c0d3530 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e527bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6e4c1626b0 con 0x7f6e4c0b8080 2024-04-13T03:15:07.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e527bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4c0d0b60 con 0x7f6e4c1622d0 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e4b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6e4c0d1c00 0x7f6e4c0b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41900/0 (socket says 172.21.15.106:41900) 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e4b7fe640 1 -- 172.21.15.106:0/2408623075 learned_addr learned my addr 172.21.15.106:0/2408623075 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e4bfff640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e4c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e50d58640 1 --2- 172.21.15.106:0/2408623075 >> v2:172.21.15.114:3300/0 conn(0x7f6e4c0b8080 0x7f6e4c1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668840340 0 0) 0x7f6e4c0d3530 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.684+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e34003520 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 861445626 0 0) 0x7f6e34003520 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 msgr2=0x7f6e4c0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e4c0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 >> v2:172.21.15.114:3300/0 conn(0x7f6e4c0b8080 msgr2=0x7f6e4c1688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 --2- 172.21.15.106:0/2408623075 >> v2:172.21.15.114:3300/0 conn(0x7f6e4c0b8080 0x7f6e4c1688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c0d0e40 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6e40001670 con 0x7f6e4c0d1c00 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 >> v1:172.21.15.106:6789/0 conn(0x7f6e4c0d1c00 legacy=0x7f6e4c0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e340039b0 0x7f6e34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0d0b60 con 0x7f6e34004800 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3400b9c0 con 0x7f6e340039b0 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3400bb40 con 0x7f6e4c1622d0 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4bfff640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e340039b0 0x7f6e34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e50d58640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 -- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 msgr2=0x7f6e340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 -- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e340039b0 msgr2=0x7f6e34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e340039b0 0x7f6e34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e4b7fe640 1 -- 172.21.15.106:0/2408623075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34003520 con 0x7f6e34004800 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.685+0000 7f6e50d58640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c1622d0 0x7f6e340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6e40007fd0 tx=0x7f6e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97d137e1f25301e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40007d50 con 0x7f6e34004800 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40030070 con 0x7f6e34004800 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e4affd640 1 -- 172.21.15.106:0/2408623075 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4002cd40 con 0x7f6e34004800 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e527bb640 1 -- 172.21.15.106:0/2408623075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 msgr2=0x7f6e34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e527bb640 1 --2- 172.21.15.106:0/2408623075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6e40007fd0 tx=0x7f6e34005830 comp rx=0 tx=0).stop 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.686+0000 7f6e50d58640 1 -- 172.21.15.106:0/2408623075 reap_dead start 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.687+0000 7f6e527bb640 1 -- 172.21.15.106:0/2408623075 shutdown_connections 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.687+0000 7f6e527bb640 1 -- 172.21.15.106:0/2408623075 >> 172.21.15.106:0/2408623075 conn(0x7f6e4c0d25d0 msgr2=0x7f6e4c0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.687+0000 7f6e527bb640 1 -- 172.21.15.106:0/2408623075 shutdown_connections 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.687+0000 7f6e527bb640 1 -- 172.21.15.106:0/2408623075 wait complete. 2024-04-13T03:15:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.688+0000 7f6e527bb640 1 Processor -- start 2024-04-13T03:15:07.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.688+0000 7f6e527bb640 1 -- start start 2024-04-13T03:15:07.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c1f7ad0 0x7f6e4c1f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c155380 0x7f6e4c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0d16e0 con 0x7f6e34004800 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0d1400 con 0x7f6e4c1f7ad0 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4c0cc6f0 con 0x7f6e4c155380 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36900/0 (socket says 172.21.15.106:36900) 2024-04-13T03:15:07.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 -- 172.21.15.106:0/687426999 learned_addr learned my addr 172.21.15.106:0/687426999 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e50d58640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c155380 0x7f6e4c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c1f7ad0 0x7f6e4c1f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 -- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c155380 msgr2=0x7f6e4c155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c155380 0x7f6e4c155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 -- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c1f7ad0 msgr2=0x7f6e4c1f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c1f7ad0 0x7f6e4c1f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 -- 172.21.15.106:0/687426999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c154370 con 0x7f6e34004800 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e4bfff640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0118e0 tx=0x7f6e3c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00faf0 con 0x7f6e34004800 2024-04-13T03:15:07.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00fc90 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c156050 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c014440 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.689+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c159730 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.690+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c159240 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.691+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e3c012040 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.691+0000 7f6e48ff9640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 0x7f6e3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.691+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f6e3c08c260 con 0x7f6e34004800 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.691+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 0x7f6e3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.692+0000 7f6e4b7fe640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 0x7f6e3806ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6e40034000 tx=0x7f6e40007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.695+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e3c019030 con 0x7f6e34004800 2024-04-13T03:15:07.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.906+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e4c068a10 con 0x7f6e34004800 2024-04-13T03:15:07.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.907+0000 7f6e48ff9640 1 -- 172.21.15.106:0/687426999 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5191 (secure 0 0 0) 0x7f6e3c059ef0 con 0x7f6e34004800 2024-04-13T03:15:07.908 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:07.908 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:13:37.103773+0000","modified":"2024-04-13T03:14:33.765336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.106:6837/2254289128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2254289128},{"type":"v1","addr":"172.21.15.106:6837","nonce":2254289128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-04-13T03:15:07.908 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2024-04-13T03:15:07.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 msgr2=0x7f6e3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 0x7f6e3806ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6e40034000 tx=0x7f6e40007000 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 msgr2=0x7f6e4c153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0118e0 tx=0x7f6e3c011da0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 shutdown_connections 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e3806c920 0x7f6e3806ede0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e4c155380 0x7f6e4c155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c1f7ad0 0x7f6e4c1f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 --2- 172.21.15.106:0/687426999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e34004800 0x7f6e4c153e30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.908+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 >> 172.21.15.106:0/687426999 conn(0x7f6e4c0d25d0 msgr2=0x7f6e4c0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:07.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.909+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 shutdown_connections 2024-04-13T03:15:07.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:07.909+0000 7f6e527bb640 1 -- 172.21.15.106:0/687426999 wait complete. 2024-04-13T03:15:07.920 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:08.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.020+0000 7f9af2236640 1 Processor -- start 2024-04-13T03:15:08.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.020+0000 7f9af2236640 1 -- start start 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9af2236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9aec070090 0x7f9aec160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9af2236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec070470 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9af2236640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec15da00 con 0x7f9aec0b0390 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9af2236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9aec0ceb50 con 0x7f9aec070090 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aeb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9aec070090 0x7f9aec160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aebfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9aec0cf5a0 0x7f9aec0cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41906/0 (socket says 172.21.15.106:41906) 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aebfff640 1 -- 172.21.15.106:0/2194114246 learned_addr learned my addr 172.21.15.106:0/2194114246 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aeb7fe640 1 -- 172.21.15.106:0/2194114246 >> v1:172.21.15.114:6789/0 conn(0x7f9aec0b0390 legacy=0x7f9aec160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aeb7fe640 1 -- 172.21.15.106:0/2194114246 >> v1:172.21.15.106:6789/0 conn(0x7f9aec0cf5a0 legacy=0x7f9aec0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.021+0000 7f9aeb7fe640 1 -- 172.21.15.106:0/2194114246 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec15dcc0 con 0x7f9aec070090 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9aeb7fe640 1 --2- 172.21.15.106:0/2194114246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec070090 0x7f9aec160570 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0097a0 tx=0x7f9adc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571c2a9417ef6abb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9aea7fc640 1 -- 172.21.15.106:0/2194114246 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0315a0 con 0x7f9aec070090 2024-04-13T03:15:08.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9aea7fc640 1 -- 172.21.15.106:0/2194114246 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc037070 con 0x7f9aec070090 2024-04-13T03:15:08.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9aea7fc640 1 -- 172.21.15.106:0/2194114246 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc031960 con 0x7f9aec070090 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9af2236640 1 -- 172.21.15.106:0/2194114246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec070090 msgr2=0x7f9aec160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.022+0000 7f9af2236640 1 --2- 172.21.15.106:0/2194114246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec070090 0x7f9aec160570 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0097a0 tx=0x7f9adc02b840 comp rx=0 tx=0).stop 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.023+0000 7f9af2236640 1 -- 172.21.15.106:0/2194114246 shutdown_connections 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.023+0000 7f9af2236640 1 --2- 172.21.15.106:0/2194114246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec070090 0x7f9aec160570 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.023+0000 7f9af2236640 1 -- 172.21.15.106:0/2194114246 >> 172.21.15.106:0/2194114246 conn(0x7f9aec0cd6d0 msgr2=0x7f9aec0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.023+0000 7f9af2236640 1 -- 172.21.15.106:0/2194114246 shutdown_connections 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.023+0000 7f9af2236640 1 -- 172.21.15.106:0/2194114246 wait complete. 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.024+0000 7f9af2236640 1 Processor -- start 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.024+0000 7f9af2236640 1 -- start start 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aec070090 0x7f9aec1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9aec0b0390 0x7f9aec1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec06e560 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0cf110 con 0x7f9aec070090 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec0cd220 con 0x7f9aec0b0390 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36918/0 (socket says 172.21.15.106:36918) 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 -- 172.21.15.106:0/3252850239 learned_addr learned my addr 172.21.15.106:0/3252850239 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aeaffd640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9aec0b0390 0x7f9aec1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aeb7fe640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aec070090 0x7f9aec1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 -- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9aec0b0390 msgr2=0x7f9aec1458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9aec0b0390 0x7f9aec1458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 -- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aec070090 msgr2=0x7f9aec1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aec070090 0x7f9aec1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 -- 172.21.15.106:0/3252850239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec15ee50 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9aebfff640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400ca80 tx=0x7f9ae400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4017070 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae400ac30 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec15f0d0 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4012600 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.025+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec15f690 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.026+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec068a10 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.027+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ae401a070 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.027+0000 7f9ad3fff640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 0x7f9ad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.027+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9ae408b8e0 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.027+0000 7f9aeb7fe640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 0x7f9ad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.027+0000 7f9aeb7fe640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 0x7f9ad806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9adc036000 tx=0x7f9adc03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.031+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9aec15ee50 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.207+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9aec0d0840 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.209+0000 7f9ad3fff640 1 -- 172.21.15.106:0/3252850239 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17110 (secure 0 0 0) 0x7f9ae400ae20 con 0x7f9aec0cf5a0 2024-04-13T03:15:08.210 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:08.210 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:13:50.750679+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:13:35.329204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-13T03:13:36.059991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 msgr2=0x7f9ad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 0x7f9ad806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9adc036000 tx=0x7f9adc03d040 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 msgr2=0x7f9aec15e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400ca80 tx=0x7f9ae400cf40 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 shutdown_connections 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.210+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9ad806c8f0 0x7f9ad806edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9aec0b0390 0x7f9aec1458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9aec0cf5a0 0x7f9aec15e910 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 --2- 172.21.15.106:0/3252850239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aec070090 0x7f9aec1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 >> 172.21.15.106:0/3252850239 conn(0x7f9aec0cd6d0 msgr2=0x7f9aec152570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:08.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 shutdown_connections 2024-04-13T03:15:08.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.211+0000 7f9af2236640 1 -- 172.21.15.106:0/3252850239 wait complete. 2024-04-13T03:15:08.223 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:15:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.323+0000 7f58e5eb7640 1 Processor -- start 2024-04-13T03:15:08.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.323+0000 7f58e5eb7640 1 -- start start 2024-04-13T03:15:08.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58e5eb7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f58e015e8c0 0x7f58e00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58e5eb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e00b5060 con 0x7f58e015eca0 2024-04-13T03:15:08.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58e5eb7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e00b5340 con 0x7f58e0073520 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58e5eb7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f58e006c500 con 0x7f58e015e8c0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58deffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f58e015eca0 0x7f58e0072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41922/0 (socket says 172.21.15.106:41922) 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58deffd640 1 -- 172.21.15.106:0/1613721312 learned_addr learned my addr 172.21.15.106:0/1613721312 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58df7fe640 1 --2- 172.21.15.106:0/1613721312 >> v2:172.21.15.114:3300/0 conn(0x7f58e015e8c0 0x7f58e00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068985738 0 0) 0x7f58e00b5060 con 0x7f58e015eca0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.324+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58c8003520 con 0x7f58e015eca0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3393542826 0 0) 0x7f58c8003520 con 0x7f58e015eca0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 >> v2:172.21.15.114:3300/0 conn(0x7f58e015e8c0 msgr2=0x7f58e00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 --2- 172.21.15.106:0/1613721312 >> v2:172.21.15.114:3300/0 conn(0x7f58e015e8c0 0x7f58e00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 >> v1:172.21.15.114:6789/0 conn(0x7f58e0073520 legacy=0x7f58e00b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e006c7e0 con 0x7f58e015eca0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f58d4001670 con 0x7f58e015eca0 2024-04-13T03:15:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 >> v1:172.21.15.106:6789/0 conn(0x7f58e015eca0 legacy=0x7f58e0072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58e006c500 con 0x7f58c8004800 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58c800c070 con 0x7f58c80051c0 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58c800c270 con 0x7f58c80039b0 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58df7fe640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58dffff640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 -- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58c80039b0 msgr2=0x7f58c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 -- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58c80051c0 msgr2=0x7f58c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58deffd640 1 -- 172.21.15.106:0/1613721312 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c8003520 con 0x7f58c8004800 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58df7fe640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58dffff640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.325+0000 7f58dffff640 1 -- 172.21.15.106:0/1613721312 reap_dead start 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58deffd640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c240 tx=0x7f58c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8507d106743b1361 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4007cf0 con 0x7f58c8004800 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4030070 con 0x7f58c8004800 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58de7fc640 1 -- 172.21.15.106:0/1613721312 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4005d30 con 0x7f58c8004800 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58e5eb7640 1 -- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 msgr2=0x7f58c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c240 tx=0x7f58c8005830 comp rx=0 tx=0).stop 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58e5eb7640 1 -- 172.21.15.106:0/1613721312 shutdown_connections 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/1613721312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.326+0000 7f58e5eb7640 1 -- 172.21.15.106:0/1613721312 >> 172.21.15.106:0/1613721312 conn(0x7f58e006f0d0 msgr2=0x7f58e006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.327+0000 7f58e5eb7640 1 -- 172.21.15.106:0/1613721312 shutdown_connections 2024-04-13T03:15:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.327+0000 7f58e5eb7640 1 -- 172.21.15.106:0/1613721312 wait complete. 2024-04-13T03:15:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.328+0000 7f58e5eb7640 1 Processor -- start 2024-04-13T03:15:08.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.328+0000 7f58e5eb7640 1 -- start start 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.328+0000 7f58e5eb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.328+0000 7f58e5eb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e01612b0 0x7f58e0161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e0148840 0x7f58e0148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58e00ce5e0 con 0x7f58c8004800 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58e00ce460 con 0x7f58e0148840 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f58e0072e10 con 0x7f58e01612b0 2024-04-13T03:15:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36930/0 (socket says 172.21.15.106:36930) 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 -- 172.21.15.106:0/710169663 learned_addr learned my addr 172.21.15.106:0/710169663 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58dffff640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e0148840 0x7f58e0148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 -- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e01612b0 msgr2=0x7f58e0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e01612b0 0x7f58e0161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 -- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e0148840 msgr2=0x7f58e0148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e0148840 0x7f58e0148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 -- 172.21.15.106:0/710169663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e015ab30 con 0x7f58c8004800 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58df7fe640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f58d00114e0 tx=0x7f58d00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0015870 con 0x7f58c8004800 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0015a10 con 0x7f58c8004800 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e015ae10 con 0x7f58c8004800 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0015e00 con 0x7f58c8004800 2024-04-13T03:15:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.329+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e015b6b0 con 0x7f58c8004800 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.330+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e015b200 con 0x7f58c8004800 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.330+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58d001d400 con 0x7f58c8004800 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.331+0000 7f58c7fff640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 0x7f58cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.331+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f58d008c6f0 con 0x7f58c8004800 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.331+0000 7f58deffd640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 0x7f58cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.331+0000 7f58deffd640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 0x7f58cc06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d4039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.335+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58d0019030 con 0x7f58c8004800 2024-04-13T03:15:08.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.548+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f58e0068a10 con 0x7f58c8004800 2024-04-13T03:15:08.549 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:15:08.548+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:15:08.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.550+0000 7f58c7fff640 1 -- 172.21.15.106:0/710169663 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f58d005a8a0 con 0x7f58c8004800 2024-04-13T03:15:08.550 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 msgr2=0x7f58cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 0x7f58cc06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d4039040 comp rx=0 tx=0).stop 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 msgr2=0x7f58e00b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f58d00114e0 tx=0x7f58d00119a0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 shutdown_connections 2024-04-13T03:15:08.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f58cc06c920 0x7f58cc06ede0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f58e01612b0 0x7f58e0161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58e0148840 0x7f58e0148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 --2- 172.21.15.106:0/710169663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58c8004800 0x7f58e00b2410 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 >> 172.21.15.106:0/710169663 conn(0x7f58e006f0d0 msgr2=0x7f58e00cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 shutdown_connections 2024-04-13T03:15:08.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.553+0000 7f58e5eb7640 1 -- 172.21.15.106:0/710169663 wait complete. 2024-04-13T03:15:08.558 INFO:tasks.ceph.mds.c.smithi106.stderr: -3507> 2024-04-13T03:13:58.025+0000 7f7606df1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:15:08.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:15:08.588 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:15:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.670+0000 7fe251908640 1 Processor -- start 2024-04-13T03:15:08.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.670+0000 7fe251908640 1 -- start start 2024-04-13T03:15:08.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.670+0000 7fe251908640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe24c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe251908640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe24c0cfd20 con 0x7fe24c0b3d60 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe251908640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe24c162af0 con 0x7fe24c15ed90 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe251908640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c162dd0 con 0x7fe24c15e9b0 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe24c0b3d60 0x7fe24c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41944/0 (socket says 172.21.15.106:41944) 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24b7fe640 1 -- 172.21.15.106:0/4201635189 learned_addr learned my addr 172.21.15.106:0/4201635189 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841627078 0 0) 0x7fe24c0cfd20 con 0x7fe24c0b3d60 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe238003520 con 0x7fe24c0b3d60 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24affd640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe24c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338739591 0 0) 0x7fe238003520 con 0x7fe24c0b3d60 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 msgr2=0x7fe24c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe24c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 >> v1:172.21.15.114:6789/0 conn(0x7fe24c15ed90 legacy=0x7fe24c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1630b0 con 0x7fe24c0b3d60 2024-04-13T03:15:08.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fe244001a10 con 0x7fe24c0b3d60 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 >> v1:172.21.15.106:6789/0 conn(0x7fe24c0b3d60 legacy=0x7fe24c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe2380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe238004d40 0x7fe2380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c162dd0 con 0x7fe238003f60 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe23800bb30 con 0x7fe238004d40 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe23800bd30 con 0x7fe24c15e9b0 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe2427fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24affd640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe2380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.671+0000 7fe24b7fe640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe238004d40 0x7fe2380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 -- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 msgr2=0x7fe2380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c15e9b0 0x7fe2380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 -- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe238004d40 msgr2=0x7fe2380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe238004d40 0x7fe2380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 -- 172.21.15.106:0/4201635189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe238003520 con 0x7fe238003f60 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe24b7fe640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe238004d40 0x7fe2380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe2427fc640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe238004800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe234001770 tx=0x7fe238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a9f7649cbc454 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.672+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340013c0 con 0x7fe238003f60 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2340052d0 con 0x7fe238003f60 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe24a7fc640 1 -- 172.21.15.106:0/4201635189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234003640 con 0x7fe238003f60 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 -- 172.21.15.106:0/4201635189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 msgr2=0x7fe238004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 --2- 172.21.15.106:0/4201635189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe238004800 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe234001770 tx=0x7fe238005830 comp rx=0 tx=0).stop 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe24b7fe640 1 -- 172.21.15.106:0/4201635189 reap_dead start 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 -- 172.21.15.106:0/4201635189 shutdown_connections 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 -- 172.21.15.106:0/4201635189 >> 172.21.15.106:0/4201635189 conn(0x7fe24c08bf10 msgr2=0x7fe24c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:08.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 -- 172.21.15.106:0/4201635189 shutdown_connections 2024-04-13T03:15:08.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.673+0000 7fe251908640 1 -- 172.21.15.106:0/4201635189 wait complete. 2024-04-13T03:15:08.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.674+0000 7fe251908640 1 Processor -- start 2024-04-13T03:15:08.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.674+0000 7fe251908640 1 -- start start 2024-04-13T03:15:08.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c0c8120 0x7fe24c1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24c1f5280 0x7fe24c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c162800 con 0x7fe238003f60 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c162680 con 0x7fe24c1f5280 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe251908640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c0d4d00 con 0x7fe24c0c8120 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe24affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe24affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36948/0 (socket says 172.21.15.106:36948) 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe24affd640 1 -- 172.21.15.106:0/3142881756 learned_addr learned my addr 172.21.15.106:0/3142881756 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe24b7fe640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24c1f5280 0x7fe24c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.675+0000 7fe2427fc640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c0c8120 0x7fe24c1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 -- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c0c8120 msgr2=0x7fe24c1f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c0c8120 0x7fe24c1f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 -- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24c1f5280 msgr2=0x7fe24c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24c1f5280 0x7fe24c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 -- 172.21.15.106:0/3142881756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c159fc0 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe24affd640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0119d0 tx=0x7fe23c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00faf0 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23c00fc90 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c014440 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1f5770 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.676+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c0b5540 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.677+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c1f5c90 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.677+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe23c012040 con 0x7fe238003f60 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.677+0000 7fe248ff9640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 0x7fe22c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:08.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.678+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6273+0+0 (secure 0 0 0) 0x7fe23c08c350 con 0x7fe238003f60 2024-04-13T03:15:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.678+0000 7fe2427fc640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 0x7fe22c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.678+0000 7fe2427fc640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 0x7fe22c06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe234033000 tx=0x7fe234009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.682+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe24c1f5c90 con 0x7fe238003f60 2024-04-13T03:15:08.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:08.892+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe24c068a10 con 0x7fe238003f60 2024-04-13T03:15:09.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.551+0000 7fe248ff9640 1 -- 172.21.15.106:0/3142881756 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7fe23c019030 con 0x7fe238003f60 2024-04-13T03:15:09.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 msgr2=0x7fe22c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 0x7fe22c06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe234033000 tx=0x7fe234009030 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 msgr2=0x7fe24c159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0119d0 tx=0x7fe23c011e90 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 shutdown_connections 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe22c06c8f0 0x7fe22c06edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe24c0c8120 0x7fe24c1f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24c1f5280 0x7fe24c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 --2- 172.21.15.106:0/3142881756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe238003f60 0x7fe24c159a80 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.553+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 >> 172.21.15.106:0/3142881756 conn(0x7fe24c08bf10 msgr2=0x7fe24c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.554+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 shutdown_connections 2024-04-13T03:15:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.554+0000 7fe251908640 1 -- 172.21.15.106:0/3142881756 wait complete. 2024-04-13T03:15:09.566 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:15:09.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.711+0000 7f512a99c640 1 Processor -- start 2024-04-13T03:15:09.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.711+0000 7f512a99c640 1 -- start start 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f512a99c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f512415b480 0x7f512415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f512a99c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512415ff50 con 0x7f5124072db0 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f512a99c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512406d620 con 0x7f51240d0570 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f512a99c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f512406e230 con 0x7f512415b480 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f51237fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5124072db0 0x7f512415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41958/0 (socket says 172.21.15.106:41958) 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f51237fe640 1 -- 172.21.15.106:0/3360952592 learned_addr learned my addr 172.21.15.106:0/3360952592 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599805290 0 0) 0x7f512415ff50 con 0x7f5124072db0 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5123fff640 1 --2- 172.21.15.106:0/3360952592 >> v2:172.21.15.114:3300/0 conn(0x7f512415b480 0x7f512415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f510c003520 con 0x7f5124072db0 2024-04-13T03:15:09.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729920901 0 0) 0x7f510c003520 con 0x7f5124072db0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 >> v2:172.21.15.114:3300/0 conn(0x7f512415b480 msgr2=0x7f512415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 --2- 172.21.15.106:0/3360952592 >> v2:172.21.15.114:3300/0 conn(0x7f512415b480 0x7f512415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 >> v1:172.21.15.114:6789/0 conn(0x7f51240d0570 legacy=0x7f512415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512406d900 con 0x7f5124072db0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5118001670 con 0x7f5124072db0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.712+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 >> v1:172.21.15.106:6789/0 conn(0x7f5124072db0 legacy=0x7f512415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f510c0039b0 0x7f510c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c004800 0x7f510c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 0x7f510c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f512406e230 con 0x7f510c0051c0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f510c00c160 con 0x7f510c0039b0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f510c00c360 con 0x7f510c004800 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 0x7f510c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5123fff640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f510c0039b0 0x7f510c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f51237fe640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c004800 0x7f510c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 -- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c004800 msgr2=0x7f510c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c004800 0x7f510c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 -- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f510c0039b0 msgr2=0x7f510c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f510c0039b0 0x7f510c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5128f39640 1 -- 172.21.15.106:0/3360952592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c003520 con 0x7f510c0051c0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.713+0000 7f5123fff640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f510c0039b0 0x7f510c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f5128f39640 1 -- 172.21.15.106:0/3360952592 reap_dead start 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f5128f39640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 0x7f510c00b9c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f511c001e50 tx=0x7f510c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568b815ae78654ad server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c001a70 con 0x7f510c0051c0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c003bb0 con 0x7f510c0051c0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f5122ffd640 1 -- 172.21.15.106:0/3360952592 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0050d0 con 0x7f510c0051c0 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 -- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 msgr2=0x7f510c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 0x7f510c00b9c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f511c001e50 tx=0x7f510c005830 comp rx=0 tx=0).stop 2024-04-13T03:15:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 -- 172.21.15.106:0/3360952592 shutdown_connections 2024-04-13T03:15:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 --2- 172.21.15.106:0/3360952592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f510c0051c0 0x7f510c00b9c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 -- 172.21.15.106:0/3360952592 >> 172.21.15.106:0/3360952592 conn(0x7f512406f0f0 msgr2=0x7f512406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 -- 172.21.15.106:0/3360952592 shutdown_connections 2024-04-13T03:15:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.714+0000 7f512a99c640 1 -- 172.21.15.106:0/3360952592 wait complete. 2024-04-13T03:15:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.715+0000 7f512a99c640 1 Processor -- start 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 -- start start 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c0051c0 0x7f5124154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51240b2390 0x7f51240b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f512406d280 con 0x7f5124157e00 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f512406dec0 con 0x7f51240b2390 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f512a99c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f512406d400 con 0x7f510c0051c0 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f5128f39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f5128f39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36976/0 (socket says 172.21.15.106:36976) 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.716+0000 7f5128f39640 1 -- 172.21.15.106:0/1120129518 learned_addr learned my addr 172.21.15.106:0/1120129518 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5123fff640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c0051c0 0x7f5124154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 -- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c0051c0 msgr2=0x7f5124154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f51237fe640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51240b2390 0x7f51240b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c0051c0 0x7f5124154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 -- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51240b2390 msgr2=0x7f51240b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51240b2390 0x7f51240b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 -- 172.21.15.106:0/1120129518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51240cb9a0 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5128f39640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f511c001f70 tx=0x7f511c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c006d60 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c007030 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51240cc020 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0073e0 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.717+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241f7cd0 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.718+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5124069120 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.718+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f511c007580 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.719+0000 7f5120ff9640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 0x7f511006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.719+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6273+0+0 (secure 0 0 0) 0x7f511c0ad990 con 0x7f5124157e00 2024-04-13T03:15:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.719+0000 7f5123fff640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 0x7f511006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.719+0000 7f5123fff640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 0x7f511006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f51140098b0 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.723+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f511c033030 con 0x7f5124157e00 2024-04-13T03:15:09.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:09.896+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5124073e30 con 0x7f5124157e00 2024-04-13T03:15:10.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.555+0000 7f5120ff9640 1 -- 172.21.15.106:0/1120129518 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) ==== 181+0+0 (secure 0 0 0) 0x7f511c02e3e0 con 0x7f5124157e00 2024-04-13T03:15:10.556 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 msgr2=0x7f511006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 0x7f511006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f51140098b0 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 msgr2=0x7f5124158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f511c001f70 tx=0x7f511c0014a0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 shutdown_connections 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f511006c8f0 0x7f511006edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5124157e00 0x7f5124158280 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51240b2390 0x7f51240b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 --2- 172.21.15.106:0/1120129518 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f510c0051c0 0x7f5124154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.558+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 >> 172.21.15.106:0/1120129518 conn(0x7f512406f0f0 msgr2=0x7f512415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:10.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.559+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 shutdown_connections 2024-04-13T03:15:10.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.559+0000 7f512a99c640 1 -- 172.21.15.106:0/1120129518 wait complete. 2024-04-13T03:15:10.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:15:10.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.679+0000 7f31b90ad640 1 Processor -- start 2024-04-13T03:15:10.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.679+0000 7f31b90ad640 1 -- start start 2024-04-13T03:15:10.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b90ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31b406bff0 0x7f31b406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:10.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b90ad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40cef80 con 0x7f31b40d07c0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b90ad640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40cf260 con 0x7f31b415fb50 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b90ad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f31b40cf540 con 0x7f31b406bff0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31b406bff0 0x7f31b406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b359e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f31b40d07c0 0x7f31b406ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41974/0 (socket says 172.21.15.106:41974) 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b359e640 1 -- 172.21.15.106:0/569119805 learned_addr learned my addr 172.21.15.106:0/569119805 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b259c640 1 -- 172.21.15.106:0/569119805 >> v1:172.21.15.114:6789/0 conn(0x7f31b415fb50 legacy=0x7f31b406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b259c640 1 -- 172.21.15.106:0/569119805 >> v1:172.21.15.106:6789/0 conn(0x7f31b40d07c0 legacy=0x7f31b406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b1d9b640 1 -- 172.21.15.106:0/569119805 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283004232 0 0) 0x7f31b40cef80 con 0x7f31b40d07c0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.680+0000 7f31b259c640 1 -- 172.21.15.106:0/569119805 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40cf820 con 0x7f31b406bff0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.681+0000 7f31b259c640 1 --2- 172.21.15.106:0/569119805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b406bff0 0x7f31b406e6b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4a74bed14544f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.681+0000 7f31b1d9b640 1 -- 172.21.15.106:0/569119805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a80315a0 con 0x7f31b406bff0 2024-04-13T03:15:10.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.681+0000 7f31b1d9b640 1 -- 172.21.15.106:0/569119805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a8037070 con 0x7f31b406bff0 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.681+0000 7f31b1d9b640 1 -- 172.21.15.106:0/569119805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8031960 con 0x7f31b406bff0 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.681+0000 7f31b90ad640 1 -- 172.21.15.106:0/569119805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b406bff0 msgr2=0x7f31b406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 --2- 172.21.15.106:0/569119805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b406bff0 0x7f31b406e6b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802b840 comp rx=0 tx=0).stop 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 -- 172.21.15.106:0/569119805 shutdown_connections 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 --2- 172.21.15.106:0/569119805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b406bff0 0x7f31b406e6b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 -- 172.21.15.106:0/569119805 >> 172.21.15.106:0/569119805 conn(0x7f31b40cd6a0 msgr2=0x7f31b40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 -- 172.21.15.106:0/569119805 shutdown_connections 2024-04-13T03:15:10.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.682+0000 7f31b90ad640 1 -- 172.21.15.106:0/569119805 wait complete. 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.683+0000 7f31b90ad640 1 Processor -- start 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.683+0000 7f31b90ad640 1 -- start start 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 0x7f31b41518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f31b40d07c0 0x7f31b4156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40ced20 con 0x7f31b415fb50 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40ceba0 con 0x7f31b406bff0 2024-04-13T03:15:10.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b90ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f31b415ef10 con 0x7f31b40d07c0 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36996/0 (socket says 172.21.15.106:36996) 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 -- 172.21.15.106:0/768334255 learned_addr learned my addr 172.21.15.106:0/768334255 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b259c640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f31b40d07c0 0x7f31b4156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b2d9d640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 0x7f31b41518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 -- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f31b40d07c0 msgr2=0x7f31b4156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f31b40d07c0 0x7f31b4156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 -- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 msgr2=0x7f31b41518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 0x7f31b41518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 -- 172.21.15.106:0/768334255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41f5020 con 0x7f31b415fb50 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b2d9d640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 0x7f31b41518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f31b359e640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c780 tx=0x7f31ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:10.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac018070 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.684+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac00a640 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.685+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b40b3cd0 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.685+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00aab0 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.685+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b41574e0 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.685+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b4068a10 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.686+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31ac01b070 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.686+0000 7f319b7fe640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 0x7f31a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.686+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5835+0+0 (secure 0 0 0) 0x7f31ac08b1f0 con 0x7f31b415fb50 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.686+0000 7f31b2d9d640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 0x7f31a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:10.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.687+0000 7f31b2d9d640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 0x7f31a006edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f31a400a810 tx=0x7f31a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:10.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.691+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31ac0595a0 con 0x7f31b415fb50 2024-04-13T03:15:10.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:10.864+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f31b4073630 con 0x7f31b415fb50 2024-04-13T03:15:11.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.562+0000 7f319b7fe640 1 -- 172.21.15.106:0/768334255 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) ==== 169+0+0 (secure 0 0 0) 0x7f31ac058ea0 con 0x7f31b415fb50 2024-04-13T03:15:11.563 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 msgr2=0x7f31a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 0x7f31a006edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f31a400a810 tx=0x7f31a4008040 comp rx=0 tx=0).stop 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 msgr2=0x7f31b41610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c780 tx=0x7f31ac00cc40 comp rx=0 tx=0).stop 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 shutdown_connections 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f31a006c8f0 0x7f31a006edb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f31b40d07c0 0x7f31b4156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31b415fb50 0x7f31b41610b0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 --2- 172.21.15.106:0/768334255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31b406bff0 0x7f31b41518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 >> 172.21.15.106:0/768334255 conn(0x7f31b40cd6a0 msgr2=0x7f31b40b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:11.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 shutdown_connections 2024-04-13T03:15:11.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.564+0000 7f31b90ad640 1 -- 172.21.15.106:0/768334255 wait complete. 2024-04-13T03:15:11.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:15:11.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.681+0000 7f18690de640 1 Processor -- start 2024-04-13T03:15:11.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.681+0000 7f18690de640 1 -- start start 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f18690de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1864070c90 0x7f18640cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f18690de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f186415a4e0 0x7f18640ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f18690de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186415dd40 con 0x7f186415a100 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f18690de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f18640c80e0 con 0x7f186415a4e0 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f18690de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1864160670 con 0x7f1864070c90 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f1862d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f186415a100 0x7f1864070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:41978/0 (socket says 172.21.15.106:41978) 2024-04-13T03:15:11.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.682+0000 7f1862d9d640 1 -- 172.21.15.106:0/1209211403 learned_addr learned my addr 172.21.15.106:0/1209211403 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1864070c90 0x7f18640cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186359e640 1 --2- 172.21.15.106:0/1209211403 >> v2:172.21.15.114:3300/0 conn(0x7f186415a4e0 0x7f18640ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791817331 0 0) 0x7f186415dd40 con 0x7f186415a100 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f184c003520 con 0x7f186415a100 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267929310 0 0) 0x7f184c003520 con 0x7f186415a100 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1864070c90 msgr2=0x7f18640cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1864070c90 0x7f18640cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 >> v2:172.21.15.114:3300/0 conn(0x7f186415a4e0 msgr2=0x7f18640ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 --2- 172.21.15.106:0/1209211403 >> v2:172.21.15.114:3300/0 conn(0x7f186415a4e0 0x7f18640ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1864160400 con 0x7f186415a100 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f185c001670 con 0x7f186415a100 2024-04-13T03:15:11.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186359e640 1 --2- 172.21.15.106:0/1209211403 >> v2:172.21.15.114:3300/0 conn(0x7f186415a4e0 0x7f18640ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 >> v1:172.21.15.106:6789/0 conn(0x7f186415a100 legacy=0x7f1864070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0039b0 0x7f184c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c004800 0x7f184c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f184c0051c0 0x7f184c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18640c80e0 con 0x7f184c004800 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1864160670 con 0x7f184c0039b0 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f184c00bf00 con 0x7f184c0051c0 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1864070c90 0x7f18640cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f1862d9d640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0039b0 0x7f184c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c004800 0x7f184c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186359e640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f184c0051c0 0x7f184c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 -- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f184c0051c0 msgr2=0x7f184c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f184c0051c0 0x7f184c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 -- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0039b0 msgr2=0x7f184c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0039b0 0x7f184c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.683+0000 7f186259c640 1 -- 172.21.15.106:0/1209211403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c003520 con 0x7f184c004800 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.684+0000 7f186259c640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c004800 0x7f184c004c80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f185800e1f0 tx=0x7f184c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97541fa34a843688 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.684+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858009d40 con 0x7f184c004800 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.684+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185802cce0 con 0x7f184c004800 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.684+0000 7f1861d9b640 1 -- 172.21.15.106:0/1209211403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858003bb0 con 0x7f184c004800 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 -- 172.21.15.106:0/1209211403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c004800 msgr2=0x7f184c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 --2- 172.21.15.106:0/1209211403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c004800 0x7f184c004c80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f185800e1f0 tx=0x7f184c005830 comp rx=0 tx=0).stop 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f186359e640 1 -- 172.21.15.106:0/1209211403 reap_dead start 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 -- 172.21.15.106:0/1209211403 shutdown_connections 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 -- 172.21.15.106:0/1209211403 >> 172.21.15.106:0/1209211403 conn(0x7f18640d1650 msgr2=0x7f18640d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:11.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 -- 172.21.15.106:0/1209211403 shutdown_connections 2024-04-13T03:15:11.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.685+0000 7f18690de640 1 -- 172.21.15.106:0/1209211403 wait complete. 2024-04-13T03:15:11.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.686+0000 7f18690de640 1 Processor -- start 2024-04-13T03:15:11.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.686+0000 7f18690de640 1 -- start start 2024-04-13T03:15:11.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c004800 0x7f1864155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 0x7f18641f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1864161150 con 0x7f18640b1c80 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1864160c30 con 0x7f184c004800 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f18690de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1864156370 con 0x7f18641f5b80 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37020/0 (socket says 172.21.15.106:37020) 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 -- 172.21.15.106:0/975096361 learned_addr learned my addr 172.21.15.106:0/975096361 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f1862d9d640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c004800 0x7f1864155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186359e640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 0x7f18641f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 -- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 msgr2=0x7f18641f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 0x7f18641f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 -- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c004800 msgr2=0x7f1864155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c004800 0x7f1864155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 -- 172.21.15.106:0/975096361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18641f68b0 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186359e640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 0x7f18641f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.687+0000 7f186259c640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1858036040 tx=0x7f1858009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.688+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858043020 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.688+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1858003e30 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.688+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1864155a40 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.688+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858009d40 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.688+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1864156000 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.689+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1864069120 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.689+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1858046020 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.689+0000 7f184b7fe640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 0x7f185006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.689+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f18580ae520 con 0x7f18640b1c80 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.689+0000 7f1862d9d640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 0x7f185006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.690+0000 7f1862d9d640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 0x7f185006edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f185c001470 tx=0x7f185c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:11.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.694+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f185804d030 con 0x7f18640b1c80 2024-04-13T03:15:11.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.871+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1864156520 con 0x7f18640b1c80 2024-04-13T03:15:11.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.874+0000 7f184b7fe640 1 -- 172.21.15.106:0/975096361 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f185807ca50 con 0x7f18640b1c80 2024-04-13T03:15:11.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 msgr2=0x7f185006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 0x7f185006edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f185c001470 tx=0x7f185c009040 comp rx=0 tx=0).stop 2024-04-13T03:15:11.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 msgr2=0x7f18640b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1858036040 tx=0x7f1858009750 comp rx=0 tx=0).stop 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 shutdown_connections 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f185006c8f0 0x7f185006edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18641f5b80 0x7f18641f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18640b1c80 0x7f18640b2100 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 --2- 172.21.15.106:0/975096361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c004800 0x7f1864155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 >> 172.21.15.106:0/975096361 conn(0x7f18640d1650 msgr2=0x7f1864157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 shutdown_connections 2024-04-13T03:15:11.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.876+0000 7f18690de640 1 -- 172.21.15.106:0/975096361 wait complete. 2024-04-13T03:15:11.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-13T03:15:11.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.987+0000 7fb64ca36640 1 Processor -- start 2024-04-13T03:15:11.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.987+0000 7fb64ca36640 1 -- start start 2024-04-13T03:15:11.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64ca36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 0x7fb6481626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64ca36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb6480ce1f0 0x7fb648162be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64ca36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6480ce6f0 con 0x7fb64815ebf0 2024-04-13T03:15:11.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64ca36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb64815fa70 con 0x7fb6480ce1f0 2024-04-13T03:15:11.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64ca36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb64815f420 con 0x7fb64806fe90 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 0x7fb6481626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb646d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb6480ce1f0 0x7fb648162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 0x7fb6481626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:40062/0 (socket says 172.21.15.106:40062) 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb63ffff640 1 -- 172.21.15.106:0/2667053567 learned_addr learned my addr 172.21.15.106:0/2667053567 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb64659c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb64815ebf0 0x7fb64806f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:54530/0 (socket says 172.21.15.106:54530) 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb646d9d640 1 -- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 msgr2=0x7fb6481626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb646d9d640 1 --2- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 0x7fb6481626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb646d9d640 1 -- 172.21.15.106:0/2667053567 >> v1:172.21.15.114:6789/0 conn(0x7fb64815ebf0 legacy=0x7fb64806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb646d9d640 1 -- 172.21.15.106:0/2667053567 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64815f700 con 0x7fb6480ce1f0 2024-04-13T03:15:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.988+0000 7fb645d9b640 1 -- 172.21.15.106:0/2667053567 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094349190 0 0) 0x7fb6480ce6f0 con 0x7fb64815ebf0 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.989+0000 7fb646d9d640 1 --2- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6480ce1f0 0x7fb648162be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb640009d90 tx=0x7fb64002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657f33c7ee46002b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.989+0000 7fb645d9b640 1 -- 172.21.15.106:0/2667053567 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640039070 con 0x7fb6480ce1f0 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.989+0000 7fb645d9b640 1 -- 172.21.15.106:0/2667053567 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64002cc50 con 0x7fb6480ce1f0 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.989+0000 7fb645d9b640 1 -- 172.21.15.106:0/2667053567 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640034440 con 0x7fb6480ce1f0 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 -- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6480ce1f0 msgr2=0x7fb648162be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 --2- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6480ce1f0 0x7fb648162be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb640009d90 tx=0x7fb64002bea0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 -- 172.21.15.106:0/2667053567 shutdown_connections 2024-04-13T03:15:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 --2- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6480ce1f0 0x7fb648162be0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 --2- 172.21.15.106:0/2667053567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64806fe90 0x7fb6481626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 -- 172.21.15.106:0/2667053567 >> 172.21.15.106:0/2667053567 conn(0x7fb6480cd6d0 msgr2=0x7fb6480cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 -- 172.21.15.106:0/2667053567 shutdown_connections 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.990+0000 7fb64ca36640 1 -- 172.21.15.106:0/2667053567 wait complete. 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.991+0000 7fb64ca36640 1 Processor -- start 2024-04-13T03:15:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.991+0000 7fb64ca36640 1 -- start start 2024-04-13T03:15:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 0x7fb64814b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64815ebf0 0x7fb648157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480b0650 con 0x7fb6480ce1f0 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480b04d0 con 0x7fb64806fe90 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64ca36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb64815efd0 con 0x7fb64815ebf0 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 0x7fb64814b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 0x7fb64814b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:43876/0 (socket says 172.21.15.106:43876) 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37028/0 (socket says 172.21.15.106:37028) 2024-04-13T03:15:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb64659c640 1 -- 172.21.15.106:0/3157590820 learned_addr learned my addr 172.21.15.106:0/3157590820 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb646d9d640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64815ebf0 0x7fb648157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 -- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64815ebf0 msgr2=0x7fb648157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64815ebf0 0x7fb648157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 -- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 msgr2=0x7fb64814b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 0x7fb64814b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.992+0000 7fb63ffff640 1 -- 172.21.15.106:0/3157590820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64806be20 con 0x7fb6480ce1f0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb63ffff640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb63000e850 tx=0x7fb63000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630017070 con 0x7fb6480ce1f0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63000c940 con 0x7fb6480ce1f0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000ccf0 con 0x7fb6480ce1f0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb648156550 con 0x7fb6480ce1f0 2024-04-13T03:15:11.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.993+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6480ade40 con 0x7fb6480ce1f0 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.994+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb648075180 con 0x7fb6480ce1f0 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.994+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63001a070 con 0x7fb6480ce1f0 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.994+0000 7fb63effd640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 0x7fb62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.995+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7fb63008b940 con 0x7fb6480ce1f0 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.995+0000 7fb64659c640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 0x7fb62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.995+0000 7fb64659c640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 0x7fb62806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb638000f60 tx=0x7fb638007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:11.999+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb648075180 con 0x7fb6480ce1f0 2024-04-13T03:15:12.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.240+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fb648068a10 con 0x7fb6480ce1f0 2024-04-13T03:15:12.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.563+0000 7fb63effd640 1 -- 172.21.15.106:0/3157590820 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v290) ==== 166+0+0 (secure 0 0 0) 0x7fb630059830 con 0x7fb6480ce1f0 2024-04-13T03:15:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 msgr2=0x7fb62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 0x7fb62806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb638000f60 tx=0x7fb638007ea0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 msgr2=0x7fb64806b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb63000e850 tx=0x7fb63000ed10 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 shutdown_connections 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb62806c8f0 0x7fb62806edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb64815ebf0 0x7fb648157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6480ce1f0 0x7fb64806b8e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.565+0000 7fb64ca36640 1 --2- 172.21.15.106:0/3157590820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb64806fe90 0x7fb64814b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.566+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 >> 172.21.15.106:0/3157590820 conn(0x7fb6480cd6d0 msgr2=0x7fb6480ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.566+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 shutdown_connections 2024-04-13T03:15:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.566+0000 7fb64ca36640 1 -- 172.21.15.106:0/3157590820 wait complete. 2024-04-13T03:15:12.578 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-13T03:15:12.578 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-13T03:15:12.578 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:15:12.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:12.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.683+0000 7f79f08c8640 1 Processor -- start 2024-04-13T03:15:12.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.683+0000 7f79f08c8640 1 -- start start 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79f08c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79ec15e980 0x7f79ec162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79f08c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f79ec0cd550 0x7f79ec06d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79f08c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ec15fe40 con 0x7f79ec15ed60 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79f08c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f79ec1633f0 con 0x7f79ec0cd550 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79f08c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f79ec15f2c0 con 0x7f79ec15e980 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ead9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f79ec0cd550 0x7f79ec06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79e3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f79ec15ed60 0x7f79ec0cce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46224/0 (socket says 172.21.15.106:46224) 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79e3fff640 1 -- 172.21.15.106:0/1522328092 learned_addr learned my addr 172.21.15.106:0/1522328092 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ea59c640 1 --2- 172.21.15.106:0/1522328092 >> v2:172.21.15.114:3300/0 conn(0x7f79ec15e980 0x7f79ec162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ead9d640 1 -- 172.21.15.106:0/1522328092 >> v2:172.21.15.114:3300/0 conn(0x7f79ec15e980 msgr2=0x7f79ec162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ead9d640 1 --2- 172.21.15.106:0/1522328092 >> v2:172.21.15.114:3300/0 conn(0x7f79ec15e980 0x7f79ec162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ead9d640 1 -- 172.21.15.106:0/1522328092 >> v1:172.21.15.106:6789/0 conn(0x7f79ec15ed60 legacy=0x7f79ec0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79ead9d640 1 -- 172.21.15.106:0/1522328092 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec15f5a0 con 0x7f79ec0cd550 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.684+0000 7f79e9d9b640 1 -- 172.21.15.106:0/1522328092 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2408971565 0 0) 0x7f79ec15fe40 con 0x7f79ec15ed60 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79ead9d640 1 --2- 172.21.15.106:0/1522328092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec0cd550 0x7f79ec06d370 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0d31f0 tx=0x7f79e402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561802f12c97f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79e9d9b640 1 -- 172.21.15.106:0/1522328092 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4039070 con 0x7f79ec0cd550 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79e9d9b640 1 -- 172.21.15.106:0/1522328092 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e402cd00 con 0x7f79ec0cd550 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79e9d9b640 1 -- 172.21.15.106:0/1522328092 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4034660 con 0x7f79ec0cd550 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 -- 172.21.15.106:0/1522328092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec0cd550 msgr2=0x7f79ec06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 --2- 172.21.15.106:0/1522328092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec0cd550 0x7f79ec06d370 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0d31f0 tx=0x7f79e402bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:12.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 -- 172.21.15.106:0/1522328092 shutdown_connections 2024-04-13T03:15:12.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 --2- 172.21.15.106:0/1522328092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec0cd550 0x7f79ec06d370 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 --2- 172.21.15.106:0/1522328092 >> v2:172.21.15.114:3300/0 conn(0x7f79ec15e980 0x7f79ec162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.685+0000 7f79f08c8640 1 -- 172.21.15.106:0/1522328092 >> 172.21.15.106:0/1522328092 conn(0x7f79ec0d2550 msgr2=0x7f79ec0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:12.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.686+0000 7f79f08c8640 1 -- 172.21.15.106:0/1522328092 shutdown_connections 2024-04-13T03:15:12.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.686+0000 7f79f08c8640 1 -- 172.21.15.106:0/1522328092 wait complete. 2024-04-13T03:15:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.687+0000 7f79f08c8640 1 Processor -- start 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.687+0000 7f79f08c8640 1 -- start start 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.687+0000 7f79f08c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79ec0cd550 0x7f79ec0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.687+0000 7f79f08c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 0x7f79ec158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec0d1e20 con 0x7f79ec15ed60 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec15fb60 con 0x7f79ec15e980 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f79ec0d1fa0 con 0x7f79ec0cd550 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47520/0 (socket says 172.21.15.106:47520) 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 -- 172.21.15.106:0/2468186662 learned_addr learned my addr 172.21.15.106:0/2468186662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79e3fff640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 0x7f79ec158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ea59c640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79ec0cd550 0x7f79ec0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 -- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79ec0cd550 msgr2=0x7f79ec0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79ec0cd550 0x7f79ec0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 -- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 msgr2=0x7f79ec158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 0x7f79ec158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 -- 172.21.15.106:0/2468186662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec0c65a0 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79e3fff640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 0x7f79ec158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79ead9d640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009e80 tx=0x7f79e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4039050 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e403a040 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec0c9340 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e403e610 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.688+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec0c8aa0 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.689+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec069120 con 0x7f79ec15ed60 2024-04-13T03:15:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.689+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79e4047020 con 0x7f79ec15ed60 2024-04-13T03:15:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.690+0000 7f79e2ffd640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 0x7f79cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.690+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f79e40ade20 con 0x7f79ec15ed60 2024-04-13T03:15:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.690+0000 7f79ea59c640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 0x7f79cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.690+0000 7f79ea59c640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 0x7f79cc06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.694+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79e404e020 con 0x7f79ec15ed60 2024-04-13T03:15:12.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.871+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79ec0b84d0 con 0x7f79ec15ed60 2024-04-13T03:15:12.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.871+0000 7f79e2ffd640 1 -- 172.21.15.106:0/2468186662 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+13462 (secure 0 0 0) 0x7f79e4082070 con 0x7f79ec15ed60 2024-04-13T03:15:12.872 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:12.872 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":82,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:11.556088+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/2254289128":"2024-04-14T03:15:08.549162+0000","172.21.15.106:6835/2254289128":"2024-04-14T03:15:08.549162+0000","172.21.15.114:6834/390879687":"2024-04-14T03:13:50.750656+0000","172.21.15.114:6832/390879687":"2024-04-14T03:13:50.750656+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:12.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.873+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 msgr2=0x7f79cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.873+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 0x7f79cc06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc008040 comp rx=0 tx=0).stop 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.873+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 msgr2=0x7f79ec160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.873+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009e80 tx=0x7f79e4004a40 comp rx=0 tx=0).stop 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 shutdown_connections 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f79cc06c8f0 0x7f79cc06edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec15ed60 0x7f79ec160800 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79ec15e980 0x7f79ec158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 --2- 172.21.15.106:0/2468186662 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f79ec0cd550 0x7f79ec0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 >> 172.21.15.106:0/2468186662 conn(0x7f79ec0d2550 msgr2=0x7f79ec161a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:12.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 shutdown_connections 2024-04-13T03:15:12.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.874+0000 7f79f08c8640 1 -- 172.21.15.106:0/2468186662 wait complete. 2024-04-13T03:15:12.886 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:15:12.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:15:12.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.988+0000 7fba6a58c640 1 Processor -- start 2024-04-13T03:15:12.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.988+0000 7fba6a58c640 1 -- start start 2024-04-13T03:15:12.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba6a58c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba64160400 0x7fba6406c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba6a58c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba64160890 con 0x7fba6413f8f0 2024-04-13T03:15:12.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba6a58c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba64160b70 con 0x7fba6413f510 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba6a58c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fba64160e50 con 0x7fba64160400 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba64160400 0x7fba6406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba68b29640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fba6413f8f0 0x7fba640704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46240/0 (socket says 172.21.15.106:46240) 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.989+0000 7fba68b29640 1 -- 172.21.15.106:0/1428672596 learned_addr learned my addr 172.21.15.106:0/1428672596 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.990+0000 7fba637fe640 1 -- 172.21.15.106:0/1428672596 >> v1:172.21.15.114:6789/0 conn(0x7fba6413f510 legacy=0x7fba6415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.990+0000 7fba637fe640 1 -- 172.21.15.106:0/1428672596 >> v1:172.21.15.106:6789/0 conn(0x7fba6413f8f0 legacy=0x7fba640704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.990+0000 7fba62ffd640 1 -- 172.21.15.106:0/1428672596 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631961492 0 0) 0x7fba64160890 con 0x7fba6413f8f0 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.990+0000 7fba637fe640 1 -- 172.21.15.106:0/1428672596 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64161130 con 0x7fba64160400 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.990+0000 7fba637fe640 1 --2- 172.21.15.106:0/1428672596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba64160400 0x7fba6406c290 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fba580097a0 tx=0x7fba5802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81190e9123dd7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba62ffd640 1 -- 172.21.15.106:0/1428672596 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58034070 con 0x7fba64160400 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba62ffd640 1 -- 172.21.15.106:0/1428672596 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba58030610 con 0x7fba64160400 2024-04-13T03:15:12.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba62ffd640 1 -- 172.21.15.106:0/1428672596 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580309e0 con 0x7fba64160400 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 -- 172.21.15.106:0/1428672596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba64160400 msgr2=0x7fba6406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 --2- 172.21.15.106:0/1428672596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba64160400 0x7fba6406c290 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fba580097a0 tx=0x7fba5802b960 comp rx=0 tx=0).stop 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 -- 172.21.15.106:0/1428672596 shutdown_connections 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 --2- 172.21.15.106:0/1428672596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba64160400 0x7fba6406c290 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 -- 172.21.15.106:0/1428672596 >> 172.21.15.106:0/1428672596 conn(0x7fba640d13f0 msgr2=0x7fba640d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 -- 172.21.15.106:0/1428672596 shutdown_connections 2024-04-13T03:15:12.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.991+0000 7fba6a58c640 1 -- 172.21.15.106:0/1428672596 wait complete. 2024-04-13T03:15:12.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.992+0000 7fba6a58c640 1 Processor -- start 2024-04-13T03:15:12.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 -- start start 2024-04-13T03:15:12.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba6413f510 0x7fba641f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba6413f8f0 0x7fba64141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fba64070d80 con 0x7fba641416d0 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba64070c00 con 0x7fba6413f510 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba6a58c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fba64073100 con 0x7fba6413f8f0 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba68b29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba68b29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47542/0 (socket says 172.21.15.106:47542) 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba68b29640 1 -- 172.21.15.106:0/3507528054 learned_addr learned my addr 172.21.15.106:0/3507528054 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.993+0000 7fba637fe640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba6413f8f0 0x7fba64141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba63fff640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba6413f510 0x7fba641f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:12.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 -- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba6413f8f0 msgr2=0x7fba64141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba6413f8f0 0x7fba64141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 -- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba6413f510 msgr2=0x7fba641f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba6413f510 0x7fba641f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 -- 172.21.15.106:0/3507528054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64141d00 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba68b29640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00c7b0 tx=0x7fba5c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c018070 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c00a640 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba641f81b0 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00aa70 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.994+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641f8770 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.995+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba64069120 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.996+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba5c01b070 con 0x7fba641416d0 2024-04-13T03:15:12.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.996+0000 7fba60ff9640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.996+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7fba5c08aec0 con 0x7fba641416d0 2024-04-13T03:15:13.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.996+0000 7fba63fff640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:12.996+0000 7fba63fff640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 0x7fba5006edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fba54000f60 tx=0x7fba54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.000+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba5c058e50 con 0x7fba641416d0 2024-04-13T03:15:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.174+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba64071160 con 0x7fba641416d0 2024-04-13T03:15:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.174+0000 7fba60ff9640 1 -- 172.21.15.106:0/3507528054 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fba5c05d150 con 0x7fba641416d0 2024-04-13T03:15:13.176 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 msgr2=0x7fba5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 0x7fba5006edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fba54000f60 tx=0x7fba54009040 comp rx=0 tx=0).stop 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 msgr2=0x7fba6415b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00c7b0 tx=0x7fba5c00cc70 comp rx=0 tx=0).stop 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 shutdown_connections 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fba6413f8f0 0x7fba64141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba641416d0 0x7fba6415b8e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 --2- 172.21.15.106:0/3507528054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba6413f510 0x7fba641f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 >> 172.21.15.106:0/3507528054 conn(0x7fba640d13f0 msgr2=0x7fba640cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:13.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 shutdown_connections 2024-04-13T03:15:13.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.176+0000 7fba6a58c640 1 -- 172.21.15.106:0/3507528054 wait complete. 2024-04-13T03:15:13.189 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-13T03:15:13.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 Processor -- start 2024-04-13T03:15:13.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 -- start start 2024-04-13T03:15:13.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7f000cd550 0x7f7f00162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f0006d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0015fe40 con 0x7f7f0015e980 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7f001633f0 con 0x7f7f000cd550 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.292+0000 7f7f05ed5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0015f2c0 con 0x7f7f0015ed60 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7ef6ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7f000cd550 0x7f7f00162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7efffff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f0006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7eff7fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7f0015e980 0x7f7f000cce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:48704/0 (socket says 172.21.15.106:48704) 2024-04-13T03:15:13.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7eff7fe640 1 -- 172.21.15.106:0/2902341232 learned_addr learned my addr 172.21.15.106:0/2902341232 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:13.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7efffff640 1 -- 172.21.15.106:0/2902341232 >> v2:172.21.15.114:3300/0 conn(0x7f7f000cd550 msgr2=0x7f7f00162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:13.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7efffff640 1 --2- 172.21.15.106:0/2902341232 >> v2:172.21.15.114:3300/0 conn(0x7f7f000cd550 0x7f7f00162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7efffff640 1 -- 172.21.15.106:0/2902341232 >> v1:172.21.15.114:6789/0 conn(0x7f7f0015e980 legacy=0x7f7f000cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.293+0000 7f7efffff640 1 -- 172.21.15.106:0/2902341232 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0015f5a0 con 0x7f7f0015ed60 2024-04-13T03:15:13.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7efffff640 1 --2- 172.21.15.106:0/2902341232 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f0006d370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009d90 tx=0x7f7ef802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722cba54e44918ee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7efeffd640 1 -- 172.21.15.106:0/2902341232 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8039070 con 0x7f7f0015ed60 2024-04-13T03:15:13.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7efeffd640 1 -- 172.21.15.106:0/2902341232 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef802cce0 con 0x7f7f0015ed60 2024-04-13T03:15:13.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7efeffd640 1 -- 172.21.15.106:0/2902341232 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8034660 con 0x7f7f0015ed60 2024-04-13T03:15:13.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7f05ed5640 1 -- 172.21.15.106:0/2902341232 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 msgr2=0x7f7f0006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/2902341232 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f0006d370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009d90 tx=0x7f7ef802bf30 comp rx=0 tx=0).stop 2024-04-13T03:15:13.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.294+0000 7f7f05ed5640 1 -- 172.21.15.106:0/2902341232 shutdown_connections 2024-04-13T03:15:13.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.295+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/2902341232 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f0006d370 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.295+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/2902341232 >> v2:172.21.15.114:3300/0 conn(0x7f7f000cd550 0x7f7f00162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.295+0000 7f7f05ed5640 1 -- 172.21.15.106:0/2902341232 >> 172.21.15.106:0/2902341232 conn(0x7f7f000d2550 msgr2=0x7f7f000d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.295+0000 7f7f05ed5640 1 -- 172.21.15.106:0/2902341232 shutdown_connections 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.295+0000 7f7f05ed5640 1 -- 172.21.15.106:0/2902341232 wait complete. 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.296+0000 7f7f05ed5640 1 Processor -- start 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.296+0000 7f7f05ed5640 1 -- start start 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.296+0000 7f7f05ed5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.296+0000 7f7f05ed5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0015e980 0x7f7f001508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.296+0000 7f7f05ed5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f001f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7f05ed5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f000d1e20 con 0x7f7f000cd550 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7f05ed5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0015fb60 con 0x7f7f0015e980 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7f05ed5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000d1fa0 con 0x7f7f0015ed60 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47544/0 (socket says 172.21.15.106:47544) 2024-04-13T03:15:13.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 -- 172.21.15.106:0/3132048462 learned_addr learned my addr 172.21.15.106:0/3132048462 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:13.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7efffff640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f001f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7ef6ffd640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0015e980 0x7f7f001508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 -- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 msgr2=0x7f7f001f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f001f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 -- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0015e980 msgr2=0x7f7f001508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0015e980 0x7f7f001508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 -- 172.21.15.106:0/3132048462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f000b7a20 con 0x7f7f000cd550 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7eff7fe640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c400 tx=0x7f7ef000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0017070 con 0x7f7f000cd550 2024-04-13T03:15:13.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0005ce0 con 0x7f7f000cd550 2024-04-13T03:15:13.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f000b7d00 con 0x7f7f000cd550 2024-04-13T03:15:13.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00072f0 con 0x7f7f000cd550 2024-04-13T03:15:13.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.297+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f000b7710 con 0x7f7f000cd550 2024-04-13T03:15:13.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.298+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f00069120 con 0x7f7f000cd550 2024-04-13T03:15:13.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.299+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ef001a070 con 0x7f7f000cd550 2024-04-13T03:15:13.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.299+0000 7f7efcff9640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.299+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f7ef008ae40 con 0x7f7f000cd550 2024-04-13T03:15:13.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.299+0000 7f7ef6ffd640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.300+0000 7f7ef6ffd640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7f0006ffc0 tx=0x7f7ee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.303+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ef0059390 con 0x7f7f000cd550 2024-04-13T03:15:13.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.550+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f7f0006f250 con 0x7f7f000cd550 2024-04-13T03:15:13.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.564+0000 7f7efcff9640 1 -- 172.21.15.106:0/3132048462 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v291) ==== 170+0+0 (secure 0 0 0) 0x7f7ef0058c90 con 0x7f7f000cd550 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 msgr2=0x7f7ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7f0006ffc0 tx=0x7f7ee8008040 comp rx=0 tx=0).stop 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 msgr2=0x7f7f000cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c400 tx=0x7f7ef000c8c0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 shutdown_connections 2024-04-13T03:15:13.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f0015ed60 0x7f7f001f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f0015e980 0x7f7f001508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 --2- 172.21.15.106:0/3132048462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f000cd550 0x7f7f000cbfc0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 >> 172.21.15.106:0/3132048462 conn(0x7f7f000d2550 msgr2=0x7f7f0006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 shutdown_connections 2024-04-13T03:15:13.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.567+0000 7f7f05ed5640 1 -- 172.21.15.106:0/3132048462 wait complete. 2024-04-13T03:15:13.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:15:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.722+0000 7f639851e640 1 Processor -- start 2024-04-13T03:15:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.722+0000 7f639851e640 1 -- start start 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f639851e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f639015e8c0 0x7f63900ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f639851e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f63900b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f639851e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63900b5060 con 0x7f639015eca0 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f639851e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f63900b5340 con 0x7f639015e8c0 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f639851e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f639006c500 con 0x7f6390073520 2024-04-13T03:15:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63962ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f639015e8c0 0x7f63900ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f6395ab9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f639015eca0 0x7f6390072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46252/0 (socket says 172.21.15.106:46252) 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f6395ab9640 1 -- 172.21.15.106:0/1202352311 learned_addr learned my addr 172.21.15.106:0/1202352311 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f6396abb640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f63900b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63962ba640 1 -- 172.21.15.106:0/1202352311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 msgr2=0x7f63900b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63962ba640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f63900b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63962ba640 1 -- 172.21.15.106:0/1202352311 >> v1:172.21.15.106:6789/0 conn(0x7f639015eca0 legacy=0x7f6390072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63952b8640 1 -- 172.21.15.106:0/1202352311 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469205132 0 0) 0x7f63900b5060 con 0x7f639015eca0 2024-04-13T03:15:13.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.723+0000 7f63962ba640 1 -- 172.21.15.106:0/1202352311 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639006c7e0 con 0x7f639015e8c0 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.724+0000 7f63962ba640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63900ce250 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097a0 tx=0x7f638c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c74515d9f64c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.724+0000 7f63952b8640 1 -- 172.21.15.106:0/1202352311 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c034070 con 0x7f639015e8c0 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.724+0000 7f63952b8640 1 -- 172.21.15.106:0/1202352311 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c030610 con 0x7f639015e8c0 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.724+0000 7f63952b8640 1 -- 172.21.15.106:0/1202352311 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0309e0 con 0x7f639015e8c0 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 -- 172.21.15.106:0/1202352311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 msgr2=0x7f63900ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63900ce250 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097a0 tx=0x7f638c02b960 comp rx=0 tx=0).stop 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 -- 172.21.15.106:0/1202352311 shutdown_connections 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f63900b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 --2- 172.21.15.106:0/1202352311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63900ce250 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 -- 172.21.15.106:0/1202352311 >> 172.21.15.106:0/1202352311 conn(0x7f639006f0d0 msgr2=0x7f639006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:13.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 -- 172.21.15.106:0/1202352311 shutdown_connections 2024-04-13T03:15:13.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.725+0000 7f639851e640 1 -- 172.21.15.106:0/1202352311 wait complete. 2024-04-13T03:15:13.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.726+0000 7f639851e640 1 Processor -- start 2024-04-13T03:15:13.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.726+0000 7f639851e640 1 -- start start 2024-04-13T03:15:13.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f6390161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f639015eca0 0x7f6390154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63900b4d80 con 0x7f639015e8c0 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63900b4c00 con 0x7f639015eca0 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f639006cd00 con 0x7f6390073520 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47566/0 (socket says 172.21.15.106:47566) 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 -- 172.21.15.106:0/687036335 learned_addr learned my addr 172.21.15.106:0/687036335 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 -- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 msgr2=0x7f6390161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f6390161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 -- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f639015eca0 msgr2=0x7f6390154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f639015eca0 0x7f6390154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 -- 172.21.15.106:0/687036335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6390156c30 con 0x7f639015e8c0 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6395ab9640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f638000c460 tx=0x7f638000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63800074e0 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63800047e0 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380004bd0 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6390156ed0 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.727+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6390157490 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.728+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6390068a10 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.729+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6380011070 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.729+0000 7f6386ffd640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 0x7f637406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.729+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f638008b5e0 con 0x7f639015e8c0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.729+0000 7f63962ba640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 0x7f637406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:13.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.730+0000 7f63962ba640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 0x7f637406edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f638c036000 tx=0x7f638c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:13.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.734+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f638000e280 con 0x7f639015e8c0 2024-04-13T03:15:13.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.909+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f63900cdbf0 con 0x7f639015e8c0 2024-04-13T03:15:13.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.913+0000 7f6386ffd640 1 -- 172.21.15.106:0/687036335 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f6380059b30 con 0x7f639015e8c0 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 msgr2=0x7f637406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 0x7f637406edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f638c036000 tx=0x7f638c030000 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 msgr2=0x7f63901566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f638000c460 tx=0x7f638000c920 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 shutdown_connections 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f637406c8f0 0x7f637406edb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f639015eca0 0x7f6390154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639015e8c0 0x7f63901566f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 --2- 172.21.15.106:0/687036335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6390073520 0x7f6390161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:13.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.915+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 >> 172.21.15.106:0/687036335 conn(0x7f639006f0d0 msgr2=0x7f639006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:13.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.916+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 shutdown_connections 2024-04-13T03:15:13.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:13.916+0000 7f639851e640 1 -- 172.21.15.106:0/687036335 wait complete. 2024-04-13T03:15:13.927 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:15:13.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:15:13.966 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:15:14.017 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:15:14.017 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:14.017 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:14.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:15:14.033 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:15:14.033 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:14.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:15:14.050 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:14.050 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:14.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:15:14.088 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:15:14.088 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:14.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:15:14.106 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:15:14.107 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:14.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.227+0000 7f2956fcc640 1 Processor -- start 2024-04-13T03:15:14.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.227+0000 7f2956fcc640 1 -- start start 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2956fcc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f295006e480 0x7f295006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2956fcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295015d690 con 0x7f2950156b50 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2956fcc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295015d970 con 0x7f2950160c10 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2956fcc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2950161080 con 0x7f295006e480 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2954d68640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2950156b50 0x7f295015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46264/0 (socket says 172.21.15.106:46264) 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2954d68640 1 -- 172.21.15.106:0/137523880 learned_addr learned my addr 172.21.15.106:0/137523880 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2955569640 1 --2- 172.21.15.106:0/137523880 >> v2:172.21.15.114:3300/0 conn(0x7f295006e480 0x7f295006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892316579 0 0) 0x7f295015d690 con 0x7f2950156b50 2024-04-13T03:15:14.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2940003520 con 0x7f2950156b50 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630426764 0 0) 0x7f295015d970 con 0x7f2950160c10 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f295015d690 con 0x7f2950160c10 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918490108 0 0) 0x7f2940003520 con 0x7f2950156b50 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 >> v2:172.21.15.114:3300/0 conn(0x7f295006e480 msgr2=0x7f295006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 --2- 172.21.15.106:0/137523880 >> v2:172.21.15.114:3300/0 conn(0x7f295006e480 0x7f295006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 >> v1:172.21.15.114:6789/0 conn(0x7f2950160c10 legacy=0x7f295006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950160410 con 0x7f2950156b50 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2944001670 con 0x7f2950156b50 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2955569640 1 --2- 172.21.15.106:0/137523880 >> v2:172.21.15.114:3300/0 conn(0x7f295006e480 0x7f295006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 >> v1:172.21.15.106:6789/0 conn(0x7f2950156b50 legacy=0x7f295015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2940003b40 0x7f2940003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29400051c0 0x7f294000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2950161080 con 0x7f2940004800 2024-04-13T03:15:14.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f294000c040 con 0x7f29400051c0 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f294000c240 con 0x7f2940003b40 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f294ffff640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2954d68640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2940003b40 0x7f2940003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.228+0000 7f2955569640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29400051c0 0x7f294000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 -- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2940003b40 msgr2=0x7f2940003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2940003b40 0x7f2940003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 -- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29400051c0 msgr2=0x7f294000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29400051c0 0x7f294000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 -- 172.21.15.106:0/137523880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2940003520 con 0x7f2940004800 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f2955569640 1 -- 172.21.15.106:0/137523880 reap_dead start 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f2954d68640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2940003b40 0x7f2940003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294ffff640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2940004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f293c0018d0 tx=0x7f2940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39bf73e1a9ce6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c037070 con 0x7f2940004800 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293c0052d0 con 0x7f2940004800 2024-04-13T03:15:14.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.229+0000 7f294f7fe640 1 -- 172.21.15.106:0/137523880 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0014f0 con 0x7f2940004800 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 -- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 msgr2=0x7f2940004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2940004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f293c0018d0 tx=0x7f2940005830 comp rx=0 tx=0).stop 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 -- 172.21.15.106:0/137523880 shutdown_connections 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 --2- 172.21.15.106:0/137523880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2940004c80 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 -- 172.21.15.106:0/137523880 >> 172.21.15.106:0/137523880 conn(0x7f29500b1f10 msgr2=0x7f29500b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 -- 172.21.15.106:0/137523880 shutdown_connections 2024-04-13T03:15:14.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.230+0000 7f2956fcc640 1 -- 172.21.15.106:0/137523880 wait complete. 2024-04-13T03:15:14.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.231+0000 7f2956fcc640 1 Processor -- start 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.231+0000 7f2956fcc640 1 -- start start 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29500ad2a0 0x7f2950157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 0x7f29500aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2950156f30 con 0x7f2940004800 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29500afaa0 con 0x7f29500ad2a0 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2956fcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f29500ad780 con 0x7f29500ae820 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47582/0 (socket says 172.21.15.106:47582) 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 -- 172.21.15.106:0/231675755 learned_addr learned my addr 172.21.15.106:0/231675755 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f294ffff640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29500ad2a0 0x7f2950157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2955569640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 0x7f29500aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 -- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 msgr2=0x7f29500aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 0x7f29500aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 -- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29500ad2a0 msgr2=0x7f2950157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29500ad2a0 0x7f2950157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2954d68640 1 -- 172.21.15.106:0/231675755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500ada60 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.232+0000 7f2955569640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 0x7f29500aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f2954d68640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f294400e700 tx=0x7f294400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294400c6c0 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2944018070 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294400ca80 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29500add40 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.233+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29500ae300 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.234+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2950068a10 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.234+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f294400cc20 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.234+0000 7f294d7fa640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 0x7f293406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.235+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f294408b490 con 0x7f2940004800 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.235+0000 7f294ffff640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 0x7f293406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.235+0000 7f294ffff640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 0x7f293406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f293c034000 tx=0x7f293c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.239+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29440599e0 con 0x7f2940004800 2024-04-13T03:15:14.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.452+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29500d11f0 con 0x7f2940004800 2024-04-13T03:15:14.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.452+0000 7f294d7fa640 1 -- 172.21.15.106:0/231675755 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3875 (secure 0 0 0) 0x7f29440592e0 con 0x7f2940004800 2024-04-13T03:15:14.453 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:14.454 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/888661684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":888661684},{"type":"v1","addr":"172.21.15.114:6835","nonce":888661684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2169758388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2169758388},{"type":"v1","addr":"172.21.15.114:6834","nonce":2169758388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6813,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/847658996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":847658996},{"type":"v1","addr":"172.21.15.106:6836","nonce":847658996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":7446,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1351218772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1351218772},{"type":"v1","addr":"172.21.15.106:6837","nonce":1351218772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-04-13T03:15:14.454 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 46 2024-04-13T03:15:14.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 msgr2=0x7f293406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 0x7f293406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f293c034000 tx=0x7f293c031000 comp rx=0 tx=0).stop 2024-04-13T03:15:14.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 msgr2=0x7f2950155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f294400e700 tx=0x7f294400ebc0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 shutdown_connections 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f293406c920 0x7f293406ede0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29500ae820 0x7f29500aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29500ad2a0 0x7f2950157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 --2- 172.21.15.106:0/231675755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2940004800 0x7f2950155510 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 >> 172.21.15.106:0/231675755 conn(0x7f29500b1f10 msgr2=0x7f2950156180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.454+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 shutdown_connections 2024-04-13T03:15:14.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.455+0000 7f2956fcc640 1 -- 172.21.15.106:0/231675755 wait complete. 2024-04-13T03:15:14.467 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:15:14.467 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:15:14.467 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:15:14.467 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:15:14.467 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:15:14.467 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:15:14.467 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:15:14.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:15:14.468 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:15:14.468 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:15:14.468 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:15:14.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:15:14.468 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:15:14.468 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:15:14.469 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:15:14.469 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:15:14.476 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:15:14.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:15:14.477 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:15:14.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:15:14.480 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:15:14.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:15:14.481 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:15:14.481 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:15:14.519 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:15:14.519 DEBUG:teuthology.parallel:result is None 2024-04-13T03:15:14.520 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:15:14.520 DEBUG:teuthology.parallel:result is None 2024-04-13T03:15:14.522 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:15:14.522 DEBUG:teuthology.parallel:result is None 2024-04-13T03:15:14.523 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:15:14.523 DEBUG:teuthology.parallel:result is None 2024-04-13T03:15:14.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:15:14.596 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:15:14.597 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:15:14.597 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:15:14.596+0000 7f45c11fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-13T03:15:14.605 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:15:14.606 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:15:14.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.627+0000 7f8849555640 1 Processor -- start 2024-04-13T03:15:14.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.627+0000 7f8849555640 1 -- start start 2024-04-13T03:15:14.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8849555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 0x7f88440d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8849555640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f884415ed90 0x7f88440d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8849555640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88440cfd20 con 0x7f88440b3d60 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8849555640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8844162af0 con 0x7f884415ed90 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8849555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8844162dd0 con 0x7f884415e9b0 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f88437fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f88440b3d60 0x7f88440d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46272/0 (socket says 172.21.15.106:46272) 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f88437fe640 1 -- 172.21.15.106:0/84744962 learned_addr learned my addr 172.21.15.106:0/84744962 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8842ffd640 1 --2- 172.21.15.106:0/84744962 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 0x7f88440d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f88427fc640 1 --2- 172.21.15.106:0/84744962 >> v2:172.21.15.114:3300/0 conn(0x7f884415ed90 0x7f88440d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8841ffb640 1 -- 172.21.15.106:0/84744962 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190253780 0 0) 0x7f88440cfd20 con 0x7f88440b3d60 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8841ffb640 1 -- 172.21.15.106:0/84744962 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f882c003520 con 0x7f88440b3d60 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8842ffd640 1 -- 172.21.15.106:0/84744962 >> v2:172.21.15.114:3300/0 conn(0x7f884415ed90 msgr2=0x7f88440d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8842ffd640 1 --2- 172.21.15.106:0/84744962 >> v2:172.21.15.114:3300/0 conn(0x7f884415ed90 0x7f88440d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8842ffd640 1 -- 172.21.15.106:0/84744962 >> v1:172.21.15.106:6789/0 conn(0x7f88440b3d60 legacy=0x7f88440d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.628+0000 7f8842ffd640 1 -- 172.21.15.106:0/84744962 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88441630b0 con 0x7f884415e9b0 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.629+0000 7f8842ffd640 1 --2- 172.21.15.106:0/84744962 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 0x7f88440d2300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88340097a0 tx=0x7f883402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2d3788fc8130c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.629+0000 7f8841ffb640 1 -- 172.21.15.106:0/84744962 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834034070 con 0x7f884415e9b0 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.629+0000 7f8841ffb640 1 -- 172.21.15.106:0/84744962 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834030610 con 0x7f884415e9b0 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.629+0000 7f8841ffb640 1 -- 172.21.15.106:0/84744962 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340309e0 con 0x7f884415e9b0 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 -- 172.21.15.106:0/84744962 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 msgr2=0x7f88440d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 --2- 172.21.15.106:0/84744962 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 0x7f88440d2300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88340097a0 tx=0x7f883402b960 comp rx=0 tx=0).stop 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 -- 172.21.15.106:0/84744962 shutdown_connections 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 --2- 172.21.15.106:0/84744962 >> v2:172.21.15.114:3300/0 conn(0x7f884415ed90 0x7f88440d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 --2- 172.21.15.106:0/84744962 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f884415e9b0 0x7f88440d2300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 -- 172.21.15.106:0/84744962 >> 172.21.15.106:0/84744962 conn(0x7f884408bf10 msgr2=0x7f88440b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 -- 172.21.15.106:0/84744962 shutdown_connections 2024-04-13T03:15:14.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.630+0000 7f8849555640 1 -- 172.21.15.106:0/84744962 wait complete. 2024-04-13T03:15:14.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.631+0000 7f8849555640 1 Processor -- start 2024-04-13T03:15:14.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.631+0000 7f8849555640 1 -- start start 2024-04-13T03:15:14.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88440b3d60 0x7f884406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8844155d00 0x7f88440b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8844162800 con 0x7f884415e9b0 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8844162680 con 0x7f88440b3d60 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8849555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f884406eb20 con 0x7f8844155d00 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47600/0 (socket says 172.21.15.106:47600) 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 -- 172.21.15.106:0/3931179146 learned_addr learned my addr 172.21.15.106:0/3931179146 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88437fe640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8844155d00 0x7f88440b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f8842ffd640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88440b3d60 0x7f884406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 -- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8844155d00 msgr2=0x7f88440b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8844155d00 0x7f88440b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 -- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88440b3d60 msgr2=0x7f884406dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88440b3d60 0x7f884406dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 -- 172.21.15.106:0/3931179146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88441616e0 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.632+0000 7f88427fc640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f883800e870 tx=0x7f883800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.633+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838017070 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.633+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f883800c940 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.633+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88441619c0 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.633+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883800ccf0 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.633+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88440c9310 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.634+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8844068a10 con 0x7f884415e9b0 2024-04-13T03:15:14.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.634+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f883801a070 con 0x7f884415e9b0 2024-04-13T03:15:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.634+0000 7f882b7fe640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.635+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f883808bc80 con 0x7f884415e9b0 2024-04-13T03:15:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.635+0000 7f8842ffd640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:14.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.635+0000 7f8842ffd640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 0x7f883006edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8834036000 tx=0x7f883403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:14.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.639+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8838059cd0 con 0x7f884415e9b0 2024-04-13T03:15:14.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:14.823+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f88440ced60 con 0x7f884415e9b0 2024-04-13T03:15:15.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.936+0000 7f882b7fe640 1 -- 172.21.15.106:0/3931179146 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v87) ==== 97+0+0 (secure 0 0 0) 0x7f883805dfd0 con 0x7f884415e9b0 2024-04-13T03:15:15.938 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:15:15.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 msgr2=0x7f883006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:15.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 0x7f883006edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8834036000 tx=0x7f883403b040 comp rx=0 tx=0).stop 2024-04-13T03:15:15.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 msgr2=0x7f88440b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:15.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f883800e870 tx=0x7f883800ed30 comp rx=0 tx=0).stop 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 shutdown_connections 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8844155d00 0x7f88440b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f884415e9b0 0x7f88440b43e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 --2- 172.21.15.106:0/3931179146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88440b3d60 0x7f884406dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 >> 172.21.15.106:0/3931179146 conn(0x7f884408bf10 msgr2=0x7f88440b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 shutdown_connections 2024-04-13T03:15:15.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:15.938+0000 7f8849555640 1 -- 172.21.15.106:0/3931179146 wait complete. 2024-04-13T03:15:15.953 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:15:16.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 Processor -- start 2024-04-13T03:15:16.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 -- start start 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa190070090 0x7fa190160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 0x7fa1900cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa190070470 con 0x7fa1900b0390 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900ceb50 con 0x7fa190070090 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.056+0000 7fa196ead640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900cf110 con 0x7fa1900cf5a0 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa190070090 0x7fa190160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa190070090 0x7fa190160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47610/0 (socket says 172.21.15.106:47610) 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 -- 172.21.15.106:0/448879202 learned_addr learned my addr 172.21.15.106:0/448879202 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:16.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa19544a640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 0x7fa1900cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 -- 172.21.15.106:0/448879202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 msgr2=0x7fa1900cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 0x7fa1900cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 -- 172.21.15.106:0/448879202 >> v1:172.21.15.114:6789/0 conn(0x7fa1900b0390 legacy=0x7fa190160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa194c49640 1 -- 172.21.15.106:0/448879202 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19015da00 con 0x7fa190070090 2024-04-13T03:15:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.057+0000 7fa19544a640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 0x7fa1900cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:16.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa194c49640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190070090 0x7fa190160570 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa184009870 tx=0x7fa18402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55370241c84a42f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa18f7fe640 1 -- 172.21.15.106:0/448879202 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840317c0 con 0x7fa190070090 2024-04-13T03:15:16.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa18f7fe640 1 -- 172.21.15.106:0/448879202 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa184037070 con 0x7fa190070090 2024-04-13T03:15:16.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa18f7fe640 1 -- 172.21.15.106:0/448879202 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184031b80 con 0x7fa190070090 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 -- 172.21.15.106:0/448879202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190070090 msgr2=0x7fa190160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190070090 0x7fa190160570 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa184009870 tx=0x7fa18402b860 comp rx=0 tx=0).stop 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 -- 172.21.15.106:0/448879202 shutdown_connections 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa1900cf5a0 0x7fa1900cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 --2- 172.21.15.106:0/448879202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa190070090 0x7fa190160570 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 -- 172.21.15.106:0/448879202 >> 172.21.15.106:0/448879202 conn(0x7fa1900cd6d0 msgr2=0x7fa1900cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 -- 172.21.15.106:0/448879202 shutdown_connections 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.058+0000 7fa196ead640 1 -- 172.21.15.106:0/448879202 wait complete. 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.059+0000 7fa196ead640 1 Processor -- start 2024-04-13T03:15:16.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 -- start start 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa190070090 0x7fa1901526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1900b0390 0x7fa1901f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa19006e560 con 0x7fa1900cf5a0 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900ceda0 con 0x7fa1900b0390 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa196ead640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900cd220 con 0x7fa190070090 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.060+0000 7fa19544a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47622/0 (socket says 172.21.15.106:47622) 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 -- 172.21.15.106:0/3406797971 learned_addr learned my addr 172.21.15.106:0/3406797971 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa18ffff640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1900b0390 0x7fa1901f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa194c49640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa190070090 0x7fa1901526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 -- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa190070090 msgr2=0x7fa1901526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa190070090 0x7fa1901526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 -- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1900b0390 msgr2=0x7fa1901f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1900b0390 0x7fa1901f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 -- 172.21.15.106:0/3406797971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19009aa20 con 0x7fa1900cf5a0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa19544a640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa18800cd50 tx=0x7fa18800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18800f680 con 0x7fa1900cf5a0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18800f840 con 0x7fa1900cf5a0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa190156230 con 0x7fa1900cf5a0 2024-04-13T03:15:16.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18800fc30 con 0x7fa1900cf5a0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.061+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa190156a70 con 0x7fa1900cf5a0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.062+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190068a10 con 0x7fa1900cf5a0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.062+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1880213f0 con 0x7fa1900cf5a0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.063+0000 7fa18d7fa640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.063+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7fa18808c6b0 con 0x7fa1900cf5a0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.063+0000 7fa194c49640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.063+0000 7fa194c49640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 0x7fa17406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa184036000 tx=0x7fa18403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.067+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa190156230 con 0x7fa1900cf5a0 2024-04-13T03:15:16.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.306+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa1901602e0 con 0x7fa1900cf5a0 2024-04-13T03:15:16.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.307+0000 7fa18d7fa640 1 -- 172.21.15.106:0/3406797971 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fa188014030 con 0x7fa1900cf5a0 2024-04-13T03:15:16.309 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:16.310 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:16.310 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:15:16.310 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.310 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:15:16.310 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.311 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:15:16.312 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:15:16.313 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.314 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:15:16.315 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.316 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.317 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.318 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.319 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:15:16.320 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.321 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:15:16.322 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:15:16.323 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.324 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:15:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:15:16.326 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.308+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 msgr2=0x7fa17406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.308+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 0x7fa17406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa184036000 tx=0x7fa18403d040 comp rx=0 tx=0).stop 2024-04-13T03:15:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.308+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 msgr2=0x7fa19009bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.308+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa18800cd50 tx=0x7fa18800a650 comp rx=0 tx=0).stop 2024-04-13T03:15:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 shutdown_connections 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa17406c8f0 0x7fa17406edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1900cf5a0 0x7fa19009bde0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1900b0390 0x7fa1901f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 --2- 172.21.15.106:0/3406797971 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa190070090 0x7fa1901526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 >> 172.21.15.106:0/3406797971 conn(0x7fa1900cd6d0 msgr2=0x7fa1900cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 shutdown_connections 2024-04-13T03:15:16.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.309+0000 7fa196ead640 1 -- 172.21.15.106:0/3406797971 wait complete. 2024-04-13T03:15:16.330 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.431+0000 7f645b1b4640 1 Processor -- start 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.431+0000 7f645b1b4640 1 -- start start 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.431+0000 7f645b1b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6454070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.431+0000 7f645b1b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f645415ce80 0x7f64540b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.431+0000 7f645b1b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645406fd00 con 0x7f64540b7f70 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f645b1b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f645406ffe0 con 0x7f645415ce80 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f645b1b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f64540702c0 con 0x7f645415caa0 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6459751640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f64540b7f70 0x7f64540b5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46278/0 (socket says 172.21.15.106:46278) 2024-04-13T03:15:16.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6459751640 1 -- 172.21.15.106:0/1694322109 learned_addr learned my addr 172.21.15.106:0/1694322109 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6458f50640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6454070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668915331 0 0) 0x7f645406fd00 con 0x7f64540b7f70 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6444003520 con 0x7f64540b7f70 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6453fff640 1 --2- 172.21.15.106:0/1694322109 >> v2:172.21.15.114:3300/0 conn(0x7f645415ce80 0x7f64540b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 32423699 0 0) 0x7f6444003520 con 0x7f64540b7f70 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 msgr2=0x7f6454070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6454070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 >> v2:172.21.15.114:3300/0 conn(0x7f645415ce80 msgr2=0x7f64540b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 --2- 172.21.15.106:0/1694322109 >> v2:172.21.15.114:3300/0 conn(0x7f645415ce80 0x7f64540b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64540705a0 con 0x7f64540b7f70 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f644c0019f0 con 0x7f64540b7f70 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 >> v1:172.21.15.106:6789/0 conn(0x7f64540b7f70 legacy=0x7f64540b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64440039b0 0x7f6444003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6444004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6444004d40 0x7f64440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f645406ffe0 con 0x7f6444004d40 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f64540702c0 con 0x7f64440039b0 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f644400b9c0 con 0x7f645415caa0 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6459751640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6444004d40 0x7f64440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6458f50640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64440039b0 0x7f6444003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.432+0000 7f6453fff640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6444004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 -- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 msgr2=0x7f6444004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f645415caa0 0x7f6444004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 -- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64440039b0 msgr2=0x7f6444003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64440039b0 0x7f6444003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 -- 172.21.15.106:0/1694322109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444003520 con 0x7f6444004d40 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f6459751640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6444004d40 0x7f64440051c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f644c00a880 tx=0x7f6444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e69c76ceba1d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00a4a0 con 0x7f6444004d40 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644c034650 con 0x7f6444004d40 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.433+0000 7f64537fe640 1 -- 172.21.15.106:0/1694322109 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c034a40 con 0x7f6444004d40 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 -- 172.21.15.106:0/1694322109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6444004d40 msgr2=0x7f64440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1694322109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6444004d40 0x7f64440051c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f644c00a880 tx=0x7f6444005830 comp rx=0 tx=0).stop 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f6459751640 1 -- 172.21.15.106:0/1694322109 reap_dead start 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 -- 172.21.15.106:0/1694322109 shutdown_connections 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 -- 172.21.15.106:0/1694322109 >> 172.21.15.106:0/1694322109 conn(0x7f64540d25d0 msgr2=0x7f64540d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 -- 172.21.15.106:0/1694322109 shutdown_connections 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.434+0000 7f645b1b4640 1 -- 172.21.15.106:0/1694322109 wait complete. 2024-04-13T03:15:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.435+0000 7f645b1b4640 1 Processor -- start 2024-04-13T03:15:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.435+0000 7f645b1b4640 1 -- start start 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6444004d40 0x7f64541f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64540c6f00 0x7f645415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f645406f490 con 0x7f645406d870 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f645406f990 con 0x7f64540c6f00 2024-04-13T03:15:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f645b1b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f645406f610 con 0x7f6444004d40 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47632/0 (socket says 172.21.15.106:47632) 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 -- 172.21.15.106:0/1683360797 learned_addr learned my addr 172.21.15.106:0/1683360797 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6459751640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64540c6f00 0x7f645415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6458f50640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6444004d40 0x7f64541f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 -- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6444004d40 msgr2=0x7f64541f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6444004d40 0x7f64541f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 -- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64540c6f00 msgr2=0x7f645415b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64540c6f00 0x7f645415b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.436+0000 7f6453fff640 1 -- 172.21.15.106:0/1683360797 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645415d310 con 0x7f645406d870 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f6453fff640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6440011a70 tx=0x7f6440011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644000fb50 con 0x7f645406d870 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644000fcf0 con 0x7f645406d870 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645415d5b0 con 0x7f645406d870 2024-04-13T03:15:16.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644000fea0 con 0x7f645406d870 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.437+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645406f0d0 con 0x7f645406d870 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.438+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6454068a10 con 0x7f645406d870 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.438+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6440012040 con 0x7f645406d870 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.439+0000 7f64517fa640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.439+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f644008c0c0 con 0x7f645406d870 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.439+0000 7f6458f50640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.439+0000 7f6458f50640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6448009a80 tx=0x7f6448008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.443+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6440019030 con 0x7f645406d870 2024-04-13T03:15:16.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.620+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f645415fd40 con 0x7f645406d870 2024-04-13T03:15:16.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.621+0000 7f64517fa640 1 -- 172.21.15.106:0/1683360797 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+13201 (secure 0 0 0) 0x7f644005a6b0 con 0x7f645406d870 2024-04-13T03:15:16.622 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:16.622 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":87,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:15.933478+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 msgr2=0x7f643806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6448009a80 tx=0x7f6448008040 comp rx=0 tx=0).stop 2024-04-13T03:15:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 msgr2=0x7f645406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6440011a70 tx=0x7f6440011f30 comp rx=0 tx=0).stop 2024-04-13T03:15:16.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 shutdown_connections 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64540c6f00 0x7f645415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f645406d870 0x7f645406e7d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 --2- 172.21.15.106:0/1683360797 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6444004d40 0x7f64541f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 >> 172.21.15.106:0/1683360797 conn(0x7f64540d25d0 msgr2=0x7f64540b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 shutdown_connections 2024-04-13T03:15:16.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.623+0000 7f645b1b4640 1 -- 172.21.15.106:0/1683360797 wait complete. 2024-04-13T03:15:16.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:15:16.635 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:15:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.737+0000 7f9b1942d640 1 Processor -- start 2024-04-13T03:15:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.737+0000 7f9b1942d640 1 -- start start 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b1415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b140b4730 0x7f9b140b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9b140b51c0 0x7f9b1406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9b140b3690 con 0x7f9b140b4730 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9b140b3970 con 0x7f9b140b51c0 2024-04-13T03:15:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b1942d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9b140b3c50 con 0x7f9b1415b4e0 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b140b4730 0x7f9b140b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9b140b4730 0x7f9b140b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47638/0 (socket says 172.21.15.106:47638) 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 -- 172.21.15.106:0/1864813764 learned_addr learned my addr 172.21.15.106:0/1864813764 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b137fe640 1 --2- 172.21.15.106:0/1864813764 >> v2:172.21.15.114:3300/0 conn(0x7f9b140b51c0 0x7f9b1406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b12ffd640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b1415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 -- 172.21.15.106:0/1864813764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 msgr2=0x7f9b1415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b1415fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 -- 172.21.15.106:0/1864813764 >> v2:172.21.15.114:3300/0 conn(0x7f9b140b51c0 msgr2=0x7f9b1406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 --2- 172.21.15.106:0/1864813764 >> v2:172.21.15.114:3300/0 conn(0x7f9b140b51c0 0x7f9b1406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.738+0000 7f9b127fc640 1 -- 172.21.15.106:0/1864813764 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b140b3f30 con 0x7f9b140b4730 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.739+0000 7f9b127fc640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b140b4c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9b140b4bd0 tx=0x7f9b0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85f47662d173381 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.739+0000 7f9b11ffb640 1 -- 172.21.15.106:0/1864813764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08035020 con 0x7f9b140b4730 2024-04-13T03:15:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.739+0000 7f9b11ffb640 1 -- 172.21.15.106:0/1864813764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b08031870 con 0x7f9b140b4730 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.739+0000 7f9b11ffb640 1 -- 172.21.15.106:0/1864813764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08031c40 con 0x7f9b140b4730 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 -- 172.21.15.106:0/1864813764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 msgr2=0x7f9b140b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b140b4c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9b140b4bd0 tx=0x7f9b0802b910 comp rx=0 tx=0).stop 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 -- 172.21.15.106:0/1864813764 shutdown_connections 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 --2- 172.21.15.106:0/1864813764 >> v2:172.21.15.114:3300/0 conn(0x7f9b140b51c0 0x7f9b1406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b140b4c80 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 --2- 172.21.15.106:0/1864813764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b1415fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 -- 172.21.15.106:0/1864813764 >> 172.21.15.106:0/1864813764 conn(0x7f9b140b2600 msgr2=0x7f9b140d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 -- 172.21.15.106:0/1864813764 shutdown_connections 2024-04-13T03:15:16.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.740+0000 7f9b1942d640 1 -- 172.21.15.106:0/1864813764 wait complete. 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.741+0000 7f9b1942d640 1 Processor -- start 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.741+0000 7f9b1942d640 1 -- start start 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b140b51c0 0x7f9b1414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b141f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1406d0c0 con 0x7f9b140b4730 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b140b32d0 con 0x7f9b140b51c0 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b1942d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9b140ae9c0 con 0x7f9b1415b4e0 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47642/0 (socket says 172.21.15.106:47642) 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 -- 172.21.15.106:0/3584781807 learned_addr learned my addr 172.21.15.106:0/3584781807 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 -- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 msgr2=0x7f9b141f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b141f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 -- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b140b51c0 msgr2=0x7f9b1414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b140b51c0 0x7f9b1414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.742+0000 7f9b12ffd640 1 -- 172.21.15.106:0/3584781807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141f5cc0 con 0x7f9b140b4730 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9b12ffd640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400efc0 tx=0x7f9b0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04018020 con 0x7f9b140b4730 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b04014480 con 0x7f9b140b4730 2024-04-13T03:15:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b140addb0 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04014850 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.743+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b140ae370 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.744+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b140d0390 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.744+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b04005d10 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.745+0000 7f9afb7fe640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.745+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f9b0408c400 con 0x7f9b140b4730 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.745+0000 7f9b127fc640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.745+0000 7f9b127fc640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 0x7f9b0006edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036000 tx=0x7f9b08030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:16.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.749+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b140d0390 con 0x7f9b140b4730 2024-04-13T03:15:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.924+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b140d07e0 con 0x7f9b140b4730 2024-04-13T03:15:16.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.925+0000 7f9afb7fe640 1 -- 172.21.15.106:0/3584781807 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f9b140d07e0 con 0x7f9b140b4730 2024-04-13T03:15:16.926 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:15:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 msgr2=0x7f9b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 0x7f9b0006edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036000 tx=0x7f9b08030000 comp rx=0 tx=0).stop 2024-04-13T03:15:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 msgr2=0x7f9b141f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400efc0 tx=0x7f9b0400c490 comp rx=0 tx=0).stop 2024-04-13T03:15:16.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 shutdown_connections 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b1415b4e0 0x7f9b141f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b140b51c0 0x7f9b1414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 --2- 172.21.15.106:0/3584781807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b140b4730 0x7f9b141f5780 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 >> 172.21.15.106:0/3584781807 conn(0x7f9b140b2600 msgr2=0x7f9b140b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 shutdown_connections 2024-04-13T03:15:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:16.927+0000 7f9b1942d640 1 -- 172.21.15.106:0/3584781807 wait complete. 2024-04-13T03:15:16.939 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:15:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.038+0000 7ff5cad55640 1 Processor -- start 2024-04-13T03:15:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.038+0000 7ff5cad55640 1 -- start start 2024-04-13T03:15:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5cad55640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5c406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5cad55640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff5c40b5bf0 0x7ff5c406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5cad55640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c40ce6f0 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5cad55640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c415fa70 con 0x7ff5c40b5bf0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5cad55640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c415f420 con 0x7ff5c415ebf0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff5c40ce1f0 0x7ff5c406e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46288/0 (socket says 172.21.15.106:46288) 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c3fff640 1 -- 172.21.15.106:0/55476747 learned_addr learned my addr 172.21.15.106:0/55476747 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5c406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c92f2640 1 --2- 172.21.15.106:0/55476747 >> v2:172.21.15.114:3300/0 conn(0x7ff5c40b5bf0 0x7ff5c406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2637874991 0 0) 0x7ff5c40ce6f0 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b4003520 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2831669640 0 0) 0x7ff5b4003520 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 msgr2=0x7ff5c406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5c406f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 >> v2:172.21.15.114:3300/0 conn(0x7ff5c40b5bf0 msgr2=0x7ff5c406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 --2- 172.21.15.106:0/55476747 >> v2:172.21.15.114:3300/0 conn(0x7ff5c40b5bf0 0x7ff5c406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c415f700 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c92f2640 1 --2- 172.21.15.106:0/55476747 >> v2:172.21.15.114:3300/0 conn(0x7ff5c40b5bf0 0x7ff5c406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7ff5b0001670 con 0x7ff5c40ce1f0 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5c406f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 >> v1:172.21.15.106:6789/0 conn(0x7ff5c40ce1f0 legacy=0x7ff5c406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.039+0000 7ff5c37fe640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5b4004d40 0x7ff5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c415fa70 con 0x7ff5b40039b0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c415f420 con 0x7ff5b4004d40 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b400b9c0 con 0x7ff5c415ebf0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c3fff640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c92f2640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5b4004d40 0x7ff5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 -- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 msgr2=0x7ff5b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 -- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5b4004d40 msgr2=0x7ff5b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5b4004d40 0x7ff5b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c8af1640 1 -- 172.21.15.106:0/55476747 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b4003520 con 0x7ff5b40039b0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c3fff640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c415ebf0 0x7ff5b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.040+0000 7ff5c92f2640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5b4004d40 0x7ff5b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009c90 tx=0x7ff5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e2badfc52afacd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8039070 con 0x7ff5b40039b0 2024-04-13T03:15:17.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b802c8f0 con 0x7ff5b40039b0 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5c37fe640 1 -- 172.21.15.106:0/55476747 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8009860 con 0x7ff5b40039b0 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 -- 172.21.15.106:0/55476747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 msgr2=0x7ff5b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 --2- 172.21.15.106:0/55476747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009c90 tx=0x7ff5b4005830 comp rx=0 tx=0).stop 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5c92f2640 1 -- 172.21.15.106:0/55476747 reap_dead start 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 -- 172.21.15.106:0/55476747 shutdown_connections 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 -- 172.21.15.106:0/55476747 >> 172.21.15.106:0/55476747 conn(0x7ff5c40cd6d0 msgr2=0x7ff5c40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 -- 172.21.15.106:0/55476747 shutdown_connections 2024-04-13T03:15:17.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.041+0000 7ff5cad55640 1 -- 172.21.15.106:0/55476747 wait complete. 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.042+0000 7ff5cad55640 1 Processor -- start 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.042+0000 7ff5cad55640 1 -- start start 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5c41567e0 0x7ff5c4141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c4157f80 0x7ff5c40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40b0650 con 0x7ff5b40039b0 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40b04d0 con 0x7ff5c41567e0 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5cad55640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c415efd0 con 0x7ff5c4157f80 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5c8af1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5c8af1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47662/0 (socket says 172.21.15.106:47662) 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.043+0000 7ff5c8af1640 1 -- 172.21.15.106:0/1150325435 learned_addr learned my addr 172.21.15.106:0/1150325435 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 -- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c4157f80 msgr2=0x7ff5c40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c4157f80 0x7ff5c40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 -- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5c41567e0 msgr2=0x7ff5c4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5c41567e0 0x7ff5c4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 -- 172.21.15.106:0/1150325435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c4156c60 con 0x7ff5b40039b0 2024-04-13T03:15:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c8af1640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009db0 tx=0x7ff5b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8039050 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8045070 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c4156ee0 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8009860 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.044+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41574a0 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.045+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c4068a10 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.045+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b804a460 con 0x7ff5b40039b0 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.046+0000 7ff5c17fa640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.046+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7ff5b80adac0 con 0x7ff5b40039b0 2024-04-13T03:15:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.046+0000 7ff5c3fff640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.046+0000 7ff5c3fff640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 0x7ff5a806edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b0001c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.050+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5c4156c60 con 0x7ff5b40039b0 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.226+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff5c40ccbe0 con 0x7ff5b40039b0 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.226+0000 7ff5c17fa640 1 -- 172.21.15.106:0/1150325435 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) ==== 84+0+65 (secure 0 0 0) 0x7ff5b807c0b0 con 0x7ff5b40039b0 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:15:17.227 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:15:17.228 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:15:17.228 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:17.228 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:15:17.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 msgr2=0x7ff5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 0x7ff5a806edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b0001c50 comp rx=0 tx=0).stop 2024-04-13T03:15:17.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 msgr2=0x7ff5c415dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009db0 tx=0x7ff5b8004730 comp rx=0 tx=0).stop 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 shutdown_connections 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff5a806c8f0 0x7ff5a806edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff5c4157f80 0x7ff5c40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5c41567e0 0x7ff5c4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 --2- 172.21.15.106:0/1150325435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5b40039b0 0x7ff5c415dff0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 >> 172.21.15.106:0/1150325435 conn(0x7ff5c40cd6d0 msgr2=0x7ff5c40cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 shutdown_connections 2024-04-13T03:15:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.228+0000 7ff5cad55640 1 -- 172.21.15.106:0/1150325435 wait complete. 2024-04-13T03:15:17.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:15:17.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:15:17.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.343+0000 7f10aade0640 1 Processor -- start 2024-04-13T03:15:17.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.343+0000 7f10aade0640 1 -- start start 2024-04-13T03:15:17.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10a415e880 0x7f10a406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 0x7f10a406be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 0x7f10a4162470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10a40d30a0 con 0x7f10a415e880 2024-04-13T03:15:17.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f10a40cfc60 con 0x7f10a40cf7d0 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10aade0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10a40d2a50 con 0x7f10a406c360 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10a8b7c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10a415e880 0x7f10a406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10a8b7c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10a415e880 0x7f10a406b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47678/0 (socket says 172.21.15.106:47678) 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10a8b7c640 1 -- 172.21.15.106:0/1029012299 learned_addr learned my addr 172.21.15.106:0/1029012299 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10a3fff640 1 --2- 172.21.15.106:0/1029012299 >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 0x7f10a406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.344+0000 7f10a937d640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 0x7f10a4162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 -- 172.21.15.106:0/1029012299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 msgr2=0x7f10a4162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 0x7f10a4162470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 -- 172.21.15.106:0/1029012299 >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 msgr2=0x7f10a406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/1029012299 >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 0x7f10a406be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 -- 172.21.15.106:0/1029012299 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a40d2d30 con 0x7f10a415e880 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a937d640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 0x7f10a4162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a3fff640 1 --2- 172.21.15.106:0/1029012299 >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 0x7f10a406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a415e880 0x7f10a406b8e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f10a40b3810 tx=0x7f109402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec68090392ce58e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a37fe640 1 -- 172.21.15.106:0/1029012299 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094035020 con 0x7f10a415e880 2024-04-13T03:15:17.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a37fe640 1 -- 172.21.15.106:0/1029012299 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1094031870 con 0x7f10a415e880 2024-04-13T03:15:17.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.345+0000 7f10a37fe640 1 -- 172.21.15.106:0/1029012299 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094031c40 con 0x7f10a415e880 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 -- 172.21.15.106:0/1029012299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a415e880 msgr2=0x7f10a406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a415e880 0x7f10a406b8e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f10a40b3810 tx=0x7f109402b910 comp rx=0 tx=0).stop 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 -- 172.21.15.106:0/1029012299 shutdown_connections 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a406c360 0x7f10a4162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 --2- 172.21.15.106:0/1029012299 >> v2:172.21.15.114:3300/0 conn(0x7f10a40cf7d0 0x7f10a406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 --2- 172.21.15.106:0/1029012299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a415e880 0x7f10a406b8e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 -- 172.21.15.106:0/1029012299 >> 172.21.15.106:0/1029012299 conn(0x7f10a40b6280 msgr2=0x7f10a40b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 -- 172.21.15.106:0/1029012299 shutdown_connections 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.346+0000 7f10aade0640 1 -- 172.21.15.106:0/1029012299 wait complete. 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.347+0000 7f10aade0640 1 Processor -- start 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.347+0000 7f10aade0640 1 -- start start 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a406c360 0x7f10a40b9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 0x7f10a40b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40b3bd0 con 0x7f10a40b24e0 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40b3a50 con 0x7f10a406c360 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10aade0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10a40b78c0 con 0x7f10a415e880 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47690/0 (socket says 172.21.15.106:47690) 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 -- 172.21.15.106:0/418576640 learned_addr learned my addr 172.21.15.106:0/418576640 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a406c360 0x7f10a40b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a3fff640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 0x7f10a40b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 -- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 msgr2=0x7f10a40b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 0x7f10a40b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 -- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a406c360 msgr2=0x7f10a40b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a406c360 0x7f10a40b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a937d640 1 -- 172.21.15.106:0/418576640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41e9dd0 con 0x7f10a40b24e0 2024-04-13T03:15:17.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.348+0000 7f10a3fff640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 0x7f10a40b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10a937d640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f109c00cef0 tx=0x7f109c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c01a070 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c00d860 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a41ea0b0 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c00dc50 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.349+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41ea670 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.350+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a4068a10 con 0x7f10a40b24e0 2024-04-13T03:15:17.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.350+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f109c01d070 con 0x7f10a40b24e0 2024-04-13T03:15:17.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.351+0000 7f10a17fa640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:17.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.351+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:17.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.351+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f109c08c1b0 con 0x7f10a40b24e0 2024-04-13T03:15:17.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.351+0000 7f10a8b7c640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1094036000 tx=0x7f109403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:17.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.355+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f109c05a210 con 0x7f10a40b24e0 2024-04-13T03:15:17.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.532+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f10a41621e0 con 0x7f10a40b24e0 2024-04-13T03:15:17.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.942+0000 7f10a17fa640 1 -- 172.21.15.106:0/418576640 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) ==== 138+0+0 (secure 0 0 0) 0x7f109c05e510 con 0x7f10a40b24e0 2024-04-13T03:15:17.943 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 msgr2=0x7f109006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1094036000 tx=0x7f109403c040 comp rx=0 tx=0).stop 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 msgr2=0x7f10a41e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f109c00cef0 tx=0x7f109c00a710 comp rx=0 tx=0).stop 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 shutdown_connections 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a415e880 0x7f10a40b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a40b24e0 0x7f10a41e9080 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 --2- 172.21.15.106:0/418576640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a406c360 0x7f10a40b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:17.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 >> 172.21.15.106:0/418576640 conn(0x7f10a40b6280 msgr2=0x7f10a40b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:17.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 shutdown_connections 2024-04-13T03:15:17.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:17.945+0000 7f10aade0640 1 -- 172.21.15.106:0/418576640 wait complete. 2024-04-13T03:15:17.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:15:18.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.066+0000 7f968795e640 1 Processor -- start 2024-04-13T03:15:18.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 -- start start 2024-04-13T03:15:18.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96800b5ad0 0x7f96800d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96800cd240 con 0x7f968015e820 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f96800cd520 con 0x7f96800b5ad0 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.067+0000 7f968795e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96800cd800 con 0x7f96800ce9e0 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9685efb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96800b5ad0 0x7f96800d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:48218/0 (socket says 172.21.15.106:48218) 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 -- 172.21.15.106:0/3926191260 learned_addr learned my addr 172.21.15.106:0/3926191260 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 -- 172.21.15.106:0/3926191260 >> v2:172.21.15.114:3300/0 conn(0x7f96800b5ad0 msgr2=0x7f96800d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 --2- 172.21.15.106:0/3926191260 >> v2:172.21.15.114:3300/0 conn(0x7f96800b5ad0 0x7f96800d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 -- 172.21.15.106:0/3926191260 >> v1:172.21.15.114:6789/0 conn(0x7f968015e820 legacy=0x7f96800b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.068+0000 7f9684ef9640 1 -- 172.21.15.106:0/3926191260 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800cdae0 con 0x7f96800ce9e0 2024-04-13T03:15:18.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f9684ef9640 1 --2- 172.21.15.106:0/3926191260 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9670009870 tx=0x7f967002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27d322108da1467 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f9677fff640 1 -- 172.21.15.106:0/3926191260 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035020 con 0x7f96800ce9e0 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f9677fff640 1 -- 172.21.15.106:0/3926191260 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9670031640 con 0x7f96800ce9e0 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f9677fff640 1 -- 172.21.15.106:0/3926191260 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967002d040 con 0x7f96800ce9e0 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 -- 172.21.15.106:0/3926191260 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 msgr2=0x7f96800b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 --2- 172.21.15.106:0/3926191260 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9670009870 tx=0x7f967002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 -- 172.21.15.106:0/3926191260 shutdown_connections 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 --2- 172.21.15.106:0/3926191260 >> v2:172.21.15.114:3300/0 conn(0x7f96800b5ad0 0x7f96800d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 --2- 172.21.15.106:0/3926191260 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96800ce9e0 0x7f96800b7390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 -- 172.21.15.106:0/3926191260 >> 172.21.15.106:0/3926191260 conn(0x7f96800d2510 msgr2=0x7f96800d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 -- 172.21.15.106:0/3926191260 shutdown_connections 2024-04-13T03:15:18.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.069+0000 7f968795e640 1 -- 172.21.15.106:0/3926191260 wait complete. 2024-04-13T03:15:18.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 Processor -- start 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 -- start start 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96800b5ad0 0x7f96801520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f968015e820 0x7f96800c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96800ccf50 con 0x7f96800ce9e0 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96800ccdd0 con 0x7f96800b5ad0 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f968795e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f968015ec00 con 0x7f968015e820 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f9684ef9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f9684ef9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47696/0 (socket says 172.21.15.106:47696) 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.071+0000 7f9684ef9640 1 -- 172.21.15.106:0/1503163239 learned_addr learned my addr 172.21.15.106:0/1503163239 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9685efb640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f968015e820 0x7f96800c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f96856fa640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96800b5ad0 0x7f96801520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 -- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f968015e820 msgr2=0x7f96800c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f968015e820 0x7f96800c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 -- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96800b5ad0 msgr2=0x7f96801520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96800b5ad0 0x7f96801520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 -- 172.21.15.106:0/1503163239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96801f9620 con 0x7f96800ce9e0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9684ef9640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9670000c00 tx=0x7f96700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670040070 con 0x7f96800ce9e0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700399f0 con 0x7f96800ce9e0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035040 con 0x7f96800ce9e0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96801603f0 con 0x7f96800ce9e0 2024-04-13T03:15:18.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.072+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801609b0 con 0x7f96800ce9e0 2024-04-13T03:15:18.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.073+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96800b2fc0 con 0x7f96800ce9e0 2024-04-13T03:15:18.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.074+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f967004b060 con 0x7f96800ce9e0 2024-04-13T03:15:18.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.074+0000 7f9675ffb640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 0x7f966406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:18.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.074+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5619+0+0 (secure 0 0 0) 0x7f96700ad260 con 0x7f96800ce9e0 2024-04-13T03:15:18.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.074+0000 7f96856fa640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 0x7f966406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:18.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.075+0000 7f96856fa640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 0x7f966406edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f967c001e70 tx=0x7f967c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:18.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.079+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96800b2fc0 con 0x7f96800ce9e0 2024-04-13T03:15:18.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.253+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9680069120 con 0x7f96800ce9e0 2024-04-13T03:15:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.947+0000 7f9675ffb640 1 -- 172.21.15.106:0/1503163239 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7f967007b090 con 0x7f96800ce9e0 2024-04-13T03:15:18.948 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.949+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 msgr2=0x7f966406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.949+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 0x7f966406edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f967c001e70 tx=0x7f967c009040 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.949+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 msgr2=0x7f96800c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.949+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9670000c00 tx=0x7f96700046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 shutdown_connections 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f966406c8f0 0x7f966406edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f968015e820 0x7f96800c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96800ce9e0 0x7f96800c6a70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 --2- 172.21.15.106:0/1503163239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96800b5ad0 0x7f96801520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 >> 172.21.15.106:0/1503163239 conn(0x7f96800d2510 msgr2=0x7f96800b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 shutdown_connections 2024-04-13T03:15:18.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:18.950+0000 7f968795e640 1 -- 172.21.15.106:0/1503163239 wait complete. 2024-04-13T03:15:18.970 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:15:19.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.074+0000 7efd80ba3640 1 Processor -- start 2024-04-13T03:15:19.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.074+0000 7efd80ba3640 1 -- start start 2024-04-13T03:15:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd80ba3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efd7c15b4e0 0x7efd7c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd80ba3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efd7c0b5390 0x7efd7c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd80ba3640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c0b3690 con 0x7efd7c0b4730 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd80ba3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efd7c0b3970 con 0x7efd7c15b4e0 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd80ba3640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7efd7c0b3c50 con 0x7efd7c0b5390 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efd7c15b4e0 0x7efd7c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efd7c15b4e0 0x7efd7c15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47708/0 (socket says 172.21.15.106:47708) 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7a59c640 1 -- 172.21.15.106:0/4139273388 learned_addr learned my addr 172.21.15.106:0/4139273388 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7ad9d640 1 --2- 172.21.15.106:0/4139273388 >> v2:172.21.15.114:3300/0 conn(0x7efd7c0b5390 0x7efd7c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7a59c640 1 -- 172.21.15.106:0/4139273388 >> v2:172.21.15.114:3300/0 conn(0x7efd7c0b5390 msgr2=0x7efd7c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.075+0000 7efd7a59c640 1 --2- 172.21.15.106:0/4139273388 >> v2:172.21.15.114:3300/0 conn(0x7efd7c0b5390 0x7efd7c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7a59c640 1 -- 172.21.15.106:0/4139273388 >> v1:172.21.15.114:6789/0 conn(0x7efd7c0b4730 legacy=0x7efd7c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7a59c640 1 -- 172.21.15.106:0/4139273388 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c0b3f30 con 0x7efd7c15b4e0 2024-04-13T03:15:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7a59c640 1 --2- 172.21.15.106:0/4139273388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c15fb60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0cd9e0 tx=0x7efd6c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeb14bcdb92f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7959a640 1 -- 172.21.15.106:0/4139273388 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c02ca80 con 0x7efd7c15b4e0 2024-04-13T03:15:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7959a640 1 -- 172.21.15.106:0/4139273388 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c02cc20 con 0x7efd7c15b4e0 2024-04-13T03:15:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.076+0000 7efd7959a640 1 -- 172.21.15.106:0/4139273388 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0343f0 con 0x7efd7c15b4e0 2024-04-13T03:15:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 -- 172.21.15.106:0/4139273388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 msgr2=0x7efd7c15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 --2- 172.21.15.106:0/4139273388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c15fb60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0cd9e0 tx=0x7efd6c02bee0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 -- 172.21.15.106:0/4139273388 shutdown_connections 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 --2- 172.21.15.106:0/4139273388 >> v2:172.21.15.114:3300/0 conn(0x7efd7c0b5390 0x7efd7c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 --2- 172.21.15.106:0/4139273388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c15fb60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 -- 172.21.15.106:0/4139273388 >> 172.21.15.106:0/4139273388 conn(0x7efd7c0b2600 msgr2=0x7efd7c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 -- 172.21.15.106:0/4139273388 shutdown_connections 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.077+0000 7efd80ba3640 1 -- 172.21.15.106:0/4139273388 wait complete. 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.078+0000 7efd80ba3640 1 Processor -- start 2024-04-13T03:15:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.078+0000 7efd80ba3640 1 -- start start 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efd7c0b4730 0x7efd7c1e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd7c0b5390 0x7efd7c13f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c06d0c0 con 0x7efd7c15b4e0 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0b32d0 con 0x7efd7c0b5390 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd80ba3640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c1404a0 con 0x7efd7c0b4730 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47718/0 (socket says 172.21.15.106:47718) 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd7ad9d640 1 -- 172.21.15.106:0/1939997276 learned_addr learned my addr 172.21.15.106:0/1939997276 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd79d9b640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd7c0b5390 0x7efd7c13f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.079+0000 7efd7a59c640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efd7c0b4730 0x7efd7c1e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 -- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efd7c0b4730 msgr2=0x7efd7c1e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efd7c0b4730 0x7efd7c1e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 -- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd7c0b5390 msgr2=0x7efd7c13f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd7c0b5390 0x7efd7c13f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 -- 172.21.15.106:0/1939997276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c140780 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd7ad9d640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efd7400d5d0 tx=0x7efd7400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd740125f0 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74019070 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c140a60 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd740129d0 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.080+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1e5720 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.081+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c068a10 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.082+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd74012b70 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.082+0000 7efd6affd640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 0x7efd5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.082+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6001+0+0 (secure 0 0 0) 0x7efd7408bf70 con 0x7efd7c15b4e0 2024-04-13T03:15:19.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.082+0000 7efd7a59c640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 0x7efd5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:19.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.082+0000 7efd7a59c640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 0x7efd5c06edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efd6c035000 tx=0x7efd6c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:19.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.087+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd74059c20 con 0x7efd7c15b4e0 2024-04-13T03:15:19.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.299+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efd7c06d240 con 0x7efd7c15b4e0 2024-04-13T03:15:19.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.949+0000 7efd6affd640 1 -- 172.21.15.106:0/1939997276 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) ==== 174+0+0 (secure 0 0 0) 0x7efd7405df20 con 0x7efd7c15b4e0 2024-04-13T03:15:19.950 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 13 and data pool 14 2024-04-13T03:15:19.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.954+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 msgr2=0x7efd5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.954+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 0x7efd5c06edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efd6c035000 tx=0x7efd6c030000 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.954+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 msgr2=0x7efd7c13fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.954+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efd7400d5d0 tx=0x7efd7400da90 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 shutdown_connections 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efd5c06c8f0 0x7efd5c06edb0 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efd6c035000 tx=0x7efd6c030000 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd7c15b4e0 0x7efd7c13fb80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd7c0b5390 0x7efd7c13f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 --2- 172.21.15.106:0/1939997276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efd7c0b4730 0x7efd7c1e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:19.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.955+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 >> 172.21.15.106:0/1939997276 conn(0x7efd7c0b2600 msgr2=0x7efd7c072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:19.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.971+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 shutdown_connections 2024-04-13T03:15:19.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:19.971+0000 7efd80ba3640 1 -- 172.21.15.106:0/1939997276 wait complete. 2024-04-13T03:15:19.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:20.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.093+0000 7fac73488640 1 Processor -- start 2024-04-13T03:15:20.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.093+0000 7fac73488640 1 -- start start 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac73488640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac6c0b8080 0x7fac6c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac73488640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac6c0d3530 con 0x7fac6c1622d0 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac73488640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac6c0d0b60 con 0x7fac6c0d1c00 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac73488640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c1626b0 con 0x7fac6c0b8080 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71224640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fac6c1622d0 0x7fac6c0b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46294/0 (socket says 172.21.15.106:46294) 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71a25640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac6c0b8080 0x7fac6c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71224640 1 -- 172.21.15.106:0/3442449401 learned_addr learned my addr 172.21.15.106:0/3442449401 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71a25640 1 -- 172.21.15.106:0/3442449401 >> v1:172.21.15.114:6789/0 conn(0x7fac6c0d1c00 legacy=0x7fac6c1668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:20.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71a25640 1 -- 172.21.15.106:0/3442449401 >> v1:172.21.15.106:6789/0 conn(0x7fac6c1622d0 legacy=0x7fac6c0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac63fff640 1 -- 172.21.15.106:0/3442449401 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871044433 0 0) 0x7fac6c0d3530 con 0x7fac6c1622d0 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.094+0000 7fac71a25640 1 -- 172.21.15.106:0/3442449401 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0d0e40 con 0x7fac6c0b8080 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.095+0000 7fac71a25640 1 --2- 172.21.15.106:0/3442449401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0b8080 0x7fac6c0d04d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fac64009d90 tx=0x7fac6402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef843617e3fc6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.095+0000 7fac63fff640 1 -- 172.21.15.106:0/3442449401 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64039070 con 0x7fac6c0b8080 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.095+0000 7fac63fff640 1 -- 172.21.15.106:0/3442449401 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6402cd00 con 0x7fac6c0b8080 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.095+0000 7fac63fff640 1 -- 172.21.15.106:0/3442449401 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64034660 con 0x7fac6c0b8080 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 -- 172.21.15.106:0/3442449401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0b8080 msgr2=0x7fac6c0d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 --2- 172.21.15.106:0/3442449401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0b8080 0x7fac6c0d04d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fac64009d90 tx=0x7fac6402bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:20.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 -- 172.21.15.106:0/3442449401 shutdown_connections 2024-04-13T03:15:20.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 --2- 172.21.15.106:0/3442449401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0b8080 0x7fac6c0d04d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 -- 172.21.15.106:0/3442449401 >> 172.21.15.106:0/3442449401 conn(0x7fac6c0d25d0 msgr2=0x7fac6c0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:20.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 -- 172.21.15.106:0/3442449401 shutdown_connections 2024-04-13T03:15:20.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.096+0000 7fac73488640 1 -- 172.21.15.106:0/3442449401 wait complete. 2024-04-13T03:15:20.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.097+0000 7fac73488640 1 Processor -- start 2024-04-13T03:15:20.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.097+0000 7fac73488640 1 -- start start 2024-04-13T03:15:20.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 0x7fac6c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac6c1622d0 0x7fac6c1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d16e0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d1400 con 0x7fac6c0b8080 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac73488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0cc6f0 con 0x7fac6c1622d0 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47738/0 (socket says 172.21.15.106:47738) 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 -- 172.21.15.106:0/3159046361 learned_addr learned my addr 172.21.15.106:0/3159046361 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac71224640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 0x7fac6c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac71a25640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac6c1622d0 0x7fac6c1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 -- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac6c1622d0 msgr2=0x7fac6c1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac6c1622d0 0x7fac6c1f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 -- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 msgr2=0x7fac6c1513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 0x7fac6c1513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 -- 172.21.15.106:0/3159046361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0cc450 con 0x7fac6c0d1c00 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac71224640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 0x7fac6c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.098+0000 7fac70a23640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c450 tx=0x7fac5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.099+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0052d0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.099+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c005470 con 0x7fac6c0d1c00 2024-04-13T03:15:20.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.099+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c012870 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.099+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c0cb8d0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.099+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c0cbe90 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.100+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c068a10 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.100+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac5c012a10 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.100+0000 7fac61ffb640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.101+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fac5c08b890 con 0x7fac6c0d1c00 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.101+0000 7fac71224640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.101+0000 7fac71224640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 0x7fac5006edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac68000f60 tx=0x7fac68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.105+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac5c059bc0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.282+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac6c151fd0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.283+0000 7fac61ffb640 1 -- 172.21.15.106:0/3159046361 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16934 (secure 0 0 0) 0x7fac5c0594c0 con 0x7fac6c0d1c00 2024-04-13T03:15:20.285 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:20.285 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":90,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:19.300992+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 msgr2=0x7fac5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 0x7fac5006edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac68000f60 tx=0x7fac68009040 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 msgr2=0x7fac6c0b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c450 tx=0x7fac5c00c910 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 shutdown_connections 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac5006c8f0 0x7fac5006edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac6c1622d0 0x7fac6c1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac6c0d1c00 0x7fac6c0b38c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 --2- 172.21.15.106:0/3159046361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac6c0b8080 0x7fac6c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 >> 172.21.15.106:0/3159046361 conn(0x7fac6c0d25d0 msgr2=0x7fac6c15e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 shutdown_connections 2024-04-13T03:15:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.286+0000 7fac73488640 1 -- 172.21.15.106:0/3159046361 wait complete. 2024-04-13T03:15:20.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:20.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.398+0000 7f38b92de640 1 Processor -- start 2024-04-13T03:15:20.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.398+0000 7f38b92de640 1 -- start start 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b415e9b0 0x7f38b40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 0x7f38b40d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38b40b3d60 0x7f38b40d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f38b40d5070 con 0x7f38b40b3d60 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f38b40cfd20 con 0x7f38b415ed90 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b92de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38b4162af0 con 0x7f38b415e9b0 2024-04-13T03:15:20.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38b40b3d60 0x7f38b40d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f38b40b3d60 0x7f38b40d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47744/0 (socket says 172.21.15.106:47744) 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 -- 172.21.15.106:0/3330836620 learned_addr learned my addr 172.21.15.106:0/3330836620 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38abfff640 1 --2- 172.21.15.106:0/3330836620 >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 0x7f38b40d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 -- 172.21.15.106:0/3330836620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b415e9b0 msgr2=0x7f38b40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b2ffd640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b415e9b0 0x7f38b40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b415e9b0 0x7f38b40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 -- 172.21.15.106:0/3330836620 >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 msgr2=0x7f38b40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 --2- 172.21.15.106:0/3330836620 >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 0x7f38b40d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38b37fe640 1 -- 172.21.15.106:0/3330836620 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b4162dd0 con 0x7f38b40b3d60 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.399+0000 7f38abfff640 1 --2- 172.21.15.106:0/3330836620 >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 0x7f38b40d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:20.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b37fe640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b40d4b30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f38b4162d00 tx=0x7f38ac02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b54d06c2d53d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b27fc640 1 -- 172.21.15.106:0/3330836620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac02caf0 con 0x7f38b40b3d60 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b27fc640 1 -- 172.21.15.106:0/3330836620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac02cc90 con 0x7f38b40b3d60 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b27fc640 1 -- 172.21.15.106:0/3330836620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac034610 con 0x7f38b40b3d60 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b92de640 1 -- 172.21.15.106:0/3330836620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 msgr2=0x7f38b40d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.400+0000 7f38b92de640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b40d4b30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f38b4162d00 tx=0x7f38ac02bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 -- 172.21.15.106:0/3330836620 shutdown_connections 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b40d4b30 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 --2- 172.21.15.106:0/3330836620 >> v2:172.21.15.114:3300/0 conn(0x7f38b415ed90 0x7f38b40d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 --2- 172.21.15.106:0/3330836620 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b415e9b0 0x7f38b40d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 -- 172.21.15.106:0/3330836620 >> 172.21.15.106:0/3330836620 conn(0x7f38b408bf10 msgr2=0x7f38b40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:20.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 -- 172.21.15.106:0/3330836620 shutdown_connections 2024-04-13T03:15:20.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.401+0000 7f38b92de640 1 -- 172.21.15.106:0/3330836620 wait complete. 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 Processor -- start 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 -- start start 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b415e9b0 0x7f38b406bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 0x7f38b40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4162680 con 0x7f38b40b3d60 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4163390 con 0x7f38b415e9b0 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.402+0000 7f38b92de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f38b4162800 con 0x7f38b40c9170 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47758/0 (socket says 172.21.15.106:47758) 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 -- 172.21.15.106:0/4254197112 learned_addr learned my addr 172.21.15.106:0/4254197112 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38abfff640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b415e9b0 0x7f38b406bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b37fe640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 0x7f38b40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 -- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 msgr2=0x7f38b40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 0x7f38b40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 -- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b415e9b0 msgr2=0x7f38b406bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b415e9b0 0x7f38b406bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 -- 172.21.15.106:0/4254197112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b40c9c30 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b37fe640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 0x7f38b40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b2ffd640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c7f0 tx=0x7f38a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4015070 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a4004240 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b40c9f10 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4007620 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.403+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b40ca4d0 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.404+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b40ca800 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.405+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38a40077c0 con 0x7f38b40b3d60 2024-04-13T03:15:20.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.405+0000 7f38ab7fe640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.405+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f38a408bf60 con 0x7f38b40b3d60 2024-04-13T03:15:20.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.405+0000 7f38abfff640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.405+0000 7f38abfff640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.409+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38a405a200 con 0x7f38b40b3d60 2024-04-13T03:15:20.577 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:15:20.577+0000 7f45bb1f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:15:20.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.586+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38b40d53a0 con 0x7f38b40b3d60 2024-04-13T03:15:20.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.587+0000 7f38ab7fe640 1 -- 172.21.15.106:0/4254197112 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16934 (secure 0 0 0) 0x7f38a4059b00 con 0x7f38b40b3d60 2024-04-13T03:15:20.588 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:20.588 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":90,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:19.300992+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:20.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 msgr2=0x7f389406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c008040 comp rx=0 tx=0).stop 2024-04-13T03:15:20.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 msgr2=0x7f38b414d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c7f0 tx=0x7f38a400ccb0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 shutdown_connections 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f38b40c9170 0x7f38b40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b415e9b0 0x7f38b406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 --2- 172.21.15.106:0/4254197112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b40b3d60 0x7f38b414d8d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 >> 172.21.15.106:0/4254197112 conn(0x7f38b408bf10 msgr2=0x7f38b40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 shutdown_connections 2024-04-13T03:15:20.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.589+0000 7f38b92de640 1 -- 172.21.15.106:0/4254197112 wait complete. 2024-04-13T03:15:20.601 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:15:20.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.704+0000 7f3f7e0ad640 1 Processor -- start 2024-04-13T03:15:20.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.704+0000 7f3f7e0ad640 1 -- start start 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f7e0ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f780704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f7e0ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f780709e0 0x7f3f7806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f7e0ad640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f780b5cf0 con 0x7f3f780b1c30 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f7e0ad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3f780b5fd0 con 0x7f3f780709e0 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f7e0ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3f780b62b0 con 0x7f3f7815ee40 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f780709e0 0x7f3f7806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f780709e0 0x7f3f7806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47760/0 (socket says 172.21.15.106:47760) 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 -- 172.21.15.106:0/1455422868 learned_addr learned my addr 172.21.15.106:0/1455422868 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f76ffd640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f780704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 -- 172.21.15.106:0/1455422868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 msgr2=0x7f3f780704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f780704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 -- 172.21.15.106:0/1455422868 >> v1:172.21.15.114:6789/0 conn(0x7f3f780b1c30 legacy=0x7f3f780b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.705+0000 7f3f77fff640 1 -- 172.21.15.106:0/1455422868 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780b6590 con 0x7f3f780709e0 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.706+0000 7f3f77fff640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780709e0 0x7f3f7806d660 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009e80 tx=0x7f3f7002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3f98911f760538 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.706+0000 7f3f767fc640 1 -- 172.21.15.106:0/1455422868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7002caf0 con 0x7f3f780709e0 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.706+0000 7f3f767fc640 1 -- 172.21.15.106:0/1455422868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7002cc90 con 0x7f3f780709e0 2024-04-13T03:15:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.706+0000 7f3f767fc640 1 -- 172.21.15.106:0/1455422868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70034610 con 0x7f3f780709e0 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1455422868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780709e0 msgr2=0x7f3f7806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780709e0 0x7f3f7806d660 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009e80 tx=0x7f3f7002bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1455422868 shutdown_connections 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780709e0 0x7f3f7806d660 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1455422868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f780704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1455422868 >> 172.21.15.106:0/1455422868 conn(0x7f3f78071870 msgr2=0x7f3f78071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1455422868 shutdown_connections 2024-04-13T03:15:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.707+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1455422868 wait complete. 2024-04-13T03:15:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.708+0000 7f3f7e0ad640 1 Processor -- start 2024-04-13T03:15:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.708+0000 7f3f7e0ad640 1 -- start start 2024-04-13T03:15:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 0x7f3f781f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f78148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7806df30 con 0x7f3f780b1c30 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7806ddb0 con 0x7f3f780709e0 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7806e0b0 con 0x7f3f7815ee40 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47772/0 (socket says 172.21.15.106:47772) 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 -- 172.21.15.106:0/1227345715 learned_addr learned my addr 172.21.15.106:0/1227345715 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f77fff640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f78148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f777fe640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 0x7f3f781f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 -- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 msgr2=0x7f3f78148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f78148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 -- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 msgr2=0x7f3f781f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 0x7f3f781f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 -- 172.21.15.106:0/1227345715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f78147250 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f777fe640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 0x7f3f781f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f77fff640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f78148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f76ffd640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d620 tx=0x7f3f6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00b620 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c019070 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00ba00 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.709+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f78147530 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.710+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f78147af0 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.711+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f78068a10 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.711+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f6c00bba0 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.711+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.711+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f3f6c08c0e0 con 0x7f3f780b1c30 2024-04-13T03:15:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.711+0000 7f3f777fe640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:20.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.712+0000 7f3f777fe640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 0x7f3f6406edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3f68001680 tx=0x7f3f68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:20.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.716+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f6c05a380 con 0x7f3f780b1c30 2024-04-13T03:15:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:20.926+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3f7815a670 con 0x7f3f780b1c30 2024-04-13T03:15:21.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.009+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1227345715 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) ==== 118+0+0 (secure 0 0 0) 0x7f3f6c059c80 con 0x7f3f780b1c30 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 msgr2=0x7f3f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 0x7f3f6406edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3f68001680 tx=0x7f3f68009040 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 msgr2=0x7f3f781482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00d620 tx=0x7f3f6c00dae0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 shutdown_connections 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f6406c8f0 0x7f3f6406edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f7815ee40 0x7f3f78148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f780b1c30 0x7f3f781482f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 --2- 172.21.15.106:0/1227345715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f780709e0 0x7f3f781f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 >> 172.21.15.106:0/1227345715 conn(0x7f3f78071870 msgr2=0x7f3f7815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 shutdown_connections 2024-04-13T03:15:21.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.012+0000 7f3f7e0ad640 1 -- 172.21.15.106:0/1227345715 wait complete. 2024-04-13T03:15:21.023 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:15:21.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:15:21.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.126+0000 7fb16d820640 1 Processor -- start 2024-04-13T03:15:21.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.126+0000 7fb16d820640 1 -- start start 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb16d820640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb16815e9b0 0x7fb1680d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb16d820640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb16815ed90 0x7fb1680d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb16d820640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1680cfd20 con 0x7fb1680b3d60 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb16d820640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb168162af0 con 0x7fb16815ed90 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb16d820640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb168162dd0 con 0x7fb16815e9b0 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1667fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb16815ed90 0x7fb1680d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1677fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb1680b3d60 0x7fb1680d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46306/0 (socket says 172.21.15.106:46306) 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1677fe640 1 -- 172.21.15.106:0/2252799036 learned_addr learned my addr 172.21.15.106:0/2252799036 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1667fc640 1 -- 172.21.15.106:0/2252799036 >> v2:172.21.15.114:3300/0 conn(0x7fb16815e9b0 msgr2=0x7fb1680d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1667fc640 1 --2- 172.21.15.106:0/2252799036 >> v2:172.21.15.114:3300/0 conn(0x7fb16815e9b0 0x7fb1680d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1667fc640 1 -- 172.21.15.106:0/2252799036 >> v1:172.21.15.106:6789/0 conn(0x7fb1680b3d60 legacy=0x7fb1680d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb165ffb640 1 -- 172.21.15.106:0/2252799036 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865406225 0 0) 0x7fb1680cfd20 con 0x7fb1680b3d60 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.127+0000 7fb1667fc640 1 -- 172.21.15.106:0/2252799036 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1681630b0 con 0x7fb16815ed90 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb1667fc640 1 --2- 172.21.15.106:0/2252799036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb16815ed90 0x7fb1680d45f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb16808e4e0 tx=0x7fb15c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0492c63071d586 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb165ffb640 1 -- 172.21.15.106:0/2252799036 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c034070 con 0x7fb16815ed90 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb165ffb640 1 -- 172.21.15.106:0/2252799036 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c030610 con 0x7fb16815ed90 2024-04-13T03:15:21.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb165ffb640 1 -- 172.21.15.106:0/2252799036 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0309e0 con 0x7fb16815ed90 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 -- 172.21.15.106:0/2252799036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb16815ed90 msgr2=0x7fb1680d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 --2- 172.21.15.106:0/2252799036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb16815ed90 0x7fb1680d45f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb16808e4e0 tx=0x7fb15c02b960 comp rx=0 tx=0).stop 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 -- 172.21.15.106:0/2252799036 shutdown_connections 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 --2- 172.21.15.106:0/2252799036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb16815ed90 0x7fb1680d45f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 --2- 172.21.15.106:0/2252799036 >> v2:172.21.15.114:3300/0 conn(0x7fb16815e9b0 0x7fb1680d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 -- 172.21.15.106:0/2252799036 >> 172.21.15.106:0/2252799036 conn(0x7fb16808bf10 msgr2=0x7fb1680b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.128+0000 7fb16d820640 1 -- 172.21.15.106:0/2252799036 shutdown_connections 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.129+0000 7fb16d820640 1 -- 172.21.15.106:0/2252799036 wait complete. 2024-04-13T03:15:21.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 Processor -- start 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 -- start start 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb16815e9b0 0x7fb168161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 0x7fb16815f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.130+0000 7fb16d820640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb168162800 con 0x7fb1680b3d60 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb16d820640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb168162680 con 0x7fb16815f180 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb16d820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb16806eb20 con 0x7fb16815e9b0 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47800/0 (socket says 172.21.15.106:47800) 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 -- 172.21.15.106:0/3560344353 learned_addr learned my addr 172.21.15.106:0/3560344353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 -- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb16815e9b0 msgr2=0x7fb168161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb16815e9b0 0x7fb168161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb1677fe640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 0x7fb16815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:21.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 -- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 msgr2=0x7fb16815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 0x7fb16815f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 -- 172.21.15.106:0/3560344353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16806cbf0 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb1677fe640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 0x7fb16815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb166ffd640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b4d0 tx=0x7fb15800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158014070 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158010870 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1680b6cf0 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158010c60 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.131+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16806d140 con 0x7fb1680b3d60 2024-04-13T03:15:21.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.132+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb168068a10 con 0x7fb1680b3d60 2024-04-13T03:15:21.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.133+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb158017070 con 0x7fb1680b3d60 2024-04-13T03:15:21.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.133+0000 7fb14ffff640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:21.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.133+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb15808ba70 con 0x7fb1680b3d60 2024-04-13T03:15:21.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.133+0000 7fb1667fc640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:21.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.133+0000 7fb1667fc640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 0x7fb15406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb15c036000 tx=0x7fb15c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:21.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.137+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb158059da0 con 0x7fb1680b3d60 2024-04-13T03:15:21.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:21.351+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb1680d53a0 con 0x7fb1680b3d60 2024-04-13T03:15:22.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.011+0000 7fb14ffff640 1 -- 172.21.15.106:0/3560344353 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fb1580596a0 con 0x7fb1680b3d60 2024-04-13T03:15:22.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 msgr2=0x7fb15406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 0x7fb15406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb15c036000 tx=0x7fb15c030000 comp rx=0 tx=0).stop 2024-04-13T03:15:22.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 msgr2=0x7fb168152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b4d0 tx=0x7fb15800b990 comp rx=0 tx=0).stop 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 shutdown_connections 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb16815e9b0 0x7fb168161e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16815f180 0x7fb16815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 --2- 172.21.15.106:0/3560344353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1680b3d60 0x7fb168152b20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 >> 172.21.15.106:0/3560344353 conn(0x7fb16808bf10 msgr2=0x7fb1680d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 shutdown_connections 2024-04-13T03:15:22.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.013+0000 7fb16d820640 1 -- 172.21.15.106:0/3560344353 wait complete. 2024-04-13T03:15:22.027 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:15:22.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:22.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.131+0000 7fed32d7b640 1 Processor -- start 2024-04-13T03:15:22.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.131+0000 7fed32d7b640 1 -- start start 2024-04-13T03:15:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed32d7b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed2c0aebe0 0x7fed2c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed32d7b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed2c0b5ab0 0x7fed2c06e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed32d7b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0aefc0 con 0x7fed2c15ce90 2024-04-13T03:15:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed32d7b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fed2c06e180 con 0x7fed2c0aebe0 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed32d7b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fed2c0ad180 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed2c0aebe0 0x7fed2c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed2bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fed2c15ce90 0x7fed2c0b53a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46318/0 (socket says 172.21.15.106:46318) 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed2c0aebe0 0x7fed2c1603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47810/0 (socket says 172.21.15.106:47810) 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed2bfff640 1 -- 172.21.15.106:0/3748449426 learned_addr learned my addr 172.21.15.106:0/3748449426 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed31318640 1 --2- 172.21.15.106:0/3748449426 >> v2:172.21.15.114:3300/0 conn(0x7fed2c0b5ab0 0x7fed2c06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 -- 172.21.15.106:0/3748449426 >> v2:172.21.15.114:3300/0 conn(0x7fed2c0b5ab0 msgr2=0x7fed2c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 --2- 172.21.15.106:0/3748449426 >> v2:172.21.15.114:3300/0 conn(0x7fed2c0b5ab0 0x7fed2c06e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 -- 172.21.15.106:0/3748449426 >> v1:172.21.15.106:6789/0 conn(0x7fed2c15ce90 legacy=0x7fed2c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed2b7fe640 1 -- 172.21.15.106:0/3748449426 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2059858939 0 0) 0x7fed20001110 con 0x7fed2c15ce90 2024-04-13T03:15:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.132+0000 7fed30b17640 1 -- 172.21.15.106:0/3748449426 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c160ca0 con 0x7fed2c0aebe0 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.133+0000 7fed30b17640 1 --2- 172.21.15.106:0/3748449426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0aebe0 0x7fed2c1603a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06e300 tx=0x7fed2402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdce1b3755851b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.133+0000 7fed2b7fe640 1 -- 172.21.15.106:0/3748449426 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24039070 con 0x7fed2c0aebe0 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.133+0000 7fed2b7fe640 1 -- 172.21.15.106:0/3748449426 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2402cbd0 con 0x7fed2c0aebe0 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.133+0000 7fed2b7fe640 1 -- 172.21.15.106:0/3748449426 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24034440 con 0x7fed2c0aebe0 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 -- 172.21.15.106:0/3748449426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0aebe0 msgr2=0x7fed2c1603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 --2- 172.21.15.106:0/3748449426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0aebe0 0x7fed2c1603a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06e300 tx=0x7fed2402be20 comp rx=0 tx=0).stop 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 -- 172.21.15.106:0/3748449426 shutdown_connections 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 --2- 172.21.15.106:0/3748449426 >> v2:172.21.15.114:3300/0 conn(0x7fed2c0b5ab0 0x7fed2c06e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 --2- 172.21.15.106:0/3748449426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0aebe0 0x7fed2c1603a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 -- 172.21.15.106:0/3748449426 >> 172.21.15.106:0/3748449426 conn(0x7fed2c0b1f10 msgr2=0x7fed2c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 -- 172.21.15.106:0/3748449426 shutdown_connections 2024-04-13T03:15:22.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.134+0000 7fed32d7b640 1 -- 172.21.15.106:0/3748449426 wait complete. 2024-04-13T03:15:22.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.135+0000 7fed32d7b640 1 Processor -- start 2024-04-13T03:15:22.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.135+0000 7fed32d7b640 1 -- start start 2024-04-13T03:15:22.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed2c0aebe0 0x7fed2c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 0x7fed2c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c06de10 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c161260 con 0x7fed2c15ce90 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c1608e0 con 0x7fed2c0aebe0 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47826/0 (socket says 172.21.15.106:47826) 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 -- 172.21.15.106:0/1159385158 learned_addr learned my addr 172.21.15.106:0/1159385158 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed31318640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 0x7fed2c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed30b17640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed2c0aebe0 0x7fed2c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 -- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed2c0aebe0 msgr2=0x7fed2c154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed2c0aebe0 0x7fed2c154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 -- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 msgr2=0x7fed2c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 0x7fed2c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 -- 172.21.15.106:0/1159385158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c155500 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed31318640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 0x7fed2c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed2bfff640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c420 tx=0x7fed2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed200074e0 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed200047e0 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20004c10 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c15a910 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.136+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c15aed0 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.137+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c068a10 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.138+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed20011070 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.138+0000 7fed297fa640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.138+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fed2008b820 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.138+0000 7fed30b17640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.139+0000 7fed30b17640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 0x7fed1806edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fed24036000 tx=0x7fed24030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.143+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed2c0aefc0 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.352+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2c06ba20 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.353+0000 7fed297fa640 1 -- 172.21.15.106:0/1159385158 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5195 (secure 0 0 0) 0x7fed2000e280 con 0x7fed2c0b5ab0 2024-04-13T03:15:22.354 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:22.354 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:21.352812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/3360662124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3360662124},{"type":"v1","addr":"172.21.15.106:6837","nonce":3360662124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-13T03:15:22.354 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 51 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 msgr2=0x7fed1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 0x7fed1806edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fed24036000 tx=0x7fed24030000 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 msgr2=0x7fed2c15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c420 tx=0x7fed2000c8e0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 shutdown_connections 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed2c15ce90 0x7fed2c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed2c0b5ab0 0x7fed2c15a0c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 --2- 172.21.15.106:0/1159385158 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fed2c0aebe0 0x7fed2c154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 >> 172.21.15.106:0/1159385158 conn(0x7fed2c0b1f10 msgr2=0x7fed2c06f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 shutdown_connections 2024-04-13T03:15:22.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.355+0000 7fed32d7b640 1 -- 172.21.15.106:0/1159385158 wait complete. 2024-04-13T03:15:22.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:22.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.467+0000 7fe575795640 1 Processor -- start 2024-04-13T03:15:22.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.467+0000 7fe575795640 1 -- start start 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe575795640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe57015e9b0 0x7fe5700d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe575795640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5700d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe575795640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5700cfd20 con 0x7fe57015ed90 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe575795640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe570162af0 con 0x7fe57015e9b0 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe575795640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe570162dd0 con 0x7fe5700b3d60 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe57015e9b0 0x7fe5700d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe567fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe57015ed90 0x7fe5700d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46322/0 (socket says 172.21.15.106:46322) 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe57015e9b0 0x7fe5700d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47836/0 (socket says 172.21.15.106:47836) 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe567fff640 1 -- 172.21.15.106:0/1303539439 learned_addr learned my addr 172.21.15.106:0/1303539439 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:22.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56f7fe640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5700d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 -- 172.21.15.106:0/1303539439 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 msgr2=0x7fe5700d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5700d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 -- 172.21.15.106:0/1303539439 >> v1:172.21.15.106:6789/0 conn(0x7fe57015ed90 legacy=0x7fe5700d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56effd640 1 -- 172.21.15.106:0/1303539439 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5701630b0 con 0x7fe57015e9b0 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.468+0000 7fe56e7fc640 1 -- 172.21.15.106:0/1303539439 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119873603 0 0) 0x7fe5700cfd20 con 0x7fe57015ed90 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.469+0000 7fe56effd640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700d2300 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097a0 tx=0x7fe56002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151b8f023425f587 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.469+0000 7fe56e7fc640 1 -- 172.21.15.106:0/1303539439 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560034070 con 0x7fe57015e9b0 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.469+0000 7fe56e7fc640 1 -- 172.21.15.106:0/1303539439 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe560030610 con 0x7fe57015e9b0 2024-04-13T03:15:22.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.469+0000 7fe56e7fc640 1 -- 172.21.15.106:0/1303539439 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600309e0 con 0x7fe57015e9b0 2024-04-13T03:15:22.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 -- 172.21.15.106:0/1303539439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 msgr2=0x7fe5700d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700d2300 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097a0 tx=0x7fe56002b960 comp rx=0 tx=0).stop 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 -- 172.21.15.106:0/1303539439 shutdown_connections 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5700d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 --2- 172.21.15.106:0/1303539439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700d2300 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 -- 172.21.15.106:0/1303539439 >> 172.21.15.106:0/1303539439 conn(0x7fe57008bf10 msgr2=0x7fe5700b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 -- 172.21.15.106:0/1303539439 shutdown_connections 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.470+0000 7fe575795640 1 -- 172.21.15.106:0/1303539439 wait complete. 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.471+0000 7fe575795640 1 Processor -- start 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.471+0000 7fe575795640 1 -- start start 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5701610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe570159b00 0x7fe57014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe570162800 con 0x7fe57015e9b0 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe570162680 con 0x7fe570159b00 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe5700d4d00 con 0x7fe5700b3d60 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47848/0 (socket says 172.21.15.106:47848) 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 -- 172.21.15.106:0/2622192106 learned_addr learned my addr 172.21.15.106:0/2622192106 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe56f7fe640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe570159b00 0x7fe57014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe56effd640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5701610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 -- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 msgr2=0x7fe5701610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5701610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 -- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe570159b00 msgr2=0x7fe57014f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe570159b00 0x7fe57014f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 -- 172.21.15.106:0/2622192106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5700cc3e0 con 0x7fe57015e9b0 2024-04-13T03:15:22.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe567fff640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe558001eb0 tx=0x7fe55800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558017070 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5580074e0 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580059e0 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5700cc6c0 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.472+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5700ccc80 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.473+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe570068a10 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.474+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe55801a070 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.474+0000 7fe5677fe640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 0x7fe55006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.474+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe55808b110 con 0x7fe57015e9b0 2024-04-13T03:15:22.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.474+0000 7fe56effd640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 0x7fe55006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.474+0000 7fe56effd640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 0x7fe55006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe560036000 tx=0x7fe56003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.478+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe558059470 con 0x7fe57015e9b0 2024-04-13T03:15:22.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.654+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5700b8290 con 0x7fe57015e9b0 2024-04-13T03:15:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.655+0000 7fe5677fe640 1 -- 172.21.15.106:0/2622192106 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16921 (secure 0 0 0) 0x7fe558058d70 con 0x7fe57015e9b0 2024-04-13T03:15:22.656 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:22.656 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":91,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:20.959782+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 msgr2=0x7fe55006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 0x7fe55006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe560036000 tx=0x7fe56003b040 comp rx=0 tx=0).stop 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 msgr2=0x7fe5700cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe558001eb0 tx=0x7fe55800c890 comp rx=0 tx=0).stop 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 shutdown_connections 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe55006c8f0 0x7fe55006edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe570159b00 0x7fe57014f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe57015e9b0 0x7fe5700cba70 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 --2- 172.21.15.106:0/2622192106 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5700b3d60 0x7fe5701610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 >> 172.21.15.106:0/2622192106 conn(0x7fe57008bf10 msgr2=0x7fe5700d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 shutdown_connections 2024-04-13T03:15:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.657+0000 7fe575795640 1 -- 172.21.15.106:0/2622192106 wait complete. 2024-04-13T03:15:22.669 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:15:22.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:15:22.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.768+0000 7fec1deb7640 1 Processor -- start 2024-04-13T03:15:22.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.768+0000 7fec1deb7640 1 -- start start 2024-04-13T03:15:22.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec1deb7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fec180ce580 0x7fec180d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec1deb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec180b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec1deb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec180b5d60 con 0x7fec180b5940 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec1deb7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fec180d1be0 con 0x7fec180ce580 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec1deb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec180d0b40 con 0x7fec180d06c0 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec17fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fec180b5940 0x7fec181627f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47122/0 (socket says 172.21.15.106:47122) 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec17fff640 1 -- 172.21.15.106:0/115032655 learned_addr learned my addr 172.21.15.106:0/115032655 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec177fe640 1 --2- 172.21.15.106:0/115032655 >> v2:172.21.15.114:3300/0 conn(0x7fec180ce580 0x7fec180d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec16ffd640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec180b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1148852497 0 0) 0x7fec180b5d60 con 0x7fec180b5940 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec00003520 con 0x7fec180b5940 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2528146919 0 0) 0x7fec00003520 con 0x7fec180b5940 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 msgr2=0x7fec180b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec180b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 >> v2:172.21.15.114:3300/0 conn(0x7fec180ce580 msgr2=0x7fec180d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 --2- 172.21.15.106:0/115032655 >> v2:172.21.15.114:3300/0 conn(0x7fec180ce580 0x7fec180d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec180d0e20 con 0x7fec180b5940 2024-04-13T03:15:22.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fec10001a10 con 0x7fec180b5940 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 >> v1:172.21.15.106:6789/0 conn(0x7fec180b5940 legacy=0x7fec181627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.769+0000 7fec167fc640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec000039b0 0x7fec00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec167fc640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec167fc640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec00004d40 0x7fec000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec180d1be0 con 0x7fec000039b0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec180d0b40 con 0x7fec00004d40 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec0000b9c0 con 0x7fec180d06c0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec000039b0 0x7fec00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec17fff640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec00004d40 0x7fec000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec16ffd640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 -- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 msgr2=0x7fec00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec180d06c0 0x7fec00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 -- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec00004d40 msgr2=0x7fec000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec00004d40 0x7fec000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.770+0000 7fec177fe640 1 -- 172.21.15.106:0/115032655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec00003520 con 0x7fec000039b0 2024-04-13T03:15:22.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec177fe640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec000039b0 0x7fec00003dd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fec08009af0 tx=0x7fec00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140d33540d152880 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08039070 con 0x7fec000039b0 2024-04-13T03:15:22.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0802c710 con 0x7fec000039b0 2024-04-13T03:15:22.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec167fc640 1 -- 172.21.15.106:0/115032655 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08009690 con 0x7fec000039b0 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 -- 172.21.15.106:0/115032655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec000039b0 msgr2=0x7fec00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 --2- 172.21.15.106:0/115032655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec000039b0 0x7fec00003dd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fec08009af0 tx=0x7fec00005830 comp rx=0 tx=0).stop 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec17fff640 1 -- 172.21.15.106:0/115032655 reap_dead start 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 -- 172.21.15.106:0/115032655 shutdown_connections 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 -- 172.21.15.106:0/115032655 >> 172.21.15.106:0/115032655 conn(0x7fec180d25b0 msgr2=0x7fec180d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 -- 172.21.15.106:0/115032655 shutdown_connections 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.771+0000 7fec1deb7640 1 -- 172.21.15.106:0/115032655 wait complete. 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.772+0000 7fec1deb7640 1 Processor -- start 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 -- start start 2024-04-13T03:15:22.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec000039b0 0x7fec1814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 0x7fec1815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec180d16c0 con 0x7fec1815db20 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec180d13e0 con 0x7fec000039b0 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec1deb7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fec180ce960 con 0x7fec1815c0e0 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec17fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec17fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53726/0 (socket says 172.21.15.106:53726) 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec17fff640 1 -- 172.21.15.106:0/1740735580 learned_addr learned my addr 172.21.15.106:0/1740735580 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.773+0000 7fec177fe640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec000039b0 0x7fec1814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec16ffd640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 0x7fec1815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 -- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 msgr2=0x7fec1815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 0x7fec1815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 -- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec000039b0 msgr2=0x7fec1814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec000039b0 0x7fec1814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 -- 172.21.15.106:0/1740735580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1814deb0 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec16ffd640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 0x7fec1815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec17fff640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fec10002090 tx=0x7fec1000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1000edc0 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec10018070 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1815b130 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10013660 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.774+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1815cf30 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.775+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec18074750 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.776+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec10013800 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.776+0000 7febfffff640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 0x7fec0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.776+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fec1008c020 con 0x7fec1815db20 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.776+0000 7fec177fe640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 0x7fec0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:22.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.776+0000 7fec177fe640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 0x7fec0406edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec08036000 tx=0x7fec08030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:22.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.780+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec1005a2c0 con 0x7fec1815db20 2024-04-13T03:15:22.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.951+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec180d3400 con 0x7fec0406c8f0 2024-04-13T03:15:22.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.957+0000 7febfffff640 1 -- 172.21.15.106:0/1740735580 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252072 (secure 0 0 0) 0x7fec180d3400 con 0x7fec0406c8f0 2024-04-13T03:15:22.960 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:22.962 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 msgr2=0x7fec0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 0x7fec0406edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec08036000 tx=0x7fec08030000 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 msgr2=0x7fec1815dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fec10002090 tx=0x7fec1000e530 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 shutdown_connections 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fec0406c8f0 0x7fec0406edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fec1815c0e0 0x7fec1815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec1815db20 0x7fec1815dfa0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 --2- 172.21.15.106:0/1740735580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec000039b0 0x7fec1814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 >> 172.21.15.106:0/1740735580 conn(0x7fec180d25b0 msgr2=0x7fec18163130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 shutdown_connections 2024-04-13T03:15:22.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:22.960+0000 7fec1deb7640 1 -- 172.21.15.106:0/1740735580 wait complete. 2024-04-13T03:15:22.972 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-04-13T03:15:22.440178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255312,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719408,"statfs":{"total":773094113280,"available":772832673792,"internally_reserved":0,"allocated":10231808,"data_stored":5321836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.494015"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245999+0000","last_change":"2024-04-13T03:15:19.964347+0000","last_active":"2024-04-13T03:15:21.245999+0000","last_peered":"2024-04-13T03:15:21.245999+0000","last_clean":"2024-04-13T03:15:21.245999+0000","last_became_active":"2024-04-13T03:15:19.964070+0000","last_became_peered":"2024-04-13T03:15:19.964070+0000","last_unstale":"2024-04-13T03:15:21.245999+0000","last_undegraded":"2024-04-13T03:15:21.245999+0000","last_fullsized":"2024-04-13T03:15:21.245999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:08:32.030220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246463+0000","last_change":"2024-04-13T03:15:19.966839+0000","last_active":"2024-04-13T03:15:21.246463+0000","last_peered":"2024-04-13T03:15:21.246463+0000","last_clean":"2024-04-13T03:15:21.246463+0000","last_became_active":"2024-04-13T03:15:19.966574+0000","last_became_peered":"2024-04-13T03:15:19.966574+0000","last_unstale":"2024-04-13T03:15:21.246463+0000","last_undegraded":"2024-04-13T03:15:21.246463+0000","last_fullsized":"2024-04-13T03:15:21.246463+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:12.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245762+0000","last_change":"2024-04-13T03:15:19.968398+0000","last_active":"2024-04-13T03:15:21.245762+0000","last_peered":"2024-04-13T03:15:21.245762+0000","last_clean":"2024-04-13T03:15:21.245762+0000","last_became_active":"2024-04-13T03:15:19.968118+0000","last_became_peered":"2024-04-13T03:15:19.968118+0000","last_unstale":"2024-04-13T03:15:21.245762+0000","last_undegraded":"2024-04-13T03:15:21.245762+0000","last_fullsized":"2024-04-13T03:15:21.245762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:15:58.438037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965799+0000","last_change":"2024-04-13T03:15:19.967764+0000","last_active":"2024-04-13T03:15:20.965799+0000","last_peered":"2024-04-13T03:15:20.965799+0000","last_clean":"2024-04-13T03:15:20.965799+0000","last_became_active":"2024-04-13T03:15:19.966897+0000","last_became_peered":"2024-04-13T03:15:19.966897+0000","last_unstale":"2024-04-13T03:15:20.965799+0000","last_undegraded":"2024-04-13T03:15:20.965799+0000","last_fullsized":"2024-04-13T03:15:20.965799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:09:20.958767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247545+0000","last_change":"2024-04-13T03:15:19.968823+0000","last_active":"2024-04-13T03:15:21.247545+0000","last_peered":"2024-04-13T03:15:21.247545+0000","last_clean":"2024-04-13T03:15:21.247545+0000","last_became_active":"2024-04-13T03:15:19.968564+0000","last_became_peered":"2024-04-13T03:15:19.968564+0000","last_unstale":"2024-04-13T03:15:21.247545+0000","last_undegraded":"2024-04-13T03:15:21.247545+0000","last_fullsized":"2024-04-13T03:15:21.247545+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:06:09.027110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964610+0000","last_change":"2024-04-13T03:15:19.975653+0000","last_active":"2024-04-13T03:15:20.964610+0000","last_peered":"2024-04-13T03:15:20.964610+0000","last_clean":"2024-04-13T03:15:20.964610+0000","last_became_active":"2024-04-13T03:15:19.975360+0000","last_became_peered":"2024-04-13T03:15:19.975360+0000","last_unstale":"2024-04-13T03:15:20.964610+0000","last_undegraded":"2024-04-13T03:15:20.964610+0000","last_fullsized":"2024-04-13T03:15:20.964610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:14.511264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247173+0000","last_change":"2024-04-13T03:15:19.971621+0000","last_active":"2024-04-13T03:15:21.247173+0000","last_peered":"2024-04-13T03:15:21.247173+0000","last_clean":"2024-04-13T03:15:21.247173+0000","last_became_active":"2024-04-13T03:15:19.971367+0000","last_became_peered":"2024-04-13T03:15:19.971367+0000","last_unstale":"2024-04-13T03:15:21.247173+0000","last_undegraded":"2024-04-13T03:15:21.247173+0000","last_fullsized":"2024-04-13T03:15:21.247173+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:49.786951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965579+0000","last_change":"2024-04-13T03:15:19.968071+0000","last_active":"2024-04-13T03:15:20.965579+0000","last_peered":"2024-04-13T03:15:20.965579+0000","last_clean":"2024-04-13T03:15:20.965579+0000","last_became_active":"2024-04-13T03:15:19.967788+0000","last_became_peered":"2024-04-13T03:15:19.967788+0000","last_unstale":"2024-04-13T03:15:20.965579+0000","last_undegraded":"2024-04-13T03:15:20.965579+0000","last_fullsized":"2024-04-13T03:15:20.965579+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:59:00.328223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246448+0000","last_change":"2024-04-13T03:15:19.970244+0000","last_active":"2024-04-13T03:15:21.246448+0000","last_peered":"2024-04-13T03:15:21.246448+0000","last_clean":"2024-04-13T03:15:21.246448+0000","last_became_active":"2024-04-13T03:15:19.969974+0000","last_became_peered":"2024-04-13T03:15:19.969974+0000","last_unstale":"2024-04-13T03:15:21.246448+0000","last_undegraded":"2024-04-13T03:15:21.246448+0000","last_fullsized":"2024-04-13T03:15:21.246448+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245517+0000","last_change":"2024-04-13T03:15:19.971517+0000","last_active":"2024-04-13T03:15:21.245517+0000","last_peered":"2024-04-13T03:15:21.245517+0000","last_clean":"2024-04-13T03:15:21.245517+0000","last_became_active":"2024-04-13T03:15:19.971249+0000","last_became_peered":"2024-04-13T03:15:19.971249+0000","last_unstale":"2024-04-13T03:15:21.245517+0000","last_undegraded":"2024-04-13T03:15:21.245517+0000","last_fullsized":"2024-04-13T03:15:21.245517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:44:17.964010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247240+0000","last_change":"2024-04-13T03:15:19.980895+0000","last_active":"2024-04-13T03:15:21.247240+0000","last_peered":"2024-04-13T03:15:21.247240+0000","last_clean":"2024-04-13T03:15:21.247240+0000","last_became_active":"2024-04-13T03:15:19.980624+0000","last_became_peered":"2024-04-13T03:15:19.980624+0000","last_unstale":"2024-04-13T03:15:21.247240+0000","last_undegraded":"2024-04-13T03:15:21.247240+0000","last_fullsized":"2024-04-13T03:15:21.247240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:09.461685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245412+0000","last_change":"2024-04-13T03:15:19.969165+0000","last_active":"2024-04-13T03:15:21.245412+0000","last_peered":"2024-04-13T03:15:21.245412+0000","last_clean":"2024-04-13T03:15:21.245412+0000","last_became_active":"2024-04-13T03:15:19.968854+0000","last_became_peered":"2024-04-13T03:15:19.968854+0000","last_unstale":"2024-04-13T03:15:21.245412+0000","last_undegraded":"2024-04-13T03:15:21.245412+0000","last_fullsized":"2024-04-13T03:15:21.245412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:25.178271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965679+0000","last_change":"2024-04-13T03:15:19.975395+0000","last_active":"2024-04-13T03:15:20.965679+0000","last_peered":"2024-04-13T03:15:20.965679+0000","last_clean":"2024-04-13T03:15:20.965679+0000","last_became_active":"2024-04-13T03:15:19.974911+0000","last_became_peered":"2024-04-13T03:15:19.974911+0000","last_unstale":"2024-04-13T03:15:20.965679+0000","last_undegraded":"2024-04-13T03:15:20.965679+0000","last_fullsized":"2024-04-13T03:15:20.965679+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:52.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246810+0000","last_change":"2024-04-13T03:15:19.974152+0000","last_active":"2024-04-13T03:15:21.246810+0000","last_peered":"2024-04-13T03:15:21.246810+0000","last_clean":"2024-04-13T03:15:21.246810+0000","last_became_active":"2024-04-13T03:15:19.973745+0000","last_became_peered":"2024-04-13T03:15:19.973745+0000","last_unstale":"2024-04-13T03:15:21.246810+0000","last_undegraded":"2024-04-13T03:15:21.246810+0000","last_fullsized":"2024-04-13T03:15:21.246810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:42.417633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965632+0000","last_change":"2024-04-13T03:15:19.970797+0000","last_active":"2024-04-13T03:15:20.965632+0000","last_peered":"2024-04-13T03:15:20.965632+0000","last_clean":"2024-04-13T03:15:20.965632+0000","last_became_active":"2024-04-13T03:15:19.970506+0000","last_became_peered":"2024-04-13T03:15:19.970506+0000","last_unstale":"2024-04-13T03:15:20.965632+0000","last_undegraded":"2024-04-13T03:15:20.965632+0000","last_fullsized":"2024-04-13T03:15:20.965632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:47.736587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964550+0000","last_change":"2024-04-13T03:15:19.979929+0000","last_active":"2024-04-13T03:15:20.964550+0000","last_peered":"2024-04-13T03:15:20.964550+0000","last_clean":"2024-04-13T03:15:20.964550+0000","last_became_active":"2024-04-13T03:15:19.979310+0000","last_became_peered":"2024-04-13T03:15:19.979310+0000","last_unstale":"2024-04-13T03:15:20.964550+0000","last_undegraded":"2024-04-13T03:15:20.964550+0000","last_fullsized":"2024-04-13T03:15:20.964550+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:45:20.338779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965050+0000","last_change":"2024-04-13T03:15:19.979589+0000","last_active":"2024-04-13T03:15:20.965050+0000","last_peered":"2024-04-13T03:15:20.965050+0000","last_clean":"2024-04-13T03:15:20.965050+0000","last_became_active":"2024-04-13T03:15:19.977584+0000","last_became_peered":"2024-04-13T03:15:19.977584+0000","last_unstale":"2024-04-13T03:15:20.965050+0000","last_undegraded":"2024-04-13T03:15:20.965050+0000","last_fullsized":"2024-04-13T03:15:20.965050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:20.025715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245662+0000","last_change":"2024-04-13T03:15:19.969038+0000","last_active":"2024-04-13T03:15:21.245662+0000","last_peered":"2024-04-13T03:15:21.245662+0000","last_clean":"2024-04-13T03:15:21.245662+0000","last_became_active":"2024-04-13T03:15:19.968571+0000","last_became_peered":"2024-04-13T03:15:19.968571+0000","last_unstale":"2024-04-13T03:15:21.245662+0000","last_undegraded":"2024-04-13T03:15:21.245662+0000","last_fullsized":"2024-04-13T03:15:21.245662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:33.741751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247121+0000","last_change":"2024-04-13T03:15:19.974321+0000","last_active":"2024-04-13T03:15:21.247121+0000","last_peered":"2024-04-13T03:15:21.247121+0000","last_clean":"2024-04-13T03:15:21.247121+0000","last_became_active":"2024-04-13T03:15:19.973828+0000","last_became_peered":"2024-04-13T03:15:19.973828+0000","last_unstale":"2024-04-13T03:15:21.247121+0000","last_undegraded":"2024-04-13T03:15:21.247121+0000","last_fullsized":"2024-04-13T03:15:21.247121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:23:51.416101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965881+0000","last_change":"2024-04-13T03:15:19.970897+0000","last_active":"2024-04-13T03:15:20.965881+0000","last_peered":"2024-04-13T03:15:20.965881+0000","last_clean":"2024-04-13T03:15:20.965881+0000","last_became_active":"2024-04-13T03:15:19.970640+0000","last_became_peered":"2024-04-13T03:15:19.970640+0000","last_unstale":"2024-04-13T03:15:20.965881+0000","last_undegraded":"2024-04-13T03:15:20.965881+0000","last_fullsized":"2024-04-13T03:15:20.965881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:03.433211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247541+0000","last_change":"2024-04-13T03:15:19.966154+0000","last_active":"2024-04-13T03:15:21.247541+0000","last_peered":"2024-04-13T03:15:21.247541+0000","last_clean":"2024-04-13T03:15:21.247541+0000","last_became_active":"2024-04-13T03:15:19.965884+0000","last_became_peered":"2024-04-13T03:15:19.965884+0000","last_unstale":"2024-04-13T03:15:21.247541+0000","last_undegraded":"2024-04-13T03:15:21.247541+0000","last_fullsized":"2024-04-13T03:15:21.247541+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:21.487590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246988+0000","last_change":"2024-04-13T03:15:19.973557+0000","last_active":"2024-04-13T03:15:21.246988+0000","last_peered":"2024-04-13T03:15:21.246988+0000","last_clean":"2024-04-13T03:15:21.246988+0000","last_became_active":"2024-04-13T03:15:19.972480+0000","last_became_peered":"2024-04-13T03:15:19.972480+0000","last_unstale":"2024-04-13T03:15:21.246988+0000","last_undegraded":"2024-04-13T03:15:21.246988+0000","last_fullsized":"2024-04-13T03:15:21.246988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:48:37.673448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966548+0000","last_change":"2024-04-13T03:15:19.975042+0000","last_active":"2024-04-13T03:15:20.966548+0000","last_peered":"2024-04-13T03:15:20.966548+0000","last_clean":"2024-04-13T03:15:20.966548+0000","last_became_active":"2024-04-13T03:15:19.974732+0000","last_became_peered":"2024-04-13T03:15:19.974732+0000","last_unstale":"2024-04-13T03:15:20.966548+0000","last_undegraded":"2024-04-13T03:15:20.966548+0000","last_fullsized":"2024-04-13T03:15:20.966548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:58:11.629615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245682+0000","last_change":"2024-04-13T03:15:19.967816+0000","last_active":"2024-04-13T03:15:21.245682+0000","last_peered":"2024-04-13T03:15:21.245682+0000","last_clean":"2024-04-13T03:15:21.245682+0000","last_became_active":"2024-04-13T03:15:19.967503+0000","last_became_peered":"2024-04-13T03:15:19.967503+0000","last_unstale":"2024-04-13T03:15:21.245682+0000","last_undegraded":"2024-04-13T03:15:21.245682+0000","last_fullsized":"2024-04-13T03:15:21.245682+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:50.645873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246486+0000","last_change":"2024-04-13T03:15:19.966356+0000","last_active":"2024-04-13T03:15:21.246486+0000","last_peered":"2024-04-13T03:15:21.246486+0000","last_clean":"2024-04-13T03:15:21.246486+0000","last_became_active":"2024-04-13T03:15:19.966081+0000","last_became_peered":"2024-04-13T03:15:19.966081+0000","last_unstale":"2024-04-13T03:15:21.246486+0000","last_undegraded":"2024-04-13T03:15:21.246486+0000","last_fullsized":"2024-04-13T03:15:21.246486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:50.929473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246714+0000","last_change":"2024-04-13T03:15:19.971998+0000","last_active":"2024-04-13T03:15:21.246714+0000","last_peered":"2024-04-13T03:15:21.246714+0000","last_clean":"2024-04-13T03:15:21.246714+0000","last_became_active":"2024-04-13T03:15:19.971721+0000","last_became_peered":"2024-04-13T03:15:19.971721+0000","last_unstale":"2024-04-13T03:15:21.246714+0000","last_undegraded":"2024-04-13T03:15:21.246714+0000","last_fullsized":"2024-04-13T03:15:21.246714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:50.929473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245562+0000","last_change":"2024-04-13T03:15:19.968494+0000","last_active":"2024-04-13T03:15:21.245562+0000","last_peered":"2024-04-13T03:15:21.245562+0000","last_clean":"2024-04-13T03:15:21.245562+0000","last_became_active":"2024-04-13T03:15:19.968242+0000","last_became_peered":"2024-04-13T03:15:19.968242+0000","last_unstale":"2024-04-13T03:15:21.245562+0000","last_undegraded":"2024-04-13T03:15:21.245562+0000","last_fullsized":"2024-04-13T03:15:21.245562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:40:45.329981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965941+0000","last_change":"2024-04-13T03:15:19.977127+0000","last_active":"2024-04-13T03:15:20.965941+0000","last_peered":"2024-04-13T03:15:20.965941+0000","last_clean":"2024-04-13T03:15:20.965941+0000","last_became_active":"2024-04-13T03:15:19.976856+0000","last_became_peered":"2024-04-13T03:15:19.976856+0000","last_unstale":"2024-04-13T03:15:20.965941+0000","last_undegraded":"2024-04-13T03:15:20.965941+0000","last_fullsized":"2024-04-13T03:15:20.965941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:36:10.037015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247089+0000","last_change":"2024-04-13T03:15:19.969207+0000","last_active":"2024-04-13T03:15:21.247089+0000","last_peered":"2024-04-13T03:15:21.247089+0000","last_clean":"2024-04-13T03:15:21.247089+0000","last_became_active":"2024-04-13T03:15:19.968177+0000","last_became_peered":"2024-04-13T03:15:19.968177+0000","last_unstale":"2024-04-13T03:15:21.247089+0000","last_undegraded":"2024-04-13T03:15:21.247089+0000","last_fullsized":"2024-04-13T03:15:21.247089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:02:55.255018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247240+0000","last_change":"2024-04-13T03:15:19.969647+0000","last_active":"2024-04-13T03:15:21.247240+0000","last_peered":"2024-04-13T03:15:21.247240+0000","last_clean":"2024-04-13T03:15:21.247240+0000","last_became_active":"2024-04-13T03:15:19.969379+0000","last_became_peered":"2024-04-13T03:15:19.969379+0000","last_unstale":"2024-04-13T03:15:21.247240+0000","last_undegraded":"2024-04-13T03:15:21.247240+0000","last_fullsized":"2024-04-13T03:15:21.247240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:10:18.534253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246159+0000","last_change":"2024-04-13T03:15:19.972364+0000","last_active":"2024-04-13T03:15:21.246159+0000","last_peered":"2024-04-13T03:15:21.246159+0000","last_clean":"2024-04-13T03:15:21.246159+0000","last_became_active":"2024-04-13T03:15:19.972090+0000","last_became_peered":"2024-04-13T03:15:19.972090+0000","last_unstale":"2024-04-13T03:15:21.246159+0000","last_undegraded":"2024-04-13T03:15:21.246159+0000","last_fullsized":"2024-04-13T03:15:21.246159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964603+0000","last_change":"2024-04-13T03:15:19.979558+0000","last_active":"2024-04-13T03:15:20.964603+0000","last_peered":"2024-04-13T03:15:20.964603+0000","last_clean":"2024-04-13T03:15:20.964603+0000","last_became_active":"2024-04-13T03:15:19.978735+0000","last_became_peered":"2024-04-13T03:15:19.978735+0000","last_unstale":"2024-04-13T03:15:20.964603+0000","last_undegraded":"2024-04-13T03:15:20.964603+0000","last_fullsized":"2024-04-13T03:15:20.964603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:53:10.421528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964899+0000","last_change":"2024-04-13T03:15:19.977378+0000","last_active":"2024-04-13T03:15:20.964899+0000","last_peered":"2024-04-13T03:15:20.964899+0000","last_clean":"2024-04-13T03:15:20.964899+0000","last_became_active":"2024-04-13T03:15:19.976433+0000","last_became_peered":"2024-04-13T03:15:19.976433+0000","last_unstale":"2024-04-13T03:15:20.964899+0000","last_undegraded":"2024-04-13T03:15:20.964899+0000","last_fullsized":"2024-04-13T03:15:20.964899+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:04:29.507885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.967155+0000","last_change":"2024-04-13T03:15:18.974373+0000","last_active":"2024-04-13T03:15:20.967155+0000","last_peered":"2024-04-13T03:15:20.967155+0000","last_clean":"2024-04-13T03:15:20.967155+0000","last_became_active":"2024-04-13T03:15:18.973461+0000","last_became_peered":"2024-04-13T03:15:18.973461+0000","last_unstale":"2024-04-13T03:15:20.967155+0000","last_undegraded":"2024-04-13T03:15:20.967155+0000","last_fullsized":"2024-04-13T03:15:20.967155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:12:04.167831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964947+0000","last_change":"2024-04-13T03:15:19.972289+0000","last_active":"2024-04-13T03:15:20.964947+0000","last_peered":"2024-04-13T03:15:20.964947+0000","last_clean":"2024-04-13T03:15:20.964947+0000","last_became_active":"2024-04-13T03:15:19.963471+0000","last_became_peered":"2024-04-13T03:15:19.963471+0000","last_unstale":"2024-04-13T03:15:20.964947+0000","last_undegraded":"2024-04-13T03:15:20.964947+0000","last_fullsized":"2024-04-13T03:15:20.964947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:01.136698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247038+0000","last_change":"2024-04-13T03:15:18.968924+0000","last_active":"2024-04-13T03:15:21.247038+0000","last_peered":"2024-04-13T03:15:21.247038+0000","last_clean":"2024-04-13T03:15:21.247038+0000","last_became_active":"2024-04-13T03:15:18.968653+0000","last_became_peered":"2024-04-13T03:15:18.968653+0000","last_unstale":"2024-04-13T03:15:21.247038+0000","last_undegraded":"2024-04-13T03:15:21.247038+0000","last_fullsized":"2024-04-13T03:15:21.247038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:34:22.836564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247301+0000","last_change":"2024-04-13T03:15:19.973001+0000","last_active":"2024-04-13T03:15:21.247301+0000","last_peered":"2024-04-13T03:15:21.247301+0000","last_clean":"2024-04-13T03:15:21.247301+0000","last_became_active":"2024-04-13T03:15:19.972725+0000","last_became_peered":"2024-04-13T03:15:19.972725+0000","last_unstale":"2024-04-13T03:15:21.247301+0000","last_undegraded":"2024-04-13T03:15:21.247301+0000","last_fullsized":"2024-04-13T03:15:21.247301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:30.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247099+0000","last_change":"2024-04-13T03:15:18.973196+0000","last_active":"2024-04-13T03:15:21.247099+0000","last_peered":"2024-04-13T03:15:21.247099+0000","last_clean":"2024-04-13T03:15:21.247099+0000","last_became_active":"2024-04-13T03:15:18.972930+0000","last_became_peered":"2024-04-13T03:15:18.972930+0000","last_unstale":"2024-04-13T03:15:21.247099+0000","last_undegraded":"2024-04-13T03:15:21.247099+0000","last_fullsized":"2024-04-13T03:15:21.247099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:49.925520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965073+0000","last_change":"2024-04-13T03:15:19.972396+0000","last_active":"2024-04-13T03:15:20.965073+0000","last_peered":"2024-04-13T03:15:20.965073+0000","last_clean":"2024-04-13T03:15:20.965073+0000","last_became_active":"2024-04-13T03:15:19.963504+0000","last_became_peered":"2024-04-13T03:15:19.963504+0000","last_unstale":"2024-04-13T03:15:20.965073+0000","last_undegraded":"2024-04-13T03:15:20.965073+0000","last_fullsized":"2024-04-13T03:15:20.965073+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:00.727075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966982+0000","last_change":"2024-04-13T03:15:18.973253+0000","last_active":"2024-04-13T03:15:20.966982+0000","last_peered":"2024-04-13T03:15:20.966982+0000","last_clean":"2024-04-13T03:15:20.966982+0000","last_became_active":"2024-04-13T03:15:18.973016+0000","last_became_peered":"2024-04-13T03:15:18.973016+0000","last_unstale":"2024-04-13T03:15:20.966982+0000","last_undegraded":"2024-04-13T03:15:20.966982+0000","last_fullsized":"2024-04-13T03:15:20.966982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965928+0000","last_change":"2024-04-13T03:15:19.970488+0000","last_active":"2024-04-13T03:15:20.965928+0000","last_peered":"2024-04-13T03:15:20.965928+0000","last_clean":"2024-04-13T03:15:20.965928+0000","last_became_active":"2024-04-13T03:15:19.970218+0000","last_became_peered":"2024-04-13T03:15:19.970218+0000","last_unstale":"2024-04-13T03:15:20.965928+0000","last_undegraded":"2024-04-13T03:15:20.965928+0000","last_fullsized":"2024-04-13T03:15:20.965928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:41:48.557267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247624+0000","last_change":"2024-04-13T03:15:18.960938+0000","last_active":"2024-04-13T03:15:21.247624+0000","last_peered":"2024-04-13T03:15:21.247624+0000","last_clean":"2024-04-13T03:15:21.247624+0000","last_became_active":"2024-04-13T03:15:18.960534+0000","last_became_peered":"2024-04-13T03:15:18.960534+0000","last_unstale":"2024-04-13T03:15:21.247624+0000","last_undegraded":"2024-04-13T03:15:21.247624+0000","last_fullsized":"2024-04-13T03:15:21.247624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:37:30.553956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966905+0000","last_change":"2024-04-13T03:15:19.964778+0000","last_active":"2024-04-13T03:15:20.966905+0000","last_peered":"2024-04-13T03:15:20.966905+0000","last_clean":"2024-04-13T03:15:20.966905+0000","last_became_active":"2024-04-13T03:15:19.964503+0000","last_became_peered":"2024-04-13T03:15:19.964503+0000","last_unstale":"2024-04-13T03:15:20.966905+0000","last_undegraded":"2024-04-13T03:15:20.966905+0000","last_fullsized":"2024-04-13T03:15:20.966905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:47.736587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966086+0000","last_change":"2024-04-13T03:15:18.967368+0000","last_active":"2024-04-13T03:15:20.966086+0000","last_peered":"2024-04-13T03:15:20.966086+0000","last_clean":"2024-04-13T03:15:20.966086+0000","last_became_active":"2024-04-13T03:15:18.967051+0000","last_became_peered":"2024-04-13T03:15:18.967051+0000","last_unstale":"2024-04-13T03:15:20.966086+0000","last_undegraded":"2024-04-13T03:15:20.966086+0000","last_fullsized":"2024-04-13T03:15:20.966086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964745+0000","last_change":"2024-04-13T03:15:19.975957+0000","last_active":"2024-04-13T03:15:20.964745+0000","last_peered":"2024-04-13T03:15:20.964745+0000","last_clean":"2024-04-13T03:15:20.964745+0000","last_became_active":"2024-04-13T03:15:19.975053+0000","last_became_peered":"2024-04-13T03:15:19.975053+0000","last_unstale":"2024-04-13T03:15:20.964745+0000","last_undegraded":"2024-04-13T03:15:20.964745+0000","last_fullsized":"2024-04-13T03:15:20.964745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:51:42.407125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245642+0000","last_change":"2024-04-13T03:15:18.974497+0000","last_active":"2024-04-13T03:15:21.245642+0000","last_peered":"2024-04-13T03:15:21.245642+0000","last_clean":"2024-04-13T03:15:21.245642+0000","last_became_active":"2024-04-13T03:15:18.974188+0000","last_became_peered":"2024-04-13T03:15:18.974188+0000","last_unstale":"2024-04-13T03:15:21.245642+0000","last_undegraded":"2024-04-13T03:15:21.245642+0000","last_fullsized":"2024-04-13T03:15:21.245642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:20.483637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965963+0000","last_change":"2024-04-13T03:15:19.975155+0000","last_active":"2024-04-13T03:15:20.965963+0000","last_peered":"2024-04-13T03:15:20.965963+0000","last_clean":"2024-04-13T03:15:20.965963+0000","last_became_active":"2024-04-13T03:15:19.974234+0000","last_became_peered":"2024-04-13T03:15:19.974234+0000","last_unstale":"2024-04-13T03:15:20.965963+0000","last_undegraded":"2024-04-13T03:15:20.965963+0000","last_fullsized":"2024-04-13T03:15:20.965963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246337+0000","last_change":"2024-04-13T03:15:18.974213+0000","last_active":"2024-04-13T03:15:21.246337+0000","last_peered":"2024-04-13T03:15:21.246337+0000","last_clean":"2024-04-13T03:15:21.246337+0000","last_became_active":"2024-04-13T03:15:18.972521+0000","last_became_peered":"2024-04-13T03:15:18.972521+0000","last_unstale":"2024-04-13T03:15:21.246337+0000","last_undegraded":"2024-04-13T03:15:21.246337+0000","last_fullsized":"2024-04-13T03:15:21.246337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:42.526775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246655+0000","last_change":"2024-04-13T03:15:19.969956+0000","last_active":"2024-04-13T03:15:21.246655+0000","last_peered":"2024-04-13T03:15:21.246655+0000","last_clean":"2024-04-13T03:15:21.246655+0000","last_became_active":"2024-04-13T03:15:19.969647+0000","last_became_peered":"2024-04-13T03:15:19.969647+0000","last_unstale":"2024-04-13T03:15:21.246655+0000","last_undegraded":"2024-04-13T03:15:21.246655+0000","last_fullsized":"2024-04-13T03:15:21.246655+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:12.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246663+0000","last_change":"2024-04-13T03:15:18.973458+0000","last_active":"2024-04-13T03:15:21.246663+0000","last_peered":"2024-04-13T03:15:21.246663+0000","last_clean":"2024-04-13T03:15:21.246663+0000","last_became_active":"2024-04-13T03:15:18.973183+0000","last_became_peered":"2024-04-13T03:15:18.973183+0000","last_unstale":"2024-04-13T03:15:21.246663+0000","last_undegraded":"2024-04-13T03:15:21.246663+0000","last_fullsized":"2024-04-13T03:15:21.246663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:58:59.324270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246943+0000","last_change":"2024-04-13T03:15:19.972097+0000","last_active":"2024-04-13T03:15:21.246943+0000","last_peered":"2024-04-13T03:15:21.246943+0000","last_clean":"2024-04-13T03:15:21.246943+0000","last_became_active":"2024-04-13T03:15:19.971816+0000","last_became_peered":"2024-04-13T03:15:19.971816+0000","last_unstale":"2024-04-13T03:15:21.246943+0000","last_undegraded":"2024-04-13T03:15:21.246943+0000","last_fullsized":"2024-04-13T03:15:21.246943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:26.384212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245885+0000","last_change":"2024-04-13T03:15:18.973131+0000","last_active":"2024-04-13T03:15:21.245885+0000","last_peered":"2024-04-13T03:15:21.245885+0000","last_clean":"2024-04-13T03:15:21.245885+0000","last_became_active":"2024-04-13T03:15:18.972829+0000","last_became_peered":"2024-04-13T03:15:18.972829+0000","last_unstale":"2024-04-13T03:15:21.245885+0000","last_undegraded":"2024-04-13T03:15:21.245885+0000","last_fullsized":"2024-04-13T03:15:21.245885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:06:08.023157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245799+0000","last_change":"2024-04-13T03:15:19.963852+0000","last_active":"2024-04-13T03:15:21.245799+0000","last_peered":"2024-04-13T03:15:21.245799+0000","last_clean":"2024-04-13T03:15:21.245799+0000","last_became_active":"2024-04-13T03:15:19.963593+0000","last_became_peered":"2024-04-13T03:15:19.963593+0000","last_unstale":"2024-04-13T03:15:21.245799+0000","last_undegraded":"2024-04-13T03:15:21.245799+0000","last_fullsized":"2024-04-13T03:15:21.245799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:40:49.104144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247341+0000","last_change":"2024-04-13T03:15:18.964113+0000","last_active":"2024-04-13T03:15:21.247341+0000","last_peered":"2024-04-13T03:15:21.247341+0000","last_clean":"2024-04-13T03:15:21.247341+0000","last_became_active":"2024-04-13T03:15:18.963567+0000","last_became_peered":"2024-04-13T03:15:18.963567+0000","last_unstale":"2024-04-13T03:15:21.247341+0000","last_undegraded":"2024-04-13T03:15:21.247341+0000","last_fullsized":"2024-04-13T03:15:21.247341+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966086+0000","last_change":"2024-04-13T03:15:19.969514+0000","last_active":"2024-04-13T03:15:20.966086+0000","last_peered":"2024-04-13T03:15:20.966086+0000","last_clean":"2024-04-13T03:15:20.966086+0000","last_became_active":"2024-04-13T03:15:19.969251+0000","last_became_peered":"2024-04-13T03:15:19.969251+0000","last_unstale":"2024-04-13T03:15:20.966086+0000","last_undegraded":"2024-04-13T03:15:20.966086+0000","last_fullsized":"2024-04-13T03:15:20.966086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965012+0000","last_change":"2024-04-13T03:15:18.974486+0000","last_active":"2024-04-13T03:15:20.965012+0000","last_peered":"2024-04-13T03:15:20.965012+0000","last_clean":"2024-04-13T03:15:20.965012+0000","last_became_active":"2024-04-13T03:15:18.974178+0000","last_became_peered":"2024-04-13T03:15:18.974178+0000","last_unstale":"2024-04-13T03:15:20.965012+0000","last_undegraded":"2024-04-13T03:15:20.965012+0000","last_fullsized":"2024-04-13T03:15:20.965012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:48.782998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965720+0000","last_change":"2024-04-13T03:15:19.967462+0000","last_active":"2024-04-13T03:15:20.965720+0000","last_peered":"2024-04-13T03:15:20.965720+0000","last_clean":"2024-04-13T03:15:20.965720+0000","last_became_active":"2024-04-13T03:15:19.967050+0000","last_became_peered":"2024-04-13T03:15:19.967050+0000","last_unstale":"2024-04-13T03:15:20.965720+0000","last_undegraded":"2024-04-13T03:15:20.965720+0000","last_fullsized":"2024-04-13T03:15:20.965720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:07.224694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247136+0000","last_change":"2024-04-13T03:15:18.973728+0000","last_active":"2024-04-13T03:15:21.247136+0000","last_peered":"2024-04-13T03:15:21.247136+0000","last_clean":"2024-04-13T03:15:21.247136+0000","last_became_active":"2024-04-13T03:15:18.973479+0000","last_became_peered":"2024-04-13T03:15:18.973479+0000","last_unstale":"2024-04-13T03:15:21.247136+0000","last_undegraded":"2024-04-13T03:15:21.247136+0000","last_fullsized":"2024-04-13T03:15:21.247136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:33.321965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246472+0000","last_change":"2024-04-13T03:15:19.964411+0000","last_active":"2024-04-13T03:15:21.246472+0000","last_peered":"2024-04-13T03:15:21.246472+0000","last_clean":"2024-04-13T03:15:21.246472+0000","last_became_active":"2024-04-13T03:15:19.964130+0000","last_became_peered":"2024-04-13T03:15:19.964130+0000","last_unstale":"2024-04-13T03:15:21.246472+0000","last_undegraded":"2024-04-13T03:15:21.246472+0000","last_fullsized":"2024-04-13T03:15:21.246472+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964824+0000","last_change":"2024-04-13T03:15:18.968822+0000","last_active":"2024-04-13T03:15:20.964824+0000","last_peered":"2024-04-13T03:15:20.964824+0000","last_clean":"2024-04-13T03:15:20.964824+0000","last_became_active":"2024-04-13T03:15:18.968528+0000","last_became_peered":"2024-04-13T03:15:18.968528+0000","last_unstale":"2024-04-13T03:15:20.964824+0000","last_undegraded":"2024-04-13T03:15:20.964824+0000","last_fullsized":"2024-04-13T03:15:20.964824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:25.380259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246124+0000","last_change":"2024-04-13T03:15:19.975015+0000","last_active":"2024-04-13T03:15:21.246124+0000","last_peered":"2024-04-13T03:15:21.246124+0000","last_clean":"2024-04-13T03:15:21.246124+0000","last_became_active":"2024-04-13T03:15:19.974732+0000","last_became_peered":"2024-04-13T03:15:19.974732+0000","last_unstale":"2024-04-13T03:15:21.246124+0000","last_undegraded":"2024-04-13T03:15:21.246124+0000","last_fullsized":"2024-04-13T03:15:21.246124+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:18.467921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965657+0000","last_change":"2024-04-13T03:15:18.974120+0000","last_active":"2024-04-13T03:15:20.965657+0000","last_peered":"2024-04-13T03:15:20.965657+0000","last_clean":"2024-04-13T03:15:20.965657+0000","last_became_active":"2024-04-13T03:15:18.973879+0000","last_became_peered":"2024-04-13T03:15:18.973879+0000","last_unstale":"2024-04-13T03:15:20.965657+0000","last_undegraded":"2024-04-13T03:15:20.965657+0000","last_fullsized":"2024-04-13T03:15:20.965657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:48:17.505348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245918+0000","last_change":"2024-04-13T03:15:19.966899+0000","last_active":"2024-04-13T03:15:21.245918+0000","last_peered":"2024-04-13T03:15:21.245918+0000","last_clean":"2024-04-13T03:15:21.245918+0000","last_became_active":"2024-04-13T03:15:19.966597+0000","last_became_peered":"2024-04-13T03:15:19.966597+0000","last_unstale":"2024-04-13T03:15:21.245918+0000","last_undegraded":"2024-04-13T03:15:21.245918+0000","last_fullsized":"2024-04-13T03:15:21.245918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:00:46.165705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965809+0000","last_change":"2024-04-13T03:15:18.960533+0000","last_active":"2024-04-13T03:15:20.965809+0000","last_peered":"2024-04-13T03:15:20.965809+0000","last_clean":"2024-04-13T03:15:20.965809+0000","last_became_active":"2024-04-13T03:15:18.960250+0000","last_became_peered":"2024-04-13T03:15:18.960250+0000","last_unstale":"2024-04-13T03:15:20.965809+0000","last_undegraded":"2024-04-13T03:15:20.965809+0000","last_fullsized":"2024-04-13T03:15:20.965809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:36:58.800747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965148+0000","last_change":"2024-04-13T03:15:19.975603+0000","last_active":"2024-04-13T03:15:20.965148+0000","last_peered":"2024-04-13T03:15:20.965148+0000","last_clean":"2024-04-13T03:15:20.965148+0000","last_became_active":"2024-04-13T03:15:19.975380+0000","last_became_peered":"2024-04-13T03:15:19.975380+0000","last_unstale":"2024-04-13T03:15:20.965148+0000","last_undegraded":"2024-04-13T03:15:20.965148+0000","last_fullsized":"2024-04-13T03:15:20.965148+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:42.826594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.967182+0000","last_change":"2024-04-13T03:15:18.975085+0000","last_active":"2024-04-13T03:15:20.967182+0000","last_peered":"2024-04-13T03:15:20.967182+0000","last_clean":"2024-04-13T03:15:20.967182+0000","last_became_active":"2024-04-13T03:15:18.974827+0000","last_became_peered":"2024-04-13T03:15:18.974827+0000","last_unstale":"2024-04-13T03:15:20.967182+0000","last_undegraded":"2024-04-13T03:15:20.967182+0000","last_fullsized":"2024-04-13T03:15:20.967182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:06.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966978+0000","last_change":"2024-04-13T03:15:19.970168+0000","last_active":"2024-04-13T03:15:20.966978+0000","last_peered":"2024-04-13T03:15:20.966978+0000","last_clean":"2024-04-13T03:15:20.966978+0000","last_became_active":"2024-04-13T03:15:19.969904+0000","last_became_peered":"2024-04-13T03:15:19.969904+0000","last_unstale":"2024-04-13T03:15:20.966978+0000","last_undegraded":"2024-04-13T03:15:20.966978+0000","last_fullsized":"2024-04-13T03:15:20.966978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:52.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247413+0000","last_change":"2024-04-13T03:15:18.974450+0000","last_active":"2024-04-13T03:15:21.247413+0000","last_peered":"2024-04-13T03:15:21.247413+0000","last_clean":"2024-04-13T03:15:21.247413+0000","last_became_active":"2024-04-13T03:15:18.974157+0000","last_became_peered":"2024-04-13T03:15:18.974157+0000","last_unstale":"2024-04-13T03:15:21.247413+0000","last_undegraded":"2024-04-13T03:15:21.247413+0000","last_fullsized":"2024-04-13T03:15:21.247413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246862+0000","last_change":"2024-04-13T03:15:19.968337+0000","last_active":"2024-04-13T03:15:21.246862+0000","last_peered":"2024-04-13T03:15:21.246862+0000","last_clean":"2024-04-13T03:15:21.246862+0000","last_became_active":"2024-04-13T03:15:19.968010+0000","last_became_peered":"2024-04-13T03:15:19.968010+0000","last_unstale":"2024-04-13T03:15:21.246862+0000","last_undegraded":"2024-04-13T03:15:21.246862+0000","last_fullsized":"2024-04-13T03:15:21.246862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:34.325918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247355+0000","last_change":"2024-04-13T03:15:18.964360+0000","last_active":"2024-04-13T03:15:21.247355+0000","last_peered":"2024-04-13T03:15:21.247355+0000","last_clean":"2024-04-13T03:15:21.247355+0000","last_became_active":"2024-04-13T03:15:18.963856+0000","last_became_peered":"2024-04-13T03:15:18.963856+0000","last_unstale":"2024-04-13T03:15:21.247355+0000","last_undegraded":"2024-04-13T03:15:21.247355+0000","last_fullsized":"2024-04-13T03:15:21.247355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:12:04.167831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964684+0000","last_change":"2024-04-13T03:15:19.974674+0000","last_active":"2024-04-13T03:15:20.964684+0000","last_peered":"2024-04-13T03:15:20.964684+0000","last_clean":"2024-04-13T03:15:20.964684+0000","last_became_active":"2024-04-13T03:15:19.974428+0000","last_became_peered":"2024-04-13T03:15:19.974428+0000","last_unstale":"2024-04-13T03:15:20.964684+0000","last_undegraded":"2024-04-13T03:15:20.964684+0000","last_fullsized":"2024-04-13T03:15:20.964684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:22:14.125862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246240+0000","last_change":"2024-04-13T03:15:18.966755+0000","last_active":"2024-04-13T03:15:21.246240+0000","last_peered":"2024-04-13T03:15:21.246240+0000","last_clean":"2024-04-13T03:15:21.246240+0000","last_became_active":"2024-04-13T03:15:18.966488+0000","last_became_peered":"2024-04-13T03:15:18.966488+0000","last_unstale":"2024-04-13T03:15:21.246240+0000","last_undegraded":"2024-04-13T03:15:21.246240+0000","last_fullsized":"2024-04-13T03:15:21.246240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:33.302316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246235+0000","last_change":"2024-04-13T03:15:19.966246+0000","last_active":"2024-04-13T03:15:21.246235+0000","last_peered":"2024-04-13T03:15:21.246235+0000","last_clean":"2024-04-13T03:15:21.246235+0000","last_became_active":"2024-04-13T03:15:19.965970+0000","last_became_peered":"2024-04-13T03:15:19.965970+0000","last_unstale":"2024-04-13T03:15:21.246235+0000","last_undegraded":"2024-04-13T03:15:21.246235+0000","last_fullsized":"2024-04-13T03:15:21.246235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246100+0000","last_change":"2024-04-13T03:15:18.970400+0000","last_active":"2024-04-13T03:15:21.246100+0000","last_peered":"2024-04-13T03:15:21.246100+0000","last_clean":"2024-04-13T03:15:21.246100+0000","last_became_active":"2024-04-13T03:15:18.970152+0000","last_became_peered":"2024-04-13T03:15:18.970152+0000","last_unstale":"2024-04-13T03:15:21.246100+0000","last_undegraded":"2024-04-13T03:15:21.246100+0000","last_fullsized":"2024-04-13T03:15:21.246100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:33:28.051506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247145+0000","last_change":"2024-04-13T03:15:19.971403+0000","last_active":"2024-04-13T03:15:21.247145+0000","last_peered":"2024-04-13T03:15:21.247145+0000","last_clean":"2024-04-13T03:15:21.247145+0000","last_became_active":"2024-04-13T03:15:19.971119+0000","last_became_peered":"2024-04-13T03:15:19.971119+0000","last_unstale":"2024-04-13T03:15:21.247145+0000","last_undegraded":"2024-04-13T03:15:21.247145+0000","last_fullsized":"2024-04-13T03:15:21.247145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:34.325918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246552+0000","last_change":"2024-04-13T03:15:18.966351+0000","last_active":"2024-04-13T03:15:21.246552+0000","last_peered":"2024-04-13T03:15:21.246552+0000","last_clean":"2024-04-13T03:15:21.246552+0000","last_became_active":"2024-04-13T03:15:18.966048+0000","last_became_peered":"2024-04-13T03:15:18.966048+0000","last_unstale":"2024-04-13T03:15:21.246552+0000","last_undegraded":"2024-04-13T03:15:21.246552+0000","last_fullsized":"2024-04-13T03:15:21.246552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:58:10.625662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964831+0000","last_change":"2024-04-13T03:15:19.966143+0000","last_active":"2024-04-13T03:15:20.964831+0000","last_peered":"2024-04-13T03:15:20.964831+0000","last_clean":"2024-04-13T03:15:20.964831+0000","last_became_active":"2024-04-13T03:15:19.963355+0000","last_became_peered":"2024-04-13T03:15:19.963355+0000","last_unstale":"2024-04-13T03:15:20.964831+0000","last_undegraded":"2024-04-13T03:15:20.964831+0000","last_fullsized":"2024-04-13T03:15:20.964831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:26:37.985001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245309+0000","last_change":"2024-04-13T03:15:18.966450+0000","last_active":"2024-04-13T03:15:21.245309+0000","last_peered":"2024-04-13T03:15:21.245309+0000","last_clean":"2024-04-13T03:15:21.245309+0000","last_became_active":"2024-04-13T03:15:18.966150+0000","last_became_peered":"2024-04-13T03:15:18.966150+0000","last_unstale":"2024-04-13T03:15:21.245309+0000","last_undegraded":"2024-04-13T03:15:21.245309+0000","last_fullsized":"2024-04-13T03:15:21.245309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:42:35.734922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966644+0000","last_change":"2024-04-13T03:15:19.964081+0000","last_active":"2024-04-13T03:15:20.966644+0000","last_peered":"2024-04-13T03:15:20.966644+0000","last_clean":"2024-04-13T03:15:20.966644+0000","last_became_active":"2024-04-13T03:15:19.963798+0000","last_became_peered":"2024-04-13T03:15:19.963798+0000","last_unstale":"2024-04-13T03:15:20.966644+0000","last_undegraded":"2024-04-13T03:15:20.966644+0000","last_fullsized":"2024-04-13T03:15:20.966644+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:34.306269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247537+0000","last_change":"2024-04-13T03:15:18.975811+0000","last_active":"2024-04-13T03:15:21.247537+0000","last_peered":"2024-04-13T03:15:21.247537+0000","last_clean":"2024-04-13T03:15:21.247537+0000","last_became_active":"2024-04-13T03:15:18.975442+0000","last_became_peered":"2024-04-13T03:15:18.975442+0000","last_unstale":"2024-04-13T03:15:21.247537+0000","last_undegraded":"2024-04-13T03:15:21.247537+0000","last_fullsized":"2024-04-13T03:15:21.247537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:48:36.669495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965143+0000","last_change":"2024-04-13T03:15:19.979056+0000","last_active":"2024-04-13T03:15:20.965143+0000","last_peered":"2024-04-13T03:15:20.965143+0000","last_clean":"2024-04-13T03:15:20.965143+0000","last_became_active":"2024-04-13T03:15:19.978768+0000","last_became_peered":"2024-04-13T03:15:19.978768+0000","last_unstale":"2024-04-13T03:15:20.965143+0000","last_undegraded":"2024-04-13T03:15:20.965143+0000","last_fullsized":"2024-04-13T03:15:20.965143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:10.108929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247159+0000","last_change":"2024-04-13T03:15:18.974236+0000","last_active":"2024-04-13T03:15:21.247159+0000","last_peered":"2024-04-13T03:15:21.247159+0000","last_clean":"2024-04-13T03:15:21.247159+0000","last_became_active":"2024-04-13T03:15:18.973950+0000","last_became_peered":"2024-04-13T03:15:18.973950+0000","last_unstale":"2024-04-13T03:15:21.247159+0000","last_undegraded":"2024-04-13T03:15:21.247159+0000","last_fullsized":"2024-04-13T03:15:21.247159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:06.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965093+0000","last_change":"2024-04-13T03:15:19.965920+0000","last_active":"2024-04-13T03:15:20.965093+0000","last_peered":"2024-04-13T03:15:20.965093+0000","last_clean":"2024-04-13T03:15:20.965093+0000","last_became_active":"2024-04-13T03:15:19.965475+0000","last_became_peered":"2024-04-13T03:15:19.965475+0000","last_unstale":"2024-04-13T03:15:20.965093+0000","last_undegraded":"2024-04-13T03:15:20.965093+0000","last_fullsized":"2024-04-13T03:15:20.965093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:53:13.000921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247446+0000","last_change":"2024-04-13T03:15:18.963844+0000","last_active":"2024-04-13T03:15:21.247446+0000","last_peered":"2024-04-13T03:15:21.247446+0000","last_clean":"2024-04-13T03:15:21.247446+0000","last_became_active":"2024-04-13T03:15:18.963512+0000","last_became_peered":"2024-04-13T03:15:18.963512+0000","last_unstale":"2024-04-13T03:15:21.247446+0000","last_undegraded":"2024-04-13T03:15:21.247446+0000","last_fullsized":"2024-04-13T03:15:21.247446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:03:21.856679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246705+0000","last_change":"2024-04-13T03:15:19.966815+0000","last_active":"2024-04-13T03:15:21.246705+0000","last_peered":"2024-04-13T03:15:21.246705+0000","last_clean":"2024-04-13T03:15:21.246705+0000","last_became_active":"2024-04-13T03:15:19.966553+0000","last_became_peered":"2024-04-13T03:15:19.966553+0000","last_unstale":"2024-04-13T03:15:21.246705+0000","last_undegraded":"2024-04-13T03:15:21.246705+0000","last_fullsized":"2024-04-13T03:15:21.246705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:42.417633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966988+0000","last_change":"2024-04-13T03:15:18.968687+0000","last_active":"2024-04-13T03:15:20.966988+0000","last_peered":"2024-04-13T03:15:20.966988+0000","last_clean":"2024-04-13T03:15:20.966988+0000","last_became_active":"2024-04-13T03:15:18.968379+0000","last_became_peered":"2024-04-13T03:15:18.968379+0000","last_unstale":"2024-04-13T03:15:20.966988+0000","last_undegraded":"2024-04-13T03:15:20.966988+0000","last_fullsized":"2024-04-13T03:15:20.966988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:34:22.836564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245303+0000","last_change":"2024-04-13T03:15:19.975504+0000","last_active":"2024-04-13T03:15:21.245303+0000","last_peered":"2024-04-13T03:15:21.245303+0000","last_clean":"2024-04-13T03:15:21.245303+0000","last_became_active":"2024-04-13T03:15:19.975260+0000","last_became_peered":"2024-04-13T03:15:19.975260+0000","last_unstale":"2024-04-13T03:15:21.245303+0000","last_undegraded":"2024-04-13T03:15:21.245303+0000","last_fullsized":"2024-04-13T03:15:21.245303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:39:05.606358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965154+0000","last_change":"2024-04-13T03:15:18.968767+0000","last_active":"2024-04-13T03:15:20.965154+0000","last_peered":"2024-04-13T03:15:20.965154+0000","last_clean":"2024-04-13T03:15:20.965154+0000","last_became_active":"2024-04-13T03:15:18.968495+0000","last_became_peered":"2024-04-13T03:15:18.968495+0000","last_unstale":"2024-04-13T03:15:20.965154+0000","last_undegraded":"2024-04-13T03:15:20.965154+0000","last_fullsized":"2024-04-13T03:15:20.965154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:02:54.251065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245596+0000","last_change":"2024-04-13T03:15:19.971844+0000","last_active":"2024-04-13T03:15:21.245596+0000","last_peered":"2024-04-13T03:15:21.245596+0000","last_clean":"2024-04-13T03:15:21.245596+0000","last_became_active":"2024-04-13T03:15:19.971592+0000","last_became_peered":"2024-04-13T03:15:19.971592+0000","last_unstale":"2024-04-13T03:15:21.245596+0000","last_undegraded":"2024-04-13T03:15:21.245596+0000","last_fullsized":"2024-04-13T03:15:21.245596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:05.752317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246885+0000","last_change":"2024-04-13T03:15:18.965952+0000","last_active":"2024-04-13T03:15:21.246885+0000","last_peered":"2024-04-13T03:15:21.246885+0000","last_clean":"2024-04-13T03:15:21.246885+0000","last_became_active":"2024-04-13T03:15:18.965639+0000","last_became_peered":"2024-04-13T03:15:18.965639+0000","last_unstale":"2024-04-13T03:15:21.246885+0000","last_undegraded":"2024-04-13T03:15:21.246885+0000","last_fullsized":"2024-04-13T03:15:21.246885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:00.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966754+0000","last_change":"2024-04-13T03:15:19.975072+0000","last_active":"2024-04-13T03:15:20.966754+0000","last_peered":"2024-04-13T03:15:20.966754+0000","last_clean":"2024-04-13T03:15:20.966754+0000","last_became_active":"2024-04-13T03:15:19.974781+0000","last_became_peered":"2024-04-13T03:15:19.974781+0000","last_unstale":"2024-04-13T03:15:20.966754+0000","last_undegraded":"2024-04-13T03:15:20.966754+0000","last_fullsized":"2024-04-13T03:15:20.966754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:59:00.328223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966823+0000","last_change":"2024-04-13T03:15:18.973260+0000","last_active":"2024-04-13T03:15:20.966823+0000","last_peered":"2024-04-13T03:15:20.966823+0000","last_clean":"2024-04-13T03:15:20.966823+0000","last_became_active":"2024-04-13T03:15:18.972942+0000","last_became_peered":"2024-04-13T03:15:18.972942+0000","last_unstale":"2024-04-13T03:15:20.966823+0000","last_undegraded":"2024-04-13T03:15:20.966823+0000","last_fullsized":"2024-04-13T03:15:20.966823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:48:17.505348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964972+0000","last_change":"2024-04-13T03:15:19.963057+0000","last_active":"2024-04-13T03:15:20.964972+0000","last_peered":"2024-04-13T03:15:20.964972+0000","last_clean":"2024-04-13T03:15:20.964972+0000","last_became_active":"2024-04-13T03:15:19.961927+0000","last_became_peered":"2024-04-13T03:15:19.961927+0000","last_unstale":"2024-04-13T03:15:20.964972+0000","last_undegraded":"2024-04-13T03:15:20.964972+0000","last_fullsized":"2024-04-13T03:15:20.964972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:10.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246430+0000","last_change":"2024-04-13T03:15:18.973612+0000","last_active":"2024-04-13T03:15:21.246430+0000","last_peered":"2024-04-13T03:15:21.246430+0000","last_clean":"2024-04-13T03:15:21.246430+0000","last_became_active":"2024-04-13T03:15:18.973369+0000","last_became_peered":"2024-04-13T03:15:18.973369+0000","last_unstale":"2024-04-13T03:15:21.246430+0000","last_undegraded":"2024-04-13T03:15:21.246430+0000","last_fullsized":"2024-04-13T03:15:21.246430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:11.727025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247499+0000","last_change":"2024-04-13T03:15:19.970984+0000","last_active":"2024-04-13T03:15:21.247499+0000","last_peered":"2024-04-13T03:15:21.247499+0000","last_clean":"2024-04-13T03:15:21.247499+0000","last_became_active":"2024-04-13T03:15:19.970730+0000","last_became_peered":"2024-04-13T03:15:19.970730+0000","last_unstale":"2024-04-13T03:15:21.247499+0000","last_undegraded":"2024-04-13T03:15:21.247499+0000","last_fullsized":"2024-04-13T03:15:21.247499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:25:07.869053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965970+0000","last_change":"2024-04-13T03:15:18.973820+0000","last_active":"2024-04-13T03:15:20.965970+0000","last_peered":"2024-04-13T03:15:20.965970+0000","last_clean":"2024-04-13T03:15:20.965970+0000","last_became_active":"2024-04-13T03:15:18.973569+0000","last_became_peered":"2024-04-13T03:15:18.973569+0000","last_unstale":"2024-04-13T03:15:20.965970+0000","last_undegraded":"2024-04-13T03:15:20.965970+0000","last_fullsized":"2024-04-13T03:15:20.965970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:48.011391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246485+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:15:21.246485+0000","last_peered":"2024-04-13T03:15:21.246485+0000","last_clean":"2024-04-13T03:15:21.246485+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:15:21.246485+0000","last_undegraded":"2024-04-13T03:15:21.246485+0000","last_fullsized":"2024-04-13T03:15:21.246485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1107459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":519251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":517195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":517105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":1109721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:15:22.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:15:23.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.112+0000 7f28b20b7640 1 Processor -- start 2024-04-13T03:15:23.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.112+0000 7f28b20b7640 1 -- start start 2024-04-13T03:15:23.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28b20b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28ac06f160 0x7f28ac166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28b20b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ac0998d0 con 0x7f28ac0cdf50 2024-04-13T03:15:23.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28b20b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ac073020 con 0x7f28ac06cb10 2024-04-13T03:15:23.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28b20b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f28ac073300 con 0x7f28ac06f160 2024-04-13T03:15:23.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28abfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28ac06f160 0x7f28ac166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28ab7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f28ac0cdf50 0x7f28ac0ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47126/0 (socket says 172.21.15.106:47126) 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28ab7fe640 1 -- 172.21.15.106:0/276376495 learned_addr learned my addr 172.21.15.106:0/276376495 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28abfff640 1 -- 172.21.15.106:0/276376495 >> v1:172.21.15.114:6789/0 conn(0x7f28ac06cb10 legacy=0x7f28ac072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28abfff640 1 -- 172.21.15.106:0/276376495 >> v1:172.21.15.106:6789/0 conn(0x7f28ac0cdf50 legacy=0x7f28ac0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28abfff640 1 -- 172.21.15.106:0/276376495 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0735e0 con 0x7f28ac06f160 2024-04-13T03:15:23.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.113+0000 7f28aa7fc640 1 -- 172.21.15.106:0/276376495 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249123439 0 0) 0x7f28ac0998d0 con 0x7f28ac0cdf50 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.114+0000 7f28abfff640 1 --2- 172.21.15.106:0/276376495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac06f160 0x7f28ac166170 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009d90 tx=0x7f28a402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3565b4c37ae70515 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.114+0000 7f28aa7fc640 1 -- 172.21.15.106:0/276376495 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4039070 con 0x7f28ac06f160 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.114+0000 7f28aa7fc640 1 -- 172.21.15.106:0/276376495 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a402cd00 con 0x7f28ac06f160 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.114+0000 7f28aa7fc640 1 -- 172.21.15.106:0/276376495 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4034660 con 0x7f28ac06f160 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 -- 172.21.15.106:0/276376495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac06f160 msgr2=0x7f28ac166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 --2- 172.21.15.106:0/276376495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac06f160 0x7f28ac166170 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009d90 tx=0x7f28a402bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 -- 172.21.15.106:0/276376495 shutdown_connections 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 --2- 172.21.15.106:0/276376495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac06f160 0x7f28ac166170 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 -- 172.21.15.106:0/276376495 >> 172.21.15.106:0/276376495 conn(0x7f28ac0b1f10 msgr2=0x7f28ac0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 -- 172.21.15.106:0/276376495 shutdown_connections 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.115+0000 7f28b20b7640 1 -- 172.21.15.106:0/276376495 wait complete. 2024-04-13T03:15:23.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.116+0000 7f28b20b7640 1 Processor -- start 2024-04-13T03:15:23.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.116+0000 7f28b20b7640 1 -- start start 2024-04-13T03:15:23.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ac06cb10 0x7f28ac0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 0x7f28ac157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0b5520 con 0x7f28ac0cdf50 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0b53a0 con 0x7f28ac06f160 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28ac0b56a0 con 0x7f28ac06cb10 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53744/0 (socket says 172.21.15.106:53744) 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 -- 172.21.15.106:0/3856888513 learned_addr learned my addr 172.21.15.106:0/3856888513 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28aaffd640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 0x7f28ac157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 -- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ac06cb10 msgr2=0x7f28ac0aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28ab7fe640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ac06cb10 0x7f28ac0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ac06cb10 0x7f28ac0aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 -- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 msgr2=0x7f28ac157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 0x7f28ac157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 -- 172.21.15.106:0/3856888513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0af4f0 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28aaffd640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 0x7f28ac157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28abfff640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4044020 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a403a040 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4039050 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.117+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac0c4280 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.118+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac0c1d30 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.118+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac068a10 con 0x7f28ac0cdf50 2024-04-13T03:15:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.119+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28a402cbc0 con 0x7f28ac0cdf50 2024-04-13T03:15:23.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.119+0000 7f2893fff640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 0x7f289806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.119+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f28a40ae2b0 con 0x7f28ac0cdf50 2024-04-13T03:15:23.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.119+0000 7f28ab7fe640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 0x7f289806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.120+0000 7f28ab7fe640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 0x7f289806ede0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f289c000f60 tx=0x7f289c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.124+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28a404e060 con 0x7f28ac0cdf50 2024-04-13T03:15:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.295+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28ac0998d0 con 0x7f289806c920 2024-04-13T03:15:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.301+0000 7f2893fff640 1 -- 172.21.15.106:0/3856888513 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252072 (secure 0 0 0) 0x7f28ac0998d0 con 0x7f289806c920 2024-04-13T03:15:23.305 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:15:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 msgr2=0x7f289806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 0x7f289806ede0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f289c000f60 tx=0x7f289c009040 comp rx=0 tx=0).stop 2024-04-13T03:15:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 msgr2=0x7f28ac1f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4004a40 comp rx=0 tx=0).stop 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 shutdown_connections 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f289806c920 0x7f289806ede0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ac0cdf50 0x7f28ac1f81d0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ac06f160 0x7f28ac157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 --2- 172.21.15.106:0/3856888513 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ac06cb10 0x7f28ac0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 >> 172.21.15.106:0/3856888513 conn(0x7f28ac0b1f10 msgr2=0x7f28ac0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 shutdown_connections 2024-04-13T03:15:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.304+0000 7f28b20b7640 1 -- 172.21.15.106:0/3856888513 wait complete. 2024-04-13T03:15:23.316 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-04-13T03:15:22.440178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255312,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719408,"statfs":{"total":773094113280,"available":772832673792,"internally_reserved":0,"allocated":10231808,"data_stored":5321836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.494015"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245999+0000","last_change":"2024-04-13T03:15:19.964347+0000","last_active":"2024-04-13T03:15:21.245999+0000","last_peered":"2024-04-13T03:15:21.245999+0000","last_clean":"2024-04-13T03:15:21.245999+0000","last_became_active":"2024-04-13T03:15:19.964070+0000","last_became_peered":"2024-04-13T03:15:19.964070+0000","last_unstale":"2024-04-13T03:15:21.245999+0000","last_undegraded":"2024-04-13T03:15:21.245999+0000","last_fullsized":"2024-04-13T03:15:21.245999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:08:32.030220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246463+0000","last_change":"2024-04-13T03:15:19.966839+0000","last_active":"2024-04-13T03:15:21.246463+0000","last_peered":"2024-04-13T03:15:21.246463+0000","last_clean":"2024-04-13T03:15:21.246463+0000","last_became_active":"2024-04-13T03:15:19.966574+0000","last_became_peered":"2024-04-13T03:15:19.966574+0000","last_unstale":"2024-04-13T03:15:21.246463+0000","last_undegraded":"2024-04-13T03:15:21.246463+0000","last_fullsized":"2024-04-13T03:15:21.246463+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:12.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245762+0000","last_change":"2024-04-13T03:15:19.968398+0000","last_active":"2024-04-13T03:15:21.245762+0000","last_peered":"2024-04-13T03:15:21.245762+0000","last_clean":"2024-04-13T03:15:21.245762+0000","last_became_active":"2024-04-13T03:15:19.968118+0000","last_became_peered":"2024-04-13T03:15:19.968118+0000","last_unstale":"2024-04-13T03:15:21.245762+0000","last_undegraded":"2024-04-13T03:15:21.245762+0000","last_fullsized":"2024-04-13T03:15:21.245762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:15:58.438037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965799+0000","last_change":"2024-04-13T03:15:19.967764+0000","last_active":"2024-04-13T03:15:20.965799+0000","last_peered":"2024-04-13T03:15:20.965799+0000","last_clean":"2024-04-13T03:15:20.965799+0000","last_became_active":"2024-04-13T03:15:19.966897+0000","last_became_peered":"2024-04-13T03:15:19.966897+0000","last_unstale":"2024-04-13T03:15:20.965799+0000","last_undegraded":"2024-04-13T03:15:20.965799+0000","last_fullsized":"2024-04-13T03:15:20.965799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:09:20.958767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247545+0000","last_change":"2024-04-13T03:15:19.968823+0000","last_active":"2024-04-13T03:15:21.247545+0000","last_peered":"2024-04-13T03:15:21.247545+0000","last_clean":"2024-04-13T03:15:21.247545+0000","last_became_active":"2024-04-13T03:15:19.968564+0000","last_became_peered":"2024-04-13T03:15:19.968564+0000","last_unstale":"2024-04-13T03:15:21.247545+0000","last_undegraded":"2024-04-13T03:15:21.247545+0000","last_fullsized":"2024-04-13T03:15:21.247545+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:06:09.027110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964610+0000","last_change":"2024-04-13T03:15:19.975653+0000","last_active":"2024-04-13T03:15:20.964610+0000","last_peered":"2024-04-13T03:15:20.964610+0000","last_clean":"2024-04-13T03:15:20.964610+0000","last_became_active":"2024-04-13T03:15:19.975360+0000","last_became_peered":"2024-04-13T03:15:19.975360+0000","last_unstale":"2024-04-13T03:15:20.964610+0000","last_undegraded":"2024-04-13T03:15:20.964610+0000","last_fullsized":"2024-04-13T03:15:20.964610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:49:14.511264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247173+0000","last_change":"2024-04-13T03:15:19.971621+0000","last_active":"2024-04-13T03:15:21.247173+0000","last_peered":"2024-04-13T03:15:21.247173+0000","last_clean":"2024-04-13T03:15:21.247173+0000","last_became_active":"2024-04-13T03:15:19.971367+0000","last_became_peered":"2024-04-13T03:15:19.971367+0000","last_unstale":"2024-04-13T03:15:21.247173+0000","last_undegraded":"2024-04-13T03:15:21.247173+0000","last_fullsized":"2024-04-13T03:15:21.247173+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:49.786951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965579+0000","last_change":"2024-04-13T03:15:19.968071+0000","last_active":"2024-04-13T03:15:20.965579+0000","last_peered":"2024-04-13T03:15:20.965579+0000","last_clean":"2024-04-13T03:15:20.965579+0000","last_became_active":"2024-04-13T03:15:19.967788+0000","last_became_peered":"2024-04-13T03:15:19.967788+0000","last_unstale":"2024-04-13T03:15:20.965579+0000","last_undegraded":"2024-04-13T03:15:20.965579+0000","last_fullsized":"2024-04-13T03:15:20.965579+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:59:00.328223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246448+0000","last_change":"2024-04-13T03:15:19.970244+0000","last_active":"2024-04-13T03:15:21.246448+0000","last_peered":"2024-04-13T03:15:21.246448+0000","last_clean":"2024-04-13T03:15:21.246448+0000","last_became_active":"2024-04-13T03:15:19.969974+0000","last_became_peered":"2024-04-13T03:15:19.969974+0000","last_unstale":"2024-04-13T03:15:21.246448+0000","last_undegraded":"2024-04-13T03:15:21.246448+0000","last_fullsized":"2024-04-13T03:15:21.246448+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245517+0000","last_change":"2024-04-13T03:15:19.971517+0000","last_active":"2024-04-13T03:15:21.245517+0000","last_peered":"2024-04-13T03:15:21.245517+0000","last_clean":"2024-04-13T03:15:21.245517+0000","last_became_active":"2024-04-13T03:15:19.971249+0000","last_became_peered":"2024-04-13T03:15:19.971249+0000","last_unstale":"2024-04-13T03:15:21.245517+0000","last_undegraded":"2024-04-13T03:15:21.245517+0000","last_fullsized":"2024-04-13T03:15:21.245517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:44:17.964010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247240+0000","last_change":"2024-04-13T03:15:19.980895+0000","last_active":"2024-04-13T03:15:21.247240+0000","last_peered":"2024-04-13T03:15:21.247240+0000","last_clean":"2024-04-13T03:15:21.247240+0000","last_became_active":"2024-04-13T03:15:19.980624+0000","last_became_peered":"2024-04-13T03:15:19.980624+0000","last_unstale":"2024-04-13T03:15:21.247240+0000","last_undegraded":"2024-04-13T03:15:21.247240+0000","last_fullsized":"2024-04-13T03:15:21.247240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:09.461685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245412+0000","last_change":"2024-04-13T03:15:19.969165+0000","last_active":"2024-04-13T03:15:21.245412+0000","last_peered":"2024-04-13T03:15:21.245412+0000","last_clean":"2024-04-13T03:15:21.245412+0000","last_became_active":"2024-04-13T03:15:19.968854+0000","last_became_peered":"2024-04-13T03:15:19.968854+0000","last_unstale":"2024-04-13T03:15:21.245412+0000","last_undegraded":"2024-04-13T03:15:21.245412+0000","last_fullsized":"2024-04-13T03:15:21.245412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:58:25.178271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965679+0000","last_change":"2024-04-13T03:15:19.975395+0000","last_active":"2024-04-13T03:15:20.965679+0000","last_peered":"2024-04-13T03:15:20.965679+0000","last_clean":"2024-04-13T03:15:20.965679+0000","last_became_active":"2024-04-13T03:15:19.974911+0000","last_became_peered":"2024-04-13T03:15:19.974911+0000","last_unstale":"2024-04-13T03:15:20.965679+0000","last_undegraded":"2024-04-13T03:15:20.965679+0000","last_fullsized":"2024-04-13T03:15:20.965679+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:52.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246810+0000","last_change":"2024-04-13T03:15:19.974152+0000","last_active":"2024-04-13T03:15:21.246810+0000","last_peered":"2024-04-13T03:15:21.246810+0000","last_clean":"2024-04-13T03:15:21.246810+0000","last_became_active":"2024-04-13T03:15:19.973745+0000","last_became_peered":"2024-04-13T03:15:19.973745+0000","last_unstale":"2024-04-13T03:15:21.246810+0000","last_undegraded":"2024-04-13T03:15:21.246810+0000","last_fullsized":"2024-04-13T03:15:21.246810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:42.417633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965632+0000","last_change":"2024-04-13T03:15:19.970797+0000","last_active":"2024-04-13T03:15:20.965632+0000","last_peered":"2024-04-13T03:15:20.965632+0000","last_clean":"2024-04-13T03:15:20.965632+0000","last_became_active":"2024-04-13T03:15:19.970506+0000","last_became_peered":"2024-04-13T03:15:19.970506+0000","last_unstale":"2024-04-13T03:15:20.965632+0000","last_undegraded":"2024-04-13T03:15:20.965632+0000","last_fullsized":"2024-04-13T03:15:20.965632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:47.736587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964550+0000","last_change":"2024-04-13T03:15:19.979929+0000","last_active":"2024-04-13T03:15:20.964550+0000","last_peered":"2024-04-13T03:15:20.964550+0000","last_clean":"2024-04-13T03:15:20.964550+0000","last_became_active":"2024-04-13T03:15:19.979310+0000","last_became_peered":"2024-04-13T03:15:19.979310+0000","last_unstale":"2024-04-13T03:15:20.964550+0000","last_undegraded":"2024-04-13T03:15:20.964550+0000","last_fullsized":"2024-04-13T03:15:20.964550+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:45:20.338779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965050+0000","last_change":"2024-04-13T03:15:19.979589+0000","last_active":"2024-04-13T03:15:20.965050+0000","last_peered":"2024-04-13T03:15:20.965050+0000","last_clean":"2024-04-13T03:15:20.965050+0000","last_became_active":"2024-04-13T03:15:19.977584+0000","last_became_peered":"2024-04-13T03:15:19.977584+0000","last_unstale":"2024-04-13T03:15:20.965050+0000","last_undegraded":"2024-04-13T03:15:20.965050+0000","last_fullsized":"2024-04-13T03:15:20.965050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:20.025715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245662+0000","last_change":"2024-04-13T03:15:19.969038+0000","last_active":"2024-04-13T03:15:21.245662+0000","last_peered":"2024-04-13T03:15:21.245662+0000","last_clean":"2024-04-13T03:15:21.245662+0000","last_became_active":"2024-04-13T03:15:19.968571+0000","last_became_peered":"2024-04-13T03:15:19.968571+0000","last_unstale":"2024-04-13T03:15:21.245662+0000","last_undegraded":"2024-04-13T03:15:21.245662+0000","last_fullsized":"2024-04-13T03:15:21.245662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:33.741751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247121+0000","last_change":"2024-04-13T03:15:19.974321+0000","last_active":"2024-04-13T03:15:21.247121+0000","last_peered":"2024-04-13T03:15:21.247121+0000","last_clean":"2024-04-13T03:15:21.247121+0000","last_became_active":"2024-04-13T03:15:19.973828+0000","last_became_peered":"2024-04-13T03:15:19.973828+0000","last_unstale":"2024-04-13T03:15:21.247121+0000","last_undegraded":"2024-04-13T03:15:21.247121+0000","last_fullsized":"2024-04-13T03:15:21.247121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:23:51.416101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965881+0000","last_change":"2024-04-13T03:15:19.970897+0000","last_active":"2024-04-13T03:15:20.965881+0000","last_peered":"2024-04-13T03:15:20.965881+0000","last_clean":"2024-04-13T03:15:20.965881+0000","last_became_active":"2024-04-13T03:15:19.970640+0000","last_became_peered":"2024-04-13T03:15:19.970640+0000","last_unstale":"2024-04-13T03:15:20.965881+0000","last_undegraded":"2024-04-13T03:15:20.965881+0000","last_fullsized":"2024-04-13T03:15:20.965881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:04:03.433211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247541+0000","last_change":"2024-04-13T03:15:19.966154+0000","last_active":"2024-04-13T03:15:21.247541+0000","last_peered":"2024-04-13T03:15:21.247541+0000","last_clean":"2024-04-13T03:15:21.247541+0000","last_became_active":"2024-04-13T03:15:19.965884+0000","last_became_peered":"2024-04-13T03:15:19.965884+0000","last_unstale":"2024-04-13T03:15:21.247541+0000","last_undegraded":"2024-04-13T03:15:21.247541+0000","last_fullsized":"2024-04-13T03:15:21.247541+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:21.487590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246988+0000","last_change":"2024-04-13T03:15:19.973557+0000","last_active":"2024-04-13T03:15:21.246988+0000","last_peered":"2024-04-13T03:15:21.246988+0000","last_clean":"2024-04-13T03:15:21.246988+0000","last_became_active":"2024-04-13T03:15:19.972480+0000","last_became_peered":"2024-04-13T03:15:19.972480+0000","last_unstale":"2024-04-13T03:15:21.246988+0000","last_undegraded":"2024-04-13T03:15:21.246988+0000","last_fullsized":"2024-04-13T03:15:21.246988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:48:37.673448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966548+0000","last_change":"2024-04-13T03:15:19.975042+0000","last_active":"2024-04-13T03:15:20.966548+0000","last_peered":"2024-04-13T03:15:20.966548+0000","last_clean":"2024-04-13T03:15:20.966548+0000","last_became_active":"2024-04-13T03:15:19.974732+0000","last_became_peered":"2024-04-13T03:15:19.974732+0000","last_unstale":"2024-04-13T03:15:20.966548+0000","last_undegraded":"2024-04-13T03:15:20.966548+0000","last_fullsized":"2024-04-13T03:15:20.966548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:58:11.629615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245682+0000","last_change":"2024-04-13T03:15:19.967816+0000","last_active":"2024-04-13T03:15:21.245682+0000","last_peered":"2024-04-13T03:15:21.245682+0000","last_clean":"2024-04-13T03:15:21.245682+0000","last_became_active":"2024-04-13T03:15:19.967503+0000","last_became_peered":"2024-04-13T03:15:19.967503+0000","last_unstale":"2024-04-13T03:15:21.245682+0000","last_undegraded":"2024-04-13T03:15:21.245682+0000","last_fullsized":"2024-04-13T03:15:21.245682+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:50.645873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246486+0000","last_change":"2024-04-13T03:15:19.966356+0000","last_active":"2024-04-13T03:15:21.246486+0000","last_peered":"2024-04-13T03:15:21.246486+0000","last_clean":"2024-04-13T03:15:21.246486+0000","last_became_active":"2024-04-13T03:15:19.966081+0000","last_became_peered":"2024-04-13T03:15:19.966081+0000","last_unstale":"2024-04-13T03:15:21.246486+0000","last_undegraded":"2024-04-13T03:15:21.246486+0000","last_fullsized":"2024-04-13T03:15:21.246486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:50.929473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246714+0000","last_change":"2024-04-13T03:15:19.971998+0000","last_active":"2024-04-13T03:15:21.246714+0000","last_peered":"2024-04-13T03:15:21.246714+0000","last_clean":"2024-04-13T03:15:21.246714+0000","last_became_active":"2024-04-13T03:15:19.971721+0000","last_became_peered":"2024-04-13T03:15:19.971721+0000","last_unstale":"2024-04-13T03:15:21.246714+0000","last_undegraded":"2024-04-13T03:15:21.246714+0000","last_fullsized":"2024-04-13T03:15:21.246714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:50.929473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245562+0000","last_change":"2024-04-13T03:15:19.968494+0000","last_active":"2024-04-13T03:15:21.245562+0000","last_peered":"2024-04-13T03:15:21.245562+0000","last_clean":"2024-04-13T03:15:21.245562+0000","last_became_active":"2024-04-13T03:15:19.968242+0000","last_became_peered":"2024-04-13T03:15:19.968242+0000","last_unstale":"2024-04-13T03:15:21.245562+0000","last_undegraded":"2024-04-13T03:15:21.245562+0000","last_fullsized":"2024-04-13T03:15:21.245562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:40:45.329981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965941+0000","last_change":"2024-04-13T03:15:19.977127+0000","last_active":"2024-04-13T03:15:20.965941+0000","last_peered":"2024-04-13T03:15:20.965941+0000","last_clean":"2024-04-13T03:15:20.965941+0000","last_became_active":"2024-04-13T03:15:19.976856+0000","last_became_peered":"2024-04-13T03:15:19.976856+0000","last_unstale":"2024-04-13T03:15:20.965941+0000","last_undegraded":"2024-04-13T03:15:20.965941+0000","last_fullsized":"2024-04-13T03:15:20.965941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:36:10.037015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247089+0000","last_change":"2024-04-13T03:15:19.969207+0000","last_active":"2024-04-13T03:15:21.247089+0000","last_peered":"2024-04-13T03:15:21.247089+0000","last_clean":"2024-04-13T03:15:21.247089+0000","last_became_active":"2024-04-13T03:15:19.968177+0000","last_became_peered":"2024-04-13T03:15:19.968177+0000","last_unstale":"2024-04-13T03:15:21.247089+0000","last_undegraded":"2024-04-13T03:15:21.247089+0000","last_fullsized":"2024-04-13T03:15:21.247089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:02:55.255018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247240+0000","last_change":"2024-04-13T03:15:19.969647+0000","last_active":"2024-04-13T03:15:21.247240+0000","last_peered":"2024-04-13T03:15:21.247240+0000","last_clean":"2024-04-13T03:15:21.247240+0000","last_became_active":"2024-04-13T03:15:19.969379+0000","last_became_peered":"2024-04-13T03:15:19.969379+0000","last_unstale":"2024-04-13T03:15:21.247240+0000","last_undegraded":"2024-04-13T03:15:21.247240+0000","last_fullsized":"2024-04-13T03:15:21.247240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:10:18.534253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246159+0000","last_change":"2024-04-13T03:15:19.972364+0000","last_active":"2024-04-13T03:15:21.246159+0000","last_peered":"2024-04-13T03:15:21.246159+0000","last_clean":"2024-04-13T03:15:21.246159+0000","last_became_active":"2024-04-13T03:15:19.972090+0000","last_became_peered":"2024-04-13T03:15:19.972090+0000","last_unstale":"2024-04-13T03:15:21.246159+0000","last_undegraded":"2024-04-13T03:15:21.246159+0000","last_fullsized":"2024-04-13T03:15:21.246159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964603+0000","last_change":"2024-04-13T03:15:19.979558+0000","last_active":"2024-04-13T03:15:20.964603+0000","last_peered":"2024-04-13T03:15:20.964603+0000","last_clean":"2024-04-13T03:15:20.964603+0000","last_became_active":"2024-04-13T03:15:19.978735+0000","last_became_peered":"2024-04-13T03:15:19.978735+0000","last_unstale":"2024-04-13T03:15:20.964603+0000","last_undegraded":"2024-04-13T03:15:20.964603+0000","last_fullsized":"2024-04-13T03:15:20.964603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:53:10.421528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964899+0000","last_change":"2024-04-13T03:15:19.977378+0000","last_active":"2024-04-13T03:15:20.964899+0000","last_peered":"2024-04-13T03:15:20.964899+0000","last_clean":"2024-04-13T03:15:20.964899+0000","last_became_active":"2024-04-13T03:15:19.976433+0000","last_became_peered":"2024-04-13T03:15:19.976433+0000","last_unstale":"2024-04-13T03:15:20.964899+0000","last_undegraded":"2024-04-13T03:15:20.964899+0000","last_fullsized":"2024-04-13T03:15:20.964899+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:04:29.507885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.967155+0000","last_change":"2024-04-13T03:15:18.974373+0000","last_active":"2024-04-13T03:15:20.967155+0000","last_peered":"2024-04-13T03:15:20.967155+0000","last_clean":"2024-04-13T03:15:20.967155+0000","last_became_active":"2024-04-13T03:15:18.973461+0000","last_became_peered":"2024-04-13T03:15:18.973461+0000","last_unstale":"2024-04-13T03:15:20.967155+0000","last_undegraded":"2024-04-13T03:15:20.967155+0000","last_fullsized":"2024-04-13T03:15:20.967155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:12:04.167831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964947+0000","last_change":"2024-04-13T03:15:19.972289+0000","last_active":"2024-04-13T03:15:20.964947+0000","last_peered":"2024-04-13T03:15:20.964947+0000","last_clean":"2024-04-13T03:15:20.964947+0000","last_became_active":"2024-04-13T03:15:19.963471+0000","last_became_peered":"2024-04-13T03:15:19.963471+0000","last_unstale":"2024-04-13T03:15:20.964947+0000","last_undegraded":"2024-04-13T03:15:20.964947+0000","last_fullsized":"2024-04-13T03:15:20.964947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:01.136698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247038+0000","last_change":"2024-04-13T03:15:18.968924+0000","last_active":"2024-04-13T03:15:21.247038+0000","last_peered":"2024-04-13T03:15:21.247038+0000","last_clean":"2024-04-13T03:15:21.247038+0000","last_became_active":"2024-04-13T03:15:18.968653+0000","last_became_peered":"2024-04-13T03:15:18.968653+0000","last_unstale":"2024-04-13T03:15:21.247038+0000","last_undegraded":"2024-04-13T03:15:21.247038+0000","last_fullsized":"2024-04-13T03:15:21.247038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:34:22.836564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247301+0000","last_change":"2024-04-13T03:15:19.973001+0000","last_active":"2024-04-13T03:15:21.247301+0000","last_peered":"2024-04-13T03:15:21.247301+0000","last_clean":"2024-04-13T03:15:21.247301+0000","last_became_active":"2024-04-13T03:15:19.972725+0000","last_became_peered":"2024-04-13T03:15:19.972725+0000","last_unstale":"2024-04-13T03:15:21.247301+0000","last_undegraded":"2024-04-13T03:15:21.247301+0000","last_fullsized":"2024-04-13T03:15:21.247301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:40:30.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247099+0000","last_change":"2024-04-13T03:15:18.973196+0000","last_active":"2024-04-13T03:15:21.247099+0000","last_peered":"2024-04-13T03:15:21.247099+0000","last_clean":"2024-04-13T03:15:21.247099+0000","last_became_active":"2024-04-13T03:15:18.972930+0000","last_became_peered":"2024-04-13T03:15:18.972930+0000","last_unstale":"2024-04-13T03:15:21.247099+0000","last_undegraded":"2024-04-13T03:15:21.247099+0000","last_fullsized":"2024-04-13T03:15:21.247099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:13:49.925520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965073+0000","last_change":"2024-04-13T03:15:19.972396+0000","last_active":"2024-04-13T03:15:20.965073+0000","last_peered":"2024-04-13T03:15:20.965073+0000","last_clean":"2024-04-13T03:15:20.965073+0000","last_became_active":"2024-04-13T03:15:19.963504+0000","last_became_peered":"2024-04-13T03:15:19.963504+0000","last_unstale":"2024-04-13T03:15:20.965073+0000","last_undegraded":"2024-04-13T03:15:20.965073+0000","last_fullsized":"2024-04-13T03:15:20.965073+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:00.727075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966982+0000","last_change":"2024-04-13T03:15:18.973253+0000","last_active":"2024-04-13T03:15:20.966982+0000","last_peered":"2024-04-13T03:15:20.966982+0000","last_clean":"2024-04-13T03:15:20.966982+0000","last_became_active":"2024-04-13T03:15:18.973016+0000","last_became_peered":"2024-04-13T03:15:18.973016+0000","last_unstale":"2024-04-13T03:15:20.966982+0000","last_undegraded":"2024-04-13T03:15:20.966982+0000","last_fullsized":"2024-04-13T03:15:20.966982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965928+0000","last_change":"2024-04-13T03:15:19.970488+0000","last_active":"2024-04-13T03:15:20.965928+0000","last_peered":"2024-04-13T03:15:20.965928+0000","last_clean":"2024-04-13T03:15:20.965928+0000","last_became_active":"2024-04-13T03:15:19.970218+0000","last_became_peered":"2024-04-13T03:15:19.970218+0000","last_unstale":"2024-04-13T03:15:20.965928+0000","last_undegraded":"2024-04-13T03:15:20.965928+0000","last_fullsized":"2024-04-13T03:15:20.965928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:41:48.557267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247624+0000","last_change":"2024-04-13T03:15:18.960938+0000","last_active":"2024-04-13T03:15:21.247624+0000","last_peered":"2024-04-13T03:15:21.247624+0000","last_clean":"2024-04-13T03:15:21.247624+0000","last_became_active":"2024-04-13T03:15:18.960534+0000","last_became_peered":"2024-04-13T03:15:18.960534+0000","last_unstale":"2024-04-13T03:15:21.247624+0000","last_undegraded":"2024-04-13T03:15:21.247624+0000","last_fullsized":"2024-04-13T03:15:21.247624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:37:30.553956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966905+0000","last_change":"2024-04-13T03:15:19.964778+0000","last_active":"2024-04-13T03:15:20.966905+0000","last_peered":"2024-04-13T03:15:20.966905+0000","last_clean":"2024-04-13T03:15:20.966905+0000","last_became_active":"2024-04-13T03:15:19.964503+0000","last_became_peered":"2024-04-13T03:15:19.964503+0000","last_unstale":"2024-04-13T03:15:20.966905+0000","last_undegraded":"2024-04-13T03:15:20.966905+0000","last_fullsized":"2024-04-13T03:15:20.966905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:29:47.736587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966086+0000","last_change":"2024-04-13T03:15:18.967368+0000","last_active":"2024-04-13T03:15:20.966086+0000","last_peered":"2024-04-13T03:15:20.966086+0000","last_clean":"2024-04-13T03:15:20.966086+0000","last_became_active":"2024-04-13T03:15:18.967051+0000","last_became_peered":"2024-04-13T03:15:18.967051+0000","last_unstale":"2024-04-13T03:15:20.966086+0000","last_undegraded":"2024-04-13T03:15:20.966086+0000","last_fullsized":"2024-04-13T03:15:20.966086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964745+0000","last_change":"2024-04-13T03:15:19.975957+0000","last_active":"2024-04-13T03:15:20.964745+0000","last_peered":"2024-04-13T03:15:20.964745+0000","last_clean":"2024-04-13T03:15:20.964745+0000","last_became_active":"2024-04-13T03:15:19.975053+0000","last_became_peered":"2024-04-13T03:15:19.975053+0000","last_unstale":"2024-04-13T03:15:20.964745+0000","last_undegraded":"2024-04-13T03:15:20.964745+0000","last_fullsized":"2024-04-13T03:15:20.964745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:51:42.407125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245642+0000","last_change":"2024-04-13T03:15:18.974497+0000","last_active":"2024-04-13T03:15:21.245642+0000","last_peered":"2024-04-13T03:15:21.245642+0000","last_clean":"2024-04-13T03:15:21.245642+0000","last_became_active":"2024-04-13T03:15:18.974188+0000","last_became_peered":"2024-04-13T03:15:18.974188+0000","last_unstale":"2024-04-13T03:15:21.245642+0000","last_undegraded":"2024-04-13T03:15:21.245642+0000","last_fullsized":"2024-04-13T03:15:21.245642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:20.483637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965963+0000","last_change":"2024-04-13T03:15:19.975155+0000","last_active":"2024-04-13T03:15:20.965963+0000","last_peered":"2024-04-13T03:15:20.965963+0000","last_clean":"2024-04-13T03:15:20.965963+0000","last_became_active":"2024-04-13T03:15:19.974234+0000","last_became_peered":"2024-04-13T03:15:19.974234+0000","last_unstale":"2024-04-13T03:15:20.965963+0000","last_undegraded":"2024-04-13T03:15:20.965963+0000","last_fullsized":"2024-04-13T03:15:20.965963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:21.980518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246337+0000","last_change":"2024-04-13T03:15:18.974213+0000","last_active":"2024-04-13T03:15:21.246337+0000","last_peered":"2024-04-13T03:15:21.246337+0000","last_clean":"2024-04-13T03:15:21.246337+0000","last_became_active":"2024-04-13T03:15:18.972521+0000","last_became_peered":"2024-04-13T03:15:18.972521+0000","last_unstale":"2024-04-13T03:15:21.246337+0000","last_undegraded":"2024-04-13T03:15:21.246337+0000","last_fullsized":"2024-04-13T03:15:21.246337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:42.526775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246655+0000","last_change":"2024-04-13T03:15:19.969956+0000","last_active":"2024-04-13T03:15:21.246655+0000","last_peered":"2024-04-13T03:15:21.246655+0000","last_clean":"2024-04-13T03:15:21.246655+0000","last_became_active":"2024-04-13T03:15:19.969647+0000","last_became_peered":"2024-04-13T03:15:19.969647+0000","last_unstale":"2024-04-13T03:15:21.246655+0000","last_undegraded":"2024-04-13T03:15:21.246655+0000","last_fullsized":"2024-04-13T03:15:21.246655+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:12.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246663+0000","last_change":"2024-04-13T03:15:18.973458+0000","last_active":"2024-04-13T03:15:21.246663+0000","last_peered":"2024-04-13T03:15:21.246663+0000","last_clean":"2024-04-13T03:15:21.246663+0000","last_became_active":"2024-04-13T03:15:18.973183+0000","last_became_peered":"2024-04-13T03:15:18.973183+0000","last_unstale":"2024-04-13T03:15:21.246663+0000","last_undegraded":"2024-04-13T03:15:21.246663+0000","last_fullsized":"2024-04-13T03:15:21.246663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:58:59.324270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246943+0000","last_change":"2024-04-13T03:15:19.972097+0000","last_active":"2024-04-13T03:15:21.246943+0000","last_peered":"2024-04-13T03:15:21.246943+0000","last_clean":"2024-04-13T03:15:21.246943+0000","last_became_active":"2024-04-13T03:15:19.971816+0000","last_became_peered":"2024-04-13T03:15:19.971816+0000","last_unstale":"2024-04-13T03:15:21.246943+0000","last_undegraded":"2024-04-13T03:15:21.246943+0000","last_fullsized":"2024-04-13T03:15:21.246943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:26.384212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245885+0000","last_change":"2024-04-13T03:15:18.973131+0000","last_active":"2024-04-13T03:15:21.245885+0000","last_peered":"2024-04-13T03:15:21.245885+0000","last_clean":"2024-04-13T03:15:21.245885+0000","last_became_active":"2024-04-13T03:15:18.972829+0000","last_became_peered":"2024-04-13T03:15:18.972829+0000","last_unstale":"2024-04-13T03:15:21.245885+0000","last_undegraded":"2024-04-13T03:15:21.245885+0000","last_fullsized":"2024-04-13T03:15:21.245885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:06:08.023157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245799+0000","last_change":"2024-04-13T03:15:19.963852+0000","last_active":"2024-04-13T03:15:21.245799+0000","last_peered":"2024-04-13T03:15:21.245799+0000","last_clean":"2024-04-13T03:15:21.245799+0000","last_became_active":"2024-04-13T03:15:19.963593+0000","last_became_peered":"2024-04-13T03:15:19.963593+0000","last_unstale":"2024-04-13T03:15:21.245799+0000","last_undegraded":"2024-04-13T03:15:21.245799+0000","last_fullsized":"2024-04-13T03:15:21.245799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:40:49.104144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247341+0000","last_change":"2024-04-13T03:15:18.964113+0000","last_active":"2024-04-13T03:15:21.247341+0000","last_peered":"2024-04-13T03:15:21.247341+0000","last_clean":"2024-04-13T03:15:21.247341+0000","last_became_active":"2024-04-13T03:15:18.963567+0000","last_became_peered":"2024-04-13T03:15:18.963567+0000","last_unstale":"2024-04-13T03:15:21.247341+0000","last_undegraded":"2024-04-13T03:15:21.247341+0000","last_fullsized":"2024-04-13T03:15:21.247341+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:00.449083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966086+0000","last_change":"2024-04-13T03:15:19.969514+0000","last_active":"2024-04-13T03:15:20.966086+0000","last_peered":"2024-04-13T03:15:20.966086+0000","last_clean":"2024-04-13T03:15:20.966086+0000","last_became_active":"2024-04-13T03:15:19.969251+0000","last_became_peered":"2024-04-13T03:15:19.969251+0000","last_unstale":"2024-04-13T03:15:20.966086+0000","last_undegraded":"2024-04-13T03:15:20.966086+0000","last_fullsized":"2024-04-13T03:15:20.966086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965012+0000","last_change":"2024-04-13T03:15:18.974486+0000","last_active":"2024-04-13T03:15:20.965012+0000","last_peered":"2024-04-13T03:15:20.965012+0000","last_clean":"2024-04-13T03:15:20.965012+0000","last_became_active":"2024-04-13T03:15:18.974178+0000","last_became_peered":"2024-04-13T03:15:18.974178+0000","last_unstale":"2024-04-13T03:15:20.965012+0000","last_undegraded":"2024-04-13T03:15:20.965012+0000","last_fullsized":"2024-04-13T03:15:20.965012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:40:48.782998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965720+0000","last_change":"2024-04-13T03:15:19.967462+0000","last_active":"2024-04-13T03:15:20.965720+0000","last_peered":"2024-04-13T03:15:20.965720+0000","last_clean":"2024-04-13T03:15:20.965720+0000","last_became_active":"2024-04-13T03:15:19.967050+0000","last_became_peered":"2024-04-13T03:15:19.967050+0000","last_unstale":"2024-04-13T03:15:20.965720+0000","last_undegraded":"2024-04-13T03:15:20.965720+0000","last_fullsized":"2024-04-13T03:15:20.965720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:07.224694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247136+0000","last_change":"2024-04-13T03:15:18.973728+0000","last_active":"2024-04-13T03:15:21.247136+0000","last_peered":"2024-04-13T03:15:21.247136+0000","last_clean":"2024-04-13T03:15:21.247136+0000","last_became_active":"2024-04-13T03:15:18.973479+0000","last_became_peered":"2024-04-13T03:15:18.973479+0000","last_unstale":"2024-04-13T03:15:21.247136+0000","last_undegraded":"2024-04-13T03:15:21.247136+0000","last_fullsized":"2024-04-13T03:15:21.247136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:33.321965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246472+0000","last_change":"2024-04-13T03:15:19.964411+0000","last_active":"2024-04-13T03:15:21.246472+0000","last_peered":"2024-04-13T03:15:21.246472+0000","last_clean":"2024-04-13T03:15:21.246472+0000","last_became_active":"2024-04-13T03:15:19.964130+0000","last_became_peered":"2024-04-13T03:15:19.964130+0000","last_unstale":"2024-04-13T03:15:21.246472+0000","last_undegraded":"2024-04-13T03:15:21.246472+0000","last_fullsized":"2024-04-13T03:15:21.246472+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964824+0000","last_change":"2024-04-13T03:15:18.968822+0000","last_active":"2024-04-13T03:15:20.964824+0000","last_peered":"2024-04-13T03:15:20.964824+0000","last_clean":"2024-04-13T03:15:20.964824+0000","last_became_active":"2024-04-13T03:15:18.968528+0000","last_became_peered":"2024-04-13T03:15:18.968528+0000","last_unstale":"2024-04-13T03:15:20.964824+0000","last_undegraded":"2024-04-13T03:15:20.964824+0000","last_fullsized":"2024-04-13T03:15:20.964824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:25.380259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246124+0000","last_change":"2024-04-13T03:15:19.975015+0000","last_active":"2024-04-13T03:15:21.246124+0000","last_peered":"2024-04-13T03:15:21.246124+0000","last_clean":"2024-04-13T03:15:21.246124+0000","last_became_active":"2024-04-13T03:15:19.974732+0000","last_became_peered":"2024-04-13T03:15:19.974732+0000","last_unstale":"2024-04-13T03:15:21.246124+0000","last_undegraded":"2024-04-13T03:15:21.246124+0000","last_fullsized":"2024-04-13T03:15:21.246124+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:18.467921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965657+0000","last_change":"2024-04-13T03:15:18.974120+0000","last_active":"2024-04-13T03:15:20.965657+0000","last_peered":"2024-04-13T03:15:20.965657+0000","last_clean":"2024-04-13T03:15:20.965657+0000","last_became_active":"2024-04-13T03:15:18.973879+0000","last_became_peered":"2024-04-13T03:15:18.973879+0000","last_unstale":"2024-04-13T03:15:20.965657+0000","last_undegraded":"2024-04-13T03:15:20.965657+0000","last_fullsized":"2024-04-13T03:15:20.965657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:48:17.505348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245918+0000","last_change":"2024-04-13T03:15:19.966899+0000","last_active":"2024-04-13T03:15:21.245918+0000","last_peered":"2024-04-13T03:15:21.245918+0000","last_clean":"2024-04-13T03:15:21.245918+0000","last_became_active":"2024-04-13T03:15:19.966597+0000","last_became_peered":"2024-04-13T03:15:19.966597+0000","last_unstale":"2024-04-13T03:15:21.245918+0000","last_undegraded":"2024-04-13T03:15:21.245918+0000","last_fullsized":"2024-04-13T03:15:21.245918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:00:46.165705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965809+0000","last_change":"2024-04-13T03:15:18.960533+0000","last_active":"2024-04-13T03:15:20.965809+0000","last_peered":"2024-04-13T03:15:20.965809+0000","last_clean":"2024-04-13T03:15:20.965809+0000","last_became_active":"2024-04-13T03:15:18.960250+0000","last_became_peered":"2024-04-13T03:15:18.960250+0000","last_unstale":"2024-04-13T03:15:20.965809+0000","last_undegraded":"2024-04-13T03:15:20.965809+0000","last_fullsized":"2024-04-13T03:15:20.965809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:36:58.800747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965148+0000","last_change":"2024-04-13T03:15:19.975603+0000","last_active":"2024-04-13T03:15:20.965148+0000","last_peered":"2024-04-13T03:15:20.965148+0000","last_clean":"2024-04-13T03:15:20.965148+0000","last_became_active":"2024-04-13T03:15:19.975380+0000","last_became_peered":"2024-04-13T03:15:19.975380+0000","last_unstale":"2024-04-13T03:15:20.965148+0000","last_undegraded":"2024-04-13T03:15:20.965148+0000","last_fullsized":"2024-04-13T03:15:20.965148+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:38:42.826594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.967182+0000","last_change":"2024-04-13T03:15:18.975085+0000","last_active":"2024-04-13T03:15:20.967182+0000","last_peered":"2024-04-13T03:15:20.967182+0000","last_clean":"2024-04-13T03:15:20.967182+0000","last_became_active":"2024-04-13T03:15:18.974827+0000","last_became_peered":"2024-04-13T03:15:18.974827+0000","last_unstale":"2024-04-13T03:15:20.967182+0000","last_undegraded":"2024-04-13T03:15:20.967182+0000","last_fullsized":"2024-04-13T03:15:20.967182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:06.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966978+0000","last_change":"2024-04-13T03:15:19.970168+0000","last_active":"2024-04-13T03:15:20.966978+0000","last_peered":"2024-04-13T03:15:20.966978+0000","last_clean":"2024-04-13T03:15:20.966978+0000","last_became_active":"2024-04-13T03:15:19.969904+0000","last_became_peered":"2024-04-13T03:15:19.969904+0000","last_unstale":"2024-04-13T03:15:20.966978+0000","last_undegraded":"2024-04-13T03:15:20.966978+0000","last_fullsized":"2024-04-13T03:15:20.966978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:47:52.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247413+0000","last_change":"2024-04-13T03:15:18.974450+0000","last_active":"2024-04-13T03:15:21.247413+0000","last_peered":"2024-04-13T03:15:21.247413+0000","last_clean":"2024-04-13T03:15:21.247413+0000","last_became_active":"2024-04-13T03:15:18.974157+0000","last_became_peered":"2024-04-13T03:15:18.974157+0000","last_unstale":"2024-04-13T03:15:21.247413+0000","last_undegraded":"2024-04-13T03:15:21.247413+0000","last_fullsized":"2024-04-13T03:15:21.247413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246862+0000","last_change":"2024-04-13T03:15:19.968337+0000","last_active":"2024-04-13T03:15:21.246862+0000","last_peered":"2024-04-13T03:15:21.246862+0000","last_clean":"2024-04-13T03:15:21.246862+0000","last_became_active":"2024-04-13T03:15:19.968010+0000","last_became_peered":"2024-04-13T03:15:19.968010+0000","last_unstale":"2024-04-13T03:15:21.246862+0000","last_undegraded":"2024-04-13T03:15:21.246862+0000","last_fullsized":"2024-04-13T03:15:21.246862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:34.325918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247355+0000","last_change":"2024-04-13T03:15:18.964360+0000","last_active":"2024-04-13T03:15:21.247355+0000","last_peered":"2024-04-13T03:15:21.247355+0000","last_clean":"2024-04-13T03:15:21.247355+0000","last_became_active":"2024-04-13T03:15:18.963856+0000","last_became_peered":"2024-04-13T03:15:18.963856+0000","last_unstale":"2024-04-13T03:15:21.247355+0000","last_undegraded":"2024-04-13T03:15:21.247355+0000","last_fullsized":"2024-04-13T03:15:21.247355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:12:04.167831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964684+0000","last_change":"2024-04-13T03:15:19.974674+0000","last_active":"2024-04-13T03:15:20.964684+0000","last_peered":"2024-04-13T03:15:20.964684+0000","last_clean":"2024-04-13T03:15:20.964684+0000","last_became_active":"2024-04-13T03:15:19.974428+0000","last_became_peered":"2024-04-13T03:15:19.974428+0000","last_unstale":"2024-04-13T03:15:20.964684+0000","last_undegraded":"2024-04-13T03:15:20.964684+0000","last_fullsized":"2024-04-13T03:15:20.964684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:22:14.125862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246240+0000","last_change":"2024-04-13T03:15:18.966755+0000","last_active":"2024-04-13T03:15:21.246240+0000","last_peered":"2024-04-13T03:15:21.246240+0000","last_clean":"2024-04-13T03:15:21.246240+0000","last_became_active":"2024-04-13T03:15:18.966488+0000","last_became_peered":"2024-04-13T03:15:18.966488+0000","last_unstale":"2024-04-13T03:15:21.246240+0000","last_undegraded":"2024-04-13T03:15:21.246240+0000","last_fullsized":"2024-04-13T03:15:21.246240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:33.302316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246235+0000","last_change":"2024-04-13T03:15:19.966246+0000","last_active":"2024-04-13T03:15:21.246235+0000","last_peered":"2024-04-13T03:15:21.246235+0000","last_clean":"2024-04-13T03:15:21.246235+0000","last_became_active":"2024-04-13T03:15:19.965970+0000","last_became_peered":"2024-04-13T03:15:19.965970+0000","last_unstale":"2024-04-13T03:15:21.246235+0000","last_undegraded":"2024-04-13T03:15:21.246235+0000","last_fullsized":"2024-04-13T03:15:21.246235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:43.530729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246100+0000","last_change":"2024-04-13T03:15:18.970400+0000","last_active":"2024-04-13T03:15:21.246100+0000","last_peered":"2024-04-13T03:15:21.246100+0000","last_clean":"2024-04-13T03:15:21.246100+0000","last_became_active":"2024-04-13T03:15:18.970152+0000","last_became_peered":"2024-04-13T03:15:18.970152+0000","last_unstale":"2024-04-13T03:15:21.246100+0000","last_undegraded":"2024-04-13T03:15:21.246100+0000","last_fullsized":"2024-04-13T03:15:21.246100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:33:28.051506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247145+0000","last_change":"2024-04-13T03:15:19.971403+0000","last_active":"2024-04-13T03:15:21.247145+0000","last_peered":"2024-04-13T03:15:21.247145+0000","last_clean":"2024-04-13T03:15:21.247145+0000","last_became_active":"2024-04-13T03:15:19.971119+0000","last_became_peered":"2024-04-13T03:15:19.971119+0000","last_unstale":"2024-04-13T03:15:21.247145+0000","last_undegraded":"2024-04-13T03:15:21.247145+0000","last_fullsized":"2024-04-13T03:15:21.247145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:57:34.325918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246552+0000","last_change":"2024-04-13T03:15:18.966351+0000","last_active":"2024-04-13T03:15:21.246552+0000","last_peered":"2024-04-13T03:15:21.246552+0000","last_clean":"2024-04-13T03:15:21.246552+0000","last_became_active":"2024-04-13T03:15:18.966048+0000","last_became_peered":"2024-04-13T03:15:18.966048+0000","last_unstale":"2024-04-13T03:15:21.246552+0000","last_undegraded":"2024-04-13T03:15:21.246552+0000","last_fullsized":"2024-04-13T03:15:21.246552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:58:10.625662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964831+0000","last_change":"2024-04-13T03:15:19.966143+0000","last_active":"2024-04-13T03:15:20.964831+0000","last_peered":"2024-04-13T03:15:20.964831+0000","last_clean":"2024-04-13T03:15:20.964831+0000","last_became_active":"2024-04-13T03:15:19.963355+0000","last_became_peered":"2024-04-13T03:15:19.963355+0000","last_unstale":"2024-04-13T03:15:20.964831+0000","last_undegraded":"2024-04-13T03:15:20.964831+0000","last_fullsized":"2024-04-13T03:15:20.964831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:26:37.985001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245309+0000","last_change":"2024-04-13T03:15:18.966450+0000","last_active":"2024-04-13T03:15:21.245309+0000","last_peered":"2024-04-13T03:15:21.245309+0000","last_clean":"2024-04-13T03:15:21.245309+0000","last_became_active":"2024-04-13T03:15:18.966150+0000","last_became_peered":"2024-04-13T03:15:18.966150+0000","last_unstale":"2024-04-13T03:15:21.245309+0000","last_undegraded":"2024-04-13T03:15:21.245309+0000","last_fullsized":"2024-04-13T03:15:21.245309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:42:35.734922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966644+0000","last_change":"2024-04-13T03:15:19.964081+0000","last_active":"2024-04-13T03:15:20.966644+0000","last_peered":"2024-04-13T03:15:20.966644+0000","last_clean":"2024-04-13T03:15:20.966644+0000","last_became_active":"2024-04-13T03:15:19.963798+0000","last_became_peered":"2024-04-13T03:15:19.963798+0000","last_unstale":"2024-04-13T03:15:20.966644+0000","last_undegraded":"2024-04-13T03:15:20.966644+0000","last_fullsized":"2024-04-13T03:15:20.966644+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:34.306269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247537+0000","last_change":"2024-04-13T03:15:18.975811+0000","last_active":"2024-04-13T03:15:21.247537+0000","last_peered":"2024-04-13T03:15:21.247537+0000","last_clean":"2024-04-13T03:15:21.247537+0000","last_became_active":"2024-04-13T03:15:18.975442+0000","last_became_peered":"2024-04-13T03:15:18.975442+0000","last_unstale":"2024-04-13T03:15:21.247537+0000","last_undegraded":"2024-04-13T03:15:21.247537+0000","last_fullsized":"2024-04-13T03:15:21.247537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:48:36.669495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965143+0000","last_change":"2024-04-13T03:15:19.979056+0000","last_active":"2024-04-13T03:15:20.965143+0000","last_peered":"2024-04-13T03:15:20.965143+0000","last_clean":"2024-04-13T03:15:20.965143+0000","last_became_active":"2024-04-13T03:15:19.978768+0000","last_became_peered":"2024-04-13T03:15:19.978768+0000","last_unstale":"2024-04-13T03:15:20.965143+0000","last_undegraded":"2024-04-13T03:15:20.965143+0000","last_fullsized":"2024-04-13T03:15:20.965143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:10.108929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247159+0000","last_change":"2024-04-13T03:15:18.974236+0000","last_active":"2024-04-13T03:15:21.247159+0000","last_peered":"2024-04-13T03:15:21.247159+0000","last_clean":"2024-04-13T03:15:21.247159+0000","last_became_active":"2024-04-13T03:15:18.973950+0000","last_became_peered":"2024-04-13T03:15:18.973950+0000","last_unstale":"2024-04-13T03:15:21.247159+0000","last_undegraded":"2024-04-13T03:15:21.247159+0000","last_fullsized":"2024-04-13T03:15:21.247159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:06.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965093+0000","last_change":"2024-04-13T03:15:19.965920+0000","last_active":"2024-04-13T03:15:20.965093+0000","last_peered":"2024-04-13T03:15:20.965093+0000","last_clean":"2024-04-13T03:15:20.965093+0000","last_became_active":"2024-04-13T03:15:19.965475+0000","last_became_peered":"2024-04-13T03:15:19.965475+0000","last_unstale":"2024-04-13T03:15:20.965093+0000","last_undegraded":"2024-04-13T03:15:20.965093+0000","last_fullsized":"2024-04-13T03:15:20.965093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:53:13.000921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247446+0000","last_change":"2024-04-13T03:15:18.963844+0000","last_active":"2024-04-13T03:15:21.247446+0000","last_peered":"2024-04-13T03:15:21.247446+0000","last_clean":"2024-04-13T03:15:21.247446+0000","last_became_active":"2024-04-13T03:15:18.963512+0000","last_became_peered":"2024-04-13T03:15:18.963512+0000","last_unstale":"2024-04-13T03:15:21.247446+0000","last_undegraded":"2024-04-13T03:15:21.247446+0000","last_fullsized":"2024-04-13T03:15:21.247446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:03:21.856679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246705+0000","last_change":"2024-04-13T03:15:19.966815+0000","last_active":"2024-04-13T03:15:21.246705+0000","last_peered":"2024-04-13T03:15:21.246705+0000","last_clean":"2024-04-13T03:15:21.246705+0000","last_became_active":"2024-04-13T03:15:19.966553+0000","last_became_peered":"2024-04-13T03:15:19.966553+0000","last_unstale":"2024-04-13T03:15:21.246705+0000","last_undegraded":"2024-04-13T03:15:21.246705+0000","last_fullsized":"2024-04-13T03:15:21.246705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:52:42.417633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966988+0000","last_change":"2024-04-13T03:15:18.968687+0000","last_active":"2024-04-13T03:15:20.966988+0000","last_peered":"2024-04-13T03:15:20.966988+0000","last_clean":"2024-04-13T03:15:20.966988+0000","last_became_active":"2024-04-13T03:15:18.968379+0000","last_became_peered":"2024-04-13T03:15:18.968379+0000","last_unstale":"2024-04-13T03:15:20.966988+0000","last_undegraded":"2024-04-13T03:15:20.966988+0000","last_fullsized":"2024-04-13T03:15:20.966988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:34:22.836564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245303+0000","last_change":"2024-04-13T03:15:19.975504+0000","last_active":"2024-04-13T03:15:21.245303+0000","last_peered":"2024-04-13T03:15:21.245303+0000","last_clean":"2024-04-13T03:15:21.245303+0000","last_became_active":"2024-04-13T03:15:19.975260+0000","last_became_peered":"2024-04-13T03:15:19.975260+0000","last_unstale":"2024-04-13T03:15:21.245303+0000","last_undegraded":"2024-04-13T03:15:21.245303+0000","last_fullsized":"2024-04-13T03:15:21.245303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:39:05.606358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965154+0000","last_change":"2024-04-13T03:15:18.968767+0000","last_active":"2024-04-13T03:15:20.965154+0000","last_peered":"2024-04-13T03:15:20.965154+0000","last_clean":"2024-04-13T03:15:20.965154+0000","last_became_active":"2024-04-13T03:15:18.968495+0000","last_became_peered":"2024-04-13T03:15:18.968495+0000","last_unstale":"2024-04-13T03:15:20.965154+0000","last_undegraded":"2024-04-13T03:15:20.965154+0000","last_fullsized":"2024-04-13T03:15:20.965154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:02:54.251065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.245596+0000","last_change":"2024-04-13T03:15:19.971844+0000","last_active":"2024-04-13T03:15:21.245596+0000","last_peered":"2024-04-13T03:15:21.245596+0000","last_clean":"2024-04-13T03:15:21.245596+0000","last_became_active":"2024-04-13T03:15:19.971592+0000","last_became_peered":"2024-04-13T03:15:19.971592+0000","last_unstale":"2024-04-13T03:15:21.245596+0000","last_undegraded":"2024-04-13T03:15:21.245596+0000","last_fullsized":"2024-04-13T03:15:21.245596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:05.752317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246885+0000","last_change":"2024-04-13T03:15:18.965952+0000","last_active":"2024-04-13T03:15:21.246885+0000","last_peered":"2024-04-13T03:15:21.246885+0000","last_clean":"2024-04-13T03:15:21.246885+0000","last_became_active":"2024-04-13T03:15:18.965639+0000","last_became_peered":"2024-04-13T03:15:18.965639+0000","last_unstale":"2024-04-13T03:15:21.246885+0000","last_undegraded":"2024-04-13T03:15:21.246885+0000","last_fullsized":"2024-04-13T03:15:21.246885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:00.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966754+0000","last_change":"2024-04-13T03:15:19.975072+0000","last_active":"2024-04-13T03:15:20.966754+0000","last_peered":"2024-04-13T03:15:20.966754+0000","last_clean":"2024-04-13T03:15:20.966754+0000","last_became_active":"2024-04-13T03:15:19.974781+0000","last_became_peered":"2024-04-13T03:15:19.974781+0000","last_unstale":"2024-04-13T03:15:20.966754+0000","last_undegraded":"2024-04-13T03:15:20.966754+0000","last_fullsized":"2024-04-13T03:15:20.966754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:59:00.328223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.966823+0000","last_change":"2024-04-13T03:15:18.973260+0000","last_active":"2024-04-13T03:15:20.966823+0000","last_peered":"2024-04-13T03:15:20.966823+0000","last_clean":"2024-04-13T03:15:20.966823+0000","last_became_active":"2024-04-13T03:15:18.972942+0000","last_became_peered":"2024-04-13T03:15:18.972942+0000","last_unstale":"2024-04-13T03:15:20.966823+0000","last_undegraded":"2024-04-13T03:15:20.966823+0000","last_fullsized":"2024-04-13T03:15:20.966823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:48:17.505348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.964972+0000","last_change":"2024-04-13T03:15:19.963057+0000","last_active":"2024-04-13T03:15:20.964972+0000","last_peered":"2024-04-13T03:15:20.964972+0000","last_clean":"2024-04-13T03:15:20.964972+0000","last_became_active":"2024-04-13T03:15:19.961927+0000","last_became_peered":"2024-04-13T03:15:19.961927+0000","last_unstale":"2024-04-13T03:15:20.964972+0000","last_undegraded":"2024-04-13T03:15:20.964972+0000","last_fullsized":"2024-04-13T03:15:20.964972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:10.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246430+0000","last_change":"2024-04-13T03:15:18.973612+0000","last_active":"2024-04-13T03:15:21.246430+0000","last_peered":"2024-04-13T03:15:21.246430+0000","last_clean":"2024-04-13T03:15:21.246430+0000","last_became_active":"2024-04-13T03:15:18.973369+0000","last_became_peered":"2024-04-13T03:15:18.973369+0000","last_unstale":"2024-04-13T03:15:21.246430+0000","last_undegraded":"2024-04-13T03:15:21.246430+0000","last_fullsized":"2024-04-13T03:15:21.246430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:11.727025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.247499+0000","last_change":"2024-04-13T03:15:19.970984+0000","last_active":"2024-04-13T03:15:21.247499+0000","last_peered":"2024-04-13T03:15:21.247499+0000","last_clean":"2024-04-13T03:15:21.247499+0000","last_became_active":"2024-04-13T03:15:19.970730+0000","last_became_peered":"2024-04-13T03:15:19.970730+0000","last_unstale":"2024-04-13T03:15:21.247499+0000","last_undegraded":"2024-04-13T03:15:21.247499+0000","last_fullsized":"2024-04-13T03:15:21.247499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:18.943875+0000","last_clean_scrub_stamp":"2024-04-13T03:15:18.943875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:25:07.869053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:20.965970+0000","last_change":"2024-04-13T03:15:18.973820+0000","last_active":"2024-04-13T03:15:20.965970+0000","last_peered":"2024-04-13T03:15:20.965970+0000","last_clean":"2024-04-13T03:15:20.965970+0000","last_became_active":"2024-04-13T03:15:18.973569+0000","last_became_peered":"2024-04-13T03:15:18.973569+0000","last_unstale":"2024-04-13T03:15:20.965970+0000","last_undegraded":"2024-04-13T03:15:20.965970+0000","last_fullsized":"2024-04-13T03:15:20.965970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:15:17.939922+0000","last_clean_scrub_stamp":"2024-04-13T03:15:17.939922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:24:48.011391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-13T03:15:21.246485+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:15:21.246485+0000","last_peered":"2024-04-13T03:15:21.246485+0000","last_clean":"2024-04-13T03:15:21.246485+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:15:21.246485+0000","last_undegraded":"2024-04-13T03:15:21.246485+0000","last_fullsized":"2024-04-13T03:15:21.246485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1107459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1142784,"data_stored":519251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":517195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":517035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":517105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":1109721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:15:23.319 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:15:23.320 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:15:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.463+0000 7fc914b8c640 1 Processor -- start 2024-04-13T03:15:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.463+0000 7fc914b8c640 1 -- start start 2024-04-13T03:15:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc914b8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9100704f0 0x7fc9100708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc914b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc914b8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc910070e10 con 0x7fc91015f080 2024-04-13T03:15:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc914b8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc910162f40 con 0x7fc9100704f0 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc914b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc91015fbf0 con 0x7fc9100cf2f0 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9100704f0 0x7fc9100708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90dd9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc91015f080 0x7fc91015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47128/0 (socket says 172.21.15.106:47128) 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9100704f0 0x7fc9100708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53746/0 (socket says 172.21.15.106:53746) 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90dd9b640 1 -- 172.21.15.106:0/2776546193 learned_addr learned my addr 172.21.15.106:0/2776546193 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90ed9d640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 -- 172.21.15.106:0/2776546193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 msgr2=0x7fc9100cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 -- 172.21.15.106:0/2776546193 >> v1:172.21.15.106:6789/0 conn(0x7fc91015f080 legacy=0x7fc91015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90e59c640 1 -- 172.21.15.106:0/2776546193 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91015fe40 con 0x7fc9100704f0 2024-04-13T03:15:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.464+0000 7fc90d59a640 1 -- 172.21.15.106:0/2776546193 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011984007 0 0) 0x7fc910070e10 con 0x7fc91015f080 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.465+0000 7fc90e59c640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9100704f0 0x7fc9100708d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc908009d90 tx=0x7fc90802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cdfd4ac22a05a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.465+0000 7fc90d59a640 1 -- 172.21.15.106:0/2776546193 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908039070 con 0x7fc9100704f0 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.465+0000 7fc90d59a640 1 -- 172.21.15.106:0/2776546193 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90802cc50 con 0x7fc9100704f0 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.465+0000 7fc90d59a640 1 -- 172.21.15.106:0/2776546193 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908034440 con 0x7fc9100704f0 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 -- 172.21.15.106:0/2776546193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9100704f0 msgr2=0x7fc9100708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9100704f0 0x7fc9100708d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc908009d90 tx=0x7fc90802bea0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 -- 172.21.15.106:0/2776546193 shutdown_connections 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 --2- 172.21.15.106:0/2776546193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9100704f0 0x7fc9100708d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 -- 172.21.15.106:0/2776546193 >> 172.21.15.106:0/2776546193 conn(0x7fc9100d13f0 msgr2=0x7fc9100d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 -- 172.21.15.106:0/2776546193 shutdown_connections 2024-04-13T03:15:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.466+0000 7fc914b8c640 1 -- 172.21.15.106:0/2776546193 wait complete. 2024-04-13T03:15:23.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.467+0000 7fc914b8c640 1 Processor -- start 2024-04-13T03:15:23.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.467+0000 7fc914b8c640 1 -- start start 2024-04-13T03:15:23.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9100704f0 0x7fc9100ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101628f0 con 0x7fc91015f080 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc910162610 con 0x7fc9100704f0 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc914b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc910162bd0 con 0x7fc9100cf2f0 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53750/0 (socket says 172.21.15.106:53750) 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 -- 172.21.15.106:0/1040116409 learned_addr learned my addr 172.21.15.106:0/1040116409 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90e59c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9100704f0 0x7fc9100ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90dd9b640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 -- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 msgr2=0x7fc9100ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 -- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9100704f0 msgr2=0x7fc9100ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9100704f0 0x7fc9100ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.468+0000 7fc90ed9d640 1 -- 172.21.15.106:0/1040116409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91006cdb0 con 0x7fc91015f080 2024-04-13T03:15:23.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc90ed9d640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e820 tx=0x7fc90000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900017070 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90000ee70 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91006d090 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000cb60 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.469+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91006d650 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.470+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc910069120 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.470+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc90001a070 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.470+0000 7fc8feffd640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.471+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc90008be50 con 0x7fc91015f080 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.471+0000 7fc90e59c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.471+0000 7fc90e59c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 0x7fc8f006edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc908036000 tx=0x7fc908030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.475+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc900059aa0 con 0x7fc91015f080 2024-04-13T03:15:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.717+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc910071330 con 0x7fc91015f080 2024-04-13T03:15:23.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.719+0000 7fc8feffd640 1 -- 172.21.15.106:0/1040116409 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fc90005dda0 con 0x7fc91015f080 2024-04-13T03:15:23.720 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:15:23.721 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:15:23.721 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:15:23.721 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:15:23.721 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:15:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 msgr2=0x7fc8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 0x7fc8f006edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc908036000 tx=0x7fc908030000 comp rx=0 tx=0).stop 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 msgr2=0x7fc91006c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e820 tx=0x7fc90000ec20 comp rx=0 tx=0).stop 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 shutdown_connections 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9100cf2f0 0x7fc9100ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc91015f080 0x7fc91006c160 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 --2- 172.21.15.106:0/1040116409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9100704f0 0x7fc9100ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 >> 172.21.15.106:0/1040116409 conn(0x7fc9100d13f0 msgr2=0x7fc9100b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 shutdown_connections 2024-04-13T03:15:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.722+0000 7fc914b8c640 1 -- 172.21.15.106:0/1040116409 wait complete. 2024-04-13T03:15:23.733 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:23.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.832+0000 7f0e78036640 1 Processor -- start 2024-04-13T03:15:23.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 -- start start 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e70160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e700b81f0 0x7f0e700ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700b54c0 con 0x7f0e700cd2f0 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0e70071320 con 0x7f0e700b81f0 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.833+0000 7f0e78036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0e70070d60 con 0x7f0e7015cb00 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e700b81f0 0x7f0e700ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0e700b81f0 0x7f0e700ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53764/0 (socket says 172.21.15.106:53764) 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 -- 172.21.15.106:0/2991014680 learned_addr learned my addr 172.21.15.106:0/2991014680 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e75dd2640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e70160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 -- 172.21.15.106:0/2991014680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 msgr2=0x7f0e70160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e70160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 -- 172.21.15.106:0/2991014680 >> v1:172.21.15.114:6789/0 conn(0x7f0e700cd2f0 legacy=0x7f0e7006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 -- 172.21.15.106:0/2991014680 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70071040 con 0x7f0e700b81f0 2024-04-13T03:15:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e75dd2640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e70160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:23.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.834+0000 7f0e755d1640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700b81f0 0x7f0e700ccdb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009870 tx=0x7f0e6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5565171bea0d6049 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e74dd0640 1 -- 172.21.15.106:0/2991014680 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60035020 con 0x7f0e700b81f0 2024-04-13T03:15:23.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e74dd0640 1 -- 172.21.15.106:0/2991014680 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e60031640 con 0x7f0e700b81f0 2024-04-13T03:15:23.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e74dd0640 1 -- 172.21.15.106:0/2991014680 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6002d040 con 0x7f0e700b81f0 2024-04-13T03:15:23.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 -- 172.21.15.106:0/2991014680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700b81f0 msgr2=0x7f0e700ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700b81f0 0x7f0e700ccdb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009870 tx=0x7f0e6002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 -- 172.21.15.106:0/2991014680 shutdown_connections 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700b81f0 0x7f0e700ccdb0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 --2- 172.21.15.106:0/2991014680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e70160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 -- 172.21.15.106:0/2991014680 >> 172.21.15.106:0/2991014680 conn(0x7f0e700d2530 msgr2=0x7f0e700d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 -- 172.21.15.106:0/2991014680 shutdown_connections 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.835+0000 7f0e78036640 1 -- 172.21.15.106:0/2991014680 wait complete. 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.836+0000 7f0e78036640 1 Processor -- start 2024-04-13T03:15:23.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.836+0000 7f0e78036640 1 -- start start 2024-04-13T03:15:23.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e700b81f0 0x7f0e700c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e700cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700d1ce0 con 0x7f0e700cd2f0 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700d07b0 con 0x7f0e700b81f0 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e78036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700d1e60 con 0x7f0e7015cb00 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e755d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e755d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53768/0 (socket says 172.21.15.106:53768) 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e755d1640 1 -- 172.21.15.106:0/988408485 learned_addr learned my addr 172.21.15.106:0/988408485 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e75dd2640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e700b81f0 0x7f0e700c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.837+0000 7f0e765d3640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e700cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 -- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 msgr2=0x7f0e700cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e700cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 -- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e700b81f0 msgr2=0x7f0e700c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:23.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e700b81f0 0x7f0e700c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 -- 172.21.15.106:0/988408485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7015d7e0 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e755d1640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60040070 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e60039bf0 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7015da50 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60035040 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.838+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7006eb10 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.839+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e701f5550 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.839+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e6004b060 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.840+0000 7f0e667fc640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 0x7f0e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.840+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f0e600ad5f0 con 0x7f0e700cd2f0 2024-04-13T03:15:23.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.840+0000 7f0e75dd2640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 0x7f0e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:23.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.840+0000 7f0e75dd2640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 0x7f0e5406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ab60 tx=0x7f0e68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:23.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:23.844+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e701f5550 con 0x7f0e700cd2f0 2024-04-13T03:15:24.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.056+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e70068a10 con 0x7f0e700cd2f0 2024-04-13T03:15:24.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.056+0000 7f0e667fc640 1 -- 172.21.15.106:0/988408485 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5195 (secure 0 0 0) 0x7f0e700b54c0 con 0x7f0e700cd2f0 2024-04-13T03:15:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:24.058 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:21.352812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/3360662124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3360662124},{"type":"v1","addr":"172.21.15.106:6837","nonce":3360662124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-13T03:15:24.058 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 51 2024-04-13T03:15:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 msgr2=0x7f0e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 0x7f0e5406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ab60 tx=0x7f0e68008040 comp rx=0 tx=0).stop 2024-04-13T03:15:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 msgr2=0x7f0e700c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e600046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 shutdown_connections 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0e5406c8f0 0x7f0e5406edb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0e7015cb00 0x7f0e700cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e700cd2f0 0x7f0e700c9b40 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 --2- 172.21.15.106:0/988408485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e700b81f0 0x7f0e700c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.058+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 >> 172.21.15.106:0/988408485 conn(0x7f0e700d2530 msgr2=0x7f0e700d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.059+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 shutdown_connections 2024-04-13T03:15:24.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.059+0000 7f0e78036640 1 -- 172.21.15.106:0/988408485 wait complete. 2024-04-13T03:15:24.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:15:21.352812+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/3360662124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 3360662124}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3360662124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7518, 'qdb_cluster': [7518]} 2024-04-13T03:15:24.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:15:24.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-13T03:15:24.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.576+0000 7f83b7258640 1 Processor -- start 2024-04-13T03:15:24.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.576+0000 7f83b7258640 1 -- start start 2024-04-13T03:15:24.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83b7258640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83b00cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83b7258640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b006d220 con 0x7f83b00b47e0 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83b7258640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b00b3740 con 0x7f83b0145d20 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83b7258640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83b00b3a20 con 0x7f83b00b4c20 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83affff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f83b00b47e0 0x7f83b015cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47140/0 (socket says 172.21.15.106:47140) 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83affff640 1 -- 172.21.15.106:0/2784967625 learned_addr learned my addr 172.21.15.106:0/2784967625 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006801182 0 0) 0x7f83b006d220 con 0x7f83b00b47e0 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8398003520 con 0x7f83b00b47e0 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.577+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83b00cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 901550628 0 0) 0x7f8398003520 con 0x7f83b00b47e0 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 msgr2=0x7f83b00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83b00cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 >> v1:172.21.15.114:6789/0 conn(0x7f83b0145d20 legacy=0x7f83b006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b00b3d00 con 0x7f83b00b47e0 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f83a4001670 con 0x7f83b00b47e0 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 >> v1:172.21.15.106:6789/0 conn(0x7f83b00b47e0 legacy=0x7f83b015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8398003f20 0x7f8398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8398004d40 0x7f83980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00b3a20 con 0x7f8398004d40 2024-04-13T03:15:24.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f839800ba20 con 0x7f8398003f20 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f839800bc20 con 0x7f83b00b4c20 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8398004d40 0x7f83980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83affff640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8398003f20 0x7f8398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 -- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 msgr2=0x7f83980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b00b4c20 0x7f83980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 -- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8398003f20 msgr2=0x7f8398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8398003f20 0x7f8398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.578+0000 7f83b57f5640 1 -- 172.21.15.106:0/2784967625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8398003520 con 0x7f8398004d40 2024-04-13T03:15:24.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.579+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8398004d40 0x7f83980051c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009fc0 tx=0x7f8398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d8109e78f8acee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.579+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8039070 con 0x7f8398004d40 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.579+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a802cc20 con 0x7f8398004d40 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.579+0000 7f83af7fe640 1 -- 172.21.15.106:0/2784967625 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8009c10 con 0x7f8398004d40 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 -- 172.21.15.106:0/2784967625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8398004d40 msgr2=0x7f83980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 --2- 172.21.15.106:0/2784967625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8398004d40 0x7f83980051c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009fc0 tx=0x7f8398005830 comp rx=0 tx=0).stop 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b57f5640 1 -- 172.21.15.106:0/2784967625 reap_dead start 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 -- 172.21.15.106:0/2784967625 shutdown_connections 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 -- 172.21.15.106:0/2784967625 >> 172.21.15.106:0/2784967625 conn(0x7f83b00b2600 msgr2=0x7f83b00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 -- 172.21.15.106:0/2784967625 shutdown_connections 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.580+0000 7f83b7258640 1 -- 172.21.15.106:0/2784967625 wait complete. 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.581+0000 7f83b7258640 1 Processor -- start 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.581+0000 7f83b7258640 1 -- start start 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b015ebd0 0x7f83b01f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b01f3780 0x7f83b013cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0146100 con 0x7f83b0159ef0 2024-04-13T03:15:24.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00b42c0 con 0x7f83b01f3780 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83b0146280 con 0x7f83b015ebd0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53772/0 (socket says 172.21.15.106:53772) 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 -- 172.21.15.106:0/3320050313 learned_addr learned my addr 172.21.15.106:0/3320050313 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b57f5640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b01f3780 0x7f83b013cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83affff640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b015ebd0 0x7f83b01f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 -- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b015ebd0 msgr2=0x7f83b01f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b015ebd0 0x7f83b01f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 -- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b01f3780 msgr2=0x7f83b013cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b01f3780 0x7f83b013cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 -- 172.21.15.106:0/3320050313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b015f030 con 0x7f83b0159ef0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f83a000f850 tx=0x7f83a000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83ad7fa640 1 -- 172.21.15.106:0/3320050313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0018020 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83ad7fa640 1 -- 172.21.15.106:0/3320050313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a000d8e0 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b015f2e0 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83ad7fa640 1 -- 172.21.15.106:0/3320050313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a000dcb0 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.582+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b015f8a0 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.584+0000 7f83ad7fa640 1 -- 172.21.15.106:0/3320050313 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83a001b020 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.584+0000 7f83ad7fa640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 0x7f839c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.585+0000 7f83affff640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 0x7f839c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.585+0000 7f83ad7fa640 1 -- 172.21.15.106:0/3320050313 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f83a008c1f0 con 0x7f83b0159ef0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.585+0000 7f83affff640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 0x7f839c06ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f83b00cd650 tx=0x7f83a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.585+0000 7f83ba046740 1 Processor -- start 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.585+0000 7f83ba046740 1 -- start start 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb704c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb704f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb7042c990 con 0x55eb704c22c0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eb703d3620 con 0x55eb704eede0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55eb70428950 con 0x55eb704ef1c0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53784/0 (socket says 172.21.15.106:53784) 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 -- 172.21.15.106:0/1561514342 learned_addr learned my addr 172.21.15.106:0/1561514342 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83affff640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb704f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb704c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 -- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 msgr2=0x55eb704f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb704f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 -- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 msgr2=0x55eb704c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb704c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b57f5640 1 -- 172.21.15.106:0/1561514342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7013f2d0 con 0x55eb704c22c0 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83affff640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb704f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.586+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb704c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83b57f5640 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f83a800e230 tx=0x7f83a802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b254465eac654fb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83967fc640 1 -- 172.21.15.106:0/1561514342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8043020 con 0x55eb704c22c0 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83967fc640 1 -- 172.21.15.106:0/1561514342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a803a040 con 0x55eb704c22c0 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83967fc640 1 -- 172.21.15.106:0/1561514342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8039070 con 0x55eb704c22c0 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 -- 172.21.15.106:0/1561514342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 msgr2=0x55eb704f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f83a800e230 tx=0x7f83a802e070 comp rx=0 tx=0).stop 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 -- 172.21.15.106:0/1561514342 shutdown_connections 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb704f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704c22c0 0x55eb704f20b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 --2- 172.21.15.106:0/1561514342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb704c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 -- 172.21.15.106:0/1561514342 >> 172.21.15.106:0/1561514342 conn(0x55eb704c3ff0 msgr2=0x55eb7050cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 -- 172.21.15.106:0/1561514342 shutdown_connections 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.587+0000 7f83ba046740 1 -- 172.21.15.106:0/1561514342 wait complete. 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.588+0000 7f83ba046740 1 -- start start 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.588+0000 7f83ba046740 1 Processor -- start 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb70507330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb704c6590 con 0x55eb704eede0 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eb703e69e0 con 0x55eb704ee280 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55eb704c5980 con 0x55eb704ef1c0 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83ba046740 10 client.0 _ll_register_callbacks cb 0x55eb70422220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53800/0 (socket says 172.21.15.106:53800) 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 -- 172.21.15.106:0/498773146 learned_addr learned my addr 172.21.15.106:0/498773146 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83affff640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb70507330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b57f5640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 -- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 msgr2=0x55eb70507330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb70507330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 -- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 msgr2=0x55eb704f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 -- 172.21.15.106:0/498773146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb702096b0 con 0x55eb704eede0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f83a0059a90 tx=0x7f83a005a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb51dafe08bc14c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.589+0000 7f838f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00722d0 con 0x55eb704eede0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f83a00591e0 con 0x55eb704eede0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a803a040 con 0x55eb704eede0 2024-04-13T03:15:24.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f83a0062970 con 0x55eb704eede0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 10 client.7665 fetch_fsmap learned FSMap version 51 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55eb7051d7d0 con 0x55eb704eede0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a803a040 con 0x55eb704eede0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f83a0060340 con 0x55eb704eede0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 10 client.7665 fetch_fsmap finished waiting for FSMap version 51 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 10 client.7665 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 20 client.7665 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb705042b0 0x55eb70520b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 4 client.7665 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 --> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7043a3b0 con 0x55eb705042b0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83affff640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb705042b0 0x55eb70520b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f838f7fe640 10 client.7665 ms_handle_connect on v2:172.21.15.106:6835/3360662124 2024-04-13T03:15:24.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.590+0000 7f83affff640 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb705042b0 0x55eb70520b20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cacf888cd2e51ab server_cookie=30c507a66bdf1ef4 in_seq=0 out_seq=0 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f838f7fe640 1 -- 172.21.15.106:0/498773146 <== mds.0 v2:172.21.15.106:6835/3360662124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb7043a3b0 con 0x55eb705042b0 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f838f7fe640 10 client.7665 handle_command_reply: tid=0 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 client.7665 shutdown 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 msgr2=0x55eb7051cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f83a0059a90 tx=0x7f83a005a490 comp rx=0 tx=0).stop 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 shutdown_connections 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb705042b0 0x55eb70520b20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ef1c0 0x55eb70507330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f838f7fe640 0 client.7665 ms_handle_reset on v2:172.21.15.106:6835/3360662124 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 --2- 172.21.15.106:0/498773146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704eede0 0x55eb7051cca0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 >> 172.21.15.106:0/498773146 conn(0x55eb704c22c0 msgr2=0x55eb704f1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 shutdown_connections 2024-04-13T03:15:24.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 1 -- 172.21.15.106:0/498773146 wait complete. 2024-04-13T03:15:24.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.591+0000 7f83ba046740 20 client.7665 trim_cache size 0 max 16384 2024-04-13T03:15:24.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.609+0000 7f83ba046740 1 Processor -- start 2024-04-13T03:15:24.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.609+0000 7f83ba046740 1 -- start start 2024-04-13T03:15:24.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 0x55eb7050ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01fedd0 con 0x55eb704ef1c0 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01fe1d0 con 0x55eb704ee280 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f83b01fe780 con 0x55eb704eede0 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53808/0 (socket says 172.21.15.106:53808) 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 -- 172.21.15.106:0/2377448743 learned_addr learned my addr 172.21.15.106:0/2377448743 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83affff640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 0x55eb7050ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 -- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 msgr2=0x55eb7050ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 0x55eb7050ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 -- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 msgr2=0x55eb704c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 -- 172.21.15.106:0/2377448743 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb704ecec0 con 0x55eb704ef1c0 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83affff640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 0x55eb7050ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f83a8040000 tx=0x7f83a8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10bd5071b71e200 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f8395ffb640 1 -- 172.21.15.106:0/2377448743 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a804e070 con 0x55eb704ef1c0 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f8395ffb640 1 -- 172.21.15.106:0/2377448743 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8003d70 con 0x55eb704ef1c0 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.610+0000 7f8395ffb640 1 -- 172.21.15.106:0/2377448743 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8043030 con 0x55eb704ef1c0 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 -- 172.21.15.106:0/2377448743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 msgr2=0x55eb704c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f83a8040000 tx=0x7f83a8030000 comp rx=0 tx=0).stop 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 -- 172.21.15.106:0/2377448743 shutdown_connections 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704eede0 0x55eb7050ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb704c75a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 --2- 172.21.15.106:0/2377448743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704ee280 0x55eb704c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 -- 172.21.15.106:0/2377448743 >> 172.21.15.106:0/2377448743 conn(0x55eb704c22c0 msgr2=0x55eb704f1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 -- 172.21.15.106:0/2377448743 shutdown_connections 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.611+0000 7f83ba046740 1 -- 172.21.15.106:0/2377448743 wait complete. 2024-04-13T03:15:24.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 -- start start 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 Processor -- start 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 0x55eb705074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb7050e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb7051dfa0 con 0x55eb704ef1c0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eb7051e1c0 con 0x55eb704eede0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.612+0000 7f83ba046740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55eb705046a0 con 0x55eb704ee280 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83ba046740 10 client.0 _ll_register_callbacks cb 0x7f83a0059b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53812/0 (socket says 172.21.15.106:53812) 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 -- 172.21.15.106:0/2458940094 learned_addr learned my addr 172.21.15.106:0/2458940094 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83affff640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb7050e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 0x55eb705074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 -- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 msgr2=0x55eb705074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 0x55eb705074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 -- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 msgr2=0x55eb7050e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb7050e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 -- 172.21.15.106:0/2458940094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb705302d0 con 0x55eb704ef1c0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 0x55eb705074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83b57f5640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f83a8002230 tx=0x7f83a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723cba9199081ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f838f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a804e040 con 0x55eb704ef1c0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f83a802c960 con 0x55eb704ef1c0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a8003d70 con 0x55eb704ef1c0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f83a8062050 con 0x55eb704ef1c0 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83ba046740 10 client.7671 fetch_fsmap learned FSMap version 51 2024-04-13T03:15:24.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55eb7056e2b0 con 0x55eb704ef1c0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.613+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8003d70 con 0x55eb704ef1c0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f83a8065020 con 0x55eb704ef1c0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 10 client.7671 fetch_fsmap finished waiting for FSMap version 51 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 10 client.7671 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 20 client.7671 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb7056d980 0x55eb7051c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 4 client.7671 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 --> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] -- command(tid 0: {"prefix": "status"}) -- 0x55eb70434220 con 0x55eb7056d980 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb7056d980 0x55eb7051c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f838f7fe640 10 client.7671 ms_handle_connect on v2:172.21.15.106:6835/3360662124 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83b4ff4640 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb7056d980 0x55eb7051c030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3802ce85a0b6e8e server_cookie=e5be483ec7fc9b5c in_seq=0 out_seq=0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f838f7fe640 1 -- 172.21.15.106:0/2458940094 <== mds.0 v2:172.21.15.106:6835/3360662124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55eb70434220 con 0x55eb7056d980 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f838f7fe640 10 client.7671 handle_command_reply: tid=0 2024-04-13T03:15:24.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 client.7671 shutdown 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 msgr2=0x55eb7050e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f83a8002230 tx=0x7f83a8004ab0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 shutdown_connections 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:6835/3360662124,v1:172.21.15.106:6837/3360662124] conn(0x55eb7056d980 0x55eb7051c030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb704ef1c0 0x55eb7050e620 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f838f7fe640 0 client.7671 ms_handle_reset on v2:172.21.15.106:6835/3360662124 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb704eede0 0x55eb7050e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 --2- 172.21.15.106:0/2458940094 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55eb704ee280 0x55eb705074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.614+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 >> 172.21.15.106:0/2458940094 conn(0x55eb704c22c0 msgr2=0x55eb7051f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 shutdown_connections 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83ba046740 1 -- 172.21.15.106:0/2458940094 wait complete. 2024-04-13T03:15:24.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83ba046740 20 client.7671 trim_cache size 0 max 16384 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 7518, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.6650168079999998, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 51, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 91, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 91, 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 10.01716062 2024-04-13T03:15:24.624 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:24.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 msgr2=0x7f839c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 0x7f839c06ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f83b00cd650 tx=0x7f83a4009040 comp rx=0 tx=0).stop 2024-04-13T03:15:24.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 msgr2=0x7f83b0141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f83a000f850 tx=0x7f83a000fd10 comp rx=0 tx=0).stop 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 shutdown_connections 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f839c06c920 0x7f839c06ede0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f83b015ebd0 0x7f83b01f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b01f3780 0x7f83b013cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 --2- 172.21.15.106:0/3320050313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83b0159ef0 0x7f83b0141250 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 >> 172.21.15.106:0/3320050313 conn(0x7f83b00b2600 msgr2=0x7f83b00b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 shutdown_connections 2024-04-13T03:15:24.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.615+0000 7f83b7258640 1 -- 172.21.15.106:0/3320050313 wait complete. 2024-04-13T03:15:24.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:15:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.726+0000 7f7b1bbc8640 1 Processor -- start 2024-04-13T03:15:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.726+0000 7f7b1bbc8640 1 -- start start 2024-04-13T03:15:24.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1bbc8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 0x7f7b140d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1bbc8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b140d0c40 0x7f7b140ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1bbc8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1415cf90 con 0x7f7b1406dfa0 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1bbc8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7b14153bb0 con 0x7f7b140d0c40 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1bbc8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1415d270 con 0x7f7b14070e40 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1a165640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b140d0c40 0x7f7b140ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b19964640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7b1406dfa0 0x7f7b14070730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47142/0 (socket says 172.21.15.106:47142) 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b19964640 1 -- 172.21.15.106:0/2370506340 learned_addr learned my addr 172.21.15.106:0/2370506340 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b19163640 1 --2- 172.21.15.106:0/2370506340 >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 0x7f7b140d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1a165640 1 -- 172.21.15.106:0/2370506340 >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 msgr2=0x7f7b140d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1a165640 1 --2- 172.21.15.106:0/2370506340 >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 0x7f7b140d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1a165640 1 -- 172.21.15.106:0/2370506340 >> v1:172.21.15.106:6789/0 conn(0x7f7b1406dfa0 legacy=0x7f7b14070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b1a165640 1 -- 172.21.15.106:0/2370506340 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1415d550 con 0x7f7b140d0c40 2024-04-13T03:15:24.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.727+0000 7f7b19163640 1 --2- 172.21.15.106:0/2370506340 >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 0x7f7b140d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.728+0000 7f7b1a165640 1 --2- 172.21.15.106:0/2370506340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b140d0c40 0x7f7b140ce190 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7b140cfc60 tx=0x7f7b0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ff961703b296b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.728+0000 7f7b18962640 1 -- 172.21.15.106:0/2370506340 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0315a0 con 0x7f7b140d0c40 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.728+0000 7f7b18962640 1 -- 172.21.15.106:0/2370506340 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c037070 con 0x7f7b140d0c40 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.728+0000 7f7b18962640 1 -- 172.21.15.106:0/2370506340 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031960 con 0x7f7b140d0c40 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/2370506340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b140d0c40 msgr2=0x7f7b140ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/2370506340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b140d0c40 0x7f7b140ce190 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7b140cfc60 tx=0x7f7b0c02b840 comp rx=0 tx=0).stop 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/2370506340 shutdown_connections 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/2370506340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b140d0c40 0x7f7b140ce190 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/2370506340 >> v2:172.21.15.114:3300/0 conn(0x7f7b14070e40 0x7f7b140d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/2370506340 >> 172.21.15.106:0/2370506340 conn(0x7f7b1406f290 msgr2=0x7f7b1406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/2370506340 shutdown_connections 2024-04-13T03:15:24.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.729+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/2370506340 wait complete. 2024-04-13T03:15:24.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.730+0000 7f7b1bbc8640 1 Processor -- start 2024-04-13T03:15:24.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.730+0000 7f7b1bbc8640 1 -- start start 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b1406dfa0 0x7f7b141564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 0x7f7b14154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140d1070 con 0x7f7b14070e40 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1415cc40 con 0x7f7b140d0c40 2024-04-13T03:15:24.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1bbc8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7b140d11f0 con 0x7f7b1406dfa0 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53832/0 (socket says 172.21.15.106:53832) 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 -- 172.21.15.106:0/1099150276 learned_addr learned my addr 172.21.15.106:0/1099150276 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 -- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b1406dfa0 msgr2=0x7f7b141564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1a165640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 0x7f7b14154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19964640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b1406dfa0 0x7f7b141564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b1406dfa0 0x7f7b141564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 -- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 msgr2=0x7f7b14154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 0x7f7b14154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 -- 172.21.15.106:0/1099150276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b141e81f0 con 0x7f7b14070e40 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b1a165640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 0x7f7b14154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.731+0000 7f7b19163640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c820 tx=0x7f7b0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.732+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015070 con 0x7f7b14070e40 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.732+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04007a30 con 0x7f7b14070e40 2024-04-13T03:15:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.732+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b141e7c30 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.732+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04007e00 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.732+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b140b1120 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.733+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b14068a10 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.733+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b04018070 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.733+0000 7f7b0a7fc640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 0x7f7af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.734+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f7b0408bc20 con 0x7f7b14070e40 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.734+0000 7f7b19964640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 0x7f7af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.734+0000 7f7b19964640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 0x7f7af806edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7b1414eee0 tx=0x7f7b10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:24.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.738+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b04059f50 con 0x7f7b14070e40 2024-04-13T03:15:24.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.950+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7b140cd7a0 con 0x7f7b14070e40 2024-04-13T03:15:24.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.950+0000 7f7b0a7fc640 1 -- 172.21.15.106:0/1099150276 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f7b04059850 con 0x7f7b14070e40 2024-04-13T03:15:24.951 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 msgr2=0x7f7af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 0x7f7af806edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7b1414eee0 tx=0x7f7b10009040 comp rx=0 tx=0).stop 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 msgr2=0x7f7b14153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c820 tx=0x7f7b0400cc20 comp rx=0 tx=0).stop 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 shutdown_connections 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7af806c8f0 0x7f7af806edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b140d0c40 0x7f7b14154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b14070e40 0x7f7b14153d30 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 --2- 172.21.15.106:0/1099150276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b1406dfa0 0x7f7b141564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:24.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 >> 172.21.15.106:0/1099150276 conn(0x7f7b1406f290 msgr2=0x7f7b1415dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:24.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 shutdown_connections 2024-04-13T03:15:24.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:24.952+0000 7f7b1bbc8640 1 -- 172.21.15.106:0/1099150276 wait complete. 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:15:24.964 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:24.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:24.964 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:24.980 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42224sec preferred_lft 42224sec 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:15:24.981 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:24.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:24.983 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:15:25.046 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:15:25.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.048 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:15:25.103 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:15:25.104 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:15:25.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:15:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:25.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.238 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:15:25.252 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:15:25.253 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:15:25.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:25.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.253 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:15:25.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:25.319 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:25.374 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:25.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:25.375 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:25.429 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:15:25.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.429 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:25.514 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:25.514 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:25.521 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:25.521 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:25.521 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:15:25.521 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:15:25.521 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:15:25.522 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:15:25.522 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:15:25.522 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:15:25.522 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:15:25.522 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:15:25.524 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:15:25.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.524 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:25.550 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:15:25.625 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:15:25.636 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:15:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:15:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:25.652 INFO:teuthology.orchestra.run.smithi106.stdout:client7683 2024-04-13T03:15:25.652 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:25.652 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:25.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.708 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-13T03:15:25.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.767 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-13T03:15:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.824 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-13T03:15:25.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:25.879 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-13T03:15:25.940 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:15:25.940 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:15:25.940 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:15:25.940 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:15:25.941 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-13T03:15:25.941 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:15:26.046 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:15:26.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:26.054 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:26.055 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:26.110 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:26.110 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:26.166 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:15:26.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.304+0000 7f2a2902d640 1 Processor -- start 2024-04-13T03:15:26.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.304+0000 7f2a2902d640 1 -- start start 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2902d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a24071390 0x7f2a240cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2902d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2415f420 con 0x7f2a2415ed80 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2902d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2415f700 con 0x7f2a2406e8e0 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2902d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2406ecc0 con 0x7f2a24071390 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a24071390 0x7f2a240cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a1bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2a2415ed80 0x7f2a24070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47148/0 (socket says 172.21.15.106:47148) 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a1bfff640 1 -- 172.21.15.106:0/3409889615 learned_addr learned my addr 172.21.15.106:0/3409889615 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:26.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2259c640 1 -- 172.21.15.106:0/3409889615 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099667636 0 0) 0x7f2a2415f420 con 0x7f2a2415ed80 2024-04-13T03:15:26.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2259c640 1 -- 172.21.15.106:0/3409889615 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a10003520 con 0x7f2a2415ed80 2024-04-13T03:15:26.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2359e640 1 -- 172.21.15.106:0/3409889615 >> v1:172.21.15.114:6789/0 conn(0x7f2a2406e8e0 legacy=0x7f2a240cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2359e640 1 -- 172.21.15.106:0/3409889615 >> v1:172.21.15.106:6789/0 conn(0x7f2a2415ed80 legacy=0x7f2a24070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.305+0000 7f2a2359e640 1 -- 172.21.15.106:0/3409889615 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2415f9e0 con 0x7f2a24071390 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2359e640 1 --2- 172.21.15.106:0/3409889615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a24071390 0x7f2a240cd770 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009d90 tx=0x7f2a1c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda346613e39c0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2259c640 1 -- 172.21.15.106:0/3409889615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c039070 con 0x7f2a24071390 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2259c640 1 -- 172.21.15.106:0/3409889615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c02cd00 con 0x7f2a24071390 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2259c640 1 -- 172.21.15.106:0/3409889615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c034660 con 0x7f2a24071390 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 -- 172.21.15.106:0/3409889615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a24071390 msgr2=0x7f2a240cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3409889615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a24071390 0x7f2a240cd770 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009d90 tx=0x7f2a1c02bf50 comp rx=0 tx=0).stop 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 -- 172.21.15.106:0/3409889615 shutdown_connections 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3409889615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a24071390 0x7f2a240cd770 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 -- 172.21.15.106:0/3409889615 >> 172.21.15.106:0/3409889615 conn(0x7f2a2406f180 msgr2=0x7f2a2406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 -- 172.21.15.106:0/3409889615 shutdown_connections 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.306+0000 7f2a2902d640 1 -- 172.21.15.106:0/3409889615 wait complete. 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 Processor -- start 2024-04-13T03:15:26.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 -- start start 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a2406e8e0 0x7f2a24154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a24071390 0x7f2a24149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2a240707f0 con 0x7f2a2415ed80 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2415f1e0 con 0x7f2a24071390 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.308+0000 7f2a2902d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2a240cf790 con 0x7f2a2406e8e0 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53852/0 (socket says 172.21.15.106:53852) 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 -- 172.21.15.106:0/3464102220 learned_addr learned my addr 172.21.15.106:0/3464102220 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:26.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a1bfff640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a24071390 0x7f2a24149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a22d9d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a2406e8e0 0x7f2a24154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 -- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a2406e8e0 msgr2=0x7f2a24154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a2406e8e0 0x7f2a24154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 -- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a24071390 msgr2=0x7f2a24149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a24071390 0x7f2a24149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 -- 172.21.15.106:0/3464102220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a240b74b0 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2359e640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009d90 tx=0x7f2a1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c039050 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c045070 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c009450 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a241548b0 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.309+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a240b4ce0 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.310+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a24068a10 con 0x7f2a2415ed80 2024-04-13T03:15:26.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.311+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a1c03e850 con 0x7f2a2415ed80 2024-04-13T03:15:26.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.311+0000 7f2a1b7fe640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 0x7f2a0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.311+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f2a1c0ae8d0 con 0x7f2a2415ed80 2024-04-13T03:15:26.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.311+0000 7f2a22d9d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 0x7f2a0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.311+0000 7f2a22d9d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 0x7f2a0406edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a14001640 tx=0x7f2a14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.315+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a1c07cbe0 con 0x7f2a2415ed80 2024-04-13T03:15:26.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.490+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a24162a40 con 0x7f2a2415ed80 2024-04-13T03:15:26.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.491+0000 7f2a1b7fe640 1 -- 172.21.15.106:0/3464102220 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16921 (secure 0 0 0) 0x7f2a1c07c4e0 con 0x7f2a2415ed80 2024-04-13T03:15:26.492 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:26.492 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":91,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:15:20.959782+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:15:26.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 msgr2=0x7f2a0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 0x7f2a0406edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a14001640 tx=0x7f2a14009040 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 msgr2=0x7f2a240b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009d90 tx=0x7f2a1c004a40 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 shutdown_connections 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2a0406c8f0 0x7f2a0406edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a2415ed80 0x7f2a240b52c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a24071390 0x7f2a24149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 --2- 172.21.15.106:0/3464102220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2a2406e8e0 0x7f2a24154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 >> 172.21.15.106:0/3464102220 conn(0x7f2a2406f180 msgr2=0x7f2a24161700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 shutdown_connections 2024-04-13T03:15:26.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.493+0000 7f2a2902d640 1 -- 172.21.15.106:0/3464102220 wait complete. 2024-04-13T03:15:26.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:15:17.537136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:15:18.258394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:15:26.506 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:15:26.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.607+0000 7fc7c07b4640 1 Processor -- start 2024-04-13T03:15:26.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.607+0000 7fc7c07b4640 1 -- start start 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.608+0000 7fc7c07b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7b813f510 0x7fc7b815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.608+0000 7fc7c07b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7b8160400 0x7fc7b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.608+0000 7fc7c07b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b8160890 con 0x7fc7b813f8f0 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.608+0000 7fc7c07b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b8160b70 con 0x7fc7b813f510 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.608+0000 7fc7c07b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b8160e50 con 0x7fc7b8160400 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bed51640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc7b813f8f0 0x7fc7b80704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47160/0 (socket says 172.21.15.106:47160) 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bed51640 1 -- 172.21.15.106:0/1123960496 learned_addr learned my addr 172.21.15.106:0/1123960496 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7be550640 1 --2- 172.21.15.106:0/1123960496 >> v2:172.21.15.114:3300/0 conn(0x7fc7b813f510 0x7fc7b815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bdd4f640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7b8160400 0x7fc7b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940346205 0 0) 0x7fc7b8160890 con 0x7fc7b813f8f0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7a4003520 con 0x7fc7b813f8f0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2315500491 0 0) 0x7fc7a4003520 con 0x7fc7b813f8f0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7b8160400 msgr2=0x7fc7b806c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7b8160400 0x7fc7b806c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 >> v2:172.21.15.114:3300/0 conn(0x7fc7b813f510 msgr2=0x7fc7b815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 --2- 172.21.15.106:0/1123960496 >> v2:172.21.15.114:3300/0 conn(0x7fc7b813f510 0x7fc7b815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b8161130 con 0x7fc7b813f8f0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc7b00019f0 con 0x7fc7b813f8f0 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 >> v1:172.21.15.106:6789/0 conn(0x7fc7b813f8f0 legacy=0x7fc7b80704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a40039b0 0x7fc7a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a4004360 0x7fc7a4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a4004eb0 0x7fc7a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8160b70 con 0x7fc7a4004eb0 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8160e50 con 0x7fc7a4004360 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a400bf00 con 0x7fc7a40039b0 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bed51640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a4004eb0 0x7fc7a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7be550640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a40039b0 0x7fc7a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.609+0000 7fc7bdd4f640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a4004360 0x7fc7a4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 -- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a40039b0 msgr2=0x7fc7a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a40039b0 0x7fc7a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 -- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a4004360 msgr2=0x7fc7a4004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7a4004360 0x7fc7a4004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 -- 172.21.15.106:0/1123960496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a4003520 con 0x7fc7a4004eb0 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bed51640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a4004eb0 0x7fc7a400b9c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000a7b0 tx=0x7fc7a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406cf770621f1f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000a3d0 con 0x7fc7a4004eb0 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0034650 con 0x7fc7a4004eb0 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.610+0000 7fc7bd54e640 1 -- 172.21.15.106:0/1123960496 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0034a40 con 0x7fc7a4004eb0 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 -- 172.21.15.106:0/1123960496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a4004eb0 msgr2=0x7fc7a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/1123960496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7a4004eb0 0x7fc7a400b9c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000a7b0 tx=0x7fc7a4005830 comp rx=0 tx=0).stop 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7bed51640 1 -- 172.21.15.106:0/1123960496 reap_dead start 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 -- 172.21.15.106:0/1123960496 shutdown_connections 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 -- 172.21.15.106:0/1123960496 >> 172.21.15.106:0/1123960496 conn(0x7fc7b80d13f0 msgr2=0x7fc7b80d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 -- 172.21.15.106:0/1123960496 shutdown_connections 2024-04-13T03:15:26.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.611+0000 7fc7c07b4640 1 -- 172.21.15.106:0/1123960496 wait complete. 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.612+0000 7fc7c07b4640 1 Processor -- start 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.612+0000 7fc7c07b4640 1 -- start start 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a4004eb0 0x7fc7b814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7b80ca750 0x7fc7b80cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8070d80 con 0x7fc7b80c6250 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8070c00 con 0x7fc7b80ca750 2024-04-13T03:15:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7c07b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b806c290 con 0x7fc7a4004eb0 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53866/0 (socket says 172.21.15.106:53866) 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 -- 172.21.15.106:0/817095031 learned_addr learned my addr 172.21.15.106:0/817095031 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7be550640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a4004eb0 0x7fc7b814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bed51640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7b80ca750 0x7fc7b80cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 -- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a4004eb0 msgr2=0x7fc7b814cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a4004eb0 0x7fc7b814cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 -- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7b80ca750 msgr2=0x7fc7b80cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7b80ca750 0x7fc7b80cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.613+0000 7fc7bdd4f640 1 -- 172.21.15.106:0/817095031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b80c6810 con 0x7fc7b80c6250 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7bdd4f640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4011970 tx=0x7fc7b4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4019070 con 0x7fc7b80c6250 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4004240 con 0x7fc7b80c6250 2024-04-13T03:15:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400f820 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b80c6af0 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.614+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b80c70b0 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.615+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8069120 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.615+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7b401c070 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.616+0000 7fc7aeffd640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 0x7fc79c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.616+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc7b408c120 con 0x7fc7b80c6250 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.616+0000 7fc7be550640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 0x7fc79c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.616+0000 7fc7be550640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 0x7fc79c06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009bc0 tx=0x7fc7a8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.620+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7b4059cd0 con 0x7fc7b80c6250 2024-04-13T03:15:26.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.793+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7b806d760 con 0x7fc7b80c6250 2024-04-13T03:15:26.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.794+0000 7fc7aeffd640 1 -- 172.21.15.106:0/817095031 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fc7b405dfd0 con 0x7fc7b80c6250 2024-04-13T03:15:26.795 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 msgr2=0x7fc79c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 0x7fc79c06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009bc0 tx=0x7fc7a8009190 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 msgr2=0x7fc7b813ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4011970 tx=0x7fc7b4011d70 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 shutdown_connections 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc79c06c8f0 0x7fc79c06edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7b80ca750 0x7fc7b80cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7b80c6250 0x7fc7b813ecc0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 --2- 172.21.15.106:0/817095031 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7a4004eb0 0x7fc7b814cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 >> 172.21.15.106:0/817095031 conn(0x7fc7b80d13f0 msgr2=0x7fc7b80704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:26.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 shutdown_connections 2024-04-13T03:15:26.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.796+0000 7fc7c07b4640 1 -- 172.21.15.106:0/817095031 wait complete. 2024-04-13T03:15:26.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:15:26.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.910+0000 7f4df8c76640 1 Processor -- start 2024-04-13T03:15:26.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.910+0000 7f4df8c76640 1 -- start start 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df8c76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4df415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df8c76640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df40b5cf0 con 0x7f4df4070bb0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df8c76640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df40b5fd0 con 0x7f4df40b54a0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df8c76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4df40b62b0 con 0x7f4df40b1c30 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4df4070bb0 0x7f4df406d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47176/0 (socket says 172.21.15.106:47176) 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df2d9d640 1 -- 172.21.15.106:0/3607243374 learned_addr learned my addr 172.21.15.106:0/3607243374 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696131571 0 0) 0x7f4df40b5cf0 con 0x7f4df4070bb0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df259c640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4df415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.911+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ddc003520 con 0x7f4df4070bb0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 102886570 0 0) 0x7f4ddc003520 con 0x7f4df4070bb0 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 msgr2=0x7f4df415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4df415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 >> v1:172.21.15.114:6789/0 conn(0x7f4df40b54a0 legacy=0x7f4df40704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df40b6590 con 0x7f4df4070bb0 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4dec001670 con 0x7f4df4070bb0 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 >> v1:172.21.15.106:6789/0 conn(0x7f4df4070bb0 legacy=0x7f4df406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4ddc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc003f20 0x7f4ddc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ddc004d40 0x7f4ddc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df40b62b0 con 0x7f4ddc004d40 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00ba20 con 0x7f4ddc003f20 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc00bc20 con 0x7f4df40b1c30 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ddc004d40 0x7f4ddc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df259c640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4ddc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df1d9b640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc003f20 0x7f4ddc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 -- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 msgr2=0x7f4ddc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df40b1c30 0x7f4ddc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 -- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc003f20 msgr2=0x7f4ddc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc003f20 0x7f4ddc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.912+0000 7f4df2d9d640 1 -- 172.21.15.106:0/3607243374 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc003520 con 0x7f4ddc004d40 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.913+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ddc004d40 0x7f4ddc0051c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c240 tx=0x7f4ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421f26210d982d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.913+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007d10 con 0x7f4ddc004d40 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.913+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec030070 con 0x7f4ddc004d40 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.913+0000 7f4df159a640 1 -- 172.21.15.106:0/3607243374 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec005d70 con 0x7f4ddc004d40 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 -- 172.21.15.106:0/3607243374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ddc004d40 msgr2=0x7f4ddc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 --2- 172.21.15.106:0/3607243374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ddc004d40 0x7f4ddc0051c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c240 tx=0x7f4ddc005830 comp rx=0 tx=0).stop 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df2d9d640 1 -- 172.21.15.106:0/3607243374 reap_dead start 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 -- 172.21.15.106:0/3607243374 shutdown_connections 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 -- 172.21.15.106:0/3607243374 >> 172.21.15.106:0/3607243374 conn(0x7f4df4071870 msgr2=0x7f4df4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 -- 172.21.15.106:0/3607243374 shutdown_connections 2024-04-13T03:15:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.914+0000 7f4df8c76640 1 -- 172.21.15.106:0/3607243374 wait complete. 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.915+0000 7f4df8c76640 1 Processor -- start 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.915+0000 7f4df8c76640 1 -- start start 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc004d40 0x7f4df4140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df4151680 0x7f4df4152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df415efc0 con 0x7f4df40d04c0 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4df415ee40 con 0x7f4ddc004d40 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4df415f140 con 0x7f4df4151680 2024-04-13T03:15:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53888/0 (socket says 172.21.15.106:53888) 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 -- 172.21.15.106:0/625165834 learned_addr learned my addr 172.21.15.106:0/625165834 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df259c640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc004d40 0x7f4df4140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df1d9b640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df4151680 0x7f4df4152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 -- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df4151680 msgr2=0x7f4df4152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df4151680 0x7f4df4152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 -- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc004d40 msgr2=0x7f4df4140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc004d40 0x7f4df4140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 -- 172.21.15.106:0/625165834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df4153ba0 con 0x7f4df40d04c0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df2d9d640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4dec000c00 tx=0x7f4df40b6870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007d10 con 0x7f4df40d04c0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec004bf0 con 0x7f4df40d04c0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df4153e20 con 0x7f4df40d04c0 2024-04-13T03:15:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec001c50 con 0x7f4df40d04c0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.916+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41543e0 con 0x7f4df40d04c0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.917+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df4068a10 con 0x7f4df40d04c0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.918+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dec007070 con 0x7f4df40d04c0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.918+0000 7f4de2ffd640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 0x7f4dd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.918+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f4dec0ac9e0 con 0x7f4df40d04c0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.918+0000 7f4df259c640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 0x7f4dd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.919+0000 7f4df259c640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 0x7f4dd406ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4de4007520 tx=0x7f4de400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:26.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:26.923+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dec047060 con 0x7f4df40d04c0 2024-04-13T03:15:27.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.100+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4df40711f0 con 0x7f4df40d04c0 2024-04-13T03:15:27.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.101+0000 7f4de2ffd640 1 -- 172.21.15.106:0/625165834 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f4df4153ba0 con 0x7f4df40d04c0 2024-04-13T03:15:27.102 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 msgr2=0x7f4dd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 0x7f4dd406ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4de4007520 tx=0x7f4de400b040 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 msgr2=0x7f4df40d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4dec000c00 tx=0x7f4df40b6870 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 shutdown_connections 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4dd406c920 0x7f4dd406ede0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df4151680 0x7f4df4152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df40d04c0 0x7f4df40d0970 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 --2- 172.21.15.106:0/625165834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ddc004d40 0x7f4df4140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 >> 172.21.15.106:0/625165834 conn(0x7f4df4071870 msgr2=0x7f4df40704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 shutdown_connections 2024-04-13T03:15:27.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.103+0000 7f4df8c76640 1 -- 172.21.15.106:0/625165834 wait complete. 2024-04-13T03:15:27.115 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:15:27.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.217+0000 7effe2bf7640 1 Processor -- start 2024-04-13T03:15:27.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.217+0000 7effe2bf7640 1 -- start start 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.218+0000 7effe2bf7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 0x7effdc15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.218+0000 7effe2bf7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effdc06f200 0x7effdc06f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.218+0000 7effe2bf7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effdc15c200 con 0x7effdc15e830 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.218+0000 7effe2bf7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7effdc06fba0 con 0x7effdc06f200 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.218+0000 7effe2bf7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7effdc0b5090 con 0x7effdc0af800 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effe0993640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 0x7effdc15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effdbfff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effdc06f200 0x7effdc06f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effe1194640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7effdc15e830 0x7effdc15ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:34478/0 (socket says 172.21.15.106:34478) 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effe1194640 1 -- 172.21.15.106:0/3376573957 learned_addr learned my addr 172.21.15.106:0/3376573957 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effdbfff640 1 -- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 msgr2=0x7effdc15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effdbfff640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 0x7effdc15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effdbfff640 1 -- 172.21.15.106:0/3376573957 >> v1:172.21.15.114:6789/0 conn(0x7effdc15e830 legacy=0x7effdc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effdbfff640 1 -- 172.21.15.106:0/3376573957 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc0b3ff0 con 0x7effdc06f200 2024-04-13T03:15:27.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.219+0000 7effe0993640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 0x7effdc15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effdbfff640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc06f660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effd00097a0 tx=0x7effd002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc6016723f6cac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effdb7fe640 1 -- 172.21.15.106:0/3376573957 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0035020 con 0x7effdc06f200 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effdb7fe640 1 -- 172.21.15.106:0/3376573957 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0031830 con 0x7effdc06f200 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effdb7fe640 1 -- 172.21.15.106:0/3376573957 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0031c00 con 0x7effdc06f200 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 -- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 msgr2=0x7effdc06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc06f660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effd00097a0 tx=0x7effd002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 -- 172.21.15.106:0/3376573957 shutdown_connections 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc06f660 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 --2- 172.21.15.106:0/3376573957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc0af800 0x7effdc15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.220+0000 7effe2bf7640 1 -- 172.21.15.106:0/3376573957 >> 172.21.15.106:0/3376573957 conn(0x7effdc0b2600 msgr2=0x7effdc0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.221+0000 7effe2bf7640 1 -- 172.21.15.106:0/3376573957 shutdown_connections 2024-04-13T03:15:27.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.221+0000 7effe2bf7640 1 -- 172.21.15.106:0/3376573957 wait complete. 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 Processor -- start 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 -- start start 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc15e830 0x7effdc14be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effdc0b4890 con 0x7effdc0af800 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7effdc0b45b0 con 0x7effdc06f200 2024-04-13T03:15:27.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.222+0000 7effe2bf7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7effdc0b4b70 con 0x7effdc15e830 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53904/0 (socket says 172.21.15.106:53904) 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 -- 172.21.15.106:0/918107902 learned_addr learned my addr 172.21.15.106:0/918107902 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effe0993640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effe1194640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc15e830 0x7effdc14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 -- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc15e830 msgr2=0x7effdc14be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc15e830 0x7effdc14be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 -- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 msgr2=0x7effdc149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 -- 172.21.15.106:0/918107902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc14b820 con 0x7effdc0af800 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effe0993640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effdbfff640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7effd0034040 tx=0x7effd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0042070 con 0x7effdc0af800 2024-04-13T03:15:27.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd003cb20 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc14bb60 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0035040 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.223+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc14d5c0 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.224+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effdc068a10 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.225+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effd0038020 con 0x7effdc0af800 2024-04-13T03:15:27.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.225+0000 7effd97fa640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 0x7effc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.225+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7effd00af940 con 0x7effdc0af800 2024-04-13T03:15:27.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.225+0000 7effe0993640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 0x7effc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.226+0000 7effe0993640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 0x7effc806ede0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effdc06c840 tx=0x7effcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.230+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effdc14bb60 con 0x7effdc0af800 2024-04-13T03:15:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.401+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effdc146ec0 con 0x7effdc0af800 2024-04-13T03:15:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.402+0000 7effd97fa640 1 -- 172.21.15.106:0/918107902 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7effd007dbe0 con 0x7effdc0af800 2024-04-13T03:15:27.403 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 msgr2=0x7effc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 0x7effc806ede0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effdc06c840 tx=0x7effcc008040 comp rx=0 tx=0).stop 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 msgr2=0x7effdc15cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7effd0034040 tx=0x7effd00046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 shutdown_connections 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effc806c920 0x7effc806ede0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7effdc15e830 0x7effdc14be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effdc0af800 0x7effdc15cf40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 --2- 172.21.15.106:0/918107902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effdc06f200 0x7effdc149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 >> 172.21.15.106:0/918107902 conn(0x7effdc0b2600 msgr2=0x7effdc06c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 shutdown_connections 2024-04-13T03:15:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.404+0000 7effe2bf7640 1 -- 172.21.15.106:0/918107902 wait complete. 2024-04-13T03:15:27.416 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:15:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.516+0000 7f4757836640 1 Processor -- start 2024-04-13T03:15:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 -- start start 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f475015b4e0 0x7f475015fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f47500b5390 0x7f475006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47500b3690 con 0x7f47500b4730 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f47500b3970 con 0x7f475015b4e0 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f4757836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f47500b3c50 con 0x7f47500b5390 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.517+0000 7f47555d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f475015b4e0 0x7f475015fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f475015b4e0 0x7f475015fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53910/0 (socket says 172.21.15.106:53910) 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 -- 172.21.15.106:0/3400699110 learned_addr learned my addr 172.21.15.106:0/3400699110 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f4755dd3640 1 --2- 172.21.15.106:0/3400699110 >> v2:172.21.15.114:3300/0 conn(0x7f47500b5390 0x7f475006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 -- 172.21.15.106:0/3400699110 >> v2:172.21.15.114:3300/0 conn(0x7f47500b5390 msgr2=0x7f475006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 --2- 172.21.15.106:0/3400699110 >> v2:172.21.15.114:3300/0 conn(0x7f47500b5390 0x7f475006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 -- 172.21.15.106:0/3400699110 >> v1:172.21.15.114:6789/0 conn(0x7f47500b4730 legacy=0x7f47500b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 -- 172.21.15.106:0/3400699110 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500b3f30 con 0x7f475015b4e0 2024-04-13T03:15:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.518+0000 7f47555d2640 1 --2- 172.21.15.106:0/3400699110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f475015fb60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f47500cd9e0 tx=0x7f474c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7519cdd01142c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4743fff640 1 -- 172.21.15.106:0/3400699110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c034070 con 0x7f475015b4e0 2024-04-13T03:15:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4743fff640 1 -- 172.21.15.106:0/3400699110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c030860 con 0x7f475015b4e0 2024-04-13T03:15:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4743fff640 1 -- 172.21.15.106:0/3400699110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c030c30 con 0x7f475015b4e0 2024-04-13T03:15:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 -- 172.21.15.106:0/3400699110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 msgr2=0x7f475015fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 --2- 172.21.15.106:0/3400699110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f475015fb60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f47500cd9e0 tx=0x7f474c02b9a0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 -- 172.21.15.106:0/3400699110 shutdown_connections 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 --2- 172.21.15.106:0/3400699110 >> v2:172.21.15.114:3300/0 conn(0x7f47500b5390 0x7f475006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 --2- 172.21.15.106:0/3400699110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f475015fb60 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 -- 172.21.15.106:0/3400699110 >> 172.21.15.106:0/3400699110 conn(0x7f47500b2600 msgr2=0x7f47500d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 -- 172.21.15.106:0/3400699110 shutdown_connections 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.519+0000 7f4757836640 1 -- 172.21.15.106:0/3400699110 wait complete. 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.520+0000 7f4757836640 1 Processor -- start 2024-04-13T03:15:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.520+0000 7f4757836640 1 -- start start 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47500b4730 0x7f47500cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 0x7f4750145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f475006d0c0 con 0x7f475015b4e0 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f47500b32d0 con 0x7f47500b4730 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4757836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f47500ce420 con 0x7f47500b5390 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4755dd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4755dd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53916/0 (socket says 172.21.15.106:53916) 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4755dd3640 1 -- 172.21.15.106:0/2981253199 learned_addr learned my addr 172.21.15.106:0/2981253199 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f47555d2640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47500b4730 0x7f47500cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.521+0000 7f4754dd1640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 0x7f4750145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 -- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 msgr2=0x7f4750145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 0x7f4750145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 -- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47500b4730 msgr2=0x7f47500cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47500b4730 0x7f47500cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 -- 172.21.15.106:0/2981253199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500ce700 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4754dd1640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 0x7f4750145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4755dd3640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4748007e30 tx=0x7f474800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748014620 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4748019070 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47500ce9e0 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748014a00 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.522+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47500cefa0 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.523+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4750068a10 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.524+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4748014ba0 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.524+0000 7f4741ffb640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.524+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f474808c540 con 0x7f475015b4e0 2024-04-13T03:15:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.524+0000 7f47555d2640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.524+0000 7f47555d2640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f474c036000 tx=0x7f474c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.528+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f474805a7e0 con 0x7f475015b4e0 2024-04-13T03:15:27.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.719+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f475006d240 con 0x7f475015b4e0 2024-04-13T03:15:27.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.719+0000 7f4741ffb640 1 -- 172.21.15.106:0/2981253199 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f474805a0e0 con 0x7f475015b4e0 2024-04-13T03:15:27.720 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 msgr2=0x7f473406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f474c036000 tx=0x7f474c03b040 comp rx=0 tx=0).stop 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 msgr2=0x7f47500aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4748007e30 tx=0x7f474800e4b0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 shutdown_connections 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f47500b5390 0x7f4750145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475015b4e0 0x7f47500aee30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 --2- 172.21.15.106:0/2981253199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47500b4730 0x7f47500cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 >> 172.21.15.106:0/2981253199 conn(0x7f47500b2600 msgr2=0x7f47500afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 shutdown_connections 2024-04-13T03:15:27.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.721+0000 7f4757836640 1 -- 172.21.15.106:0/2981253199 wait complete. 2024-04-13T03:15:27.732 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:15:27.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:27.733 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:27.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:15:27.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:27.784 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:15:27.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:27.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.953+0000 7f9396d88640 1 Processor -- start 2024-04-13T03:15:27.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.953+0000 7f9396d88640 1 -- start start 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9396d88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 0x7f9390160c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9396d88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93900cf5a0 0x7f93900cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9396d88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9390070470 con 0x7f9390070090 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9396d88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f93900ceb50 con 0x7f93900cf5a0 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9396d88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f93900cf110 con 0x7f93900b0390 2024-04-13T03:15:27.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9395325640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93900cf5a0 0x7f93900cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9395325640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93900cf5a0 0x7f93900cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53922/0 (socket says 172.21.15.106:53922) 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9395325640 1 -- 172.21.15.106:0/817056612 learned_addr learned my addr 172.21.15.106:0/817056612 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f938ffff640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 0x7f9390160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9395325640 1 -- 172.21.15.106:0/817056612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 msgr2=0x7f9390160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.954+0000 7f9395325640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 0x7f9390160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f9395325640 1 -- 172.21.15.106:0/817056612 >> v1:172.21.15.114:6789/0 conn(0x7f9390070090 legacy=0x7f9390160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f9395325640 1 -- 172.21.15.106:0/817056612 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939015da00 con 0x7f93900cf5a0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f938ffff640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 0x7f9390160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f9395325640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900cf5a0 0x7f93900cfa30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9388009e50 tx=0x7f938802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c84d8c8d96fdaf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f938f7fe640 1 -- 172.21.15.106:0/817056612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938802ca70 con 0x7f93900cf5a0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f938f7fe640 1 -- 172.21.15.106:0/817056612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938802cc10 con 0x7f93900cf5a0 2024-04-13T03:15:27.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.955+0000 7f938f7fe640 1 -- 172.21.15.106:0/817056612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880343f0 con 0x7f93900cf5a0 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 -- 172.21.15.106:0/817056612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900cf5a0 msgr2=0x7f93900cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900cf5a0 0x7f93900cfa30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9388009e50 tx=0x7f938802bed0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 -- 172.21.15.106:0/817056612 shutdown_connections 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900cf5a0 0x7f93900cfa30 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 --2- 172.21.15.106:0/817056612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93900b0390 0x7f9390160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 -- 172.21.15.106:0/817056612 >> 172.21.15.106:0/817056612 conn(0x7f93900cd6d0 msgr2=0x7f93900cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 -- 172.21.15.106:0/817056612 shutdown_connections 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.956+0000 7f9396d88640 1 -- 172.21.15.106:0/817056612 wait complete. 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.957+0000 7f9396d88640 1 Processor -- start 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.957+0000 7f9396d88640 1 -- start start 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9390070090 0x7f93901f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93900cf5a0 0x7f9390147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f939006e560 con 0x7f93900b0390 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93900ceda0 con 0x7f93900cf5a0 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9396d88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f93900cd220 con 0x7f9390070090 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53938/0 (socket says 172.21.15.106:53938) 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 -- 172.21.15.106:0/1635081930 learned_addr learned my addr 172.21.15.106:0/1635081930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9395325640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93900cf5a0 0x7f9390147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f9394b24640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9390070090 0x7f93901f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 -- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9390070090 msgr2=0x7f93901f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9390070090 0x7f93901f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 -- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93900cf5a0 msgr2=0x7f9390147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93900cf5a0 0x7f9390147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.958+0000 7f938ffff640 1 -- 172.21.15.106:0/1635081930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939009bcb0 con 0x7f93900b0390 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f938ffff640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f938400d7d0 tx=0x7f938400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384004030 con 0x7f93900b0390 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840041d0 con 0x7f93900b0390 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939009bf90 con 0x7f93900b0390 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840096a0 con 0x7f93900b0390 2024-04-13T03:15:27.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.959+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939009c550 con 0x7f93900b0390 2024-04-13T03:15:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.960+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9390068a10 con 0x7f93900b0390 2024-04-13T03:15:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.960+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9384009840 con 0x7f93900b0390 2024-04-13T03:15:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.961+0000 7f938d7fa640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:27.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.961+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f938408be90 con 0x7f93900b0390 2024-04-13T03:15:27.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.961+0000 7f9394b24640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:27.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.961+0000 7f9394b24640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 0x7f937c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f93800016e0 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:27.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:27.965+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9384059ae0 con 0x7f93900b0390 2024-04-13T03:15:28.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.178+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93900d0840 con 0x7f93900b0390 2024-04-13T03:15:28.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.179+0000 7f938d7fa640 1 -- 172.21.15.106:0/1635081930 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5195 (secure 0 0 0) 0x7f938405dde0 con 0x7f93900b0390 2024-04-13T03:15:28.180 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:28.180 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:21.352812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/3360662124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3360662124},{"type":"v1","addr":"172.21.15.106:6837","nonce":3360662124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-13T03:15:28.180 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 51 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 msgr2=0x7f937c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 0x7f937c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f93800016e0 tx=0x7f9380009040 comp rx=0 tx=0).stop 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 msgr2=0x7f939009e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f938400d7d0 tx=0x7f938400dc90 comp rx=0 tx=0).stop 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 shutdown_connections 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93900cf5a0 0x7f9390147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93900b0390 0x7f939009e5c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 --2- 172.21.15.106:0/1635081930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9390070090 0x7f93901f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 >> 172.21.15.106:0/1635081930 conn(0x7f93900cd6d0 msgr2=0x7f93900b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 shutdown_connections 2024-04-13T03:15:28.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.181+0000 7f9396d88640 1 -- 172.21.15.106:0/1635081930 wait complete. 2024-04-13T03:15:28.192 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-13T03:15:28.321 INFO:teuthology.orchestra.run.smithi106.stdout:{"message":"","return_code":0} 2024-04-13T03:15:28.322 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:15:28.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:15:28.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.423+0000 7f4b20d9c640 1 Processor -- start 2024-04-13T03:15:28.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.423+0000 7f4b20d9c640 1 -- start start 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b20d9c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 0x7f4b1c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b20d9c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b1c15eca0 0x7f4b1c072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b20d9c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c0b5060 con 0x7f4b1c073350 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b20d9c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c0b5340 con 0x7f4b1c15eca0 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b20d9c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c06c500 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b19d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4b1c15eca0 0x7f4b1c072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b1a59c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 0x7f4b1c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b1ad9d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4b1c073350 0x7f4b1c0b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:34496/0 (socket says 172.21.15.106:34496) 2024-04-13T03:15:28.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.424+0000 7f4b1ad9d640 1 -- 172.21.15.106:0/2603707451 learned_addr learned my addr 172.21.15.106:0/2603707451 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.425+0000 7f4b19d9b640 1 -- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 msgr2=0x7f4b1c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.425+0000 7f4b19d9b640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 0x7f4b1c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.425+0000 7f4b19d9b640 1 -- 172.21.15.106:0/2603707451 >> v1:172.21.15.114:6789/0 conn(0x7f4b1c073350 legacy=0x7f4b1c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.425+0000 7f4b19d9b640 1 -- 172.21.15.106:0/2603707451 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c06c7e0 con 0x7f4b1c15eca0 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.425+0000 7f4b1a59c640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 0x7f4b1c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:15:28.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b19d9b640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c15eca0 0x7f4b1c072e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009d90 tx=0x7f4b1002bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca51b2bc4a173e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b1959a640 1 -- 172.21.15.106:0/2603707451 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10039070 con 0x7f4b1c15eca0 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b1959a640 1 -- 172.21.15.106:0/2603707451 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1002cce0 con 0x7f4b1c15eca0 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b1959a640 1 -- 172.21.15.106:0/2603707451 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10034660 con 0x7f4b1c15eca0 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 -- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c15eca0 msgr2=0x7f4b1c072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c15eca0 0x7f4b1c072e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009d90 tx=0x7f4b1002bf30 comp rx=0 tx=0).stop 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 -- 172.21.15.106:0/2603707451 shutdown_connections 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c15eca0 0x7f4b1c072e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/2603707451 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15e8c0 0x7f4b1c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 -- 172.21.15.106:0/2603707451 >> 172.21.15.106:0/2603707451 conn(0x7f4b1c06f0d0 msgr2=0x7f4b1c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 -- 172.21.15.106:0/2603707451 shutdown_connections 2024-04-13T03:15:28.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.426+0000 7f4b20d9c640 1 -- 172.21.15.106:0/2603707451 wait complete. 2024-04-13T03:15:28.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 Processor -- start 2024-04-13T03:15:28.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 -- start start 2024-04-13T03:15:28.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c073350 0x7f4b1c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15eca0 0x7f4b1c1f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c06cd00 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c0b4d50 con 0x7f4b1c073350 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.428+0000 7f4b20d9c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c06ce80 con 0x7f4b1c15eca0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53954/0 (socket says 172.21.15.106:53954) 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 -- 172.21.15.106:0/3085103065 learned_addr learned my addr 172.21.15.106:0/3085103065 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b1a59c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c073350 0x7f4b1c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b1ad9d640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15eca0 0x7f4b1c1f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 -- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15eca0 msgr2=0x7f4b1c1f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15eca0 0x7f4b1c1f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 -- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c073350 msgr2=0x7f4b1c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c073350 0x7f4b1c0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 -- 172.21.15.106:0/3085103065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c0cb640 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b19d9b640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4b10037040 tx=0x7f4b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10039050 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b10045070 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c1594e0 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1003e850 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.429+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c159aa0 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.430+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1c068a10 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.431+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b1003e9f0 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.431+0000 7f4b0affd640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 0x7f4afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.431+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f4b100ae9f0 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.431+0000 7f4b1a59c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 0x7f4afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.431+0000 7f4b1a59c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 0x7f4afc06edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009560 tx=0x7f4b14009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.435+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b1007ce40 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.648+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b1c0cdbf0 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.650 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:15:28.649+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:15:28.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.650+0000 7f4b0affd640 1 -- 172.21.15.106:0/3085103065 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7f4b100ae020 con 0x7f4b1c15e8c0 2024-04-13T03:15:28.651 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.653+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 msgr2=0x7f4afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.653+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 0x7f4afc06edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009560 tx=0x7f4b14009190 comp rx=0 tx=0).stop 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.653+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 msgr2=0x7f4b1c155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.653+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4b10037040 tx=0x7f4b10004a40 comp rx=0 tx=0).stop 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 shutdown_connections 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4afc06c8f0 0x7f4afc06edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4b1c15eca0 0x7f4b1c1f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b1c15e8c0 0x7f4b1c155700 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 --2- 172.21.15.106:0/3085103065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b1c073350 0x7f4b1c0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 >> 172.21.15.106:0/3085103065 conn(0x7f4b1c06f0d0 msgr2=0x7f4b1c06f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 shutdown_connections 2024-04-13T03:15:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.654+0000 7f4b20d9c640 1 -- 172.21.15.106:0/3085103065 wait complete. 2024-04-13T03:15:28.656 INFO:tasks.ceph.mds.a.smithi106.stderr: -6206> 2024-04-13T03:15:14.596+0000 7f45c11fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-13T03:15:28.656 INFO:tasks.ceph.mds.a.smithi106.stderr: -5095> 2024-04-13T03:15:20.577+0000 7f45bb1f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:15:28.667 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-13T03:15:28.687 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:15:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.707+0000 7f0007e63f40 1 Processor -- start 2024-04-13T03:15:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.707+0000 7f0007e63f40 1 -- start start 2024-04-13T03:15:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961e9dfc00 0x55961ea524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 0x55961ea6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55961e9db780 con 0x55961e9c06d0 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55961e93ed00 con 0x55961e9dfc00 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0007e63f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55961e9dba20 con 0x55961ea6ebe0 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54000/0 (socket says 172.21.15.106:54000) 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 -- 172.21.15.106:0/3982367800 learned_addr learned my addr 172.21.15.106:0/3982367800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0004f4e640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961e9dfc00 0x55961ea524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0005f50640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 0x55961ea6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 -- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 msgr2=0x55961ea6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 0x55961ea6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 -- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961e9dfc00 msgr2=0x55961ea524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961e9dfc00 0x55961ea524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f000574f640 1 -- 172.21.15.106:0/3982367800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961ea6d910 con 0x55961e9c06d0 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.708+0000 7f0005f50640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 0x55961ea6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f000574f640 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efff800a7d0 tx=0x7efff800a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa1910d31f5d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7efff77fe640 1 -- 172.21.15.106:0/3982367800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8018070 con 0x55961e9c06d0 2024-04-13T03:15:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7efff77fe640 1 -- 172.21.15.106:0/3982367800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8007600 con 0x55961e9c06d0 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7efff77fe640 1 -- 172.21.15.106:0/3982367800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80079d0 con 0x55961e9c06d0 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 -- 172.21.15.106:0/3982367800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 msgr2=0x55961e9c4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efff800a7d0 tx=0x7efff800a8c0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 -- 172.21.15.106:0/3982367800 shutdown_connections 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961ea6ebe0 0x55961ea6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961e9dfc00 0x55961ea524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 --2- 172.21.15.106:0/3982367800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961e9c4160 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 -- 172.21.15.106:0/3982367800 >> 172.21.15.106:0/3982367800 conn(0x55961e9c1ae0 msgr2=0x55961ea6efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 -- 172.21.15.106:0/3982367800 shutdown_connections 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.709+0000 7f0007e63f40 1 -- 172.21.15.106:0/3982367800 wait complete. 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 Processor -- start 2024-04-13T03:15:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 -- start start 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961eb05910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961e9dfc00 0x55961ea41640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961ea6ebe0 0x55961e995b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55961e9c4a30 con 0x55961e9c06d0 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55961ea541d0 con 0x55961ea6ebe0 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.710+0000 7f0007e63f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55961ea52a30 con 0x55961e9dfc00 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961eb05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961eb05910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54004/0 (socket says 172.21.15.106:54004) 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 -- 172.21.15.106:0/1134817317 learned_addr learned my addr 172.21.15.106:0/1134817317 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f0004f4e640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961e9dfc00 0x55961ea41640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f0005f50640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961ea6ebe0 0x55961e995b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 -- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961e9dfc00 msgr2=0x55961ea41640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55961e9dfc00 0x55961ea41640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 -- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961ea6ebe0 msgr2=0x55961e995b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55961ea6ebe0 0x55961e995b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 -- 172.21.15.106:0/1134817317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961ea41b80 con 0x55961e9c06d0 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f000574f640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961eb05910 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efff8016040 tx=0x7efff8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7efff67fc640 1 -- 172.21.15.106:0/1134817317 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8018050 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7efff67fc640 1 -- 172.21.15.106:0/1134817317 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8024070 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55961eb05f40 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7efff67fc640 1 -- 172.21.15.106:0/1134817317 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8013840 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.711+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961eb06500 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.712+0000 7efff67fc640 1 -- 172.21.15.106:0/1134817317 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efff80139e0 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7efff67fc640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effe806b2d0 0x7effe806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7efff67fc640 1 -- 172.21.15.106:0/1134817317 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7efff808bd40 con 0x55961e9c06d0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0004f4e640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effe806b2d0 0x7effe806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0007e63f40 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55961e997600 0x55961e997a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55961e997f50 con 0x55961e997600 2024-04-13T03:15:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0005f50640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55961e997600 0x55961e997a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0004f4e640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effe806b2d0 0x7effe806d790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55961e9987f0 tx=0x7efffc0017f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.713+0000 7f0005f50640 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55961e997600 0x55961e997a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0005f50640 1 -- 172.21.15.106:0/1134817317 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v92'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f000005e040 con 0x55961e997600 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55961e997600 msgr2=0x55961e997a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55961e997600 0x55961e997a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effe806b2d0 msgr2=0x7effe806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7effe806b2d0 0x7effe806d790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55961e9987f0 tx=0x7efffc0017f0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 msgr2=0x55961eb05910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 --2- 172.21.15.106:0/1134817317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55961e9c06d0 0x55961eb05910 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efff8016040 tx=0x7efff8003020 comp rx=0 tx=0).stop 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0005f50640 1 -- 172.21.15.106:0/1134817317 reap_dead start 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 shutdown_connections 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.716+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 >> 172.21.15.106:0/1134817317 conn(0x55961e9c1ae0 msgr2=0x55961e998dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.717+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 shutdown_connections 2024-04-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.717+0000 7f0007e63f40 1 -- 172.21.15.106:0/1134817317 wait complete. 2024-04-13T03:15:28.725 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.804+0000 7f3c13873280 1 -- start start 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.804+0000 7f3c13873280 1 Processor -- start 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 0x56436ccec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436bf21800 0x56436ccec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56436cd10180 con 0x56436bf21c00 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56436bf46a80 con 0x56436bf21800 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56436bf47800 con 0x56436bf20c00 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54018/0 (socket says 172.21.15.106:54018) 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 -- 172.21.15.106:0/3598191013 learned_addr learned my addr 172.21.15.106:0/3598191013 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c115fb640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436bf21800 0x56436ccec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 0x56436ccec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 -- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 msgr2=0x56436ccec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 0x56436ccec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 -- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436bf21800 msgr2=0x56436ccec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436bf21800 0x56436ccec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 -- 172.21.15.106:0/3598191013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436bf58780 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 0x56436ccec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.805+0000 7f3c125fd640 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x56436cd55320 tx=0x56436ccb6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05241566f399d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c0f5f7640 1 -- 172.21.15.106:0/3598191013 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf564e0 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c0f5f7640 1 -- 172.21.15.106:0/3598191013 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56436bf4c700 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c0f5f7640 1 -- 172.21.15.106:0/3598191013 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf56b60 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436bf594a0 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56436bf58000 con 0x56436bf21c00 2024-04-13T03:15:28.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 4 waiting for MDS map... 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c0f5f7640 1 -- 172.21.15.106:0/3598191013 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x56436cd32600 con 0x56436bf21c00 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c0f5f7640 1 -- 172.21.15.106:0/3598191013 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 52) ==== 2885+0+0 (secure 0 0 0) 0x56436bf7d180 con 0x56436bf21c00 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 4 Got MDS map 52 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 10 main: JournalTool::main 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 4 main: JournalTool: connecting to RADOS... 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.806+0000 7f3c13873280 1 Processor -- start 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 -- start start 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56436bf47b00 con 0x56436cd01c00 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56436cd10d80 con 0x56436cd01800 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56436cd10f00 con 0x56436cd01400 2024-04-13T03:15:28.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54034/0 (socket says 172.21.15.106:54034) 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 -- 172.21.15.106:0/169657566 learned_addr learned my addr 172.21.15.106:0/169657566 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 -- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01400 msgr2=0x56436cced080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 -- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01800 msgr2=0x56436cced600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.807+0000 7f3c115fb640 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c115fb640 1 -- 172.21.15.106:0/169657566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436bf583c0 con 0x56436cd01c00 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c115fb640 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x56436ce4e660 tx=0x56436ccb4780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a085b67cd6d43d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c0d5f3640 1 -- 172.21.15.106:0/169657566 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf569c0 con 0x56436cd01c00 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c0d5f3640 1 -- 172.21.15.106:0/169657566 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56436bf4cfc0 con 0x56436cd01c00 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c0d5f3640 1 -- 172.21.15.106:0/169657566 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf564e0 con 0x56436cd01c00 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 -- 172.21.15.106:0/169657566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 msgr2=0x56436ccedb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x56436ce4e660 tx=0x56436ccb4780 comp rx=0 tx=0).stop 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 -- 172.21.15.106:0/169657566 shutdown_connections 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 --2- 172.21.15.106:0/169657566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 -- 172.21.15.106:0/169657566 >> 172.21.15.106:0/169657566 conn(0x56436cd01000 msgr2=0x56436c00d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 -- 172.21.15.106:0/169657566 shutdown_connections 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.808+0000 7f3c13873280 1 -- 172.21.15.106:0/169657566 wait complete. 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 Processor -- start 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 -- start start 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56436cd11200 con 0x56436cd01c00 2024-04-13T03:15:28.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56436cd11080 con 0x56436cd01400 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56436cd11380 con 0x56436cd01800 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54038/0 (socket says 172.21.15.106:54038) 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.809+0000 7f3c115fb640 1 -- 172.21.15.106:0/2666703543 learned_addr learned my addr 172.21.15.106:0/2666703543 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 -- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01800 msgr2=0x56436cced600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 -- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01400 msgr2=0x56436cced080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 -- 172.21.15.106:0/2666703543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436bf58780 con 0x56436cd01c00 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c115fb640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x56436ce4e990 tx=0x56436ccb4a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c0bdf0640 1 -- 172.21.15.106:0/2666703543 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf56b60 con 0x56436cd01c00 2024-04-13T03:15:28.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c0bdf0640 1 -- 172.21.15.106:0/2666703543 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56436bf4ce00 con 0x56436cd01c00 2024-04-13T03:15:28.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56436bf594a0 con 0x56436cd01c00 2024-04-13T03:15:28.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c0bdf0640 1 -- 172.21.15.106:0/2666703543 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436bf56000 con 0x56436cd01c00 2024-04-13T03:15:28.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436bf58960 con 0x56436cd01c00 2024-04-13T03:15:28.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.810+0000 7f3c13873280 4 main: JournalTool: resolving pool 13 2024-04-13T03:15:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.811+0000 7f3c0bdf0640 1 -- 172.21.15.106:0/2666703543 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x56436ccf4c00 con 0x56436cd01c00 2024-04-13T03:15:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.811+0000 7f3c0bdf0640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 0x56436ccee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c0bdf0640 1 -- 172.21.15.106:0/2666703543 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x56436ccf6600 con 0x56436cd01c00 2024-04-13T03:15:28.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c13873280 4 main: JournalTool: creating IoCtx.. 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c13873280 4 main: Executing for rank 0 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 0x56436ccee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 0x56436ccee100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x56436c0246f0 tx=0x56436c22aa80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c13873280 1 -- start start 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.812+0000 7f3c13873280 1 Processor -- start 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436c29b400 0x56436ccee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436c29b800 0x56436cceec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56436cd10f00 con 0x56436c29bc00 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56436cd10d80 con 0x56436c29b800 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c13873280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56436cd11b00 con 0x56436c29b400 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54054/0 (socket says 172.21.15.106:54054) 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 learned_addr learned my addr 172.21.15.106:0/6554343 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436c29b400 0x56436ccee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436c29b400 msgr2=0x56436ccee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436c29b800 0x56436cceec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436c29b400 0x56436ccee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436c29b800 msgr2=0x56436cceec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436c29b800 0x56436cceec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436bf59680 con 0x56436c29bc00 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436c29b800 0x56436cceec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c115fb640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x56436cead350 tx=0x56436ce0bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6344fb31fe7ce086 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c08dea640 1 -- 172.21.15.106:0/6554343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436c227520 con 0x56436c29bc00 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.813+0000 7f3c08dea640 1 -- 172.21.15.106:0/6554343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56436bf4ca80 con 0x56436c29bc00 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436bf583c0 con 0x56436c29bc00 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56436bf58780 con 0x56436c29bc00 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 4 waiting for MDS map... 2024-04-13T03:15:28.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c08dea640 1 -- 172.21.15.106:0/6554343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56436c2276c0 con 0x56436c29bc00 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c08dea640 1 -- 172.21.15.106:0/6554343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x56436c2fc600 con 0x56436c29bc00 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c08dea640 1 -- 172.21.15.106:0/6554343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 52) ==== 2885+0+0 (secure 0 0 0) 0x56436bf7d500 con 0x56436c29bc00 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 4 Got MDS map 52 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x56436c2ff800 0x56436ccef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x56436c2ffc00 con 0x56436c2ff800 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x56436c2ff800 0x56436ccef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.814+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x56436c2ff800 0x56436ccef700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=1b8af618921a4c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.817+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56436c022000 con 0x56436c2ff800 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.817+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x56436c31e000 0x56436c324000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.817+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x56436c31e400 con 0x56436c31e000 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.817+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x56436c31e000 0x56436c324000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.817+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x56436c31e000 0x56436c324000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=88622c7f29283126 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c11dfc640 1 -- 172.21.15.106:0/6554343 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56436c022500 con 0x56436c31e000 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c075e7640 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) -- 0x56436c31fc00 con 0x56436c31e000 2024-04-13T03:15:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c075e7640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56436c32a000 0x56436c325b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c075e7640 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) -- 0x56436c32a400 con 0x56436c32a000 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c11dfc640 1 -- 172.21.15.106:0/6554343 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56436c022500 con 0x56436c31e000 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56436c32a000 0x56436c325b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.818+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56436c32a000 0x56436c325b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=820b38db5558d13d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.821+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c32a000 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stdout:old journal was 4212724~2605 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stdout:new journal start will be 8388608 (4173279 bytes past old end) 2024-04-13T03:15:28.826 INFO:teuthology.orchestra.run.smithi106.stdout:writing journal head 2024-04-13T03:15:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.821+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31e800 con 0x56436c31e000 2024-04-13T03:15:28.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c11dfc640 1 -- 172.21.15.106:0/6554343 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v92'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56436c022500 con 0x56436c31e000 2024-04-13T03:15:28.827 INFO:teuthology.orchestra.run.smithi106.stdout:writing EResetJournal entry 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31ec00 con 0x56436c32a000 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x56436c31f000 0x56436c324580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31f400 con 0x56436c31f000 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31f800 con 0x56436c2ff800 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x56436c32a800 0x56436c324b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c32ac00 con 0x56436c32a800 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c32b000 con 0x56436c32a000 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x56436c32b400 0x56436c325080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31e400 con 0x56436c32b400 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c115fb640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x56436c32a800 0x56436c324b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x56436c31f000 0x56436c324580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x56436c32b400 0x56436c325080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.822+0000 7f3c115fb640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x56436c32a800 0x56436c324b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=cbc9e48e38541c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.823+0000 7f3c11dfc640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x56436c31f000 0x56436c324580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=79763e841e2da4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.823+0000 7f3c125fd640 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x56436c32b400 0x56436c325080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=cc7a12f012fdab9b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.823+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(6 200.00000002 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c32a000 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.824+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(8 200.00000004 [delete] v92'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c2ff800 2024-04-13T03:15:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.824+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(10 200.00000006 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c32a000 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.824+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(9 200.00000005 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022780 con 0x56436c32a800 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.825+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(11 200.00000007 [delete] v92'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c32b400 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.826+0000 7f3c11dfc640 1 -- 172.21.15.106:0/6554343 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(7 200.00000003 [delete] v92'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56436c022500 con 0x56436c31f000 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.826+0000 7f3c075e7640 1 -- 172.21.15.106:0/6554343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x56436c31ec00 con 0x56436c32a000 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c125fd640 1 -- 172.21.15.106:0/6554343 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v92'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56436c022000 con 0x56436c32a000 2024-04-13T03:15:28.832 INFO:teuthology.orchestra.run.smithi106.stdout:done 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x56436c31e000 msgr2=0x56436c324000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x56436c31e000 0x56436c324000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x56436c32a800 msgr2=0x56436c324b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x56436c32a800 0x56436c324b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56436c32a000 msgr2=0x56436c325b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56436c32a000 0x56436c325b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x56436c31f000 msgr2=0x56436c324580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x56436c31f000 0x56436c324580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x56436c2ff800 msgr2=0x56436ccef700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x56436c2ff800 0x56436ccef700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x56436c32b400 msgr2=0x56436c325080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x56436c32b400 0x56436c325080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c115fb640 1 -- 172.21.15.106:0/6554343 reap_dead start 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 msgr2=0x56436ccef180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.827+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x56436cead350 tx=0x56436ce0bf80 comp rx=0 tx=0).stop 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 shutdown_connections 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/6554343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436c29bc00 0x56436ccef180 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 >> 172.21.15.106:0/6554343 conn(0x56436c29b000 msgr2=0x56436c00d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 shutdown_connections 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/6554343 wait complete. 2024-04-13T03:15:28.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 msgr2=0x56436ccee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 0x56436ccee100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x56436c0246f0 tx=0x56436c22aa80 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 msgr2=0x56436ccedb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x56436ce4e990 tx=0x56436ccb4a20 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 shutdown_connections 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x56436c29a000 0x56436ccee100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436cd01800 0x56436cced600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436cd01c00 0x56436ccedb80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/2666703543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436cd01400 0x56436cced080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 >> 172.21.15.106:0/2666703543 conn(0x56436cd01000 msgr2=0x56436c00d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 shutdown_connections 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/2666703543 wait complete. 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 msgr2=0x56436ccecb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x56436cd55320 tx=0x56436ccb6a20 comp rx=0 tx=0).stop 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 shutdown_connections 2024-04-13T03:15:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56436bf21c00 0x56436ccecb00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56436bf21800 0x56436ccec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 --2- 172.21.15.106:0/3598191013 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56436bf20c00 0x56436ccec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 >> 172.21.15.106:0/3598191013 conn(0x56436bf21000 msgr2=0x56436c00c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 shutdown_connections 2024-04-13T03:15:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.828+0000 7f3c13873280 1 -- 172.21.15.106:0/3598191013 wait complete. 2024-04-13T03:15:28.838 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.112842, result: old journal was 4212724~2605 new journal start will be 8388608 (4173279 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-13T03:15:28.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:15:28.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.979+0000 7f2bf06de640 1 Processor -- start 2024-04-13T03:15:28.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.979+0000 7f2bf06de640 1 -- start start 2024-04-13T03:15:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.980+0000 7f2bf06de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2be80d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.980+0000 7f2bf06de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be815ddd0 con 0x7f2be80d1a60 2024-04-13T03:15:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.980+0000 7f2bf06de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be815d4d0 con 0x7f2be80cfea0 2024-04-13T03:15:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.980+0000 7f2bf06de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2be815e100 con 0x7f2be80704a0 2024-04-13T03:15:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2beec7b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2be80d1a60 0x7f2be815e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47200/0 (socket says 172.21.15.106:47200) 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2beec7b640 1 -- 172.21.15.106:0/3946401530 learned_addr learned my addr 172.21.15.106:0/3946401530 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206028748 0 0) 0x7f2be815ddd0 con 0x7f2be80d1a60 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bd4003520 con 0x7f2be80d1a60 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bedc79640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2be80d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4119750716 0 0) 0x7f2bd4003520 con 0x7f2be80d1a60 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 msgr2=0x7f2be80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2be80d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 >> v1:172.21.15.114:6789/0 conn(0x7f2be80cfea0 legacy=0x7f2be815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be815d7b0 con 0x7f2be80d1a60 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2be00019f0 con 0x7f2be80d1a60 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 >> v1:172.21.15.106:6789/0 conn(0x7f2be80d1a60 legacy=0x7f2be815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd40039e0 0x7f2bd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2bd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2be815e100 con 0x7f2bd4004800 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd400ba70 con 0x7f2bd40039e0 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd400bc70 con 0x7f2be80704a0 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bedc79640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2bd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2bee47a640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd40039e0 0x7f2bd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.981+0000 7f2beec7b640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 -- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 msgr2=0x7f2bd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be80704a0 0x7f2bd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 -- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd40039e0 msgr2=0x7f2bd4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd40039e0 0x7f2bd4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 -- 172.21.15.106:0/3946401530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd4003520 con 0x7f2bd4004800 2024-04-13T03:15:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bedc79640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2bd4004c80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009a90 tx=0x7f2bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfe5df8019fdfb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4039070 con 0x7f2bd4004800 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.982+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be402c680 con 0x7f2bd4004800 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bed478640 1 -- 172.21.15.106:0/3946401530 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4009690 con 0x7f2bd4004800 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 -- 172.21.15.106:0/3946401530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 msgr2=0x7f2bd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 --2- 172.21.15.106:0/3946401530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2bd4004c80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009a90 tx=0x7f2bd4005830 comp rx=0 tx=0).stop 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2beec7b640 1 -- 172.21.15.106:0/3946401530 reap_dead start 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 -- 172.21.15.106:0/3946401530 shutdown_connections 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 -- 172.21.15.106:0/3946401530 >> 172.21.15.106:0/3946401530 conn(0x7f2be8071870 msgr2=0x7f2be8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 -- 172.21.15.106:0/3946401530 shutdown_connections 2024-04-13T03:15:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.983+0000 7f2bf06de640 1 -- 172.21.15.106:0/3946401530 wait complete. 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.984+0000 7f2bf06de640 1 Processor -- start 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.984+0000 7f2bf06de640 1 -- start start 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be81f4b60 0x7f2be8155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be81562f0 0x7f2be8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2be80d1520 con 0x7f2bd4004800 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2be80d1ef0 con 0x7f2be81f4b60 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bf06de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2be80d16a0 con 0x7f2be81562f0 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54066/0 (socket says 172.21.15.106:54066) 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 -- 172.21.15.106:0/2834262073 learned_addr learned my addr 172.21.15.106:0/2834262073 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2beec7b640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be81562f0 0x7f2be8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bedc79640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be81f4b60 0x7f2be8155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 -- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be81562f0 msgr2=0x7f2be8159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be81562f0 0x7f2be8159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 -- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be81f4b60 msgr2=0x7f2be8155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be81f4b60 0x7f2be8155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 -- 172.21.15.106:0/2834262073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81f5370 con 0x7f2bd4004800 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.985+0000 7f2bee47a640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800f850 tx=0x7f2bd800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.986+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800fe40 con 0x7f2bd4004800 2024-04-13T03:15:28.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.986+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8018070 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.986+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81f5650 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.986+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800d730 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.986+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be8152f20 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.987+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be8068a10 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.987+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bd800d8d0 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.987+0000 7f2bdeffd640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.987+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f2bd808ba70 con 0x7f2bd4004800 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.987+0000 7f2bedc79640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.988+0000 7f2bedc79640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2be4036000 tx=0x7f2be4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:28.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:28.992+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bd8059670 con 0x7f2bd4004800 2024-04-13T03:15:29.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.204+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2be8072360 con 0x7f2bd4004800 2024-04-13T03:15:29.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.652+0000 7f2bdeffd640 1 -- 172.21.15.106:0/2834262073 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v53) ==== 162+0+0 (secure 0 0 0) 0x7f2bd805d970 con 0x7f2bd4004800 2024-04-13T03:15:29.653 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 msgr2=0x7f2bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2be4036000 tx=0x7f2be4030000 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 msgr2=0x7f2be8157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800f850 tx=0x7f2bd800fd10 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 shutdown_connections 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2be81562f0 0x7f2be8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be81f4b60 0x7f2be8155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 --2- 172.21.15.106:0/2834262073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bd4004800 0x7f2be8157400 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 >> 172.21.15.106:0/2834262073 conn(0x7f2be8071870 msgr2=0x7f2be815c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:29.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 shutdown_connections 2024-04-13T03:15:29.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.654+0000 7f2bf06de640 1 -- 172.21.15.106:0/2834262073 wait complete. 2024-04-13T03:15:29.668 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.769+0000 7f0423108640 1 Processor -- start 2024-04-13T03:15:29.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.769+0000 7f0423108640 1 -- start start 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0423108640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f041c0d1c00 0x7f041c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0423108640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f041c0b8080 0x7f041c1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0423108640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041c0d3530 con 0x7f041c1622d0 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0423108640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f041c1626b0 con 0x7f041c0d1c00 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0423108640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f041c0d0b60 con 0x7f041c0b8080 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f041c0d1c00 0x7f041c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0420ea4640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f041c1622d0 0x7f041c0b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47212/0 (socket says 172.21.15.106:47212) 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f0420ea4640 1 -- 172.21.15.106:0/1230435777 learned_addr learned my addr 172.21.15.106:0/1230435777 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041bfff640 1 -- 172.21.15.106:0/1230435777 >> v2:172.21.15.114:3300/0 conn(0x7f041c0b8080 msgr2=0x7f041c1688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:29.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041bfff640 1 --2- 172.21.15.106:0/1230435777 >> v2:172.21.15.114:3300/0 conn(0x7f041c0b8080 0x7f041c1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041bfff640 1 -- 172.21.15.106:0/1230435777 >> v1:172.21.15.106:6789/0 conn(0x7f041c1622d0 legacy=0x7f041c0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041bfff640 1 -- 172.21.15.106:0/1230435777 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c0d0e40 con 0x7f041c0d1c00 2024-04-13T03:15:29.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.770+0000 7f041b7fe640 1 -- 172.21.15.106:0/1230435777 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594604159 0 0) 0x7f041c0d3530 con 0x7f041c1622d0 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f041bfff640 1 --2- 172.21.15.106:0/1230435777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c0d1c00 0x7f041c0d04d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ceef0 tx=0x7f040802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615f9d4f132491f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f041b7fe640 1 -- 172.21.15.106:0/1230435777 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408035020 con 0x7f041c0d1c00 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f041b7fe640 1 -- 172.21.15.106:0/1230435777 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0408031830 con 0x7f041c0d1c00 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f041b7fe640 1 -- 172.21.15.106:0/1230435777 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408031c00 con 0x7f041c0d1c00 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 -- 172.21.15.106:0/1230435777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c0d1c00 msgr2=0x7f041c0d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 --2- 172.21.15.106:0/1230435777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c0d1c00 0x7f041c0d04d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ceef0 tx=0x7f040802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 -- 172.21.15.106:0/1230435777 shutdown_connections 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 --2- 172.21.15.106:0/1230435777 >> v2:172.21.15.114:3300/0 conn(0x7f041c0b8080 0x7f041c1688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 --2- 172.21.15.106:0/1230435777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c0d1c00 0x7f041c0d04d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 -- 172.21.15.106:0/1230435777 >> 172.21.15.106:0/1230435777 conn(0x7f041c0d25d0 msgr2=0x7f041c0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 -- 172.21.15.106:0/1230435777 shutdown_connections 2024-04-13T03:15:29.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.771+0000 7f0423108640 1 -- 172.21.15.106:0/1230435777 wait complete. 2024-04-13T03:15:29.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 Processor -- start 2024-04-13T03:15:29.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 -- start start 2024-04-13T03:15:29.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f041c0b8080 0x7f041c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f041c0d1c00 0x7f041c1427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f041c0d16e0 con 0x7f041c1622d0 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f041c0d1400 con 0x7f041c0b8080 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.773+0000 7f0423108640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f041c0cc6f0 con 0x7f041c0d1c00 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54086/0 (socket says 172.21.15.106:54086) 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 -- 172.21.15.106:0/2548524342 learned_addr learned my addr 172.21.15.106:0/2548524342 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 -- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f041c0d1c00 msgr2=0x7f041c1427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f041c0d1c00 0x7f041c1427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 -- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f041c0b8080 msgr2=0x7f041c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f041c0b8080 0x7f041c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 -- 172.21.15.106:0/2548524342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c15f150 con 0x7f041c1622d0 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04216a5640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f041400b9a0 tx=0x7f041400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:29.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414019070 con 0x7f041c1622d0 2024-04-13T03:15:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041400cbc0 con 0x7f041c1622d0 2024-04-13T03:15:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414014440 con 0x7f041c1622d0 2024-04-13T03:15:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c0ca790 con 0x7f041c1622d0 2024-04-13T03:15:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.774+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c0cad50 con 0x7f041c1622d0 2024-04-13T03:15:29.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.775+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c068a10 con 0x7f041c1622d0 2024-04-13T03:15:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.776+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f041401c070 con 0x7f041c1622d0 2024-04-13T03:15:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.776+0000 7f04197fa640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 0x7f040006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.776+0000 7f0420ea4640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 0x7f040006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.776+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f041408bb00 con 0x7f041c1622d0 2024-04-13T03:15:29.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.776+0000 7f0420ea4640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 0x7f040006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0410007fd0 tx=0x7f0410007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:29.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.781+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0414059d50 con 0x7f041c1622d0 2024-04-13T03:15:29.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.995+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f041c0b3ab0 con 0x7f041c1622d0 2024-04-13T03:15:29.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.995+0000 7f04197fa640 1 -- 172.21.15.106:0/2548524342 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5186 (secure 0 0 0) 0x7f0414059650 con 0x7f041c1622d0 2024-04-13T03:15:29.997 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:29.997 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:29.655034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:replay","state_seq":1,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-13T03:15:29.997 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 54 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.997+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 msgr2=0x7f040006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.997+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 0x7f040006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0410007fd0 tx=0x7f0410007960 comp rx=0 tx=0).stop 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 msgr2=0x7f041c1f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f041400b9a0 tx=0x7f041400be60 comp rx=0 tx=0).stop 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 shutdown_connections 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f040006c8f0 0x7f040006edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f041c0d1c00 0x7f041c1427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f041c1622d0 0x7f041c1f7af0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:29.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 --2- 172.21.15.106:0/2548524342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f041c0b8080 0x7f041c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:30.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 >> 172.21.15.106:0/2548524342 conn(0x7f041c0d25d0 msgr2=0x7f041c0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:30.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 shutdown_connections 2024-04-13T03:15:30.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:29.998+0000 7f0423108640 1 -- 172.21.15.106:0/2548524342 wait complete. 2024-04-13T03:15:30.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:15:29.655034+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/635681585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 635681585}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 635681585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:15:30.412 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7728:up:replay 2024-04-13T03:15:31.413 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.531+0000 7f68eaade640 1 Processor -- start 2024-04-13T03:15:31.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.531+0000 7f68eaade640 1 -- start start 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68eaade640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f68e415e9b0 0x7f68e40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68eaade640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e40cfd20 con 0x7f68e40b3d60 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68eaade640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e4162af0 con 0x7f68e415ed90 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68eaade640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f68e4162dd0 con 0x7f68e415e9b0 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e907b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f68e40b3d60 0x7f68e40d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47224/0 (socket says 172.21.15.106:47224) 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e907b640 1 -- 172.21.15.106:0/250482676 learned_addr learned my addr 172.21.15.106:0/250482676 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283936260 0 0) 0x7f68e40cfd20 con 0x7f68e40b3d60 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e887a640 1 --2- 172.21.15.106:0/250482676 >> v2:172.21.15.114:3300/0 conn(0x7f68e415e9b0 0x7f68e40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68d0003520 con 0x7f68e40b3d60 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3831050099 0 0) 0x7f68d0003520 con 0x7f68e40b3d60 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 >> v2:172.21.15.114:3300/0 conn(0x7f68e415e9b0 msgr2=0x7f68e40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 --2- 172.21.15.106:0/250482676 >> v2:172.21.15.114:3300/0 conn(0x7f68e415e9b0 0x7f68e40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 >> v1:172.21.15.114:6789/0 conn(0x7f68e415ed90 legacy=0x7f68e40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e41630b0 con 0x7f68e40b3d60 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.532+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f68dc0019f0 con 0x7f68e40b3d60 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 >> v1:172.21.15.106:6789/0 conn(0x7f68e40b3d60 legacy=0x7f68e40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00039b0 0x7f68d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4162dd0 con 0x7f68d00051c0 2024-04-13T03:15:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68d000c070 con 0x7f68d0004800 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68d000c270 con 0x7f68d00039b0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e3fff640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e887a640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00039b0 0x7f68d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 -- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00039b0 msgr2=0x7f68d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00039b0 0x7f68d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 -- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68d0004800 msgr2=0x7f68d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.533+0000 7f68e907b640 1 -- 172.21.15.106:0/250482676 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d0003520 con 0x7f68d00051c0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.534+0000 7f68e907b640 1 -- 172.21.15.106:0/250482676 reap_dead start 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.534+0000 7f68e907b640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00a840 tx=0x7f68d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4082953a781d1988 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.534+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00a460 con 0x7f68d00051c0 2024-04-13T03:15:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.534+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc034650 con 0x7f68d00051c0 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.534+0000 7f68e37fe640 1 -- 172.21.15.106:0/250482676 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc034a40 con 0x7f68d00051c0 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 -- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 msgr2=0x7f68d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00a840 tx=0x7f68d0005830 comp rx=0 tx=0).stop 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 -- 172.21.15.106:0/250482676 shutdown_connections 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 --2- 172.21.15.106:0/250482676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 -- 172.21.15.106:0/250482676 >> 172.21.15.106:0/250482676 conn(0x7f68e408bf10 msgr2=0x7f68e40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 -- 172.21.15.106:0/250482676 shutdown_connections 2024-04-13T03:15:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.535+0000 7f68eaade640 1 -- 172.21.15.106:0/250482676 wait complete. 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.536+0000 7f68eaade640 1 Processor -- start 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.536+0000 7f68eaade640 1 -- start start 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00051c0 0x7f68e414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e414a790 0x7f68e414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4162800 con 0x7f68e40b4140 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4162680 con 0x7f68e414a790 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68eaade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68e406eb20 con 0x7f68d00051c0 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54102/0 (socket says 172.21.15.106:54102) 2024-04-13T03:15:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 -- 172.21.15.106:0/746570114 learned_addr learned my addr 172.21.15.106:0/746570114 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e887a640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00051c0 0x7f68e414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e907b640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e414a790 0x7f68e414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 -- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00051c0 msgr2=0x7f68e414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00051c0 0x7f68e414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 -- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e414a790 msgr2=0x7f68e414ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.537+0000 7f68e3fff640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e414a790 0x7f68e414ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68e3fff640 1 -- 172.21.15.106:0/746570114 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e414bd80 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68e3fff640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f68cc001f00 tx=0x7f68cc00f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc01a070 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc005320 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e414c060 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc006a40 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.538+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e414c620 con 0x7f68e40b4140 2024-04-13T03:15:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.539+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e4068a10 con 0x7f68e40b4140 2024-04-13T03:15:31.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.539+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68cc01d070 con 0x7f68e40b4140 2024-04-13T03:15:31.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.540+0000 7f68e17fa640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 0x7f68c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:31.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.540+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f68cc08b3c0 con 0x7f68e40b4140 2024-04-13T03:15:31.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.540+0000 7f68e887a640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 0x7f68c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:31.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.540+0000 7f68e887a640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 0x7f68c406ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009b40 tx=0x7f68d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:31.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.545+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68cc058fc0 con 0x7f68e40b4140 2024-04-13T03:15:31.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.759+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68e40d53a0 con 0x7f68e40b4140 2024-04-13T03:15:31.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.760+0000 7f68e17fa640 1 -- 172.21.15.106:0/746570114 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5186 (secure 0 0 0) 0x7f68cc05d2c0 con 0x7f68e40b4140 2024-04-13T03:15:31.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:31.761 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:30.659837+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-13T03:15:31.761 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 56 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 msgr2=0x7f68c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 0x7f68c406ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009b40 tx=0x7f68d4009190 comp rx=0 tx=0).stop 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 msgr2=0x7f68e4160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f68cc001f00 tx=0x7f68cc00f990 comp rx=0 tx=0).stop 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 shutdown_connections 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c920 0x7f68c406ede0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e414a790 0x7f68e414ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e40b4140 0x7f68e4160440 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 --2- 172.21.15.106:0/746570114 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68d00051c0 0x7f68e414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:31.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 >> 172.21.15.106:0/746570114 conn(0x7f68e408bf10 msgr2=0x7f68e40b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:31.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 shutdown_connections 2024-04-13T03:15:31.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:31.762+0000 7f68eaade640 1 -- 172.21.15.106:0/746570114 wait complete. 2024-04-13T03:15:32.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:15:30.659837+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.106:6837/635681585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 635681585}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 635681585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:15:32.178 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7728:up:rejoin 2024-04-13T03:15:33.178 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:15:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.297+0000 7f53d8d8c640 1 Processor -- start 2024-04-13T03:15:33.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.298+0000 7f53d8d8c640 1 -- start start 2024-04-13T03:15:33.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53d415e9b0 0x7f53d40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f53d415ed90 0x7f53d40d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d40b3d60 0x7f53d40d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f53d40d5070 con 0x7f53d415e9b0 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f53d40cfd20 con 0x7f53d415ed90 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d8d8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f53d4162af0 con 0x7f53d40b3d60 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53d415e9b0 0x7f53d40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53d415e9b0 0x7f53d40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54406/0 (socket says 172.21.15.106:54406) 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 -- 172.21.15.106:0/558824586 learned_addr learned my addr 172.21.15.106:0/558824586 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d1d9b640 1 --2- 172.21.15.106:0/558824586 >> v2:172.21.15.114:3300/0 conn(0x7f53d415ed90 0x7f53d40d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d2d9d640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d40b3d60 0x7f53d40d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 -- 172.21.15.106:0/558824586 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d40b3d60 msgr2=0x7f53d40d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d40b3d60 0x7f53d40d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 -- 172.21.15.106:0/558824586 >> v2:172.21.15.114:3300/0 conn(0x7f53d415ed90 msgr2=0x7f53d40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:33.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 --2- 172.21.15.106:0/558824586 >> v2:172.21.15.114:3300/0 conn(0x7f53d415ed90 0x7f53d40d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.299+0000 7f53d259c640 1 -- 172.21.15.106:0/558824586 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4162dd0 con 0x7f53d415e9b0 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.300+0000 7f53d259c640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d415e9b0 0x7f53d40d2300 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f53d408ebd0 tx=0x7f53c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5adc410ef99140 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d159a640 1 -- 172.21.15.106:0/558824586 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8034070 con 0x7f53d415e9b0 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d159a640 1 -- 172.21.15.106:0/558824586 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c8030860 con 0x7f53d415e9b0 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d159a640 1 -- 172.21.15.106:0/558824586 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8030c30 con 0x7f53d415e9b0 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 -- 172.21.15.106:0/558824586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d415e9b0 msgr2=0x7f53d40d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d415e9b0 0x7f53d40d2300 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f53d408ebd0 tx=0x7f53c802b9a0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 -- 172.21.15.106:0/558824586 shutdown_connections 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d40b3d60 0x7f53d40d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/558824586 >> v2:172.21.15.114:3300/0 conn(0x7f53d415ed90 0x7f53d40d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/558824586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d415e9b0 0x7f53d40d2300 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 -- 172.21.15.106:0/558824586 >> 172.21.15.106:0/558824586 conn(0x7f53d408bf10 msgr2=0x7f53d40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.301+0000 7f53d8d8c640 1 -- 172.21.15.106:0/558824586 shutdown_connections 2024-04-13T03:15:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.302+0000 7f53d8d8c640 1 -- 172.21.15.106:0/558824586 wait complete. 2024-04-13T03:15:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.303+0000 7f53d8d8c640 1 Processor -- start 2024-04-13T03:15:33.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.303+0000 7f53d8d8c640 1 -- start start 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d415e9b0 0x7f53d41486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d4161a90 0x7f53d414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4162680 con 0x7f53d40b3d60 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4163390 con 0x7f53d4161a90 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d8d8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f53d4162800 con 0x7f53d415e9b0 2024-04-13T03:15:33.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54408/0 (socket says 172.21.15.106:54408) 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d259c640 1 -- 172.21.15.106:0/3863286221 learned_addr learned my addr 172.21.15.106:0/3863286221 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d1d9b640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d415e9b0 0x7f53d41486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.304+0000 7f53d2d9d640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d4161a90 0x7f53d414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 -- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d415e9b0 msgr2=0x7f53d41486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d415e9b0 0x7f53d41486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 -- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d4161a90 msgr2=0x7f53d414add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d4161a90 0x7f53d414add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 -- 172.21.15.106:0/3863286221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d40b6e30 con 0x7f53d40b3d60 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d259c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000c00 tx=0x7f53c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8048070 con 0x7f53d40b3d60 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c8030660 con 0x7f53d40b3d60 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d40cc630 con 0x7f53d40b3d60 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8034040 con 0x7f53d40b3d60 2024-04-13T03:15:33.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.305+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d40ccbf0 con 0x7f53d40b3d60 2024-04-13T03:15:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.306+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d40cc440 con 0x7f53d40b3d60 2024-04-13T03:15:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.307+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53c804b070 con 0x7f53d40b3d60 2024-04-13T03:15:33.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.308+0000 7f53c2ffd640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.308+0000 7f53d1d9b640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.308+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f53c80ade20 con 0x7f53d40b3d60 2024-04-13T03:15:33.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.308+0000 7f53d1d9b640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f53d41f83b0 tx=0x7f53c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:33.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.312+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53c807ba20 con 0x7f53d40b3d60 2024-04-13T03:15:33.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.525+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d4068a10 con 0x7f53d40b3d60 2024-04-13T03:15:33.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.526+0000 7f53c2ffd640 1 -- 172.21.15.106:0/3863286221 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5193 (secure 0 0 0) 0x7f53c807fd20 con 0x7f53d40b3d60 2024-04-13T03:15:33.527 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:15:33.527 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:32.666909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7728,"qdb_cluster":[7728]},"id":6}]} 2024-04-13T03:15:33.527 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 57 2024-04-13T03:15:33.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.527+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 msgr2=0x7f53b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.527+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f53d41f83b0 tx=0x7f53c4008040 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.527+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 msgr2=0x7f53d40cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.527+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000c00 tx=0x7f53c8004730 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 shutdown_connections 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53d415e9b0 0x7f53d41486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d4161a90 0x7f53d414add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 --2- 172.21.15.106:0/3863286221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53d40b3d60 0x7f53d40cd110 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 >> 172.21.15.106:0/3863286221 conn(0x7f53d408bf10 msgr2=0x7f53d40b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 shutdown_connections 2024-04-13T03:15:33.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:33.528+0000 7f53d8d8c640 1 -- 172.21.15.106:0/3863286221 wait complete. 2024-04-13T03:15:33.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:15:32.666909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.106:6837/635681585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 635681585}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 635681585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7728, 'qdb_cluster': [7728]} 2024-04-13T03:15:33.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:15:33.958 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-13T03:15:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.064+0000 7f87555cf640 1 Processor -- start 2024-04-13T03:15:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.064+0000 7f87555cf640 1 -- start start 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f87555cf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f875015e9b0 0x7f87500d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f87555cf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f87500d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f87555cf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87500cfd20 con 0x7f875015ed90 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f87555cf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8750162af0 con 0x7f875015e9b0 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f87555cf640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8750162dd0 con 0x7f87500b3d60 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f875015e9b0 0x7f87500d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f875015e9b0 0x7f87500d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54414/0 (socket says 172.21.15.106:54414) 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874e7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f875015ed90 0x7f87500d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53506/0 (socket says 172.21.15.106:53506) 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 -- 172.21.15.106:0/1314651760 learned_addr learned my addr 172.21.15.106:0/1314651760 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874f7fe640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f87500d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 -- 172.21.15.106:0/1314651760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 msgr2=0x7f87500d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f87500d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 -- 172.21.15.106:0/1314651760 >> v1:172.21.15.106:6789/0 conn(0x7f875015ed90 legacy=0x7f87500d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874effd640 1 -- 172.21.15.106:0/1314651760 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87501630b0 con 0x7f875015e9b0 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.065+0000 7f874dffb640 1 -- 172.21.15.106:0/1314651760 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350132180 0 0) 0x7f87500cfd20 con 0x7f875015ed90 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.066+0000 7f874effd640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87500d2300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f874002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d7b5d06866a3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.066+0000 7f874dffb640 1 -- 172.21.15.106:0/1314651760 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874002f840 con 0x7f875015e9b0 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.066+0000 7f874dffb640 1 -- 172.21.15.106:0/1314651760 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874002f9e0 con 0x7f875015e9b0 2024-04-13T03:15:34.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.066+0000 7f874dffb640 1 -- 172.21.15.106:0/1314651760 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874002fdb0 con 0x7f875015e9b0 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 -- 172.21.15.106:0/1314651760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 msgr2=0x7f87500d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87500d2300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f874002ba20 comp rx=0 tx=0).stop 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 -- 172.21.15.106:0/1314651760 shutdown_connections 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f87500d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 --2- 172.21.15.106:0/1314651760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87500d2300 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 -- 172.21.15.106:0/1314651760 >> 172.21.15.106:0/1314651760 conn(0x7f875008bf10 msgr2=0x7f87500b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 -- 172.21.15.106:0/1314651760 shutdown_connections 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.067+0000 7f87555cf640 1 -- 172.21.15.106:0/1314651760 wait complete. 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.068+0000 7f87555cf640 1 Processor -- start 2024-04-13T03:15:34.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.068+0000 7f87555cf640 1 -- start start 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f875014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 0x7f87501f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8750162800 con 0x7f875015e9b0 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8750162680 con 0x7f87501f7fc0 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f87555cf640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f87500d4d00 con 0x7f87500b3d60 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54428/0 (socket says 172.21.15.106:54428) 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 -- 172.21.15.106:0/4007630354 learned_addr learned my addr 172.21.15.106:0/4007630354 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874effd640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f875014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874f7fe640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 0x7f87501f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 -- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 msgr2=0x7f875014bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f875014bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 -- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 msgr2=0x7f87501f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 0x7f87501f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 -- 172.21.15.106:0/4007630354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87501f8a60 con 0x7f875015e9b0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874f7fe640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 0x7f87501f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.069+0000 7f874e7fc640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f874400c430 tx=0x7f874400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.070+0000 7f87377fe640 1 -- 172.21.15.106:0/4007630354 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744017070 con 0x7f875015e9b0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.070+0000 7f87377fe640 1 -- 172.21.15.106:0/4007630354 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87440074e0 con 0x7f875015e9b0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.070+0000 7f87377fe640 1 -- 172.21.15.106:0/4007630354 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440059e0 con 0x7f875015e9b0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.070+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87501f8d40 con 0x7f875015e9b0 2024-04-13T03:15:34.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.070+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875014b440 con 0x7f875015e9b0 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.071+0000 7f87377fe640 1 -- 172.21.15.106:0/4007630354 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f874401a070 con 0x7f875015e9b0 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.072+0000 7f87377fe640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 0x7f873c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.072+0000 7f874effd640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 0x7f873c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.072+0000 7f87377fe640 1 -- 172.21.15.106:0/4007630354 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f874408b180 con 0x7f875015e9b0 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.072+0000 7f874effd640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 0x7f873c06ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8740036000 tx=0x7f8740008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.073+0000 7f87583bd740 1 Processor -- start 2024-04-13T03:15:34.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.073+0000 7f87583bd740 1 -- start start 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76ade0 0x55cb8f73dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f76db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f6a8990 con 0x55cb8f73e2c0 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f64f620 con 0x55cb8f76b1c0 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cb8f6a4950 con 0x55cb8f76ade0 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54442/0 (socket says 172.21.15.106:54442) 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 -- 172.21.15.106:0/2635795796 learned_addr learned my addr 172.21.15.106:0/2635795796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 -- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76ade0 msgr2=0x55cb8f73dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76ade0 0x55cb8f73dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 -- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 msgr2=0x55cb8f76db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f76db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 -- 172.21.15.106:0/2635795796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8f3bb2d0 con 0x55cb8f73e2c0 2024-04-13T03:15:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f874f7fe640 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8748007410 tx=0x7f87480078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbb8c51a7069ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f8734ff9640 1 -- 172.21.15.106:0/2635795796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874801a070 con 0x55cb8f73e2c0 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f8734ff9640 1 -- 172.21.15.106:0/2635795796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8748003da0 con 0x55cb8f73e2c0 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.074+0000 7f8734ff9640 1 -- 172.21.15.106:0/2635795796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748004230 con 0x55cb8f73e2c0 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 -- 172.21.15.106:0/2635795796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 msgr2=0x55cb8f76e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8748007410 tx=0x7f87480078d0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 -- 172.21.15.106:0/2635795796 shutdown_connections 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f73e2c0 0x55cb8f76e0b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f76db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 --2- 172.21.15.106:0/2635795796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76ade0 0x55cb8f73dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 -- 172.21.15.106:0/2635795796 >> 172.21.15.106:0/2635795796 conn(0x55cb8f73fff0 msgr2=0x55cb8f788b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 -- 172.21.15.106:0/2635795796 shutdown_connections 2024-04-13T03:15:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.075+0000 7f87583bd740 1 -- 172.21.15.106:0/2635795796 wait complete. 2024-04-13T03:15:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 -- start start 2024-04-13T03:15:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 Processor -- start 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f781a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f781f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f742590 con 0x55cb8f76ade0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f6629e0 con 0x55cb8f76b1c0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cb8f741980 con 0x55cb8f76eb60 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.076+0000 7f87583bd740 10 client.0 _ll_register_callbacks cb 0x55cb8f69e220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54454/0 (socket says 172.21.15.106:54454) 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 -- 172.21.15.106:0/622463561 learned_addr learned my addr 172.21.15.106:0/622463561 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874e7fc640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f781a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874f7fe640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f781f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 -- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 msgr2=0x55cb8f781f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f781f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 -- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 msgr2=0x55cb8f781a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f781a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 -- 172.21.15.106:0/622463561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8f4856b0 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f874effd640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f87400090b0 tx=0x7f87400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a200c6bf3f20d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740051060 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8740040410 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8748003da0 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8740057050 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f87583bd740 10 client.7797 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55cb8f7702d0 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8748003da0 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.077+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f874005a020 con 0x55cb8f76ade0 2024-04-13T03:15:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 10 client.7797 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 10 client.7797 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 20 client.7797 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f7807f0 0x55cb8f7a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 4 client.7797 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb8f6b63b0 con 0x55cb8f7807f0 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f874e7fc640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f7807f0 0x55cb8f7a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f872dffb640 10 client.7797 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.078+0000 7f874e7fc640 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f7807f0 0x55cb8f7a75e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b0ddb0c894beb6 server_cookie=b75a64b641ad4985 in_seq=0 out_seq=0 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f872dffb640 1 -- 172.21.15.106:0/622463561 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb8f6b63b0 con 0x55cb8f7807f0 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f872dffb640 10 client.7797 handle_command_reply: tid=0 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 client.7797 shutdown 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 msgr2=0x55cb8f75e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f87400090b0 tx=0x7f87400046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 shutdown_connections 2024-04-13T03:15:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f7807f0 0x55cb8f7a75e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f781f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f872dffb640 0 client.7797 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f781a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 --2- 172.21.15.106:0/622463561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f75e130 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 >> 172.21.15.106:0/622463561 conn(0x55cb8f73e2c0 msgr2=0x55cb8f76db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 shutdown_connections 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 1 -- 172.21.15.106:0/622463561 wait complete. 2024-04-13T03:15:34.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.079+0000 7f87583bd740 20 client.7797 trim_cache size 0 max 16384 2024-04-13T03:15:34.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.097+0000 7f87583bd740 1 Processor -- start 2024-04-13T03:15:34.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.097+0000 7f87583bd740 1 -- start start 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f73cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f76dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8750201530 con 0x55cb8f76ade0 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8750200550 con 0x55cb8f76b1c0 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8750149db0 con 0x55cb8f76eb60 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54464/0 (socket says 172.21.15.106:54464) 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 -- 172.21.15.106:0/2845351311 learned_addr learned my addr 172.21.15.106:0/2845351311 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874e7fc640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f73cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874f7fe640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f76dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 -- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 msgr2=0x55cb8f76dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f76dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 -- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 msgr2=0x55cb8f73cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f73cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874effd640 1 -- 172.21.15.106:0/2845351311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8f73da50 con 0x55cb8f76ade0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874e7fc640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f73cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.098+0000 7f874f7fe640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f76dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f874effd640 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f874002f7d0 tx=0x7f8740049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57f84b3d7c79175 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f872ffff640 1 -- 172.21.15.106:0/2845351311 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740040610 con 0x55cb8f76ade0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f872ffff640 1 -- 172.21.15.106:0/2845351311 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8740004040 con 0x55cb8f76ade0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f872ffff640 1 -- 172.21.15.106:0/2845351311 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87400043f0 con 0x55cb8f76ade0 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 -- 172.21.15.106:0/2845351311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 msgr2=0x55cb8f744170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f874002f7d0 tx=0x7f8740049000 comp rx=0 tx=0).stop 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 -- 172.21.15.106:0/2845351311 shutdown_connections 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76eb60 0x55cb8f76dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76b1c0 0x55cb8f73cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 --2- 172.21.15.106:0/2845351311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f76ade0 0x55cb8f744170 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 -- 172.21.15.106:0/2845351311 >> 172.21.15.106:0/2845351311 conn(0x55cb8f73e2c0 msgr2=0x55cb8f7418e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 -- 172.21.15.106:0/2845351311 shutdown_connections 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.099+0000 7f87583bd740 1 -- 172.21.15.106:0/2845351311 wait complete. 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.100+0000 7f87583bd740 1 -- start start 2024-04-13T03:15:34.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.100+0000 7f87583bd740 1 Processor -- start 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76b1c0 0x55cb8f7ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76eb60 0x55cb8f79b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f781090 con 0x55cb8f7430a0 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8750162000 con 0x55cb8f76eb60 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55cb8f769320 con 0x55cb8f76b1c0 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 10 client.0 _ll_register_callbacks cb 0x7f87400096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54480/0 (socket says 172.21.15.106:54480) 2024-04-13T03:15:34.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 -- 172.21.15.106:0/3425246194 learned_addr learned my addr 172.21.15.106:0/3425246194 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874effd640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76b1c0 0x55cb8f7ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874e7fc640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76eb60 0x55cb8f79b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 -- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76b1c0 msgr2=0x55cb8f7ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76b1c0 0x55cb8f7ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 -- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76eb60 msgr2=0x55cb8f79b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76eb60 0x55cb8f79b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 -- 172.21.15.106:0/3425246194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8f79bbf0 con 0x55cb8f7430a0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f874f7fe640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8748009f70 tx=0x7f8748004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd4946f9d68979a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f872dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748026070 con 0x55cb8f7430a0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8748015920 con 0x55cb8f7430a0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.101+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8740004040 con 0x55cb8f7430a0 2024-04-13T03:15:34.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8748031050 con 0x55cb8f7430a0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 10 client.7803 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55cb8f770b90 con 0x55cb8f7430a0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8740004040 con 0x55cb8f7430a0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f8748034020 con 0x55cb8f7430a0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 10 client.7803 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 10 client.7803 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 20 client.7803 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f77e500 0x55cb8f77e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 4 client.7803 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "status"}) -- 0x55cb8f6b0220 con 0x55cb8f77e500 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f874effd640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f77e500 0x55cb8f77e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 10 client.7803 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f874effd640 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f77e500 0x55cb8f77e8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183645b7c55da9ad server_cookie=23291676b05fc91e in_seq=0 out_seq=0 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 1 -- 172.21.15.106:0/3425246194 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cb8f6b0220 con 0x55cb8f77e500 2024-04-13T03:15:34.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f872dffb640 10 client.7803 handle_command_reply: tid=0 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.102+0000 7f87583bd740 1 client.7803 shutdown 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 msgr2=0x55cb8f7435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8748009f70 tx=0x7f8748004ab0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 shutdown_connections 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55cb8f77e500 0x55cb8f77e8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb8f7430a0 0x55cb8f7435f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8f76eb60 0x55cb8f79b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f872dffb640 0 client.7803 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 --2- 172.21.15.106:0/3425246194 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55cb8f76b1c0 0x55cb8f7ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 >> 172.21.15.106:0/3425246194 conn(0x55cb8f73e2c0 msgr2=0x55cb8f743f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 shutdown_connections 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 1 -- 172.21.15.106:0/3425246194 wait complete. 2024-04-13T03:15:34.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87583bd740 20 client.7803 trim_cache size 0 max 16384 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 7728, 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.4443831300000003, 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 57, 2024-04-13T03:15:34.111 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 92, 2024-04-13T03:15:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T03:15:34.112 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 5.4144482030000001 2024-04-13T03:15:34.112 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:34.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 msgr2=0x7f873c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 0x7f873c06ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8740036000 tx=0x7f8740008030 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 msgr2=0x7f87501487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f874400c430 tx=0x7f874400c8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 shutdown_connections 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f873c06c920 0x7f873c06ede0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87501f7fc0 0x7f87501f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f875015e9b0 0x7f87501487c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 --2- 172.21.15.106:0/4007630354 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f87500b3d60 0x7f875014bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 >> 172.21.15.106:0/4007630354 conn(0x7f875008bf10 msgr2=0x7f87500b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 shutdown_connections 2024-04-13T03:15:34.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.103+0000 7f87555cf640 1 -- 172.21.15.106:0/4007630354 wait complete. 2024-04-13T03:15:34.115 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:15:34.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.213+0000 7f98c478c640 1 Processor -- start 2024-04-13T03:15:34.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.213+0000 7f98c478c640 1 -- start start 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c478c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 0x7f98bc15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c478c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc158c50 con 0x7f98bc15f460 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c478c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc0b3800 con 0x7f98bc158870 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c478c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc15ea90 con 0x7f98bc0b32d0 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c2d29640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f98bc15f460 0x7f98bc15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53508/0 (socket says 172.21.15.106:53508) 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c2d29640 1 -- 172.21.15.106:0/601499997 learned_addr learned my addr 172.21.15.106:0/601499997 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1d27640 1 --2- 172.21.15.106:0/601499997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 0x7f98bc15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763291777 0 0) 0x7f98bc158c50 con 0x7f98bc15f460 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98a8003520 con 0x7f98bc15f460 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301603816 0 0) 0x7f98bc0b3800 con 0x7f98bc158870 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98bc158c50 con 0x7f98bc158870 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1d27640 1 -- 172.21.15.106:0/601499997 >> v1:172.21.15.114:6789/0 conn(0x7f98bc158870 legacy=0x7f98bc06faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1d27640 1 -- 172.21.15.106:0/601499997 >> v1:172.21.15.106:6789/0 conn(0x7f98bc15f460 legacy=0x7f98bc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.214+0000 7f98c1d27640 1 -- 172.21.15.106:0/601499997 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc0b3ae0 con 0x7f98bc0b32d0 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.215+0000 7f98c1d27640 1 --2- 172.21.15.106:0/601499997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 0x7f98bc15ef20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00a970 tx=0x7f98ac02bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f560afeae7fbe8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.215+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac039070 con 0x7f98bc0b32d0 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.215+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac02ca10 con 0x7f98bc0b32d0 2024-04-13T03:15:34.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.215+0000 7f98c1526640 1 -- 172.21.15.106:0/601499997 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00b040 con 0x7f98bc0b32d0 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 -- 172.21.15.106:0/601499997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 msgr2=0x7f98bc15ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 --2- 172.21.15.106:0/601499997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 0x7f98bc15ef20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00a970 tx=0x7f98ac02bc60 comp rx=0 tx=0).stop 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 -- 172.21.15.106:0/601499997 shutdown_connections 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 --2- 172.21.15.106:0/601499997 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc0b32d0 0x7f98bc15ef20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 -- 172.21.15.106:0/601499997 >> 172.21.15.106:0/601499997 conn(0x7f98bc0d0340 msgr2=0x7f98bc0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 -- 172.21.15.106:0/601499997 shutdown_connections 2024-04-13T03:15:34.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.216+0000 7f98c478c640 1 -- 172.21.15.106:0/601499997 wait complete. 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.217+0000 7f98c478c640 1 Processor -- start 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.217+0000 7f98c478c640 1 -- start start 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98bc0b32d0 0x7f98bc1f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc15f460 0x7f98bc06f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc15bb00 con 0x7f98bc158870 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc15b980 con 0x7f98bc0b32d0 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c478c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc06b8e0 con 0x7f98bc15f460 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54494/0 (socket says 172.21.15.106:54494) 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 -- 172.21.15.106:0/1646433964 learned_addr learned my addr 172.21.15.106:0/1646433964 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c2d29640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc15f460 0x7f98bc06f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c2528640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98bc0b32d0 0x7f98bc1f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 -- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc15f460 msgr2=0x7f98bc06f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc15f460 0x7f98bc06f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 -- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98bc0b32d0 msgr2=0x7f98bc1f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98bc0b32d0 0x7f98bc1f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 -- 172.21.15.106:0/1646433964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc1f8440 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98c1d27640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00a970 tx=0x7f98ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.218+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac039040 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.219+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac044070 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.219+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1f8720 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.219+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac035d10 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.219+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc155d10 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.220+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc068a10 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.220+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98ac049460 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.220+0000 7f98b2ffd640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.221+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f98ac0adbd0 con 0x7f98bc158870 2024-04-13T03:15:34.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.221+0000 7f98c2528640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:34.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.221+0000 7f98c2528640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 0x7f98a006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007fd0 tx=0x7f98b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:34.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.225+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98ac07b860 con 0x7f98bc158870 2024-04-13T03:15:34.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.436+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98bc15b560 con 0x7f98bc158870 2024-04-13T03:15:34.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.436+0000 7f98b2ffd640 1 -- 172.21.15.106:0/1646433964 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7f98ac07fb60 con 0x7f98bc158870 2024-04-13T03:15:34.437 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 msgr2=0x7f98a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 0x7f98a006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007fd0 tx=0x7f98b8009040 comp rx=0 tx=0).stop 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 msgr2=0x7f98bc1f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00a970 tx=0x7f98ac0046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 shutdown_connections 2024-04-13T03:15:34.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f98a006c8f0 0x7f98a006edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98bc15f460 0x7f98bc06f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98bc158870 0x7f98bc1f3d40 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 --2- 172.21.15.106:0/1646433964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98bc0b32d0 0x7f98bc1f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 >> 172.21.15.106:0/1646433964 conn(0x7f98bc0d0340 msgr2=0x7f98bc0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 shutdown_connections 2024-04-13T03:15:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34.438+0000 7f98c478c640 1 -- 172.21.15.106:0/1646433964 wait complete. 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:15:34.450 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:34.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.450 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:15:34.466 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42215sec preferred_lft 42215sec 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:15:34.467 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:15:34.468 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:15:34.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.469 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:15:34.531 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:15:34.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.533 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:15:34.589 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:15:34.589 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:15:34.589 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:15:34.589 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:15:34.589 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:15:34.589 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:15:34.589 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:15:34.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:15:34.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:15:34.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:15:34.590 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:15:34.660 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:15:34.659+0000 7f8e6d3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:15:34.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:15:34.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:34.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.725 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:15:34.779 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:15:34.780 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:15:34.780 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:34.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.780 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:15:34.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:34.847 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:34.901 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:15:34.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:15:34.902 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:34.956 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:15:34.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:34.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:35.041 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:35.041 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:35.050 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:15:35.051 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:15:35.052 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:15:35.053 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:15:35.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:35.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:15:35.079 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:15:35.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:15:35.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:15:35.175 INFO:teuthology.orchestra.run.smithi106.stdout:client7812 2024-04-13T03:15:35.175 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:15:35.175 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:15:35.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:35.191 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-13T03:15:35.247 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:15:35.246+0000 7f8e693e9640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-13T03:15:35.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:15:35.248 INFO:teuthology.orchestra.run.smithi106.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-13T03:15:35.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:35.248 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-13T03:15:35.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.392+0000 7f719d384640 1 Processor -- start 2024-04-13T03:15:35.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.392+0000 7f719d384640 1 -- start start 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f719d384640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f71980d3490 0x7f71980d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f719d384640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980b3920 0x7f719815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f719d384640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719815c870 con 0x7f71980d2370 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f719d384640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f71980b1b00 con 0x7f71980d3490 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f719d384640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f71980d5420 con 0x7f71980b3920 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f71980d3490 0x7f71980d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f71977fe640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980b3920 0x7f719815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f71980d3490 0x7f71980d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:54508/0 (socket says 172.21.15.106:54508) 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f718ffff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f71980d2370 0x7f71980b13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:34742/0 (socket says 172.21.15.106:34742) 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 -- 172.21.15.106:0/605324226 learned_addr learned my addr 172.21.15.106:0/605324226 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 -- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980b3920 msgr2=0x7f719815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 --2- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980b3920 0x7f719815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 -- 172.21.15.106:0/605324226 >> v1:172.21.15.114:6789/0 conn(0x7f71980d2370 legacy=0x7f71980b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.393+0000 7f7196ffd640 1 -- 172.21.15.106:0/605324226 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719815ee70 con 0x7f71980d3490 2024-04-13T03:15:35.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.394+0000 7f7196ffd640 1 --2- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980d3490 0x7f71980d3870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71880098a0 tx=0x7f718802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0b8c0dba030a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.394+0000 7f71967fc640 1 -- 172.21.15.106:0/605324226 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188035020 con 0x7f71980d3490 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.394+0000 7f71967fc640 1 -- 172.21.15.106:0/605324226 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7188031640 con 0x7f71980d3490 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.394+0000 7f71967fc640 1 -- 172.21.15.106:0/605324226 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718802d040 con 0x7f71980d3490 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 -- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980d3490 msgr2=0x7f71980d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 --2- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980d3490 0x7f71980d3870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71880098a0 tx=0x7f718802b8f0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 -- 172.21.15.106:0/605324226 shutdown_connections 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 --2- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980b3920 0x7f719815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 --2- 172.21.15.106:0/605324226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980d3490 0x7f71980d3870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 -- 172.21.15.106:0/605324226 >> 172.21.15.106:0/605324226 conn(0x7f71980b9760 msgr2=0x7f71980b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 -- 172.21.15.106:0/605324226 shutdown_connections 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.395+0000 7f719d384640 1 -- 172.21.15.106:0/605324226 wait complete. 2024-04-13T03:15:35.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.396+0000 7f719d384640 1 Processor -- start 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.396+0000 7f719d384640 1 -- start start 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980b3920 0x7f719815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 0x7f71980b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71980d2ae0 con 0x7f71980b6910 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71980d2800 con 0x7f71980b3920 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f719d384640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f71980d2dc0 con 0x7f71980d3490 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54500/0 (socket says 172.21.15.106:54500) 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 -- 172.21.15.106:0/2139580747 learned_addr learned my addr 172.21.15.106:0/2139580747 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980b3920 0x7f719815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f718ffff640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 0x7f71980b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 -- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 msgr2=0x7f71980b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 0x7f71980b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 -- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980b3920 msgr2=0x7f719815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980b3920 0x7f719815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 -- 172.21.15.106:0/2139580747 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719806c9b0 con 0x7f71980b6910 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f718ffff640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 0x7f71980b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f71977fe640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f719000efc0 tx=0x7f719000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.397+0000 7f718f7fe640 1 -- 172.21.15.106:0/2139580747 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190018020 con 0x7f71980b6910 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.398+0000 7f718f7fe640 1 -- 172.21.15.106:0/2139580747 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7190014480 con 0x7f71980b6910 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.398+0000 7f718f7fe640 1 -- 172.21.15.106:0/2139580747 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190014870 con 0x7f71980b6910 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.398+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719806cc90 con 0x7f71980b6910 2024-04-13T03:15:35.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.398+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719806d250 con 0x7f71980b6910 2024-04-13T03:15:35.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.399+0000 7f718f7fe640 1 -- 172.21.15.106:0/2139580747 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7190005d10 con 0x7f71980b6910 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.399+0000 7f718f7fe640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 0x7f717806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.399+0000 7f718f7fe640 1 -- 172.21.15.106:0/2139580747 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f719008ca40 con 0x7f71980b6910 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.399+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 0x7f717806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.400+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 0x7f717806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7188036000 tx=0x7f718803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.400+0000 7f71a0168740 1 Processor -- start 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.400+0000 7f71a0168740 1 -- start start 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea89540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea59c90 0x5634aea89a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634ae982200 con 0x5634aea867b0 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5634ae9c3d60 con 0x5634aea86b90 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5634ae960a10 con 0x5634aea59c90 2024-04-13T03:15:35.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54514/0 (socket says 172.21.15.106:54514) 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f7196ffd640 1 -- 172.21.15.106:0/2738953400 learned_addr learned my addr 172.21.15.106:0/2738953400 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f718ffff640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea89540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.401+0000 7f71977fe640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea59c90 0x5634aea89a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 -- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea59c90 msgr2=0x5634aea89a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea59c90 0x5634aea89a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 -- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 msgr2=0x5634aea89540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea89540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 -- 172.21.15.106:0/2738953400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ae98a930 con 0x5634aea867b0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f718ffff640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea89540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f7196ffd640 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f718802bf00 tx=0x7f7188030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664f27a1142a88ba server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f718cff9640 1 -- 172.21.15.106:0/2738953400 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188035040 con 0x5634aea867b0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f718cff9640 1 -- 172.21.15.106:0/2738953400 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718803a420 con 0x5634aea867b0 2024-04-13T03:15:35.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f718cff9640 1 -- 172.21.15.106:0/2738953400 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718803a7f0 con 0x5634aea867b0 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f71a0168740 1 -- 172.21.15.106:0/2738953400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 msgr2=0x5634aea59750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.402+0000 7f71a0168740 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f718802bf00 tx=0x7f7188030000 comp rx=0 tx=0).stop 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 -- 172.21.15.106:0/2738953400 shutdown_connections 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea59c90 0x5634aea89a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea89540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 --2- 172.21.15.106:0/2738953400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea59750 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 -- 172.21.15.106:0/2738953400 >> 172.21.15.106:0/2738953400 conn(0x5634aea5b9c0 msgr2=0x5634aeaa4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 -- 172.21.15.106:0/2738953400 shutdown_connections 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 -- 172.21.15.106:0/2738953400 wait complete. 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 -- start start 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.403+0000 7f71a0168740 1 Processor -- start 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea9eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aea78310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634ae95e6e0 con 0x5634aea867b0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5634ae9bfd20 con 0x5634aea86b90 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5634aea5df60 con 0x5634aea9d9d0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71a0168740 10 client.0 _ll_register_callbacks cb 0x5634ae9d0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54524/0 (socket says 172.21.15.106:54524) 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 -- 172.21.15.106:0/1563644055 learned_addr learned my addr 172.21.15.106:0/1563644055 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f718ffff640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea9eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71977fe640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aea78310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 -- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 msgr2=0x5634aea78310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aea78310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 -- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 msgr2=0x5634aea9eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea9eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f7196ffd640 1 -- 172.21.15.106:0/1563644055 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ae7a27b0 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.404+0000 7f71977fe640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aea78310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7196ffd640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f71880099c0 tx=0x7f7188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd3fdfac63f9c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718803ae30 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f7188057430 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718803a420 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f7188064050 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 10 client.7821 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x5634ae802f40 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718803a420 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f7188067020 con 0x5634aea867b0 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 10 client.7821 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:35.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 10 client.7821 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 20 client.7821 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aea8c700 0x5634aea8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 4 client.7821 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634ae9d7d10 con 0x5634aea8c700 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.405+0000 7f718ffff640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aea8c700 0x5634aea8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f718ffff640 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aea8c700 0x5634aea8cae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765ea7e4a74878dc server_cookie=fb605ff47af9ae29 in_seq=0 out_seq=0 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f7175ffb640 10 client.7821 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f7175ffb640 1 -- 172.21.15.106:0/1563644055 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634ae9d7d10 con 0x5634aea8c700 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f7175ffb640 10 client.7821 handle_command_reply: tid=0 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 client.7821 shutdown 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 msgr2=0x5634aea77b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f71880099c0 tx=0x7f7188004730 comp rx=0 tx=0).stop 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 shutdown_connections 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aea8c700 0x5634aea8cae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aea78310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aea9eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f7175ffb640 0 client.7821 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 --2- 172.21.15.106:0/1563644055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aea77b70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 >> 172.21.15.106:0/1563644055 conn(0x5634aea59c90 msgr2=0x5634aea89530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 shutdown_connections 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 1 -- 172.21.15.106:0/1563644055 wait complete. 2024-04-13T03:15:35.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.406+0000 7f71a0168740 20 client.7821 trim_cache size 0 max 16384 2024-04-13T03:15:35.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 Processor -- start 2024-04-13T03:15:35.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 -- start start 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 0x5634aeb18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea86b90 0x5634aea88f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7198201650 con 0x5634aea9d9d0 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7198200630 con 0x5634aea867b0 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f71980cdf50 con 0x5634aea86b90 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.424+0000 7f71977fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54536/0 (socket says 172.21.15.106:54536) 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 -- 172.21.15.106:0/1428000859 learned_addr learned my addr 172.21.15.106:0/1428000859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f718ffff640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea86b90 0x5634aea88f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f7196ffd640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 0x5634aeb18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 -- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea86b90 msgr2=0x5634aea88f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea86b90 0x5634aea88f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 -- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 msgr2=0x5634aeb18210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 0x5634aeb18210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 -- 172.21.15.106:0/1428000859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ae6960a0 con 0x5634aea9d9d0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f7196ffd640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 0x5634aeb18210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f71977fe640 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f719008ad90 tx=0x7f719005add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beaab764793a274 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f7177fff640 1 -- 172.21.15.106:0/1428000859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190072020 con 0x5634aea9d9d0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f7177fff640 1 -- 172.21.15.106:0/1428000859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7190063170 con 0x5634aea9d9d0 2024-04-13T03:15:35.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.425+0000 7f7177fff640 1 -- 172.21.15.106:0/1428000859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190056c00 con 0x5634aea9d9d0 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- 172.21.15.106:0/1428000859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 msgr2=0x5634aea894c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f719008ad90 tx=0x7f719005add0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- 172.21.15.106:0/1428000859 shutdown_connections 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea86b90 0x5634aea88f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea9d9d0 0x5634aea894c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 --2- 172.21.15.106:0/1428000859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea867b0 0x5634aeb18210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- 172.21.15.106:0/1428000859 >> 172.21.15.106:0/1428000859 conn(0x5634aea59c90 msgr2=0x5634aea9bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- 172.21.15.106:0/1428000859 shutdown_connections 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- 172.21.15.106:0/1428000859 wait complete. 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 -- start start 2024-04-13T03:15:35.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.426+0000 7f71a0168740 1 Processor -- start 2024-04-13T03:15:35.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aeaa4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aeac5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7198200010 con 0x5634aea867b0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5634aea5b120 con 0x5634aea86b90 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5634aeb1fa00 con 0x5634aea9d9d0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71a0168740 10 client.0 _ll_register_callbacks cb 0x7f71880317a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54542/0 (socket says 172.21.15.106:54542) 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 -- 172.21.15.106:0/3395181762 learned_addr learned my addr 172.21.15.106:0/3395181762 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f718ffff640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aeaa4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f71977fe640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aeac5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 -- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 msgr2=0x5634aeac5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aeac5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 -- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 msgr2=0x5634aeaa4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aeaa4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 -- 172.21.15.106:0/3395181762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634aeab8820 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7196ffd640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f71880090b0 tx=0x7f7188008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e8989f1156d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.427+0000 7f7175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718806d060 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f7188057c90 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7190063170 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f7188073050 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 10 client.7827 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x5634aeb1f800 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7190063170 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f7188076020 con 0x5634aea867b0 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 10 client.7827 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 10 client.7827 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 20 client.7827 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:35.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aeb152c0 0x5634aeac6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 4 client.7827 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x5634ae9d1780 con 0x5634aeb152c0 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f718ffff640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aeb152c0 0x5634aeac6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f7175ffb640 10 client.7827 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.428+0000 7f718ffff640 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aeb152c0 0x5634aeac6040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0465b517514e2a8 server_cookie=164315b9c382ce41 in_seq=0 out_seq=0 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f7175ffb640 1 -- 172.21.15.106:0/3395181762 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5634ae9d1780 con 0x5634aeb152c0 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f7175ffb640 10 client.7827 handle_command_reply: tid=0 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 client.7827 shutdown 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 msgr2=0x5634aeaa4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f71880090b0 tx=0x7f7188008030 comp rx=0 tx=0).stop 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 shutdown_connections 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5634aeb152c0 0x5634aeac6040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5634aea9d9d0 0x5634aeac5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634aea86b90 0x5634aeaa4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 --2- 172.21.15.106:0/3395181762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634aea867b0 0x5634aeaa4980 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 >> 172.21.15.106:0/3395181762 conn(0x5634aea59c90 msgr2=0x5634aea76820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f7175ffb640 0 client.7827 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 shutdown_connections 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 1 -- 172.21.15.106:0/3395181762 wait complete. 2024-04-13T03:15:35.437 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f8e693e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0) n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5628b7fa4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-13T03:15:35.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.429+0000 7f71a0168740 20 client.7827 trim_cache size 0 max 16384 2024-04-13T03:15:35.438 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:35.438 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:15:35.438 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "c8b94afe-3953-4cf4-a425-184951c1242e", 2024-04-13T03:15:35.438 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:15:35.438 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 msgr2=0x7f717806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 0x7f717806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7188036000 tx=0x7f718803c040 comp rx=0 tx=0).stop 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 msgr2=0x7f71981f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f719000efc0 tx=0x7f719000c490 comp rx=0 tx=0).stop 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 shutdown_connections 2024-04-13T03:15:35.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717806c8f0 0x7f717806edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71980d3490 0x7f71980b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71980b6910 0x7f71981f4700 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 --2- 172.21.15.106:0/2139580747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71980b3920 0x7f719815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 >> 172.21.15.106:0/2139580747 conn(0x7f71980b9760 msgr2=0x7f71980b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 shutdown_connections 2024-04-13T03:15:35.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.430+0000 7f719d384640 1 -- 172.21.15.106:0/2139580747 wait complete. 2024-04-13T03:15:35.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:15:35.441 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:15:35.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.542+0000 7f4fc395e640 1 Processor -- start 2024-04-13T03:15:35.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.542+0000 7f4fc395e640 1 -- start start 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc395e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc395e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fbc070bb0 0x7f4fbc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc395e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fbc0b5cf0 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc395e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4fbc0b5fd0 con 0x7f4fbc070bb0 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc395e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc0b62b0 con 0x7f4fbc0b1c30 2024-04-13T03:15:35.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc1efb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fbc070bb0 0x7f4fbc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc0ef9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4fbc0b54a0 0x7f4fbc0704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53510/0 (socket says 172.21.15.106:53510) 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc0ef9640 1 -- 172.21.15.106:0/3325561675 learned_addr learned my addr 172.21.15.106:0/3325561675 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fb3fff640 1 -- 172.21.15.106:0/3325561675 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170636134 0 0) 0x7f4fbc0b5cf0 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.543+0000 7f4fb3fff640 1 -- 172.21.15.106:0/3325561675 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fa8003520 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fc1efb640 1 -- 172.21.15.106:0/3325561675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 msgr2=0x7f4fbc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fc1efb640 1 -- 172.21.15.106:0/3325561675 >> v1:172.21.15.106:6789/0 conn(0x7f4fbc0b54a0 legacy=0x7f4fbc0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fc1efb640 1 -- 172.21.15.106:0/3325561675 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc0b6590 con 0x7f4fbc070bb0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc06d660 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40097d0 tx=0x7f4fb402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff3404ff63a4c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fb3fff640 1 -- 172.21.15.106:0/3325561675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035020 con 0x7f4fbc070bb0 2024-04-13T03:15:35.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fb3fff640 1 -- 172.21.15.106:0/3325561675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb4031800 con 0x7f4fbc070bb0 2024-04-13T03:15:35.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.544+0000 7f4fb3fff640 1 -- 172.21.15.106:0/3325561675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031bb0 con 0x7f4fbc070bb0 2024-04-13T03:15:35.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 -- 172.21.15.106:0/3325561675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc070bb0 msgr2=0x7f4fbc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc06d660 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40097d0 tx=0x7f4fb402b8a0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 -- 172.21.15.106:0/3325561675 shutdown_connections 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc06d660 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 --2- 172.21.15.106:0/3325561675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 -- 172.21.15.106:0/3325561675 >> 172.21.15.106:0/3325561675 conn(0x7f4fbc071870 msgr2=0x7f4fbc071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 -- 172.21.15.106:0/3325561675 shutdown_connections 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.545+0000 7f4fc395e640 1 -- 172.21.15.106:0/3325561675 wait complete. 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.546+0000 7f4fc395e640 1 Processor -- start 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.546+0000 7f4fc395e640 1 -- start start 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc06dd20 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc06dba0 con 0x7f4fbc070bb0 2024-04-13T03:15:35.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc395e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc06dea0 con 0x7f4fbc0b1c30 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54562/0 (socket says 172.21.15.106:54562) 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 -- 172.21.15.106:0/1349891308 learned_addr learned my addr 172.21.15.106:0/1349891308 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 -- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 msgr2=0x7f4fbc1f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc1f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 -- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc070bb0 msgr2=0x7f4fbc1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc1efb640 1 -- 172.21.15.106:0/1349891308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc154820 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.547+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402beb0 tx=0x7f4fb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fb1ffb640 1 -- 172.21.15.106:0/1349891308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4041070 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fb1ffb640 1 -- 172.21.15.106:0/1349891308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb403ad00 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fb1ffb640 1 -- 172.21.15.106:0/1349891308 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035040 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc1f8640 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.548+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1f5d10 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.550+0000 7f4fb1ffb640 1 -- 172.21.15.106:0/1349891308 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fb4038020 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.550+0000 7f4fb1ffb640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 0x7f4fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.550+0000 7f4fb1ffb640 1 -- 172.21.15.106:0/1349891308 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f4fb40ad860 con 0x7f4fbc0b54a0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.550+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 0x7f4fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 0x7f4fa006edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009a60 tx=0x7f4fb8003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 Processor -- start 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 -- start start 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f19058dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19055e4d0 0x55f19058e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1904c9170 con 0x55f19058aff0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f19046f7f0 con 0x55f19055e4d0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.551+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f1904c5130 con 0x55f19058b3d0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54578/0 (socket says 172.21.15.106:54578) 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 -- 172.21.15.106:0/4083836115 learned_addr learned my addr 172.21.15.106:0/4083836115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 -- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 msgr2=0x55f19058dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f19058dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 -- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19055e4d0 msgr2=0x55f19058e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19055e4d0 0x55f19058e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 -- 172.21.15.106:0/4083836115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1902a57d0 con 0x55f19058aff0 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016470 tx=0x7f4fb8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390094b9c646e989 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4f9effd640 1 -- 172.21.15.106:0/4083836115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8026070 con 0x55f19058aff0 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4f9effd640 1 -- 172.21.15.106:0/4083836115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80147e0 con 0x55f19058aff0 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.552+0000 7f4f9effd640 1 -- 172.21.15.106:0/4083836115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8014b90 con 0x55f19058aff0 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- 172.21.15.106:0/4083836115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 msgr2=0x55f19055df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016470 tx=0x7f4fb8012000 comp rx=0 tx=0).stop 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- 172.21.15.106:0/4083836115 shutdown_connections 2024-04-13T03:15:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f19058dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19055e4d0 0x55f19058e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 --2- 172.21.15.106:0/4083836115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19055df90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- 172.21.15.106:0/4083836115 >> 172.21.15.106:0/4083836115 conn(0x55f190560200 msgr2=0x55f1905a8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- 172.21.15.106:0/4083836115 shutdown_connections 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- 172.21.15.106:0/4083836115 wait complete. 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 -- start start 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.553+0000 7f4fc6829740 1 Processor -- start 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f1905909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 0x55f19057bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1905628a0 con 0x55f19058aff0 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f190482d00 con 0x55f19057baa0 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f1905650e0 con 0x55f19058b3d0 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc6829740 10 client.0 _ll_register_callbacks cb 0x55f1904826f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54584/0 (socket says 172.21.15.106:54584) 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.554+0000 7f4fc16fa640 1 -- 172.21.15.106:0/2956231611 learned_addr learned my addr 172.21.15.106:0/2956231611 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 0x55f19057bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f1905909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 -- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 msgr2=0x55f1905909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f1905909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 -- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 msgr2=0x55f19057bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 0x55f19057bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 -- 172.21.15.106:0/2956231611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1901db3f0 con 0x55f19058aff0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 0x55f19057bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f1905909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016d00 tx=0x7f4fb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f89e11dec2f878 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb802f070 con 0x55f19058aff0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f4fb8014da0 con 0x55f19058aff0 2024-04-13T03:15:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb80147e0 con 0x55f19058aff0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4fb803a050 con 0x55f19058aff0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 10 client.7842 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55f19019b0f0 con 0x55f19058aff0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80147e0 con 0x55f19058aff0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f4fb803d020 con 0x55f19058aff0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 10 client.7842 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 10 client.7842 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 20 client.7842 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f19058f9b0 0x55f19058fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 4 client.7842 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.555+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1904d6b90 con 0x55f19058f9b0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f19058f9b0 0x55f19058fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f19058f9b0 0x55f19058fdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20e3a61bfffca29 server_cookie=cb5864d10327787c in_seq=0 out_seq=0 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4f97fff640 10 client.7842 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4f97fff640 1 -- 172.21.15.106:0/2956231611 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1904d6b90 con 0x55f19058f9b0 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4f97fff640 10 client.7842 handle_command_reply: tid=0 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4fc6829740 1 client.7842 shutdown 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 msgr2=0x55f19057fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.556+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016d00 tx=0x7f4fb8004ab0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 shutdown_connections 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f19058f9b0 0x55f19058fdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19058b3d0 0x55f1905909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19057baa0 0x55f19057bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2956231611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f19057fb30 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4f97fff640 0 client.7842 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 >> 172.21.15.106:0/2956231611 conn(0x55f19055e4d0 msgr2=0x55f190561f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 shutdown_connections 2024-04-13T03:15:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 1 -- 172.21.15.106:0/2956231611 wait complete. 2024-04-13T03:15:35.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.557+0000 7f4fc6829740 20 client.7842 trim_cache size 0 max 16384 2024-04-13T03:15:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.574+0000 7f4fc6829740 1 Processor -- start 2024-04-13T03:15:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.574+0000 7f4fc6829740 1 -- start start 2024-04-13T03:15:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19057baa0 0x55f19055f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f190561d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1905c34e0 con 0x55f19058aff0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc201460 con 0x55f19058b3d0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc200480 con 0x55f19057baa0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54590/0 (socket says 172.21.15.106:54590) 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 -- 172.21.15.106:0/2865436023 learned_addr learned my addr 172.21.15.106:0/2865436023 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19057baa0 0x55f19055f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f190561d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 -- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19057baa0 msgr2=0x55f19055f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19057baa0 0x55f19055f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 -- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 msgr2=0x55f190561d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f190561d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 -- 172.21.15.106:0/2865436023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19057c290 con 0x55f19058aff0 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f60 tx=0x7f4fac018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e449b1c53348ee3f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.575+0000 7f4f9e7fc640 1 -- 172.21.15.106:0/2865436023 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0075d0 con 0x55f19058aff0 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4f9e7fc640 1 -- 172.21.15.106:0/2865436023 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac021440 con 0x55f19058aff0 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4f9e7fc640 1 -- 172.21.15.106:0/2865436023 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac021830 con 0x55f19058aff0 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 -- 172.21.15.106:0/2865436023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 msgr2=0x55f190591650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f60 tx=0x7f4fac018810 comp rx=0 tx=0).stop 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 -- 172.21.15.106:0/2865436023 shutdown_connections 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f190561d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19058aff0 0x55f190591650 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 --2- 172.21.15.106:0/2865436023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19057baa0 0x55f19055f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 -- 172.21.15.106:0/2865436023 >> 172.21.15.106:0/2865436023 conn(0x55f19055e4d0 msgr2=0x55f19057f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 -- 172.21.15.106:0/2865436023 shutdown_connections 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.576+0000 7f4fc6829740 1 -- 172.21.15.106:0/2865436023 wait complete. 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.577+0000 7f4fc6829740 1 -- start start 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.577+0000 7f4fc6829740 1 Processor -- start 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f1905b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 0x55f1905b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1fff50 con 0x55f19057baa0 2024-04-13T03:15:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1f72a0 con 0x55f19058b3d0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f19055e9a0 con 0x55f19055ce80 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 10 client.0 _ll_register_callbacks cb 0x7f4fb8004930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54600/0 (socket says 172.21.15.106:54600) 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 -- 172.21.15.106:0/164306330 learned_addr learned my addr 172.21.15.106:0/164306330 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f1905b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 0x55f1905b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 -- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 msgr2=0x55f1905b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 0x55f1905b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 -- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 msgr2=0x55f1905b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f1905b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 -- 172.21.15.106:0/164306330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1905a34e0 con 0x55f19057baa0 2024-04-13T03:15:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc1efb640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 0x55f1905b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc16fa640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80346a0 tx=0x7f4fb8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3cd6327f337b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb802f040 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f4fb802bcb0 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac021440 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4fb8046050 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 10 client.7848 fetch_fsmap learned FSMap version 57 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.578+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55f19058dca0 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac021440 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f4fb8049020 con 0x55f19057baa0 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 10 client.7848 fetch_fsmap finished waiting for FSMap version 57 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 10 client.7848 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 20 client.7848 populate_metadata read hostname 'smithi106' 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f190621c40 0x55f190561d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:15:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 4 client.7848 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1904d0a00 con 0x55f190621c40 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f190621c40 0x55f190561d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4f97fff640 10 client.7848 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc0ef9640 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f190621c40 0x55f190561d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb4afb80c29a8bd server_cookie=84441f9797ddfb82 in_seq=0 out_seq=0 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4f97fff640 1 -- 172.21.15.106:0/164306330 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55f1904d0a00 con 0x55f190621c40 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4f97fff640 10 client.7848 handle_command_reply: tid=0 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.579+0000 7f4fc6829740 1 client.7848 shutdown 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 msgr2=0x55f19058e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80346a0 tx=0x7f4fb8015000 comp rx=0 tx=0).stop 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 shutdown_connections 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55f190621c40 0x55f190561d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f19055ce80 0x55f1905b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f19058b3d0 0x55f1905b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4f97fff640 0 client.7848 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 --2- 172.21.15.106:0/164306330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f19057baa0 0x55f19058e1c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 >> 172.21.15.106:0/164306330 conn(0x55f19055e4d0 msgr2=0x55f190591650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 shutdown_connections 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 1 -- 172.21.15.106:0/164306330 wait complete. 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc6829740 20 client.7848 trim_cache size 0 max 16384 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:15:35.589 INFO:teuthology.orchestra.run.smithi106.stdout: "c8b94afe-3953-4cf4-a425-184951c1242e": { 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir", 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "c8b94afe-3953-4cf4-a425-184951c1242e", 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive" 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:15:35.590 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 msgr2=0x7f4fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 0x7f4fa006edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009a60 tx=0x7f4fb8003990 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 msgr2=0x7f4fbc0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402beb0 tx=0x7f4fb40046c0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 shutdown_connections 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4fa006c8f0 0x7f4fa006edb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4fbc0b1c30 0x7f4fbc1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fbc0b54a0 0x7f4fbc0b3350 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 --2- 172.21.15.106:0/1349891308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc070bb0 0x7f4fbc1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 >> 172.21.15.106:0/1349891308 conn(0x7f4fbc071870 msgr2=0x7f4fbc0700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:15:35.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 shutdown_connections 2024-04-13T03:15:35.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:15:35.580+0000 7f4fc395e640 1 -- 172.21.15.106:0/1349891308 wait complete. 2024-04-13T03:15:35.593 INFO:tasks.cephfs.filesystem:scrub status for tag:c8b94afe-3953-4cf4-a425-184951c1242e - {'path': '/dir', 'tag': 'c8b94afe-3953-4cf4-a425-184951c1242e', 'options': 'recursive'} 2024-04-13T03:15:39.683 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:15:39.682+0000 7f707b3f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-13T03:15:55.732 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:15:55.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:16:05.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:05.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:16:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.707+0000 7f96f260f640 1 Processor -- start 2024-04-13T03:16:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.707+0000 7f96f260f640 1 -- start start 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96f260f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96ec15ee40 0x7f96ec0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96f260f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96ec0709e0 0x7f96ec06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96f260f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ec0b5cf0 con 0x7f96ec0b1c30 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96f260f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec0b5fd0 con 0x7f96ec15ee40 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96f260f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec0b62b0 con 0x7f96ec0709e0 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96eb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96ec15ee40 0x7f96ec0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96eb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f96ec15ee40 0x7f96ec0704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38044/0 (socket says 172.21.15.106:38044) 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.708+0000 7f96eb7fe640 1 -- 172.21.15.106:0/407998693 learned_addr learned my addr 172.21.15.106:0/407998693 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96f0bac640 1 --2- 172.21.15.106:0/407998693 >> v2:172.21.15.114:3300/0 conn(0x7f96ec0709e0 0x7f96ec06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96eb7fe640 1 -- 172.21.15.106:0/407998693 >> v2:172.21.15.114:3300/0 conn(0x7f96ec0709e0 msgr2=0x7f96ec06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/407998693 >> v2:172.21.15.114:3300/0 conn(0x7f96ec0709e0 0x7f96ec06d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96eb7fe640 1 -- 172.21.15.106:0/407998693 >> v1:172.21.15.114:6789/0 conn(0x7f96ec0b1c30 legacy=0x7f96ec0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96eb7fe640 1 -- 172.21.15.106:0/407998693 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0b6590 con 0x7f96ec15ee40 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.709+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/407998693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec0704a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f96ec073f70 tx=0x7f96e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807e02881fbaa2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96eaffd640 1 -- 172.21.15.106:0/407998693 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0035020 con 0x7f96ec15ee40 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96eaffd640 1 -- 172.21.15.106:0/407998693 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e0031870 con 0x7f96ec15ee40 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96eaffd640 1 -- 172.21.15.106:0/407998693 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0031c40 con 0x7f96ec15ee40 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 -- 172.21.15.106:0/407998693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 msgr2=0x7f96ec0704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 --2- 172.21.15.106:0/407998693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec0704a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f96ec073f70 tx=0x7f96e002b910 comp rx=0 tx=0).stop 2024-04-13T03:16:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 -- 172.21.15.106:0/407998693 shutdown_connections 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 --2- 172.21.15.106:0/407998693 >> v2:172.21.15.114:3300/0 conn(0x7f96ec0709e0 0x7f96ec06d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 --2- 172.21.15.106:0/407998693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec0704a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 -- 172.21.15.106:0/407998693 >> 172.21.15.106:0/407998693 conn(0x7f96ec071870 msgr2=0x7f96ec071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 -- 172.21.15.106:0/407998693 shutdown_connections 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.710+0000 7f96f260f640 1 -- 172.21.15.106:0/407998693 wait complete. 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.711+0000 7f96f260f640 1 Processor -- start 2024-04-13T03:16:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 -- start start 2024-04-13T03:16:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 0x7f96ec0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96ec0b1c30 0x7f96ec1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec06df30 con 0x7f96ec15ee40 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec06ddb0 con 0x7f96ec0709e0 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f260f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec06e0b0 con 0x7f96ec0b1c30 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f0bac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f0bac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38046/0 (socket says 172.21.15.106:38046) 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.712+0000 7f96f0bac640 1 -- 172.21.15.106:0/2793753694 learned_addr learned my addr 172.21.15.106:0/2793753694 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96ebfff640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 0x7f96ec0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96ec0b1c30 0x7f96ec1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 -- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96ec0b1c30 msgr2=0x7f96ec1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96ec0b1c30 0x7f96ec1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 -- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 msgr2=0x7f96ec0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 0x7f96ec0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 -- 172.21.15.106:0/2793753694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0cb820 con 0x7f96ec15ee40 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96ebfff640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 0x7f96ec0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f0bac640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f96e400cd70 tx=0x7f96e400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96e8ff9640 1 -- 172.21.15.106:0/2793753694 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4003b90 con 0x7f96ec15ee40 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96e8ff9640 1 -- 172.21.15.106:0/2793753694 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e4003d30 con 0x7f96ec15ee40 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec161350 con 0x7f96ec15ee40 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96e8ff9640 1 -- 172.21.15.106:0/2793753694 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4004240 con 0x7f96ec15ee40 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.713+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec0cbac0 con 0x7f96ec15ee40 2024-04-13T03:16:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.715+0000 7f96e8ff9640 1 -- 172.21.15.106:0/2793753694 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96e40043e0 con 0x7f96ec15ee40 2024-04-13T03:16:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.715+0000 7f96e8ff9640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.715+0000 7f96e8ff9640 1 -- 172.21.15.106:0/2793753694 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f96e408c810 con 0x7f96ec15ee40 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.715+0000 7f96ebfff640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.716+0000 7f96ebfff640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 0x7f96d806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96dc003eb0 tx=0x7f96dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.716+0000 7f96f53fe740 1 Processor -- start 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.716+0000 7f96f53fe740 1 -- start start 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f3ff0 0x5610ed2c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5610ed232170 con 0x5610ed2c74d0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610ed1d87f0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed22e130 con 0x5610ed2f43d0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38052/0 (socket says 172.21.15.106:38052) 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 -- 172.21.15.106:0/3835431911 learned_addr learned my addr 172.21.15.106:0/3835431911 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96ebfff640 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f3ff0 0x5610ed2c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 -- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 msgr2=0x5610ed2f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 -- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f3ff0 msgr2=0x5610ed2c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f3ff0 0x5610ed2c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 -- 172.21.15.106:0/3835431911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ed00e7d0 con 0x5610ed2c74d0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.717+0000 7f96f0bac640 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004af0 tx=0x7f96e405aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b15aa63ea75d0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96d1ffb640 1 -- 172.21.15.106:0/3835431911 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4089070 con 0x5610ed2c74d0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96d1ffb640 1 -- 172.21.15.106:0/3835431911 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e4062f40 con 0x5610ed2c74d0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96d1ffb640 1 -- 172.21.15.106:0/3835431911 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40046d0 con 0x5610ed2c74d0 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 -- 172.21.15.106:0/3835431911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 msgr2=0x5610ed2f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004af0 tx=0x7f96e405aba0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 -- 172.21.15.106:0/3835431911 shutdown_connections 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2c74d0 0x5610ed2f72c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 --2- 172.21.15.106:0/3835431911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f3ff0 0x5610ed2c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 -- 172.21.15.106:0/3835431911 >> 172.21.15.106:0/3835431911 conn(0x5610ed2c9200 msgr2=0x5610ed311d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 -- 172.21.15.106:0/3835431911 shutdown_connections 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.718+0000 7f96f53fe740 1 -- 172.21.15.106:0/3835431911 wait complete. 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.719+0000 7f96f53fe740 1 -- start start 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.719+0000 7f96f53fe740 1 Processor -- start 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2e7090 0x5610ed2e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5610ed2cb8a0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610ed1ebd00 con 0x5610ed2e7090 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed2ce0e0 con 0x5610ed2f43d0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f53fe740 10 client.0 _ll_register_callbacks cb 0x5610ed1eb6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38060/0 (socket says 172.21.15.106:38060) 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 -- 172.21.15.106:0/1147158223 learned_addr learned my addr 172.21.15.106:0/1147158223 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96f0bac640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2e7090 0x5610ed2e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 -- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 msgr2=0x5610ed2fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 -- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2e7090 msgr2=0x5610ed2e3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2e7090 0x5610ed2e3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 -- 172.21.15.106:0/1147158223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ecf443f0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96ebfff640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00de90 tx=0x7f96dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97307a6a82aa48e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.720+0000 7f96caffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc014e30 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f96dc0125e0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e4062f40 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f96dc02d050 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 10 client.7860 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5610ecf040f0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e4062f40 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f96dc030020 con 0x5610ed2f3ff0 2024-04-13T03:16:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 10 client.7860 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 10 client.7860 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 20 client.7860 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed325ca0 0x5610ed326080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 4 client.7860 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ed23fb90 con 0x5610ed325ca0 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.721+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed325ca0 0x5610ed326080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96caffd640 10 client.7860 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed325ca0 0x5610ed326080 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e043a0d56f01e08 server_cookie=beb8d21334ef84f6 in_seq=0 out_seq=0 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96caffd640 1 -- 172.21.15.106:0/1147158223 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610ed23fb90 con 0x5610ed325ca0 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96caffd640 10 client.7860 handle_command_reply: tid=0 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 client.7860 shutdown 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 msgr2=0x5610ed2f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00de90 tx=0x7f96dc0036f0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 shutdown_connections 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed325ca0 0x5610ed326080 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2f43d0 0x5610ed2fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2e7090 0x5610ed2e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96caffd640 0 client.7860 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 --2- 172.21.15.106:0/1147158223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f8c20 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 >> 172.21.15.106:0/1147158223 conn(0x5610ed2c74d0 msgr2=0x5610ed2cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 shutdown_connections 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.722+0000 7f96f53fe740 1 -- 172.21.15.106:0/1147158223 wait complete. 2024-04-13T03:16:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.723+0000 7f96f53fe740 20 client.7860 trim_cache size 0 max 16384 2024-04-13T03:16:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.740+0000 7f96f53fe740 1 Processor -- start 2024-04-13T03:16:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.740+0000 7f96f53fe740 1 -- start start 2024-04-13T03:16:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed385170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec2016b0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec2006e0 con 0x5610ed2f43d0 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec1f8e40 con 0x5610ed2e7090 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38070/0 (socket says 172.21.15.106:38070) 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 -- 172.21.15.106:0/313016666 learned_addr learned my addr 172.21.15.106:0/313016666 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 -- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 msgr2=0x5610ed2fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 -- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 msgr2=0x5610ed385170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed385170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 -- 172.21.15.106:0/313016666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ed2c2f10 con 0x5610ed2f3ff0 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.741+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f96e00090b0 tx=0x7f96e0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c423cf09079b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96d17fa640 1 -- 172.21.15.106:0/313016666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0035040 con 0x5610ed2f3ff0 2024-04-13T03:16:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96d17fa640 1 -- 172.21.15.106:0/313016666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e00043a0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96d17fa640 1 -- 172.21.15.106:0/313016666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0031c00 con 0x5610ed2f3ff0 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 -- 172.21.15.106:0/313016666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 msgr2=0x5610ed2f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f96e00090b0 tx=0x7f96e0030000 comp rx=0 tx=0).stop 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 -- 172.21.15.106:0/313016666 shutdown_connections 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed385170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2f7f70 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 --2- 172.21.15.106:0/313016666 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 -- 172.21.15.106:0/313016666 >> 172.21.15.106:0/313016666 conn(0x5610ed2c74d0 msgr2=0x5610ed2f8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 -- 172.21.15.106:0/313016666 shutdown_connections 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.742+0000 7f96f53fe740 1 -- 172.21.15.106:0/313016666 wait complete. 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.743+0000 7f96f53fe740 1 -- start start 2024-04-13T03:16:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.743+0000 7f96f53fe740 1 Processor -- start 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed2f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0cd0a0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610ed2fafd0 con 0x5610ed2f43d0 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed2c67c0 con 0x5610ed2e7090 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f53fe740 10 client.0 _ll_register_callbacks cb 0x7f96dc00dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38084/0 (socket says 172.21.15.106:38084) 2024-04-13T03:16:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 -- 172.21.15.106:0/401301546 learned_addr learned my addr 172.21.15.106:0/401301546 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96ebfff640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96f0bac640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed2f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 -- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 msgr2=0x5610ed2cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 -- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 msgr2=0x5610ed2f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed2f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 -- 172.21.15.106:0/401301546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ed2cb4c0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96eb7fe640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009280 tx=0x7f96e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f64f189bafbe07 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.744+0000 7f96caffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0035040 con 0x5610ed2f3ff0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f96e0031bd0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e00043a0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f96e005e050 con 0x5610ed2f3ff0 2024-04-13T03:16:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 10 client.7866 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5610ed384bd0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e00043a0 con 0x5610ed2f3ff0 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f96e0061020 con 0x5610ed2f3ff0 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 10 client.7866 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 10 client.7866 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 20 client.7866 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed2cc630 0x5610ed2f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 4 client.7866 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610ed239a00 con 0x5610ed2cc630 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96ebfff640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed2cc630 0x5610ed2f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.745+0000 7f96caffd640 10 client.7866 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96ebfff640 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed2cc630 0x5610ed2f6d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b138fe8464ba22c7 server_cookie=a6baab41379a7c00 in_seq=0 out_seq=0 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96caffd640 1 -- 172.21.15.106:0/401301546 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610ed239a00 con 0x5610ed2cc630 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96caffd640 10 client.7866 handle_command_reply: tid=0 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 client.7866 shutdown 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 msgr2=0x5610ed2cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009280 tx=0x7f96e0004730 comp rx=0 tx=0).stop 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 shutdown_connections 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5610ed2cc630 0x5610ed2f6d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed2f43d0 0x5610ed2f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed2f3ff0 0x5610ed2cbf90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96caffd640 0 client.7866 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 --2- 172.21.15.106:0/401301546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed2e7090 0x5610ed2cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 >> 172.21.15.106:0/401301546 conn(0x5610ed2c74d0 msgr2=0x5610ed2f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 shutdown_connections 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 1 -- 172.21.15.106:0/401301546 wait complete. 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.746+0000 7f96f53fe740 20 client.7866 trim_cache size 0 max 16384 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:16:05.759 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 msgr2=0x7f96d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 0x7f96d806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96dc003eb0 tx=0x7f96dc009040 comp rx=0 tx=0).stop 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 msgr2=0x7f96ec162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f96e400cd70 tx=0x7f96e400a660 comp rx=0 tx=0).stop 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 shutdown_connections 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96ec0b1c30 0x7f96ec1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96ec15ee40 0x7f96ec162340 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 --2- 172.21.15.106:0/2793753694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96ec0709e0 0x7f96ec0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 >> 172.21.15.106:0/2793753694 conn(0x7f96ec071870 msgr2=0x7f96ec0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 shutdown_connections 2024-04-13T03:16:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.747+0000 7f96f260f640 1 -- 172.21.15.106:0/2793753694 wait complete. 2024-04-13T03:16:05.763 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:16:05.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 Processor -- start 2024-04-13T03:16:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 -- start start 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0634070090 0x7f0634160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0634070470 con 0x7f06340cf5a0 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063415da00 con 0x7f06340b0390 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.863+0000 7f063aca2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f06340ceb50 con 0x7f0634070090 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f0638a3e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0634070090 0x7f0634160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f063923f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f06340cf5a0 0x7f06340cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37040/0 (socket says 172.21.15.106:37040) 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f063923f640 1 -- 172.21.15.106:0/1140883069 learned_addr learned my addr 172.21.15.106:0/1140883069 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f0638a3e640 1 -- 172.21.15.106:0/1140883069 >> v1:172.21.15.114:6789/0 conn(0x7f06340b0390 legacy=0x7f0634160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f0638a3e640 1 -- 172.21.15.106:0/1140883069 >> v1:172.21.15.106:6789/0 conn(0x7f06340cf5a0 legacy=0x7f06340cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f06337fe640 1 -- 172.21.15.106:0/1140883069 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350618719 0 0) 0x7f0634070470 con 0x7f06340cf5a0 2024-04-13T03:16:05.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.864+0000 7f0638a3e640 1 -- 172.21.15.106:0/1140883069 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063415dcc0 con 0x7f0634070090 2024-04-13T03:16:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f0638a3e640 1 --2- 172.21.15.106:0/1140883069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0634070090 0x7f0634160570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec168a5fc756a268 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f06337fe640 1 -- 172.21.15.106:0/1140883069 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06240315a0 con 0x7f0634070090 2024-04-13T03:16:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f06337fe640 1 -- 172.21.15.106:0/1140883069 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f0634070090 2024-04-13T03:16:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f06337fe640 1 -- 172.21.15.106:0/1140883069 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624031960 con 0x7f0634070090 2024-04-13T03:16:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 -- 172.21.15.106:0/1140883069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0634070090 msgr2=0x7f0634160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 --2- 172.21.15.106:0/1140883069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0634070090 0x7f0634160570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402b840 comp rx=0 tx=0).stop 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 -- 172.21.15.106:0/1140883069 shutdown_connections 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 --2- 172.21.15.106:0/1140883069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0634070090 0x7f0634160570 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 -- 172.21.15.106:0/1140883069 >> 172.21.15.106:0/1140883069 conn(0x7f06340cd6d0 msgr2=0x7f06340cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 -- 172.21.15.106:0/1140883069 shutdown_connections 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.865+0000 7f063aca2640 1 -- 172.21.15.106:0/1140883069 wait complete. 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.866+0000 7f063aca2640 1 Processor -- start 2024-04-13T03:16:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 -- start start 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 0x7f0634152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 0x7f063414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f063406e560 con 0x7f06340b0390 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06340cf110 con 0x7f0634070090 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f063aca2640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06340cd220 con 0x7f06340cf5a0 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f0633fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f0633fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38104/0 (socket says 172.21.15.106:38104) 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.867+0000 7f0633fff640 1 -- 172.21.15.106:0/3152689548 learned_addr learned my addr 172.21.15.106:0/3152689548 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0638a3e640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 0x7f0634152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f063923f640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 0x7f063414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 -- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 msgr2=0x7f063414ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 0x7f063414ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 -- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 msgr2=0x7f0634152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 0x7f0634152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 -- 172.21.15.106:0/3152689548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341e7680 con 0x7f06340b0390 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0638a3e640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 0x7f0634152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f063923f640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 0x7f063414ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f0633fff640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f062800c460 tx=0x7f062800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628016020 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0628012870 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628012c60 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063415eb60 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.868+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0634154e60 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.869+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0634068a10 con 0x7f06340b0390 2024-04-13T03:16:05.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.870+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0628019020 con 0x7f06340b0390 2024-04-13T03:16:05.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.870+0000 7f06317fa640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:05.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.870+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f062808bd40 con 0x7f06340b0390 2024-04-13T03:16:05.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.870+0000 7f0638a3e640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:05.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.870+0000 7f0638a3e640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0624036000 tx=0x7f0624008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:05.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:05.875+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0628020020 con 0x7f06340b0390 2024-04-13T03:16:06.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.085+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06340d0840 con 0x7f06340b0390 2024-04-13T03:16:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.086+0000 7f06317fa640 1 -- 172.21.15.106:0/3152689548 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5193 (secure 0 0 0) 0x7f06280182c0 con 0x7f06340b0390 2024-04-13T03:16:06.087 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:06.087 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:39.667524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7728,"qdb_cluster":[7728]},"id":6}]} 2024-04-13T03:16:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 58 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 msgr2=0x7f062006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0624036000 tx=0x7f0624008030 comp rx=0 tx=0).stop 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 msgr2=0x7f0634154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f062800c460 tx=0x7f062800c920 comp rx=0 tx=0).stop 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 shutdown_connections 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f062006c8f0 0x7f062006edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06340cf5a0 0x7f063414ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06340b0390 0x7f0634154920 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 --2- 172.21.15.106:0/3152689548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634070090 0x7f0634152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 >> 172.21.15.106:0/3152689548 conn(0x7f06340cd6d0 msgr2=0x7f06340cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 shutdown_connections 2024-04-13T03:16:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.088+0000 7f063aca2640 1 -- 172.21.15.106:0/3152689548 wait complete. 2024-04-13T03:16:06.100 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-13T03:16:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.202+0000 7f98bb036640 1 Processor -- start 2024-04-13T03:16:06.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.202+0000 7f98bb036640 1 -- start start 2024-04-13T03:16:06.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98b40b54a0 0x7f98b40704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 0x7f98b4162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40b5cf0 con 0x7f98b40b54a0 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40b5fd0 con 0x7f98b40709e0 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98bb036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98b40b62b0 con 0x7f98b40b1c30 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98b40b54a0 0x7f98b40704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98b40b54a0 0x7f98b40704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38106/0 (socket says 172.21.15.106:38106) 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 -- 172.21.15.106:0/2058760840 learned_addr learned my addr 172.21.15.106:0/2058760840 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b95d3640 1 --2- 172.21.15.106:0/2058760840 >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 0x7f98b4162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 -- 172.21.15.106:0/2058760840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 msgr2=0x7f98b415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 -- 172.21.15.106:0/2058760840 >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 msgr2=0x7f98b4162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 --2- 172.21.15.106:0/2058760840 >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 0x7f98b4162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b3fff640 1 -- 172.21.15.106:0/2058760840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40b6590 con 0x7f98b40b54a0 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.203+0000 7f98b95d3640 1 --2- 172.21.15.106:0/2058760840 >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 0x7f98b4162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98b3fff640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b40704a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009870 tx=0x7f98a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3e63f0013078dd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98b37fe640 1 -- 172.21.15.106:0/2058760840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0035020 con 0x7f98b40b54a0 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98b37fe640 1 -- 172.21.15.106:0/2058760840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a0031640 con 0x7f98b40b54a0 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98b37fe640 1 -- 172.21.15.106:0/2058760840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a002d040 con 0x7f98b40b54a0 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98bb036640 1 -- 172.21.15.106:0/2058760840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 msgr2=0x7f98b40704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.204+0000 7f98bb036640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b40704a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009870 tx=0x7f98a002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 -- 172.21.15.106:0/2058760840 shutdown_connections 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 --2- 172.21.15.106:0/2058760840 >> v2:172.21.15.114:3300/0 conn(0x7f98b40709e0 0x7f98b4162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b40704a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 --2- 172.21.15.106:0/2058760840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 -- 172.21.15.106:0/2058760840 >> 172.21.15.106:0/2058760840 conn(0x7f98b4071870 msgr2=0x7f98b4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 -- 172.21.15.106:0/2058760840 shutdown_connections 2024-04-13T03:16:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.205+0000 7f98bb036640 1 -- 172.21.15.106:0/2058760840 wait complete. 2024-04-13T03:16:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.206+0000 7f98bb036640 1 Processor -- start 2024-04-13T03:16:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.206+0000 7f98bb036640 1 -- start start 2024-04-13T03:16:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 0x7f98b40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b40d1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4162af0 con 0x7f98b40b54a0 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4162970 con 0x7f98b40709e0 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98b4162c70 con 0x7f98b40b1c30 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38110/0 (socket says 172.21.15.106:38110) 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 -- 172.21.15.106:0/1210968796 learned_addr learned my addr 172.21.15.106:0/1210968796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 -- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 msgr2=0x7f98b40d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b40d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 -- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 msgr2=0x7f98b40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 0x7f98b40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 0x7f98b40cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 -- 172.21.15.106:0/1210968796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40cccc0 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 0x7f98b40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b95d3640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00cef0 tx=0x7f98ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b17fa640 1 -- 172.21.15.106:0/1210968796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac01a070 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b17fa640 1 -- 172.21.15.106:0/1210968796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac00d840 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b40ccfa0 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98b17fa640 1 -- 172.21.15.106:0/1210968796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00dc30 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.207+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b40cd560 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.209+0000 7f98b17fa640 1 -- 172.21.15.106:0/1210968796 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98ac01d070 con 0x7f98b40b54a0 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.209+0000 7f98b17fa640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.209+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.209+0000 7f98b17fa640 1 -- 172.21.15.106:0/1210968796 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f98ac08c540 con 0x7f98b40b54a0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.210+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 0x7f989806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f98a8008890 tx=0x7f98a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.210+0000 7f98bde2a740 1 Processor -- start 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.210+0000 7f98bde2a740 1 -- start start 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x562672086ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 0x562672082f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562671fc0fe0 con 0x5626720866d0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562671f5d860 con 0x562672083af0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x562671fbcfa0 con 0x56267205b800 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38112/0 (socket says 172.21.15.106:38112) 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 -- 172.21.15.106:0/1175791932 learned_addr learned my addr 172.21.15.106:0/1175791932 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b3fff640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x562672086ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b95d3640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 0x562672082f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 -- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 msgr2=0x562672082f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 0x562672082f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 -- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 msgr2=0x562672086ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x562672086ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b8dd2640 1 -- 172.21.15.106:0/1175791932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562671f8c1a0 con 0x5626720866d0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.211+0000 7f98b95d3640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 0x562672082f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f98a800dad0 tx=0x7f98a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c701c215ecd6b8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f989e7fc640 1 -- 172.21.15.106:0/1175791932 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8019050 con 0x5626720866d0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f989e7fc640 1 -- 172.21.15.106:0/1175791932 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8023070 con 0x5626720866d0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f989e7fc640 1 -- 172.21.15.106:0/1175791932 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8012c50 con 0x5626720866d0 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 -- 172.21.15.106:0/1175791932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 msgr2=0x56267205d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f98a800dad0 tx=0x7f98a8010000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 -- 172.21.15.106:0/1175791932 shutdown_connections 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56267205b800 0x562672082f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x562672086ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 --2- 172.21.15.106:0/1175791932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267205d560 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 -- 172.21.15.106:0/1175791932 >> 172.21.15.106:0/1175791932 conn(0x562671fefdc0 msgr2=0x5626720a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 -- 172.21.15.106:0/1175791932 shutdown_connections 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.212+0000 7f98bde2a740 1 -- 172.21.15.106:0/1175791932 wait complete. 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.213+0000 7f98bde2a740 1 -- start start 2024-04-13T03:16:06.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.213+0000 7f98bde2a740 1 Processor -- start 2024-04-13T03:16:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56267205b800 0x562672087dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x56267207d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562672056b00 con 0x5626720866d0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562671f5b1d0 con 0x56267205b800 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5626720a1c90 con 0x562672083af0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98bde2a740 10 client.0 _ll_register_callbacks cb 0x562671f7e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38128/0 (socket says 172.21.15.106:38128) 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 -- 172.21.15.106:0/2318313116 learned_addr learned my addr 172.21.15.106:0/2318313116 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b3fff640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x56267207d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56267205b800 0x562672087dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 -- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 msgr2=0x56267207d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x56267207d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 -- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56267205b800 msgr2=0x562672087dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56267205b800 0x562672087dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 -- 172.21.15.106:0/2318313116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562671d9f7c0 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98b95d3640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00afd0 tx=0x7f98ac05a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34e1a9c2c354063 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.214+0000 7f98937fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0627d0 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f98ac062970 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a8023070 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f98ac08c070 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 10 client.7887 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x562671dfff50 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8023070 con 0x5626720866d0 2024-04-13T03:16:06.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f98ac0594d0 con 0x5626720866d0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 10 client.7887 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 10 client.7887 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 20 client.7887 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5626720bd1b0 0x5626720bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 4 client.7887 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562671f604a0 con 0x5626720bd1b0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5626720bd1b0 0x5626720bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98937fe640 10 client.7887 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.215+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5626720bd1b0 0x5626720bf5a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64dfadc61e963cf3 server_cookie=4b9bc9f59362559e in_seq=0 out_seq=0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98937fe640 1 -- 172.21.15.106:0/2318313116 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562671f604a0 con 0x5626720bd1b0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98937fe640 10 client.7887 handle_command_reply: tid=0 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 client.7887 shutdown 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 msgr2=0x56267207f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00afd0 tx=0x7f98ac05a1c0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 shutdown_connections 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x5626720bd1b0 0x5626720bf5a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x56267207d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626720866d0 0x56267207f030 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98937fe640 0 client.7887 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2318313116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56267205b800 0x562672087dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 >> 172.21.15.106:0/2318313116 conn(0x562671fefdc0 msgr2=0x562672055e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 shutdown_connections 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 1 -- 172.21.15.106:0/2318313116 wait complete. 2024-04-13T03:16:06.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.216+0000 7f98bde2a740 20 client.7887 trim_cache size 0 max 16384 2024-04-13T03:16:06.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.229+0000 7f98bde2a740 1 Processor -- start 2024-04-13T03:16:06.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.229+0000 7f98bde2a740 1 -- start start 2024-04-13T03:16:06.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x5626720c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5626720866d0 0x5626720c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4200500 con 0x56267205b800 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41f8420 con 0x562672083af0 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f98b41fffe0 con 0x5626720866d0 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38134/0 (socket says 172.21.15.106:38134) 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 -- 172.21.15.106:0/2018732190 learned_addr learned my addr 172.21.15.106:0/2018732190 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 -- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5626720866d0 msgr2=0x5626720c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5626720866d0 0x5626720c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 -- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 msgr2=0x5626720c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x5626720c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.230+0000 7f98b8dd2640 1 -- 172.21.15.106:0/2018732190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562671c930b0 con 0x56267205b800 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x56267205b5c0 tx=0x7f98a8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfc23e7d146a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f989dffb640 1 -- 172.21.15.106:0/2018732190 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8019050 con 0x56267205b800 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f989dffb640 1 -- 172.21.15.106:0/2018732190 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8023040 con 0x56267205b800 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f989dffb640 1 -- 172.21.15.106:0/2018732190 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8035030 con 0x56267205b800 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 -- 172.21.15.106:0/2018732190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 msgr2=0x56267207e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x56267205b5c0 tx=0x7f98a8012000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 -- 172.21.15.106:0/2018732190 shutdown_connections 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5626720866d0 0x5626720c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562672083af0 0x5626720c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 --2- 172.21.15.106:0/2018732190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267207e900 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 -- 172.21.15.106:0/2018732190 >> 172.21.15.106:0/2018732190 conn(0x562671fefdc0 msgr2=0x56267207bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 -- 172.21.15.106:0/2018732190 shutdown_connections 2024-04-13T03:16:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.231+0000 7f98bde2a740 1 -- 172.21.15.106:0/2018732190 wait complete. 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.232+0000 7f98bde2a740 1 -- start start 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.232+0000 7f98bde2a740 1 Processor -- start 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x5626720c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626720866d0 0x5626720822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562672088310 con 0x56267205b800 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56267207cce0 con 0x5626720866d0 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56267207ab00 con 0x562672083af0 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 10 client.0 _ll_register_callbacks cb 0x7f98ac05a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38142/0 (socket says 172.21.15.106:38142) 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 -- 172.21.15.106:0/680329480 learned_addr learned my addr 172.21.15.106:0/680329480 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b95d3640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626720866d0 0x5626720822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b3fff640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x5626720c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 -- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 msgr2=0x5626720c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x5626720c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 -- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626720866d0 msgr2=0x5626720822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626720866d0 0x5626720822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 -- 172.21.15.106:0/680329480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267207f0b0 con 0x56267205b800 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b3fff640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x5626720c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98b8dd2640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f98a800d590 tx=0x7f98a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01f3e6f25fe3f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98937fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8035030 con 0x56267205b800 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f98a803fc80 con 0x56267205b800 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.233+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a8023040 con 0x56267205b800 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f98a8019020 con 0x56267205b800 2024-04-13T03:16:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 10 client.7893 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x56267205db40 con 0x56267205b800 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8023040 con 0x56267205b800 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f98a8049080 con 0x56267205b800 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 10 client.7893 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 10 client.7893 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 20 client.7893 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x562672082850 0x562672126a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 4 client.7893 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562671fcea00 con 0x562672082850 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98b3fff640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x562672082850 0x562672126a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 10 client.7893 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98b3fff640 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x562672082850 0x562672126a40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df90811091d477ba server_cookie=6bce5316be2a3ed4 in_seq=0 out_seq=0 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 1 -- 172.21.15.106:0/680329480 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+418 (crc 0 0 0) 0x562671fcea00 con 0x562672082850 2024-04-13T03:16:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98937fe640 10 client.7893 handle_command_reply: tid=0 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.234+0000 7f98bde2a740 1 client.7893 shutdown 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 msgr2=0x56267211b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f98a800d590 tx=0x7f98a8004730 comp rx=0 tx=0).stop 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 shutdown_connections 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x562672082850 0x562672126a40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562672083af0 0x5626720c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626720866d0 0x5626720822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 --2- 172.21.15.106:0/680329480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56267205b800 0x56267211b080 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 >> 172.21.15.106:0/680329480 conn(0x562671fefdc0 msgr2=0x56267207e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98937fe640 0 client.7893 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 shutdown_connections 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 1 -- 172.21.15.106:0/680329480 wait complete. 2024-04-13T03:16:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bde2a740 20 client.7893 trim_cache size 0 max 16384 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "dir_frag", 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 523692687, 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627776, 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "frag": "*", 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir" 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:16:06.245 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "backtrace", 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 941769863, 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627776, 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir" 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "backtrace", 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 3964833038, 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/testdir/hardlink" 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:06.246 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 msgr2=0x7f989806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 0x7f989806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f98a8008890 tx=0x7f98a8008040 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 msgr2=0x7f98b41e9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00cef0 tx=0x7f98ac00a710 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 shutdown_connections 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98b40b1c30 0x7f98b40d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.235+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98b40b54a0 0x7f98b41e9060 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.236+0000 7f98bb036640 1 --2- 172.21.15.106:0/1210968796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40709e0 0x7f98b40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.236+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 >> 172.21.15.106:0/1210968796 conn(0x7f98b4071870 msgr2=0x7f98b40b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.236+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 shutdown_connections 2024-04-13T03:16:06.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.236+0000 7f98bb036640 1 -- 172.21.15.106:0/1210968796 wait complete. 2024-04-13T03:16:06.249 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:16:06.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.347+0000 7fde5271e640 1 Processor -- start 2024-04-13T03:16:06.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.347+0000 7fde5271e640 1 -- start start 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde5271e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fde4c0b33b0 0x7fde4c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde5271e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c15f4d0 con 0x7fde4c158f60 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde5271e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c15f760 con 0x7fde4c0b3de0 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde5271e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fde4c15fa40 con 0x7fde4c0b33b0 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fde4c158f60 0x7fde4c15edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37056/0 (socket says 172.21.15.106:37056) 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4bfff640 1 -- 172.21.15.106:0/869075297 learned_addr learned my addr 172.21.15.106:0/869075297 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166047796 0 0) 0x7fde4c15f4d0 con 0x7fde4c158f60 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde34003520 con 0x7fde4c158f60 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4b7fe640 1 --2- 172.21.15.106:0/869075297 >> v2:172.21.15.114:3300/0 conn(0x7fde4c0b33b0 0x7fde4c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1952779224 0 0) 0x7fde34003520 con 0x7fde4c158f60 2024-04-13T03:16:06.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 >> v2:172.21.15.114:3300/0 conn(0x7fde4c0b33b0 msgr2=0x7fde4c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 --2- 172.21.15.106:0/869075297 >> v2:172.21.15.114:3300/0 conn(0x7fde4c0b33b0 0x7fde4c06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 >> v1:172.21.15.114:6789/0 conn(0x7fde4c0b3de0 legacy=0x7fde4c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c15fd20 con 0x7fde4c158f60 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fde40001670 con 0x7fde4c158f60 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 >> v1:172.21.15.106:6789/0 conn(0x7fde4c158f60 legacy=0x7fde4c15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340051c0 0x7fde3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c15fa40 con 0x7fde340039b0 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde3400c070 con 0x7fde340051c0 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.348+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fde3400c270 con 0x7fde34004800 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde50cbb640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340051c0 0x7fde3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4b7fe640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 -- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde34004800 msgr2=0x7fde34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 -- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340051c0 msgr2=0x7fde3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340051c0 0x7fde3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde4bfff640 1 -- 172.21.15.106:0/869075297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde34003520 con 0x7fde340039b0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.349+0000 7fde50cbb640 1 -- 172.21.15.106:0/869075297 reap_dead start 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde4bfff640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c240 tx=0x7fde34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f511807ccbed527 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40007cf0 con 0x7fde340039b0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde40030070 con 0x7fde340039b0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde4affd640 1 -- 172.21.15.106:0/869075297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40005d30 con 0x7fde340039b0 2024-04-13T03:16:06.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 -- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 msgr2=0x7fde34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c240 tx=0x7fde34005830 comp rx=0 tx=0).stop 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 -- 172.21.15.106:0/869075297 shutdown_connections 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 --2- 172.21.15.106:0/869075297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 -- 172.21.15.106:0/869075297 >> 172.21.15.106:0/869075297 conn(0x7fde4c0d0340 msgr2=0x7fde4c0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 -- 172.21.15.106:0/869075297 shutdown_connections 2024-04-13T03:16:06.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.350+0000 7fde5271e640 1 -- 172.21.15.106:0/869075297 wait complete. 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.351+0000 7fde5271e640 1 Processor -- start 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 -- start start 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340039b0 0x7fde4c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde4c15d620 0x7fde4c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0b4200 con 0x7fde4c1570b0 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0adea0 con 0x7fde340039b0 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde5271e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c0b4380 con 0x7fde4c15d620 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde50cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde50cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38156/0 (socket says 172.21.15.106:38156) 2024-04-13T03:16:06.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde50cbb640 1 -- 172.21.15.106:0/50256677 learned_addr learned my addr 172.21.15.106:0/50256677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde4bfff640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340039b0 0x7fde4c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.352+0000 7fde4b7fe640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde4c15d620 0x7fde4c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 -- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde4c15d620 msgr2=0x7fde4c15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde4c15d620 0x7fde4c15daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 -- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340039b0 msgr2=0x7fde4c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340039b0 0x7fde4c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 -- 172.21.15.106:0/50256677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c156ac0 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde50cbb640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0b3d30 tx=0x7fde4400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44003b80 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde44018070 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde440050d0 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c15e190 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.353+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1594e0 con 0x7fde4c1570b0 2024-04-13T03:16:06.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.354+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c15e6b0 con 0x7fde4c1570b0 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.354+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde44005270 con 0x7fde4c1570b0 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.355+0000 7fde48ff9640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 0x7fde3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.355+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7fde4408baf0 con 0x7fde4c1570b0 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.355+0000 7fde4bfff640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 0x7fde3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.355+0000 7fde4bfff640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 0x7fde3806edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde40034000 tx=0x7fde40007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.359+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde4c15e6b0 con 0x7fde4c1570b0 2024-04-13T03:16:06.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.574+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fde4c068a10 con 0x7fde4c1570b0 2024-04-13T03:16:06.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.574+0000 7fde48ff9640 1 -- 172.21.15.106:0/50256677 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fde44059dd0 con 0x7fde4c1570b0 2024-04-13T03:16:06.576 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:06.576 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-13T03:16:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 msgr2=0x7fde3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 0x7fde3806edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde40034000 tx=0x7fde40007000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 msgr2=0x7fde4c157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0b3d30 tx=0x7fde4400fc70 comp rx=0 tx=0).stop 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 shutdown_connections 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fde3806c8f0 0x7fde3806edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fde4c15d620 0x7fde4c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde4c1570b0 0x7fde4c157560 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 --2- 172.21.15.106:0/50256677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde340039b0 0x7fde4c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 >> 172.21.15.106:0/50256677 conn(0x7fde4c0d0340 msgr2=0x7fde4c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 shutdown_connections 2024-04-13T03:16:06.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.576+0000 7fde5271e640 1 -- 172.21.15.106:0/50256677 wait complete. 2024-04-13T03:16:06.588 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:16:06.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:06.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-13T03:16:06.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.692+0000 7f6045620640 1 Processor -- start 2024-04-13T03:16:06.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.692+0000 7f6045620640 1 -- start start 2024-04-13T03:16:06.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6045620640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60400d3490 0x7f60400d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6045620640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60400b3920 0x7f604015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6045620640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604015c870 con 0x7f60400d2370 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6045620640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60400b1b00 con 0x7f60400b3920 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6045620640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f60400d5420 con 0x7f60400d3490 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60400b3920 0x7f604015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6037fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f60400d2370 0x7f60400b13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37064/0 (socket says 172.21.15.106:37064) 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f6037fff640 1 -- 172.21.15.106:0/483414770 learned_addr learned my addr 172.21.15.106:0/483414770 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603effd640 1 --2- 172.21.15.106:0/483414770 >> v2:172.21.15.114:3300/0 conn(0x7f60400d3490 0x7f60400d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603f7fe640 1 -- 172.21.15.106:0/483414770 >> v2:172.21.15.114:3300/0 conn(0x7f60400d3490 msgr2=0x7f60400d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603f7fe640 1 --2- 172.21.15.106:0/483414770 >> v2:172.21.15.114:3300/0 conn(0x7f60400d3490 0x7f60400d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603f7fe640 1 -- 172.21.15.106:0/483414770 >> v1:172.21.15.106:6789/0 conn(0x7f60400d2370 legacy=0x7f60400b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603f7fe640 1 -- 172.21.15.106:0/483414770 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604015ee70 con 0x7f60400b3920 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.693+0000 7f603e7fc640 1 -- 172.21.15.106:0/483414770 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444240264 0 0) 0x7f604015c870 con 0x7f60400d2370 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.694+0000 7f603f7fe640 1 --2- 172.21.15.106:0/483414770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60400b3920 0x7f604015f360 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f60400b8700 tx=0x7f603802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ab93b844318dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.694+0000 7f603e7fc640 1 -- 172.21.15.106:0/483414770 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038039070 con 0x7f60400b3920 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.694+0000 7f603e7fc640 1 -- 172.21.15.106:0/483414770 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603802cbd0 con 0x7f60400b3920 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.694+0000 7f603e7fc640 1 -- 172.21.15.106:0/483414770 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038034440 con 0x7f60400b3920 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 -- 172.21.15.106:0/483414770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60400b3920 msgr2=0x7f604015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 --2- 172.21.15.106:0/483414770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60400b3920 0x7f604015f360 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f60400b8700 tx=0x7f603802be20 comp rx=0 tx=0).stop 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 -- 172.21.15.106:0/483414770 shutdown_connections 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 --2- 172.21.15.106:0/483414770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60400b3920 0x7f604015f360 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 --2- 172.21.15.106:0/483414770 >> v2:172.21.15.114:3300/0 conn(0x7f60400d3490 0x7f60400d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 -- 172.21.15.106:0/483414770 >> 172.21.15.106:0/483414770 conn(0x7f60400b9760 msgr2=0x7f60400b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 -- 172.21.15.106:0/483414770 shutdown_connections 2024-04-13T03:16:06.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.695+0000 7f6045620640 1 -- 172.21.15.106:0/483414770 wait complete. 2024-04-13T03:16:06.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 Processor -- start 2024-04-13T03:16:06.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 -- start start 2024-04-13T03:16:06.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60400b3920 0x7f604015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60400d3490 0x7f60400a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60400d2ae0 con 0x7f604015b900 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.696+0000 7f6045620640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60400d2800 con 0x7f60400d3490 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f6045620640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60400d2dc0 con 0x7f60400b3920 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38164/0 (socket says 172.21.15.106:38164) 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 -- 172.21.15.106:0/3186081180 learned_addr learned my addr 172.21.15.106:0/3186081180 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f6037fff640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60400d3490 0x7f60400a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603effd640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60400b3920 0x7f604015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 -- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60400b3920 msgr2=0x7f604015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60400b3920 0x7f604015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 -- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60400d3490 msgr2=0x7f60400a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60400d3490 0x7f60400a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 -- 172.21.15.106:0/3186081180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60401f4470 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f603f7fe640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6038037040 tx=0x7f6038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f60377fe640 1 -- 172.21.15.106:0/3186081180 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038044020 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f60377fe640 1 -- 172.21.15.106:0/3186081180 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603803a040 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60401f4750 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f60377fe640 1 -- 172.21.15.106:0/3186081180 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038039050 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.697+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401f8590 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.699+0000 7f60377fe640 1 -- 172.21.15.106:0/3186081180 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6038003bb0 con 0x7f604015b900 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.699+0000 7f60377fe640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.699+0000 7f60377fe640 1 -- 172.21.15.106:0/3186081180 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f60380ae160 con 0x7f604015b900 2024-04-13T03:16:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.699+0000 7f603effd640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.700+0000 7f603effd640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 0x7f602006ede0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6030006fd0 tx=0x7f6030008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.700+0000 7f6048405740 1 Processor -- start 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.700+0000 7f6048405740 1 -- start start 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 0x55d954a7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954a802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d9549e50a0 con 0x55d954a80d40 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d95498b930 con 0x55d954a81120 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d9549e1200 con 0x55d954aa7db0 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38180/0 (socket says 172.21.15.106:38180) 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 -- 172.21.15.106:0/3607178952 learned_addr learned my addr 172.21.15.106:0/3607178952 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603f7fe640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954a802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6037fff640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 0x55d954a7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 -- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 msgr2=0x55d954a7fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 0x55d954a7fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 -- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 msgr2=0x55d954a802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954a802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f603effd640 1 -- 172.21.15.106:0/3607178952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9547bf7e0 con 0x55d954a80d40 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.701+0000 7f6037fff640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 0x55d954a7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f603effd640 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f60300041f0 tx=0x7f6030013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28d969370fe7a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6034ff9640 1 -- 172.21.15.106:0/3607178952 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030027070 con 0x55d954a80d40 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6034ff9640 1 -- 172.21.15.106:0/3607178952 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60300038e0 con 0x55d954a80d40 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6034ff9640 1 -- 172.21.15.106:0/3607178952 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030003c90 con 0x55d954a80d40 2024-04-13T03:16:06.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 -- 172.21.15.106:0/3607178952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 msgr2=0x55d954aa7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f60300041f0 tx=0x7f6030013000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 -- 172.21.15.106:0/3607178952 shutdown_connections 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954aa7db0 0x55d954a7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954a802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 --2- 172.21.15.106:0/3607178952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa7870 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 -- 172.21.15.106:0/3607178952 >> 172.21.15.106:0/3607178952 conn(0x55d954a476e0 msgr2=0x55d954ac5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 -- 172.21.15.106:0/3607178952 shutdown_connections 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.702+0000 7f6048405740 1 -- 172.21.15.106:0/3607178952 wait complete. 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.703+0000 7f6048405740 1 -- start start 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.703+0000 7f6048405740 1 Processor -- start 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954ac9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954ac9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d954aa73e0 con 0x55d954a80d40 2024-04-13T03:16:06.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d95499ecf0 con 0x55d954a81120 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d954aa0160 con 0x55d954a9efc0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6048405740 10 client.0 _ll_register_callbacks cb 0x55d95499e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38196/0 (socket says 172.21.15.106:38196) 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 -- 172.21.15.106:0/4191598259 learned_addr learned my addr 172.21.15.106:0/4191598259 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603f7fe640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954ac9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f6037fff640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954ac9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 -- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 msgr2=0x55d954ac9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954ac9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 -- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 msgr2=0x55d954ac9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954ac9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 -- 172.21.15.106:0/4191598259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95481ff70 con 0x55d954a80d40 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f603effd640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6030007fd0 tx=0x7f6030004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78099ef09f1ec664 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.704+0000 7f601dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030030060 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f60300166c0 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60300038e0 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6030036050 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 10 client.7914 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55d9546b7170 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60300038e0 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f6030039020 con 0x55d954a80d40 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 10 client.7914 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 10 client.7914 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 20 client.7914 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954ac4ec0 0x55d954aabc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 4 client.7914 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9549f2ac0 con 0x55d954ac4ec0 2024-04-13T03:16:06.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6037fff640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954ac4ec0 0x55d954aabc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f601dffb640 10 client.7914 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.705+0000 7f6037fff640 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954ac4ec0 0x55d954aabc00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31da590ba4af5642 server_cookie=5a0dad108784db3d in_seq=0 out_seq=0 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f601dffb640 1 -- 172.21.15.106:0/4191598259 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9549f2ac0 con 0x55d954ac4ec0 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f601dffb640 10 client.7914 handle_command_reply: tid=0 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 client.7914 shutdown 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 msgr2=0x55d954aa5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6030007fd0 tx=0x7f6030004730 comp rx=0 tx=0).stop 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 shutdown_connections 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954ac4ec0 0x55d954aabc00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954ac9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954ac9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f601dffb640 0 client.7914 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 --2- 172.21.15.106:0/4191598259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954aa5d30 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 >> 172.21.15.106:0/4191598259 conn(0x55d954a476e0 msgr2=0x55d954aaaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 shutdown_connections 2024-04-13T03:16:06.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 1 -- 172.21.15.106:0/4191598259 wait complete. 2024-04-13T03:16:06.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.706+0000 7f6048405740 20 client.7914 trim_cache size 0 max 16384 2024-04-13T03:16:06.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 Processor -- start 2024-04-13T03:16:06.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 -- start start 2024-04-13T03:16:06.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6040201470 con 0x55d954a80d40 2024-04-13T03:16:06.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6040200470 con 0x55d954a9efc0 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.724+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60401ff8a0 con 0x55d954a81120 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38212/0 (socket says 172.21.15.106:38212) 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 -- 172.21.15.106:0/1582737045 learned_addr learned my addr 172.21.15.106:0/1582737045 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603f7fe640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f6037fff640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 -- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 msgr2=0x55d954aa55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 -- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a9efc0 msgr2=0x55d954aa5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 -- 172.21.15.106:0/1582737045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d954a7d110 con 0x55d954a80d40 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f6037fff640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f603effd640 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6030007790 tx=0x7f6030028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9a439ddf490b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f601ffff640 1 -- 172.21.15.106:0/1582737045 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030027040 con 0x55d954a80d40 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f601ffff640 1 -- 172.21.15.106:0/1582737045 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030016de0 con 0x55d954a80d40 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f601ffff640 1 -- 172.21.15.106:0/1582737045 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300168c0 con 0x55d954a80d40 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f6048405740 1 -- 172.21.15.106:0/1582737045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 msgr2=0x55d954abb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.725+0000 7f6048405740 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6030007790 tx=0x7f6030028000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 -- 172.21.15.106:0/1582737045 shutdown_connections 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 --2- 172.21.15.106:0/1582737045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954abb930 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 -- 172.21.15.106:0/1582737045 >> 172.21.15.106:0/1582737045 conn(0x55d954a476e0 msgr2=0x55d954aaa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 -- 172.21.15.106:0/1582737045 shutdown_connections 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 -- 172.21.15.106:0/1582737045 wait complete. 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 -- start start 2024-04-13T03:16:06.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.726+0000 7f6048405740 1 Processor -- start 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d954aa6420 con 0x55d954a80d40 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60401ffe20 con 0x55d954a81120 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d954adff20 con 0x55d954a9efc0 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6048405740 10 client.0 _ll_register_callbacks cb 0x7f60300040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38226/0 (socket says 172.21.15.106:38226) 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 -- 172.21.15.106:0/3324310161 learned_addr learned my addr 172.21.15.106:0/3324310161 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6037fff640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603f7fe640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 -- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 msgr2=0x55d954aa5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 -- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 msgr2=0x55d954aa55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 -- 172.21.15.106:0/3324310161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d954ade490 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f6037fff640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f603effd640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6030006dc0 tx=0x7f6030016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8032872aebbbac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.727+0000 7f601dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030027040 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f6030016ad0 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6030016de0 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6030047050 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 10 client.7920 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55d954b3fcf0 con 0x55d954a80d40 2024-04-13T03:16:06.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030016de0 con 0x55d954a80d40 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f603004a020 con 0x55d954a80d40 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 10 client.7920 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 10 client.7920 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 20 client.7920 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954aa39c0 0x55d954ac42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 4 client.7920 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.728+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55d9549ec930 con 0x55d954aa39c0 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f6037fff640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954aa39c0 0x55d954ac42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f601dffb640 10 client.7920 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f6037fff640 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954aa39c0 0x55d954ac42e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19fa5abe9ef590c server_cookie=32251da3fce63424 in_seq=0 out_seq=0 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f601dffb640 1 -- 172.21.15.106:0/3324310161 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d9549ec930 con 0x55d954aa39c0 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f601dffb640 10 client.7920 handle_command_reply: tid=0 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f6048405740 1 client.7920 shutdown 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 msgr2=0x55d954b3f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.729+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6030006dc0 tx=0x7f6030016000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 shutdown_connections 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x55d954aa39c0 0x55d954ac42e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d954a9efc0 0x55d954aa5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d954a81120 0x55d954aa55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 --2- 172.21.15.106:0/3324310161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d954a80d40 0x55d954b3f750 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 >> 172.21.15.106:0/3324310161 conn(0x55d954a476e0 msgr2=0x55d954aae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f601dffb640 0 client.7920 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 shutdown_connections 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 1 -- 172.21.15.106:0/3324310161 wait complete. 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6048405740 20 client.7920 trim_cache size 0 max 16384 2024-04-13T03:16:06.737 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:06.738 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:16:06.738 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "761ca48c-8883-4340-9215-da2d234562ce", 2024-04-13T03:16:06.738 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:16:06.738 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:06.738 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f8e693e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)->f(v1 m2024-04-13T03:15:25.822275+0000) n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)->n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5628b7fa4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 msgr2=0x7f602006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 0x7f602006ede0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6030006fd0 tx=0x7f6030008040 comp rx=0 tx=0).stop 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 msgr2=0x7f60400cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.730+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6038037040 tx=0x7f6038004a40 comp rx=0 tx=0).stop 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 shutdown_connections 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f604015b900 0x7f60400cbbd0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60400d3490 0x7f60400a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 --2- 172.21.15.106:0/3186081180 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60400b3920 0x7f604015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 >> 172.21.15.106:0/3186081180 conn(0x7f60400b9760 msgr2=0x7f60400b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 shutdown_connections 2024-04-13T03:16:06.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.731+0000 7f6045620640 1 -- 172.21.15.106:0/3186081180 wait complete. 2024-04-13T03:16:06.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:06.746 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:16:06.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.850+0000 7f5810b1e640 1 Processor -- start 2024-04-13T03:16:06.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.850+0000 7f5810b1e640 1 -- start start 2024-04-13T03:16:06.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f5810b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 0x7f580c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f5810b1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f580c0ccd40 0x7f580c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f5810b1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580c0d0640 con 0x7f580c15d380 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f5810b1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f580c0d0f70 con 0x7f580c0ccd40 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f5810b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f580c0d0920 con 0x7f580c0b7f40 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f580ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f580c0ccd40 0x7f580c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f580a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f580c15d380 0x7f580c06f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37080/0 (socket says 172.21.15.106:37080) 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.851+0000 7f580a59c640 1 -- 172.21.15.106:0/3765725469 learned_addr learned my addr 172.21.15.106:0/3765725469 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580ad9d640 1 -- 172.21.15.106:0/3765725469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 msgr2=0x7f580c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f5809d9b640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 0x7f580c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580959a640 1 -- 172.21.15.106:0/3765725469 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019623937 0 0) 0x7f580c0d0640 con 0x7f580c15d380 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580ad9d640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 0x7f580c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580ad9d640 1 -- 172.21.15.106:0/3765725469 >> v1:172.21.15.106:6789/0 conn(0x7f580c15d380 legacy=0x7f580c06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580ad9d640 1 -- 172.21.15.106:0/3765725469 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c0d0c00 con 0x7f580c0ccd40 2024-04-13T03:16:06.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f5809d9b640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 0x7f580c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.852+0000 7f580ad9d640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0ccd40 0x7f580c0cd1d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f58040097a0 tx=0x7f580402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadfd5126ebfd1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f580959a640 1 -- 172.21.15.106:0/3765725469 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804031780 con 0x7f580c0ccd40 2024-04-13T03:16:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f580959a640 1 -- 172.21.15.106:0/3765725469 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804037070 con 0x7f580c0ccd40 2024-04-13T03:16:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f580959a640 1 -- 172.21.15.106:0/3765725469 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804031b40 con 0x7f580c0ccd40 2024-04-13T03:16:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 -- 172.21.15.106:0/3765725469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0ccd40 msgr2=0x7f580c0cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0ccd40 0x7f580c0cd1d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f58040097a0 tx=0x7f580402b820 comp rx=0 tx=0).stop 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 -- 172.21.15.106:0/3765725469 shutdown_connections 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0ccd40 0x7f580c0cd1d0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 --2- 172.21.15.106:0/3765725469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c0b7f40 0x7f580c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 -- 172.21.15.106:0/3765725469 >> 172.21.15.106:0/3765725469 conn(0x7f580c0d2530 msgr2=0x7f580c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 -- 172.21.15.106:0/3765725469 shutdown_connections 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.853+0000 7f5810b1e640 1 -- 172.21.15.106:0/3765725469 wait complete. 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 Processor -- start 2024-04-13T03:16:06.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 -- start start 2024-04-13T03:16:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f580c0ccd40 0x7f580c0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c15d380 0x7f580c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f580c0d1580 con 0x7f580c0b7f40 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f580c0d1400 con 0x7f580c0ccd40 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f5810b1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f580c0d1700 con 0x7f580c15d380 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38246/0 (socket says 172.21.15.106:38246) 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.855+0000 7f580a59c640 1 -- 172.21.15.106:0/2651119560 learned_addr learned my addr 172.21.15.106:0/2651119560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 -- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c15d380 msgr2=0x7f580c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c15d380 0x7f580c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 -- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f580c0ccd40 msgr2=0x7f580c0ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f580c0ccd40 0x7f580c0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 -- 172.21.15.106:0/2651119560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c157550 con 0x7f580c0b7f40 2024-04-13T03:16:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f580a59c640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c7c0 tx=0x7f57fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f5802ffd640 1 -- 172.21.15.106:0/2651119560 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc018070 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f5802ffd640 1 -- 172.21.15.106:0/2651119560 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc00a640 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c14f3a0 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f5802ffd640 1 -- 172.21.15.106:0/2651119560 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00aa30 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.856+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c0cb200 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.857+0000 7f5802ffd640 1 -- 172.21.15.106:0/2651119560 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57fc01b070 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.858+0000 7f5802ffd640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.858+0000 7f5802ffd640 1 -- 172.21.15.106:0/2651119560 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f57fc08b3b0 con 0x7f580c0b7f40 2024-04-13T03:16:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.858+0000 7f5809d9b640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.858+0000 7f5809d9b640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006fd0 tx=0x7f57f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.859+0000 7f5813a09740 1 Processor -- start 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.859+0000 7f5813a09740 1 -- start start 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813eff0 0x561f08111f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f081124d0 0x561f081422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561f0807d170 con 0x561f0813f3d0 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f080237f0 con 0x561f081124d0 2024-04-13T03:16:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f08079130 con 0x561f0813eff0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38262/0 (socket says 172.21.15.106:38262) 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 -- 172.21.15.106:0/1031776220 learned_addr learned my addr 172.21.15.106:0/1031776220 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f580ad9d640 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f081124d0 0x561f081422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f580a59c640 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813eff0 0x561f08111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 -- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813eff0 msgr2=0x561f08111f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813eff0 0x561f08111f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 -- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f081124d0 msgr2=0x561f081422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f081124d0 0x561f081422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 -- 172.21.15.106:0/1031776220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f07e597d0 con 0x561f0813f3d0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f57f400cb20 tx=0x7f57f400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92897394b21f4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f57ebfff640 1 -- 172.21.15.106:0/1031776220 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019040 con 0x561f0813f3d0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f57ebfff640 1 -- 172.21.15.106:0/1031776220 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4022070 con 0x561f0813f3d0 2024-04-13T03:16:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.860+0000 7f57ebfff640 1 -- 172.21.15.106:0/1031776220 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4013da0 con 0x561f0813f3d0 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 -- 172.21.15.106:0/1031776220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 msgr2=0x561f08141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f57f400cb20 tx=0x7f57f400f000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 -- 172.21.15.106:0/1031776220 shutdown_connections 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f081124d0 0x561f081422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813f3d0 0x561f08141d80 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 --2- 172.21.15.106:0/1031776220 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813eff0 0x561f08111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 -- 172.21.15.106:0/1031776220 >> 172.21.15.106:0/1031776220 conn(0x561f08114200 msgr2=0x561f0815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 -- 172.21.15.106:0/1031776220 shutdown_connections 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.861+0000 7f5813a09740 1 -- 172.21.15.106:0/1031776220 wait complete. 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.862+0000 7f5813a09740 1 -- start start 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.862+0000 7f5813a09740 1 Processor -- start 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813f3d0 0x561f081403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0816d0a0 0x561f08138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561f081168a0 con 0x561f0813eff0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f08036d00 con 0x561f0813f3d0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f081190e0 con 0x561f0816d0a0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 10 client.0 _ll_register_callbacks cb 0x561f080366f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38268/0 (socket says 172.21.15.106:38268) 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 -- 172.21.15.106:0/3891656242 learned_addr learned my addr 172.21.15.106:0/3891656242 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5809d9b640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813f3d0 0x561f081403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 -- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0816d0a0 msgr2=0x561f08138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0816d0a0 0x561f08138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 -- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813f3d0 msgr2=0x561f081403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813f3d0 0x561f081403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 -- 172.21.15.106:0/3891656242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f07d8f3f0 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f580a59c640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0552f0 tx=0x7f57fc059700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db98b2561d69423f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f57e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0615e0 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f57fc061780 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.863+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f4022070 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0891a0 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 10 client.7935 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561f07d4f0f0 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4022070 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f57fc071170 con 0x561f0813eff0 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 10 client.7935 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 10 client.7935 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 20 client.7935 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f0815d0c0 0x561f081705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 4 client.7935 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f0808ab90 con 0x561f0815d0c0 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5809d9b640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f0815d0c0 0x561f081705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f57e97fa640 10 client.7935 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.864+0000 7f5809d9b640 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f0815d0c0 0x561f081705b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3353825391d51e95 server_cookie=9a56329e38d4272c in_seq=0 out_seq=0 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f57e97fa640 1 -- 172.21.15.106:0/3891656242 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f0808ab90 con 0x561f0815d0c0 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f57e97fa640 10 client.7935 handle_command_reply: tid=0 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 client.7935 shutdown 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 msgr2=0x561f08128b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0552f0 tx=0x7f57fc059700 comp rx=0 tx=0).stop 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 shutdown_connections 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f0815d0c0 0x561f081705b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0816d0a0 0x561f08138290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813f3d0 0x561f081403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 --2- 172.21.15.106:0/3891656242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f08128b30 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f57e97fa640 0 client.7935 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 >> 172.21.15.106:0/3891656242 conn(0x561f081124d0 msgr2=0x561f08116e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 shutdown_connections 2024-04-13T03:16:06.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 1 -- 172.21.15.106:0/3891656242 wait complete. 2024-04-13T03:16:06.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.865+0000 7f5813a09740 20 client.7935 trim_cache size 0 max 16384 2024-04-13T03:16:06.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 Processor -- start 2024-04-13T03:16:06.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 -- start start 2024-04-13T03:16:06.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813eff0 0x561f08180ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08115d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ff3e0 con 0x561f0816d0a0 2024-04-13T03:16:06.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1fe3c0 con 0x561f0813eff0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f580c1588e0 con 0x561f0813f3d0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f580ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f580ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38272/0 (socket says 172.21.15.106:38272) 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.883+0000 7f580ad9d640 1 -- 172.21.15.106:0/686088703 learned_addr learned my addr 172.21.15.106:0/686088703 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580a59c640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813eff0 0x561f08180ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f5809d9b640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 -- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 msgr2=0x561f08115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 -- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813eff0 msgr2=0x561f08180ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813eff0 0x561f08180ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 -- 172.21.15.106:0/686088703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f081702d0 con 0x561f0816d0a0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f5809d9b640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f580ad9d640 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f580402be30 tx=0x7f580402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8fec463185d3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f57eb7fe640 1 -- 172.21.15.106:0/686088703 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580404b070 con 0x561f0816d0a0 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f57eb7fe640 1 -- 172.21.15.106:0/686088703 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804037040 con 0x561f0816d0a0 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.884+0000 7f57eb7fe640 1 -- 172.21.15.106:0/686088703 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804031c20 con 0x561f0816d0a0 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- 172.21.15.106:0/686088703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 msgr2=0x561f0816fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f580402be30 tx=0x7f580402e000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- 172.21.15.106:0/686088703 shutdown_connections 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0816d0a0 0x561f0816fd90 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 --2- 172.21.15.106:0/686088703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0813eff0 0x561f08180ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- 172.21.15.106:0/686088703 >> 172.21.15.106:0/686088703 conn(0x561f081124d0 msgr2=0x561f08142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- 172.21.15.106:0/686088703 shutdown_connections 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- 172.21.15.106:0/686088703 wait complete. 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 -- start start 2024-04-13T03:16:06.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.885+0000 7f5813a09740 1 Processor -- start 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08141410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0816d0a0 0x561f08141950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f580c158f30 con 0x561f0813eff0 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f0813ade0 con 0x561f0816d0a0 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561f0815e000 con 0x561f0813f3d0 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5813a09740 10 client.0 _ll_register_callbacks cb 0x7f57fc001400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38278/0 (socket says 172.21.15.106:38278) 2024-04-13T03:16:06.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580a59c640 1 -- 172.21.15.106:0/1469393276 learned_addr learned my addr 172.21.15.106:0/1469393276 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580ad9d640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0816d0a0 0x561f08141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580a59c640 1 -- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 msgr2=0x561f08141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.886+0000 7f580a59c640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f580a59c640 1 -- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0816d0a0 msgr2=0x561f08141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f580a59c640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0816d0a0 0x561f08141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f580a59c640 1 -- 172.21.15.106:0/1469393276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f0813e760 con 0x561f0813eff0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f580a59c640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f57fc059050 tx=0x7f57fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77676bfc70292f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc075070 con 0x561f0813eff0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f57fc0617e0 con 0x561f0813eff0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5804037040 con 0x561f0813eff0 2024-04-13T03:16:06.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0589d0 con 0x561f0813eff0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 10 client.7941 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561f0812c990 con 0x561f0813eff0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804037040 con 0x561f0813eff0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f57fc078020 con 0x561f0813eff0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 10 client.7941 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 10 client.7941 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 20 client.7941 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f081da590 0x561f0813d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 4 client.7941 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f08084a00 con 0x561f081da590 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.887+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f081da590 0x561f0813d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f57e97fa640 10 client.7941 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5809d9b640 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f081da590 0x561f0813d6e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d17aef48778adf server_cookie=a381e0ed39a7006a in_seq=0 out_seq=0 2024-04-13T03:16:06.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f57e97fa640 1 -- 172.21.15.106:0/1469393276 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x561f08084a00 con 0x561f081da590 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f57e97fa640 10 client.7941 handle_command_reply: tid=0 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 client.7941 shutdown 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 msgr2=0x561f0812c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f57fc059050 tx=0x7f57fc014000 comp rx=0 tx=0).stop 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 shutdown_connections 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x561f081da590 0x561f0813d6e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561f0813f3d0 0x561f08141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f0816d0a0 0x561f08141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f57e97fa640 0 client.7941 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 --2- 172.21.15.106:0/1469393276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561f0813eff0 0x561f0812c450 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 >> 172.21.15.106:0/1469393276 conn(0x561f081124d0 msgr2=0x561f081390d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 shutdown_connections 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 1 -- 172.21.15.106:0/1469393276 wait complete. 2024-04-13T03:16:06.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.888+0000 7f5813a09740 20 client.7941 trim_cache size 0 max 16384 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "761ca48c-8883-4340-9215-da2d234562ce": { 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir", 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "761ca48c-8883-4340-9215-da2d234562ce", 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:06.898 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:06.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 msgr2=0x7f57ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006fd0 tx=0x7f57f4008040 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 msgr2=0x7f580c14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c7c0 tx=0x7f57fc00cc80 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 shutdown_connections 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f580c15d380 0x7f580c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f580c0ccd40 0x7f580c0ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 --2- 172.21.15.106:0/2651119560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f580c0b7f40 0x7f580c14ee60 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 >> 172.21.15.106:0/2651119560 conn(0x7f580c0d2530 msgr2=0x7f580c0c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 shutdown_connections 2024-04-13T03:16:06.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:06.889+0000 7f5810b1e640 1 -- 172.21.15.106:0/2651119560 wait complete. 2024-04-13T03:16:06.900 INFO:tasks.cephfs.filesystem:scrub status for tag:761ca48c-8883-4340-9215-da2d234562ce - {'path': '/dir', 'tag': '761ca48c-8883-4340-9215-da2d234562ce', 'options': 'recursive,repair'} 2024-04-13T03:16:25.871 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:16:25.875 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:16:36.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:36.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:16:37.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.015+0000 7f9d8e236640 1 Processor -- start 2024-04-13T03:16:37.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.015+0000 7f9d8e236640 1 -- start start 2024-04-13T03:16:37.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d8e236640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9d8806bff0 0x7f9d8806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d8e236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d880cef80 con 0x7f9d880d07c0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d8e236640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d880cf260 con 0x7f9d8815fb50 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d8e236640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9d880cf540 con 0x7f9d8806bff0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d87fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9d880d07c0 0x7f9d8806ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57820/0 (socket says 172.21.15.106:57820) 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d87fff640 1 -- 172.21.15.106:0/2900216387 learned_addr learned my addr 172.21.15.106:0/2900216387 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/2900216387 >> v2:172.21.15.114:3300/0 conn(0x7f9d8806bff0 0x7f9d8806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342565343 0 0) 0x7f9d880cef80 con 0x7f9d880d07c0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d70003520 con 0x7f9d880d07c0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3974694518 0 0) 0x7f9d70003520 con 0x7f9d880d07c0 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 >> v2:172.21.15.114:3300/0 conn(0x7f9d8806bff0 msgr2=0x7f9d8806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 --2- 172.21.15.106:0/2900216387 >> v2:172.21.15.114:3300/0 conn(0x7f9d8806bff0 0x7f9d8806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 >> v1:172.21.15.114:6789/0 conn(0x7f9d8815fb50 legacy=0x7f9d8806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d880cf820 con 0x7f9d880d07c0 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9d800019f0 con 0x7f9d880d07c0 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 >> v1:172.21.15.106:6789/0 conn(0x7f9d880d07c0 legacy=0x7f9d8806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880cf540 con 0x7f9d700051c0 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000c160 con 0x7f9d700039b0 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.016+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7000c360 con 0x7f9d70004800 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/2900216387 >> v2:172.21.15.114:3300/0 conn(0x7f9d8806bff0 0x7f9d8806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d877fe640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 -- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d70004800 msgr2=0x7f9d70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 -- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700039b0 msgr2=0x7f9d70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 -- 172.21.15.106:0/2900216387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d70003520 con 0x7f9d700051c0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.017+0000 7f9d87fff640 1 -- 172.21.15.106:0/2900216387 reap_dead start 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d87fff640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000a830 tx=0x7f9d70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c934fdf6c749eba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000a480 con 0x7f9d700051c0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80034630 con 0x7f9d700051c0 2024-04-13T03:16:37.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d867fc640 1 -- 172.21.15.106:0/2900216387 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80034a00 con 0x7f9d700051c0 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 -- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 msgr2=0x7f9d7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000a830 tx=0x7f9d70005830 comp rx=0 tx=0).stop 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 -- 172.21.15.106:0/2900216387 shutdown_connections 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 --2- 172.21.15.106:0/2900216387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 -- 172.21.15.106:0/2900216387 >> 172.21.15.106:0/2900216387 conn(0x7f9d880cd6a0 msgr2=0x7f9d880cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 -- 172.21.15.106:0/2900216387 shutdown_connections 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.018+0000 7f9d8e236640 1 -- 172.21.15.106:0/2900216387 wait complete. 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 Processor -- start 2024-04-13T03:16:37.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 -- start start 2024-04-13T03:16:37.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700051c0 0x7f9d881487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d881527f0 0x7f9d880b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880ced20 con 0x7f9d88158600 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880ceba0 con 0x7f9d700051c0 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.020+0000 7f9d8e236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8815ef10 con 0x7f9d881527f0 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51086/0 (socket says 172.21.15.106:51086) 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 -- 172.21.15.106:0/955125618 learned_addr learned my addr 172.21.15.106:0/955125618 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d877fe640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700051c0 0x7f9d881487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d881527f0 0x7f9d880b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 -- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d881527f0 msgr2=0x7f9d880b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d881527f0 0x7f9d880b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 -- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700051c0 msgr2=0x7f9d881487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700051c0 0x7f9d881487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 -- 172.21.15.106:0/955125618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d880b3e80 con 0x7f9d88158600 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d87fff640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000a830 tx=0x7f9d880cf430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d6ffff640 1 -- 172.21.15.106:0/955125618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80039040 con 0x7f9d88158600 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d6ffff640 1 -- 172.21.15.106:0/955125618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80044070 con 0x7f9d88158600 2024-04-13T03:16:37.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d6ffff640 1 -- 172.21.15.106:0/955125618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000a480 con 0x7f9d88158600 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d88157560 con 0x7f9d88158600 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.021+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d88157b20 con 0x7f9d88158600 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.023+0000 7f9d6ffff640 1 -- 172.21.15.106:0/955125618 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d80034630 con 0x7f9d88158600 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.023+0000 7f9d6ffff640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 0x7f9d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.023+0000 7f9d6ffff640 1 -- 172.21.15.106:0/955125618 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f9d800ad9d0 con 0x7f9d88158600 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.023+0000 7f9d877fe640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 0x7f9d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.024+0000 7f9d877fe640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 0x7f9d7406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.024+0000 7f9d91024740 1 Processor -- start 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.024+0000 7f9d91024740 1 -- start start 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645cfff0 0x564d645a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645a34d0 0x564d645d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d6450e170 con 0x564d645d03d0 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564d644b47f0 con 0x564d645cfff0 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564d6450a130 con 0x564d645a34d0 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51092/0 (socket says 172.21.15.106:51092) 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3193920314 learned_addr learned my addr 172.21.15.106:0/3193920314 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645cfff0 0x564d645a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d87fff640 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645a34d0 0x564d645d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645a34d0 msgr2=0x564d645d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645a34d0 0x564d645d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645cfff0 msgr2=0x564d645a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645cfff0 0x564d645a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.025+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3193920314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d642ea7d0 con 0x564d645d03d0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009cf0 tx=0x7f9d7c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1f2de6eb81294e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d6d7fa640 1 -- 172.21.15.106:0/3193920314 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c019070 con 0x564d645d03d0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d6d7fa640 1 -- 172.21.15.106:0/3193920314 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c004030 con 0x564d645d03d0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d6d7fa640 1 -- 172.21.15.106:0/3193920314 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00f040 con 0x564d645d03d0 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d91024740 1 -- 172.21.15.106:0/3193920314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 msgr2=0x564d645d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d91024740 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009cf0 tx=0x7f9d7c009190 comp rx=0 tx=0).stop 2024-04-13T03:16:37.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.026+0000 7f9d91024740 1 -- 172.21.15.106:0/3193920314 shutdown_connections 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645a34d0 0x564d645d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645d03d0 0x564d645d2d80 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 --2- 172.21.15.106:0/3193920314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645cfff0 0x564d645a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 -- 172.21.15.106:0/3193920314 >> 172.21.15.106:0/3193920314 conn(0x564d645a5200 msgr2=0x564d645edd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 -- 172.21.15.106:0/3193920314 shutdown_connections 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 -- 172.21.15.106:0/3193920314 wait complete. 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 -- start start 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.027+0000 7f9d91024740 1 Processor -- start 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64601840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d645a78a0 con 0x564d645c1890 2024-04-13T03:16:37.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564d644c7d00 con 0x564d645d03d0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564d645aa0e0 con 0x564d645cfff0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d91024740 10 client.0 _ll_register_callbacks cb 0x564d644c76f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51098/0 (socket says 172.21.15.106:51098) 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 -- 172.21.15.106:0/3036587084 learned_addr learned my addr 172.21.15.106:0/3036587084 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64601840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 -- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 msgr2=0x564d645d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 -- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 msgr2=0x564d64601840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64601840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 -- 172.21.15.106:0/3036587084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d642203f0 con 0x564d645c1890 2024-04-13T03:16:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d87fff640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9d80001cf0 tx=0x7f9d8007af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28aef8134bc0936 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.028+0000 7f9d667fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80084120 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f9d80077bb0 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c004030 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9d8007a990 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 10 client.7953 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x564d641e00f0 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c004030 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f9d8007ab30 con 0x564d645c1890 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 10 client.7953 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 10 client.7953 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 20 client.7953 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d645d1a40 0x564d645d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 4 client.7953 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d6451bb90 con 0x564d645d1a40 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.029+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d645d1a40 0x564d645d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d667fc640 10 client.7953 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d645d1a40 0x564d645d1e20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=569417decf0afa0 server_cookie=40ca179a2481f6d in_seq=0 out_seq=0 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d667fc640 1 -- 172.21.15.106:0/3036587084 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d6451bb90 con 0x564d645d1a40 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d667fc640 10 client.7953 handle_command_reply: tid=0 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 client.7953 shutdown 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 msgr2=0x564d64601d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9d80001cf0 tx=0x7f9d8007af60 comp rx=0 tx=0).stop 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 shutdown_connections 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d645d1a40 0x564d645d1e20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d64601d80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64601840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d667fc640 0 client.7953 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 --2- 172.21.15.106:0/3036587084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 >> 172.21.15.106:0/3036587084 conn(0x564d645a34d0 msgr2=0x564d645f1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 shutdown_connections 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.030+0000 7f9d91024740 1 -- 172.21.15.106:0/3036587084 wait complete. 2024-04-13T03:16:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.031+0000 7f9d91024740 20 client.7953 trim_cache size 0 max 16384 2024-04-13T03:16:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.048+0000 7f9d91024740 1 Processor -- start 2024-04-13T03:16:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.048+0000 7f9d91024740 1 -- start start 2024-04-13T03:16:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645c1890 0x564d645d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645d03d0 0x564d645a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d645ec3a0 con 0x564d645cfff0 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88201e30 con 0x564d645c1890 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9d88200e10 con 0x564d645d03d0 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51108/0 (socket says 172.21.15.106:51108) 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3677094290 learned_addr learned my addr 172.21.15.106:0/3677094290 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645d03d0 msgr2=0x564d645a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645d03d0 0x564d645a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645c1890 msgr2=0x564d645d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645c1890 0x564d645d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 -- 172.21.15.106:0/3677094290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d64612010 con 0x564d645cfff0 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009cc0 tx=0x7f9d7c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e8e1264cae025 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d6cff9640 1 -- 172.21.15.106:0/3677094290 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c02e070 con 0x564d645cfff0 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d6cff9640 1 -- 172.21.15.106:0/3677094290 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c004030 con 0x564d645cfff0 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.049+0000 7f9d6cff9640 1 -- 172.21.15.106:0/3677094290 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c019040 con 0x564d645cfff0 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 -- 172.21.15.106:0/3677094290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 msgr2=0x564d645a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009cc0 tx=0x7f9d7c010000 comp rx=0 tx=0).stop 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 -- 172.21.15.106:0/3677094290 shutdown_connections 2024-04-13T03:16:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645d03d0 0x564d645a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645cfff0 0x564d645a7990 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 --2- 172.21.15.106:0/3677094290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645c1890 0x564d645d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 -- 172.21.15.106:0/3677094290 >> 172.21.15.106:0/3677094290 conn(0x564d645a34d0 msgr2=0x564d645d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 -- 172.21.15.106:0/3677094290 shutdown_connections 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.050+0000 7f9d91024740 1 -- 172.21.15.106:0/3677094290 wait complete. 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 -- start start 2024-04-13T03:16:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 Processor -- start 2024-04-13T03:16:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64667750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88200780 con 0x564d645c1890 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88200130 con 0x564d645d03d0 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.051+0000 7f9d91024740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564d646684e0 con 0x564d645cfff0 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 10 client.0 _ll_register_callbacks cb 0x7f9d8007b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51116/0 (socket says 172.21.15.106:51116) 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 -- 172.21.15.106:0/3157125202 learned_addr learned my addr 172.21.15.106:0/3157125202 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 -- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 msgr2=0x564d645ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 -- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 msgr2=0x564d64667750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64667750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 -- 172.21.15.106:0/3157125202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d645a7990 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d877fe640 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9d78012ae0 tx=0x7f9d78007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc273450f31d609a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78007670 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f9d78010040 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c004030 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9d780069e0 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 10 client.7959 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x564d64668220 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c004030 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f9d78006b80 con 0x564d645c1890 2024-04-13T03:16:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 10 client.7959 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.052+0000 7f9d91024740 10 client.7959 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 20 client.7959 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d6465e3e0 0x564d64667c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 4 client.7959 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d64515a00 con 0x564d6465e3e0 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d6465e3e0 0x564d64667c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d667fc640 10 client.7959 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d86ffd640 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d6465e3e0 0x564d64667c90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd29a499fd483855 server_cookie=1c78210b33a2ceae in_seq=0 out_seq=0 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d667fc640 1 -- 172.21.15.106:0/3157125202 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d64515a00 con 0x564d6465e3e0 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d667fc640 10 client.7959 handle_command_reply: tid=0 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 client.7959 shutdown 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 msgr2=0x564d645d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9d78012ae0 tx=0x7f9d78007c80 comp rx=0 tx=0).stop 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 shutdown_connections 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564d6465e3e0 0x564d64667c90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564d645cfff0 0x564d645ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d645d03d0 0x564d64667750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 --2- 172.21.15.106:0/3157125202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d645c1890 0x564d645d4ee0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d667fc640 0 client.7959 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 >> 172.21.15.106:0/3157125202 conn(0x564d645a34d0 msgr2=0x564d645cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 shutdown_connections 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 1 -- 172.21.15.106:0/3157125202 wait complete. 2024-04-13T03:16:37.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.053+0000 7f9d91024740 20 client.7959 trim_cache size 0 max 16384 2024-04-13T03:16:37.063 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:37.063 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:16:37.063 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:16:37.063 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:37.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 msgr2=0x7f9d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 0x7f9d7406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d78009040 comp rx=0 tx=0).stop 2024-04-13T03:16:37.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 msgr2=0x7f9d880aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000a830 tx=0x7f9d880cf430 comp rx=0 tx=0).stop 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 shutdown_connections 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9d7406c8f0 0x7f9d7406edb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9d881527f0 0x7f9d880b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88158600 0x7f9d880aed10 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 --2- 172.21.15.106:0/955125618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d700051c0 0x7f9d881487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 >> 172.21.15.106:0/955125618 conn(0x7f9d880cd6a0 msgr2=0x7f9d880b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 shutdown_connections 2024-04-13T03:16:37.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.054+0000 7f9d8e236640 1 -- 172.21.15.106:0/955125618 wait complete. 2024-04-13T03:16:37.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:16:37.067 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.166+0000 7fe5d379c640 1 Processor -- start 2024-04-13T03:16:37.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.166+0000 7fe5d379c640 1 -- start start 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d379c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5cc0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d379c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5cc163510 con 0x7fe5cc0caf30 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d379c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5cc0cd700 con 0x7fe5cc144f10 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d379c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe5cc0cd4d0 con 0x7fe5cc1630c0 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d0d37640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe5cc0caf30 0x7fe5cc0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57836/0 (socket says 172.21.15.106:57836) 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d0d37640 1 -- 172.21.15.106:0/494744085 learned_addr learned my addr 172.21.15.106:0/494744085 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5cc0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075085600 0 0) 0x7fe5cc163510 con 0x7fe5cc0caf30 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5b4003520 con 0x7fe5cc0caf30 2024-04-13T03:16:37.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.167+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000817712 0 0) 0x7fe5cc0cd700 con 0x7fe5cc144f10 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5cc163510 con 0x7fe5cc144f10 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2072491831 0 0) 0x7fe5b4003520 con 0x7fe5cc0caf30 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 msgr2=0x7fe5cc0d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5cc0d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 >> v1:172.21.15.114:6789/0 conn(0x7fe5cc144f10 legacy=0x7fe5cc0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc0cd9e0 con 0x7fe5cc0caf30 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fe5c0001670 con 0x7fe5cc0caf30 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 >> v1:172.21.15.106:6789/0 conn(0x7fe5cc0caf30 legacy=0x7fe5cc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5b4003b70 0x7fe5b4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4004d40 0x7fe5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b400ba70 con 0x7fe5b4003b70 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b400bbf0 con 0x7fe5b4004d40 2024-04-13T03:16:37.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b400bdf0 con 0x7fe5cc1630c0 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5b4003b70 0x7fe5b4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d0d37640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4004d40 0x7fe5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 -- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 msgr2=0x7fe5b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 -- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4004d40 msgr2=0x7fe5b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4004d40 0x7fe5b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1538640 1 -- 172.21.15.106:0/494744085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b4003520 con 0x7fe5b4003b70 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d0d37640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5cc1630c0 0x7fe5b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.168+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4004d40 0x7fe5b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d1538640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5b4003b70 0x7fe5b4003f70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc001fd0 tx=0x7fe5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5096dadffd9d3e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc001bf0 con 0x7fe5b4003b70 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc003b70 con 0x7fe5b4003b70 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5cbfff640 1 -- 172.21.15.106:0/494744085 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0050d0 con 0x7fe5b4003b70 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d379c640 1 -- 172.21.15.106:0/494744085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5b4003b70 msgr2=0x7fe5b4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d379c640 1 --2- 172.21.15.106:0/494744085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5b4003b70 0x7fe5b4003f70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc001fd0 tx=0x7fe5b4005830 comp rx=0 tx=0).stop 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d1d39640 1 -- 172.21.15.106:0/494744085 reap_dead start 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d379c640 1 -- 172.21.15.106:0/494744085 shutdown_connections 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d379c640 1 -- 172.21.15.106:0/494744085 >> 172.21.15.106:0/494744085 conn(0x7fe5cc06f180 msgr2=0x7fe5cc06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.169+0000 7fe5d379c640 1 -- 172.21.15.106:0/494744085 shutdown_connections 2024-04-13T03:16:37.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.170+0000 7fe5d379c640 1 -- 172.21.15.106:0/494744085 wait complete. 2024-04-13T03:16:37.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 Processor -- start 2024-04-13T03:16:37.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 -- start start 2024-04-13T03:16:37.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5b4003b70 0x7fe5cc1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 0x7fe5cc072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc06c150 con 0x7fe5cc0b0100 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc0cf120 con 0x7fe5cc0afca0 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d379c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe5cc06c2d0 con 0x7fe5b4003b70 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.171+0000 7fe5d1d39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51136/0 (socket says 172.21.15.106:51136) 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 -- 172.21.15.106:0/2565080498 learned_addr learned my addr 172.21.15.106:0/2565080498 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d0d37640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 0x7fe5cc072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1538640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5b4003b70 0x7fe5cc1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 -- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5b4003b70 msgr2=0x7fe5cc1f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5b4003b70 0x7fe5cc1f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 -- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 msgr2=0x7fe5cc072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 0x7fe5cc072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 -- 172.21.15.106:0/2565080498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1f88e0 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d0d37640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 0x7fe5cc072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d1d39640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc072980 tx=0x7fe5c4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4019020 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4003610 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4003a00 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc14f270 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.172+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc14f830 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.173+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc068a10 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.174+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5c401c020 con 0x7fe5cc0b0100 2024-04-13T03:16:37.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.174+0000 7fe5c9ffb640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 0x7fe5b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.174+0000 7fe5d1538640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 0x7fe5b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.174+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fe5c408bf40 con 0x7fe5cc0b0100 2024-04-13T03:16:37.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.174+0000 7fe5d1538640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 0x7fe5b806edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc034000 tx=0x7fe5bc009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.179+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5c4023030 con 0x7fe5cc0b0100 2024-04-13T03:16:37.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.392+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5cc14ce90 con 0x7fe5cc0b0100 2024-04-13T03:16:37.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.393+0000 7fe5c9ffb640 1 -- 172.21.15.106:0/2565080498 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5193 (secure 0 0 0) 0x7fe5c405a1b0 con 0x7fe5cc0b0100 2024-04-13T03:16:37.394 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:37.394 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:39.667524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7728,"qdb_cluster":[7728]},"id":6}]} 2024-04-13T03:16:37.394 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 58 2024-04-13T03:16:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 msgr2=0x7fe5b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 0x7fe5b806edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc034000 tx=0x7fe5bc009030 comp rx=0 tx=0).stop 2024-04-13T03:16:37.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 msgr2=0x7fe5cc0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc072980 tx=0x7fe5c4011920 comp rx=0 tx=0).stop 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 shutdown_connections 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5b806c8f0 0x7fe5b806edb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0b0100 0x7fe5cc0732e0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc0afca0 0x7fe5cc072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 --2- 172.21.15.106:0/2565080498 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5b4003b70 0x7fe5cc1f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 >> 172.21.15.106:0/2565080498 conn(0x7fe5cc06f180 msgr2=0x7fe5cc0ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 shutdown_connections 2024-04-13T03:16:37.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.395+0000 7fe5d379c640 1 -- 172.21.15.106:0/2565080498 wait complete. 2024-04-13T03:16:37.407 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-13T03:16:37.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.508+0000 7f433ecde640 1 Processor -- start 2024-04-13T03:16:37.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.508+0000 7f433ecde640 1 -- start start 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ecde640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4338156b40 0x7f433806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ecde640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f433806ebf0 0x7f433815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ecde640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433815d690 con 0x7f433815d110 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ecde640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f433806b8e0 con 0x7f433806ebf0 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ecde640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f433806bea0 con 0x7f4338156b40 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433d27b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f433815d110 0x7f433806c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57842/0 (socket says 172.21.15.106:57842) 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433d27b640 1 -- 172.21.15.106:0/3187457618 learned_addr learned my addr 172.21.15.106:0/3187457618 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f432ffff640 1 --2- 172.21.15.106:0/3187457618 >> v2:172.21.15.114:3300/0 conn(0x7f433806ebf0 0x7f433815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f433ca7a640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4338156b40 0x7f433806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f432f7fe640 1 -- 172.21.15.106:0/3187457618 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 876601947 0 0) 0x7f433815d690 con 0x7f433815d110 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.509+0000 7f432f7fe640 1 -- 172.21.15.106:0/3187457618 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4320003520 con 0x7f433815d110 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432ffff640 1 -- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4338156b40 msgr2=0x7f433806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432ffff640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4338156b40 0x7f433806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432ffff640 1 -- 172.21.15.106:0/3187457618 >> v1:172.21.15.106:6789/0 conn(0x7f433815d110 legacy=0x7f433806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432ffff640 1 -- 172.21.15.106:0/3187457618 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433815d970 con 0x7f433806ebf0 2024-04-13T03:16:37.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432ffff640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f433806ebf0 0x7f433815cbd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43280097a0 tx=0x7f432802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1cbdf561864526 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.510+0000 7f432f7fe640 1 -- 172.21.15.106:0/3187457618 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328035020 con 0x7f433806ebf0 2024-04-13T03:16:37.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f432f7fe640 1 -- 172.21.15.106:0/3187457618 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4328031830 con 0x7f433806ebf0 2024-04-13T03:16:37.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f432f7fe640 1 -- 172.21.15.106:0/3187457618 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328031be0 con 0x7f433806ebf0 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 -- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f433806ebf0 msgr2=0x7f433815cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f433806ebf0 0x7f433815cbd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43280097a0 tx=0x7f432802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 -- 172.21.15.106:0/3187457618 shutdown_connections 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f433806ebf0 0x7f433815cbd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 --2- 172.21.15.106:0/3187457618 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4338156b40 0x7f433806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 -- 172.21.15.106:0/3187457618 >> 172.21.15.106:0/3187457618 conn(0x7f43380cd6a0 msgr2=0x7f43380cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 -- 172.21.15.106:0/3187457618 shutdown_connections 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.511+0000 7f433ecde640 1 -- 172.21.15.106:0/3187457618 wait complete. 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.512+0000 7f433ecde640 1 Processor -- start 2024-04-13T03:16:37.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.512+0000 7f433ecde640 1 -- start start 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f433806ebf0 0x7f43380c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4338156b40 0x7f43380b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f433806d880 con 0x7f433815d110 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4338156f20 con 0x7f4338156b40 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ecde640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f433806da00 con 0x7f433806ebf0 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51138/0 (socket says 172.21.15.106:51138) 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 -- 172.21.15.106:0/2502005659 learned_addr learned my addr 172.21.15.106:0/2502005659 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f432ffff640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4338156b40 0x7f43380b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f433806ebf0 0x7f43380c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 -- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f433806ebf0 msgr2=0x7f43380c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f433806ebf0 0x7f43380c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 -- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4338156b40 msgr2=0x7f43380b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.513+0000 7f433d27b640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4338156b40 0x7f43380b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f433d27b640 1 -- 172.21.15.106:0/2502005659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380b2780 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f433d27b640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f433000ca20 tx=0x7f433000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f432d7fa640 1 -- 172.21.15.106:0/2502005659 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017070 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f432d7fa640 1 -- 172.21.15.106:0/2502005659 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433000abd0 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f432d7fa640 1 -- 172.21.15.106:0/2502005659 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330012400 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380ca510 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.514+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43381f8890 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.515+0000 7f432d7fa640 1 -- 172.21.15.106:0/2502005659 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f433001a070 con 0x7f433815d110 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.516+0000 7f432d7fa640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 0x7f432406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.516+0000 7f432d7fa640 1 -- 172.21.15.106:0/2502005659 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f433008ba10 con 0x7f433815d110 2024-04-13T03:16:37.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.516+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 0x7f432406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.516+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 0x7f432406ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4334009ad0 tx=0x7f4334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 Processor -- start 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 -- start start 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fcb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448ff4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564448f2efe0 con 0x564448fc9800 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564448ecb860 con 0x564448ff46d0 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.517+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564448f2afa0 con 0x564448ff1af0 2024-04-13T03:16:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51154/0 (socket says 172.21.15.106:51154) 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 -- 172.21.15.106:0/1808278926 learned_addr learned my addr 172.21.15.106:0/1808278926 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 -- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 msgr2=0x564448ff4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448ff4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 -- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 msgr2=0x564448fcb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fcb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 -- 172.21.15.106:0/1808278926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564448efa1a0 con 0x564448fc9800 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f433d27b640 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4330055a10 tx=0x7f433005d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9926fbbfb81a6e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f431e7fc640 1 -- 172.21.15.106:0/1808278926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330055c20 con 0x564448fc9800 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f431e7fc640 1 -- 172.21.15.106:0/1808278926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330061c50 con 0x564448fc9800 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.518+0000 7f431e7fc640 1 -- 172.21.15.106:0/1808278926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330055dc0 con 0x564448fc9800 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- 172.21.15.106:0/1808278926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 msgr2=0x564448ff0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4330055a10 tx=0x7f433005d9f0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- 172.21.15.106:0/1808278926 shutdown_connections 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448ff4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448ff0f90 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 --2- 172.21.15.106:0/1808278926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fcb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- 172.21.15.106:0/1808278926 >> 172.21.15.106:0/1808278926 conn(0x564448f5ddc0 msgr2=0x56444900f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- 172.21.15.106:0/1808278926 shutdown_connections 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- 172.21.15.106:0/1808278926 wait complete. 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.519+0000 7f4341bad740 1 -- start start 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 Processor -- start 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448feb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fe7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564448fc4b00 con 0x564448fc9800 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564448ec91d0 con 0x564448ff46d0 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56444900fc90 con 0x564448ff1af0 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f4341bad740 10 client.0 _ll_register_callbacks cb 0x564448eec010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51160/0 (socket says 172.21.15.106:51160) 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.520+0000 7f433ca7a640 1 -- 172.21.15.106:0/2645490867 learned_addr learned my addr 172.21.15.106:0/2645490867 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433d27b640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fe7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 -- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 msgr2=0x564448feb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f432ffff640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448feb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448feb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 -- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 msgr2=0x564448fe7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fe7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 -- 172.21.15.106:0/2645490867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564448d0d7c0 con 0x564448fc9800 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4334015720 tx=0x7f4334009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4a24dbde8a4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334014770 con 0x564448fc9800 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f4334014910 con 0x564448fc9800 2024-04-13T03:16:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4330061c50 con 0x564448fc9800 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f433402d050 con 0x564448fc9800 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 10 client.7977 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x564448d6df50 con 0x564448fc9800 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330061c50 con 0x564448fc9800 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f4334014b10 con 0x564448fc9800 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 10 client.7977 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 10 client.7977 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.521+0000 7f4341bad740 20 client.7977 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564448fe9d50 0x564449021970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f4341bad740 4 client.7977 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564448ece4a0 con 0x564448fe9d50 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f432ffff640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564448fe9d50 0x564449021970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f43177fe640 10 client.7977 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f432ffff640 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564448fe9d50 0x564449021970 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c353479013b4b878 server_cookie=cd1f061fab1d467f in_seq=0 out_seq=0 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f43177fe640 1 -- 172.21.15.106:0/2645490867 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564448ece4a0 con 0x564448fe9d50 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f43177fe640 10 client.7977 handle_command_reply: tid=0 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.522+0000 7f4341bad740 1 client.7977 shutdown 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 msgr2=0x564449020470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4334015720 tx=0x7f4334009190 comp rx=0 tx=0).stop 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 shutdown_connections 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564448fe9d50 0x564449021970 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448feb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564448fe7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 --2- 172.21.15.106:0/2645490867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564449020470 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f43177fe640 0 client.7977 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 >> 172.21.15.106:0/2645490867 conn(0x564448f5ddc0 msgr2=0x564448fc35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 shutdown_connections 2024-04-13T03:16:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 1 -- 172.21.15.106:0/2645490867 wait complete. 2024-04-13T03:16:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.523+0000 7f4341bad740 20 client.7977 trim_cache size 0 max 16384 2024-04-13T03:16:37.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.536+0000 7f4341bad740 1 Processor -- start 2024-04-13T03:16:37.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.536+0000 7f4341bad740 1 -- start start 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448fca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564449020280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4338200640 con 0x564448fc9800 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43380cf390 con 0x564448ff46d0 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f43381ffff0 con 0x564448ff1af0 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51162/0 (socket says 172.21.15.106:51162) 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 -- 172.21.15.106:0/3509665911 learned_addr learned my addr 172.21.15.106:0/3509665911 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 -- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 msgr2=0x564448fca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448fca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 -- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 msgr2=0x564449020280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564449020280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 -- 172.21.15.106:0/3509665911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564448c010b0 con 0x564448fc9800 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f433ca7a640 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4334014a30 tx=0x7f433401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0f346ef03cb0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f431dffb640 1 -- 172.21.15.106:0/3509665911 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334035070 con 0x564448fc9800 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f431dffb640 1 -- 172.21.15.106:0/3509665911 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433401d040 con 0x564448fc9800 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.537+0000 7f431dffb640 1 -- 172.21.15.106:0/3509665911 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334031430 con 0x564448fc9800 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- 172.21.15.106:0/3509665911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 msgr2=0x5644490218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4334014a30 tx=0x7f433401f000 comp rx=0 tx=0).stop 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- 172.21.15.106:0/3509665911 shutdown_connections 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff1af0 0x564448fca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff46d0 0x564449020280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 --2- 172.21.15.106:0/3509665911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x5644490218f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- 172.21.15.106:0/3509665911 >> 172.21.15.106:0/3509665911 conn(0x564448f5ddc0 msgr2=0x564448feaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- 172.21.15.106:0/3509665911 shutdown_connections 2024-04-13T03:16:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- 172.21.15.106:0/3509665911 wait complete. 2024-04-13T03:16:37.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.538+0000 7f4341bad740 1 -- start start 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 Processor -- start 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff1af0 0x5644490218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff46d0 0x564449021e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564448fe9bd0 con 0x564448fc9800 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5644490225f0 con 0x564448ff1af0 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x564448fea190 con 0x564448ff46d0 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f4341bad740 10 client.0 _ll_register_callbacks cb 0x7f4334011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.539+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51172/0 (socket says 172.21.15.106:51172) 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 -- 172.21.15.106:0/2456721252 learned_addr learned my addr 172.21.15.106:0/2456721252 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f432ffff640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff1af0 0x5644490218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 -- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff46d0 msgr2=0x564449021e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433d27b640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff46d0 0x564449021e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff46d0 0x564449021e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 -- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff1af0 msgr2=0x5644490218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff1af0 0x5644490218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 -- 172.21.15.106:0/2456721252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564448fe4c50 con 0x564448fc9800 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f433ca7a640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f433400b010 tx=0x7f4334009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27956ad4e70ecfa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334035070 con 0x564448fc9800 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f4334009200 con 0x564448fc9800 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433401d040 con 0x564448fc9800 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4334005990 con 0x564448fc9800 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 10 client.7983 fetch_fsmap learned FSMap version 58 2024-04-13T03:16:37.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x564448fe4e50 con 0x564448fc9800 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433401d040 con 0x564448fc9800 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f4334005b30 con 0x564448fc9800 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 10 client.7983 fetch_fsmap finished waiting for FSMap version 58 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 10 client.7983 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 20 client.7983 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564449089140 0x56444901dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 4 client.7983 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.540+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 --> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564448f3ca00 con 0x564449089140 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f432ffff640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564449089140 0x56444901dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f43177fe640 10 client.7983 ms_handle_connect on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f432ffff640 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564449089140 0x56444901dbb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80af893b063154a server_cookie=e681d6de127f74c0 in_seq=0 out_seq=0 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f43177fe640 1 -- 172.21.15.106:0/2456721252 <== mds.0 v2:172.21.15.106:6835/635681585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x564448f3ca00 con 0x564449089140 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f43177fe640 10 client.7983 handle_command_reply: tid=0 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 client.7983 shutdown 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 msgr2=0x564448fc3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f433400b010 tx=0x7f4334009f90 comp rx=0 tx=0).stop 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 shutdown_connections 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:6835/635681585,v1:172.21.15.106:6837/635681585] conn(0x564449089140 0x56444901dbb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x564448ff46d0 0x564449021e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564448ff1af0 0x5644490218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f43177fe640 0 client.7983 ms_handle_reset on v2:172.21.15.106:6835/635681585 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 --2- 172.21.15.106:0/2456721252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564448fc9800 0x564448fc3d30 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 >> 172.21.15.106:0/2456721252 conn(0x564448f5ddc0 msgr2=0x564448ff0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 shutdown_connections 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 1 -- 172.21.15.106:0/2456721252 wait complete. 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.541+0000 7f4341bad740 20 client.7983 trim_cache size 0 max 16384 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:37.548 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: "damage_type": "backtrace", 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 3964833038, 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 1099511627777, 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/testdir/hardlink" 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:37.549 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 msgr2=0x7f432406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 0x7f432406ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4334009ad0 tx=0x7f4334008040 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 msgr2=0x7f43380b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f433000ca20 tx=0x7f433000cee0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 shutdown_connections 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f432406c920 0x7f432406ede0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f433815d110 0x7f43380b3a20 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4338156b40 0x7f43380b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 --2- 172.21.15.106:0/2502005659 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f433806ebf0 0x7f43380c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 >> 172.21.15.106:0/2502005659 conn(0x7f43380cd6a0 msgr2=0x7f433806cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 shutdown_connections 2024-04-13T03:16:37.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.542+0000 7f433ecde640 1 -- 172.21.15.106:0/2502005659 wait complete. 2024-04-13T03:16:37.553 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:16:37.553 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:16:37.569 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:16:37.569 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:16:37.625 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:16:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.764+0000 7f4a8926d640 1 Processor -- start 2024-04-13T03:16:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.764+0000 7f4a8926d640 1 -- start start 2024-04-13T03:16:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a8926d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a840b62c0 0x7f4a8406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a8926d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a8926d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8406dda0 con 0x7f4a8406d790 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a8926d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4a840b66a0 con 0x7f4a840b62c0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a8926d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8406c170 con 0x7f4a84073520 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a82ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a840b62c0 0x7f4a8406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a82ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a840b62c0 0x7f4a8406d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51174/0 (socket says 172.21.15.106:51174) 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.765+0000 7f4a82ffd640 1 -- 172.21.15.106:0/1712733412 learned_addr learned my addr 172.21.15.106:0/1712733412 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a837fe640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a82ffd640 1 -- 172.21.15.106:0/1712733412 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 msgr2=0x7f4a840b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a82ffd640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a82ffd640 1 -- 172.21.15.106:0/1712733412 >> v1:172.21.15.114:6789/0 conn(0x7f4a8406d790 legacy=0x7f4a84072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a82ffd640 1 -- 172.21.15.106:0/1712733412 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8406c450 con 0x7f4a840b62c0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.766+0000 7f4a82ffd640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a8406d250 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009e50 tx=0x7f4a7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850704e58b8b2ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a81ffb640 1 -- 172.21.15.106:0/1712733412 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7402cb00 con 0x7f4a840b62c0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a81ffb640 1 -- 172.21.15.106:0/1712733412 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7402cca0 con 0x7f4a840b62c0 2024-04-13T03:16:37.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a81ffb640 1 -- 172.21.15.106:0/1712733412 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74034610 con 0x7f4a840b62c0 2024-04-13T03:16:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 -- 172.21.15.106:0/1712733412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 msgr2=0x7f4a8406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a8406d250 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009e50 tx=0x7f4a7402bf60 comp rx=0 tx=0).stop 2024-04-13T03:16:37.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 -- 172.21.15.106:0/1712733412 shutdown_connections 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 --2- 172.21.15.106:0/1712733412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a8406d250 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 -- 172.21.15.106:0/1712733412 >> 172.21.15.106:0/1712733412 conn(0x7f4a8406f840 msgr2=0x7f4a840d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 -- 172.21.15.106:0/1712733412 shutdown_connections 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.767+0000 7f4a8926d640 1 -- 172.21.15.106:0/1712733412 wait complete. 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.768+0000 7f4a8926d640 1 Processor -- start 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.768+0000 7f4a8926d640 1 -- start start 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8406d790 0x7f4a84138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841633b0 con 0x7f4a840b62c0 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8406ca10 con 0x7f4a8406d790 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a8926d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8406cf30 con 0x7f4a84073520 2024-04-13T03:16:37.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51178/0 (socket says 172.21.15.106:51178) 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a837fe640 1 -- 172.21.15.106:0/208838375 learned_addr learned my addr 172.21.15.106:0/208838375 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a82ffd640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8406d790 0x7f4a84138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.769+0000 7f4a827fc640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 -- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 msgr2=0x7f4a840b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 -- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8406d790 msgr2=0x7f4a84138400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8406d790 0x7f4a84138400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 -- 172.21.15.106:0/208838375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a84138aa0 con 0x7f4a840b62c0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a837fe640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00d800 tx=0x7f4a7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00b6c0 con 0x7f4a840b62c0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c019070 con 0x7f4a840b62c0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a84138d80 con 0x7f4a840b62c0 2024-04-13T03:16:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00bac0 con 0x7f4a840b62c0 2024-04-13T03:16:37.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.770+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a84139340 con 0x7f4a840b62c0 2024-04-13T03:16:37.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.771+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84069120 con 0x7f4a840b62c0 2024-04-13T03:16:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.771+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a7c00bc60 con 0x7f4a840b62c0 2024-04-13T03:16:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.772+0000 7f4a6b7fe640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.772+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f4a7c08c200 con 0x7f4a840b62c0 2024-04-13T03:16:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.772+0000 7f4a82ffd640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:37.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.772+0000 7f4a82ffd640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 0x7f4a7006edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035000 tx=0x7f4a74030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:37.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.776+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a7c059c60 con 0x7f4a840b62c0 2024-04-13T03:16:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.953+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a84148450 con 0x7f4a840b62c0 2024-04-13T03:16:37.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.954+0000 7f4a6b7fe640 1 -- 172.21.15.106:0/208838375 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17245 (secure 0 0 0) 0x7f4a7c05df60 con 0x7f4a840b62c0 2024-04-13T03:16:37.955 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:37.955 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":94,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:16:07.733944+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 msgr2=0x7f4a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 0x7f4a7006edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a74035000 tx=0x7f4a74030000 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 msgr2=0x7f4a84161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00d800 tx=0x7f4a7c00dcc0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 shutdown_connections 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4a7006c8f0 0x7f4a7006edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4a84073520 0x7f4a840b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a840b62c0 0x7f4a84161520 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 --2- 172.21.15.106:0/208838375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a8406d790 0x7f4a84138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 >> 172.21.15.106:0/208838375 conn(0x7f4a8406f840 msgr2=0x7f4a840cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 shutdown_connections 2024-04-13T03:16:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:37.956+0000 7f4a8926d640 1 -- 172.21.15.106:0/208838375 wait complete. 2024-04-13T03:16:37.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:15:17.537136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:15:18.258394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:16:37.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:16:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.069+0000 7fc11471e640 1 Processor -- start 2024-04-13T03:16:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.069+0000 7fc11471e640 1 -- start start 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc11471e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc10c0af800 0x7fc10c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc11471e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c15c200 con 0x7fc10c06f690 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc11471e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c0b5090 con 0x7fc10c06f200 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc11471e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc10c15f650 con 0x7fc10c0af800 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc112cbb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc10c06f690 0x7fc10c15ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57846/0 (socket says 172.21.15.106:57846) 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.070+0000 7fc112cbb640 1 -- 172.21.15.106:0/2149631324 learned_addr learned my addr 172.21.15.106:0/2149631324 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542852403 0 0) 0x7fc104001110 con 0x7fc10c06f690 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f8003520 con 0x7fc10c06f690 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 --2- 172.21.15.106:0/2149631324 >> v2:172.21.15.114:3300/0 conn(0x7fc10c0af800 0x7fc10c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2149899748 0 0) 0x7fc0f8003520 con 0x7fc10c06f690 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 >> v2:172.21.15.114:3300/0 conn(0x7fc10c0af800 msgr2=0x7fc10c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 --2- 172.21.15.106:0/2149631324 >> v2:172.21.15.114:3300/0 conn(0x7fc10c0af800 0x7fc10c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 >> v1:172.21.15.114:6789/0 conn(0x7fc10c06f200 legacy=0x7fc10c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c06fb00 con 0x7fc10c06f690 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc104001840 con 0x7fc10c06f690 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 >> v1:172.21.15.106:6789/0 conn(0x7fc10c06f690 legacy=0x7fc10c15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc0f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0f8004800 0x7fc0f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0f80051c0 0x7fc0f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c15f650 con 0x7fc0f80039b0 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f800c060 con 0x7fc0f80051c0 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f800c260 con 0x7fc0f8004800 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc0f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc112cbb640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0f80051c0 0x7fc0f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc111cb9640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0f8004800 0x7fc0f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 -- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0f8004800 msgr2=0x7fc0f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0f8004800 0x7fc0f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 -- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0f80051c0 msgr2=0x7fc0f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0f80051c0 0x7fc0f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc1124ba640 1 -- 172.21.15.106:0/2149631324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f8003520 con 0x7fc0f80039b0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc111cb9640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc0f8004800 0x7fc0f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc112cbb640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0f80051c0 0x7fc0f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.071+0000 7fc112cbb640 1 -- 172.21.15.106:0/2149631324 reap_dead start 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.072+0000 7fc1124ba640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc0f8003dd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fc0 tx=0x7fc0f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec4727aab82b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.072+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108039070 con 0x7fc0f80039b0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.072+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10802cc20 con 0x7fc0f80039b0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.072+0000 7fc1114b8640 1 -- 172.21.15.106:0/2149631324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108009c10 con 0x7fc0f80039b0 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 -- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 msgr2=0x7fc0f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc0f8003dd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fc0 tx=0x7fc0f8005830 comp rx=0 tx=0).stop 2024-04-13T03:16:38.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 -- 172.21.15.106:0/2149631324 shutdown_connections 2024-04-13T03:16:38.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 --2- 172.21.15.106:0/2149631324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc0f8003dd0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 -- 172.21.15.106:0/2149631324 >> 172.21.15.106:0/2149631324 conn(0x7fc10c0b2600 msgr2=0x7fc10c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 -- 172.21.15.106:0/2149631324 shutdown_connections 2024-04-13T03:16:38.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.073+0000 7fc11471e640 1 -- 172.21.15.106:0/2149631324 wait complete. 2024-04-13T03:16:38.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.074+0000 7fc11471e640 1 Processor -- start 2024-04-13T03:16:38.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.074+0000 7fc11471e640 1 -- start start 2024-04-13T03:16:38.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.074+0000 7fc11471e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c15ddd0 0x7fc10c141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc10c1417c0 0x7fc10c144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0b45b0 con 0x7fc0f80039b0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0b42d0 con 0x7fc10c15ddd0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0b4890 con 0x7fc10c1417c0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51200/0 (socket says 172.21.15.106:51200) 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 -- 172.21.15.106:0/738394961 learned_addr learned my addr 172.21.15.106:0/738394961 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc112cbb640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc10c1417c0 0x7fc10c144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc111cb9640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c15ddd0 0x7fc10c141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 -- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc10c1417c0 msgr2=0x7fc10c144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc10c1417c0 0x7fc10c144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 -- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c15ddd0 msgr2=0x7fc10c141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c15ddd0 0x7fc10c141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 -- 172.21.15.106:0/738394961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c15e2d0 con 0x7fc0f80039b0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc1124ba640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e230 tx=0x7fc108009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108039050 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10803a040 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c15e5b0 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108009c10 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.075+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c15eb70 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.076+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c0b55c0 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.077+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc108003bb0 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.077+0000 7fc102ffd640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.077+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fc1080ae500 con 0x7fc0f80039b0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.077+0000 7fc111cb9640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.077+0000 7fc111cb9640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 0x7fc0f006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc10c140370 tx=0x7fc0fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.081+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc10c0b55c0 con 0x7fc0f80039b0 2024-04-13T03:16:38.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.254+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc10c068a10 con 0x7fc0f80039b0 2024-04-13T03:16:38.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.255+0000 7fc102ffd640 1 -- 172.21.15.106:0/738394961 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fc10807c770 con 0x7fc0f80039b0 2024-04-13T03:16:38.256 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 msgr2=0x7fc0f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 0x7fc0f006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc10c140370 tx=0x7fc0fc009040 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 msgr2=0x7fc10c143450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e230 tx=0x7fc108009750 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 shutdown_connections 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc10c1417c0 0x7fc10c144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c15ddd0 0x7fc10c141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 --2- 172.21.15.106:0/738394961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f80039b0 0x7fc10c143450 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 >> 172.21.15.106:0/738394961 conn(0x7fc10c0b2600 msgr2=0x7fc10c071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 shutdown_connections 2024-04-13T03:16:38.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.257+0000 7fc11471e640 1 -- 172.21.15.106:0/738394961 wait complete. 2024-04-13T03:16:38.268 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:16:38.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.370+0000 7f7dbf436640 1 Processor -- start 2024-04-13T03:16:38.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.370+0000 7f7dbf436640 1 -- start start 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.371+0000 7f7dbf436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.371+0000 7f7dbf436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7db80b3d60 0x7f7db80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.371+0000 7f7dbf436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db80cfd20 con 0x7f7db815ed90 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.371+0000 7f7dbf436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7db8162af0 con 0x7f7db80b3d60 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.371+0000 7f7dbf436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7db8162dd0 con 0x7f7db815e9b0 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd9d3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7db80b3d60 0x7f7db80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbc9d1640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7db815ed90 0x7f7db80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42778/0 (socket says 172.21.15.106:42778) 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:33170/0 (socket says 172.21.15.106:33170) 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/616259836 learned_addr learned my addr 172.21.15.106:0/616259836 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/616259836 >> v2:172.21.15.114:3300/0 conn(0x7f7db80b3d60 msgr2=0x7f7db80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 --2- 172.21.15.106:0/616259836 >> v2:172.21.15.114:3300/0 conn(0x7f7db80b3d60 0x7f7db80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/616259836 >> v1:172.21.15.114:6789/0 conn(0x7f7db815ed90 legacy=0x7f7db80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.372+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/616259836 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db81630b0 con 0x7f7db815e9b0 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbd1d2640 1 --2- 172.21.15.106:0/616259836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009e50 tx=0x7f7db402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3922f2bae65098f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7daffff640 1 -- 172.21.15.106:0/616259836 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db402cb90 con 0x7f7db815e9b0 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7daffff640 1 -- 172.21.15.106:0/616259836 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db402cd30 con 0x7f7db815e9b0 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7daffff640 1 -- 172.21.15.106:0/616259836 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4034610 con 0x7f7db815e9b0 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 -- 172.21.15.106:0/616259836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 msgr2=0x7f7db80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 --2- 172.21.15.106:0/616259836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009e50 tx=0x7f7db402bff0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 -- 172.21.15.106:0/616259836 shutdown_connections 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 --2- 172.21.15.106:0/616259836 >> v2:172.21.15.114:3300/0 conn(0x7f7db80b3d60 0x7f7db80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 --2- 172.21.15.106:0/616259836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80d2300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.373+0000 7f7dbf436640 1 -- 172.21.15.106:0/616259836 >> 172.21.15.106:0/616259836 conn(0x7f7db808bf10 msgr2=0x7f7db80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.374+0000 7f7dbf436640 1 -- 172.21.15.106:0/616259836 shutdown_connections 2024-04-13T03:16:38.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.374+0000 7f7dbf436640 1 -- 172.21.15.106:0/616259836 wait complete. 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 Processor -- start 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 -- start start 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7db8158e50 0x7f7db8159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8162800 con 0x7f7db80b3d60 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8162680 con 0x7f7db8158e50 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbf436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80d4d00 con 0x7f7db815e9b0 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbd1d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbd1d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51212/0 (socket says 172.21.15.106:51212) 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.375+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/3537766894 learned_addr learned my addr 172.21.15.106:0/3537766894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd9d3640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7db8158e50 0x7f7db8159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbc9d1640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 msgr2=0x7f7db80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7db8158e50 msgr2=0x7f7db8159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7db8158e50 0x7f7db8159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 -- 172.21.15.106:0/3537766894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8154390 con 0x7f7db80b3d60 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbd1d2640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009f70 tx=0x7f7db4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4043020 con 0x7f7db80b3d60 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db403e450 con 0x7f7db80b3d60 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4009510 con 0x7f7db80b3d60 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db81e8f90 con 0x7f7db80b3d60 2024-04-13T03:16:38.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.376+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db80c8470 con 0x7f7db80b3d60 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.377+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8068a10 con 0x7f7db80b3d60 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.378+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7db4046020 con 0x7f7db80b3d60 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.378+0000 7f7dadffb640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.378+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f7db40ae310 con 0x7f7db80b3d60 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.378+0000 7f7dbc9d1640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.379+0000 7f7dbc9d1640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000fb0 tx=0x7f7da8007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.383+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7db404d030 con 0x7f7db80b3d60 2024-04-13T03:16:38.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.557+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7db80b3a00 con 0x7f7db80b3d60 2024-04-13T03:16:38.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.557+0000 7f7dadffb640 1 -- 172.21.15.106:0/3537766894 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f7db407c580 con 0x7f7db80b3d60 2024-04-13T03:16:38.558 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 msgr2=0x7f7d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000fb0 tx=0x7f7da8007c90 comp rx=0 tx=0).stop 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 msgr2=0x7f7db8153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009f70 tx=0x7f7db4004a40 comp rx=0 tx=0).stop 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 shutdown_connections 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7db815e9b0 0x7f7db80c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7db8158e50 0x7f7db8159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 --2- 172.21.15.106:0/3537766894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7db80b3d60 0x7f7db8153e50 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 >> 172.21.15.106:0/3537766894 conn(0x7f7db808bf10 msgr2=0x7f7db80d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 shutdown_connections 2024-04-13T03:16:38.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.559+0000 7f7dbf436640 1 -- 172.21.15.106:0/3537766894 wait complete. 2024-04-13T03:16:38.570 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:16:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.673+0000 7f66f1060640 1 Processor -- start 2024-04-13T03:16:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.673+0000 7f66f1060640 1 -- start start 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66f1060640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f66ec073520 0x7f66ec0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66f1060640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66ec15eca0 0x7f66ec0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66f1060640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ec0b5060 con 0x7f66ec15e8c0 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66f1060640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec0b5340 con 0x7f66ec15eca0 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66f1060640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec06c500 con 0x7f66ec073520 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66ec15eca0 0x7f66ec0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66ec15eca0 0x7f66ec0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51218/0 (socket says 172.21.15.106:51218) 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 -- 172.21.15.106:0/1363068859 learned_addr learned my addr 172.21.15.106:0/1363068859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66ea59c640 1 --2- 172.21.15.106:0/1363068859 >> v2:172.21.15.114:3300/0 conn(0x7f66ec073520 0x7f66ec0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 -- 172.21.15.106:0/1363068859 >> v2:172.21.15.114:3300/0 conn(0x7f66ec073520 msgr2=0x7f66ec0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 --2- 172.21.15.106:0/1363068859 >> v2:172.21.15.114:3300/0 conn(0x7f66ec073520 0x7f66ec0ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 -- 172.21.15.106:0/1363068859 >> v1:172.21.15.114:6789/0 conn(0x7f66ec15e8c0 legacy=0x7f66ec072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.674+0000 7f66eb59e640 1 -- 172.21.15.106:0/1363068859 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec06c7e0 con 0x7f66ec15eca0 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.675+0000 7f66eb59e640 1 --2- 172.21.15.106:0/1363068859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec0b4430 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f66ec072ae0 tx=0x7f66e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7498328f98e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.675+0000 7f66e9d9b640 1 -- 172.21.15.106:0/1363068859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4034070 con 0x7f66ec15eca0 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.675+0000 7f66e9d9b640 1 -- 172.21.15.106:0/1363068859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e4030860 con 0x7f66ec15eca0 2024-04-13T03:16:38.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.675+0000 7f66e9d9b640 1 -- 172.21.15.106:0/1363068859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4030c30 con 0x7f66ec15eca0 2024-04-13T03:16:38.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 -- 172.21.15.106:0/1363068859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 msgr2=0x7f66ec0b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 --2- 172.21.15.106:0/1363068859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec0b4430 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f66ec072ae0 tx=0x7f66e402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 -- 172.21.15.106:0/1363068859 shutdown_connections 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 --2- 172.21.15.106:0/1363068859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec0b4430 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 --2- 172.21.15.106:0/1363068859 >> v2:172.21.15.114:3300/0 conn(0x7f66ec073520 0x7f66ec0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 -- 172.21.15.106:0/1363068859 >> 172.21.15.106:0/1363068859 conn(0x7f66ec06f0d0 msgr2=0x7f66ec06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 -- 172.21.15.106:0/1363068859 shutdown_connections 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.676+0000 7f66f1060640 1 -- 172.21.15.106:0/1363068859 wait complete. 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.677+0000 7f66f1060640 1 Processor -- start 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.677+0000 7f66f1060640 1 -- start start 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 0x7f66ec143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66ec15e8c0 0x7f66ec15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b4d40 con 0x7f66ec15eca0 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b4bc0 con 0x7f66ec15e8c0 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66f1060640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66ec06cd00 con 0x7f66ec073520 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66eb59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66eb59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51220/0 (socket says 172.21.15.106:51220) 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66eb59e640 1 -- 172.21.15.106:0/2610017674 learned_addr learned my addr 172.21.15.106:0/2610017674 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66ea59c640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66ec15e8c0 0x7f66ec15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66ead9d640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 0x7f66ec143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66eb59e640 1 -- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 msgr2=0x7f66ec143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.678+0000 7f66eb59e640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 0x7f66ec143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66eb59e640 1 -- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66ec15e8c0 msgr2=0x7f66ec15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66eb59e640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66ec15e8c0 0x7f66ec15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66eb59e640 1 -- 172.21.15.106:0/2610017674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec155730 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66ead9d640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 0x7f66ec143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66eb59e640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009960 tx=0x7f66e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4034040 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e40433f0 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec155a10 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e40437e0 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.679+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec155fd0 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.680+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec068a10 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.680+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66e4037070 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.681+0000 7f66d37fe640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 0x7f66d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.681+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f66e40ae0b0 con 0x7f66ec15eca0 2024-04-13T03:16:38.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.681+0000 7f66ead9d640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 0x7f66d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.681+0000 7f66ead9d640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 0x7f66d806edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66dc004520 tx=0x7f66dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.685+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66e403e070 con 0x7f66ec15eca0 2024-04-13T03:16:38.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.861+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66ec073dd0 con 0x7f66ec15eca0 2024-04-13T03:16:38.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.861+0000 7f66d37fe640 1 -- 172.21.15.106:0/2610017674 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f66e407c290 con 0x7f66ec15eca0 2024-04-13T03:16:38.862 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 msgr2=0x7f66d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 0x7f66d806edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66dc004520 tx=0x7f66dc009040 comp rx=0 tx=0).stop 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 msgr2=0x7f66ec15f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009960 tx=0x7f66e40046c0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 shutdown_connections 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66d806c8f0 0x7f66d806edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec15eca0 0x7f66ec15f5f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66ec15e8c0 0x7f66ec15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 --2- 172.21.15.106:0/2610017674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66ec073520 0x7f66ec143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 >> 172.21.15.106:0/2610017674 conn(0x7f66ec06f0d0 msgr2=0x7f66ec06f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 shutdown_connections 2024-04-13T03:16:38.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.863+0000 7f66f1060640 1 -- 172.21.15.106:0/2610017674 wait complete. 2024-04-13T03:16:38.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:16:38.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.975+0000 7fa4b6636640 1 Processor -- start 2024-04-13T03:16:38.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4b6636640 1 -- start start 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4b6636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 0x7fa4b00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4b6636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b00cfd20 con 0x7fa4b015e9b0 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4b6636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b0162af0 con 0x7fa4b015ed90 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4b6636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b0162dd0 con 0x7fa4b00b3d60 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4affff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa4b015e9b0 0x7fa4b00d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57854/0 (socket says 172.21.15.106:57854) 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.976+0000 7fa4affff640 1 -- 172.21.15.106:0/4172334793 learned_addr learned my addr 172.21.15.106:0/4172334793 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4b4bd3640 1 --2- 172.21.15.106:0/4172334793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 0x7fa4b00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927306752 0 0) 0x7fa4b00cfd20 con 0x7fa4b015e9b0 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa498003520 con 0x7fa4b015e9b0 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803296992 0 0) 0x7fa4b0162af0 con 0x7fa4b015ed90 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b00cfd20 con 0x7fa4b015ed90 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 415913229 0 0) 0x7fa498003520 con 0x7fa4b015e9b0 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/4172334793 >> v1:172.21.15.114:6789/0 conn(0x7fa4b015ed90 legacy=0x7fa4b00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/4172334793 >> v1:172.21.15.106:6789/0 conn(0x7fa4b015e9b0 legacy=0x7fa4b00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.977+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/4172334793 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b01630b0 con 0x7fa4b00b3d60 2024-04-13T03:16:38.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b4bd3640 1 --2- 172.21.15.106:0/4172334793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 0x7fa4b00d2300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009dc0 tx=0x7fa4a802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd309dbe10c9ea6a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8039070 con 0x7fa4b00b3d60 2024-04-13T03:16:38.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a802c9a0 con 0x7fa4b00b3d60 2024-04-13T03:16:38.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4aeffd640 1 -- 172.21.15.106:0/4172334793 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0001670 con 0x7fa4b00b3d60 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 -- 172.21.15.106:0/4172334793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 msgr2=0x7fa4b00d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 --2- 172.21.15.106:0/4172334793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 0x7fa4b00d2300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009dc0 tx=0x7fa4a802be70 comp rx=0 tx=0).stop 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 -- 172.21.15.106:0/4172334793 shutdown_connections 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 --2- 172.21.15.106:0/4172334793 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b00b3d60 0x7fa4b00d2300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 -- 172.21.15.106:0/4172334793 >> 172.21.15.106:0/4172334793 conn(0x7fa4b008bf10 msgr2=0x7fa4b00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 -- 172.21.15.106:0/4172334793 shutdown_connections 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.978+0000 7fa4b6636640 1 -- 172.21.15.106:0/4172334793 wait complete. 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 Processor -- start 2024-04-13T03:16:38.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 -- start start 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b00b3d60 0x7fa4b0161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b015e9b0 0x7fa4b015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0162800 con 0x7fa4b00a1e30 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0162680 con 0x7fa4b00b3d60 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.980+0000 7fa4b6636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b00d4d00 con 0x7fa4b015e9b0 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51224/0 (socket says 172.21.15.106:51224) 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/2897735358 learned_addr learned my addr 172.21.15.106:0/2897735358 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4af7fe640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b015e9b0 0x7fa4b015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4affff640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b00b3d60 0x7fa4b0161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b015e9b0 msgr2=0x7fa4b015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b015e9b0 0x7fa4b015bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b00b3d60 msgr2=0x7fa4b0161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b00b3d60 0x7fa4b0161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 -- 172.21.15.106:0/2897735358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b0161da0 con 0x7fa4b00a1e30 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b4bd3640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa4a802c480 tx=0x7fa4a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8039050 con 0x7fa4b00a1e30 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8045070 con 0x7fa4b00a1e30 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b01607c0 con 0x7fa4b00a1e30 2024-04-13T03:16:38.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8009450 con 0x7fa4b00a1e30 2024-04-13T03:16:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.981+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b0160d80 con 0x7fa4b00a1e30 2024-04-13T03:16:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.982+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00d13c0 con 0x7fa4b00a1e30 2024-04-13T03:16:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.983+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4a8040690 con 0x7fa4b00a1e30 2024-04-13T03:16:38.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.983+0000 7fa4acff9640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 0x7fa49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:38.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.983+0000 7fa4affff640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 0x7fa49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:38.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.983+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fa4a80ae530 con 0x7fa4b00a1e30 2024-04-13T03:16:38.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.984+0000 7fa4affff640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 0x7fa49c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0000fd0 tx=0x7fa4a0001150 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:38.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:38.988+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4b00d13c0 con 0x7fa4b00a1e30 2024-04-13T03:16:39.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.174+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa4b00d53a0 con 0x7fa4b00a1e30 2024-04-13T03:16:39.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.174+0000 7fa4acff9640 1 -- 172.21.15.106:0/2897735358 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) ==== 80+0+124 (secure 0 0 0) 0x7fa4a807c0c0 con 0x7fa4b00a1e30 2024-04-13T03:16:39.175 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/3360662124 2024-04-14T03:15:28.649985+0000 2024-04-13T03:16:39.175 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/3360662124 2024-04-14T03:15:28.649985+0000 2024-04-13T03:16:39.175 INFO:teuthology.orchestra.run.smithi106.stderr:listed 2 entries 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 msgr2=0x7fa49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 0x7fa49c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0000fd0 tx=0x7fa4a0001150 comp rx=0 tx=0).stop 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 msgr2=0x7fa4b00cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa4a802c480 tx=0x7fa4a8004a40 comp rx=0 tx=0).stop 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 shutdown_connections 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa49c06c8f0 0x7fa49c06edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa4b015e9b0 0x7fa4b015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b00a1e30 0x7fa4b00cbff0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 --2- 172.21.15.106:0/2897735358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b00b3d60 0x7fa4b0161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 >> 172.21.15.106:0/2897735358 conn(0x7fa4b008bf10 msgr2=0x7fa4b00b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:39.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 shutdown_connections 2024-04-13T03:16:39.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.176+0000 7fa4b6636640 1 -- 172.21.15.106:0/2897735358 wait complete. 2024-04-13T03:16:39.187 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:16:39.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:39.188 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:16:39.217 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:16:39.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:39.217 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:16:39.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:39.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.412+0000 7f99dd480640 1 Processor -- start 2024-04-13T03:16:39.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.412+0000 7f99dd480640 1 -- start start 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99dd480640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f99d815e9b0 0x7f99d80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99dd480640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99d815ed90 0x7f99d80d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99dd480640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d80cfd20 con 0x7f99d80b3d60 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99dd480640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f99d8162af0 con 0x7f99d815ed90 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99dd480640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f99d8162dd0 con 0x7f99d815e9b0 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99d815ed90 0x7f99d80d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f99d815ed90 0x7f99d80d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51230/0 (socket says 172.21.15.106:51230) 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 -- 172.21.15.106:0/1199727739 learned_addr learned my addr 172.21.15.106:0/1199727739 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 -- 172.21.15.106:0/1199727739 >> v2:172.21.15.114:3300/0 conn(0x7f99d815e9b0 msgr2=0x7f99d80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d6ffd640 1 --2- 172.21.15.106:0/1199727739 >> v2:172.21.15.114:3300/0 conn(0x7f99d815e9b0 0x7f99d80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 --2- 172.21.15.106:0/1199727739 >> v2:172.21.15.114:3300/0 conn(0x7f99d815e9b0 0x7f99d80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 -- 172.21.15.106:0/1199727739 >> v1:172.21.15.114:6789/0 conn(0x7f99d80b3d60 legacy=0x7f99d80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.413+0000 7f99d67fc640 1 -- 172.21.15.106:0/1199727739 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81630b0 con 0x7f99d815ed90 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.414+0000 7f99d67fc640 1 --2- 172.21.15.106:0/1199727739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d815ed90 0x7f99d80d45f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f99d808e4e0 tx=0x7f99cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ee08d0dd6c6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.414+0000 7f99d5ffb640 1 -- 172.21.15.106:0/1199727739 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004230 con 0x7f99d815ed90 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.414+0000 7f99d5ffb640 1 -- 172.21.15.106:0/1199727739 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99cc0043d0 con 0x7f99d815ed90 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.414+0000 7f99d5ffb640 1 -- 172.21.15.106:0/1199727739 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc030840 con 0x7f99d815ed90 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 -- 172.21.15.106:0/1199727739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d815ed90 msgr2=0x7f99d80d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 --2- 172.21.15.106:0/1199727739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d815ed90 0x7f99d80d45f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f99d808e4e0 tx=0x7f99cc02b980 comp rx=0 tx=0).stop 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 -- 172.21.15.106:0/1199727739 shutdown_connections 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 --2- 172.21.15.106:0/1199727739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d815ed90 0x7f99d80d45f0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 --2- 172.21.15.106:0/1199727739 >> v2:172.21.15.114:3300/0 conn(0x7f99d815e9b0 0x7f99d80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 -- 172.21.15.106:0/1199727739 >> 172.21.15.106:0/1199727739 conn(0x7f99d808bf10 msgr2=0x7f99d80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 -- 172.21.15.106:0/1199727739 shutdown_connections 2024-04-13T03:16:39.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.415+0000 7f99dd480640 1 -- 172.21.15.106:0/1199727739 wait complete. 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.416+0000 7f99dd480640 1 Processor -- start 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.416+0000 7f99dd480640 1 -- start start 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99d815e9b0 0x7f99d815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 0x7f99d80c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8162800 con 0x7f99d80b3d60 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8162680 con 0x7f99d80b4160 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99dd480640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f99d806eb20 con 0x7f99d815e9b0 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51236/0 (socket says 172.21.15.106:51236) 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 -- 172.21.15.106:0/991611725 learned_addr learned my addr 172.21.15.106:0/991611725 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d77fe640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 0x7f99d80c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 -- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99d815e9b0 msgr2=0x7f99d815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99d815e9b0 0x7f99d815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 -- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 msgr2=0x7f99d80c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 0x7f99d80c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 -- 172.21.15.106:0/991611725 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d80cad10 con 0x7f99d80b3d60 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d77fe640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 0x7f99d80c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.417+0000 7f99d6ffd640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c820 tx=0x7f99c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8015070 con 0x7f99d80b3d60 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8004240 con 0x7f99d80b3d60 2024-04-13T03:16:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d80c8e30 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c800ce10 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d80caa00 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.418+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d80c94f0 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.419+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99c8007830 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.419+0000 7f99bf7fe640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 0x7f99c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.420+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f99c808bcb0 con 0x7f99d80b3d60 2024-04-13T03:16:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.420+0000 7f99d67fc640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 0x7f99c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.420+0000 7f99d67fc640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 0x7f99c406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99cc035000 tx=0x7f99cc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.424+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99c8059ea0 con 0x7f99d80b3d60 2024-04-13T03:16:39.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.636+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d8068a10 con 0x7f99d80b3d60 2024-04-13T03:16:39.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.636+0000 7f99bf7fe640 1 -- 172.21.15.106:0/991611725 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5193 (secure 0 0 0) 0x7f99c80597a0 con 0x7f99d80b3d60 2024-04-13T03:16:39.637 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:39.637 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:15:39.667524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/635681585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":635681585},{"type":"v1","addr":"172.21.15.106:6837","nonce":635681585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7728,"qdb_cluster":[7728]},"id":6}]} 2024-04-13T03:16:39.638 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 58 2024-04-13T03:16:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 msgr2=0x7f99c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 0x7f99c406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99cc035000 tx=0x7f99cc030000 comp rx=0 tx=0).stop 2024-04-13T03:16:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 msgr2=0x7f99d8152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c820 tx=0x7f99c800cce0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 shutdown_connections 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f99c406c8f0 0x7f99c406edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f99d815e9b0 0x7f99d815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d80b4160 0x7f99d80c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 --2- 172.21.15.106:0/991611725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99d80b3d60 0x7f99d8152530 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 >> 172.21.15.106:0/991611725 conn(0x7f99d808bf10 msgr2=0x7f99d80d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 shutdown_connections 2024-04-13T03:16:39.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.638+0000 7f99dd480640 1 -- 172.21.15.106:0/991611725 wait complete. 2024-04-13T03:16:39.650 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-13T03:16:39.777 INFO:teuthology.orchestra.run.smithi106.stdout:{"message":"","return_code":0} 2024-04-13T03:16:39.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:16:39.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:16:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.877+0000 7f8230be9640 1 Processor -- start 2024-04-13T03:16:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.877+0000 7f8230be9640 1 -- start start 2024-04-13T03:16:39.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f8230be9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f822c06f200 0x7f822c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f8230be9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c0b5090 con 0x7f822c0af800 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f8230be9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c0b3ff0 con 0x7f822c06b8e0 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f8230be9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f822c0b42d0 con 0x7f822c06f200 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f822c0af800 0x7f822c15bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57864/0 (socket says 172.21.15.106:57864) 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822a59c640 1 -- 172.21.15.106:0/3997555155 learned_addr learned my addr 172.21.15.106:0/3997555155 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f8229d9b640 1 --2- 172.21.15.106:0/3997555155 >> v2:172.21.15.114:3300/0 conn(0x7f822c06f200 0x7f822c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828225259 0 0) 0x7f822c0b5090 con 0x7f822c0af800 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8214003520 con 0x7f822c0af800 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 488497760 0 0) 0x7f8214003520 con 0x7f822c0af800 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 >> v2:172.21.15.114:3300/0 conn(0x7f822c06f200 msgr2=0x7f822c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 --2- 172.21.15.106:0/3997555155 >> v2:172.21.15.114:3300/0 conn(0x7f822c06f200 0x7f822c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 >> v1:172.21.15.114:6789/0 conn(0x7f822c06b8e0 legacy=0x7f822c06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0b45b0 con 0x7f822c0af800 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f821c001670 con 0x7f822c0af800 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 >> v1:172.21.15.106:6789/0 conn(0x7f822c0af800 legacy=0x7f822c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.878+0000 7f822959a640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f82140039b0 0x7f8214003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822959a640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f8214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822959a640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82140051c0 0x7f821400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b42d0 con 0x7f8214004800 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f821400c060 con 0x7f82140051c0 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f821400c260 con 0x7f82140039b0 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f8214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822a59c640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f82140039b0 0x7f8214003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822ad9d640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82140051c0 0x7f821400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 -- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f82140039b0 msgr2=0x7f8214003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f82140039b0 0x7f8214003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 -- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82140051c0 msgr2=0x7f821400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82140051c0 0x7f821400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f8229d9b640 1 -- 172.21.15.106:0/3997555155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8214003520 con 0x7f8214004800 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822ad9d640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82140051c0 0x7f821400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.879+0000 7f822ad9d640 1 -- 172.21.15.106:0/3997555155 reap_dead start 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8229d9b640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f8214004c80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f822000aac0 tx=0x7f8214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e446430c70a28367 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000b040 con 0x7f8214004800 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8220034600 con 0x7f8214004800 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f822959a640 1 -- 172.21.15.106:0/3997555155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200349d0 con 0x7f8214004800 2024-04-13T03:16:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 -- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 msgr2=0x7f8214004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f8214004c80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f822000aac0 tx=0x7f8214005830 comp rx=0 tx=0).stop 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 -- 172.21.15.106:0/3997555155 shutdown_connections 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 --2- 172.21.15.106:0/3997555155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f8214004c80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 -- 172.21.15.106:0/3997555155 >> 172.21.15.106:0/3997555155 conn(0x7f822c0b2600 msgr2=0x7f822c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 -- 172.21.15.106:0/3997555155 shutdown_connections 2024-04-13T03:16:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.880+0000 7f8230be9640 1 -- 172.21.15.106:0/3997555155 wait complete. 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 Processor -- start 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 -- start start 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 0x7f822c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 0x7f822c14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b3cc0 con 0x7f8214004800 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b4b70 con 0x7f822c14ca30 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.882+0000 7f8230be9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f822c06fef0 con 0x7f822c15d620 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51258/0 (socket says 172.21.15.106:51258) 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 -- 172.21.15.106:0/2297781809 learned_addr learned my addr 172.21.15.106:0/2297781809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822ad9d640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 0x7f822c14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 -- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 msgr2=0x7f822c15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f8229d9b640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 0x7f822c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 0x7f822c15daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 -- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 msgr2=0x7f822c14ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 0x7f822c14ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 -- 172.21.15.106:0/2297781809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c15e080 con 0x7f8214004800 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822ad9d640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 0x7f822c14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f8229d9b640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 0x7f822c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f822a59c640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e7f0 tx=0x7f821c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00c8d0 con 0x7f8214004800 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c018070 con 0x7f8214004800 2024-04-13T03:16:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00cc70 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c14d720 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.883+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c15c8f0 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.884+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c068a10 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.885+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f821c01d460 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.885+0000 7f821affd640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 0x7f820c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.885+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f821c08bcd0 con 0x7f8214004800 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.885+0000 7f8229d9b640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 0x7f820c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.885+0000 7f8229d9b640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 0x7f820c06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8220037000 tx=0x7f8220032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:39.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:39.889+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f821c059eb0 con 0x7f8214004800 2024-04-13T03:16:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.105+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f822c0ccad0 con 0x7f8214004800 2024-04-13T03:16:40.106 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:16:40.106+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:16:40.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.107+0000 7f821affd640 1 -- 172.21.15.106:0/2297781809 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f821c060080 con 0x7f8214004800 2024-04-13T03:16:40.108 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 msgr2=0x7f820c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 0x7f820c06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8220037000 tx=0x7f8220032000 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 msgr2=0x7f822c15ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e7f0 tx=0x7f821c00ecb0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 shutdown_connections 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f820c06c8f0 0x7f820c06edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f822c15d620 0x7f822c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f822c14ca30 0x7f822c14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 --2- 172.21.15.106:0/2297781809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8214004800 0x7f822c15ef80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 >> 172.21.15.106:0/2297781809 conn(0x7f822c0b2600 msgr2=0x7f822c154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 shutdown_connections 2024-04-13T03:16:40.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.110+0000 7f8230be9640 1 -- 172.21.15.106:0/2297781809 wait complete. 2024-04-13T03:16:40.114 INFO:tasks.ceph.mds.a.smithi106.stderr: -4639> 2024-04-13T03:15:34.659+0000 7f8e6d3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:16:40.114 INFO:tasks.ceph.mds.a.smithi106.stderr: -4234> 2024-04-13T03:15:35.246+0000 7f8e693e9640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-13T03:16:40.114 INFO:tasks.ceph.mds.a.smithi106.stderr: -4064> 2024-04-13T03:15:35.429+0000 7f8e693e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0) n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5628b7fa4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-13T03:16:40.117 INFO:tasks.ceph.mds.a.smithi106.stderr: -2219> 2024-04-13T03:16:06.730+0000 7f8e693e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)->f(v1 m2024-04-13T03:15:25.822275+0000) n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)->n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5628b7fa4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-13T03:15:25.822275+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-13T03:15:25.938272+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-13T03:16:40.123 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-13T03:16:40.143 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:16:40.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.160+0000 7f9a4be2b740 1 -- start start 2024-04-13T03:16:40.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.160+0000 7f9a4be2b740 1 Processor -- start 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cc4dec00 0x55c1cc650000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 0x55c1cc650b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4180 con 0x55c1cc4df800 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4300 con 0x55c1cc4dfc00 2024-04-13T03:16:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4480 con 0x55c1cc4dec00 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51294/0 (socket says 172.21.15.106:51294) 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 -- 172.21.15.106:0/765716183 learned_addr learned my addr 172.21.15.106:0/765716183 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 0x55c1cc650b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a493fc640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cc4dec00 0x55c1cc650000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 -- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cc4dec00 msgr2=0x55c1cc650000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cc4dec00 0x55c1cc650000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 -- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 msgr2=0x55c1cc650b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 0x55c1cc650b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a48bfb640 1 -- 172.21.15.106:0/765716183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cc50b680 con 0x55c1cc4df800 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.161+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 0x55c1cc650b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55c1cd303c80 tx=0x55c1cd27c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e6f91df23fb955 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a46bf7640 1 -- 172.21.15.106:0/765716183 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc5144e0 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a46bf7640 1 -- 172.21.15.106:0/765716183 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c1cc52a540 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a46bf7640 1 -- 172.21.15.106:0/765716183 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc514b60 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1cc50b4a0 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c1cc50ba40 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 4 waiting for MDS map... 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a46bf7640 1 -- 172.21.15.106:0/765716183 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x55c1cd3bea00 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a46bf7640 1 -- 172.21.15.106:0/765716183 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 59) ==== 2885+0+0 (secure 0 0 0) 0x55c1cc539180 con 0x55c1cc4df800 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 4 Got MDS map 59 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 4 datascan.main: Using metadata pool output 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.162+0000 7f9a4be2b740 4 datascan.main: connecting to RADOS... 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 Processor -- start 2024-04-13T03:16:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 -- start start 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4600 con 0x55c1cd3bbc00 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4780 con 0x55c1cd3bb400 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4900 con 0x55c1cd3bb800 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51304/0 (socket says 172.21.15.106:51304) 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 -- 172.21.15.106:0/2598166469 learned_addr learned my addr 172.21.15.106:0/2598166469 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a493fc640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 -- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 msgr2=0x55c1cc651600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 -- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bb400 msgr2=0x55c1cc651080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a48bfb640 1 -- 172.21.15.106:0/2598166469 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cc50b860 con 0x55c1cd3bbc00 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.163+0000 7f9a493fc640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55c1cd3ccf00 tx=0x55c1cd278c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b6e5b7fe6d3fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a44bf3640 1 -- 172.21.15.106:0/2598166469 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc514340 con 0x55c1cd3bbc00 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a44bf3640 1 -- 172.21.15.106:0/2598166469 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c1cc52ae00 con 0x55c1cd3bbc00 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a44bf3640 1 -- 172.21.15.106:0/2598166469 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc5144e0 con 0x55c1cd3bbc00 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 -- 172.21.15.106:0/2598166469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 msgr2=0x55c1cc651b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55c1cd3ccf00 tx=0x55c1cd278c60 comp rx=0 tx=0).stop 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 -- 172.21.15.106:0/2598166469 shutdown_connections 2024-04-13T03:16:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/2598166469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 -- 172.21.15.106:0/2598166469 >> 172.21.15.106:0/2598166469 conn(0x55c1cd3bb000 msgr2=0x55c1cc5c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 -- 172.21.15.106:0/2598166469 shutdown_connections 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.164+0000 7f9a4be2b740 1 -- 172.21.15.106:0/2598166469 wait complete. 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 Processor -- start 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 -- start start 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a5b00 con 0x55c1cd3bb800 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c1cd3a5980 con 0x55c1cd3bbc00 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a4be2b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c1cd3a4600 con 0x55c1cd3bb400 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a493fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a493fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51308/0 (socket says 172.21.15.106:51308) 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 learned_addr learned my addr 172.21.15.106:0/3464789343 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.165+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb400 msgr2=0x55c1cc651080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bbc00 msgr2=0x55c1cc651b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cd3bbc00 0x55c1cc651b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cc50b680 con 0x55c1cd3bb800 2024-04-13T03:16:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cd3bb400 0x55c1cc651080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55c1cd3cdcb0 tx=0x55c1cd3dba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a433f0640 1 -- 172.21.15.106:0/3464789343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc514340 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a433f0640 1 -- 172.21.15.106:0/3464789343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c1cc52ac40 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1cc50b4a0 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a433f0640 1 -- 172.21.15.106:0/3464789343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c1cc5144e0 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1cc50ba40 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.166+0000 7f9a4be2b740 4 datascan.init: resolving metadata pool 13 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a433f0640 1 -- 172.21.15.106:0/3464789343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55c1cd2c6c00 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a433f0640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 0x55c1cc652100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a433f0640 1 -- 172.21.15.106:0/3464789343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x55c1cd2e8a00 con 0x55c1cd3bb800 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a4be2b740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a4be2b740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-13T03:16:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a4be2b740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 0x55c1cc652100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c1cc7ee000 0x55c1cc652680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.167+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ee400 con 0x55c1cc7ee000 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.168+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c1cc7ee000 0x55c1cc652680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.168+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 0x55c1cc652100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55c1cd27a1e0 tx=0x55c1cd278720 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.168+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c1cc7ee000 0x55c1cc652680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a443f2640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c1cd3ba800 0x55c1cc872000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2c4c00 con 0x55c1cd3ba800 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c1cd3ba800 0x55c1cc872000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c1cd3ba800 0x55c1cc872000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a44bf3640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c1cd2c4400 0x55c1cc872580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ee400 con 0x55c1cd2c4400 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.170+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c1cd2c4400 0x55c1cc872580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.171+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c1cd2c4400 0x55c1cc872580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.172+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.172+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2c4800 con 0x55c1cd3ba800 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2c4000 con 0x55c1cd3ba800 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a443f2640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c1cd2c4c00 0x55c1cc872b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf400 con 0x55c1cd2c4c00 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c1cd2c4c00 0x55c1cc872b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c1cd2c4c00 0x55c1cc872b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.173+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef400 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf000 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(8 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a44bf3640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c1cd2c4800 0x55c1cc873600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2c4000 con 0x55c1cd2c4800 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.174+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c1cd2c4800 0x55c1cc873600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.175+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c1cd2c4800 0x55c1cc873600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.177+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd2c4800 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.177+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef800 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.177+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.177+0000 7f9a44bf3640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c1cc7ee400 0x55c1cc873b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.177+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef400 con 0x55c1cc7ee400 2024-04-13T03:16:40.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.178+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c1cc7ee400 0x55c1cc873b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.178+0000 7f9a493fc640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c1cc7ee400 0x55c1cc873b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.180+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.180+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7efc00 con 0x55c1cc7ee000 2024-04-13T03:16:40.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef800 con 0x55c1cd3ba800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(13 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e000 con 0x55c1cd2c4400 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.181+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf000 con 0x55c1cd3ba800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.182+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(15 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.182+0000 7f9a443f2640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c1cc88e400 0x55c1cc873080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.182+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e800 con 0x55c1cc88e400 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.182+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c1cc88e400 0x55c1cc873080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.182+0000 7f9a48bfb640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c1cc88e400 0x55c1cc873080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.183+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.183+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f400 con 0x55c1cd2c4800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.183+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd2c4800 2024-04-13T03:16:40.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.183+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88ec00 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f800 con 0x55c1cc7ee000 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f000 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.184+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88fc00 con 0x55c1cc88e400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892000 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892000 con 0x55c1cd3ba800 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(23 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.185+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892400 con 0x55c1cd3ba800 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(24 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893800 con 0x55c1cd3ba800 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892800 con 0x55c1cc7ee400 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(27 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.186+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892c00 con 0x55c1cc7ee000 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88ec00 con 0x55c1cc7ee400 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a44bf3640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 0x55c1cc874100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.187+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef400 con 0x55c1cc88f000 2024-04-13T03:16:40.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.188+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 0x55c1cc874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.188+0000 7f9a49bfd640 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 0x55c1cc874100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.190+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(31 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cc88f000 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.190+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893400 con 0x55c1cc88e400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.190+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(32 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.190+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e800 con 0x55c1cd2c4c00 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(33 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892c00 con 0x55c1cc88e400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(34 [pgnls start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f800 con 0x55c1cc7ee400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 7 ==== osd_op_reply(35 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.191+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892000 con 0x55c1cd2c4400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(36 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf400 con 0x55c1cc88e400 2024-04-13T03:16:40.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(37 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892400 con 0x55c1cd2c4800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(38 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd2c4800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.192+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf000 con 0x55c1cd2c4800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(39 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd2c4800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893c00 con 0x55c1cc88f000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(40 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cc88f000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7efc00 con 0x55c1cd3ba800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 9 ==== osd_op_reply(41 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88fc00 con 0x55c1cc7ee000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 5 ==== osd_op_reply(42 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.193+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef800 con 0x55c1cc88f000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.194+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(43 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cc88f000 2024-04-13T03:16:40.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.194+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e000 con 0x55c1cc7ee400 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.194+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 8 ==== osd_op_reply(44 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.194+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2c4000 con 0x55c1cd2c4800 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.6 v2:172.21.15.114:6816/1761123940 5 ==== osd_op_reply(45 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd2c4800 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f400 con 0x55c1cc7ee400 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 9 ==== osd_op_reply(46 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892800 con 0x55c1cd3ba800 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 10 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.195+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef400 con 0x55c1cd2c4c00 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 5 ==== osd_op_reply(48 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892000 con 0x55c1cd3ba800 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 11 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7efc00 con 0x55c1cc88f000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cc88f000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.196+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893800 con 0x55c1cc7ee000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 6 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef800 con 0x55c1cc7ee000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.2 v2:172.21.15.106:6824/4087145286 7 ==== osd_op_reply(52 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc7ee000 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cd2cf000 con 0x55c1cd3ba800 2024-04-13T03:16:40.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 12 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.197+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892400 con 0x55c1cc7ee400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 10 ==== osd_op_reply(54 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88ec00 con 0x55c1cd2c4400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893c00 con 0x55c1cc7ee400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 11 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.198+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f800 con 0x55c1cc88e400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.199+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v91'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.199+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ee800 con 0x55c1cc88e400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.199+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.199+0000 7f9a4be2b740 10 datascan.operator(): handling object 1099511627777.0 2024-04-13T03:16:40.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.199+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7eec00 con 0x55c1cc88e400 2024-04-13T03:16:40.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(59 10000000001.00000000 [call] v95'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc7ef000 con 0x55c1cc88e400 2024-04-13T03:16:40.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.5 v2:172.21.15.114:6808/1239989021 9 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cc88e400 2024-04-13T03:16:40.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88e000 con 0x55c1cd2c4400 2024-04-13T03:16:40.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(61 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cd2c4400 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.201+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893000 con 0x55c1cc7ee400 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 <== osd.4 v2:172.21.15.114:6800/1124906481 12 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272780 con 0x55c1cc7ee400 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc88f400 con 0x55c1cc88f000 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cc88f000 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc893400 con 0x55c1cd3ba800 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.202+0000 7f9a44bf3640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc8a2000 con 0x55c1cd3ba800 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a49bfd640 1 -- 172.21.15.106:0/3464789343 <== osd.0 v2:172.21.15.106:6808/3573709684 14 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272500 con 0x55c1cd3ba800 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a443f2640 1 -- 172.21.15.106:0/3464789343 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c1cc892c00 con 0x55c1cd2c4c00 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a48bfb640 1 -- 172.21.15.106:0/3464789343 <== osd.3 v2:172.21.15.106:6811/987945743 6 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c1cd272000 con 0x55c1cd2c4c00 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c1cd3ba800 msgr2=0x55c1cc872000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c1cd3ba800 0x55c1cc872000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c1cd2c4400 msgr2=0x55c1cc872580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c1cd2c4400 0x55c1cc872580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c1cc7ee000 msgr2=0x55c1cc652680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c1cc7ee000 0x55c1cc652680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c1cd2c4c00 msgr2=0x55c1cc872b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c1cd2c4c00 0x55c1cc872b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c1cc7ee400 msgr2=0x55c1cc873b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c1cc7ee400 0x55c1cc873b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c1cc88e400 msgr2=0x55c1cc873080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c1cc88e400 0x55c1cc873080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c1cd2c4800 msgr2=0x55c1cc873600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c1cd2c4800 0x55c1cc873600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 msgr2=0x55c1cc874100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 reap_dead start 2024-04-13T03:16:40.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 0x55c1cc874100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 reap_dead start 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a493fc640 1 -- 172.21.15.106:0/3464789343 reap_dead start 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 msgr2=0x55c1cc652100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 0x55c1cc652100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55c1cd27a1e0 tx=0x55c1cd278720 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 msgr2=0x55c1cc651600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55c1cd3cdcb0 tx=0x55c1cd3dba40 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 shutdown_connections 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c1cd2c5c00 0x55c1cc652100 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c1cc88f000 0x55c1cc874100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/3464789343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cd3bb800 0x55c1cc651600 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 >> 172.21.15.106:0/3464789343 conn(0x55c1cd3bb000 msgr2=0x55c1cc5c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 shutdown_connections 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.203+0000 7f9a4be2b740 1 -- 172.21.15.106:0/3464789343 wait complete. 2024-04-13T03:16:40.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 msgr2=0x55c1cc650580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55c1cd303c80 tx=0x55c1cd27c180 comp rx=0 tx=0).stop 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 shutdown_connections 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1cc4dfc00 0x55c1cc650b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1cc4df800 0x55c1cc650580 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 --2- 172.21.15.106:0/765716183 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c1cc4dec00 0x55c1cc650000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 >> 172.21.15.106:0/765716183 conn(0x55c1cc4df000 msgr2=0x55c1cc5c4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 shutdown_connections 2024-04-13T03:16:40.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.204+0000 7f9a4be2b740 1 -- 172.21.15.106:0/765716183 wait complete. 2024-04-13T03:16:40.211 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.088421, result: 2024-04-13T03:16:40.211 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-13T03:16:40.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.289+0000 7f99fb00f740 1 -- start start 2024-04-13T03:16:40.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.289+0000 7f99fb00f740 1 Processor -- start 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 0x5590735d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 0x5590735d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5590735dc180 con 0x559072805c00 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5590735dc300 con 0x559072805800 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5590735dc480 con 0x559072804c00 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51322/0 (socket says 172.21.15.106:51322) 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 -- 172.21.15.106:0/2202749197 learned_addr learned my addr 172.21.15.106:0/2202749197 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 0x5590735d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 0x5590735d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 -- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 msgr2=0x5590735d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 0x5590735d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 -- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 msgr2=0x5590735d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 0x5590735d8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 -- 172.21.15.106:0/2202749197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559072831680 con 0x559072805c00 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 0x5590735d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 0x5590735d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:40.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.290+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55907362f470 tx=0x5590735a2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7eac8322e522de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99f61f7640 1 -- 172.21.15.106:0/2202749197 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a9c0 con 0x559072805c00 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99f61f7640 1 -- 172.21.15.106:0/2202749197 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559072850e00 con 0x559072805c00 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99f61f7640 1 -- 172.21.15.106:0/2202749197 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a4e0 con 0x559072805c00 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559072831860 con 0x559072805c00 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5590728312c0 con 0x559072805c00 2024-04-13T03:16:40.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 4 waiting for MDS map... 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99f61f7640 1 -- 172.21.15.106:0/2202749197 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x5590736d2200 con 0x559072805c00 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99f61f7640 1 -- 172.21.15.106:0/2202749197 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 59) ==== 2885+0+0 (secure 0 0 0) 0x55907285f180 con 0x559072805c00 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 4 Got MDS map 59 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 4 datascan.main: Using metadata pool output 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 4 datascan.main: connecting to RADOS... 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.291+0000 7f99fb00f740 1 Processor -- start 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 -- start start 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5590735dc600 con 0x559073738000 2024-04-13T03:16:40.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5590735dc780 con 0x559073738400 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5590735dc900 con 0x559073738800 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51332/0 (socket says 172.21.15.106:51332) 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 -- 172.21.15.106:0/2297575011 learned_addr learned my addr 172.21.15.106:0/2297575011 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 -- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738800 msgr2=0x5590735d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 -- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 msgr2=0x5590735d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.292+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99f81fb640 1 -- 172.21.15.106:0/2297575011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590728314a0 con 0x559073738000 2024-04-13T03:16:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x559073614870 tx=0x5590736e7860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588003ec17044798 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99f41f3640 1 -- 172.21.15.106:0/2297575011 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a9c0 con 0x559073738000 2024-04-13T03:16:40.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99f41f3640 1 -- 172.21.15.106:0/2297575011 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559072850a80 con 0x559073738000 2024-04-13T03:16:40.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99f41f3640 1 -- 172.21.15.106:0/2297575011 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a4e0 con 0x559073738000 2024-04-13T03:16:40.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 -- 172.21.15.106:0/2297575011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 msgr2=0x5590735d9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x559073614870 tx=0x5590736e7860 comp rx=0 tx=0).stop 2024-04-13T03:16:40.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 -- 172.21.15.106:0/2297575011 shutdown_connections 2024-04-13T03:16:40.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2297575011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 -- 172.21.15.106:0/2297575011 >> 172.21.15.106:0/2297575011 conn(0x5590735f7c00 msgr2=0x5590728e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 -- 172.21.15.106:0/2297575011 shutdown_connections 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.293+0000 7f99fb00f740 1 -- 172.21.15.106:0/2297575011 wait complete. 2024-04-13T03:16:40.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 Processor -- start 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 -- start start 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5590735ddc80 con 0x559073738800 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5590735ddb00 con 0x559073738400 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99fb00f740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5590735dd980 con 0x559073738000 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99f81fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.294+0000 7f99f81fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51338/0 (socket says 172.21.15.106:51338) 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 learned_addr learned my addr 172.21.15.106:0/2501119994 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738000 msgr2=0x5590735d9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559073738000 0x5590735d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 msgr2=0x5590735d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590728310e0 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559073738400 0x5590735d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x559073615b30 tx=0x5590735a2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f29f0640 1 -- 172.21.15.106:0/2501119994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a000 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f29f0640 1 -- 172.21.15.106:0/2501119994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559072851340 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99f29f0640 1 -- 172.21.15.106:0/2501119994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55907283a9c0 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559072831a40 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590728303c0 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.295+0000 7f99fb00f740 4 datascan.init: resolving metadata pool 13 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.296+0000 7f99f29f0640 1 -- 172.21.15.106:0/2501119994 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5590736fec00 con 0x559073738800 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.296+0000 7f99f29f0640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 0x5590735da100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f29f0640 1 -- 172.21.15.106:0/2501119994 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x559073746a00 con 0x559073738800 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 0x5590735da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x559073738c00 0x5590735da680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559073738c00 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x559073738c00 0x5590735da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 0x5590735da100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5590735a0f30 tx=0x559072b1ed80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x559073738c00 0x5590735da680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x559072b91000 0x5590735dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.297+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072b91400 con 0x559072b91000 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.298+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x559072b91000 0x5590735dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.298+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x559072b91000 0x5590735dac00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.298+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x559072b91800 0x5590735db180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91c00 con 0x559072b91800 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x559072b91800 0x5590735db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x559072b91800 0x5590735db180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.299+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559073738c00 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f41f3640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559072ba4400 0x559072baa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072ba4400 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559072ba4400 0x559072baa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559072ba4400 0x559072baa000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559073738c00 2024-04-13T03:16:40.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.300+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559073738c00 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f39f2640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x559072ba5400 0x559072baa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559072ba5400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x559072ba5400 0x559072baa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x559072ba5400 0x559072baa580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559072ba4400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.301+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072ba4400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5800 con 0x559072b91000 2024-04-13T03:16:40.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559072ba4400 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f41f3640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x559072ba5c00 0x559072baab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.302+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4000 con 0x559072ba5c00 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x559072ba5c00 0x559072baab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f89fc640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x559072ba5c00 0x559072baab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5000 con 0x559072b91800 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.303+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559073738c00 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5400 con 0x559072ba4400 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4800 con 0x559073738c00 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f39f2640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x559072bb5800 0x559072bab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5c00 con 0x559072bb5800 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.304+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x559072bb5800 0x559072bab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.305+0000 7f99f81fb640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x559072bb5800 0x559072bab600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.305+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.305+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072b91000 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.305+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.305+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b90c00 con 0x559072ba5c00 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072b91800 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559072ba5c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4000 con 0x559072bb5800 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.306+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559072ba5c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5000 con 0x559073738c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559073738c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91c00 con 0x559073738c00 2024-04-13T03:16:40.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.307+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072ba5c00 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072ba5400 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559072b91800 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.308+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5400 con 0x559072ba5c00 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072ba5400 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f41f3640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 0x559072bab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5c00 con 0x559072bb4800 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 0x559072bab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.309+0000 7f99f91fd640 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 0x559072bab080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559072ba5400 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.310+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4000 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b90c00 con 0x559072ba5c00 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5000 con 0x559072ba4400 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5800 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.311+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91400 con 0x559072b91000 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559072b91000 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8000 con 0x559072bb4800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.312+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559073738c00 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8400 con 0x559072b91800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559072bb4800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.313+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8800 con 0x559072ba5c00 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8000 con 0x559072b91000 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8c00 con 0x559072ba5c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.314+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9c00 con 0x559073738c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9000 con 0x559072ba5400 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559073738c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9400 con 0x559072bb4800 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.315+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91400 con 0x559072b91800 2024-04-13T03:16:40.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9800 con 0x559072b91800 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559073738c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.316+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91c00 con 0x559072ba5c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072ba4400 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072ba5c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5c00 con 0x559072bb5800 2024-04-13T03:16:40.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.317+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9c00 con 0x559072b91800 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce000 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4000 con 0x559072ba4400 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(62 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.318+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce400 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5000 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce800 con 0x559072ba5400 2024-04-13T03:16:40.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5400 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcec00 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.319+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5800 con 0x559072b91000 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.320+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.320+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf000 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.320+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 11 ==== osd_op_reply(69 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.320+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072ba5c00 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf400 con 0x559072b91800 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559073738c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf800 con 0x559072ba4400 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf400 con 0x559073738c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.321+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcfc00 con 0x559072bb5800 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce000 con 0x559072b91000 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072ba5c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.322+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559072b91800 2024-04-13T03:16:40.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 10 ==== osd_op_reply(78 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072ba5c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072bb5800 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.323+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce800 con 0x559072ba5c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce000 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9800 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce400 con 0x559072ba5c00 2024-04-13T03:16:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.324+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9c00 con 0x559072ba5400 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559072b91800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91400 con 0x559072ba5c00 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5800 con 0x559072ba5400 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.325+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91c00 con 0x559072bb4800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8000 con 0x559072bb5800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5400 con 0x559072ba5400 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.326+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9000 con 0x559072bb5800 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5c00 con 0x559072ba5c00 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bc9400 con 0x559072ba4400 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 13 ==== osd_op_reply(95 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.327+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559072bb5800 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 11 ==== osd_op_reply(96 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072b91000 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559072b91000 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072bb4800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.328+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559073738c00 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5c00 con 0x559072b91800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 12 ==== osd_op_reply(101 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcec00 con 0x559072bb4800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.329+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4000 con 0x559072ba5c00 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf000 con 0x559072b91000 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5000 con 0x559072ba5c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf800 con 0x559073738c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.330+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072b91400 con 0x559072ba5400 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf400 con 0x559073738c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5000 con 0x559072bb4800 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcfc00 con 0x559072b91800 2024-04-13T03:16:40.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.331+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce800 con 0x559072b91800 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.2 v2:172.21.15.106:6824/4087145286 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072b91800 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcec00 con 0x559073738c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce400 con 0x559072ba5c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb5400 con 0x559072ba4400 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.332+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559073739000 con 0x559072ba5c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4400 con 0x559072bb5800 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99fb00f740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99fb00f740 10 datascan.operator(): handling object 10000000001.0 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.333+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8800 con 0x559072bb5800 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072bc8c00 con 0x559072ba4400 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 15 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072b90c00 con 0x559072b91000 2024-04-13T03:16:40.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.6 v2:172.21.15.114:6816/1761123940 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x559073596500 con 0x559072b91000 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 20 datascan.get_frag_of: frag is * 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.334+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072bce000 con 0x559072ba4400 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 16 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf800 con 0x559072ba4400 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 17 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 20 datascan.read_dentry: file_head not found in result 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.335+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4000 con 0x559072ba4400 2024-04-13T03:16:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.336+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 18 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v95'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.336+0000 7f99fb00f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.336+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4c00 con 0x559072bb5800 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.336+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.5 v2:172.21.15.114:6808/1239989021 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072bb5800 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.336+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bb4c00 con 0x559072ba4400 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.1 v2:172.21.15.106:6800/1701180838 19 ==== osd_op_reply(124 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559072ba4400 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcf400 con 0x559072ba5c00 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 <== osd.4 v2:172.21.15.114:6800/1124906481 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596780 con 0x559072ba5c00 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba5800 con 0x559072bb4800 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.7 v2:172.21.15.114:6824/300957724 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072bb4800 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bcec00 con 0x559073738c00 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.337+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f39f2640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072ba4800 con 0x559073738c00 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f81fb640 1 -- 172.21.15.106:0/2501119994 <== osd.0 v2:172.21.15.106:6808/3573709684 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596000 con 0x559073738c00 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f41f3640 1 -- 172.21.15.106:0/2501119994 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x559072bce000 con 0x559072ba5400 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f91fd640 1 -- 172.21.15.106:0/2501119994 <== osd.3 v2:172.21.15.106:6811/987945743 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559073596500 con 0x559072ba5400 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x559073738c00 msgr2=0x5590735da680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x559073738c00 0x5590735da680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559072ba4400 msgr2=0x559072baa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x559072ba4400 0x559072baa000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x559072b91800 msgr2=0x5590735db180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x559072b91800 0x5590735db180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x559072ba5400 msgr2=0x559072baa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x559072ba5400 0x559072baa580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x559072ba5c00 msgr2=0x559072baab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 reap_dead start 2024-04-13T03:16:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x559072ba5c00 0x559072baab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x559072bb5800 msgr2=0x559072bab600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x559072bb5800 0x559072bab600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x559072b91000 msgr2=0x5590735dac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x559072b91000 0x5590735dac00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99f89fc640 1 -- 172.21.15.106:0/2501119994 reap_dead start 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 msgr2=0x559072bab080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 0x559072bab080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 msgr2=0x5590735da100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 0x5590735da100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5590735a0f30 tx=0x559072b1ed80 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 msgr2=0x5590735d9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x559073615b30 tx=0x5590735a2c60 comp rx=0 tx=0).stop 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 shutdown_connections 2024-04-13T03:16:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x559072b90000 0x5590735da100 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x559072bb4800 0x559072bab080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2501119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559073738800 0x5590735d9b80 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 >> 172.21.15.106:0/2501119994 conn(0x5590735f7c00 msgr2=0x5590728e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 shutdown_connections 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.338+0000 7f99fb00f740 1 -- 172.21.15.106:0/2501119994 wait complete. 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 msgr2=0x5590735d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55907362f470 tx=0x5590735a2e40 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 shutdown_connections 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559072805c00 0x5590735d8b00 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559072805800 0x5590735d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 --2- 172.21.15.106:0/2202749197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x559072804c00 0x5590735d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 >> 172.21.15.106:0/2202749197 conn(0x559072805000 msgr2=0x5590728e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 shutdown_connections 2024-04-13T03:16:40.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.339+0000 7f99fb00f740 1 -- 172.21.15.106:0/2202749197 wait complete. 2024-04-13T03:16:40.370 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.158289, result: 2024-04-13T03:16:40.370 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-13T03:16:40.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 -- start start 2024-04-13T03:16:40.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 Processor -- start 2024-04-13T03:16:40.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae6936c00 0x557ae7700000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 0x557ae7700b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c780 con 0x557ae6937800 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c600 con 0x557ae6936c00 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557ae771c480 con 0x557ae6937c00 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c49dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c49dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51350/0 (socket says 172.21.15.106:51350) 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c49dfb640 1 -- 172.21.15.106:0/1265831115 learned_addr learned my addr 172.21.15.106:0/1265831115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 0x557ae7700b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.424+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae6936c00 0x557ae7700000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 -- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 msgr2=0x557ae7700b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 0x557ae7700b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 -- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae6936c00 msgr2=0x557ae7700000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae6936c00 0x557ae7700000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 -- 172.21.15.106:0/1265831115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae6963680 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 0x557ae7700b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x557ae7753c50 tx=0x557ae76d2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f524ef844341d765 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c47df7640 1 -- 172.21.15.106:0/1265831115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696c4e0 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c47df7640 1 -- 172.21.15.106:0/1265831115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557ae6982e00 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c47df7640 1 -- 172.21.15.106:0/1265831115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696cb60 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae6963860 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557ae69632c0 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 4 waiting for MDS map... 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c47df7640 1 -- 172.21.15.106:0/1265831115 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x557ae7814a00 con 0x557ae6937800 2024-04-13T03:16:40.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c47df7640 1 -- 172.21.15.106:0/1265831115 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 59) ==== 2885+0+0 (secure 0 0 0) 0x557ae6991180 con 0x557ae6937800 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 4 Got MDS map 59 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 4 datascan.main: Using metadata pool output 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.425+0000 7f9c4d02b740 4 datascan.main: connecting to RADOS... 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 Processor -- start 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 -- start start 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c300 con 0x557ae77f9400 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c000 con 0x557ae77f9c00 2024-04-13T03:16:40.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.426+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557ae695ca80 con 0x557ae77f9800 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51364/0 (socket says 172.21.15.106:51364) 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 -- 172.21.15.106:0/1806556673 learned_addr learned my addr 172.21.15.106:0/1806556673 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 -- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 msgr2=0x557ae7701600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 -- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 msgr2=0x557ae7701b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 -- 172.21.15.106:0/1806556673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae69634a0 con 0x557ae77f9400 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x557ae7752f30 tx=0x557ae76d0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3abcc6739447fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c45df3640 1 -- 172.21.15.106:0/1806556673 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696c9c0 con 0x557ae77f9400 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c45df3640 1 -- 172.21.15.106:0/1806556673 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557ae6982c40 con 0x557ae77f9400 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c45df3640 1 -- 172.21.15.106:0/1806556673 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696c4e0 con 0x557ae77f9400 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1806556673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 msgr2=0x557ae7701080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x557ae7752f30 tx=0x557ae76d0a80 comp rx=0 tx=0).stop 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.427+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1806556673 shutdown_connections 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1806556673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1806556673 >> 172.21.15.106:0/1806556673 conn(0x557ae77f9000 msgr2=0x557ae6a1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1806556673 shutdown_connections 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1806556673 wait complete. 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 Processor -- start 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.428+0000 7f9c4d02b740 1 -- start start 2024-04-13T03:16:40.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c300 con 0x557ae77f9c00 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557ae771c000 con 0x557ae77f9400 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557ae771d200 con 0x557ae77f9800 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51376/0 (socket says 172.21.15.106:51376) 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 learned_addr learned my addr 172.21.15.106:0/3648437963 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 msgr2=0x557ae7701600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae77f9800 0x557ae7701600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9400 msgr2=0x557ae7701080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae6963680 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae77f9400 0x557ae7701080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x557ae780f7d0 tx=0x557ae76d09c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c445f0640 1 -- 172.21.15.106:0/3648437963 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696c340 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c445f0640 1 -- 172.21.15.106:0/3648437963 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557ae6982c40 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ae6963860 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c445f0640 1 -- 172.21.15.106:0/3648437963 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557ae696c9c0 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae69630e0 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.429+0000 7f9c4d02b740 4 datascan.init: resolving metadata pool 13 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.430+0000 7f9c445f0640 1 -- 172.21.15.106:0/3648437963 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x557ae787cc00 con 0x557ae77f9c00 2024-04-13T03:16:40.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c445f0640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 0x557ae7702100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c445f0640 1 -- 172.21.15.106:0/3648437963 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x557ae7888600 con 0x557ae77f9c00 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 0x557ae7702100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4d02b740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4d02b740 4 datascan.main: resolving metadata pool 13 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557ae7879400 0x557ae7702680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7879800 con 0x557ae7879400 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557ae7879400 0x557ae7702680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 0x557ae7702100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x557ae7752f30 tx=0x557ae76d0660 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.431+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557ae7879400 0x557ae7702680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x557ae7879800 0x557ae6cba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c40000 con 0x557ae7879800 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x557ae7879800 0x557ae6cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x557ae7879800 0x557ae6cba000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.432+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x557ae6c40000 0x557ae6cba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae771f400 con 0x557ae6c40000 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x557ae6c40000 0x557ae6cba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x557ae6c40000 0x557ae6cba580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.433+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae771f400 con 0x557ae7879400 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x557ae771f400 0x557ae6cbab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c40400 con 0x557ae771f400 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x557ae771f400 0x557ae6cbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x557ae771f400 0x557ae6cbab00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x557ae6c40400 0x557ae6cbb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.434+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878800 con 0x557ae6c40400 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x557ae6c40400 0x557ae6cbb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x557ae6c40400 0x557ae6cbb600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878800 con 0x557ae7879800 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.435+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878800 con 0x557ae771f400 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.436+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.436+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x557ae7878800 0x557ae6cbbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.436+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878c00 con 0x557ae7878800 2024-04-13T03:16:40.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.436+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x557ae7878800 0x557ae6cbbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.436+0000 7f9c4a5fc640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x557ae7878800 0x557ae6cbbb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878c00 con 0x557ae6c40400 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878c00 con 0x557ae6c40000 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(11 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae7878c00 con 0x557ae7879800 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x557ae7878c00 0x557ae6cbc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.437+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c40c00 con 0x557ae7878c00 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x557ae7878c00 0x557ae6cbc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x557ae7878c00 0x557ae6cbc680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c49dfb640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x557ae6c40c00 0x557ae6cbc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae6c40c00 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x557ae6c40c00 0x557ae6cbc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.438+0000 7f9c4adfd640 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x557ae6c40c00 0x557ae6cbc100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40c00 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7879400 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878c00 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.439+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae771f400 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7879800 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878800 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.440+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae6c40c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.441+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7879400 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae6c40400 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.442+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7879400 2024-04-13T03:16:40.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878800 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878c00 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(28 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.443+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7879400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae7878800 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae771f400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.444+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae771f400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 5 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-13T03:16:40.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae7879c00 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.445+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde400 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 20 datascan.scan_links: injected ino detected 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde800 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdec00 con 0x557ae7879800 2024-04-13T03:16:40.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf000 con 0x557ae6c40400 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.446+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf400 con 0x557ae771f400 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40000 2024-04-13T03:16:40.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde400 con 0x557ae7879400 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.447+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf800 con 0x557ae7878c00 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-13T03:16:40.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae7879c00 con 0x557ae7879400 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.448+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41800 con 0x557ae7879400 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41400 con 0x557ae7879400 2024-04-13T03:16:40.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.449+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41000 con 0x557ae771f400 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 11 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 10 ==== osd_op_reply(48 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.450+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40000 2024-04-13T03:16:40.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 12 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae771f400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 8 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.451+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.452+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(52 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.452+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.452+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 11 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.452+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae771f400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 9 ==== osd_op_reply(54 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878800 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40400 2024-04-13T03:16:40.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40000 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 5 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.453+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 12 ==== osd_op_reply(58 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 8 ==== osd_op_reply(59 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.454+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.455+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 13 ==== osd_op_reply(61 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.455+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.455+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 9 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.455+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae771f400 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 10 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878800 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.456+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 10 ==== osd_op_reply(67 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 11 ==== osd_op_reply(68 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 12 ==== osd_op_reply(69 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.457+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 14 ==== osd_op_reply(70 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae6c40400 2024-04-13T03:16:40.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(71 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 15 ==== osd_op_reply(72 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.458+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878800 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(73 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878c00 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 13 ==== osd_op_reply(74 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879400 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 16 ==== osd_op_reply(75 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.459+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7878800 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(76 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7878800 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae771f400 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 11 ==== osd_op_reply(77 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae771f400 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 12 ==== osd_op_reply(78 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.460+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde400 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdfc00 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae7879c00 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 20 datascan.scan_links: injected ino detected 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.461+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c40800 con 0x557ae7879800 2024-04-13T03:16:40.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c40800 con 0x557ae7879800 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae7879c00 con 0x557ae6c40400 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba500 con 0x557ae6c40400 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde800 con 0x557ae771f400 2024-04-13T03:16:40.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.462+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdec00 con 0x557ae6c40000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.2 v2:172.21.15.106:6824/4087145286 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x557ae76ba000 con 0x557ae6c40000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf000 con 0x557ae7879400 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.463+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde400 con 0x557ae7878c00 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-13T03:16:40.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdfc00 con 0x557ae7879400 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.464+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41800 con 0x557ae7879400 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-13T03:16:40.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41400 con 0x557ae7879400 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.465+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41c00 con 0x557ae771f400 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.6 v2:172.21.15.114:6816/1761123940 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x557ae76ba780 con 0x557ae771f400 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-13T03:16:40.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6c41000 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.466+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde000 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v95'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 10 datascan.scan_links: processing 1 injected_inos 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde400 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 20 datascan.scan_links: first is now 2 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.467+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cde800 con 0x557ae7879800 2024-04-13T03:16:40.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.468+0000 7f9c4adfd640 1 -- 172.21.15.106:0/3648437963 <== osd.1 v2:172.21.15.106:6800/1701180838 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v95'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x557ae76ba500 con 0x557ae7879800 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.468+0000 7f9c4d02b740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.468+0000 7f9c4d02b740 10 datascan.scan_links: updating inotable 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.468+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdec00 con 0x557ae7879400 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.469+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.469+0000 7f9c4d02b740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.469+0000 7f9c4d02b740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.469+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf000 con 0x557ae7879400 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 <== osd.7 v2:172.21.15.114:6824/300957724 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v95'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x557ae76ba000 con 0x557ae7879400 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 10 datascan.scan_links: updating snaptable 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x557ae6cdf400 con 0x557ae7878c00 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c49dfb640 1 -- 172.21.15.106:0/3648437963 <== osd.4 v2:172.21.15.114:6800/1124906481 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x557ae76ba780 con 0x557ae7878c00 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x557ae6c40400 msgr2=0x557ae6cbb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x557ae6c40400 0x557ae6cbb600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x557ae7879800 msgr2=0x557ae6cba000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x557ae7879800 0x557ae6cba000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x557ae6c40000 msgr2=0x557ae6cba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x557ae6c40000 0x557ae6cba580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x557ae6c40c00 msgr2=0x557ae6cbc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x557ae6c40c00 0x557ae6cbc100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x557ae7878c00 msgr2=0x557ae6cbc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x557ae7878c00 0x557ae6cbc680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 reap_dead start 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x557ae7878800 msgr2=0x557ae6cbbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x557ae7878800 0x557ae6cbbb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x557ae771f400 msgr2=0x557ae6cbab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x557ae771f400 0x557ae6cbab00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557ae7879400 msgr2=0x557ae7702680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x557ae7879400 0x557ae7702680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 reap_dead start 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.470+0000 7f9c4a5fc640 1 -- 172.21.15.106:0/3648437963 reap_dead start 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 msgr2=0x557ae7702100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 0x557ae7702100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x557ae7752f30 tx=0x557ae76d0660 comp rx=0 tx=0).stop 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 msgr2=0x557ae7701b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x557ae780f7d0 tx=0x557ae76d09c0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 shutdown_connections 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x557ae7879000 0x557ae7702100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/3648437963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae77f9c00 0x557ae7701b80 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 >> 172.21.15.106:0/3648437963 conn(0x557ae77f9000 msgr2=0x557ae6a1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 shutdown_connections 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/3648437963 wait complete. 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 msgr2=0x557ae7700580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x557ae7753c50 tx=0x557ae76d2120 comp rx=0 tx=0).stop 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 shutdown_connections 2024-04-13T03:16:40.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557ae6937c00 0x557ae7700b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae6937800 0x557ae7700580 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 --2- 172.21.15.106:0/1265831115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ae6936c00 0x557ae7700000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 >> 172.21.15.106:0/1265831115 conn(0x557ae6937000 msgr2=0x557ae6a1c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 shutdown_connections 2024-04-13T03:16:40.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.471+0000 7f9c4d02b740 1 -- 172.21.15.106:0/1265831115 wait complete. 2024-04-13T03:16:40.493 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.123098, result: 2024-04-13T03:16:40.493 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:16:40.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.626+0000 7f7b49460640 1 Processor -- start 2024-04-13T03:16:40.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 -- start start 2024-04-13T03:16:40.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 0x7f7b440d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7b440b69c0 0x7f7b440d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b440d4110 con 0x7f7b440d4a90 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7b440cfbf0 con 0x7f7b440b69c0 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b49460640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7b440b3fd0 con 0x7f7b440b3bf0 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.627+0000 7f7b437fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7b440d4a90 0x7f7b4415e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57892/0 (socket says 172.21.15.106:57892) 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b437fe640 1 -- 172.21.15.106:0/215311882 learned_addr learned my addr 172.21.15.106:0/215311882 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/215311882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 0x7f7b440d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b427fc640 1 --2- 172.21.15.106:0/215311882 >> v2:172.21.15.114:3300/0 conn(0x7f7b440b69c0 0x7f7b440d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b41ffb640 1 -- 172.21.15.106:0/215311882 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467003169 0 0) 0x7f7b440d4110 con 0x7f7b440d4a90 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b41ffb640 1 -- 172.21.15.106:0/215311882 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b2c003520 con 0x7f7b440d4a90 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b42ffd640 1 -- 172.21.15.106:0/215311882 >> v2:172.21.15.114:3300/0 conn(0x7f7b440b69c0 msgr2=0x7f7b440d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/215311882 >> v2:172.21.15.114:3300/0 conn(0x7f7b440b69c0 0x7f7b440d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b42ffd640 1 -- 172.21.15.106:0/215311882 >> v1:172.21.15.106:6789/0 conn(0x7f7b440d4a90 legacy=0x7f7b4415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.628+0000 7f7b42ffd640 1 -- 172.21.15.106:0/215311882 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b440d4f20 con 0x7f7b440b3bf0 2024-04-13T03:16:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/215311882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 0x7f7b440d2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009d90 tx=0x7f7b3402bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f682f631c5da8f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b41ffb640 1 -- 172.21.15.106:0/215311882 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34039070 con 0x7f7b440b3bf0 2024-04-13T03:16:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b41ffb640 1 -- 172.21.15.106:0/215311882 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3402cb30 con 0x7f7b440b3bf0 2024-04-13T03:16:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b41ffb640 1 -- 172.21.15.106:0/215311882 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34034440 con 0x7f7b440b3bf0 2024-04-13T03:16:40.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b49460640 1 -- 172.21.15.106:0/215311882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 msgr2=0x7f7b440d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.629+0000 7f7b49460640 1 --2- 172.21.15.106:0/215311882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 0x7f7b440d2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009d90 tx=0x7f7b3402bd80 comp rx=0 tx=0).stop 2024-04-13T03:16:40.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 -- 172.21.15.106:0/215311882 shutdown_connections 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 --2- 172.21.15.106:0/215311882 >> v2:172.21.15.114:3300/0 conn(0x7f7b440b69c0 0x7f7b440d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 --2- 172.21.15.106:0/215311882 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440b3bf0 0x7f7b440d2c70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 -- 172.21.15.106:0/215311882 >> 172.21.15.106:0/215311882 conn(0x7f7b440b96e0 msgr2=0x7f7b440b9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 -- 172.21.15.106:0/215311882 shutdown_connections 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.630+0000 7f7b49460640 1 -- 172.21.15.106:0/215311882 wait complete. 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.631+0000 7f7b49460640 1 Processor -- start 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.631+0000 7f7b49460640 1 -- start start 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 0x7f7b44159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440d4a90 0x7f7b4409ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b440d3ac0 con 0x7f7b440b69c0 2024-04-13T03:16:40.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b440d37e0 con 0x7f7b440b3bf0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7b440d3da0 con 0x7f7b440d4a90 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51388/0 (socket says 172.21.15.106:51388) 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 -- 172.21.15.106:0/318170729 learned_addr learned my addr 172.21.15.106:0/318170729 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 0x7f7b44159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b437fe640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440d4a90 0x7f7b4409ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 -- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440d4a90 msgr2=0x7f7b4409ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440d4a90 0x7f7b4409ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 -- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 msgr2=0x7f7b44159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 0x7f7b44159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 -- 172.21.15.106:0/318170729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b440a13c0 con 0x7f7b440b69c0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 0x7f7b44159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b427fc640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e900 tx=0x7f7b3800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38017070 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3800ca00 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800cdb0 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b440a16a0 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.632+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b440a1c60 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.633+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44069120 con 0x7f7b440b69c0 2024-04-13T03:16:40.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.634+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b3801a070 con 0x7f7b440b69c0 2024-04-13T03:16:40.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.634+0000 7f7b2b7fe640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 0x7f7b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:40.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.634+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f7b3808c170 con 0x7f7b440b69c0 2024-04-13T03:16:40.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.634+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 0x7f7b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:40.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.635+0000 7f7b42ffd640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 0x7f7b3006edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b34036000 tx=0x7f7b34030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:40.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.639+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b3805a380 con 0x7f7b440b69c0 2024-04-13T03:16:40.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:40.851+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7b440d5320 con 0x7f7b440b69c0 2024-04-13T03:16:41.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.111+0000 7f7b2b7fe640 1 -- 172.21.15.106:0/318170729 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v60) ==== 162+0+0 (secure 0 0 0) 0x7f7b38059c80 con 0x7f7b440b69c0 2024-04-13T03:16:41.112 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 msgr2=0x7f7b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 0x7f7b3006edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b34036000 tx=0x7f7b34030000 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 msgr2=0x7f7b4409e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e900 tx=0x7f7b3800edc0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 shutdown_connections 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b3006c8f0 0x7f7b3006edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b440d4a90 0x7f7b4409ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b440b69c0 0x7f7b4409e4f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 --2- 172.21.15.106:0/318170729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b440b3bf0 0x7f7b44159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 >> 172.21.15.106:0/318170729 conn(0x7f7b440b96e0 msgr2=0x7f7b440ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:41.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 shutdown_connections 2024-04-13T03:16:41.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.114+0000 7f7b49460640 1 -- 172.21.15.106:0/318170729 wait complete. 2024-04-13T03:16:41.128 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:41.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.228+0000 7f8dd39c8640 1 Processor -- start 2024-04-13T03:16:41.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 -- start start 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 0x7f8dcc0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dcc0d2310 con 0x7f8dcc0ccba0 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8dcc0b1f30 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.229+0000 7f8dd39c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b2210 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd0f63640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 0x7f8dcc0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1f65640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8dcc0ccba0 0x7f8dcc0b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:42846/0 (socket says 172.21.15.106:42846) 2024-04-13T03:16:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1f65640 1 -- 172.21.15.106:0/3496961698 learned_addr learned my addr 172.21.15.106:0/3496961698 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:33352/0 (socket says 172.21.15.106:33352) 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 -- 172.21.15.106:0/3496961698 >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 msgr2=0x7f8dcc0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 --2- 172.21.15.106:0/3496961698 >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 0x7f8dcc0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 -- 172.21.15.106:0/3496961698 >> v1:172.21.15.114:6789/0 conn(0x7f8dcc0ccba0 legacy=0x7f8dcc0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd1764640 1 -- 172.21.15.106:0/3496961698 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc0b24f0 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.230+0000 7f8dd0f63640 1 --2- 172.21.15.106:0/3496961698 >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 0x7f8dcc0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.231+0000 7f8dd1764640 1 --2- 172.21.15.106:0/3496961698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009d90 tx=0x7f8dc402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8b0e2632733e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.231+0000 7f8dcbfff640 1 -- 172.21.15.106:0/3496961698 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4039070 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.231+0000 7f8dcbfff640 1 -- 172.21.15.106:0/3496961698 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc402cc50 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.231+0000 7f8dcbfff640 1 -- 172.21.15.106:0/3496961698 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4034440 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 -- 172.21.15.106:0/3496961698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 msgr2=0x7f8dcc160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/3496961698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009d90 tx=0x7f8dc402bea0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 -- 172.21.15.106:0/3496961698 shutdown_connections 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/3496961698 >> v2:172.21.15.114:3300/0 conn(0x7f8dcc0c60a0 0x7f8dcc0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/3496961698 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc15c7d0 0x7f8dcc160dd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 -- 172.21.15.106:0/3496961698 >> 172.21.15.106:0/3496961698 conn(0x7f8dcc0d2550 msgr2=0x7f8dcc0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 -- 172.21.15.106:0/3496961698 shutdown_connections 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.232+0000 7f8dd39c8640 1 -- 172.21.15.106:0/3496961698 wait complete. 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.233+0000 7f8dd39c8640 1 Processor -- start 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.233+0000 7f8dd39c8640 1 -- start start 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc0ccba0 0x7f8dcc1e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dcc15c7d0 0x7f8dcc159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc15cbb0 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0cd010 con 0x7f8dcc15c7d0 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc15a050 con 0x7f8dcc0ccba0 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51394/0 (socket says 172.21.15.106:51394) 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 -- 172.21.15.106:0/449515442 learned_addr learned my addr 172.21.15.106:0/449515442 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1f65640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dcc15c7d0 0x7f8dcc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd0f63640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc0ccba0 0x7f8dcc1e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 -- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc0ccba0 msgr2=0x7f8dcc1e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc0ccba0 0x7f8dcc1e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 -- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dcc15c7d0 msgr2=0x7f8dcc159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dcc15c7d0 0x7f8dcc159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 -- 172.21.15.106:0/449515442 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc0cbe00 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd1764640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4037040 tx=0x7f8dc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4039050 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc4045070 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc0cc0e0 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.234+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4009450 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.235+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc0cc6a0 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.236+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc069120 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.236+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dc403e610 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.236+0000 7f8dc9ffb640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.237+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f8dc40ae9a0 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.237+0000 7f8dd0f63640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:41.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.237+0000 7f8dd0f63640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009560 tx=0x7f8dc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:41.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.241+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dc407ca90 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.455+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dcc0d31b0 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.455+0000 7f8dc9ffb640 1 -- 172.21.15.106:0/449515442 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5189 (secure 0 0 0) 0x7f8dc407c390 con 0x7f8dcc0c60a0 2024-04-13T03:16:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:41.457 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:41.114490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-13T03:16:41.457 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 61 2024-04-13T03:16:41.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 msgr2=0x7f8db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009560 tx=0x7f8dc0008040 comp rx=0 tx=0).stop 2024-04-13T03:16:41.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 msgr2=0x7f8dcc0b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4037040 tx=0x7f8dc4004a40 comp rx=0 tx=0).stop 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 shutdown_connections 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8dcc0ccba0 0x7f8dcc1e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dcc15c7d0 0x7f8dcc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 --2- 172.21.15.106:0/449515442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dcc0c60a0 0x7f8dcc0b7a20 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:41.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 >> 172.21.15.106:0/449515442 conn(0x7f8dcc0d2550 msgr2=0x7f8dcc15ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:41.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 shutdown_connections 2024-04-13T03:16:41.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:41.457+0000 7f8dd39c8640 1 -- 172.21.15.106:0/449515442 wait complete. 2024-04-13T03:16:41.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:16:41.114490+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8028}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8028': {'gid': 8028, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/2398448953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2398448953}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2398448953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:16:41.872 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8028:up:replay 2024-04-13T03:16:42.873 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:42.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.984+0000 7f3a29660640 1 Processor -- start 2024-04-13T03:16:42.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.984+0000 7f3a29660640 1 -- start start 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a29660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a29660640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a29660640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a240cfd20 con 0x7f3a240b3d60 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a29660640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3a24162af0 con 0x7f3a2415ed90 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a29660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a24162dd0 con 0x7f3a2415e9b0 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a227fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a22ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a227fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:32812/0 (socket says 172.21.15.106:32812) 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a227fc640 1 -- 172.21.15.106:0/1840653914 learned_addr learned my addr 172.21.15.106:0/1840653914 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:42.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a237fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3a240b3d60 0x7f3a240d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46932/0 (socket says 172.21.15.106:46932) 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.985+0000 7f3a22ffd640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:43190/0 (socket says 172.21.15.106:43190) 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.986+0000 7f3a22ffd640 1 -- 172.21.15.106:0/1840653914 >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 msgr2=0x7f3a240d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.986+0000 7f3a22ffd640 1 --2- 172.21.15.106:0/1840653914 >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.986+0000 7f3a22ffd640 1 -- 172.21.15.106:0/1840653914 >> v1:172.21.15.114:6789/0 conn(0x7f3a240b3d60 legacy=0x7f3a240d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.986+0000 7f3a22ffd640 1 -- 172.21.15.106:0/1840653914 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a241630b0 con 0x7f3a2415e9b0 2024-04-13T03:16:42.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.986+0000 7f3a227fc640 1 --2- 172.21.15.106:0/1840653914 >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:16:42.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a22ffd640 1 --2- 172.21.15.106:0/1840653914 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a140097a0 tx=0x7f3a1402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c39757eafbc62ba server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a21ffb640 1 -- 172.21.15.106:0/1840653914 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14034070 con 0x7f3a2415e9b0 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a21ffb640 1 -- 172.21.15.106:0/1840653914 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a14030610 con 0x7f3a2415e9b0 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a21ffb640 1 -- 172.21.15.106:0/1840653914 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a140309e0 con 0x7f3a2415e9b0 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 -- 172.21.15.106:0/1840653914 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 msgr2=0x7f3a240d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 --2- 172.21.15.106:0/1840653914 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a140097a0 tx=0x7f3a1402b960 comp rx=0 tx=0).stop 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 -- 172.21.15.106:0/1840653914 shutdown_connections 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 --2- 172.21.15.106:0/1840653914 >> v2:172.21.15.114:3300/0 conn(0x7f3a2415ed90 0x7f3a240d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 --2- 172.21.15.106:0/1840653914 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a2415e9b0 0x7f3a240d2300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 -- 172.21.15.106:0/1840653914 >> 172.21.15.106:0/1840653914 conn(0x7f3a2408bf10 msgr2=0x7f3a240b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.987+0000 7f3a29660640 1 -- 172.21.15.106:0/1840653914 shutdown_connections 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.988+0000 7f3a29660640 1 -- 172.21.15.106:0/1840653914 wait complete. 2024-04-13T03:16:42.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 Processor -- start 2024-04-13T03:16:42.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 -- start start 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a240b3d60 0x7f3a24159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a240c9eb0 0x7f3a24160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24162800 con 0x7f3a2415e9b0 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24162680 con 0x7f3a240b3d60 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.989+0000 7f3a29660640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2406eb20 con 0x7f3a240c9eb0 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49368/0 (socket says 172.21.15.106:49368) 2024-04-13T03:16:42.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 -- 172.21.15.106:0/389694751 learned_addr learned my addr 172.21.15.106:0/389694751 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a237fe640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a240c9eb0 0x7f3a24160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a22ffd640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a240b3d60 0x7f3a24159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 -- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a240c9eb0 msgr2=0x7f3a24160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a240c9eb0 0x7f3a24160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 -- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a240b3d60 msgr2=0x7f3a24159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a240b3d60 0x7f3a24159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 -- 172.21.15.106:0/389694751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a240ca430 con 0x7f3a2415e9b0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a227fc640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9e0 tx=0x7f3a1800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18005990 con 0x7f3a2415e9b0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a18005b30 con 0x7f3a2415e9b0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a240ca710 con 0x7f3a2415e9b0 2024-04-13T03:16:42.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800c9d0 con 0x7f3a2415e9b0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.990+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a240cacd0 con 0x7f3a2415e9b0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.991+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a24068a10 con 0x7f3a2415e9b0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.992+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a1800cb70 con 0x7f3a2415e9b0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.992+0000 7f3a0b7fe640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 0x7f3a1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.992+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f3a1808c140 con 0x7f3a2415e9b0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.992+0000 7f3a22ffd640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 0x7f3a1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:42.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.993+0000 7f3a22ffd640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 0x7f3a1006edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a14036000 tx=0x7f3a1403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:42.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:42.997+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a18017020 con 0x7f3a2415e9b0 2024-04-13T03:16:43.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.210+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a240d53a0 con 0x7f3a2415e9b0 2024-04-13T03:16:43.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.211+0000 7f3a0b7fe640 1 -- 172.21.15.106:0/389694751 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5189 (secure 0 0 0) 0x7f3a1800ce60 con 0x7f3a2415e9b0 2024-04-13T03:16:43.212 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:43.212 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:42.119782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-13T03:16:43.212 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 63 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 msgr2=0x7f3a1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 0x7f3a1006edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a14036000 tx=0x7f3a1403b040 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 msgr2=0x7f3a240c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9e0 tx=0x7f3a1800eea0 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 shutdown_connections 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3a1006c8f0 0x7f3a1006edb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3a240c9eb0 0x7f3a24160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a2415e9b0 0x7f3a240c9970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 --2- 172.21.15.106:0/389694751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a240b3d60 0x7f3a24159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:43.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 >> 172.21.15.106:0/389694751 conn(0x7f3a2408bf10 msgr2=0x7f3a240b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:43.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 shutdown_connections 2024-04-13T03:16:43.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:43.213+0000 7f3a29660640 1 -- 172.21.15.106:0/389694751 wait complete. 2024-04-13T03:16:43.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:16:42.119782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8028}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8028': {'gid': 8028, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.106:6837/2398448953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2398448953}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2398448953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:16:43.630 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8028:up:rejoin 2024-04-13T03:16:44.631 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:16:44.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.742+0000 7f354051e640 1 Processor -- start 2024-04-13T03:16:44.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.742+0000 7f354051e640 1 -- start start 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f354051e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3538160c10 0x7f353806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f354051e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f353806e2b0 0x7f353806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f354051e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f353815d690 con 0x7f3538156b50 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f354051e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f353806e700 con 0x7f3538160c10 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f354051e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3538161080 con 0x7f353806e2b0 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3538160c10 0x7f353806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3538160c10 0x7f353806dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49382/0 (socket says 172.21.15.106:49382) 2024-04-13T03:16:44.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 -- 172.21.15.106:0/9469232 learned_addr learned my addr 172.21.15.106:0/9469232 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:44.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353eabb640 1 --2- 172.21.15.106:0/9469232 >> v2:172.21.15.114:3300/0 conn(0x7f353806e2b0 0x7f353806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 -- 172.21.15.106:0/9469232 >> v2:172.21.15.114:3300/0 conn(0x7f353806e2b0 msgr2=0x7f353806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:44.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 --2- 172.21.15.106:0/9469232 >> v2:172.21.15.114:3300/0 conn(0x7f353806e2b0 0x7f353806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 -- 172.21.15.106:0/9469232 >> v1:172.21.15.114:6789/0 conn(0x7f3538156b50 legacy=0x7f353815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:44.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.743+0000 7f353dab9640 1 -- 172.21.15.106:0/9469232 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353815d970 con 0x7f3538160c10 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.744+0000 7f353dab9640 1 --2- 172.21.15.106:0/9469232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538160c10 0x7f353806dd70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f35380cdd50 tx=0x7f352802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5f5d1fac3ca19 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.744+0000 7f353d2b8640 1 -- 172.21.15.106:0/9469232 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528035020 con 0x7f3538160c10 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.744+0000 7f353d2b8640 1 -- 172.21.15.106:0/9469232 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528031870 con 0x7f3538160c10 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.744+0000 7f353d2b8640 1 -- 172.21.15.106:0/9469232 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528031c40 con 0x7f3538160c10 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 -- 172.21.15.106:0/9469232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538160c10 msgr2=0x7f353806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 --2- 172.21.15.106:0/9469232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538160c10 0x7f353806dd70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f35380cdd50 tx=0x7f352802b910 comp rx=0 tx=0).stop 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 -- 172.21.15.106:0/9469232 shutdown_connections 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 --2- 172.21.15.106:0/9469232 >> v2:172.21.15.114:3300/0 conn(0x7f353806e2b0 0x7f353806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 --2- 172.21.15.106:0/9469232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538160c10 0x7f353806dd70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 -- 172.21.15.106:0/9469232 >> 172.21.15.106:0/9469232 conn(0x7f35380b1f10 msgr2=0x7f35380b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:44.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 -- 172.21.15.106:0/9469232 shutdown_connections 2024-04-13T03:16:44.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.745+0000 7f354051e640 1 -- 172.21.15.106:0/9469232 wait complete. 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.746+0000 7f354051e640 1 Processor -- start 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.746+0000 7f354051e640 1 -- start start 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 0x7f35380cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3538160c10 0x7f353815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f353815d320 con 0x7f3538156b50 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35381606f0 con 0x7f353806e2b0 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3538156f30 con 0x7f3538160c10 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49396/0 (socket says 172.21.15.106:49396) 2024-04-13T03:16:44.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 -- 172.21.15.106:0/3818547568 learned_addr learned my addr 172.21.15.106:0/3818547568 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353e2ba640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 0x7f35380cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353eabb640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3538160c10 0x7f353815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 -- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3538160c10 msgr2=0x7f353815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3538160c10 0x7f353815f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 -- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 msgr2=0x7f35380cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 0x7f35380cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 -- 172.21.15.106:0/3818547568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35380cf280 con 0x7f3538156b50 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353e2ba640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 0x7f35380cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f353dab9640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f35280090b0 tx=0x7f35280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528041070 con 0x7f3538156b50 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352803ad00 con 0x7f3538156b50 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528035040 con 0x7f3538156b50 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.747+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3538154550 con 0x7f3538156b50 2024-04-13T03:16:44.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.748+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3538154b10 con 0x7f3538156b50 2024-04-13T03:16:44.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.749+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35380afaa0 con 0x7f3538156b50 2024-04-13T03:16:44.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.749+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3528038020 con 0x7f3538156b50 2024-04-13T03:16:44.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.749+0000 7f352effd640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 0x7f351c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:44.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.749+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f35280adc30 con 0x7f3538156b50 2024-04-13T03:16:44.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.749+0000 7f353e2ba640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 0x7f351c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:44.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.750+0000 7f353e2ba640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 0x7f351c06edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f353400a830 tx=0x7f3534008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:44.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.754+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35380afaa0 con 0x7f3538156b50 2024-04-13T03:16:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.964+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f353806d110 con 0x7f3538156b50 2024-04-13T03:16:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.965+0000 7f352effd640 1 -- 172.21.15.106:0/3818547568 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7f353806d110 con 0x7f3538156b50 2024-04-13T03:16:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:16:44.966 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:16:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 msgr2=0x7f351c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 0x7f351c06edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f353400a830 tx=0x7f3534008040 comp rx=0 tx=0).stop 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 msgr2=0x7f35380ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f35280090b0 tx=0x7f35280046c0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 shutdown_connections 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f351c06c8f0 0x7f351c06edb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3538160c10 0x7f353815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3538156b50 0x7f35380ccda0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 --2- 172.21.15.106:0/3818547568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353806e2b0 0x7f35380cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:44.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 >> 172.21.15.106:0/3818547568 conn(0x7f35380b1f10 msgr2=0x7f3538072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:44.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 shutdown_connections 2024-04-13T03:16:44.969 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:44.967+0000 7f354051e640 1 -- 172.21.15.106:0/3818547568 wait complete. 2024-04-13T03:16:45.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:15:19.300957+0000', 'modified': '2024-04-13T03:16:44.123357+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8028}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8028': {'gid': 8028, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.106:6837/2398448953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 2398448953}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2398448953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8028, 'qdb_cluster': [8028]} 2024-04-13T03:16:45.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:16:45.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-13T03:16:45.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.484+0000 7fbe2b915640 1 Processor -- start 2024-04-13T03:16:45.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.484+0000 7fbe2b915640 1 -- start start 2024-04-13T03:16:45.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe2b915640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbe2415ee40 0x7fbe240704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe2b915640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe240b5cf0 con 0x7fbe240709e0 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe2b915640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe240b5fd0 con 0x7fbe240b1c30 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe2b915640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbe240b62b0 con 0x7fbe2415ee40 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe29eb2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbe240709e0 0x7fbe2406d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35122/0 (socket says 172.21.15.106:35122) 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe29eb2640 1 -- 172.21.15.106:0/658556619 learned_addr learned my addr 172.21.15.106:0/658556619 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/658556619 >> v2:172.21.15.114:3300/0 conn(0x7fbe2415ee40 0x7fbe240704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037470157 0 0) 0x7fbe240b5cf0 con 0x7fbe240709e0 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe10003520 con 0x7fbe240709e0 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 912554654 0 0) 0x7fbe240b5fd0 con 0x7fbe240b1c30 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.485+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe240b5cf0 con 0x7fbe240b1c30 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.486+0000 7fbe28eb0640 1 -- 172.21.15.106:0/658556619 >> v1:172.21.15.114:6789/0 conn(0x7fbe240b1c30 legacy=0x7fbe240b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.486+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1695534440 0 0) 0x7fbe10003520 con 0x7fbe240709e0 2024-04-13T03:16:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.486+0000 7fbe28eb0640 1 -- 172.21.15.106:0/658556619 >> v1:172.21.15.106:6789/0 conn(0x7fbe240709e0 legacy=0x7fbe2406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.486+0000 7fbe28eb0640 1 -- 172.21.15.106:0/658556619 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe240b6590 con 0x7fbe2415ee40 2024-04-13T03:16:45.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.486+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/658556619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe2415ee40 0x7fbe240704a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbe140097a0 tx=0x7fbe1402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569a1800e2d4d2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14038070 con 0x7fbe2415ee40 2024-04-13T03:16:45.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe14031500 con 0x7fbe2415ee40 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe1bfff640 1 -- 172.21.15.106:0/658556619 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c001670 con 0x7fbe2415ee40 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 -- 172.21.15.106:0/658556619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe2415ee40 msgr2=0x7fbe240704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 --2- 172.21.15.106:0/658556619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe2415ee40 0x7fbe240704a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbe140097a0 tx=0x7fbe1402b820 comp rx=0 tx=0).stop 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 -- 172.21.15.106:0/658556619 shutdown_connections 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 --2- 172.21.15.106:0/658556619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe2415ee40 0x7fbe240704a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 -- 172.21.15.106:0/658556619 >> 172.21.15.106:0/658556619 conn(0x7fbe24071870 msgr2=0x7fbe24071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 -- 172.21.15.106:0/658556619 shutdown_connections 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.487+0000 7fbe2b915640 1 -- 172.21.15.106:0/658556619 wait complete. 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.488+0000 7fbe2b915640 1 Processor -- start 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.488+0000 7fbe2b915640 1 -- start start 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe240b1c30 0x7fbe240d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 0x7fbe24156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2406e100 con 0x7fbe240709e0 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2406df80 con 0x7fbe240b1c30 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe2b915640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2406e280 con 0x7fbe2415ee40 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49402/0 (socket says 172.21.15.106:49402) 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 -- 172.21.15.106:0/988481473 learned_addr learned my addr 172.21.15.106:0/988481473 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe240b1c30 0x7fbe240d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 0x7fbe24156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 -- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 msgr2=0x7fbe24156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 0x7fbe24156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 -- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe240b1c30 msgr2=0x7fbe240d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe240b1c30 0x7fbe240d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe296b1640 1 -- 172.21.15.106:0/988481473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe241f8210 con 0x7fbe240709e0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.489+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 0x7fbe24156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe296b1640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c800 tx=0x7fbe2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe19ffb640 1 -- 172.21.15.106:0/988481473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20003b70 con 0x7fbe240709e0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe19ffb640 1 -- 172.21.15.106:0/988481473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20003d10 con 0x7fbe240709e0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe241f84f0 con 0x7fbe240709e0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe19ffb640 1 -- 172.21.15.106:0/988481473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000aa90 con 0x7fbe240709e0 2024-04-13T03:16:45.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.490+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2415faa0 con 0x7fbe240709e0 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.491+0000 7fbe19ffb640 1 -- 172.21.15.106:0/988481473 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe2000ac30 con 0x7fbe240709e0 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.492+0000 7fbe19ffb640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 0x7fbe0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.492+0000 7fbe19ffb640 1 -- 172.21.15.106:0/988481473 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fbe2008be00 con 0x7fbe240709e0 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.492+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 0x7fbe0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.492+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 0x7fbe0806edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe14035000 tx=0x7fbe14008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.493+0000 7fbe2e7d1740 1 Processor -- start 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.493+0000 7fbe2e7d1740 1 -- start start 2024-04-13T03:16:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e081c0 0x556d19e0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19ddb2c0 0x556d19e0b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556d19d45990 con 0x556d19e07de0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556d19cec620 con 0x556d19ddb2c0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556d19d41950 con 0x556d19e081c0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49412/0 (socket says 172.21.15.106:49412) 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 -- 172.21.15.106:0/3274416218 learned_addr learned my addr 172.21.15.106:0/3274416218 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19ddb2c0 0x556d19e0b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e081c0 0x556d19e0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 -- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e081c0 msgr2=0x556d19e0ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e081c0 0x556d19e0ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 -- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19ddb2c0 msgr2=0x556d19e0b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19ddb2c0 0x556d19e0b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 -- 172.21.15.106:0/3274416218 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d19a582d0 con 0x556d19e07de0 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe296b1640 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbe2008a020 tx=0x7fbe200599f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4804f11b5b7036 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe0effd640 1 -- 172.21.15.106:0/3274416218 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe200600c0 con 0x556d19e07de0 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe0effd640 1 -- 172.21.15.106:0/3274416218 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20057c30 con 0x556d19e07de0 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.494+0000 7fbe0effd640 1 -- 172.21.15.106:0/3274416218 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2005abb0 con 0x556d19e07de0 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/3274416218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 msgr2=0x556d19ddad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbe2008a020 tx=0x7fbe200599f0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/3274416218 shutdown_connections 2024-04-13T03:16:45.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e081c0 0x556d19e0ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19ddb2c0 0x556d19e0b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/3274416218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e07de0 0x556d19ddad80 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/3274416218 >> 172.21.15.106:0/3274416218 conn(0x556d19ddcff0 msgr2=0x556d19e25b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/3274416218 shutdown_connections 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/3274416218 wait complete. 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 -- start start 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.495+0000 7fbe2e7d1740 1 Processor -- start 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19df60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19df65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556d19ddf590 con 0x556d19e0c180 2024-04-13T03:16:45.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556d19cff9e0 con 0x556d19e081c0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556d19dde980 con 0x556d19e07de0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe2e7d1740 10 client.0 _ll_register_callbacks cb 0x556d19d3b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.496+0000 7fbe29eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49424/0 (socket says 172.21.15.106:49424) 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2438931965 learned_addr learned my addr 172.21.15.106:0/2438931965 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19df65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19df60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 msgr2=0x556d19df60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19df60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 msgr2=0x556d19df65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19df65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2438931965 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d19b226b0 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19df60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00f4b0 tx=0x7fbe1c00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3015a78cfb284b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c005ba0 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fbe1c005d40 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe20057c30 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c020050 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe2e7d1740 10 client.8094 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x556d19df7690 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20057c30 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7fbe1c023020 con 0x556d19e0c180 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.497+0000 7fbe2e7d1740 10 client.8094 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:45.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe2e7d1740 10 client.8094 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe2e7d1740 20 client.8094 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e1cf50 0x556d19e09430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe2e7d1740 4 client.8094 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d19d533b0 con 0x556d19e1cf50 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e1cf50 0x556d19e09430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe03fff640 10 client.8094 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e1cf50 0x556d19e09430 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11cf4148347ce4a server_cookie=3f9aa4e1851413df in_seq=0 out_seq=0 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe03fff640 1 -- 172.21.15.106:0/2438931965 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d19d533b0 con 0x556d19e1cf50 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.498+0000 7fbe03fff640 10 client.8094 handle_command_reply: tid=0 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 client.8094 shutdown 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 msgr2=0x556d19df7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00f4b0 tx=0x7fbe1c00f970 comp rx=0 tx=0).stop 2024-04-13T03:16:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 shutdown_connections 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19df7e90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19df65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e1cf50 0x556d19e09430 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2438931965 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19df60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe03fff640 0 client.8094 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 >> 172.21.15.106:0/2438931965 conn(0x556d19ddb2c0 msgr2=0x556d19e29080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 shutdown_connections 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2438931965 wait complete. 2024-04-13T03:16:45.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.499+0000 7fbe2e7d1740 20 client.8094 trim_cache size 0 max 16384 2024-04-13T03:16:45.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.517+0000 7fbe2e7d1740 1 Processor -- start 2024-04-13T03:16:45.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.517+0000 7fbe2e7d1740 1 -- start start 2024-04-13T03:16:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e07de0 0x556d19e49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e0c180 0x556d19df6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24200600 con 0x556d19e081c0 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24161420 con 0x556d19e07de0 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbe241fff30 con 0x556d19e0c180 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49432/0 (socket says 172.21.15.106:49432) 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 -- 172.21.15.106:0/1513992280 learned_addr learned my addr 172.21.15.106:0/1513992280 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe296b1640 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e07de0 0x556d19e49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e0c180 0x556d19df6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 -- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e0c180 msgr2=0x556d19df6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e0c180 0x556d19df6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 -- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e07de0 msgr2=0x556d19e49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e07de0 0x556d19e49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 -- 172.21.15.106:0/1513992280 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d19e0f0c0 con 0x556d19e081c0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098c0 tx=0x7fbe14030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530845203bc4f8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe0e7fc640 1 -- 172.21.15.106:0/1513992280 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14038040 con 0x556d19e081c0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe0e7fc640 1 -- 172.21.15.106:0/1513992280 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1403eb20 con 0x556d19e081c0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.518+0000 7fbe0e7fc640 1 -- 172.21.15.106:0/1513992280 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1404d460 con 0x556d19e081c0 2024-04-13T03:16:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/1513992280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 msgr2=0x556d19e356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098c0 tx=0x7fbe14030000 comp rx=0 tx=0).stop 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/1513992280 shutdown_connections 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e0c180 0x556d19df6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e081c0 0x556d19e356a0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/1513992280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e07de0 0x556d19e49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/1513992280 >> 172.21.15.106:0/1513992280 conn(0x556d19ddb2c0 msgr2=0x556d19e0b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/1513992280 shutdown_connections 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/1513992280 wait complete. 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 -- start start 2024-04-13T03:16:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.519+0000 7fbe2e7d1740 1 Processor -- start 2024-04-13T03:16:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19ea03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19e34ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556d19e1cdd0 con 0x556d19e0c180 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556d19e0c6a0 con 0x556d19e081c0 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x556d19e1d3c0 con 0x556d19e07de0 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe2e7d1740 10 client.0 _ll_register_callbacks cb 0x7fbe1c0015d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe29eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.520+0000 7fbe29eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49440/0 (socket says 172.21.15.106:49440) 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2188247867 learned_addr learned my addr 172.21.15.106:0/2188247867 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe28eb0640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19e34ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19ea03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 msgr2=0x556d19ea03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19ea03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 msgr2=0x556d19e34ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19e34ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 -- 172.21.15.106:0/2188247867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d19df5910 con 0x556d19e0c180 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19ea03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe29eb2640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000ea0 tx=0x7fbe1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8d3df7338a4f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00f260 con 0x556d19e0c180 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fbe1c029b80 con 0x556d19e0c180 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1403eb20 con 0x556d19e0c180 2024-04-13T03:16:45.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c034050 con 0x556d19e0c180 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 10 client.8100 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x556d19ea0920 con 0x556d19e0c180 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1403eb20 con 0x556d19e0c180 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7fbe1c037020 con 0x556d19e0c180 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 10 client.8100 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 10 client.8100 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 20 client.8100 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e48b90 0x556d19e48f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 4 client.8100 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.521+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "status"}) -- 0x556d19d4d220 con 0x556d19e48b90 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e48b90 0x556d19e48f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe03fff640 10 client.8100 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe296b1640 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e48b90 0x556d19e48f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3e2499b6b430cc server_cookie=55bbdc3eea0345c0 in_seq=0 out_seq=0 2024-04-13T03:16:45.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe03fff640 1 -- 172.21.15.106:0/2188247867 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x556d19d4d220 con 0x556d19e48b90 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe03fff640 10 client.8100 handle_command_reply: tid=0 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 client.8100 shutdown 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 msgr2=0x556d19dde8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000ea0 tx=0x7fbe1c0057f0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 shutdown_connections 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556d19e0c180 0x556d19dde8e0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d19e081c0 0x556d19e34ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x556d19e48b90 0x556d19e48f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 --2- 172.21.15.106:0/2188247867 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x556d19e07de0 0x556d19ea03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe03fff640 0 client.8100 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 >> 172.21.15.106:0/2188247867 conn(0x556d19ddb2c0 msgr2=0x556d19e0b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 shutdown_connections 2024-04-13T03:16:45.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 1 -- 172.21.15.106:0/2188247867 wait complete. 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.522+0000 7fbe2e7d1740 20 client.8100 trim_cache size 0 max 16384 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 8028, 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:16:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.404570681, 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 64, 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 95, 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 95, 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 5.3780078629999997 2024-04-13T03:16:45.533 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:45.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 msgr2=0x7fbe0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 0x7fbe0806edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe14035000 tx=0x7fbe14008030 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 msgr2=0x7fbe241f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c800 tx=0x7fbe2000ccc0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 shutdown_connections 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbe0806c8f0 0x7fbe0806edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbe2415ee40 0x7fbe24156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe240b1c30 0x7fbe240d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 --2- 172.21.15.106:0/988481473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe240709e0 0x7fbe241f7910 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 >> 172.21.15.106:0/988481473 conn(0x7fbe24071870 msgr2=0x7fbe2415e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 shutdown_connections 2024-04-13T03:16:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.523+0000 7fbe2b915640 1 -- 172.21.15.106:0/988481473 wait complete. 2024-04-13T03:16:45.536 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:16:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.635+0000 7f8dfa3dc640 1 Processor -- start 2024-04-13T03:16:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.635+0000 7f8dfa3dc640 1 -- start start 2024-04-13T03:16:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8dfa3dc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df4145a10 0x7f8df40b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8dfa3dc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df4145fd0 con 0x7f8df40cf2c0 2024-04-13T03:16:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8dfa3dc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df406ba10 con 0x7f8df4145590 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8dfa3dc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8df41462b0 con 0x7f8df4145a10 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df4145a10 0x7f8df40b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df8979640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8df40cf2c0 0x7f8df40cf6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35132/0 (socket says 172.21.15.106:35132) 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df4145a10 0x7f8df40b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49448/0 (socket says 172.21.15.106:49448) 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df8979640 1 -- 172.21.15.106:0/4232277768 learned_addr learned my addr 172.21.15.106:0/4232277768 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df37fe640 1 -- 172.21.15.106:0/4232277768 >> v1:172.21.15.114:6789/0 conn(0x7f8df4145590 legacy=0x7f8df40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df37fe640 1 -- 172.21.15.106:0/4232277768 >> v1:172.21.15.106:6789/0 conn(0x7f8df40cf2c0 legacy=0x7f8df40cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df2ffd640 1 -- 172.21.15.106:0/4232277768 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517909629 0 0) 0x7f8df4145fd0 con 0x7f8df40cf2c0 2024-04-13T03:16:45.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.636+0000 7f8df37fe640 1 -- 172.21.15.106:0/4232277768 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df406bcf0 con 0x7f8df4145a10 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.637+0000 7f8df37fe640 1 --2- 172.21.15.106:0/4232277768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df4145a10 0x7f8df40b3ad0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8de80097a0 tx=0x7f8de802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9a239642c61823 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.637+0000 7f8df2ffd640 1 -- 172.21.15.106:0/4232277768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80315a0 con 0x7f8df4145a10 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.637+0000 7f8df2ffd640 1 -- 172.21.15.106:0/4232277768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de8037070 con 0x7f8df4145a10 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.637+0000 7f8df2ffd640 1 -- 172.21.15.106:0/4232277768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8031960 con 0x7f8df4145a10 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/4232277768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df4145a10 msgr2=0x7f8df40b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/4232277768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df4145a10 0x7f8df40b3ad0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8de80097a0 tx=0x7f8de802b840 comp rx=0 tx=0).stop 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/4232277768 shutdown_connections 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/4232277768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df4145a10 0x7f8df40b3ad0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/4232277768 >> 172.21.15.106:0/4232277768 conn(0x7f8df40d0340 msgr2=0x7f8df40d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/4232277768 shutdown_connections 2024-04-13T03:16:45.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.638+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/4232277768 wait complete. 2024-04-13T03:16:45.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.639+0000 7f8dfa3dc640 1 Processor -- start 2024-04-13T03:16:45.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.639+0000 7f8dfa3dc640 1 -- start start 2024-04-13T03:16:45.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 0x7f8df41f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8df4145a10 0x7f8df40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8df406c670 con 0x7f8df40cf2c0 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8df406c4f0 con 0x7f8df4145a10 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8dfa3dc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8df406c7f0 con 0x7f8df4145590 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49452/0 (socket says 172.21.15.106:49452) 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 -- 172.21.15.106:0/1760936456 learned_addr learned my addr 172.21.15.106:0/1760936456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df8979640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8df4145a10 0x7f8df40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df37fe640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 0x7f8df41f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 -- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 msgr2=0x7f8df41f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 0x7f8df41f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 -- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8df4145a10 msgr2=0x7f8df40b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8df4145a10 0x7f8df40b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df3fff640 1 -- 172.21.15.106:0/1760936456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df41f5860 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.640+0000 7f8df37fe640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 0x7f8df41f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8df3fff640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c430 tx=0x7f8de400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4011790 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de4017070 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df40aa060 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4011b70 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.641+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df40aa620 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.642+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df4068a10 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.642+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8de4011d10 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.642+0000 7f8df0ff9640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 0x7f8de006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.642+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f8de408bb20 con 0x7f8df40cf2c0 2024-04-13T03:16:45.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.643+0000 7f8df37fe640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 0x7f8de006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:45.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.643+0000 7f8df37fe640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 0x7f8de006ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8de8036000 tx=0x7f8de803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:45.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.647+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8de4059d30 con 0x7f8df40cf2c0 2024-04-13T03:16:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.859+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8df40b0bd0 con 0x7f8df40cf2c0 2024-04-13T03:16:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.860+0000 7f8df0ff9640 1 -- 172.21.15.106:0/1760936456 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f8de4059630 con 0x7f8df40cf2c0 2024-04-13T03:16:45.861 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:16:45.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 msgr2=0x7f8de006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 0x7f8de006ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8de8036000 tx=0x7f8de803d040 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 msgr2=0x7f8df415a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c430 tx=0x7f8de400c8f0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 shutdown_connections 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8de006c920 0x7f8de006ede0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8df4145590 0x7f8df41f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8df4145a10 0x7f8df40b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 --2- 172.21.15.106:0/1760936456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df40cf2c0 0x7f8df415a960 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.861+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 >> 172.21.15.106:0/1760936456 conn(0x7f8df40d0340 msgr2=0x7f8df413ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.862+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 shutdown_connections 2024-04-13T03:16:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:45.862+0000 7f8dfa3dc640 1 -- 172.21.15.106:0/1760936456 wait complete. 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:16:45.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:16:45.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:16:45.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:45.874 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:16:45.889 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:16:45.890 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42143sec preferred_lft 42143sec 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:16:45.891 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:16:45.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:45.892 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:16:45.955 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:16:45.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:45.957 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:16:46.013 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:16:46.013 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:16:46.013 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:16:46.013 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:16:46.014 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:16:46.014 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:16:46.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:16:46.119 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:16:46.119+0000 7fa0ec7f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:16:46.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:16:46.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.149 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:16:46.163 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:16:46.163 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:16:46.164 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:16:46.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.164 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:16:46.230 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:16:46.230 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:16:46.285 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:16:46.285 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:16:46.285 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:16:46.340 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:16:46.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.340 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:16:46.425 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:16:46.425 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:16:46.433 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:16:46.434 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:16:46.434 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:16:46.434 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:16:46.434 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:16:46.436 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:16:46.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.436 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:16:46.462 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:16:46.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:16:46.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:16:46.557 INFO:teuthology.orchestra.run.smithi106.stdout:client7066 2024-04-13T03:16:46.558 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:16:46.558 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:16:46.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-13T03:16:46.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.717+0000 7f9fc6aad640 1 Processor -- start 2024-04-13T03:16:46.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.717+0000 7f9fc6aad640 1 -- start start 2024-04-13T03:16:46.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc6aad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc006bff0 0x7f9fc006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc6aad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fc00d07c0 0x7f9fc006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc6aad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc00cef80 con 0x7f9fc015fb50 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc6aad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc00cf260 con 0x7f9fc00d07c0 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc6aad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc00cf540 con 0x7f9fc006bff0 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fc00d07c0 0x7f9fc006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9fc00d07c0 0x7f9fc006ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49460/0 (socket says 172.21.15.106:49460) 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc4849640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9fc015fb50 0x7f9fc006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35142/0 (socket says 172.21.15.106:35142) 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 -- 172.21.15.106:0/692664205 learned_addr learned my addr 172.21.15.106:0/692664205 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fbffff640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc006bff0 0x7f9fc006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 -- 172.21.15.106:0/692664205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc006bff0 msgr2=0x7f9fc006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc006bff0 0x7f9fc006e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 -- 172.21.15.106:0/692664205 >> v1:172.21.15.106:6789/0 conn(0x7f9fc015fb50 legacy=0x7f9fc006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fbf7fe640 1 -- 172.21.15.106:0/692664205 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189747907 0 0) 0x7f9fc00cef80 con 0x7f9fc015fb50 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.718+0000 7f9fc504a640 1 -- 172.21.15.106:0/692664205 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc00cf820 con 0x7f9fc00d07c0 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc504a640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d07c0 0x7f9fc006ebf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009e50 tx=0x7f9fb802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f8621cc801f018 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fbf7fe640 1 -- 172.21.15.106:0/692664205 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb802ca70 con 0x7f9fc00d07c0 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fbf7fe640 1 -- 172.21.15.106:0/692664205 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb802cc10 con 0x7f9fc00d07c0 2024-04-13T03:16:46.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fbf7fe640 1 -- 172.21.15.106:0/692664205 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80343f0 con 0x7f9fc00d07c0 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 -- 172.21.15.106:0/692664205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d07c0 msgr2=0x7f9fc006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d07c0 0x7f9fc006ebf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009e50 tx=0x7f9fb802bed0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 -- 172.21.15.106:0/692664205 shutdown_connections 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc00d07c0 0x7f9fc006ebf0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/692664205 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc006bff0 0x7f9fc006e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.719+0000 7f9fc6aad640 1 -- 172.21.15.106:0/692664205 >> 172.21.15.106:0/692664205 conn(0x7f9fc00cd6a0 msgr2=0x7f9fc00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.720+0000 7f9fc6aad640 1 -- 172.21.15.106:0/692664205 shutdown_connections 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.720+0000 7f9fc6aad640 1 -- 172.21.15.106:0/692664205 wait complete. 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.721+0000 7f9fc6aad640 1 Processor -- start 2024-04-13T03:16:46.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.721+0000 7f9fc6aad640 1 -- start start 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.721+0000 7f9fc6aad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc006bff0 0x7f9fc014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.721+0000 7f9fc6aad640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00d07c0 0x7f9fc00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.721+0000 7f9fc6aad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc6aad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00ced20 con 0x7f9fc015fb50 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc6aad640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00ceba0 con 0x7f9fc006bff0 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc6aad640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc015ef10 con 0x7f9fc00d07c0 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49468/0 (socket says 172.21.15.106:49468) 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 -- 172.21.15.106:0/2654157345 learned_addr learned my addr 172.21.15.106:0/2654157345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 -- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00d07c0 msgr2=0x7f9fc00aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00d07c0 0x7f9fc00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 -- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc006bff0 msgr2=0x7f9fc014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc006bff0 0x7f9fc014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 -- 172.21.15.106:0/2654157345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc014eef0 con 0x7f9fc015fb50 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc504a640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8036040 tx=0x7f9fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fbd7fa640 1 -- 172.21.15.106:0/2654157345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8043020 con 0x7f9fc015fb50 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fbd7fa640 1 -- 172.21.15.106:0/2654157345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb803e450 con 0x7f9fc015fb50 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc014f190 con 0x7f9fc015fb50 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fbd7fa640 1 -- 172.21.15.106:0/2654157345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8009510 con 0x7f9fc015fb50 2024-04-13T03:16:46.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.722+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc014f750 con 0x7f9fc015fb50 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.724+0000 7f9fbd7fa640 1 -- 172.21.15.106:0/2654157345 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fb8046020 con 0x7f9fc015fb50 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.724+0000 7f9fbd7fa640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.724+0000 7f9fbd7fa640 1 -- 172.21.15.106:0/2654157345 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f9fb80ae5a0 con 0x7f9fc015fb50 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.724+0000 7f9fc4849640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.725+0000 7f9fc4849640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007fd0 tx=0x7f9fb0007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.725+0000 7f9fc996a740 1 Processor -- start 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.725+0000 7f9fc996a740 1 -- start start 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66ad40 0x55a7df691870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df66a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7df5cf0a0 con 0x55a7df691db0 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a7df575930 con 0x55a7df66ad40 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a7df5cb200 con 0x55a7df66b120 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49474/0 (socket says 172.21.15.106:49474) 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 -- 172.21.15.106:0/3940806647 learned_addr learned my addr 172.21.15.106:0/3940806647 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 -- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 msgr2=0x55a7df66a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df66a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 -- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66ad40 msgr2=0x55a7df691870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66ad40 0x55a7df691870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 -- 172.21.15.106:0/3940806647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7df3a97e0 con 0x55a7df691db0 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4007780 tx=0x7f9fb4007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629c69bd8b5436ac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fa67fc640 1 -- 172.21.15.106:0/3940806647 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400d8f0 con 0x55a7df691db0 2024-04-13T03:16:46.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fa67fc640 1 -- 172.21.15.106:0/3940806647 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4018070 con 0x55a7df691db0 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.726+0000 7f9fa67fc640 1 -- 172.21.15.106:0/3940806647 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400dcd0 con 0x55a7df691db0 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- 172.21.15.106:0/3940806647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 msgr2=0x55a7df669d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4007780 tx=0x7f9fb4007c40 comp rx=0 tx=0).stop 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- 172.21.15.106:0/3940806647 shutdown_connections 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df66a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df691db0 0x55a7df669d80 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3940806647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66ad40 0x55a7df691870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- 172.21.15.106:0/3940806647 >> 172.21.15.106:0/3940806647 conn(0x55a7df6316e0 msgr2=0x55a7df6afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- 172.21.15.106:0/3940806647 shutdown_connections 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- 172.21.15.106:0/3940806647 wait complete. 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.727+0000 7f9fc996a740 1 -- start start 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 Processor -- start 2024-04-13T03:16:46.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df6a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7df6913e0 con 0x55a7df66ad40 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a7df588cf0 con 0x55a7df6a7f50 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a7df68a160 con 0x55a7df66b120 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc996a740 10 client.0 _ll_register_callbacks cb 0x55a7df5886e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc4849640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.728+0000 7f9fc4849640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49478/0 (socket says 172.21.15.106:49478) 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 -- 172.21.15.106:0/2653357022 learned_addr learned my addr 172.21.15.106:0/2653357022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 -- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 msgr2=0x55a7df6a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df6a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 -- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 msgr2=0x55a7df6b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 -- 172.21.15.106:0/2653357022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7df409f70 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc4849640 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005bb0 tx=0x7f9fb0016450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9dabec94ccb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0016ac0 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f9fb0016c60 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4018070 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0016e60 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 10 client.8124 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55a7df2a1170 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4018070 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f9fb0012440 con 0x55a7df66ad40 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 10 client.8124 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 10 client.8124 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 20 client.8124 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.729+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df6c38d0 0x55a7df6c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9fc996a740 4 client.8124 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7df5dcac0 con 0x55a7df6c38d0 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9fbffff640 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df6c38d0 0x55a7df6c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9fbffff640 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df6c38d0 0x55a7df6c3d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05571cb51f11b8 server_cookie=662096deb2a128d1 in_seq=0 out_seq=0 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9f9f7fe640 10 client.8124 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/2653357022 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7df5dcac0 con 0x55a7df6c38d0 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9f9f7fe640 10 client.8124 handle_command_reply: tid=0 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.730+0000 7f9fc996a740 1 client.8124 shutdown 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 msgr2=0x55a7df6a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005bb0 tx=0x7f9fb0016450 comp rx=0 tx=0).stop 2024-04-13T03:16:46.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 shutdown_connections 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df6c38d0 0x55a7df6c3d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66b120 0x55a7df6a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9f9f7fe640 0 client.8124 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2653357022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6a7240 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 >> 172.21.15.106:0/2653357022 conn(0x55a7df6316e0 msgr2=0x55a7df663a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 shutdown_connections 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 1 -- 172.21.15.106:0/2653357022 wait complete. 2024-04-13T03:16:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.731+0000 7f9fc996a740 20 client.8124 trim_cache size 0 max 16384 2024-04-13T03:16:46.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.748+0000 7f9fc996a740 1 Processor -- start 2024-04-13T03:16:46.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.748+0000 7f9fc996a740 1 -- start start 2024-04-13T03:16:46.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66ad40 0x55a7df6d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0201830 con 0x55a7df66b120 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc02008a0 con 0x55a7df6a7f50 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc0151d90 con 0x55a7df66ad40 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49482/0 (socket says 172.21.15.106:49482) 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 -- 172.21.15.106:0/2614618494 learned_addr learned my addr 172.21.15.106:0/2614618494 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 -- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66ad40 msgr2=0x55a7df6d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66ad40 0x55a7df6d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 -- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 msgr2=0x55a7df6b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 -- 172.21.15.106:0/2614618494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7df663b30 con 0x55a7df66b120 2024-04-13T03:16:46.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fbffff640 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4015040 tx=0x7f9fb4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69aa907f53407657 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fa5ffb640 1 -- 172.21.15.106:0/2614618494 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4004400 con 0x55a7df66b120 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fa5ffb640 1 -- 172.21.15.106:0/2614618494 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4018030 con 0x55a7df66b120 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.749+0000 7f9fa5ffb640 1 -- 172.21.15.106:0/2614618494 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400db30 con 0x55a7df66b120 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 -- 172.21.15.106:0/2614618494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 msgr2=0x55a7df66a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4015040 tx=0x7f9fb4010000 comp rx=0 tx=0).stop 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 -- 172.21.15.106:0/2614618494 shutdown_connections 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df6a7f50 0x55a7df6b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66b120 0x55a7df66a300 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 --2- 172.21.15.106:0/2614618494 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df66ad40 0x55a7df6d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 -- 172.21.15.106:0/2614618494 >> 172.21.15.106:0/2614618494 conn(0x55a7df6316e0 msgr2=0x55a7df6a7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 -- 172.21.15.106:0/2614618494 shutdown_connections 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.750+0000 7f9fc996a740 1 -- 172.21.15.106:0/2614618494 wait complete. 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 -- start start 2024-04-13T03:16:46.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 Processor -- start 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 0x55a7df6a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 0x55a7df6a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7df6a7760 con 0x55a7df66ad40 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc02002a0 con 0x55a7df66b120 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.751+0000 7f9fc996a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a7df589680 con 0x55a7df6a7f50 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc996a740 10 client.0 _ll_register_callbacks cb 0x7f9fb0001220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49488/0 (socket says 172.21.15.106:49488) 2024-04-13T03:16:46.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 -- 172.21.15.106:0/3733492502 learned_addr learned my addr 172.21.15.106:0/3733492502 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc504a640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 0x55a7df6a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 0x55a7df6a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 -- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 msgr2=0x55a7df6a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 0x55a7df6a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 -- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 msgr2=0x55a7df6a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 0x55a7df6a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 -- 172.21.15.106:0/3733492502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7df668750 con 0x55a7df66ad40 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 0x55a7df6a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc504a640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 0x55a7df6a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc4849640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fd0 tx=0x7f9fb0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dece38a9eeaaa00 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb002f030 con 0x55a7df66ad40 2024-04-13T03:16:46.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4018030 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f9fb001c040 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0000f40 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc996a740 10 client.8130 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55a7df7124c0 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.752+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4018030 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f9fb003b080 con 0x55a7df66ad40 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 10 client.8130 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 10 client.8130 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 20 client.8130 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df690000 0x55a7df7114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 4 client.8130 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-13T03:16:46.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55a7df6aca30 con 0x55a7df690000 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df690000 0x55a7df7114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9f9f7fe640 10 client.8130 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fbffff640 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df690000 0x55a7df7114f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57158caf3339d928 server_cookie=3f010e6140aad09c in_seq=0 out_seq=0 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9f9f7fe640 1 -- 172.21.15.106:0/3733492502 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a7df6aca30 con 0x55a7df690000 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9f9f7fe640 10 client.8130 handle_command_reply: tid=0 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.753+0000 7f9fc996a740 1 client.8130 shutdown 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 msgr2=0x55a7df6afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fd0 tx=0x7f9fb0007000 comp rx=0 tx=0).stop 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 shutdown_connections 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55a7df690000 0x55a7df7114f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a7df6a7f50 0x55a7df6a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a7df66b120 0x55a7df6a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 --2- 172.21.15.106:0/3733492502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df66ad40 0x55a7df6afd20 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 >> 172.21.15.106:0/3733492502 conn(0x55a7df6316e0 msgr2=0x55a7df663fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9f9f7fe640 0 client.8130 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 shutdown_connections 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 1 -- 172.21.15.106:0/3733492502 wait complete. 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc996a740 20 client.8130 trim_cache size 0 max 16384 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "11cecac7-efe3-4a85-9cc2-7891850a137d", 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:16:46.762 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:46.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 msgr2=0x7f9fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007fd0 tx=0x7f9fb0007960 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 msgr2=0x7f9fc014d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.754+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8036040 tx=0x7f9fb8004a40 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 shutdown_connections 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9fac06c8f0 0x7f9fac06edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fc00d07c0 0x7f9fc00aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fc015fb50 0x7f9fc014d160 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 --2- 172.21.15.106:0/2654157345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fc006bff0 0x7f9fc014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 >> 172.21.15.106:0/2654157345 conn(0x7f9fc00cd6a0 msgr2=0x7f9fc00b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 shutdown_connections 2024-04-13T03:16:46.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.755+0000 7f9fc6aad640 1 -- 172.21.15.106:0/2654157345 wait complete. 2024-04-13T03:16:46.765 INFO:tasks.ceph.mds.a.smithi106.stderr:2024-04-13T03:16:46.756+0000 7fa0e87e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x563c2ca8eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:16:46.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:16:46.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:16:46.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.868+0000 7f7763388640 1 Processor -- start 2024-04-13T03:16:46.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.868+0000 7f7763388640 1 -- start start 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7763388640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 0x7f775c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7763388640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f775c073520 0x7f775c0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7763388640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775c0b5060 con 0x7f775c15eca0 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7763388640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f775c0b5340 con 0x7f775c073520 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7763388640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f775c06c500 con 0x7f775c15e8c0 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f775c073520 0x7f775c0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f775c073520 0x7f775c0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49502/0 (socket says 172.21.15.106:49502) 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 -- 172.21.15.106:0/87450072 learned_addr learned my addr 172.21.15.106:0/87450072 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 -- 172.21.15.106:0/87450072 >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 msgr2=0x7f775c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761124640 1 --2- 172.21.15.106:0/87450072 >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 0x7f775c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 --2- 172.21.15.106:0/87450072 >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 0x7f775c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 -- 172.21.15.106:0/87450072 >> v1:172.21.15.114:6789/0 conn(0x7f775c15eca0 legacy=0x7f775c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761925640 1 -- 172.21.15.106:0/87450072 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c06c7e0 con 0x7f775c073520 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.869+0000 7f7761124640 1 --2- 172.21.15.106:0/87450072 >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 0x7f775c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.870+0000 7f7761925640 1 --2- 172.21.15.106:0/87450072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c073520 0x7f775c0b4430 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f775c06bb40 tx=0x7f775402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82da81f023a9b124 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.870+0000 7f7753fff640 1 -- 172.21.15.106:0/87450072 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775402cb20 con 0x7f775c073520 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.870+0000 7f7753fff640 1 -- 172.21.15.106:0/87450072 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775402ccc0 con 0x7f775c073520 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.870+0000 7f7753fff640 1 -- 172.21.15.106:0/87450072 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754034610 con 0x7f775c073520 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 -- 172.21.15.106:0/87450072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c073520 msgr2=0x7f775c0b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 --2- 172.21.15.106:0/87450072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c073520 0x7f775c0b4430 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f775c06bb40 tx=0x7f775402bf80 comp rx=0 tx=0).stop 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 -- 172.21.15.106:0/87450072 shutdown_connections 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 --2- 172.21.15.106:0/87450072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c073520 0x7f775c0b4430 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 --2- 172.21.15.106:0/87450072 >> v2:172.21.15.114:3300/0 conn(0x7f775c15e8c0 0x7f775c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 -- 172.21.15.106:0/87450072 >> 172.21.15.106:0/87450072 conn(0x7f775c06f0d0 msgr2=0x7f775c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 -- 172.21.15.106:0/87450072 shutdown_connections 2024-04-13T03:16:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.871+0000 7f7763388640 1 -- 172.21.15.106:0/87450072 wait complete. 2024-04-13T03:16:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.872+0000 7f7763388640 1 Processor -- start 2024-04-13T03:16:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.872+0000 7f7763388640 1 -- start start 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f775c073520 0x7f775c160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f775c15eca0 0x7f775c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0b4d40 con 0x7f775c15e8c0 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0b4bc0 con 0x7f775c073520 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f775c06cd00 con 0x7f775c15eca0 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49516/0 (socket says 172.21.15.106:49516) 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 -- 172.21.15.106:0/2209956009 learned_addr learned my addr 172.21.15.106:0/2209956009 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7761124640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f775c073520 0x7f775c160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 -- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f775c15eca0 msgr2=0x7f775c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f775c15eca0 0x7f775c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 -- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f775c073520 msgr2=0x7f775c160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f775c073520 0x7f775c160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 -- 172.21.15.106:0/2209956009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c161310 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7760923640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b3f0 tx=0x7f774c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7751ffb640 1 -- 172.21.15.106:0/2209956009 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c015020 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7751ffb640 1 -- 172.21.15.106:0/2209956009 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c011830 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c1615f0 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.873+0000 7f7751ffb640 1 -- 172.21.15.106:0/2209956009 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c011c20 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.874+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c161bb0 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.875+0000 7f7751ffb640 1 -- 172.21.15.106:0/2209956009 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f774c018020 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.875+0000 7f7751ffb640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 0x7f774006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.875+0000 7f7751ffb640 1 -- 172.21.15.106:0/2209956009 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f774c08b930 con 0x7f775c15e8c0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.875+0000 7f7761124640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 0x7f774006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.876+0000 7f7761124640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 0x7f774006ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7758008850 tx=0x7f7758008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.876+0000 7f776616e740 1 Processor -- start 2024-04-13T03:16:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.876+0000 7f776616e740 1 -- start start 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc17df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1ab3d0 0x5648bc1add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648bc0e9170 con 0x5648bc17e4d0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5648bc08f7f0 con 0x5648bc1aaff0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5648bc0e5130 con 0x5648bc1ab3d0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49520/0 (socket says 172.21.15.106:49520) 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 -- 172.21.15.106:0/293682415 learned_addr learned my addr 172.21.15.106:0/293682415 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761124640 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc17df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7760923640 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1ab3d0 0x5648bc1add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 -- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1ab3d0 msgr2=0x5648bc1add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1ab3d0 0x5648bc1add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 -- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 msgr2=0x5648bc17df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc17df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 -- 172.21.15.106:0/293682415 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bbec57d0 con 0x5648bc17e4d0 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.877+0000 7f7761925640 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f775402c590 tx=0x7f775402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a804f3879a0f7534 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f7746ffd640 1 -- 172.21.15.106:0/293682415 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754044070 con 0x5648bc17e4d0 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f7746ffd640 1 -- 172.21.15.106:0/293682415 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775402c920 con 0x5648bc17e4d0 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f7746ffd640 1 -- 172.21.15.106:0/293682415 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775402ccf0 con 0x5648bc17e4d0 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 -- 172.21.15.106:0/293682415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 msgr2=0x5648bc1ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f775402c590 tx=0x7f775402e070 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 -- 172.21.15.106:0/293682415 shutdown_connections 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1ab3d0 0x5648bc1add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc17e4d0 0x5648bc1ae2c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 --2- 172.21.15.106:0/293682415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc17df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 -- 172.21.15.106:0/293682415 >> 172.21.15.106:0/293682415 conn(0x5648bc180200 msgr2=0x5648bc1c8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 -- 172.21.15.106:0/293682415 shutdown_connections 2024-04-13T03:16:46.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.878+0000 7f776616e740 1 -- 172.21.15.106:0/293682415 wait complete. 2024-04-13T03:16:46.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.879+0000 7f776616e740 1 -- start start 2024-04-13T03:16:46.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.879+0000 7f776616e740 1 Processor -- start 2024-04-13T03:16:46.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648bc1828a0 con 0x5648bc1ab3d0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5648bc0a2d00 con 0x5648bc1aaff0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5648bc1850e0 con 0x5648bc1a3d90 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 10 client.0 _ll_register_callbacks cb 0x5648bc0a26f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49522/0 (socket says 172.21.15.106:49522) 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 -- 172.21.15.106:0/3974025304 learned_addr learned my addr 172.21.15.106:0/3974025304 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7761124640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7761925640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 -- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 msgr2=0x5648bc1a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 -- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 msgr2=0x5648bc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 -- 172.21.15.106:0/3974025304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bbdfb3f0 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f7760923640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f774c0593d0 tx=0x7f774c05d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cf06ccec534190 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f773bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c06fe60 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f774c058890 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775402c920 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.880+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f774c058a90 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 10 client.8142 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5648bbdbb0f0 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775402c920 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f774c055190 con 0x5648bc1ab3d0 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 10 client.8142 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 10 client.8142 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 20 client.8142 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:46.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1dca00 0x5648bc1dcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 4 client.8142 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648bc0f6b90 con 0x5648bc1dca00 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f7761124640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1dca00 0x5648bc1dcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f773bfff640 10 client.8142 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.881+0000 7f7761124640 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1dca00 0x5648bc1dcde0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a6a862f8585ce2 server_cookie=59412df6275d8d5 in_seq=0 out_seq=0 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f773bfff640 1 -- 172.21.15.106:0/3974025304 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648bc0f6b90 con 0x5648bc1dca00 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f773bfff640 10 client.8142 handle_command_reply: tid=0 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 client.8142 shutdown 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 msgr2=0x5648bc195460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f774c0593d0 tx=0x7f774c05d930 comp rx=0 tx=0).stop 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 shutdown_connections 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1dca00 0x5648bc1dcde0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f773bfff640 0 client.8142 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc195460 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 --2- 172.21.15.106:0/3974025304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 >> 172.21.15.106:0/3974025304 conn(0x5648bc17e4d0 msgr2=0x5648bc181fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 shutdown_connections 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 1 -- 172.21.15.106:0/3974025304 wait complete. 2024-04-13T03:16:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.882+0000 7f776616e740 20 client.8142 trim_cache size 0 max 16384 2024-04-13T03:16:46.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.899+0000 7f776616e740 1 Processor -- start 2024-04-13T03:16:46.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.899+0000 7f776616e740 1 -- start start 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1a3d90 0x5648bc240150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1aaff0 0x5648bc1db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f775c201790 con 0x5648bc1ab3d0 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f775c2007a0 con 0x5648bc1a3d90 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f775c159a70 con 0x5648bc1aaff0 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49538/0 (socket says 172.21.15.106:49538) 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 -- 172.21.15.106:0/2470571960 learned_addr learned my addr 172.21.15.106:0/2470571960 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761124640 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1a3d90 0x5648bc240150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 -- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1aaff0 msgr2=0x5648bc1db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1aaff0 0x5648bc1db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 -- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1a3d90 msgr2=0x5648bc240150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.900+0000 7f7761925640 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1a3d90 0x5648bc240150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f7761925640 1 -- 172.21.15.106:0/2470571960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bc17fee0 con 0x5648bc1ab3d0 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f7761925640 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f775403e000 tx=0x7f7754030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975224a8468338f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f77467fc640 1 -- 172.21.15.106:0/2470571960 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754044040 con 0x5648bc1ab3d0 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f77467fc640 1 -- 172.21.15.106:0/2470571960 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775404e070 con 0x5648bc1ab3d0 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f77467fc640 1 -- 172.21.15.106:0/2470571960 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775402cda0 con 0x5648bc1ab3d0 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 -- 172.21.15.106:0/2470571960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 msgr2=0x5648bc181c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f775403e000 tx=0x7f7754030000 comp rx=0 tx=0).stop 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 -- 172.21.15.106:0/2470571960 shutdown_connections 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1aaff0 0x5648bc1db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc181c90 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 --2- 172.21.15.106:0/2470571960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1a3d90 0x5648bc240150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 -- 172.21.15.106:0/2470571960 >> 172.21.15.106:0/2470571960 conn(0x5648bc17e4d0 msgr2=0x5648bc1ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 -- 172.21.15.106:0/2470571960 shutdown_connections 2024-04-13T03:16:46.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.901+0000 7f776616e740 1 -- 172.21.15.106:0/2470571960 wait complete. 2024-04-13T03:16:46.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.902+0000 7f776616e740 1 -- start start 2024-04-13T03:16:46.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.902+0000 7f776616e740 1 Processor -- start 2024-04-13T03:16:46.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f775c200170 con 0x5648bc1ab3d0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5648bc185360 con 0x5648bc1aaff0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5648bc1ac6f0 con 0x5648bc1a3d90 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f776616e740 10 client.0 _ll_register_callbacks cb 0x7f774c059440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49548/0 (socket says 172.21.15.106:49548) 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 -- 172.21.15.106:0/3237836070 learned_addr learned my addr 172.21.15.106:0/3237836070 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761124640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7760923640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 -- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 msgr2=0x5648bc1abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 -- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 msgr2=0x5648bc1a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 -- 172.21.15.106:0/3237836070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bc1aa650 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f7761925640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7754009f10 tx=0x7f7754004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789130e025fcc650 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.903+0000 7f773bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754057060 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f7754049620 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775404e070 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7754044070 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 10 client.8148 fetch_fsmap learned FSMap version 64 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5648bc23f420 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775404e070 con 0x5648bc1ab3d0 2024-04-13T03:16:46.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f7754060080 con 0x5648bc1ab3d0 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 10 client.8148 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 10 client.8148 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 20 client.8148 populate_metadata read hostname 'smithi106' 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1c8760 0x5648bc1c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 4 client.8148 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648bc1a5580 con 0x5648bc1c8760 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f7761124640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1c8760 0x5648bc1c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f773bfff640 10 client.8148 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.904+0000 7f7761124640 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1c8760 0x5648bc1c8b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9210c48b4d6e9432 server_cookie=92db8d60ccd3aed in_seq=0 out_seq=0 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f773bfff640 1 -- 172.21.15.106:0/3237836070 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5648bc1a5580 con 0x5648bc1c8760 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f773bfff640 10 client.8148 handle_command_reply: tid=0 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 client.8148 shutdown 2024-04-13T03:16:46.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 msgr2=0x5648bc1a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7754009f10 tx=0x7f7754004ab0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 shutdown_connections 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648bc1ab3d0 0x5648bc1a46f0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648bc1aaff0 0x5648bc1a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x5648bc1c8760 0x5648bc1c8b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 --2- 172.21.15.106:0/3237836070 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5648bc1a3d90 0x5648bc1abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 >> 172.21.15.106:0/3237836070 conn(0x5648bc17e4d0 msgr2=0x5648bc1ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f773bfff640 0 client.8148 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 shutdown_connections 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 1 -- 172.21.15.106:0/3237836070 wait complete. 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.905+0000 7f776616e740 20 client.8148 trim_cache size 0 max 16384 2024-04-13T03:16:46.912 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "11cecac7-efe3-4a85-9cc2-7891850a137d": { 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/dir", 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "11cecac7-efe3-4a85-9cc2-7891850a137d", 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:16:46.913 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:16:46.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 msgr2=0x7f774006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 0x7f774006ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7758008850 tx=0x7f7758008040 comp rx=0 tx=0).stop 2024-04-13T03:16:46.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 msgr2=0x7f775c146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:16:46.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b3f0 tx=0x7f774c00b8b0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 shutdown_connections 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f774006c920 0x7f774006ede0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f775c15eca0 0x7f775c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f775c15e8c0 0x7f775c146e70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 --2- 172.21.15.106:0/2209956009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f775c073520 0x7f775c160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 >> 172.21.15.106:0/2209956009 conn(0x7f775c06f0d0 msgr2=0x7f775c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 shutdown_connections 2024-04-13T03:16:46.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:16:46.906+0000 7f7763388640 1 -- 172.21.15.106:0/2209956009 wait complete. 2024-04-13T03:16:46.917 INFO:tasks.cephfs.filesystem:scrub status for tag:11cecac7-efe3-4a85-9cc2-7891850a137d - {'path': '/dir', 'tag': '11cecac7-efe3-4a85-9cc2-7891850a137d', 'options': 'recursive,repair'} 2024-04-13T03:16:55.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:16:55.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:17:16.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:16.918 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-13T03:17:17.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.032+0000 7f28f4077640 1 Processor -- start 2024-04-13T03:17:17.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.032+0000 7f28f4077640 1 -- start start 2024-04-13T03:17:17.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f4077640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 0x7f28ec06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f4077640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28ec0d07c0 0x7f28ec06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f4077640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ec0cef80 con 0x7f28ec15ef10 2024-04-13T03:17:17.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f4077640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f28ec0cf260 con 0x7f28ec0d07c0 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f4077640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f28ec0cf540 con 0x7f28ec15fb50 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1612640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28ec0d07c0 0x7f28ec06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f2614640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f28ec15ef10 0x7f28ec162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50134/0 (socket says 172.21.15.106:50134) 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f2614640 1 -- 172.21.15.106:0/384365411 learned_addr learned my addr 172.21.15.106:0/384365411 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1612640 1 -- 172.21.15.106:0/384365411 >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 msgr2=0x7f28ec06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1e13640 1 --2- 172.21.15.106:0/384365411 >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 0x7f28ec06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1612640 1 --2- 172.21.15.106:0/384365411 >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 0x7f28ec06b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1612640 1 -- 172.21.15.106:0/384365411 >> v1:172.21.15.106:6789/0 conn(0x7f28ec15ef10 legacy=0x7f28ec162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f0e11640 1 -- 172.21.15.106:0/384365411 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903617689 0 0) 0x7f28ec0cef80 con 0x7f28ec15ef10 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1612640 1 -- 172.21.15.106:0/384365411 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0cf820 con 0x7f28ec0d07c0 2024-04-13T03:17:17.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.033+0000 7f28f1e13640 1 --2- 172.21.15.106:0/384365411 >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 0x7f28ec06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.034+0000 7f28f1612640 1 --2- 172.21.15.106:0/384365411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec06be20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f28ec06e540 tx=0x7f28dc02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1e2d91700e15e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.034+0000 7f28f0e11640 1 -- 172.21.15.106:0/384365411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc038070 con 0x7f28ec0d07c0 2024-04-13T03:17:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.034+0000 7f28f0e11640 1 -- 172.21.15.106:0/384365411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc031710 con 0x7f28ec0d07c0 2024-04-13T03:17:17.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.034+0000 7f28f0e11640 1 -- 172.21.15.106:0/384365411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc031ae0 con 0x7f28ec0d07c0 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 -- 172.21.15.106:0/384365411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 msgr2=0x7f28ec06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 --2- 172.21.15.106:0/384365411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec06be20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f28ec06e540 tx=0x7f28dc02b7b0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 -- 172.21.15.106:0/384365411 shutdown_connections 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 --2- 172.21.15.106:0/384365411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec06be20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 --2- 172.21.15.106:0/384365411 >> v2:172.21.15.114:3300/0 conn(0x7f28ec15fb50 0x7f28ec06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 -- 172.21.15.106:0/384365411 >> 172.21.15.106:0/384365411 conn(0x7f28ec0cd6a0 msgr2=0x7f28ec0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 -- 172.21.15.106:0/384365411 shutdown_connections 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.035+0000 7f28f4077640 1 -- 172.21.15.106:0/384365411 wait complete. 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.036+0000 7f28f4077640 1 Processor -- start 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.036+0000 7f28f4077640 1 -- start start 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ec15ef10 0x7f28ec14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 0x7f28ec14cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0ced20 con 0x7f28ec0d07c0 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0ceba0 con 0x7f28ec15ef10 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f4077640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec06f6c0 con 0x7f28ec15fb50 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54058/0 (socket says 172.21.15.106:54058) 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 -- 172.21.15.106:0/235640224 learned_addr learned my addr 172.21.15.106:0/235640224 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1612640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ec15ef10 0x7f28ec14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f2614640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 0x7f28ec14cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 -- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 msgr2=0x7f28ec14cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 0x7f28ec14cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 -- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ec15ef10 msgr2=0x7f28ec14b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ec15ef10 0x7f28ec14b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 -- 172.21.15.106:0/235640224 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec14e640 con 0x7f28ec0d07c0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f2614640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 0x7f28ec14cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28f1e13640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f28e800c820 tx=0x7f28e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28e27fc640 1 -- 172.21.15.106:0/235640224 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8015070 con 0x7f28ec0d07c0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28e27fc640 1 -- 172.21.15.106:0/235640224 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e80079d0 con 0x7f28ec0d07c0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.037+0000 7f28e27fc640 1 -- 172.21.15.106:0/235640224 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8007d80 con 0x7f28ec0d07c0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.038+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec14e920 con 0x7f28ec0d07c0 2024-04-13T03:17:17.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.038+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec14db40 con 0x7f28ec0d07c0 2024-04-13T03:17:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.039+0000 7f28e27fc640 1 -- 172.21.15.106:0/235640224 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28e8018070 con 0x7f28ec0d07c0 2024-04-13T03:17:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.039+0000 7f28e27fc640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 0x7f28d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.040+0000 7f28e27fc640 1 -- 172.21.15.106:0/235640224 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f28e808bd00 con 0x7f28ec0d07c0 2024-04-13T03:17:17.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.040+0000 7f28f1612640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 0x7f28d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.040+0000 7f28f1612640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 0x7f28d006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f28dc035000 tx=0x7f28dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 Processor -- start 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 -- start start 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31c8ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cbfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563b31bfb170 con 0x563b31c904d0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563b31ba17f0 con 0x563b31cbcff0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563b31bf7130 con 0x563b31cbd3d0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.041+0000 7f28f2614640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54064/0 (socket says 172.21.15.106:54064) 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 -- 172.21.15.106:0/2801274073 learned_addr learned my addr 172.21.15.106:0/2801274073 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f1e13640 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31c8ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f1612640 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cbfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 -- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 msgr2=0x563b31cbfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cbfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 -- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 msgr2=0x563b31c8ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31c8ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 -- 172.21.15.106:0/2801274073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b319d77d0 con 0x563b31c904d0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28f2614640 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001f20 tx=0x7f28e400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33f7a82ec0d89fc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28cf7fe640 1 -- 172.21.15.106:0/2801274073 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004b60 con 0x563b31c904d0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28cf7fe640 1 -- 172.21.15.106:0/2801274073 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4004d00 con 0x563b31c904d0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.042+0000 7f28cf7fe640 1 -- 172.21.15.106:0/2801274073 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40072d0 con 0x563b31c904d0 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2801274073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 msgr2=0x563b31cc02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001f20 tx=0x7f28e400d520 comp rx=0 tx=0).stop 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2801274073 shutdown_connections 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cbfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31c904d0 0x563b31cc02c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2801274073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31c8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2801274073 >> 172.21.15.106:0/2801274073 conn(0x563b31c92200 msgr2=0x563b31cdad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2801274073 shutdown_connections 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2801274073 wait complete. 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 -- start start 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.043+0000 7f28f6e5d740 1 Processor -- start 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cb33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 0x563b31cb38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563b31c948a0 con 0x563b31cbcff0 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563b31bb4d00 con 0x563b31cb1fa0 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563b31c970e0 con 0x563b31cbd3d0 2024-04-13T03:17:17.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f6e5d740 10 client.0 _ll_register_callbacks cb 0x563b31bb46f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.044+0000 7f28f1e13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54080/0 (socket says 172.21.15.106:54080) 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 -- 172.21.15.106:0/3820100423 learned_addr learned my addr 172.21.15.106:0/3820100423 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f2614640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 0x563b31cb38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1612640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cb33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 -- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 msgr2=0x563b31cb33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cb33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 -- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 msgr2=0x563b31cb38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 0x563b31cb38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 -- 172.21.15.106:0/3820100423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b3190d3f0 con 0x563b31cbcff0 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f2614640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 0x563b31cb38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f1e13640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f28e8059680 tx=0x7f28e805db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaecb76b0a5cb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e801f040 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f28e80620d0 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e4004d00 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f28e8058b40 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f6e5d740 10 client.8163 fetch_fsmap learned FSMap version 64 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x563b318cd0f0 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4004d00 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f28e8058ce0 con 0x563b31cbcff0 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f6e5d740 10 client.8163 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.045+0000 7f28f6e5d740 10 client.8163 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f6e5d740 20 client.8163 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31cb1a30 0x563b31cdd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f6e5d740 4 client.8163 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:17:17.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b31c08b90 con 0x563b31cb1a30 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f1612640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31cb1a30 0x563b31cdd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28ccff9640 10 client.8163 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.046+0000 7f28f1612640 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31cb1a30 0x563b31cdd400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac75351fd41cb4b server_cookie=d836c23005362025 in_seq=0 out_seq=0 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28ccff9640 1 -- 172.21.15.106:0/3820100423 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b31c08b90 con 0x563b31cb1a30 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28ccff9640 10 client.8163 handle_command_reply: tid=0 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 client.8163 shutdown 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 msgr2=0x563b31cc2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f28e8059680 tx=0x7f28e805db60 comp rx=0 tx=0).stop 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 shutdown_connections 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31cb1a30 0x563b31cdd400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cbd3d0 0x563b31cb33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cb1fa0 0x563b31cb38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/3820100423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31cc2820 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28ccff9640 0 client.8163 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 >> 172.21.15.106:0/3820100423 conn(0x563b31c904d0 msgr2=0x563b31c93ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 shutdown_connections 2024-04-13T03:17:17.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 1 -- 172.21.15.106:0/3820100423 wait complete. 2024-04-13T03:17:17.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.047+0000 7f28f6e5d740 20 client.8163 trim_cache size 0 max 16384 2024-04-13T03:17:17.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.064+0000 7f28f6e5d740 1 Processor -- start 2024-04-13T03:17:17.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.064+0000 7f28f6e5d740 1 -- start start 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31cb3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbd3d0 0x563b31cda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec2016f0 con 0x563b31cbcff0 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec2006d0 con 0x563b31cbd3d0 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec150320 con 0x563b31cb1fa0 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f1612640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f1612640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54086/0 (socket says 172.21.15.106:54086) 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f1612640 1 -- 172.21.15.106:0/578528678 learned_addr learned my addr 172.21.15.106:0/578528678 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f1612640 1 -- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 msgr2=0x563b31cb3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.065+0000 7f28f1612640 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31cb3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f1612640 1 -- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbd3d0 msgr2=0x563b31cda760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f1612640 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbd3d0 0x563b31cda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f1612640 1 -- 172.21.15.106:0/578528678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b31bb09f0 con 0x563b31cbcff0 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f1612640 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02b7f0 tx=0x7f28dc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85dfcc56386ebe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28ceffd640 1 -- 172.21.15.106:0/578528678 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc038040 con 0x563b31cbcff0 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28ceffd640 1 -- 172.21.15.106:0/578528678 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc056070 con 0x563b31cbcff0 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28ceffd640 1 -- 172.21.15.106:0/578528678 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc003a80 con 0x563b31cbcff0 2024-04-13T03:17:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 -- 172.21.15.106:0/578528678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 msgr2=0x563b31c8f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02b7f0 tx=0x7f28dc030000 comp rx=0 tx=0).stop 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 -- 172.21.15.106:0/578528678 shutdown_connections 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbd3d0 0x563b31cda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbcff0 0x563b31c8f070 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/578528678 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31cb3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 -- 172.21.15.106:0/578528678 >> 172.21.15.106:0/578528678 conn(0x563b31c904d0 msgr2=0x563b31caf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 -- 172.21.15.106:0/578528678 shutdown_connections 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.066+0000 7f28f6e5d740 1 -- 172.21.15.106:0/578528678 wait complete. 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.067+0000 7f28f6e5d740 1 -- start start 2024-04-13T03:17:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.067+0000 7f28f6e5d740 1 Processor -- start 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31d54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31cbb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563b31cb3d10 con 0x563b31cbd3d0 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec200140 con 0x563b31cbcff0 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x563b31cb38b0 con 0x563b31cb1fa0 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f6e5d740 10 client.0 _ll_register_callbacks cb 0x7f28e8059df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54100/0 (socket says 172.21.15.106:54100) 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 -- 172.21.15.106:0/2826920521 learned_addr learned my addr 172.21.15.106:0/2826920521 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f1e13640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31d54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f1612640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31cbb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 -- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 msgr2=0x563b31d54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31d54db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 -- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 msgr2=0x563b31cbb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31cbb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 -- 172.21.15.106:0/2826920521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b31cdc1d0 con 0x563b31cbd3d0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28f2614640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f28e4011000 tx=0x7f28e4001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc74574d452d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.068+0000 7f28ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4024020 con 0x563b31cbd3d0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f28e4007240 con 0x563b31cbd3d0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28dc056070 con 0x563b31cbd3d0 2024-04-13T03:17:17.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f28e402e050 con 0x563b31cbd3d0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 10 client.8169 fetch_fsmap learned FSMap version 64 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x563b31cff180 con 0x563b31cbd3d0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc056070 con 0x563b31cbd3d0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f28e4031020 con 0x563b31cbd3d0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 10 client.8169 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 10 client.8169 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 20 client.8169 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31c93f60 0x563b31cfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 4 client.8169 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b31c02a00 con 0x563b31c93f60 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28f1e13640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31c93f60 0x563b31cfe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.069+0000 7f28ccff9640 10 client.8169 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f1e13640 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31c93f60 0x563b31cfe1e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e10653f9492860 server_cookie=6122373c962998de in_seq=0 out_seq=0 2024-04-13T03:17:17.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28ccff9640 1 -- 172.21.15.106:0/2826920521 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b31c02a00 con 0x563b31c93f60 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28ccff9640 10 client.8169 handle_command_reply: tid=0 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 client.8169 shutdown 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 msgr2=0x563b31cbbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f28e4011000 tx=0x7f28e4001400 comp rx=0 tx=0).stop 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 shutdown_connections 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563b31cbd3d0 0x563b31cbbd00 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b31cbcff0 0x563b31cbb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x563b31c93f60 0x563b31cfe1e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 --2- 172.21.15.106:0/2826920521 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x563b31cb1fa0 0x563b31d54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28ccff9640 0 client.8169 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 >> 172.21.15.106:0/2826920521 conn(0x563b31c904d0 msgr2=0x563b31cbc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 shutdown_connections 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 1 -- 172.21.15.106:0/2826920521 wait complete. 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f6e5d740 20 client.8169 trim_cache size 0 max 16384 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:17:17.078 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:17:17.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 msgr2=0x7f28d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 0x7f28d006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f28dc035000 tx=0x7f28dc008030 comp rx=0 tx=0).stop 2024-04-13T03:17:17.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 msgr2=0x7f28ec1405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.070+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f28e800c820 tx=0x7f28e800cce0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 shutdown_connections 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f28d006c8f0 0x7f28d006edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f28ec15fb50 0x7f28ec14cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28ec15ef10 0x7f28ec14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 --2- 172.21.15.106:0/235640224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28ec0d07c0 0x7f28ec1405c0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 >> 172.21.15.106:0/235640224 conn(0x7f28ec0cd6a0 msgr2=0x7f28ec162320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 shutdown_connections 2024-04-13T03:17:17.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.071+0000 7f28f4077640 1 -- 172.21.15.106:0/235640224 wait complete. 2024-04-13T03:17:17.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:17:17.082 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:17.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.182+0000 7f259e31e640 1 Processor -- start 2024-04-13T03:17:17.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.182+0000 7f259e31e640 1 -- start start 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f259e31e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2598156b50 0x7f2598160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f259e31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f259815cc10 0x7f259815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f259e31e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259815d690 con 0x7f259806dd70 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f259e31e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f259815d970 con 0x7f2598156b50 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f259e31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2598160410 con 0x7f259815cc10 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2598156b50 0x7f2598160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2598156b50 0x7f2598160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54104/0 (socket says 172.21.15.106:54104) 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 -- 172.21.15.106:0/1199600679 learned_addr learned my addr 172.21.15.106:0/1199600679 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f25977fe640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f259815cc10 0x7f259815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 -- 172.21.15.106:0/1199600679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f259815cc10 msgr2=0x7f259815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f259815cc10 0x7f259815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 -- 172.21.15.106:0/1199600679 >> v1:172.21.15.114:6789/0 conn(0x7f259806dd70 legacy=0x7f259806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.183+0000 7f2597fff640 1 -- 172.21.15.106:0/1199600679 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981606f0 con 0x7f2598156b50 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.184+0000 7f2597fff640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2598156b50 0x7f2598160c10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2588009870 tx=0x7f258802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdb8d6c46d679a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.184+0000 7f2596ffd640 1 -- 172.21.15.106:0/1199600679 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588034070 con 0x7f2598156b50 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.184+0000 7f2596ffd640 1 -- 172.21.15.106:0/1199600679 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2588030840 con 0x7f2598156b50 2024-04-13T03:17:17.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.184+0000 7f2596ffd640 1 -- 172.21.15.106:0/1199600679 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588030c10 con 0x7f2598156b50 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 -- 172.21.15.106:0/1199600679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2598156b50 msgr2=0x7f2598160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2598156b50 0x7f2598160c10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2588009870 tx=0x7f258802b980 comp rx=0 tx=0).stop 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 -- 172.21.15.106:0/1199600679 shutdown_connections 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f259815cc10 0x7f259815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 --2- 172.21.15.106:0/1199600679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2598156b50 0x7f2598160c10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 -- 172.21.15.106:0/1199600679 >> 172.21.15.106:0/1199600679 conn(0x7f25980b1f10 msgr2=0x7f25980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 -- 172.21.15.106:0/1199600679 shutdown_connections 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.185+0000 7f259e31e640 1 -- 172.21.15.106:0/1199600679 wait complete. 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.186+0000 7f259e31e640 1 Processor -- start 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.186+0000 7f259e31e640 1 -- start start 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f259806dd70 0x7f25980cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2598156b50 0x7f25980cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25980afc20 con 0x7f259815cc10 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25980afaa0 con 0x7f259806dd70 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259e31e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f259806e200 con 0x7f2598156b50 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54114/0 (socket says 172.21.15.106:54114) 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 -- 172.21.15.106:0/2731466866 learned_addr learned my addr 172.21.15.106:0/2731466866 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f25977fe640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2598156b50 0x7f25980cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f2597fff640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f259806dd70 0x7f25980cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 -- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2598156b50 msgr2=0x7f25980cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2598156b50 0x7f25980cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 -- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f259806dd70 msgr2=0x7f25980cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f259806dd70 0x7f25980cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 -- 172.21.15.106:0/2731466866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25980cc4c0 con 0x7f259815cc10 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.187+0000 7f259c8bb640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f259000c6e0 tx=0x7f259000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.188+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590003b70 con 0x7f259815cc10 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.188+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2590003d30 con 0x7f259815cc10 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.188+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25980cc7a0 con 0x7f259815cc10 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.188+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590005310 con 0x7f259815cc10 2024-04-13T03:17:17.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.188+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25980ccd60 con 0x7f259815cc10 2024-04-13T03:17:17.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.189+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2598068a10 con 0x7f259815cc10 2024-04-13T03:17:17.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.189+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25900054b0 con 0x7f259815cc10 2024-04-13T03:17:17.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.190+0000 7f2594ff9640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 0x7f258406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.190+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f259008bb80 con 0x7f259815cc10 2024-04-13T03:17:17.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.190+0000 7f2597fff640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 0x7f258406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.190+0000 7f2597fff640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 0x7f258406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2588036000 tx=0x7f2588008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.194+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2590059620 con 0x7f259815cc10 2024-04-13T03:17:17.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.405+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f259806ba90 con 0x7f259815cc10 2024-04-13T03:17:17.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.405+0000 7f2594ff9640 1 -- 172.21.15.106:0/2731466866 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7f259005d920 con 0x7f259815cc10 2024-04-13T03:17:17.406 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:17.406 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:17:17.407 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:17:17.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 msgr2=0x7f258406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 0x7f258406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2588036000 tx=0x7f2588008030 comp rx=0 tx=0).stop 2024-04-13T03:17:17.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 msgr2=0x7f2598099840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f259000c6e0 tx=0x7f259000cba0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 shutdown_connections 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f258406c8f0 0x7f258406edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2598156b50 0x7f25980cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f259815cc10 0x7f2598099840 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 --2- 172.21.15.106:0/2731466866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f259806dd70 0x7f25980cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 >> 172.21.15.106:0/2731466866 conn(0x7f25980b1f10 msgr2=0x7f2598072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 shutdown_connections 2024-04-13T03:17:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.407+0000 7f259e31e640 1 -- 172.21.15.106:0/2731466866 wait complete. 2024-04-13T03:17:17.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-13T03:17:17.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.520+0000 7fb791d1e640 1 Processor -- start 2024-04-13T03:17:17.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.520+0000 7fb791d1e640 1 -- start start 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb791d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 0x7fb78c072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb791d1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb78c06e020 0x7fb78c073350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb791d1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c06c880 con 0x7fb78c145f60 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb791d1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c06caf0 con 0x7fb78c06e020 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb791d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c06cdd0 con 0x7fb78c06db90 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb78c06e020 0x7fb78c073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb78c145f60 0x7fb78c06c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50140/0 (socket says 172.21.15.106:50140) 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb78c06e020 0x7fb78c073350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54120/0 (socket says 172.21.15.106:54120) 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78b7fe640 1 -- 172.21.15.106:0/2457473868 learned_addr learned my addr 172.21.15.106:0/2457473868 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:17.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78affd640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 0x7fb78c072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 -- 172.21.15.106:0/2457473868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 msgr2=0x7fb78c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 0x7fb78c072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 -- 172.21.15.106:0/2457473868 >> v1:172.21.15.106:6789/0 conn(0x7fb78c145f60 legacy=0x7fb78c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78bfff640 1 -- 172.21.15.106:0/2457473868 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c06d0b0 con 0x7fb78c06e020 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.521+0000 7fb78affd640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 0x7fb78c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.522+0000 7fb78bfff640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06e020 0x7fb78c073350 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb784009d90 tx=0x7fb78402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad1f4b510ecbbda server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.522+0000 7fb78a7fc640 1 -- 172.21.15.106:0/2457473868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784039070 con 0x7fb78c06e020 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.522+0000 7fb78a7fc640 1 -- 172.21.15.106:0/2457473868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78402cc50 con 0x7fb78c06e020 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.522+0000 7fb78a7fc640 1 -- 172.21.15.106:0/2457473868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784034440 con 0x7fb78c06e020 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 -- 172.21.15.106:0/2457473868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06e020 msgr2=0x7fb78c073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06e020 0x7fb78c073350 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb784009d90 tx=0x7fb78402bea0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 -- 172.21.15.106:0/2457473868 shutdown_connections 2024-04-13T03:17:17.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06e020 0x7fb78c073350 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 --2- 172.21.15.106:0/2457473868 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c06db90 0x7fb78c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 -- 172.21.15.106:0/2457473868 >> 172.21.15.106:0/2457473868 conn(0x7fb78c06f0d0 msgr2=0x7fb78c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 -- 172.21.15.106:0/2457473868 shutdown_connections 2024-04-13T03:17:17.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.523+0000 7fb791d1e640 1 -- 172.21.15.106:0/2457473868 wait complete. 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.524+0000 7fb791d1e640 1 Processor -- start 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.524+0000 7fb791d1e640 1 -- start start 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 0x7fb78c13ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c145f60 0x7fb78c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c0cdaa0 con 0x7fb78c06db90 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c146340 con 0x7fb78c06e020 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb791d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c0cdc20 con 0x7fb78c145f60 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54136/0 (socket says 172.21.15.106:54136) 2024-04-13T03:17:17.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 -- 172.21.15.106:0/3235705045 learned_addr learned my addr 172.21.15.106:0/3235705045 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78affd640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 0x7fb78c13ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78bfff640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c145f60 0x7fb78c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 -- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c145f60 msgr2=0x7fb78c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c145f60 0x7fb78c15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 -- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 msgr2=0x7fb78c13ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 0x7fb78c13ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 -- 172.21.15.106:0/3235705045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c0cbd90 con 0x7fb78c06db90 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78affd640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 0x7fb78c13ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.525+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c3c0 tx=0x7fb77c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.526+0000 7fb773fff640 1 -- 172.21.15.106:0/3235705045 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c017070 con 0x7fb78c06db90 2024-04-13T03:17:17.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.526+0000 7fb773fff640 1 -- 172.21.15.106:0/3235705045 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c005ce0 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.526+0000 7fb773fff640 1 -- 172.21.15.106:0/3235705045 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0072f0 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.526+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c0cc070 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.526+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c0cc910 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.527+0000 7fb773fff640 1 -- 172.21.15.106:0/3235705045 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c01a070 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.527+0000 7fb773fff640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 0x7fb77806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.528+0000 7fb773fff640 1 -- 172.21.15.106:0/3235705045 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb77c08b130 con 0x7fb78c06db90 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.528+0000 7fb78affd640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 0x7fb77806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.528+0000 7fb78affd640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 0x7fb77806ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb78c157b40 tx=0x7fb780008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.528+0000 7fb794c03740 1 Processor -- start 2024-04-13T03:17:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.528+0000 7fb794c03740 1 -- start start 2024-04-13T03:17:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad40c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad431f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad36ffe0 con 0x55f1ad432af0 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad30c860 con 0x55f1ad4356d0 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f1ad36bfa0 con 0x55f1ad40a800 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb78affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb78affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54148/0 (socket says 172.21.15.106:54148) 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.529+0000 7fb78affd640 1 -- 172.21.15.106:0/736342423 learned_addr learned my addr 172.21.15.106:0/736342423 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78bfff640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad431f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad40c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 -- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 msgr2=0x55f1ad431f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad431f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 -- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 msgr2=0x55f1ad40c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad40c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 -- 172.21.15.106:0/736342423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ad33b1a0 con 0x55f1ad432af0 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad40c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb78affd640 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb780010970 tx=0x7fb780013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fda6da93b56769 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb7717fa640 1 -- 172.21.15.106:0/736342423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78001c050 con 0x55f1ad432af0 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb7717fa640 1 -- 172.21.15.106:0/736342423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78001d040 con 0x55f1ad432af0 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.530+0000 7fb7717fa640 1 -- 172.21.15.106:0/736342423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78000f040 con 0x55f1ad432af0 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- 172.21.15.106:0/736342423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 msgr2=0x55f1ad435ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb780010970 tx=0x7fb780013000 comp rx=0 tx=0).stop 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- 172.21.15.106:0/736342423 shutdown_connections 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad431f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad435ab0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 --2- 172.21.15.106:0/736342423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad40c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- 172.21.15.106:0/736342423 >> 172.21.15.106:0/736342423 conn(0x55f1ad39edc0 msgr2=0x55f1ad4504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- 172.21.15.106:0/736342423 shutdown_connections 2024-04-13T03:17:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- 172.21.15.106:0/736342423 wait complete. 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 -- start start 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.531+0000 7fb794c03740 1 Processor -- start 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad4045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad404b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad405b00 con 0x55f1ad40a800 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad30a1d0 con 0x55f1ad4356d0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f1ad450c90 con 0x55f1ad432af0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb794c03740 10 client.0 _ll_register_callbacks cb 0x55f1ad32d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54152/0 (socket says 172.21.15.106:54152) 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 -- 172.21.15.106:0/4244154721 learned_addr learned my addr 172.21.15.106:0/4244154721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78bfff640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad404b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78affd640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad4045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 -- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 msgr2=0x55f1ad4045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad4045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.532+0000 7fb78b7fe640 1 -- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 msgr2=0x55f1ad404b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad404b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb78b7fe640 1 -- 172.21.15.106:0/4244154721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ad14e7c0 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb78bfff640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad404b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb77c061260 tx=0x7fb77c059370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a10e3a80d621922 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007690 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb77c058000 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78001d040 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb77c089050 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 10 client.8190 fetch_fsmap learned FSMap version 64 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55f1ad1aef50 con 0x55f1ad40a800 2024-04-13T03:17:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78001d040 con 0x55f1ad40a800 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7fb77c05f170 con 0x55f1ad40a800 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 10 client.8190 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 10 client.8190 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 20 client.8190 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad42ac20 0x55f1ad42b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 4 client.8190 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ad30f4a0 con 0x55f1ad42ac20 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.533+0000 7fb78affd640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad42ac20 0x55f1ad42b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb78affd640 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad42ac20 0x55f1ad42b000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5534da1b2ce47c73 server_cookie=ba8d33e6ae630252 in_seq=0 out_seq=0 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb76a7fc640 10 client.8190 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb76a7fc640 1 -- 172.21.15.106:0/4244154721 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1ad30f4a0 con 0x55f1ad42ac20 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb76a7fc640 10 client.8190 handle_command_reply: tid=0 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 client.8190 shutdown 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 msgr2=0x55f1ad42f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb77c061260 tx=0x7fb77c059370 comp rx=0 tx=0).stop 2024-04-13T03:17:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 shutdown_connections 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad42ac20 0x55f1ad42b000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad4045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad404b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 --2- 172.21.15.106:0/4244154721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad42f540 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb76a7fc640 0 client.8190 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 >> 172.21.15.106:0/4244154721 conn(0x55f1ad39edc0 msgr2=0x55f1ad454090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 shutdown_connections 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.534+0000 7fb794c03740 1 -- 172.21.15.106:0/4244154721 wait complete. 2024-04-13T03:17:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.535+0000 7fb794c03740 20 client.8190 trim_cache size 0 max 16384 2024-04-13T03:17:17.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.548+0000 7fb794c03740 1 Processor -- start 2024-04-13T03:17:17.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.548+0000 7fb794c03740 1 -- start start 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.548+0000 7fb794c03740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.548+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad40b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad409670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb794c03740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c200840 con 0x55f1ad40a800 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1413b0 con 0x55f1ad4356d0 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c2001f0 con 0x55f1ad432af0 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54156/0 (socket says 172.21.15.106:54156) 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 -- 172.21.15.106:0/1778764669 learned_addr learned my addr 172.21.15.106:0/1778764669 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78affd640 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad40b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78bfff640 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad409670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 -- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 msgr2=0x55f1ad40b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad40b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 -- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 msgr2=0x55f1ad409670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad409670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 -- 172.21.15.106:0/1778764669 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ad0420b0 con 0x55f1ad40a800 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fb77c055530 tx=0x7fb77c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4449fca79ba2df server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb770ff9640 1 -- 172.21.15.106:0/1778764669 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c076070 con 0x55f1ad40a800 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb770ff9640 1 -- 172.21.15.106:0/1778764669 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c078050 con 0x55f1ad40a800 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.549+0000 7fb770ff9640 1 -- 172.21.15.106:0/1778764669 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c054e50 con 0x55f1ad40a800 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- 172.21.15.106:0/1778764669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 msgr2=0x55f1ad40b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fb77c055530 tx=0x7fb77c01e070 comp rx=0 tx=0).stop 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- 172.21.15.106:0/1778764669 shutdown_connections 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad432af0 0x55f1ad40b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad409670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 --2- 172.21.15.106:0/1778764669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad40a800 0x55f1ad40b050 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- 172.21.15.106:0/1778764669 >> 172.21.15.106:0/1778764669 conn(0x55f1ad39edc0 msgr2=0x55f1ad42d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- 172.21.15.106:0/1778764669 shutdown_connections 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- 172.21.15.106:0/1778764669 wait complete. 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 -- start start 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.550+0000 7fb794c03740 1 Processor -- start 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad4c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad42e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad4643c0 con 0x55f1ad432af0 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad429ab0 con 0x55f1ad4356d0 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55f1ad460f40 con 0x55f1ad40a800 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb794c03740 10 client.0 _ll_register_callbacks cb 0x7fb77c0587e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:17.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb78affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb78affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54160/0 (socket says 172.21.15.106:54160) 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb78affd640 1 -- 172.21.15.106:0/634054639 learned_addr learned my addr 172.21.15.106:0/634054639 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad4c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.551+0000 7fb78bfff640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad42e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 -- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 msgr2=0x55f1ad4c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad4c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 -- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 msgr2=0x55f1ad42e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad42e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 -- 172.21.15.106:0/634054639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ad4c9e00 con 0x55f1ad432af0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad4c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78bfff640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad42e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78affd640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb780022000 tx=0x7fb780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c60a5dc68d18f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:17.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78001c040 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb780026040 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c078050 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb780040050 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 10 client.8196 fetch_fsmap learned FSMap version 64 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55f1ad450ac0 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c078050 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7fb780043020 con 0x55f1ad432af0 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 10 client.8196 fetch_fsmap finished waiting for FSMap version 64 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 10 client.8196 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 20 client.8196 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad4cd510 0x55f1ad4cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 4 client.8196 mds_command: new command op to 8028 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 --> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f1ad406c90 con 0x55f1ad4cd510 2024-04-13T03:17:17.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.552+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad4cd510 0x55f1ad4cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb76a7fc640 10 client.8196 ms_handle_connect on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb78b7fe640 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad4cd510 0x55f1ad4cd8f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e5dbdcc9f40c62 server_cookie=a857eedda8ac73e in_seq=0 out_seq=0 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb76a7fc640 1 -- 172.21.15.106:0/634054639 <== mds.0 v2:172.21.15.106:6835/2398448953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1ad406c90 con 0x55f1ad4cd510 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb76a7fc640 10 client.8196 handle_command_reply: tid=0 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 client.8196 shutdown 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 msgr2=0x55f1ad42e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb780022000 tx=0x7fb780004730 comp rx=0 tx=0).stop 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 shutdown_connections 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f1ad4356d0 0x55f1ad42e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1ad432af0 0x55f1ad42e310 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.106:6835/2398448953,v1:172.21.15.106:6837/2398448953] conn(0x55f1ad4cd510 0x55f1ad4cd8f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 --2- 172.21.15.106:0/634054639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55f1ad40a800 0x55f1ad4c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 >> 172.21.15.106:0/634054639 conn(0x55f1ad39edc0 msgr2=0x55f1ad409670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb76a7fc640 0 client.8196 ms_handle_reset on v2:172.21.15.106:6835/2398448953 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 shutdown_connections 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 1 -- 172.21.15.106:0/634054639 wait complete. 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.553+0000 7fb794c03740 20 client.8196 trim_cache size 0 max 16384 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:17.562 INFO:teuthology.orchestra.run.smithi106.stdout:[] 2024-04-13T03:17:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 msgr2=0x7fb77806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 0x7fb77806ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb78c157b40 tx=0x7fb780008040 comp rx=0 tx=0).stop 2024-04-13T03:17:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 msgr2=0x7fb78c0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c3c0 tx=0x7fb77c00c880 comp rx=0 tx=0).stop 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 shutdown_connections 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb77806c920 0x7fb77806ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb78c145f60 0x7fb78c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c06e020 0x7fb78c13ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 --2- 172.21.15.106:0/3235705045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb78c06db90 0x7fb78c0b4f10 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 >> 172.21.15.106:0/3235705045 conn(0x7fb78c06f0d0 msgr2=0x7fb78c15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 shutdown_connections 2024-04-13T03:17:17.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.554+0000 7fb791d1e640 1 -- 172.21.15.106:0/3235705045 wait complete. 2024-04-13T03:17:17.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-13T03:17:17.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.663+0000 7f88402de640 1 Processor -- start 2024-04-13T03:17:17.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.663+0000 7f88402de640 1 -- start start 2024-04-13T03:17:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f88402de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f88380d3ce0 0x7f88380b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f88402de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88381635b0 con 0x7f883815ea80 2024-04-13T03:17:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f88402de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88381627f0 con 0x7f88380b68f0 2024-04-13T03:17:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f88402de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f88380b7100 con 0x7f88380d3ce0 2024-04-13T03:17:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f883d879640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f88380d3ce0 0x7f88380b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f883e07a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f883815ea80 0x7f88380d35d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50146/0 (socket says 172.21.15.106:50146) 2024-04-13T03:17:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.664+0000 7f883e07a640 1 -- 172.21.15.106:0/2762258446 learned_addr learned my addr 172.21.15.106:0/2762258446 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.665+0000 7f883d879640 1 -- 172.21.15.106:0/2762258446 >> v1:172.21.15.114:6789/0 conn(0x7f88380b68f0 legacy=0x7f88381689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.665+0000 7f883d879640 1 -- 172.21.15.106:0/2762258446 >> v1:172.21.15.106:6789/0 conn(0x7f883815ea80 legacy=0x7f88380d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.665+0000 7f883d879640 1 -- 172.21.15.106:0/2762258446 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8838162ad0 con 0x7f88380d3ce0 2024-04-13T03:17:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.665+0000 7f883d879640 1 --2- 172.21.15.106:0/2762258446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88380d3ce0 0x7f88380b63b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e56fc352856b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f883d078640 1 -- 172.21.15.106:0/2762258446 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828035020 con 0x7f88380d3ce0 2024-04-13T03:17:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f883d078640 1 -- 172.21.15.106:0/2762258446 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8828031830 con 0x7f88380d3ce0 2024-04-13T03:17:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f883d078640 1 -- 172.21.15.106:0/2762258446 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828031c00 con 0x7f88380d3ce0 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 -- 172.21.15.106:0/2762258446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88380d3ce0 msgr2=0x7f88380b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 --2- 172.21.15.106:0/2762258446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88380d3ce0 0x7f88380b63b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f88280097a0 tx=0x7f882802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 -- 172.21.15.106:0/2762258446 shutdown_connections 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 --2- 172.21.15.106:0/2762258446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88380d3ce0 0x7f88380b63b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 -- 172.21.15.106:0/2762258446 >> 172.21.15.106:0/2762258446 conn(0x7f88380b97a0 msgr2=0x7f88380b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 -- 172.21.15.106:0/2762258446 shutdown_connections 2024-04-13T03:17:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.666+0000 7f88402de640 1 -- 172.21.15.106:0/2762258446 wait complete. 2024-04-13T03:17:17.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.667+0000 7f88402de640 1 Processor -- start 2024-04-13T03:17:17.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.667+0000 7f88402de640 1 -- start start 2024-04-13T03:17:17.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f88380b68f0 0x7f88381f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 0x7f88380c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f88380b33e0 con 0x7f883815ea80 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8838163090 con 0x7f88380d3ce0 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f88402de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f883815ee60 con 0x7f88380b68f0 2024-04-13T03:17:17.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e87b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e87b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54178/0 (socket says 172.21.15.106:54178) 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e87b640 1 -- 172.21.15.106:0/1116108604 learned_addr learned my addr 172.21.15.106:0/1116108604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883d879640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 0x7f88380c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e07a640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f88380b68f0 0x7f88381f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e87b640 1 -- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f88380b68f0 msgr2=0x7f88381f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.668+0000 7f883e87b640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f88380b68f0 0x7f88381f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f883e87b640 1 -- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 msgr2=0x7f88380c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f883e87b640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 0x7f88380c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f883e87b640 1 -- 172.21.15.106:0/1116108604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381e8fb0 con 0x7f883815ea80 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f883d879640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 0x7f88380c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f883e87b640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f883000c390 tx=0x7f883000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830017070 con 0x7f883815ea80 2024-04-13T03:17:17.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88300052d0 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381e9290 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830003810 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.669+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381e9850 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.670+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838068a10 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.670+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f883001a070 con 0x7f883815ea80 2024-04-13T03:17:17.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.671+0000 7f882effd640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:17.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.671+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f883008b120 con 0x7f883815ea80 2024-04-13T03:17:17.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.671+0000 7f883e07a640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:17.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.671+0000 7f883e07a640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 0x7f881c06edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8834005c00 tx=0x7f8834009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:17.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.675+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88300592a0 con 0x7f883815ea80 2024-04-13T03:17:17.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.886+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f883815e180 con 0x7f883815ea80 2024-04-13T03:17:17.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.886+0000 7f882effd640 1 -- 172.21.15.106:0/1116108604 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8830058ba0 con 0x7f883815ea80 2024-04-13T03:17:17.887 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:17.887 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-13T03:17:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 msgr2=0x7f881c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 0x7f881c06edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8834005c00 tx=0x7f8834009040 comp rx=0 tx=0).stop 2024-04-13T03:17:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 msgr2=0x7f88380b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f883000c390 tx=0x7f883000c850 comp rx=0 tx=0).stop 2024-04-13T03:17:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 shutdown_connections 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f883815ea80 0x7f88380b5190 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88380d3ce0 0x7f88380c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 --2- 172.21.15.106:0/1116108604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f88380b68f0 0x7f88381f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 >> 172.21.15.106:0/1116108604 conn(0x7f88380b97a0 msgr2=0x7f88380b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 shutdown_connections 2024-04-13T03:17:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:17.888+0000 7f88402de640 1 -- 172.21.15.106:0/1116108604 wait complete. 2024-04-13T03:17:17.899 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-13T03:17:17.900 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:17:17.900 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:17:17.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:17:17.984 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:17:17.984 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:17:17.984 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:17.984 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:18.000 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:18.000 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:18.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:18.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 Processor -- start 2024-04-13T03:17:18.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 -- start start 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e200b5400 0x7f8e200b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e200d0890 con 0x7f8e200ce580 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e200d0b40 con 0x7f8e2015e220 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.195+0000 7f8e27c88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8e200d1be0 con 0x7f8e200b5400 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25223640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e200b5400 0x7f8e200b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25a24640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8e200ce580 0x7f8e200d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50148/0 (socket says 172.21.15.106:50148) 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25a24640 1 -- 172.21.15.106:0/3179494170 learned_addr learned my addr 172.21.15.106:0/3179494170 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:18.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25223640 1 -- 172.21.15.106:0/3179494170 >> v1:172.21.15.114:6789/0 conn(0x7f8e2015e220 legacy=0x7f8e2015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:18.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25223640 1 -- 172.21.15.106:0/3179494170 >> v1:172.21.15.106:6789/0 conn(0x7f8e200ce580 legacy=0x7f8e200d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e25223640 1 -- 172.21.15.106:0/3179494170 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e200d0e20 con 0x7f8e200b5400 2024-04-13T03:17:18.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.196+0000 7f8e24a22640 1 -- 172.21.15.106:0/3179494170 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4055052518 0 0) 0x7f8e200d0890 con 0x7f8e200ce580 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e25223640 1 --2- 172.21.15.106:0/3179494170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200b5400 0x7f8e200b5890 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0097a0 tx=0x7f8e1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7e791a748b03d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e24a22640 1 -- 172.21.15.106:0/3179494170 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035020 con 0x7f8e200b5400 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e24a22640 1 -- 172.21.15.106:0/3179494170 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c031830 con 0x7f8e200b5400 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e24a22640 1 -- 172.21.15.106:0/3179494170 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c031c00 con 0x7f8e200b5400 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 -- 172.21.15.106:0/3179494170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200b5400 msgr2=0x7f8e200b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 --2- 172.21.15.106:0/3179494170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200b5400 0x7f8e200b5890 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0097a0 tx=0x7f8e1c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 -- 172.21.15.106:0/3179494170 shutdown_connections 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 --2- 172.21.15.106:0/3179494170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200b5400 0x7f8e200b5890 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 -- 172.21.15.106:0/3179494170 >> 172.21.15.106:0/3179494170 conn(0x7f8e200d25b0 msgr2=0x7f8e200d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 -- 172.21.15.106:0/3179494170 shutdown_connections 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.197+0000 7f8e27c88640 1 -- 172.21.15.106:0/3179494170 wait complete. 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 Processor -- start 2024-04-13T03:17:18.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 -- start start 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e200b5400 0x7f8e20155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e2015e220 0x7f8e2014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e200d16c0 con 0x7f8e200ce580 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8e200d13e0 con 0x7f8e2015e220 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e27c88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8e200ce960 con 0x7f8e200b5400 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e25223640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e25223640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54204/0 (socket says 172.21.15.106:54204) 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.199+0000 7f8e25223640 1 -- 172.21.15.106:0/871878186 learned_addr learned my addr 172.21.15.106:0/871878186 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:18.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 -- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e200b5400 msgr2=0x7f8e20155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e200b5400 0x7f8e20155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 -- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e2015e220 msgr2=0x7f8e2014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e26225640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e2015e220 0x7f8e2014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e2015e220 0x7f8e2014f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 -- 172.21.15.106:0/871878186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201f8b70 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e25223640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0098c0 tx=0x7f8e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03ba80 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c03bc20 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e20154ef0 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0453f0 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.200+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e200c8d10 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.201+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e20068a10 con 0x7f8e200ce580 2024-04-13T03:17:18.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.202+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e1c04c060 con 0x7f8e200ce580 2024-04-13T03:17:18.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.202+0000 7f8e167fc640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 0x7f8e0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.202+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f8e1c0adbd0 con 0x7f8e200ce580 2024-04-13T03:17:18.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.202+0000 7f8e25a24640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 0x7f8e0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.202+0000 7f8e25a24640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 0x7f8e0406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8e10007fd0 tx=0x7f8e10007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.206+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e201f8b70 con 0x7f8e200ce580 2024-04-13T03:17:18.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.384+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e20068140 con 0x7f8e200ce580 2024-04-13T03:17:18.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.385+0000 7f8e167fc640 1 -- 172.21.15.106:0/871878186 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17375 (secure 0 0 0) 0x7f8e1c040070 con 0x7f8e200ce580 2024-04-13T03:17:18.386 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:18.386 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":95,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:16:40.106840+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 msgr2=0x7f8e0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 0x7f8e0406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8e10007fd0 tx=0x7f8e10007920 comp rx=0 tx=0).stop 2024-04-13T03:17:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 msgr2=0x7f8e201f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0098c0 tx=0x7f8e1c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 shutdown_connections 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8e0406c8f0 0x7f8e0406edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e2015e220 0x7f8e2014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e200ce580 0x7f8e201f8630 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 --2- 172.21.15.106:0/871878186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e200b5400 0x7f8e20155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 >> 172.21.15.106:0/871878186 conn(0x7f8e200d25b0 msgr2=0x7f8e200cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 shutdown_connections 2024-04-13T03:17:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.387+0000 7f8e27c88640 1 -- 172.21.15.106:0/871878186 wait complete. 2024-04-13T03:17:18.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:15:17.537136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:15:18.258394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:18.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:18.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.502+0000 7fc6ce82d640 1 Processor -- start 2024-04-13T03:17:18.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.502+0000 7fc6ce82d640 1 -- start start 2024-04-13T03:17:18.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6ce82d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc6c815ee40 0x7fc6c80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6ce82d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c80b5cf0 con 0x7fc6c80b1c30 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6ce82d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c80b5fd0 con 0x7fc6c80709e0 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6ce82d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c80b62b0 con 0x7fc6c815ee40 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc6c80b1c30 0x7fc6c80b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50156/0 (socket says 172.21.15.106:50156) 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c7fff640 1 -- 172.21.15.106:0/3518068382 learned_addr learned my addr 172.21.15.106:0/3518068382 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c77fe640 1 --2- 172.21.15.106:0/3518068382 >> v2:172.21.15.114:3300/0 conn(0x7fc6c815ee40 0x7fc6c80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262300339 0 0) 0x7fc6c80b5cf0 con 0x7fc6c80b1c30 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6b0003520 con 0x7fc6c80b1c30 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056947098 0 0) 0x7fc6c80b5fd0 con 0x7fc6c80709e0 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6c80b5cf0 con 0x7fc6c80709e0 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2929412899 0 0) 0x7fc6b0003520 con 0x7fc6c80b1c30 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 >> v2:172.21.15.114:3300/0 conn(0x7fc6c815ee40 msgr2=0x7fc6c80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:18.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 --2- 172.21.15.106:0/3518068382 >> v2:172.21.15.114:3300/0 conn(0x7fc6c815ee40 0x7fc6c80704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 >> v1:172.21.15.114:6789/0 conn(0x7fc6c80709e0 legacy=0x7fc6c806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c80b6590 con 0x7fc6c80b1c30 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc6b8001670 con 0x7fc6c80b1c30 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 >> v1:172.21.15.106:6789/0 conn(0x7fc6c80b1c30 legacy=0x7fc6c80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.503+0000 7fc6c6ffd640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6b0003b40 0x7fc6b0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c6ffd640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b0004800 0x7fc6b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c6ffd640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 0x7fc6b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80b62b0 con 0x7fc6b00051c0 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b000c040 con 0x7fc6b0003b40 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b000c240 con 0x7fc6b0004800 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 0x7fc6b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c7fff640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6b0003b40 0x7fc6b0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6c77fe640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b0004800 0x7fc6b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 -- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b0004800 msgr2=0x7fc6b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b0004800 0x7fc6b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 -- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6b0003b40 msgr2=0x7fc6b0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6b0003b40 0x7fc6b0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.504+0000 7fc6ccdca640 1 -- 172.21.15.106:0/3518068382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b0003520 con 0x7fc6b00051c0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ccdca640 1 -- 172.21.15.106:0/3518068382 reap_dead start 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ccdca640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 0x7fc6b000b9c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0001fd0 tx=0x7fc6b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64b0d63b98f2526 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0001bf0 con 0x7fc6b00051c0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c0003b70 con 0x7fc6b00051c0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6c6ffd640 1 -- 172.21.15.106:0/3518068382 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00050d0 con 0x7fc6b00051c0 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 -- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 msgr2=0x7fc6b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 0x7fc6b000b9c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0001fd0 tx=0x7fc6b0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 -- 172.21.15.106:0/3518068382 shutdown_connections 2024-04-13T03:17:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/3518068382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6b00051c0 0x7fc6b000b9c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 -- 172.21.15.106:0/3518068382 >> 172.21.15.106:0/3518068382 conn(0x7fc6c8071870 msgr2=0x7fc6c8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 -- 172.21.15.106:0/3518068382 shutdown_connections 2024-04-13T03:17:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.505+0000 7fc6ce82d640 1 -- 172.21.15.106:0/3518068382 wait complete. 2024-04-13T03:17:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.506+0000 7fc6ce82d640 1 Processor -- start 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 -- start start 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b00051c0 0x7fc6c81f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6c8154020 0x7fc6c81544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8070830 con 0x7fc6c80cdc40 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80706b0 con 0x7fc6c8154020 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6ce82d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c80b54a0 con 0x7fc6b00051c0 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6c77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54228/0 (socket says 172.21.15.106:54228) 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.507+0000 7fc6c77fe640 1 -- 172.21.15.106:0/1565979463 learned_addr learned my addr 172.21.15.106:0/1565979463 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c7fff640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b00051c0 0x7fc6c81f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6ccdca640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6c8154020 0x7fc6c81544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 -- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b00051c0 msgr2=0x7fc6c81f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b00051c0 0x7fc6c81f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 -- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6c8154020 msgr2=0x7fc6c81544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6c8154020 0x7fc6c81544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 -- 172.21.15.106:0/1565979463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81f52c0 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c77fe640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc011940 tx=0x7fc6bc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00fa90 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00fc30 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c80ccba0 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc014440 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.508+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c80cd440 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.509+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c8068a10 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.510+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6bc012040 con 0x7fc6c80cdc40 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.510+0000 7fc6c4ff9640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 0x7fc6b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.510+0000 7fc6c7fff640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 0x7fc6b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.510+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fc6bc08c390 con 0x7fc6c80cdc40 2024-04-13T03:17:18.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.510+0000 7fc6c7fff640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 0x7fc6b406ede0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80ce4f0 tx=0x7fc6b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.514+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6c81f52c0 con 0x7fc6c80cdc40 2024-04-13T03:17:18.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.690+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6c80711f0 con 0x7fc6c80cdc40 2024-04-13T03:17:18.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.690+0000 7fc6c4ff9640 1 -- 172.21.15.106:0/1565979463 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fc6bc019030 con 0x7fc6c80cdc40 2024-04-13T03:17:18.691 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:18.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 msgr2=0x7fc6b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 0x7fc6b406ede0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80ce4f0 tx=0x7fc6b8009040 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 msgr2=0x7fc6c8146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc011940 tx=0x7fc6bc011e00 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 shutdown_connections 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc6b406c920 0x7fc6b406ede0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6c8154020 0x7fc6c81544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6c80cdc40 0x7fc6c8146de0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 --2- 172.21.15.106:0/1565979463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc6b00051c0 0x7fc6c81f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.692+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 >> 172.21.15.106:0/1565979463 conn(0x7fc6c8071870 msgr2=0x7fc6c806d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.693+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 shutdown_connections 2024-04-13T03:17:18.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.693+0000 7fc6ce82d640 1 -- 172.21.15.106:0/1565979463 wait complete. 2024-04-13T03:17:18.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:17:18.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.805+0000 7f8c37e60640 1 Processor -- start 2024-04-13T03:17:18.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.806+0000 7f8c37e60640 1 -- start start 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.806+0000 7f8c37e60640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c300704a0 0x7f8c300d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.806+0000 7f8c37e60640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c3015ddd0 con 0x7f8c300cfea0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.806+0000 7f8c37e60640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c3015d4d0 con 0x7f8c300d1a60 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.806+0000 7f8c37e60640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8c3015e100 con 0x7f8c300704a0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c35bfc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8c300cfea0 0x7f8c3015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50164/0 (socket says 172.21.15.106:50164) 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c35bfc640 1 -- 172.21.15.106:0/4037884345 learned_addr learned my addr 172.21.15.106:0/4037884345 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c353fb640 1 --2- 172.21.15.106:0/4037884345 >> v2:172.21.15.114:3300/0 conn(0x7f8c300704a0 0x7f8c300d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1963287254 0 0) 0x7f8c3015ddd0 con 0x7f8c300cfea0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c1c003520 con 0x7f8c300cfea0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 240754500 0 0) 0x7f8c1c003520 con 0x7f8c300cfea0 2024-04-13T03:17:18.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c353fb640 1 -- 172.21.15.106:0/4037884345 >> v1:172.21.15.114:6789/0 conn(0x7f8c300d1a60 legacy=0x7f8c3015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c353fb640 1 -- 172.21.15.106:0/4037884345 >> v1:172.21.15.106:6789/0 conn(0x7f8c300cfea0 legacy=0x7f8c3015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.807+0000 7f8c353fb640 1 -- 172.21.15.106:0/4037884345 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3015d7b0 con 0x7f8c300704a0 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.808+0000 7f8c353fb640 1 --2- 172.21.15.106:0/4037884345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300d1520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097a0 tx=0x7f8c2c02b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c45fd32d4cc66bed server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.808+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c037070 con 0x7f8c300704a0 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.808+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c030510 con 0x7f8c300704a0 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.808+0000 7f8c34bfa640 1 -- 172.21.15.106:0/4037884345 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20001670 con 0x7f8c300704a0 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 -- 172.21.15.106:0/4037884345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 msgr2=0x7f8c300d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 --2- 172.21.15.106:0/4037884345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300d1520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097a0 tx=0x7f8c2c02b680 comp rx=0 tx=0).stop 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 -- 172.21.15.106:0/4037884345 shutdown_connections 2024-04-13T03:17:18.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 --2- 172.21.15.106:0/4037884345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300d1520 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 -- 172.21.15.106:0/4037884345 >> 172.21.15.106:0/4037884345 conn(0x7f8c30071870 msgr2=0x7f8c30071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 -- 172.21.15.106:0/4037884345 shutdown_connections 2024-04-13T03:17:18.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.809+0000 7f8c37e60640 1 -- 172.21.15.106:0/4037884345 wait complete. 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.810+0000 7f8c37e60640 1 Processor -- start 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.810+0000 7f8c37e60640 1 -- start start 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.810+0000 7f8c37e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c300cfea0 0x7f8c30159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3015eca0 con 0x7f8c300d1a60 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300d1ef0 con 0x7f8c300704a0 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3015ee20 con 0x7f8c300cfea0 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54236/0 (socket says 172.21.15.106:54236) 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 -- 172.21.15.106:0/1004129893 learned_addr learned my addr 172.21.15.106:0/1004129893 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:18.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c35bfc640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c353fb640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c300cfea0 0x7f8c30159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 -- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c300cfea0 msgr2=0x7f8c30159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c300cfea0 0x7f8c30159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 -- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 msgr2=0x7f8c300af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 -- 172.21.15.106:0/1004129893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c300af930 con 0x7f8c300d1a60 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c363fd640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c840 tx=0x7f8c2800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28017070 con 0x7f8c300d1a60 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c28003da0 con 0x7f8c300d1a60 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005290 con 0x7f8c300d1a60 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c300afc10 con 0x7f8c300d1a60 2024-04-13T03:17:18.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.811+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301f5bf0 con 0x7f8c300d1a60 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.812+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c30068a10 con 0x7f8c300d1a60 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.813+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c2801a070 con 0x7f8c300d1a60 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.813+0000 7f8c267fc640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 0x7f8c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.813+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f8c2808b420 con 0x7f8c300d1a60 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.813+0000 7f8c35bfc640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 0x7f8c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:18.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.813+0000 7f8c35bfc640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 0x7f8c1406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8c200017e0 tx=0x7f8c20001230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:18.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.817+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c28058fe0 con 0x7f8c300d1a60 2024-04-13T03:17:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.992+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c30072360 con 0x7f8c300d1a60 2024-04-13T03:17:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.993+0000 7f8c267fc640 1 -- 172.21.15.106:0/1004129893 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f8c2805d2e0 con 0x7f8c300d1a60 2024-04-13T03:17:18.994 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 msgr2=0x7f8c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 0x7f8c1406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8c200017e0 tx=0x7f8c20001230 comp rx=0 tx=0).stop 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 msgr2=0x7f8c301470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c840 tx=0x7f8c2800cd00 comp rx=0 tx=0).stop 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 shutdown_connections 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c1406c8f0 0x7f8c1406edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c300cfea0 0x7f8c30159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c300d1a60 0x7f8c301470d0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 --2- 172.21.15.106:0/1004129893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c300704a0 0x7f8c300af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 >> 172.21.15.106:0/1004129893 conn(0x7f8c30071870 msgr2=0x7f8c3006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 shutdown_connections 2024-04-13T03:17:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:18.995+0000 7f8c37e60640 1 -- 172.21.15.106:0/1004129893 wait complete. 2024-04-13T03:17:19.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:17:19.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.109+0000 7f1b2442d640 1 Processor -- start 2024-04-13T03:17:19.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.109+0000 7f1b2442d640 1 -- start start 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b2442d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b1c15fb50 0x7f1b1c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b2442d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b1c0d07c0 0x7f1b1c06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b2442d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0cef80 con 0x7f1b1c15ef10 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b2442d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c0cf260 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b2442d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c0cf540 con 0x7f1b1c15fb50 2024-04-13T03:17:19.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b219c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b1c0d07c0 0x7f1b1c06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b229ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1b1c15ef10 0x7f1b1c162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50170/0 (socket says 172.21.15.106:50170) 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b229ca640 1 -- 172.21.15.106:0/3938280724 learned_addr learned my addr 172.21.15.106:0/3938280724 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b211c7640 1 -- 172.21.15.106:0/3938280724 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950895916 0 0) 0x7f1b1c0cef80 con 0x7f1b1c15ef10 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b211c7640 1 -- 172.21.15.106:0/3938280724 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b08003520 con 0x7f1b1c15ef10 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b221c9640 1 --2- 172.21.15.106:0/3938280724 >> v2:172.21.15.114:3300/0 conn(0x7f1b1c15fb50 0x7f1b1c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b219c8640 1 -- 172.21.15.106:0/3938280724 >> v2:172.21.15.114:3300/0 conn(0x7f1b1c15fb50 msgr2=0x7f1b1c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b219c8640 1 --2- 172.21.15.106:0/3938280724 >> v2:172.21.15.114:3300/0 conn(0x7f1b1c15fb50 0x7f1b1c06b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b219c8640 1 -- 172.21.15.106:0/3938280724 >> v1:172.21.15.106:6789/0 conn(0x7f1b1c15ef10 legacy=0x7f1b1c162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.110+0000 7f1b219c8640 1 -- 172.21.15.106:0/3938280724 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c0cf820 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.111+0000 7f1b219c8640 1 --2- 172.21.15.106:0/3938280724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c06be20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c06e540 tx=0x7f1b0c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369b3664b5b42382 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.111+0000 7f1b211c7640 1 -- 172.21.15.106:0/3938280724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c038070 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.111+0000 7f1b211c7640 1 -- 172.21.15.106:0/3938280724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c031710 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.111+0000 7f1b211c7640 1 -- 172.21.15.106:0/3938280724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c031ac0 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 -- 172.21.15.106:0/3938280724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c0d07c0 msgr2=0x7f1b1c06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 --2- 172.21.15.106:0/3938280724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c06be20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c06e540 tx=0x7f1b0c02b7b0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 -- 172.21.15.106:0/3938280724 shutdown_connections 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 --2- 172.21.15.106:0/3938280724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c06be20 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 --2- 172.21.15.106:0/3938280724 >> v2:172.21.15.114:3300/0 conn(0x7f1b1c15fb50 0x7f1b1c06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 -- 172.21.15.106:0/3938280724 >> 172.21.15.106:0/3938280724 conn(0x7f1b1c0cd6a0 msgr2=0x7f1b1c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 -- 172.21.15.106:0/3938280724 shutdown_connections 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.112+0000 7f1b2442d640 1 -- 172.21.15.106:0/3938280724 wait complete. 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.113+0000 7f1b2442d640 1 Processor -- start 2024-04-13T03:17:19.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.113+0000 7f1b2442d640 1 -- start start 2024-04-13T03:17:19.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 0x7f1b1c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0ced20 con 0x7f1b1c15fb50 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0ceba0 con 0x7f1b1c0d07c0 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c06f6c0 con 0x7f1b1c15ef10 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54254/0 (socket says 172.21.15.106:54254) 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 -- 172.21.15.106:0/4029057823 learned_addr learned my addr 172.21.15.106:0/4029057823 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 -- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 msgr2=0x7f1b1c0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b219c8640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 0x7f1b1c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b221c9640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 0x7f1b1c0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 -- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 msgr2=0x7f1b1c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 -- 172.21.15.106:0/4029057823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c156680 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b219c8640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 0x7f1b1c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b221c9640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b229ca640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400cb10 tx=0x7f1b1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14017070 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1400ac70 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c156960 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400ae60 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.114+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c156f20 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.115+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c0b52e0 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.116+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b1401a070 con 0x7f1b1c15fb50 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.116+0000 7f1b12ffd640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.116+0000 7f1b221c9640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.116+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f1b1408bb50 con 0x7f1b1c15fb50 2024-04-13T03:17:19.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.117+0000 7f1b221c9640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b820 tx=0x7f1b18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.120+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b14059d60 con 0x7f1b1c15fb50 2024-04-13T03:17:19.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.299+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b1c068a10 con 0x7f1b1c15fb50 2024-04-13T03:17:19.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.299+0000 7f1b12ffd640 1 -- 172.21.15.106:0/4029057823 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f1b14059660 con 0x7f1b1c15fb50 2024-04-13T03:17:19.300 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:17:19.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 msgr2=0x7f1b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b820 tx=0x7f1b18008040 comp rx=0 tx=0).stop 2024-04-13T03:17:19.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 msgr2=0x7f1b1c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400cb10 tx=0x7f1b1400a060 comp rx=0 tx=0).stop 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 shutdown_connections 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b1c15ef10 0x7f1b1c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b1c15fb50 0x7f1b1c15e6e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 --2- 172.21.15.106:0/4029057823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b1c0d07c0 0x7f1b1c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 >> 172.21.15.106:0/4029057823 conn(0x7f1b1c0cd6a0 msgr2=0x7f1b1c0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 shutdown_connections 2024-04-13T03:17:19.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.301+0000 7f1b2442d640 1 -- 172.21.15.106:0/4029057823 wait complete. 2024-04-13T03:17:19.313 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:17:19.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 Processor -- start 2024-04-13T03:17:19.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 -- start start 2024-04-13T03:17:19.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60f415e9b0 0x7f60f40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60f40b3d60 0x7f60f40d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f40cfd20 con 0x7f60f415ed90 2024-04-13T03:17:19.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60f4162af0 con 0x7f60f40b3d60 2024-04-13T03:17:19.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.414+0000 7f60f93f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f60f4162dd0 con 0x7f60f415e9b0 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f60f40b3d60 0x7f60f40d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60ebfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f60f415ed90 0x7f60f40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50180/0 (socket says 172.21.15.106:50180) 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60ebfff640 1 -- 172.21.15.106:0/2549682030 learned_addr learned my addr 172.21.15.106:0/2549682030 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f2ffd640 1 --2- 172.21.15.106:0/2549682030 >> v2:172.21.15.114:3300/0 conn(0x7f60f415e9b0 0x7f60f40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f37fe640 1 -- 172.21.15.106:0/2549682030 >> v2:172.21.15.114:3300/0 conn(0x7f60f415e9b0 msgr2=0x7f60f40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f37fe640 1 --2- 172.21.15.106:0/2549682030 >> v2:172.21.15.114:3300/0 conn(0x7f60f415e9b0 0x7f60f40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f37fe640 1 -- 172.21.15.106:0/2549682030 >> v1:172.21.15.106:6789/0 conn(0x7f60f415ed90 legacy=0x7f60f40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f27fc640 1 -- 172.21.15.106:0/2549682030 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765391003 0 0) 0x7f60f40cfd20 con 0x7f60f415ed90 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.415+0000 7f60f37fe640 1 -- 172.21.15.106:0/2549682030 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f41630b0 con 0x7f60f40b3d60 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f37fe640 1 --2- 172.21.15.106:0/2549682030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f40b3d60 0x7f60f40d0200 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f60f408e4e0 tx=0x7f60ec02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7047fa0b4a5fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f27fc640 1 -- 172.21.15.106:0/2549682030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec039070 con 0x7f60f40b3d60 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f27fc640 1 -- 172.21.15.106:0/2549682030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec02cd00 con 0x7f60f40b3d60 2024-04-13T03:17:19.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f27fc640 1 -- 172.21.15.106:0/2549682030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec034660 con 0x7f60f40b3d60 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 -- 172.21.15.106:0/2549682030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f40b3d60 msgr2=0x7f60f40d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 --2- 172.21.15.106:0/2549682030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f40b3d60 0x7f60f40d0200 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f60f408e4e0 tx=0x7f60ec02bf50 comp rx=0 tx=0).stop 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 -- 172.21.15.106:0/2549682030 shutdown_connections 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 --2- 172.21.15.106:0/2549682030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f40b3d60 0x7f60f40d0200 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 --2- 172.21.15.106:0/2549682030 >> v2:172.21.15.114:3300/0 conn(0x7f60f415e9b0 0x7f60f40d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 -- 172.21.15.106:0/2549682030 >> 172.21.15.106:0/2549682030 conn(0x7f60f408bf10 msgr2=0x7f60f40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 -- 172.21.15.106:0/2549682030 shutdown_connections 2024-04-13T03:17:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.416+0000 7f60f93f7640 1 -- 172.21.15.106:0/2549682030 wait complete. 2024-04-13T03:17:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 Processor -- start 2024-04-13T03:17:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 -- start start 2024-04-13T03:17:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60f40b3d60 0x7f60f415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f40b67d0 0x7f60f40cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4162800 con 0x7f60f415e9b0 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4162680 con 0x7f60f40b67d0 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60f93f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60f40d4d00 con 0x7f60f40b3d60 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54276/0 (socket says 172.21.15.106:54276) 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.418+0000 7f60ebfff640 1 -- 172.21.15.106:0/1611933624 learned_addr learned my addr 172.21.15.106:0/1611933624 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 -- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60f40b3d60 msgr2=0x7f60f415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60f40b3d60 0x7f60f415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 -- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f40b67d0 msgr2=0x7f60f40cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f40b67d0 0x7f60f40cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 -- 172.21.15.106:0/1611933624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f41e8d90 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60ebfff640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f60dc001eb0 tx=0x7f60dc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc017070 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc0074e0 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0059e0 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f41e9070 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.419+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41e9630 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.420+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f4068a10 con 0x7f60f415e9b0 2024-04-13T03:17:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.421+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60dc01a070 con 0x7f60f415e9b0 2024-04-13T03:17:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.421+0000 7f60eb7fe640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 0x7f60d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.421+0000 7f60f2ffd640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 0x7f60d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.421+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f60dc08b1b0 con 0x7f60f415e9b0 2024-04-13T03:17:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.422+0000 7f60f2ffd640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 0x7f60d406ede0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60f40cca10 tx=0x7f60e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.426+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60dc0593f0 con 0x7f60f415e9b0 2024-04-13T03:17:19.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.613+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60f40ccce0 con 0x7f60f415e9b0 2024-04-13T03:17:19.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.614+0000 7f60eb7fe640 1 -- 172.21.15.106:0/1611933624 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v95) ==== 80+0+246 (secure 0 0 0) 0x7f60f40ccce0 con 0x7f60f415e9b0 2024-04-13T03:17:19.615 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/635681585 2024-04-14T03:16:40.106818+0000 2024-04-13T03:17:19.615 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/635681585 2024-04-14T03:16:40.106818+0000 2024-04-13T03:17:19.615 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/3360662124 2024-04-14T03:15:28.649985+0000 2024-04-13T03:17:19.615 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/3360662124 2024-04-14T03:15:28.649985+0000 2024-04-13T03:17:19.615 INFO:teuthology.orchestra.run.smithi106.stderr:listed 4 entries 2024-04-13T03:17:19.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.615+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 msgr2=0x7f60d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.615+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 0x7f60d406ede0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60f40cca10 tx=0x7f60e4008040 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.615+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 msgr2=0x7f60f40b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.615+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f60dc001eb0 tx=0x7f60dc00c890 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 shutdown_connections 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60d406c920 0x7f60d406ede0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f40b67d0 0x7f60f40cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60f415e9b0 0x7f60f40b6290 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 --2- 172.21.15.106:0/1611933624 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60f40b3d60 0x7f60f415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 >> 172.21.15.106:0/1611933624 conn(0x7f60f408bf10 msgr2=0x7f60f415c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 shutdown_connections 2024-04-13T03:17:19.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.616+0000 7f60f93f7640 1 -- 172.21.15.106:0/1611933624 wait complete. 2024-04-13T03:17:19.627 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:17:19.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:19.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:19.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:17:19.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:19.657 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:17:19.713 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:17:19.713 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:17:19.713 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:19.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:17:19.732 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:17:19.732 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:19.732 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:19.768 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:17:19.768 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:17:19.768 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:17:19.768 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:19.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:17:19.790 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.907+0000 7f71a0db4640 1 Processor -- start 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.907+0000 7f71a0db4640 1 -- start start 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f71a0db4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f719c0caf30 0x7f719c1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f71a0db4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f719c0d0bc0 0x7f719c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f71a0db4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719c0cd700 con 0x7f719c144f10 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f71a0db4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f719c0cd4d0 con 0x7f719c0caf30 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f71a0db4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f719c0cd9e0 con 0x7f719c0d0bc0 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7193fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f719c0caf30 0x7f719c1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f719a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f719c144f10 0x7f719c0ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50186/0 (socket says 172.21.15.106:50186) 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f719a59c640 1 -- 172.21.15.106:0/2073913584 learned_addr learned my addr 172.21.15.106:0/2073913584 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7193fff640 1 -- 172.21.15.106:0/2073913584 >> v2:172.21.15.114:3300/0 conn(0x7f719c0d0bc0 msgr2=0x7f719c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7193fff640 1 --2- 172.21.15.106:0/2073913584 >> v2:172.21.15.114:3300/0 conn(0x7f719c0d0bc0 0x7f719c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7193fff640 1 -- 172.21.15.106:0/2073913584 >> v1:172.21.15.106:6789/0 conn(0x7f719c144f10 legacy=0x7f719c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7199d9b640 1 -- 172.21.15.106:0/2073913584 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418295764 0 0) 0x7f719c0cd700 con 0x7f719c144f10 2024-04-13T03:17:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.908+0000 7f7193fff640 1 -- 172.21.15.106:0/2073913584 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c0cdcc0 con 0x7f719c0caf30 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f7193fff640 1 --2- 172.21.15.106:0/2073913584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0caf30 0x7f719c1630c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f719c162560 tx=0x7f718402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c554ba2f68e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f7199d9b640 1 -- 172.21.15.106:0/2073913584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184034070 con 0x7f719c0caf30 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f7199d9b640 1 -- 172.21.15.106:0/2073913584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7184030610 con 0x7f719c0caf30 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f7199d9b640 1 -- 172.21.15.106:0/2073913584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840309e0 con 0x7f719c0caf30 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 -- 172.21.15.106:0/2073913584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0caf30 msgr2=0x7f719c1630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 --2- 172.21.15.106:0/2073913584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0caf30 0x7f719c1630c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f719c162560 tx=0x7f718402b960 comp rx=0 tx=0).stop 2024-04-13T03:17:19.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 -- 172.21.15.106:0/2073913584 shutdown_connections 2024-04-13T03:17:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 --2- 172.21.15.106:0/2073913584 >> v2:172.21.15.114:3300/0 conn(0x7f719c0d0bc0 0x7f719c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 --2- 172.21.15.106:0/2073913584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0caf30 0x7f719c1630c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.909+0000 7f71a0db4640 1 -- 172.21.15.106:0/2073913584 >> 172.21.15.106:0/2073913584 conn(0x7f719c06f180 msgr2=0x7f719c06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.910+0000 7f71a0db4640 1 -- 172.21.15.106:0/2073913584 shutdown_connections 2024-04-13T03:17:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.910+0000 7f71a0db4640 1 -- 172.21.15.106:0/2073913584 wait complete. 2024-04-13T03:17:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.911+0000 7f71a0db4640 1 Processor -- start 2024-04-13T03:17:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.911+0000 7f71a0db4640 1 -- start start 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719c0caf30 0x7f719c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f719c144f10 0x7f719c160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f719c06c150 con 0x7f719c0d0bc0 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f719c0d0fd0 con 0x7f719c0caf30 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f719c06c2d0 con 0x7f719c144f10 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54298/0 (socket says 172.21.15.106:54298) 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 -- 172.21.15.106:0/1237294489 learned_addr learned my addr 172.21.15.106:0/1237294489 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f719a59c640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719c0caf30 0x7f719c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f719ad9d640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f719c144f10 0x7f719c160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 -- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f719c144f10 msgr2=0x7f719c160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f719c144f10 0x7f719c160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 -- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719c0caf30 msgr2=0x7f719c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719c0caf30 0x7f719c15e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 -- 172.21.15.106:0/1237294489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1f5660 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7193fff640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7184009890 tx=0x7f71840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184034040 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7184004030 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184004420 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.912+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1f48a0 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.913+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1f4e60 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.913+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c068a10 con 0x7f719c0d0bc0 2024-04-13T03:17:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.914+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7184037070 con 0x7f719c0d0bc0 2024-04-13T03:17:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.914+0000 7f7192ffd640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 0x7f717c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.914+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f71840ada80 con 0x7f719c0d0bc0 2024-04-13T03:17:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.914+0000 7f719a59c640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 0x7f717c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.915+0000 7f719a59c640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 0x7f717c06edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f718c007fd0 tx=0x7f718c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:19.919+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f718407b5b0 con 0x7f719c0d0bc0 2024-04-13T03:17:20.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.138+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f719c0cb720 con 0x7f719c0d0bc0 2024-04-13T03:17:20.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.139+0000 7f7192ffd640 1 -- 172.21.15.106:0/1237294489 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7f718407f8b0 con 0x7f719c0d0bc0 2024-04-13T03:17:20.140 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:20.140 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:17:20.140 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 msgr2=0x7f717c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 0x7f717c06edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f718c007fd0 tx=0x7f718c007de0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 msgr2=0x7f719c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7184009890 tx=0x7f71840046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 shutdown_connections 2024-04-13T03:17:20.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f717c06c8f0 0x7f717c06edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f719c144f10 0x7f719c160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c0d0bc0 0x7f719c1609c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 --2- 172.21.15.106:0/1237294489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719c0caf30 0x7f719c15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 >> 172.21.15.106:0/1237294489 conn(0x7f719c06f180 msgr2=0x7f719c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 shutdown_connections 2024-04-13T03:17:20.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.141+0000 7f71a0db4640 1 -- 172.21.15.106:0/1237294489 wait complete. 2024-04-13T03:17:20.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:20.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.253+0000 7ff87c9cc640 1 Processor -- start 2024-04-13T03:17:20.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87c9cc640 1 -- start start 2024-04-13T03:17:20.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87c9cc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8780b3d60 0x7ff8780d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87c9cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8780cfd20 con 0x7ff87815e9b0 2024-04-13T03:17:20.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87c9cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff878162af0 con 0x7ff87815ed90 2024-04-13T03:17:20.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87c9cc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff878162dd0 con 0x7ff8780b3d60 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff876d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8780b3d60 0x7ff8780d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff87815e9b0 0x7ff8780d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50192/0 (socket says 172.21.15.106:50192) 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.254+0000 7ff87659c640 1 -- 172.21.15.106:0/3475696407 learned_addr learned my addr 172.21.15.106:0/3475696407 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.255+0000 7ff876d9d640 1 -- 172.21.15.106:0/3475696407 >> v1:172.21.15.114:6789/0 conn(0x7ff87815ed90 legacy=0x7ff8780d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.255+0000 7ff876d9d640 1 -- 172.21.15.106:0/3475696407 >> v1:172.21.15.106:6789/0 conn(0x7ff87815e9b0 legacy=0x7ff8780d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.255+0000 7ff876d9d640 1 -- 172.21.15.106:0/3475696407 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781630b0 con 0x7ff8780b3d60 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.255+0000 7ff876d9d640 1 --2- 172.21.15.106:0/3475696407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8780b3d60 0x7ff8780d2300 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ff870009d90 tx=0x7ff87002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef345886ee7545b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff875d9b640 1 -- 172.21.15.106:0/3475696407 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870039070 con 0x7ff8780b3d60 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff875d9b640 1 -- 172.21.15.106:0/3475696407 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87002cd00 con 0x7ff8780b3d60 2024-04-13T03:17:20.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff875d9b640 1 -- 172.21.15.106:0/3475696407 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870034660 con 0x7ff8780b3d60 2024-04-13T03:17:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 -- 172.21.15.106:0/3475696407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8780b3d60 msgr2=0x7ff8780d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/3475696407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8780b3d60 0x7ff8780d2300 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ff870009d90 tx=0x7ff87002bf50 comp rx=0 tx=0).stop 2024-04-13T03:17:20.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 -- 172.21.15.106:0/3475696407 shutdown_connections 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/3475696407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8780b3d60 0x7ff8780d2300 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 -- 172.21.15.106:0/3475696407 >> 172.21.15.106:0/3475696407 conn(0x7ff87808bf10 msgr2=0x7ff8780b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 -- 172.21.15.106:0/3475696407 shutdown_connections 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.256+0000 7ff87c9cc640 1 -- 172.21.15.106:0/3475696407 wait complete. 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.257+0000 7ff87c9cc640 1 Processor -- start 2024-04-13T03:17:20.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.257+0000 7ff87c9cc640 1 -- start start 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8780b3d60 0x7ff87814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8780c8ef0 0x7ff87814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff878162800 con 0x7ff87815e9b0 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff878162680 con 0x7ff8780b3d60 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87c9cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff8780d4d00 con 0x7ff8780c8ef0 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54320/0 (socket says 172.21.15.106:54320) 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff86ffff640 1 -- 172.21.15.106:0/373913596 learned_addr learned my addr 172.21.15.106:0/373913596 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.258+0000 7ff87659c640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8780b3d60 0x7ff87814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff876d9d640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8780c8ef0 0x7ff87814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 -- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8780c8ef0 msgr2=0x7ff87814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8780c8ef0 0x7ff87814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 -- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8780b3d60 msgr2=0x7ff87814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8780b3d60 0x7ff87814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 -- 172.21.15.106:0/373913596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8780c9920 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86ffff640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3e0 tx=0x7ff86000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860015020 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff860011850 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860011c80 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8780c9c00 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.259+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8780ca1c0 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.260+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff878068a10 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.260+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff860018020 con 0x7ff87815e9b0 2024-04-13T03:17:20.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.261+0000 7ff86effd640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.261+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7ff86008baf0 con 0x7ff87815e9b0 2024-04-13T03:17:20.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.261+0000 7ff87659c640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.261+0000 7ff87659c640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 0x7ff85806ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff868000f60 tx=0x7ff868009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.265+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff860011e20 con 0x7ff87815e9b0 2024-04-13T03:17:20.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.441+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8780b8000 con 0x7ff87815e9b0 2024-04-13T03:17:20.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.442+0000 7ff86effd640 1 -- 172.21.15.106:0/373913596 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17375 (secure 0 0 0) 0x7ff860059c70 con 0x7ff87815e9b0 2024-04-13T03:17:20.443 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:20.443 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":95,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:16:40.106840+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 msgr2=0x7ff85806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 0x7ff85806ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff868000f60 tx=0x7ff868009040 comp rx=0 tx=0).stop 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 msgr2=0x7ff8780ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3e0 tx=0x7ff86000b8a0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 shutdown_connections 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8780c8ef0 0x7ff87814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff87815e9b0 0x7ff8780ca9c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 --2- 172.21.15.106:0/373913596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8780b3d60 0x7ff87814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 >> 172.21.15.106:0/373913596 conn(0x7ff87808bf10 msgr2=0x7ff87815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 shutdown_connections 2024-04-13T03:17:20.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.444+0000 7ff87c9cc640 1 -- 172.21.15.106:0/373913596 wait complete. 2024-04-13T03:17:20.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:15:17.537136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:15:18.258394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:20.456 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:20.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.558+0000 7f44ed5e9640 1 Processor -- start 2024-04-13T03:17:20.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.558+0000 7f44ed5e9640 1 -- start start 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44ed5e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e815ed90 0x7f44e80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44ed5e9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f44e80b3d60 0x7f44e80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44ed5e9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e80cfd20 con 0x7f44e815e9b0 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44ed5e9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f44e8162af0 con 0x7f44e80b3d60 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44ed5e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8162dd0 con 0x7f44e815ed90 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f44e815e9b0 0x7f44e80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50198/0 (socket says 172.21.15.106:50198) 2024-04-13T03:17:20.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e6ffd640 1 -- 172.21.15.106:0/1344859249 learned_addr learned my addr 172.21.15.106:0/1344859249 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883932373 0 0) 0x7f44e80cfd20 con 0x7f44e815e9b0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e77fe640 1 --2- 172.21.15.106:0/1344859249 >> v2:172.21.15.114:3300/0 conn(0x7f44e80b3d60 0x7f44e80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e67fc640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e815ed90 0x7f44e80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d0003520 con 0x7f44e815e9b0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2788963937 0 0) 0x7f44d0003520 con 0x7f44e815e9b0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e815ed90 msgr2=0x7f44e80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e815ed90 0x7f44e80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 >> v2:172.21.15.114:3300/0 conn(0x7f44e80b3d60 msgr2=0x7f44e80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 --2- 172.21.15.106:0/1344859249 >> v2:172.21.15.114:3300/0 conn(0x7f44e80b3d60 0x7f44e80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e81630b0 con 0x7f44e815e9b0 2024-04-13T03:17:20.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.559+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f44d8001690 con 0x7f44e815e9b0 2024-04-13T03:17:20.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 >> v1:172.21.15.106:6789/0 conn(0x7f44e815e9b0 legacy=0x7f44e80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44d00039b0 0x7f44d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44d0004310 0x7f44d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44d0004e50 0x7f44d00052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8162af0 con 0x7f44d0004e50 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8162dd0 con 0x7f44d0004310 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f44d000bdd0 con 0x7f44d00039b0 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44d0004e50 0x7f44d00052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e67fc640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44d0004310 0x7f44d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e6ffd640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44d00039b0 0x7f44d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 -- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44d00039b0 msgr2=0x7f44d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44d00039b0 0x7f44d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 -- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44d0004310 msgr2=0x7f44d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44d0004310 0x7f44d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e77fe640 1 -- 172.21.15.106:0/1344859249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0003520 con 0x7f44d0004e50 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.560+0000 7f44e6ffd640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44d00039b0 0x7f44d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44e77fe640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44d0004e50 0x7f44d00052d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009f90 tx=0x7f44d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8d294372ef4c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0039070 con 0x7f44d0004e50 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e002cc20 con 0x7f44d0004e50 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44e5ffb640 1 -- 172.21.15.106:0/1344859249 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0009c10 con 0x7f44d0004e50 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44ed5e9640 1 -- 172.21.15.106:0/1344859249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44d0004e50 msgr2=0x7f44d00052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.561+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/1344859249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44d0004e50 0x7f44d00052d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009f90 tx=0x7f44d0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.562+0000 7f44e77fe640 1 -- 172.21.15.106:0/1344859249 reap_dead start 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.562+0000 7f44ed5e9640 1 -- 172.21.15.106:0/1344859249 shutdown_connections 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.562+0000 7f44ed5e9640 1 -- 172.21.15.106:0/1344859249 >> 172.21.15.106:0/1344859249 conn(0x7f44e808bf10 msgr2=0x7f44e80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.562+0000 7f44ed5e9640 1 -- 172.21.15.106:0/1344859249 shutdown_connections 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.562+0000 7f44ed5e9640 1 -- 172.21.15.106:0/1344859249 wait complete. 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.563+0000 7f44ed5e9640 1 Processor -- start 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.563+0000 7f44ed5e9640 1 -- start start 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e81f8540 0x7f44e815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44e815bd40 0x7f44e81611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8162800 con 0x7f44e81f8160 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8162680 con 0x7f44e815bd40 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f44e80d4d00 con 0x7f44e81f8540 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54336/0 (socket says 172.21.15.106:54336) 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 -- 172.21.15.106:0/728412020 learned_addr learned my addr 172.21.15.106:0/728412020 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 -- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e81f8540 msgr2=0x7f44e815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e81f8540 0x7f44e815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 -- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44e815bd40 msgr2=0x7f44e81611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44e815bd40 0x7f44e81611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 -- 172.21.15.106:0/728412020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e815c1f0 con 0x7f44e81f8160 2024-04-13T03:17:20.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44e6ffd640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000fd0 tx=0x7f44d800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d800c850 con 0x7f44e81f8160 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8018070 con 0x7f44e81f8160 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e806cfb0 con 0x7f44e81f8160 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d800cc30 con 0x7f44e81f8160 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.564+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e806d570 con 0x7f44e81f8160 2024-04-13T03:17:20.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.565+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e80d13c0 con 0x7f44e81f8160 2024-04-13T03:17:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.566+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d801d460 con 0x7f44e81f8160 2024-04-13T03:17:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.566+0000 7f44cf7fe640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.566+0000 7f44e67fc640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.566+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f44d808bda0 con 0x7f44e81f8160 2024-04-13T03:17:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.567+0000 7f44e67fc640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 0x7f44d406ede0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f44e80b6f10 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.571+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44e80d13c0 con 0x7f44e81f8160 2024-04-13T03:17:20.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.746+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44e8068a10 con 0x7f44e81f8160 2024-04-13T03:17:20.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.746+0000 7f44cf7fe640 1 -- 172.21.15.106:0/728412020 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f44e815c1f0 con 0x7f44e81f8160 2024-04-13T03:17:20.747 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:20.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 msgr2=0x7f44d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 0x7f44d406ede0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f44e80b6f10 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 msgr2=0x7f44e806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000fd0 tx=0x7f44d800ef80 comp rx=0 tx=0).stop 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 shutdown_connections 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f44d406c920 0x7f44d406ede0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f44e81f8540 0x7f44e815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44e815bd40 0x7f44e81611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 --2- 172.21.15.106:0/728412020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e81f8160 0x7f44e806e560 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 >> 172.21.15.106:0/728412020 conn(0x7f44e808bf10 msgr2=0x7f44e80ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 shutdown_connections 2024-04-13T03:17:20.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.748+0000 7f44ed5e9640 1 -- 172.21.15.106:0/728412020 wait complete. 2024-04-13T03:17:20.760 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:17:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.864+0000 7f019f236640 1 Processor -- start 2024-04-13T03:17:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.864+0000 7f019f236640 1 -- start start 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019f236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f019815e8c0 0x7f01980ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019f236640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 0x7f0198072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019f236640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01980b5060 con 0x7f0198073350 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019f236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0198073780 con 0x7f019815e8c0 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019f236640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f01980b5340 con 0x7f019815eca0 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019cfd2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f019815e8c0 0x7f01980ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019d7d3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0198073350 0x7f01980b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50208/0 (socket says 172.21.15.106:50208) 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019d7d3640 1 -- 172.21.15.106:0/3427919444 learned_addr learned my addr 172.21.15.106:0/3427919444 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:20.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f0197fff640 1 --2- 172.21.15.106:0/3427919444 >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 0x7f0198072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019cfd2640 1 -- 172.21.15.106:0/3427919444 >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 msgr2=0x7f0198072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3427919444 >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 0x7f0198072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019cfd2640 1 -- 172.21.15.106:0/3427919444 >> v1:172.21.15.106:6789/0 conn(0x7f0198073350 legacy=0x7f01980b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f019cfd2640 1 -- 172.21.15.106:0/3427919444 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019806c500 con 0x7f019815e8c0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.865+0000 7f0197fff640 1 --2- 172.21.15.106:0/3427919444 >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 0x7f0198072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.866+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3427919444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815e8c0 0x7f01980ce250 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0198072ae0 tx=0x7f019002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23499e76df5f193f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.866+0000 7f01977fe640 1 -- 172.21.15.106:0/3427919444 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190039070 con 0x7f019815e8c0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.866+0000 7f01977fe640 1 -- 172.21.15.106:0/3427919444 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019002cd00 con 0x7f019815e8c0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.866+0000 7f01977fe640 1 -- 172.21.15.106:0/3427919444 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190034660 con 0x7f019815e8c0 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 -- 172.21.15.106:0/3427919444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815e8c0 msgr2=0x7f01980ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 --2- 172.21.15.106:0/3427919444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815e8c0 0x7f01980ce250 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0198072ae0 tx=0x7f019002bf50 comp rx=0 tx=0).stop 2024-04-13T03:17:20.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 -- 172.21.15.106:0/3427919444 shutdown_connections 2024-04-13T03:17:20.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 --2- 172.21.15.106:0/3427919444 >> v2:172.21.15.114:3300/0 conn(0x7f019815eca0 0x7f0198072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 --2- 172.21.15.106:0/3427919444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815e8c0 0x7f01980ce250 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 -- 172.21.15.106:0/3427919444 >> 172.21.15.106:0/3427919444 conn(0x7f019806f0d0 msgr2=0x7f019806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:20.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 -- 172.21.15.106:0/3427919444 shutdown_connections 2024-04-13T03:17:20.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.867+0000 7f019f236640 1 -- 172.21.15.106:0/3427919444 wait complete. 2024-04-13T03:17:20.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.868+0000 7f019f236640 1 Processor -- start 2024-04-13T03:17:20.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.868+0000 7f019f236640 1 -- start start 2024-04-13T03:17:20.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 0x7f019814b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 0x7f019815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01980b4cc0 con 0x7f019815eca0 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01980b4b40 con 0x7f019815e8c0 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f01980b4e40 con 0x7f0198073350 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54356/0 (socket says 172.21.15.106:54356) 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 -- 172.21.15.106:0/3597943917 learned_addr learned my addr 172.21.15.106:0/3597943917 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 0x7f019814b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f0197fff640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 0x7f019815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 -- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 msgr2=0x7f019814b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 0x7f019814b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 -- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 msgr2=0x7f019815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 0x7f019815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 -- 172.21.15.106:0/3597943917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0198158700 con 0x7f019815eca0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f0197fff640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 0x7f019815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 0x7f019814b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019d7d3640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f018800c3c0 tx=0x7f018800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188017070 con 0x7f019815eca0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188005ce0 con 0x7f019815eca0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019814b480 con 0x7f019815eca0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01880072f0 con 0x7f019815eca0 2024-04-13T03:17:20.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.869+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019815b6f0 con 0x7f019815eca0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.870+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019806f620 con 0x7f019815eca0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.871+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f018801a070 con 0x7f019815eca0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.871+0000 7f01957fa640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 0x7f018406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.871+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f018808b0c0 con 0x7f019815eca0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.871+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 0x7f018406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.872+0000 7f019cfd2640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 0x7f018406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0190036000 tx=0x7f0190030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:20.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:20.876+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f019806f620 con 0x7f019815eca0 2024-04-13T03:17:21.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.048+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0198068a10 con 0x7f019815eca0 2024-04-13T03:17:21.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.049+0000 7f01957fa640 1 -- 172.21.15.106:0/3597943917 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f019814b480 con 0x7f019815eca0 2024-04-13T03:17:21.050 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 msgr2=0x7f018406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 0x7f018406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0190036000 tx=0x7f0190030000 comp rx=0 tx=0).stop 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 msgr2=0x7f019814af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f018800c3c0 tx=0x7f018800c880 comp rx=0 tx=0).stop 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 shutdown_connections 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f018406c8f0 0x7f018406edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f019815eca0 0x7f019814af40 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f019815e8c0 0x7f019815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 --2- 172.21.15.106:0/3597943917 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0198073350 0x7f019814b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 >> 172.21.15.106:0/3597943917 conn(0x7f019806f0d0 msgr2=0x7f0198158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 shutdown_connections 2024-04-13T03:17:21.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.051+0000 7f019f236640 1 -- 172.21.15.106:0/3597943917 wait complete. 2024-04-13T03:17:21.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:17:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.166+0000 7f493ea0f640 1 Processor -- start 2024-04-13T03:17:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.166+0000 7f493ea0f640 1 -- start start 2024-04-13T03:17:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493ea0f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f493815ea80 0x7f49381624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493ea0f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49380aeae0 0x7f4938162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493ea0f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49380b3fc0 con 0x7f49381629f0 2024-04-13T03:17:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493ea0f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f49380b42a0 con 0x7f49380aeae0 2024-04-13T03:17:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493ea0f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f49380b4580 con 0x7f493815ea80 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49380aeae0 0x7f4938162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f49377fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f49381629f0 0x7f493806ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50220/0 (socket says 172.21.15.106:50220) 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49380aeae0 0x7f4938162e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54362/0 (socket says 172.21.15.106:54362) 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f49377fe640 1 -- 172.21.15.106:0/4196129346 learned_addr learned my addr 172.21.15.106:0/4196129346 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f4937fff640 1 --2- 172.21.15.106:0/4196129346 >> v2:172.21.15.114:3300/0 conn(0x7f493815ea80 0x7f49381624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 -- 172.21.15.106:0/4196129346 >> v2:172.21.15.114:3300/0 conn(0x7f493815ea80 msgr2=0x7f49381624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 --2- 172.21.15.106:0/4196129346 >> v2:172.21.15.114:3300/0 conn(0x7f493815ea80 0x7f49381624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 -- 172.21.15.106:0/4196129346 >> v1:172.21.15.106:6789/0 conn(0x7f49381629f0 legacy=0x7f493806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f493cfac640 1 -- 172.21.15.106:0/4196129346 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493806e930 con 0x7f49380aeae0 2024-04-13T03:17:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.167+0000 7f4936ffd640 1 -- 172.21.15.106:0/4196129346 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59507024 0 0) 0x7f49380b3fc0 con 0x7f49381629f0 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.168+0000 7f493cfac640 1 --2- 172.21.15.106:0/4196129346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938162e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f49380d0b80 tx=0x7f493002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc57255e49654e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.168+0000 7f4936ffd640 1 -- 172.21.15.106:0/4196129346 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930039070 con 0x7f49380aeae0 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.168+0000 7f4936ffd640 1 -- 172.21.15.106:0/4196129346 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493002cc80 con 0x7f49380aeae0 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.168+0000 7f4936ffd640 1 -- 172.21.15.106:0/4196129346 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930034660 con 0x7f49380aeae0 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 -- 172.21.15.106:0/4196129346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 msgr2=0x7f4938162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 --2- 172.21.15.106:0/4196129346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938162e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f49380d0b80 tx=0x7f493002bed0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 -- 172.21.15.106:0/4196129346 shutdown_connections 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 --2- 172.21.15.106:0/4196129346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938162e50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 --2- 172.21.15.106:0/4196129346 >> v2:172.21.15.114:3300/0 conn(0x7f493815ea80 0x7f49381624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 -- 172.21.15.106:0/4196129346 >> 172.21.15.106:0/4196129346 conn(0x7f49380b2600 msgr2=0x7f49380d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 -- 172.21.15.106:0/4196129346 shutdown_connections 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.169+0000 7f493ea0f640 1 -- 172.21.15.106:0/4196129346 wait complete. 2024-04-13T03:17:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.170+0000 7f493ea0f640 1 Processor -- start 2024-04-13T03:17:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.170+0000 7f493ea0f640 1 -- start start 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 0x7f49380cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 0x7f493806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f493806f6e0 con 0x7f49380aeae0 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f493806f560 con 0x7f49381629f0 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f493806f860 con 0x7f493815ea80 2024-04-13T03:17:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54368/0 (socket says 172.21.15.106:54368) 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 -- 172.21.15.106:0/262603387 learned_addr learned my addr 172.21.15.106:0/262603387 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493cfac640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 0x7f493806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f49377fe640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 0x7f49380cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 -- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 msgr2=0x7f49380cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 0x7f49380cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 -- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 msgr2=0x7f493806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 0x7f493806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 -- 172.21.15.106:0/262603387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49380cd0f0 con 0x7f49380aeae0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493cfac640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 0x7f493806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f49377fe640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 0x7f49380cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4937fff640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f492800d830 tx=0x7f492800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800b6c0 con 0x7f49380aeae0 2024-04-13T03:17:21.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4928019070 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49381f7960 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800ba80 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.171+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49381f7f20 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.172+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4938068a10 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.173+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f492801e460 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.173+0000 7f4934ff9640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 0x7f492406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.173+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f492808c4e0 con 0x7f49380aeae0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.173+0000 7f49377fe640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 0x7f492406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.174+0000 7f49377fe640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 0x7f492406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49380d05a0 tx=0x7f492c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.178+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f492805a660 con 0x7f49380aeae0 2024-04-13T03:17:21.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.353+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f493806e2a0 con 0x7f49380aeae0 2024-04-13T03:17:21.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.354+0000 7f4934ff9640 1 -- 172.21.15.106:0/262603387 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f4928059f60 con 0x7f49380aeae0 2024-04-13T03:17:21.355 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 msgr2=0x7f492406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 0x7f492406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49380d05a0 tx=0x7f492c009040 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 msgr2=0x7f4938154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f492800d830 tx=0x7f492800dcf0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 shutdown_connections 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492406c8f0 0x7f492406edb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493815ea80 0x7f49380cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49381629f0 0x7f493806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 --2- 172.21.15.106:0/262603387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49380aeae0 0x7f4938154040 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 >> 172.21.15.106:0/262603387 conn(0x7f49380b2600 msgr2=0x7f49380aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 shutdown_connections 2024-04-13T03:17:21.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.356+0000 7f493ea0f640 1 -- 172.21.15.106:0/262603387 wait complete. 2024-04-13T03:17:21.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:21.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.466+0000 7fd1089c8640 1 Processor -- start 2024-04-13T03:17:21.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.466+0000 7fd1089c8640 1 -- start start 2024-04-13T03:17:21.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.467+0000 7fd1089c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd10406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.467+0000 7fd1089c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd1040b5bf0 0x7fd10406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.467+0000 7fd1089c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1040ce6f0 con 0x7fd1040ce1f0 2024-04-13T03:17:21.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.467+0000 7fd1089c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd10415fa70 con 0x7fd1040b5bf0 2024-04-13T03:17:21.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.467+0000 7fd1089c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd10415f420 con 0x7fd10415ebf0 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd102d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd1040b5bf0 0x7fd10406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd101d9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd1040ce1f0 0x7fd10406e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:37216/0 (socket says 172.21.15.106:37216) 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd101d9b640 1 -- 172.21.15.106:0/3689283611 learned_addr learned my addr 172.21.15.106:0/3689283611 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd10259c640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd10406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd102d9d640 1 -- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 msgr2=0x7fd10406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd102d9d640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd10406f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd102d9d640 1 -- 172.21.15.106:0/3689283611 >> v1:172.21.15.114:6789/0 conn(0x7fd1040ce1f0 legacy=0x7fd10406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd102d9d640 1 -- 172.21.15.106:0/3689283611 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10415f700 con 0x7fd1040b5bf0 2024-04-13T03:17:21.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.468+0000 7fd10259c640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd10406f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.469+0000 7fd102d9d640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd10406fcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097a0 tx=0x7fd0fc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e503ffe33f3d6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.469+0000 7fd10159a640 1 -- 172.21.15.106:0/3689283611 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0315a0 con 0x7fd1040b5bf0 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.469+0000 7fd10159a640 1 -- 172.21.15.106:0/3689283611 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0fc037070 con 0x7fd1040b5bf0 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.469+0000 7fd10159a640 1 -- 172.21.15.106:0/3689283611 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc031960 con 0x7fd1040b5bf0 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 -- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 msgr2=0x7fd10406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd10406fcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097a0 tx=0x7fd0fc02b840 comp rx=0 tx=0).stop 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 -- 172.21.15.106:0/3689283611 shutdown_connections 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd10406fcc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 --2- 172.21.15.106:0/3689283611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd10406f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 -- 172.21.15.106:0/3689283611 >> 172.21.15.106:0/3689283611 conn(0x7fd1040cd6d0 msgr2=0x7fd1040cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 -- 172.21.15.106:0/3689283611 shutdown_connections 2024-04-13T03:17:21.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.470+0000 7fd1089c8640 1 -- 172.21.15.106:0/3689283611 wait complete. 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.471+0000 7fd1089c8640 1 Processor -- start 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.471+0000 7fd1089c8640 1 -- start start 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd1041f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd1040ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040b0650 con 0x7fd1040ce1f0 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040b04d0 con 0x7fd1040b5bf0 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd1089c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd10415efd0 con 0x7fd10415ebf0 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54370/0 (socket says 172.21.15.106:54370) 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 -- 172.21.15.106:0/578260171 learned_addr learned my addr 172.21.15.106:0/578260171 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:21.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd102d9d640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd1040ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd10259c640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd1041f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 -- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 msgr2=0x7fd1040ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd1040ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 -- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 msgr2=0x7fd1041f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd1041f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 -- 172.21.15.106:0/578260171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10415c640 con 0x7fd1040ce1f0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.472+0000 7fd101d9b640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800c400 tx=0x7fd0f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.473+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8017070 con 0x7fd1040ce1f0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.473+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f8005ce0 con 0x7fd1040ce1f0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.473+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10415c920 con 0x7fd1040ce1f0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.473+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80072f0 con 0x7fd1040ce1f0 2024-04-13T03:17:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.473+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10415cee0 con 0x7fd1040ce1f0 2024-04-13T03:17:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.474+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104068a10 con 0x7fd1040ce1f0 2024-04-13T03:17:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.474+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0f801a070 con 0x7fd1040ce1f0 2024-04-13T03:17:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.474+0000 7fd0f2ffd640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.474+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fd0f808b070 con 0x7fd1040ce1f0 2024-04-13T03:17:21.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.475+0000 7fd10259c640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.475+0000 7fd10259c640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 0x7fd0e406edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009bd0 tx=0x7fd0f400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.479+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0f8058c30 con 0x7fd1040ce1f0 2024-04-13T03:17:21.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.690+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1040cdd00 con 0x7fd1040ce1f0 2024-04-13T03:17:21.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.691+0000 7fd0f2ffd640 1 -- 172.21.15.106:0/578260171 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7fd0f805cf30 con 0x7fd1040ce1f0 2024-04-13T03:17:21.692 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:21.692 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:17:21.692 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 msgr2=0x7fd0e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 0x7fd0e406edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009bd0 tx=0x7fd0f400b040 comp rx=0 tx=0).stop 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 msgr2=0x7fd104072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800c400 tx=0x7fd0f800c8c0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 shutdown_connections 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd10415ebf0 0x7fd1040ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1040ce1f0 0x7fd104072ba0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 --2- 172.21.15.106:0/578260171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1040b5bf0 0x7fd1041f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 >> 172.21.15.106:0/578260171 conn(0x7fd1040cd6d0 msgr2=0x7fd104159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 shutdown_connections 2024-04-13T03:17:21.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.693+0000 7fd1089c8640 1 -- 172.21.15.106:0/578260171 wait complete. 2024-04-13T03:17:21.705 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:21.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.804+0000 7f93516de640 1 Processor -- start 2024-04-13T03:17:21.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.804+0000 7f93516de640 1 -- start start 2024-04-13T03:17:21.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f93516de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f93516de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f934c0b3d60 0x7f934c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f93516de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934c0cfd20 con 0x7f934c15e9b0 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f93516de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f934c162af0 con 0x7f934c0b3d60 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f93516de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f934c162dd0 con 0x7f934c15ed90 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934b7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f934c0b3d60 0x7f934c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:59610/0 (socket says 172.21.15.106:59610) 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 -- 172.21.15.106:0/2693114241 learned_addr learned my addr 172.21.15.106:0/2693114241 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934affd640 1 --1- 172.21.15.106:0/2693114241 >> v1:172.21.15.114:6789/0 conn(0x7f934c15e9b0 0x7f934c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:37226/0 (socket says 172.21.15.106:37226) 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 -- 172.21.15.106:0/2693114241 >> v2:172.21.15.114:3300/0 conn(0x7f934c0b3d60 msgr2=0x7f934c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 --2- 172.21.15.106:0/2693114241 >> v2:172.21.15.114:3300/0 conn(0x7f934c0b3d60 0x7f934c0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 -- 172.21.15.106:0/2693114241 >> v1:172.21.15.114:6789/0 conn(0x7f934c15e9b0 legacy=0x7f934c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.805+0000 7f934a7fc640 1 -- 172.21.15.106:0/2693114241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c1630b0 con 0x7f934c15ed90 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.806+0000 7f934a7fc640 1 --2- 172.21.15.106:0/2693114241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9340009850 tx=0x7f934002b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc7a17283551274 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.806+0000 7f9349ffb640 1 -- 172.21.15.106:0/2693114241 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340004230 con 0x7f934c15ed90 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.806+0000 7f9349ffb640 1 -- 172.21.15.106:0/2693114241 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93400043d0 con 0x7f934c15ed90 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.806+0000 7f9349ffb640 1 -- 172.21.15.106:0/2693114241 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340003810 con 0x7f934c15ed90 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 -- 172.21.15.106:0/2693114241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 msgr2=0x7f934c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 --2- 172.21.15.106:0/2693114241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9340009850 tx=0x7f934002b9f0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 -- 172.21.15.106:0/2693114241 shutdown_connections 2024-04-13T03:17:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 --2- 172.21.15.106:0/2693114241 >> v2:172.21.15.114:3300/0 conn(0x7f934c0b3d60 0x7f934c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 --2- 172.21.15.106:0/2693114241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15ed90 0x7f934c0d2300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 -- 172.21.15.106:0/2693114241 >> 172.21.15.106:0/2693114241 conn(0x7f934c08bf10 msgr2=0x7f934c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 -- 172.21.15.106:0/2693114241 shutdown_connections 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.807+0000 7f93516de640 1 -- 172.21.15.106:0/2693114241 wait complete. 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.808+0000 7f93516de640 1 Processor -- start 2024-04-13T03:17:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.808+0000 7f93516de640 1 -- start start 2024-04-13T03:17:21.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15e9b0 0x7f934c0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f934c0b75c0 0x7f934c158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f934c162800 con 0x7f934c0b3d60 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f934c162680 con 0x7f934c0b75c0 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f93516de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f934c0d4d00 con 0x7f934c15e9b0 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f934affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f934affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54386/0 (socket says 172.21.15.106:54386) 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f934affd640 1 -- 172.21.15.106:0/1901099717 learned_addr learned my addr 172.21.15.106:0/1901099717 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f934b7fe640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f934c0b75c0 0x7f934c158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.809+0000 7f934a7fc640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15e9b0 0x7f934c0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 -- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15e9b0 msgr2=0x7f934c0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15e9b0 0x7f934c0c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 -- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f934c0b75c0 msgr2=0x7f934c158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f934c0b75c0 0x7f934c158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 -- 172.21.15.106:0/1901099717 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0b42e0 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f934affd640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c400 tx=0x7f933c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c0074e0 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c0047e0 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c0b4580 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c004c10 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.810+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c0b4b40 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.811+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c068a10 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.811+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f933c01a020 con 0x7f934c0b3d60 2024-04-13T03:17:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.812+0000 7f93337fe640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:21.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.812+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f933c08b510 con 0x7f934c0b3d60 2024-04-13T03:17:21.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.812+0000 7f934a7fc640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:21.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.812+0000 7f934a7fc640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 0x7f933806edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9340035000 tx=0x7f9340008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:21.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.816+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f933c059020 con 0x7f934c0b3d60 2024-04-13T03:17:21.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.990+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f934c0d53a0 con 0x7f934c0b3d60 2024-04-13T03:17:21.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.991+0000 7f93337fe640 1 -- 172.21.15.106:0/1901099717 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17375 (secure 0 0 0) 0x7f933c05d320 con 0x7f934c0b3d60 2024-04-13T03:17:21.992 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:21.992 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":95,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:16:40.106840+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:21.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 msgr2=0x7f933806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 0x7f933806edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9340035000 tx=0x7f9340008030 comp rx=0 tx=0).stop 2024-04-13T03:17:21.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 msgr2=0x7f934c161d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:21.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c400 tx=0x7f933c00c8c0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 shutdown_connections 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f934c15e9b0 0x7f934c0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f934c0b75c0 0x7f934c158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 --2- 172.21.15.106:0/1901099717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934c0b3d60 0x7f934c161d20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 >> 172.21.15.106:0/1901099717 conn(0x7f934c08bf10 msgr2=0x7f934c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 shutdown_connections 2024-04-13T03:17:21.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:21.993+0000 7f93516de640 1 -- 172.21.15.106:0/1901099717 wait complete. 2024-04-13T03:17:22.006 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:17:22.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:22.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.105+0000 7f84e1ec8640 1 Processor -- start 2024-04-13T03:17:22.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.105+0000 7f84e1ec8640 1 -- start start 2024-04-13T03:17:22.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 0x7f84dc162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc15ed60 0x7f84dc0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f84dc0cd380 0x7f84dc0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc0cd7f0 con 0x7f84dc0cd380 2024-04-13T03:17:22.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc1633f0 con 0x7f84dc15e980 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84e1ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc15fe40 con 0x7f84dc15ed60 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f84dc0cd380 0x7f84dc0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f84dc0cd380 0x7f84dc0d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54394/0 (socket says 172.21.15.106:54394) 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.106+0000 7f84dbfff640 1 -- 172.21.15.106:0/1395329881 learned_addr learned my addr 172.21.15.106:0/1395329881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84db7fe640 1 --2- 172.21.15.106:0/1395329881 >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 0x7f84dc162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84d2ffd640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc15ed60 0x7f84dc0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 -- 172.21.15.106:0/1395329881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc15ed60 msgr2=0x7f84dc0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc15ed60 0x7f84dc0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 -- 172.21.15.106:0/1395329881 >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 msgr2=0x7f84dc162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 --2- 172.21.15.106:0/1395329881 >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 0x7f84dc162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 -- 172.21.15.106:0/1395329881 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc15f2c0 con 0x7f84dc0cd380 2024-04-13T03:17:22.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84db7fe640 1 --2- 172.21.15.106:0/1395329881 >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 0x7f84dc162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:22.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84dbfff640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc0cd380 0x7f84dc0d06e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0d31f0 tx=0x7f84d402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10e69408e0930f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.107+0000 7f84daffd640 1 -- 172.21.15.106:0/1395329881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d402cb00 con 0x7f84dc0cd380 2024-04-13T03:17:22.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84daffd640 1 -- 172.21.15.106:0/1395329881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d402cca0 con 0x7f84dc0cd380 2024-04-13T03:17:22.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84daffd640 1 -- 172.21.15.106:0/1395329881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4034610 con 0x7f84dc0cd380 2024-04-13T03:17:22.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 -- 172.21.15.106:0/1395329881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc0cd380 msgr2=0x7f84dc0d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc0cd380 0x7f84dc0d06e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0d31f0 tx=0x7f84d402bf60 comp rx=0 tx=0).stop 2024-04-13T03:17:22.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 -- 172.21.15.106:0/1395329881 shutdown_connections 2024-04-13T03:17:22.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc0cd380 0x7f84dc0d06e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/1395329881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc15ed60 0x7f84dc0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/1395329881 >> v2:172.21.15.114:3300/0 conn(0x7f84dc15e980 0x7f84dc162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 -- 172.21.15.106:0/1395329881 >> 172.21.15.106:0/1395329881 conn(0x7f84dc0d2550 msgr2=0x7f84dc0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 -- 172.21.15.106:0/1395329881 shutdown_connections 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.108+0000 7f84e1ec8640 1 -- 172.21.15.106:0/1395329881 wait complete. 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.109+0000 7f84e1ec8640 1 Processor -- start 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.109+0000 7f84e1ec8640 1 -- start start 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc0cd380 0x7f84dc0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84dc15ed60 0x7f84dc1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc15fb60 con 0x7f84dc15e980 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc15f880 con 0x7f84dc15ed60 2024-04-13T03:17:22.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84e1ec8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc0d1e20 con 0x7f84dc0cd380 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54396/0 (socket says 172.21.15.106:54396) 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84d2ffd640 1 -- 172.21.15.106:0/3543692404 learned_addr learned my addr 172.21.15.106:0/3543692404 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84db7fe640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc0cd380 0x7f84dc0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.110+0000 7f84dbfff640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84dc15ed60 0x7f84dc1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 -- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc0cd380 msgr2=0x7f84dc0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc0cd380 0x7f84dc0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 -- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84dc15ed60 msgr2=0x7f84dc1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84dc15ed60 0x7f84dc1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 -- 172.21.15.106:0/3543692404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc0ccae0 con 0x7f84dc15e980 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d2ffd640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f84c400d5c0 tx=0x7f84c400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40125f0 con 0x7f84dc15e980 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c4019070 con 0x7f84dc15e980 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40129d0 con 0x7f84dc15e980 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc0b56e0 con 0x7f84dc15e980 2024-04-13T03:17:22.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.111+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc0b5f80 con 0x7f84dc15e980 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.112+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc0d21c0 con 0x7f84dc15e980 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.112+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84c4012b70 con 0x7f84dc15e980 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.113+0000 7f84d8ff9640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 0x7f84bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.113+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f84c408c000 con 0x7f84dc15e980 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.113+0000 7f84db7fe640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 0x7f84bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.113+0000 7f84db7fe640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 0x7f84bc06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009950 tx=0x7f84cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.117+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84dc0d21c0 con 0x7f84dc15e980 2024-04-13T03:17:22.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.331+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84dc069120 con 0x7f84dc15e980 2024-04-13T03:17:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.332+0000 7f84d8ff9640 1 -- 172.21.15.106:0/3543692404 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7f84c4059bc0 con 0x7f84dc15e980 2024-04-13T03:17:22.333 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:22.333 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:17:22.333 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 msgr2=0x7f84bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 0x7f84bc06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009950 tx=0x7f84cc008040 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 msgr2=0x7f84dc0cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f84c400d5c0 tx=0x7f84c400da80 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 shutdown_connections 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84bc06c8f0 0x7f84bc06edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84dc15ed60 0x7f84dc1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84dc15e980 0x7f84dc0cc5a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 --2- 172.21.15.106:0/3543692404 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84dc0cd380 0x7f84dc0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 >> 172.21.15.106:0/3543692404 conn(0x7f84dc0d2550 msgr2=0x7f84dc15d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 shutdown_connections 2024-04-13T03:17:22.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.334+0000 7f84e1ec8640 1 -- 172.21.15.106:0/3543692404 wait complete. 2024-04-13T03:17:22.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.445+0000 7fac50188640 1 Processor -- start 2024-04-13T03:17:22.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.445+0000 7fac50188640 1 -- start start 2024-04-13T03:17:22.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac50188640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fac4806e8d0 0x7fac48162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac50188640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac480cf6c0 con 0x7fac480ca1c0 2024-04-13T03:17:22.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac50188640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac480cf9a0 con 0x7fac4815ea40 2024-04-13T03:17:22.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac50188640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fac480cfc80 con 0x7fac4806e8d0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4d723640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fac480ca1c0 0x7fac4806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50226/0 (socket says 172.21.15.106:50226) 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4d723640 1 -- 172.21.15.106:0/922782319 learned_addr learned my addr 172.21.15.106:0/922782319 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256383142 0 0) 0x7fac480cf6c0 con 0x7fac480ca1c0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4e725640 1 --2- 172.21.15.106:0/922782319 >> v2:172.21.15.114:3300/0 conn(0x7fac4806e8d0 0x7fac48162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac34003520 con 0x7fac480ca1c0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 79407640 0 0) 0x7fac34003520 con 0x7fac480ca1c0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 >> v2:172.21.15.114:3300/0 conn(0x7fac4806e8d0 msgr2=0x7fac48162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 --2- 172.21.15.106:0/922782319 >> v2:172.21.15.114:3300/0 conn(0x7fac4806e8d0 0x7fac48162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 >> v1:172.21.15.114:6789/0 conn(0x7fac4815ea40 legacy=0x7fac480c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac480cff60 con 0x7fac480ca1c0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fac38001670 con 0x7fac480ca1c0 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 >> v1:172.21.15.106:6789/0 conn(0x7fac480ca1c0 legacy=0x7fac4806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac340039b0 0x7fac34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 0x7fac34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac340051c0 0x7fac3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac480cfc80 con 0x7fac34004800 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac3400c160 con 0x7fac340051c0 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.446+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fac3400c360 con 0x7fac340039b0 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 0x7fac34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4df24640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac340039b0 0x7fac34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4e725640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac340051c0 0x7fac3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 -- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac340039b0 msgr2=0x7fac34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac340039b0 0x7fac34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 -- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac340051c0 msgr2=0x7fac3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac340051c0 0x7fac3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4e725640 1 -- 172.21.15.106:0/922782319 reap_dead start 2024-04-13T03:17:22.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4d723640 1 -- 172.21.15.106:0/922782319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac34003520 con 0x7fac34004800 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.447+0000 7fac4df24640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac340039b0 0x7fac34003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac4d723640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 0x7fac34004c80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c210 tx=0x7fac34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a0c479930450ec server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38007d30 con 0x7fac34004800 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac38030070 con 0x7fac34004800 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac4cf22640 1 -- 172.21.15.106:0/922782319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38005d70 con 0x7fac34004800 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 -- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 msgr2=0x7fac34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 0x7fac34004c80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c210 tx=0x7fac34005830 comp rx=0 tx=0).stop 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 -- 172.21.15.106:0/922782319 shutdown_connections 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 --2- 172.21.15.106:0/922782319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac34004800 0x7fac34004c80 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 -- 172.21.15.106:0/922782319 >> 172.21.15.106:0/922782319 conn(0x7fac480cd1e0 msgr2=0x7fac480cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 -- 172.21.15.106:0/922782319 shutdown_connections 2024-04-13T03:17:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.448+0000 7fac50188640 1 -- 172.21.15.106:0/922782319 wait complete. 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.449+0000 7fac50188640 1 Processor -- start 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.449+0000 7fac50188640 1 -- start start 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac34004800 0x7fac4814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 0x7fac480cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac480d0480 con 0x7fac480cc000 2024-04-13T03:17:22.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac480cf370 con 0x7fac4815c890 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac50188640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fac48162eb0 con 0x7fac34004800 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4e725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4e725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54414/0 (socket says 172.21.15.106:54414) 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4e725640 1 -- 172.21.15.106:0/1997681634 learned_addr learned my addr 172.21.15.106:0/1997681634 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4d723640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 0x7fac480cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4df24640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac34004800 0x7fac4814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4e725640 1 -- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac34004800 msgr2=0x7fac4814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.450+0000 7fac4e725640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac34004800 0x7fac4814c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac4e725640 1 -- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 msgr2=0x7fac480cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac4e725640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 0x7fac480cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac4e725640 1 -- 172.21.15.106:0/1997681634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4815bad0 con 0x7fac480cc000 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac4d723640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 0x7fac480cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac4e725640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fac40011bf0 tx=0x7fac4000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4000fdb0 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac40015430 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40015820 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4815bdb0 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.451+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4815c370 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.452+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4815c570 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.452+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac400159c0 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.453+0000 7fac3e7fc640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 0x7fac2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.453+0000 7fac4df24640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 0x7fac2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.453+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fac4008ccb0 con 0x7fac480cc000 2024-04-13T03:17:22.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.453+0000 7fac4df24640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 0x7fac2c06ede0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.457+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac4815c570 con 0x7fac480cc000 2024-04-13T03:17:22.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.670+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac48068a10 con 0x7fac480cc000 2024-04-13T03:17:22.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.671+0000 7fac3e7fc640 1 -- 172.21.15.106:0/1997681634 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5196 (secure 0 0 0) 0x7fac40019030 con 0x7fac480cc000 2024-04-13T03:17:22.672 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:22.672 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:15:19.300957+0000","modified":"2024-04-13T03:16:44.123357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8028":{"gid":8028,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/2398448953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2398448953},{"type":"v1","addr":"172.21.15.106:6837","nonce":2398448953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8028,"qdb_cluster":[8028]},"id":6}]} 2024-04-13T03:17:22.672 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 64 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 msgr2=0x7fac2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 0x7fac2c06ede0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 msgr2=0x7fac480b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fac40011bf0 tx=0x7fac4000f580 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 shutdown_connections 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fac2c06c920 0x7fac2c06ede0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac480cc000 0x7fac480b56f0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac4815c890 0x7fac480cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 --2- 172.21.15.106:0/1997681634 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fac34004800 0x7fac4814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 >> 172.21.15.106:0/1997681634 conn(0x7fac480cd1e0 msgr2=0x7fac4806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 shutdown_connections 2024-04-13T03:17:22.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.673+0000 7fac50188640 1 -- 172.21.15.106:0/1997681634 wait complete. 2024-04-13T03:17:22.685 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:22.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.784+0000 7f156cc08640 1 Processor -- start 2024-04-13T03:17:22.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.784+0000 7f156cc08640 1 -- start start 2024-04-13T03:17:22.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156cc08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1568156b50 0x7f1568160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156cc08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f156815d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156cc08640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156815d690 con 0x7f156815cc10 2024-04-13T03:17:22.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156cc08640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f156815d970 con 0x7f1568156b50 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156cc08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1568160410 con 0x7f156806e480 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1568156b50 0x7f1568160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1568156b50 0x7f1568160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36324/0 (socket says 172.21.15.106:36324) 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 -- 172.21.15.106:0/33718759 learned_addr learned my addr 172.21.15.106:0/33718759 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f1566d9d640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f156815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 -- 172.21.15.106:0/33718759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 msgr2=0x7f156815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f156815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 -- 172.21.15.106:0/33718759 >> v1:172.21.15.114:6789/0 conn(0x7f156815cc10 legacy=0x7f156806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.785+0000 7f156659c640 1 -- 172.21.15.106:0/33718759 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681606f0 con 0x7f1568156b50 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.786+0000 7f156659c640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f1568160c10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1558009e50 tx=0x7f155802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0f960f404d6992 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.786+0000 7f156559a640 1 -- 172.21.15.106:0/33718759 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155802cb00 con 0x7f1568156b50 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.786+0000 7f156559a640 1 -- 172.21.15.106:0/33718759 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155802cca0 con 0x7f1568156b50 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.786+0000 7f156559a640 1 -- 172.21.15.106:0/33718759 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558034610 con 0x7f1568156b50 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 -- 172.21.15.106:0/33718759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 msgr2=0x7f1568160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f1568160c10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1558009e50 tx=0x7f155802bf60 comp rx=0 tx=0).stop 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 -- 172.21.15.106:0/33718759 shutdown_connections 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f156815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 --2- 172.21.15.106:0/33718759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f1568160c10 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 -- 172.21.15.106:0/33718759 >> 172.21.15.106:0/33718759 conn(0x7f15680b1f10 msgr2=0x7f15680b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 -- 172.21.15.106:0/33718759 shutdown_connections 2024-04-13T03:17:22.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.787+0000 7f156cc08640 1 -- 172.21.15.106:0/33718759 wait complete. 2024-04-13T03:17:22.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.788+0000 7f156cc08640 1 Processor -- start 2024-04-13T03:17:22.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.788+0000 7f156cc08640 1 -- start start 2024-04-13T03:17:22.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f1568157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f156815cc10 0x7f15680c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f15680afc20 con 0x7f1568156b50 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15680afaa0 con 0x7f156815cc10 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156cc08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f156806bcb0 con 0x7f156806e480 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36328/0 (socket says 172.21.15.106:36328) 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 -- 172.21.15.106:0/5597108 learned_addr learned my addr 172.21.15.106:0/5597108 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1566d9d640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f156815cc10 0x7f15680c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156659c640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f1568157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 -- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 msgr2=0x7f1568157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f1568157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 -- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f156815cc10 msgr2=0x7f15680c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f156815cc10 0x7f15680c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f1565d9b640 1 -- 172.21.15.106:0/5597108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156815dff0 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.789+0000 7f156659c640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f1568157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f1565d9b640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c430 tx=0x7f155c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c0052d0 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155c005490 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156815e2d0 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c007260 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.790+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156815e890 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.791+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568068a10 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.791+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f155c007400 con 0x7f1568156b50 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.791+0000 7f1556ffd640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:22.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.792+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f155c08b380 con 0x7f1568156b50 2024-04-13T03:17:22.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.792+0000 7f156659c640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:22.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.792+0000 7f156659c640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 0x7f154806edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1558035000 tx=0x7f1558030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:22.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.796+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f155c059510 con 0x7f1568156b50 2024-04-13T03:17:22.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.970+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f156806ba90 con 0x7f1568156b50 2024-04-13T03:17:22.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.972+0000 7f1556ffd640 1 -- 172.21.15.106:0/5597108 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17375 (secure 0 0 0) 0x7f155c058e10 con 0x7f1568156b50 2024-04-13T03:17:22.973 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:22.973 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":95,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:16:40.106840+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:15:17.537136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-13T03:15:18.258394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.973+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 msgr2=0x7f154806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 0x7f154806edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1558035000 tx=0x7f1558030000 comp rx=0 tx=0).stop 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 msgr2=0x7f15681f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c430 tx=0x7f155c00c8f0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 shutdown_connections 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f156815cc10 0x7f15680c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1568156b50 0x7f15681f57d0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 --2- 172.21.15.106:0/5597108 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f156806e480 0x7f1568157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 >> 172.21.15.106:0/5597108 conn(0x7f15680b1f10 msgr2=0x7f156806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 shutdown_connections 2024-04-13T03:17:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:22.974+0000 7f156cc08640 1 -- 172.21.15.106:0/5597108 wait complete. 2024-04-13T03:17:22.986 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:17:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.086+0000 7f7130f8c640 1 Processor -- start 2024-04-13T03:17:23.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.086+0000 7f7130f8c640 1 -- start start 2024-04-13T03:17:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f7130f8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f712c0ba170 0x7f712c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f7130f8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712c0cfda0 con 0x7f712c0d3220 2024-04-13T03:17:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f7130f8c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712c15f600 con 0x7f712c0cf480 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f7130f8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f712c0d2d90 con 0x7f712c0ba170 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f7129d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f712c0ba170 0x7f712c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f712ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f712c0d3220 0x7f712c0d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38540/0 (socket says 172.21.15.106:38540) 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.087+0000 7f712ad9d640 1 -- 172.21.15.106:0/4005910558 learned_addr learned my addr 172.21.15.106:0/4005910558 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f7129d9b640 1 -- 172.21.15.106:0/4005910558 >> v1:172.21.15.114:6789/0 conn(0x7f712c0cf480 legacy=0x7f712c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f7129d9b640 1 -- 172.21.15.106:0/4005910558 >> v1:172.21.15.106:6789/0 conn(0x7f712c0d3220 legacy=0x7f712c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f712959a640 1 -- 172.21.15.106:0/4005910558 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4267959285 0 0) 0x7f712c0cfda0 con 0x7f712c0d3220 2024-04-13T03:17:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f7129d9b640 1 -- 172.21.15.106:0/4005910558 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c15f8e0 con 0x7f712c0ba170 2024-04-13T03:17:23.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f7129d9b640 1 --2- 172.21.15.106:0/4005910558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0ba170 0x7f712c0cf860 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f71200097a0 tx=0x7f712002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf79e05db70dc6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f712959a640 1 -- 172.21.15.106:0/4005910558 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120035020 con 0x7f712c0ba170 2024-04-13T03:17:23.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f712959a640 1 -- 172.21.15.106:0/4005910558 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120031830 con 0x7f712c0ba170 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.088+0000 7f712959a640 1 -- 172.21.15.106:0/4005910558 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120031c00 con 0x7f712c0ba170 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 -- 172.21.15.106:0/4005910558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0ba170 msgr2=0x7f712c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 --2- 172.21.15.106:0/4005910558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0ba170 0x7f712c0cf860 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f71200097a0 tx=0x7f712002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 -- 172.21.15.106:0/4005910558 shutdown_connections 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 --2- 172.21.15.106:0/4005910558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0ba170 0x7f712c0cf860 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 -- 172.21.15.106:0/4005910558 >> 172.21.15.106:0/4005910558 conn(0x7f712c0d47d0 msgr2=0x7f712c0d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 -- 172.21.15.106:0/4005910558 shutdown_connections 2024-04-13T03:17:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.089+0000 7f7130f8c640 1 -- 172.21.15.106:0/4005910558 wait complete. 2024-04-13T03:17:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.090+0000 7f7130f8c640 1 Processor -- start 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.090+0000 7f7130f8c640 1 -- start start 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 0x7f712c1f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f712c0cf480 0x7f712c14d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1592f0 con 0x7f712c0d3220 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f712c159170 con 0x7f712c0ba170 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7130f8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0d3de0 con 0x7f712c0cf480 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f712ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f712ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36356/0 (socket says 172.21.15.106:36356) 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f712ad9d640 1 -- 172.21.15.106:0/1706844653 learned_addr learned my addr 172.21.15.106:0/1706844653 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f712a59c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 0x7f712c1f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.091+0000 7f7129d9b640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f712c0cf480 0x7f712c14d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 -- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f712c0cf480 msgr2=0x7f712c14d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f712c0cf480 0x7f712c14d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 -- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 msgr2=0x7f712c1f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 0x7f712c1f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 -- 172.21.15.106:0/1706844653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c14dac0 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712a59c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 0x7f712c1f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f712ad9d640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7124001eb0 tx=0x7f712400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240074e0 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71240047e0 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124004bb0 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c0cad90 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.092+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1f8060 con 0x7f712c0d3220 2024-04-13T03:17:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.093+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c068a10 con 0x7f712c0d3220 2024-04-13T03:17:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.094+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7124011070 con 0x7f712c0d3220 2024-04-13T03:17:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.094+0000 7f711affd640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.094+0000 7f712a59c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.094+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f712408b920 con 0x7f712c0d3220 2024-04-13T03:17:23.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.094+0000 7f712a59c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f711c0045b0 tx=0x7f711c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.098+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f712400e280 con 0x7f712c0d3220 2024-04-13T03:17:23.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.311+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f712c0b86d0 con 0x7f712c0d3220 2024-04-13T03:17:23.312 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:17:23.311+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:17:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.312+0000 7f711affd640 1 -- 172.21.15.106:0/1706844653 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) ==== 161+0+0 (secure 0 0 0) 0x7f7124059b30 con 0x7f712c0d3220 2024-04-13T03:17:23.313 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 msgr2=0x7f710c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f711c0045b0 tx=0x7f711c009040 comp rx=0 tx=0).stop 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 msgr2=0x7f712c0c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7124001eb0 tx=0x7f712400c870 comp rx=0 tx=0).stop 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 shutdown_connections 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f712c0cf480 0x7f712c14d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f712c0d3220 0x7f712c0c8130 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 --2- 172.21.15.106:0/1706844653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712c0ba170 0x7f712c1f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 >> 172.21.15.106:0/1706844653 conn(0x7f712c0d47d0 msgr2=0x7f712c0b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:23.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 shutdown_connections 2024-04-13T03:17:23.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.315+0000 7f7130f8c640 1 -- 172.21.15.106:0/1706844653 wait complete. 2024-04-13T03:17:23.318 INFO:tasks.ceph.mds.a.smithi106.stderr: -2362> 2024-04-13T03:16:46.119+0000 7fa0ec7f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:17:23.319 INFO:tasks.ceph.mds.a.smithi106.stderr: -1905> 2024-04-13T03:16:46.756+0000 7fa0e87e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x563c2ca8eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-13T03:17:23.329 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:17:23.345 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:17:23.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.431+0000 7fa9c255e640 1 Processor -- start 2024-04-13T03:17:23.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.431+0000 7fa9c255e640 1 -- start start 2024-04-13T03:17:23.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9c255e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9bc06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9c255e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9bc0cef80 con 0x7fa9bc15fb50 2024-04-13T03:17:23.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9c255e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9bc0cf260 con 0x7fa9bc0d07c0 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9c255e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa9bc0cf540 con 0x7fa9bc06bff0 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9bbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa9bc15fb50 0x7fa9bc06b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38548/0 (socket says 172.21.15.106:38548) 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9bbfff640 1 -- 172.21.15.106:0/417771256 learned_addr learned my addr 172.21.15.106:0/417771256 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9bb7fe640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9bc06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 857445603 0 0) 0x7fa9bc0cef80 con 0x7fa9bc15fb50 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a4003520 con 0x7fa9bc15fb50 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173904494 0 0) 0x7fa9bc0cf260 con 0x7fa9bc0d07c0 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9bc0cef80 con 0x7fa9bc0d07c0 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1677749724 0 0) 0x7fa9a4003520 con 0x7fa9bc15fb50 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 msgr2=0x7fa9bc06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9bc06e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 >> v1:172.21.15.114:6789/0 conn(0x7fa9bc0d07c0 legacy=0x7fa9bc06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc0cf820 con 0x7fa9bc15fb50 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa9ac001670 con 0x7fa9bc15fb50 2024-04-13T03:17:23.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 >> v1:172.21.15.106:6789/0 conn(0x7fa9bc15fb50 legacy=0x7fa9bc06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9a4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a40040d0 0x7fa9a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9a4004d40 0x7fa9a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc0cf540 con 0x7fa9a4004d40 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a400bb50 con 0x7fa9a40040d0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.432+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a400bd20 con 0x7fa9bc06bff0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9a4004d40 0x7fa9a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9bb7fe640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a40040d0 0x7fa9a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9bbfff640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9a4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 -- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 msgr2=0x7fa9a4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06bff0 0x7fa9a4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 -- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a40040d0 msgr2=0x7fa9a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a40040d0 0x7fa9a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 -- 172.21.15.106:0/417771256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a4003520 con 0x7fa9a4004d40 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.433+0000 7fa9c0afb640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9a4004d40 0x7fa9a40051c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4001fa0 tx=0x7fa9a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4806ab5e3e6cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4001bf0 con 0x7fa9a4004d40 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b4003b70 con 0x7fa9a4004d40 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9baffd640 1 -- 172.21.15.106:0/417771256 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40050d0 con 0x7fa9a4004d40 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 -- 172.21.15.106:0/417771256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9a4004d40 msgr2=0x7fa9a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 --2- 172.21.15.106:0/417771256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9a4004d40 0x7fa9a40051c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4001fa0 tx=0x7fa9a4005830 comp rx=0 tx=0).stop 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c0afb640 1 -- 172.21.15.106:0/417771256 reap_dead start 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 -- 172.21.15.106:0/417771256 shutdown_connections 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 -- 172.21.15.106:0/417771256 >> 172.21.15.106:0/417771256 conn(0x7fa9bc0cd6a0 msgr2=0x7fa9bc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 -- 172.21.15.106:0/417771256 shutdown_connections 2024-04-13T03:17:23.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.434+0000 7fa9c255e640 1 -- 172.21.15.106:0/417771256 wait complete. 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.435+0000 7fa9c255e640 1 Processor -- start 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 -- start start 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 0x7fa9bc154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06ec90 0x7fa9bc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc0ced20 con 0x7fa9bc152f90 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc0ceba0 con 0x7fa9bc1488d0 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9c255e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa9bc15ef10 con 0x7fa9bc06ec90 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9bb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36392/0 (socket says 172.21.15.106:36392) 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.436+0000 7fa9bb7fe640 1 -- 172.21.15.106:0/3636239166 learned_addr learned my addr 172.21.15.106:0/3636239166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:23.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bbfff640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 0x7fa9bc154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 -- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06ec90 msgr2=0x7fa9bc06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9c0afb640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06ec90 0x7fa9bc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06ec90 0x7fa9bc06f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 -- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 msgr2=0x7fa9bc154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 0x7fa9bc154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 -- 172.21.15.106:0/3636239166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc153a30 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bbfff640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 0x7fa9bc154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9bb7fe640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00119d0 tx=0x7fa9b0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000fb50 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b000fcf0 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000fea0 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc153d10 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.437+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1542d0 con 0x7fa9bc152f90 2024-04-13T03:17:23.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.438+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc06f650 con 0x7fa9bc152f90 2024-04-13T03:17:23.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.438+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9b0012040 con 0x7fa9bc152f90 2024-04-13T03:17:23.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.439+0000 7fa9b8ff9640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 0x7fa9a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:23.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.439+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6475+0+0 (secure 0 0 0) 0x7fa9b008c540 con 0x7fa9bc152f90 2024-04-13T03:17:23.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.439+0000 7fa9bbfff640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 0x7fa9a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:23.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.439+0000 7fa9bbfff640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 0x7fa9a806edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0011e0 tx=0x7fa9ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:23.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.443+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9b0019030 con 0x7fa9bc152f90 2024-04-13T03:17:23.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:23.660+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa9bc068a10 con 0x7fa9bc152f90 2024-04-13T03:17:24.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.316+0000 7fa9b8ff9640 1 -- 172.21.15.106:0/3636239166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) ==== 104+0+0 (secure 0 0 0) 0x7fa9b005a670 con 0x7fa9bc152f90 2024-04-13T03:17:24.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 msgr2=0x7fa9a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 0x7fa9a806edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0011e0 tx=0x7fa9ac009040 comp rx=0 tx=0).stop 2024-04-13T03:17:24.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 msgr2=0x7fa9bc153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00119d0 tx=0x7fa9b0011e90 comp rx=0 tx=0).stop 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 shutdown_connections 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa9a806c8f0 0x7fa9a806edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa9bc06ec90 0x7fa9bc06f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9bc152f90 0x7fa9bc153410 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 --2- 172.21.15.106:0/3636239166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9bc1488d0 0x7fa9bc154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 >> 172.21.15.106:0/3636239166 conn(0x7fa9bc0cd6a0 msgr2=0x7fa9bc0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 shutdown_connections 2024-04-13T03:17:24.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.318+0000 7fa9c255e640 1 -- 172.21.15.106:0/3636239166 wait complete. 2024-04-13T03:17:24.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:17:24.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.476+0000 7f96e3cde640 1 Processor -- start 2024-04-13T03:17:24.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.476+0000 7f96e3cde640 1 -- start start 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e3cde640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96dc06d990 0x7f96dc0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e3cde640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc0d06e0 con 0x7f96dc15ea80 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e3cde640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc0b20e0 con 0x7f96dc0d1d70 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e3cde640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f96dc0b1e90 con 0x7f96dc06d990 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e1a7a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f96dc15ea80 0x7f96dc0b3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38554/0 (socket says 172.21.15.106:38554) 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e1a7a640 1 -- 172.21.15.106:0/2714887652 learned_addr learned my addr 172.21.15.106:0/2714887652 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e227b640 1 --2- 172.21.15.106:0/2714887652 >> v2:172.21.15.114:3300/0 conn(0x7f96dc06d990 0x7f96dc0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191258821 0 0) 0x7f96dc0d06e0 con 0x7f96dc15ea80 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.477+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c8003520 con 0x7f96dc15ea80 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869892913 0 0) 0x7f96c8003520 con 0x7f96dc15ea80 2024-04-13T03:17:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 >> v2:172.21.15.114:3300/0 conn(0x7f96dc06d990 msgr2=0x7f96dc0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 --2- 172.21.15.106:0/2714887652 >> v2:172.21.15.114:3300/0 conn(0x7f96dc06d990 0x7f96dc0ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 >> v1:172.21.15.114:6789/0 conn(0x7f96dc0d1d70 legacy=0x7f96dc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0b23c0 con 0x7f96dc15ea80 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f96d8001670 con 0x7f96dc15ea80 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 >> v1:172.21.15.106:6789/0 conn(0x7f96dc15ea80 legacy=0x7f96dc0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 0x7f96c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c8004800 0x7f96c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80051c0 0x7f96c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b1e90 con 0x7f96c80039b0 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96c800c060 con 0x7f96c8004800 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96c800c260 con 0x7f96c80051c0 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e227b640 1 --2- 172.21.15.106:0/2714887652 >> v2:172.21.15.114:3300/0 conn(0x7f96dc06d990 0x7f96dc0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 0x7f96c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1279640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c8004800 0x7f96c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e227b640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80051c0 0x7f96c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 -- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80051c0 msgr2=0x7f96c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80051c0 0x7f96c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 -- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c8004800 msgr2=0x7f96c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c8004800 0x7f96c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e1a7a640 1 -- 172.21.15.106:0/2714887652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c8003520 con 0x7f96c80039b0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.478+0000 7f96e227b640 1 -- 172.21.15.106:0/2714887652 reap_dead start 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 0x7f96c8003dd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c240 tx=0x7f96c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31aed9b0d62cb4e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8007cf0 con 0x7f96c80039b0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8030070 con 0x7f96c80039b0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e0a78640 1 -- 172.21.15.106:0/2714887652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8005d70 con 0x7f96c80039b0 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e3cde640 1 -- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 msgr2=0x7f96c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e3cde640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 0x7f96c8003dd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c240 tx=0x7f96c8005830 comp rx=0 tx=0).stop 2024-04-13T03:17:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e3cde640 1 -- 172.21.15.106:0/2714887652 shutdown_connections 2024-04-13T03:17:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e3cde640 1 --2- 172.21.15.106:0/2714887652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96c80039b0 0x7f96c8003dd0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.479+0000 7f96e3cde640 1 -- 172.21.15.106:0/2714887652 >> 172.21.15.106:0/2714887652 conn(0x7f96dc0cefa0 msgr2=0x7f96dc0cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.480+0000 7f96e3cde640 1 -- 172.21.15.106:0/2714887652 shutdown_connections 2024-04-13T03:17:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.480+0000 7f96e3cde640 1 -- 172.21.15.106:0/2714887652 wait complete. 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 Processor -- start 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 -- start start 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 0x7f96dc0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc0af380 0x7f96dc1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b2c60 con 0x7f96dc0afb00 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b2980 con 0x7f96dc0af380 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e3cde640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc15ee60 con 0x7f96c80039b0 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.481+0000 7f96e227b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36414/0 (socket says 172.21.15.106:36414) 2024-04-13T03:17:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 -- 172.21.15.106:0/3176440369 learned_addr learned my addr 172.21.15.106:0/3176440369 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e1279640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc0af380 0x7f96dc1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 0x7f96dc0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 -- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 msgr2=0x7f96dc0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 0x7f96dc0c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 -- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc0af380 msgr2=0x7f96dc1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc0af380 0x7f96dc1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 -- 172.21.15.106:0/3176440369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0b04f0 con 0x7f96dc0afb00 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 0x7f96dc0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e227b640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f96dc06c800 tx=0x7f96d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011e90 con 0x7f96dc0afb00 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d401b070 con 0x7f96dc0afb00 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003990 con 0x7f96dc0afb00 2024-04-13T03:17:24.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc0c6a20 con 0x7f96dc0afb00 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.482+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc0c6fe0 con 0x7f96dc0afb00 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.483+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc068a10 con 0x7f96dc0afb00 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.484+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96d4005d10 con 0x7f96dc0afb00 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.484+0000 7f96d27fc640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 0x7f96c006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.484+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6475+0+0 (secure 0 0 0) 0x7f96d408c4a0 con 0x7f96dc0afb00 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.484+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 0x7f96c006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.484+0000 7f96e1a7a640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 0x7f96c006ef50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f96d8034000 tx=0x7f96d8039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.488+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96d4025020 con 0x7f96dc0afb00 2024-04-13T03:17:24.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:24.664+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96dc0b84a0 con 0x7f96dc0afb00 2024-04-13T03:17:25.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.324+0000 7f96d27fc640 1 -- 172.21.15.106:0/3176440369 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7f96d405a660 con 0x7f96dc0afb00 2024-04-13T03:17:25.325 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 msgr2=0x7f96c006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 0x7f96c006ef50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f96d8034000 tx=0x7f96d8039040 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 msgr2=0x7f96dc0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f96dc06c800 tx=0x7f96d4009f90 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 shutdown_connections 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f96c006ca90 0x7f96c006ef50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96dc0afb00 0x7f96dc0affb0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc0af380 0x7f96dc1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 --2- 172.21.15.106:0/3176440369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f96c80039b0 0x7f96dc0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 >> 172.21.15.106:0/3176440369 conn(0x7f96dc0cefa0 msgr2=0x7f96dc070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:25.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 shutdown_connections 2024-04-13T03:17:25.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.326+0000 7f96e3cde640 1 -- 172.21.15.106:0/3176440369 wait complete. 2024-04-13T03:17:25.340 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:17:25.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.444+0000 7f984d108640 1 Processor -- start 2024-04-13T03:17:25.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.444+0000 7f984d108640 1 -- start start 2024-04-13T03:17:25.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f984d108640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98480704a0 0x7f9848160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f984d108640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 0x7f9848070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f984d108640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9848161160 con 0x7f98480b5dd0 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f984d108640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f98480b7720 con 0x7f98480704a0 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f984d108640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9848070dc0 con 0x7f98480b6b00 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98480704a0 0x7f9848160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98480704a0 0x7f9848160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36426/0 (socket says 172.21.15.106:36426) 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 -- 172.21.15.106:0/1573899523 learned_addr learned my addr 172.21.15.106:0/1573899523 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f983e59c640 1 --2- 172.21.15.106:0/1573899523 >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 0x7f9848070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 -- 172.21.15.106:0/1573899523 >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 msgr2=0x7f9848070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 --2- 172.21.15.106:0/1573899523 >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 0x7f9848070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 -- 172.21.15.106:0/1573899523 >> v1:172.21.15.114:6789/0 conn(0x7f98480b5dd0 legacy=0x7f984815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:25.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f9846d9d640 1 -- 172.21.15.106:0/1573899523 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480b7160 con 0x7f98480704a0 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.445+0000 7f983e59c640 1 --2- 172.21.15.106:0/1573899523 >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 0x7f9848070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f9846d9d640 1 --2- 172.21.15.106:0/1573899523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480704a0 0x7f9848160c20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f98480b86b0 tx=0x7f983802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da557fd2dd287d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984659c640 1 -- 172.21.15.106:0/1573899523 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035020 con 0x7f98480704a0 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984659c640 1 -- 172.21.15.106:0/1573899523 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9838031870 con 0x7f98480704a0 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984659c640 1 -- 172.21.15.106:0/1573899523 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838031c40 con 0x7f98480704a0 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984d108640 1 -- 172.21.15.106:0/1573899523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480704a0 msgr2=0x7f9848160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984d108640 1 --2- 172.21.15.106:0/1573899523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480704a0 0x7f9848160c20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f98480b86b0 tx=0x7f983802b910 comp rx=0 tx=0).stop 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984d108640 1 -- 172.21.15.106:0/1573899523 shutdown_connections 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984d108640 1 --2- 172.21.15.106:0/1573899523 >> v2:172.21.15.114:3300/0 conn(0x7f98480b6b00 0x7f9848070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.446+0000 7f984d108640 1 --2- 172.21.15.106:0/1573899523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480704a0 0x7f9848160c20 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.447+0000 7f984d108640 1 -- 172.21.15.106:0/1573899523 >> 172.21.15.106:0/1573899523 conn(0x7f98480717e0 msgr2=0x7f9848071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.447+0000 7f984d108640 1 -- 172.21.15.106:0/1573899523 shutdown_connections 2024-04-13T03:17:25.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.447+0000 7f984d108640 1 -- 172.21.15.106:0/1573899523 wait complete. 2024-04-13T03:17:25.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.448+0000 7f984d108640 1 Processor -- start 2024-04-13T03:17:25.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.448+0000 7f984d108640 1 -- start start 2024-04-13T03:17:25.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98480704a0 0x7f9848150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98480b6b00 0x7f98480b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b58b0 con 0x7f98480b5dd0 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b55d0 con 0x7f98480704a0 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f984815d0f0 con 0x7f98480b6b00 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36438/0 (socket says 172.21.15.106:36438) 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 -- 172.21.15.106:0/4010137021 learned_addr learned my addr 172.21.15.106:0/4010137021 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f9846d9d640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98480704a0 0x7f9848150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 -- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98480b6b00 msgr2=0x7f98480b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98480b6b00 0x7f98480b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 -- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98480704a0 msgr2=0x7f9848150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98480704a0 0x7f9848150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 -- 172.21.15.106:0/4010137021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984806d560 con 0x7f98480b5dd0 2024-04-13T03:17:25.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983e59c640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f983000c7f0 tx=0x7f983000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830015070 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830007820 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.449+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984806d840 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.450+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830007bd0 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.450+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9848151520 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.451+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848068a10 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.451+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9830018070 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.451+0000 7f983ffff640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 0x7f982806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.451+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6037+0+0 (secure 0 0 0) 0x7f983008bd80 con 0x7f98480b5dd0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.451+0000 7f9846d9d640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 0x7f982806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:25.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.452+0000 7f9846d9d640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 0x7f982806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9838036000 tx=0x7f9838008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:25.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.456+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f983005a0f0 con 0x7f98480b5dd0 2024-04-13T03:17:25.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:25.632+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98480725b0 con 0x7f98480b5dd0 2024-04-13T03:17:26.108 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:17:26.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:17:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.325+0000 7f983ffff640 1 -- 172.21.15.106:0/4010137021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7f98300599f0 con 0x7f98480b5dd0 2024-04-13T03:17:26.326 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:17:26.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.328+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 msgr2=0x7f982806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.328+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 0x7f982806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9838036000 tx=0x7f9838008030 comp rx=0 tx=0).stop 2024-04-13T03:17:26.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.328+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 msgr2=0x7f984806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.328+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f983000c7f0 tx=0x7f983000ccb0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 shutdown_connections 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f982806c8f0 0x7f982806edb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f98480b6b00 0x7f98480b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98480b5dd0 0x7f984806e320 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 --2- 172.21.15.106:0/4010137021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98480704a0 0x7f9848150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 >> 172.21.15.106:0/4010137021 conn(0x7f98480717e0 msgr2=0x7f9848157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 shutdown_connections 2024-04-13T03:17:26.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.331+0000 7f984d108640 1 -- 172.21.15.106:0/4010137021 wait complete. 2024-04-13T03:17:26.347 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:17:26.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 Processor -- start 2024-04-13T03:17:26.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 -- start start 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74640704a0 0x7f74640d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640d1a60 0x7f746415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746415ddd0 con 0x7f74640cfea0 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f746415e100 con 0x7f74640704a0 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.449+0000 7f7468be9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f746415d4d0 con 0x7f74640d1a60 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7459d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74640704a0 0x7f74640d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f746259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f74640cfea0 0x7f746415cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38562/0 (socket says 172.21.15.106:38562) 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f746259c640 1 -- 172.21.15.106:0/2852640018 learned_addr learned my addr 172.21.15.106:0/2852640018 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7462d9d640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640d1a60 0x7f746415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7459d9b640 1 -- 172.21.15.106:0/2852640018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640d1a60 msgr2=0x7f746415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7459d9b640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640d1a60 0x7f746415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7459d9b640 1 -- 172.21.15.106:0/2852640018 >> v1:172.21.15.106:6789/0 conn(0x7f74640cfea0 legacy=0x7f746415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.450+0000 7f7459d9b640 1 -- 172.21.15.106:0/2852640018 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746415d7b0 con 0x7f74640704a0 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7459d9b640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f74640d1520 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f746415e280 tx=0x7f744c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65443242ab703ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7461d9b640 1 -- 172.21.15.106:0/2852640018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035020 con 0x7f74640704a0 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7461d9b640 1 -- 172.21.15.106:0/2852640018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c031800 con 0x7f74640704a0 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7461d9b640 1 -- 172.21.15.106:0/2852640018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c031bd0 con 0x7f74640704a0 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 -- 172.21.15.106:0/2852640018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 msgr2=0x7f74640d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f74640d1520 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f746415e280 tx=0x7f744c02b8a0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 -- 172.21.15.106:0/2852640018 shutdown_connections 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640d1a60 0x7f746415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 --2- 172.21.15.106:0/2852640018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f74640d1520 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 -- 172.21.15.106:0/2852640018 >> 172.21.15.106:0/2852640018 conn(0x7f7464071870 msgr2=0x7f7464071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:26.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 -- 172.21.15.106:0/2852640018 shutdown_connections 2024-04-13T03:17:26.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.451+0000 7f7468be9640 1 -- 172.21.15.106:0/2852640018 wait complete. 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 Processor -- start 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 -- start start 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640cfea0 0x7f746415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74640d1a60 0x7f74641f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f746415ead0 con 0x7f74640704a0 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74640d1ef0 con 0x7f74640d1a60 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.453+0000 7f7468be9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f746415ec50 con 0x7f74640cfea0 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36456/0 (socket says 172.21.15.106:36456) 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 -- 172.21.15.106:0/754372714 learned_addr learned my addr 172.21.15.106:0/754372714 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f7459d9b640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640cfea0 0x7f746415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 -- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640cfea0 msgr2=0x7f746415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640cfea0 0x7f746415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 -- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74640d1a60 msgr2=0x7f74641f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f7462d9d640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74640d1a60 0x7f74641f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74640d1a60 0x7f74641f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 -- 172.21.15.106:0/754372714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746406edc0 con 0x7f74640704a0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f746259c640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f745400c400 tx=0x7f745400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454017070 con 0x7f74640704a0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7454005ce0 con 0x7f74640704a0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746406d280 con 0x7f74640704a0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540072f0 con 0x7f74640704a0 2024-04-13T03:17:26.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.454+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746406d540 con 0x7f74640704a0 2024-04-13T03:17:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.455+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7464075180 con 0x7f74640704a0 2024-04-13T03:17:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.456+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f745401a070 con 0x7f74640704a0 2024-04-13T03:17:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.456+0000 7f745b7fe640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 0x7f744406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.456+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f745408ae70 con 0x7f74640704a0 2024-04-13T03:17:26.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.456+0000 7f7459d9b640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 0x7f744406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.457+0000 7f7459d9b640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 0x7f744406edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.461+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7464075180 con 0x7f74640704a0 2024-04-13T03:17:26.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.638+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f74640737e0 con 0x7f74640704a0 2024-04-13T03:17:26.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.641+0000 7f745b7fe640 1 -- 172.21.15.106:0/754372714 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f74640737e0 con 0x7f74640704a0 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 msgr2=0x7f744406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 0x7f744406edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c008030 comp rx=0 tx=0).stop 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 msgr2=0x7f746406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f745400c400 tx=0x7f745400c8c0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 shutdown_connections 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f744406c8f0 0x7f744406edb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f74640cfea0 0x7f746415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74640d1a60 0x7f74641f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 --2- 172.21.15.106:0/754372714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74640704a0 0x7f746406dd40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 >> 172.21.15.106:0/754372714 conn(0x7f7464071870 msgr2=0x7f74640b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:26.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 shutdown_connections 2024-04-13T03:17:26.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.643+0000 7f7468be9640 1 -- 172.21.15.106:0/754372714 wait complete. 2024-04-13T03:17:26.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-13T03:17:26.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.754+0000 7fd2ad280640 1 Processor -- start 2024-04-13T03:17:26.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.754+0000 7fd2ad280640 1 -- start start 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2ad280640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2a815e9b0 0x7fd2a80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2ad280640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 0x7fd2a80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2ad280640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80cfd20 con 0x7fd2a815ed90 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2ad280640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a8162af0 con 0x7fd2a815e9b0 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2ad280640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a8162dd0 con 0x7fd2a80b3d60 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2a815e9b0 0x7fd2a80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a67fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd2a815ed90 0x7fd2a80d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38568/0 (socket says 172.21.15.106:38568) 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a67fc640 1 -- 172.21.15.106:0/3731072571 learned_addr learned my addr 172.21.15.106:0/3731072571 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a77fe640 1 --2- 172.21.15.106:0/3731072571 >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 0x7fd2a80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a6ffd640 1 -- 172.21.15.106:0/3731072571 >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 msgr2=0x7fd2a80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a6ffd640 1 --2- 172.21.15.106:0/3731072571 >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 0x7fd2a80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a6ffd640 1 -- 172.21.15.106:0/3731072571 >> v1:172.21.15.106:6789/0 conn(0x7fd2a815ed90 legacy=0x7fd2a80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a5ffb640 1 -- 172.21.15.106:0/3731072571 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552862469 0 0) 0x7fd2a80cfd20 con 0x7fd2a815ed90 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a6ffd640 1 -- 172.21.15.106:0/3731072571 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81630b0 con 0x7fd2a815e9b0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.755+0000 7fd2a77fe640 1 --2- 172.21.15.106:0/3731072571 >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 0x7fd2a80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.756+0000 7fd2a6ffd640 1 --2- 172.21.15.106:0/3731072571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a815e9b0 0x7fd2a80d2300 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80cb7b0 tx=0x7fd29802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9187ae007f7f665 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.756+0000 7fd2a5ffb640 1 -- 172.21.15.106:0/3731072571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298034070 con 0x7fd2a815e9b0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.756+0000 7fd2a5ffb640 1 -- 172.21.15.106:0/3731072571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd298030610 con 0x7fd2a815e9b0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.756+0000 7fd2a5ffb640 1 -- 172.21.15.106:0/3731072571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980309e0 con 0x7fd2a815e9b0 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 -- 172.21.15.106:0/3731072571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a815e9b0 msgr2=0x7fd2a80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 --2- 172.21.15.106:0/3731072571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a815e9b0 0x7fd2a80d2300 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80cb7b0 tx=0x7fd29802b960 comp rx=0 tx=0).stop 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 -- 172.21.15.106:0/3731072571 shutdown_connections 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 --2- 172.21.15.106:0/3731072571 >> v2:172.21.15.114:3300/0 conn(0x7fd2a80b3d60 0x7fd2a80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 --2- 172.21.15.106:0/3731072571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a815e9b0 0x7fd2a80d2300 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 -- 172.21.15.106:0/3731072571 >> 172.21.15.106:0/3731072571 conn(0x7fd2a808bf10 msgr2=0x7fd2a80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 -- 172.21.15.106:0/3731072571 shutdown_connections 2024-04-13T03:17:26.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.757+0000 7fd2ad280640 1 -- 172.21.15.106:0/3731072571 wait complete. 2024-04-13T03:17:26.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.758+0000 7fd2ad280640 1 Processor -- start 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.758+0000 7fd2ad280640 1 -- start start 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2a80b3d60 0x7fd2a80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a815e9b0 0x7fd2a814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8162800 con 0x7fd2a806e110 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8162680 con 0x7fd2a815e9b0 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2ad280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a80d4d00 con 0x7fd2a80b3d60 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36470/0 (socket says 172.21.15.106:36470) 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 -- 172.21.15.106:0/1882677495 learned_addr learned my addr 172.21.15.106:0/1882677495 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a67fc640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a815e9b0 0x7fd2a814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a6ffd640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2a80b3d60 0x7fd2a80cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 -- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2a80b3d60 msgr2=0x7fd2a80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2a80b3d60 0x7fd2a80cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 -- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a815e9b0 msgr2=0x7fd2a814fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a815e9b0 0x7fd2a814fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 -- 172.21.15.106:0/1882677495 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a80cbde0 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.759+0000 7fd2a77fe640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000cef0 tx=0x7fd2a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.760+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a001a070 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.760+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a000d860 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.760+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000dc50 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.760+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a80cc0c0 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.760+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a80cc680 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.761+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8068a10 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.761+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2a001d070 con 0x7fd2a806e110 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.761+0000 7fd28f7fe640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 0x7fd29406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:26.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.762+0000 7fd2a6ffd640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 0x7fd29406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:26.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.762+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7fd2a008c2f0 con 0x7fd2a806e110 2024-04-13T03:17:26.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.762+0000 7fd2a6ffd640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 0x7fd29406ede0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd298036000 tx=0x7fd298008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:26.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:26.766+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2a005a7e0 con 0x7fd2a806e110 2024-04-13T03:17:27.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.008+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fd2a80d53a0 con 0x7fd2a806e110 2024-04-13T03:17:27.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.329+0000 7fd28f7fe640 1 -- 172.21.15.106:0/1882677495 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v381) ==== 167+0+0 (secure 0 0 0) 0x7fd2a005a0e0 con 0x7fd2a806e110 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 msgr2=0x7fd29406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 0x7fd29406ede0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd298036000 tx=0x7fd298008030 comp rx=0 tx=0).stop 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 msgr2=0x7fd2a80cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000cef0 tx=0x7fd2a000a710 comp rx=0 tx=0).stop 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 shutdown_connections 2024-04-13T03:17:27.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd29406c920 0x7fd29406ede0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2a806e110 0x7fd2a80cce80 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a815e9b0 0x7fd2a814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 --2- 172.21.15.106:0/1882677495 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2a80b3d60 0x7fd2a80cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 >> 172.21.15.106:0/1882677495 conn(0x7fd2a808bf10 msgr2=0x7fd2a80b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 shutdown_connections 2024-04-13T03:17:27.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.332+0000 7fd2ad280640 1 -- 172.21.15.106:0/1882677495 wait complete. 2024-04-13T03:17:27.344 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-13T03:17:27.344 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:17:27.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:27.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.444+0000 7f9135c60640 1 Processor -- start 2024-04-13T03:17:27.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.444+0000 7f9135c60640 1 -- start start 2024-04-13T03:17:27.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f9135c60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f91300cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f9135c60640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9130162820 con 0x7f91300d1900 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f9135c60640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9130162b00 con 0x7f91300d4e00 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f9135c60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f91301625d0 con 0x7f91300d0a80 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f912effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f91300d1900 0x7f91300d46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38574/0 (socket says 172.21.15.106:38574) 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.445+0000 7f912effd640 1 -- 172.21.15.106:0/3676318655 learned_addr learned my addr 172.21.15.106:0/3676318655 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799181358 0 0) 0x7f9130162820 con 0x7f91300d1900 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9118003520 con 0x7f91300d1900 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912f7fe640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f91300cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2241643448 0 0) 0x7f9118003520 con 0x7f91300d1900 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 msgr2=0x7f91300cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f91300cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 >> v1:172.21.15.114:6789/0 conn(0x7f91300d4e00 legacy=0x7f91300d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9130162de0 con 0x7f91300d1900 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9124001670 con 0x7f91300d1900 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 >> v1:172.21.15.106:6789/0 conn(0x7f91300d1900 legacy=0x7f91300d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91180039e0 0x7f9118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f9118004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9118004d40 0x7f91180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f91301625d0 con 0x7f9118004d40 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f911800bb50 con 0x7f91180039e0 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f911800bd20 con 0x7f91300d0a80 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9118004d40 0x7f91180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912f7fe640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91180039e0 0x7f9118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912effd640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f9118004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 -- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 msgr2=0x7f9118004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91300d0a80 0x7f9118004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 -- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91180039e0 msgr2=0x7f9118003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91180039e0 0x7f9118003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.446+0000 7f912ffff640 1 -- 172.21.15.106:0/3676318655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9118003520 con 0x7f9118004d40 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.447+0000 7f912ffff640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9118004d40 0x7f91180051c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9128001e80 tx=0x7f9118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fff3066b5aa5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.447+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128001a70 con 0x7f9118004d40 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.447+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128003b70 con 0x7f9118004d40 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.447+0000 7f912e7fc640 1 -- 172.21.15.106:0/3676318655 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280050d0 con 0x7f9118004d40 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 -- 172.21.15.106:0/3676318655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9118004d40 msgr2=0x7f91180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 --2- 172.21.15.106:0/3676318655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9118004d40 0x7f91180051c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9128001e80 tx=0x7f9118005830 comp rx=0 tx=0).stop 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f912ffff640 1 -- 172.21.15.106:0/3676318655 reap_dead start 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 -- 172.21.15.106:0/3676318655 shutdown_connections 2024-04-13T03:17:27.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 -- 172.21.15.106:0/3676318655 >> 172.21.15.106:0/3676318655 conn(0x7f913009da70 msgr2=0x7f91300b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:27.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 -- 172.21.15.106:0/3676318655 shutdown_connections 2024-04-13T03:17:27.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.448+0000 7f9135c60640 1 -- 172.21.15.106:0/3676318655 wait complete. 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.449+0000 7f9135c60640 1 Processor -- start 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.449+0000 7f9135c60640 1 -- start start 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91301567d0 0x7f9130156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 0x7f9130157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f91300d41b0 con 0x7f913006b8e0 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9130163430 con 0x7f913006d630 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f91300d0e60 con 0x7f91301567d0 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36490/0 (socket says 172.21.15.106:36490) 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 -- 172.21.15.106:0/3727220584 learned_addr learned my addr 172.21.15.106:0/3727220584 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:27.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912ffff640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 0x7f9130157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 -- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91301567d0 msgr2=0x7f9130156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912f7fe640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91301567d0 0x7f9130156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91301567d0 0x7f9130156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 -- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 msgr2=0x7f9130157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 0x7f9130157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 -- 172.21.15.106:0/3727220584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913006c050 con 0x7f913006b8e0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912ffff640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 0x7f9130157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f912effd640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9124007310 tx=0x7f91240077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912400f9c0 con 0x7f913006b8e0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9124018070 con 0x7f913006b8e0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913006c330 con 0x7f913006b8e0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912400fda0 con 0x7f913006b8e0 2024-04-13T03:17:27.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.450+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913006c8f0 con 0x7f913006b8e0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.451+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913015a5b0 con 0x7f913006b8e0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.452+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f912401d460 con 0x7f913006b8e0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.452+0000 7f9117fff640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 0x7f911c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.452+0000 7f912f7fe640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 0x7f911c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.452+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f912408baf0 con 0x7f913006b8e0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.453+0000 7f912f7fe640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 0x7f911c06ede0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91300cda90 tx=0x7f9120008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.457+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f912405a070 con 0x7f913006b8e0 2024-04-13T03:17:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.632+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9130069120 con 0x7f913006b8e0 2024-04-13T03:17:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.633+0000 7f9117fff640 1 -- 172.21.15.106:0/3727220584 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+13594 (secure 0 0 0) 0x7f9124059970 con 0x7f913006b8e0 2024-04-13T03:17:27.634 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:27.634 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:26.324736+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/2398448953":"2024-04-14T03:17:23.312183+0000","172.21.15.106:6837/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/635681585":"2024-04-14T03:16:40.106818+0000","172.21.15.106:6835/2398448953":"2024-04-14T03:17:23.312183+0000","172.21.15.106:6837/3360662124":"2024-04-14T03:15:28.649985+0000","172.21.15.106:6835/3360662124":"2024-04-14T03:15:28.649985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 msgr2=0x7f911c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 0x7f911c06ede0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91300cda90 tx=0x7f9120008040 comp rx=0 tx=0).stop 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 msgr2=0x7f913006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9124007310 tx=0x7f91240077d0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 shutdown_connections 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f911c06c920 0x7f911c06ede0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f913006d630 0x7f9130157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f913006b8e0 0x7f913006d0f0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 --2- 172.21.15.106:0/3727220584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f91301567d0 0x7f9130156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 >> 172.21.15.106:0/3727220584 conn(0x7f913009da70 msgr2=0x7f91300d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 shutdown_connections 2024-04-13T03:17:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.635+0000 7f9135c60640 1 -- 172.21.15.106:0/3727220584 wait complete. 2024-04-13T03:17:27.647 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:27.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.750+0000 7f3234ea2640 1 Processor -- start 2024-04-13T03:17:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.750+0000 7f3234ea2640 1 -- start start 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3234ea2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3234ea2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f323006d220 con 0x7f3230145d20 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3234ea2640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32300b3740 con 0x7f32300b4c20 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3234ea2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f32300b3a20 con 0x7f32300b47e0 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3227fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f322e59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3230145d20 0x7f323006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38578/0 (socket says 172.21.15.106:38578) 2024-04-13T03:17:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f322e59c640 1 -- 172.21.15.106:0/1491016902 learned_addr learned my addr 172.21.15.106:0/1491016902 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3227fff640 1 -- 172.21.15.106:0/1491016902 >> v1:172.21.15.114:6789/0 conn(0x7f32300b4c20 legacy=0x7f32300cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3227fff640 1 -- 172.21.15.106:0/1491016902 >> v1:172.21.15.106:6789/0 conn(0x7f3230145d20 legacy=0x7f323006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f3227fff640 1 -- 172.21.15.106:0/1491016902 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32300b3d00 con 0x7f32300b47e0 2024-04-13T03:17:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.751+0000 7f322dd9b640 1 -- 172.21.15.106:0/1491016902 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440240991 0 0) 0x7f323006d220 con 0x7f3230145d20 2024-04-13T03:17:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.752+0000 7f3227fff640 1 --2- 172.21.15.106:0/1491016902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b47e0 0x7f323015cbc0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad0dfea02f6cafb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.752+0000 7f322dd9b640 1 -- 172.21.15.106:0/1491016902 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32180315a0 con 0x7f32300b47e0 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.752+0000 7f322dd9b640 1 -- 172.21.15.106:0/1491016902 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218037070 con 0x7f32300b47e0 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.752+0000 7f322dd9b640 1 -- 172.21.15.106:0/1491016902 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218031960 con 0x7f32300b47e0 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 -- 172.21.15.106:0/1491016902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b47e0 msgr2=0x7f323015cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 --2- 172.21.15.106:0/1491016902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b47e0 0x7f323015cbc0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802b840 comp rx=0 tx=0).stop 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 -- 172.21.15.106:0/1491016902 shutdown_connections 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 --2- 172.21.15.106:0/1491016902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 -- 172.21.15.106:0/1491016902 >> 172.21.15.106:0/1491016902 conn(0x7f32300b2600 msgr2=0x7f32300d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 -- 172.21.15.106:0/1491016902 shutdown_connections 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.753+0000 7f3234ea2640 1 -- 172.21.15.106:0/1491016902 wait complete. 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.754+0000 7f3234ea2640 1 Processor -- start 2024-04-13T03:17:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.754+0000 7f3234ea2640 1 -- start start 2024-04-13T03:17:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 0x7f32301329f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3230146100 con 0x7f32300b4c20 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f32300b42c0 con 0x7f3230145d20 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3230146280 con 0x7f32300b47e0 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36508/0 (socket says 172.21.15.106:36508) 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 -- 172.21.15.106:0/263581379 learned_addr learned my addr 172.21.15.106:0/263581379 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f322ed9d640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 0x7f32301329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 -- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f32300b47e0 msgr2=0x7f323015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f322e59c640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 -- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 msgr2=0x7f32301329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 0x7f32301329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 -- 172.21.15.106:0/263581379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323015d2a0 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f322ed9d640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 0x7f32301329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3227fff640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f32180098c0 tx=0x7f32180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218032840 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32180329e0 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323015d580 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218032dd0 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.755+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323015db40 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.756+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3230068a10 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.757+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f321804d3f0 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.757+0000 7f3226ffd640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.757+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f321802f030 con 0x7f32300b4c20 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.757+0000 7f322e59c640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.758+0000 7f322e59c640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 0x7f321006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3220007fd0 tx=0x7f3220009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.762+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3218040070 con 0x7f32300b4c20 2024-04-13T03:17:27.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.937+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f323015eb60 con 0x7f32300b4c20 2024-04-13T03:17:27.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.937+0000 7f3226ffd640 1 -- 172.21.15.106:0/263581379 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f321807bc00 con 0x7f32300b4c20 2024-04-13T03:17:27.938 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 msgr2=0x7f321006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 0x7f321006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3220007fd0 tx=0x7f3220009040 comp rx=0 tx=0).stop 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 msgr2=0x7f323015e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f32180098c0 tx=0x7f32180046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 shutdown_connections 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3230145d20 0x7f32301329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b4c20 0x7f323015e620 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 --2- 172.21.15.106:0/263581379 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f32300b47e0 0x7f323015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:27.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 >> 172.21.15.106:0/263581379 conn(0x7f32300b2600 msgr2=0x7f32300cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:27.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 shutdown_connections 2024-04-13T03:17:27.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:27.939+0000 7f3234ea2640 1 -- 172.21.15.106:0/263581379 wait complete. 2024-04-13T03:17:27.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-13T03:17:28.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.049+0000 7f6ffbbe0640 1 Processor -- start 2024-04-13T03:17:28.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.050+0000 7f6ffbbe0640 1 -- start start 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.050+0000 7f6ffbbe0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ff41622d0 0x7f6ff40d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.050+0000 7f6ffbbe0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 0x7f6ff41688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.050+0000 7f6ffbbe0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff40d3530 con 0x7f6ff40d1c00 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ffbbe0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff41626b0 con 0x7f6ff41622d0 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ffbbe0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff40d0b60 con 0x7f6ff40b8080 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ff41622d0 0x7f6ff40d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff917b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6ff40d1c00 0x7f6ff40b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38586/0 (socket says 172.21.15.106:38586) 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ff41622d0 0x7f6ff40d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36514/0 (socket says 172.21.15.106:36514) 2024-04-13T03:17:28.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff917b640 1 -- 172.21.15.106:0/2588952498 learned_addr learned my addr 172.21.15.106:0/2588952498 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ffa17d640 1 --2- 172.21.15.106:0/2588952498 >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 0x7f6ff41688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 -- 172.21.15.106:0/2588952498 >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 msgr2=0x7f6ff41688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 --2- 172.21.15.106:0/2588952498 >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 0x7f6ff41688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 -- 172.21.15.106:0/2588952498 >> v1:172.21.15.106:6789/0 conn(0x7f6ff40d1c00 legacy=0x7f6ff40b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff897a640 1 -- 172.21.15.106:0/2588952498 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542115317 0 0) 0x7f6ff40d3530 con 0x7f6ff40d1c00 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ff997c640 1 -- 172.21.15.106:0/2588952498 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff40d0e40 con 0x7f6ff41622d0 2024-04-13T03:17:28.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.051+0000 7f6ffa17d640 1 --2- 172.21.15.106:0/2588952498 >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 0x7f6ff41688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:28.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ff997c640 1 --2- 172.21.15.106:0/2588952498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff41622d0 0x7f6ff40d04d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40ceef0 tx=0x7f6ff002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debdfd1d1b74b7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ff897a640 1 -- 172.21.15.106:0/2588952498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0039070 con 0x7f6ff41622d0 2024-04-13T03:17:28.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ff897a640 1 -- 172.21.15.106:0/2588952498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff002cc80 con 0x7f6ff41622d0 2024-04-13T03:17:28.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ff897a640 1 -- 172.21.15.106:0/2588952498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0034660 con 0x7f6ff41622d0 2024-04-13T03:17:28.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/2588952498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff41622d0 msgr2=0x7f6ff40d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/2588952498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff41622d0 0x7f6ff40d04d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40ceef0 tx=0x7f6ff002bed0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/2588952498 shutdown_connections 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/2588952498 >> v2:172.21.15.114:3300/0 conn(0x7f6ff40b8080 0x7f6ff41688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/2588952498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff41622d0 0x7f6ff40d04d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.052+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/2588952498 >> 172.21.15.106:0/2588952498 conn(0x7f6ff40d25d0 msgr2=0x7f6ff40d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.053+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/2588952498 shutdown_connections 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.053+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/2588952498 wait complete. 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 Processor -- start 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 -- start start 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ff40b8080 0x7f6ff41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ff41622d0 0x7f6ff40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40d16e0 con 0x7f6ff40d1c00 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40d1400 con 0x7f6ff41622d0 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ffbbe0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff40cc6f0 con 0x7f6ff40b8080 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ff917b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ff917b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36530/0 (socket says 172.21.15.106:36530) 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.054+0000 7f6ff917b640 1 -- 172.21.15.106:0/1761906410 learned_addr learned my addr 172.21.15.106:0/1761906410 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ffa17d640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ff41622d0 0x7f6ff40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 -- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ff40b8080 msgr2=0x7f6ff41f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ff40b8080 0x7f6ff41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 -- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ff41622d0 msgr2=0x7f6ff40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ff41622d0 0x7f6ff40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 -- 172.21.15.106:0/1761906410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff415df00 con 0x7f6ff40d1c00 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ff917b640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400c4d0 tx=0x7f6fe400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4018070 con 0x7f6ff40d1c00 2024-04-13T03:17:28.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe40074e0 con 0x7f6ff40d1c00 2024-04-13T03:17:28.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff415e1e0 con 0x7f6ff40d1c00 2024-04-13T03:17:28.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4004a20 con 0x7f6ff40d1c00 2024-04-13T03:17:28.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.055+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff415e7a0 con 0x7f6ff40d1c00 2024-04-13T03:17:28.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.056+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4075180 con 0x7f6ff40d1c00 2024-04-13T03:17:28.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.056+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fe401b070 con 0x7f6ff40d1c00 2024-04-13T03:17:28.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.057+0000 7f6fea7fc640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 0x7f6fd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.057+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f6fe408ad90 con 0x7f6ff40d1c00 2024-04-13T03:17:28.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.057+0000 7f6ff997c640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 0x7f6fd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.057+0000 7f6ff997c640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 0x7f6fd806edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0036000 tx=0x7f6ff002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.061+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ff4075180 con 0x7f6ff40d1c00 2024-04-13T03:17:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.304+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f6ff4068a10 con 0x7f6ff40d1c00 2024-04-13T03:17:28.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.330+0000 7f6fea7fc640 1 -- 172.21.15.106:0/1761906410 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v382) ==== 149+0+0 (secure 0 0 0) 0x7f6fe4059280 con 0x7f6ff40d1c00 2024-04-13T03:17:28.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 msgr2=0x7f6fd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 0x7f6fd806edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0036000 tx=0x7f6ff002e000 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 msgr2=0x7f6ff40cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400c4d0 tx=0x7f6fe400c990 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 shutdown_connections 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6fd806c8f0 0x7f6fd806edb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ff41622d0 0x7f6ff40cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ff40d1c00 0x7f6ff40cb0a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 --2- 172.21.15.106:0/1761906410 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6ff40b8080 0x7f6ff41f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 >> 172.21.15.106:0/1761906410 conn(0x7f6ff40d25d0 msgr2=0x7f6ff40b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 shutdown_connections 2024-04-13T03:17:28.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.333+0000 7f6ffbbe0640 1 -- 172.21.15.106:0/1761906410 wait complete. 2024-04-13T03:17:28.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:17:28.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.446+0000 7f85eb7c8640 1 Processor -- start 2024-04-13T03:17:28.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 -- start start 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f85e40b1c30 0x7f85e415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 0x7f85e40704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e40b5cf0 con 0x7f85e40709e0 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f85e40b5fd0 con 0x7f85e40b1c30 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.447+0000 7f85eb7c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f85e40b62b0 con 0x7f85e40b54a0 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f85e40b1c30 0x7f85e415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9d65640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f85e40709e0 0x7f85e4162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38592/0 (socket says 172.21.15.106:38592) 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9d65640 1 -- 172.21.15.106:0/961199426 learned_addr learned my addr 172.21.15.106:0/961199426 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 --2- 172.21.15.106:0/961199426 >> v2:172.21.15.106:3300/0 conn(0x7f85e40b1c30 0x7f85e415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36538/0 (socket says 172.21.15.106:36538) 2024-04-13T03:17:28.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e8d63640 1 --2- 172.21.15.106:0/961199426 >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 0x7f85e40704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 -- 172.21.15.106:0/961199426 >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 msgr2=0x7f85e40704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:28.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 --2- 172.21.15.106:0/961199426 >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 0x7f85e40704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 -- 172.21.15.106:0/961199426 >> v1:172.21.15.106:6789/0 conn(0x7f85e40709e0 legacy=0x7f85e4162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e9564640 1 -- 172.21.15.106:0/961199426 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40b6590 con 0x7f85e40b1c30 2024-04-13T03:17:28.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.448+0000 7f85e8d63640 1 --2- 172.21.15.106:0/961199426 >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 0x7f85e40704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:28.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85e9564640 1 --2- 172.21.15.106:0/961199426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b1c30 0x7f85e415ee40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f85e406c900 tx=0x7f85d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839874fc0df1175f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85dbfff640 1 -- 172.21.15.106:0/961199426 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4035020 con 0x7f85e40b1c30 2024-04-13T03:17:28.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85dbfff640 1 -- 172.21.15.106:0/961199426 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4031830 con 0x7f85e40b1c30 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85dbfff640 1 -- 172.21.15.106:0/961199426 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4031c00 con 0x7f85e40b1c30 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 -- 172.21.15.106:0/961199426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b1c30 msgr2=0x7f85e415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/961199426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b1c30 0x7f85e415ee40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f85e406c900 tx=0x7f85d402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 -- 172.21.15.106:0/961199426 shutdown_connections 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/961199426 >> v2:172.21.15.114:3300/0 conn(0x7f85e40b54a0 0x7f85e40704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/961199426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b1c30 0x7f85e415ee40 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 -- 172.21.15.106:0/961199426 >> 172.21.15.106:0/961199426 conn(0x7f85e4071870 msgr2=0x7f85e4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 -- 172.21.15.106:0/961199426 shutdown_connections 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.449+0000 7f85eb7c8640 1 -- 172.21.15.106:0/961199426 wait complete. 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 Processor -- start 2024-04-13T03:17:28.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 -- start start 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40709e0 0x7f85e40cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 0x7f85e4162900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85e406e7f0 con 0x7f85e40b54a0 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85e406e670 con 0x7f85e40709e0 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85eb7c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f85e406e970 con 0x7f85e40b1c30 2024-04-13T03:17:28.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85e9d65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85e9d65640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36554/0 (socket says 172.21.15.106:36554) 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.451+0000 7f85e9d65640 1 -- 172.21.15.106:0/950860814 learned_addr learned my addr 172.21.15.106:0/950860814 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9564640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40709e0 0x7f85e40cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e8d63640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 0x7f85e4162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 -- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 msgr2=0x7f85e4162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 0x7f85e4162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 -- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40709e0 msgr2=0x7f85e40cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40709e0 0x7f85e40cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 -- 172.21.15.106:0/950860814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40cbf50 con 0x7f85e40b54a0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e8d63640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 0x7f85e4162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85e9d65640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00cbc0 tx=0x7f85dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc017070 con 0x7f85e40b54a0 2024-04-13T03:17:28.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc00ad40 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc012600 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e40cc230 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.452+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e40cc7f0 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.453+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e4068a10 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.453+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85dc01a070 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.454+0000 7f85d9ffb640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.454+0000 7f85e9564640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.454+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f85dc08b7f0 con 0x7f85e40b54a0 2024-04-13T03:17:28.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.454+0000 7f85e9564640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 0x7f85c806edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85d4036000 tx=0x7f85d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.458+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85dc059600 con 0x7f85e40b54a0 2024-04-13T03:17:28.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.636+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f85e415c9d0 con 0x7f85e40b54a0 2024-04-13T03:17:28.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.641+0000 7f85d9ffb640 1 -- 172.21.15.106:0/950860814 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f85dc05d900 con 0x7f85e40b54a0 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 msgr2=0x7f85c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 0x7f85c806edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85d4036000 tx=0x7f85d4008030 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 msgr2=0x7f85e40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00cbc0 tx=0x7f85dc00a430 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 shutdown_connections 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f85c806c8f0 0x7f85c806edb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85e40b1c30 0x7f85e4162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e40b54a0 0x7f85e40ccff0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 --2- 172.21.15.106:0/950860814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40709e0 0x7f85e40cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 >> 172.21.15.106:0/950860814 conn(0x7f85e4071870 msgr2=0x7f85e415e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:28.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 shutdown_connections 2024-04-13T03:17:28.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.644+0000 7f85eb7c8640 1 -- 172.21.15.106:0/950860814 wait complete. 2024-04-13T03:17:28.655 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:17:28.656 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:17:28.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:17:28.750 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:17:28.750 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:28.750 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:28.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:17:28.766 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:17:28.766 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:28.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:17:28.783 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:28.783 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:28.820 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:17:28.821 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:17:28.821 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:28.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:17:28.841 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:17:28.841 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:28.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 Processor -- start 2024-04-13T03:17:28.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 -- start start 2024-04-13T03:17:28.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c15ed90 0x7f244c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244c0cfd20 con 0x7f244c15e9b0 2024-04-13T03:17:28.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244c162af0 con 0x7f244c0b3d60 2024-04-13T03:17:28.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.961+0000 7f2451dcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f244c162dd0 con 0x7f244c15ed90 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f244c15e9b0 0x7f244c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38594/0 (socket says 172.21.15.106:38594) 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244b7fe640 1 -- 172.21.15.106:0/4261076876 learned_addr learned my addr 172.21.15.106:0/4261076876 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3938627504 0 0) 0x7f244c0cfd20 con 0x7f244c15e9b0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2434003520 con 0x7f244c15e9b0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244affd640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c15ed90 0x7f244c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 879169864 0 0) 0x7f2434003520 con 0x7f244c15e9b0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c15ed90 msgr2=0x7f244c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c15ed90 0x7f244c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 >> v1:172.21.15.114:6789/0 conn(0x7f244c0b3d60 legacy=0x7f244c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1630b0 con 0x7f244c15e9b0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f243c001690 con 0x7f244c15e9b0 2024-04-13T03:17:28.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 >> v1:172.21.15.106:6789/0 conn(0x7f244c15e9b0 legacy=0x7f244c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f2434003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2434004800 0x7f2434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2434005180 0x7f243400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f244c162dd0 con 0x7f24340039e0 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f243400bf90 con 0x7f2434005180 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f243400c190 con 0x7f2434004800 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244b7fe640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f2434003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244affd640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2434004800 0x7f2434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.962+0000 7f244bfff640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2434005180 0x7f243400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 -- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2434004800 msgr2=0x7f2434004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2434004800 0x7f2434004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 -- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2434005180 msgr2=0x7f243400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2434005180 0x7f243400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 -- 172.21.15.106:0/4261076876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2434003520 con 0x7f24340039e0 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244affd640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2434004800 0x7f2434004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244b7fe640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f2434003e00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c210 tx=0x7f2434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f556436e113c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c007d10 con 0x7f24340039e0 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c030070 con 0x7f24340039e0 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.963+0000 7f244a7fc640 1 -- 172.21.15.106:0/4261076876 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c005d30 con 0x7f24340039e0 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 -- 172.21.15.106:0/4261076876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 msgr2=0x7f2434003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 --2- 172.21.15.106:0/4261076876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f2434003e00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c210 tx=0x7f2434005830 comp rx=0 tx=0).stop 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f244bfff640 1 -- 172.21.15.106:0/4261076876 reap_dead start 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 -- 172.21.15.106:0/4261076876 shutdown_connections 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 -- 172.21.15.106:0/4261076876 >> 172.21.15.106:0/4261076876 conn(0x7f244c08bf10 msgr2=0x7f244c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 -- 172.21.15.106:0/4261076876 shutdown_connections 2024-04-13T03:17:28.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.964+0000 7f2451dcc640 1 -- 172.21.15.106:0/4261076876 wait complete. 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.965+0000 7f2451dcc640 1 Processor -- start 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.965+0000 7f2451dcc640 1 -- start start 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244c0b6240 0x7f244c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c0ca360 0x7f244c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f244c162800 con 0x7f24340039e0 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f244c162680 con 0x7f244c0b6240 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f2451dcc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f244c0d4d00 con 0x7f244c0ca360 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36572/0 (socket says 172.21.15.106:36572) 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 -- 172.21.15.106:0/1499159396 learned_addr learned my addr 172.21.15.106:0/1499159396 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244affd640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244c0b6240 0x7f244c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244bfff640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c0ca360 0x7f244c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 -- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c0ca360 msgr2=0x7f244c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c0ca360 0x7f244c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 -- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244c0b6240 msgr2=0x7f244c1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.966+0000 7f244b7fe640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244c0b6240 0x7f244c1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:28.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f244b7fe640 1 -- 172.21.15.106:0/1499159396 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c0b66a0 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f244b7fe640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c210 tx=0x7f244c163280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c001c50 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c004bf0 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c0c9680 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c007070 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.967+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c0ccb30 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.968+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244c068a10 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.968+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f243c007210 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.969+0000 7f2433fff640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 0x7f243806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.969+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f243c0acfd0 con 0x7f24340039e0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.969+0000 7f244affd640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 0x7f243806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.969+0000 7f244affd640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 0x7f243806ede0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:28.973+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f243c07ae70 con 0x7f24340039e0 2024-04-13T03:17:29.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.185+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f244c0d53a0 con 0x7f24340039e0 2024-04-13T03:17:29.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.186+0000 7f2433fff640 1 -- 172.21.15.106:0/1499159396 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+3878 (secure 0 0 0) 0x7f243c07f170 con 0x7f24340039e0 2024-04-13T03:17:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:29.187 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3728366786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3728366786},{"type":"v1","addr":"172.21.15.106:6836","nonce":3728366786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1935798436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1935798436},{"type":"v1","addr":"172.21.15.114:6835","nonce":1935798436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/2444538235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2444538235},{"type":"v1","addr":"172.21.15.114:6834","nonce":2444538235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":8325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/224729349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":224729349},{"type":"v1","addr":"172.21.15.106:6837","nonce":224729349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[]} 2024-04-13T03:17:29.187 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 66 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 msgr2=0x7f243806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 0x7f243806ede0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440008040 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 msgr2=0x7f244c0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c210 tx=0x7f244c163280 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 shutdown_connections 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f243806c920 0x7f243806ede0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f244c0ca360 0x7f244c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f244c0b6240 0x7f244c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 --2- 172.21.15.106:0/1499159396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24340039e0 0x7f244c0c8f80 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.187+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 >> 172.21.15.106:0/1499159396 conn(0x7f244c08bf10 msgr2=0x7f244c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.188+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 shutdown_connections 2024-04-13T03:17:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.188+0000 7f2451dcc640 1 -- 172.21.15.106:0/1499159396 wait complete. 2024-04-13T03:17:29.200 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:17:29.200 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:17:29.200 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:17:29.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:17:29.200 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:17:29.200 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:17:29.201 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:17:29.201 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:17:29.201 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:17:29.201 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:17:29.201 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:17:29.201 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:17:29.201 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:17:29.201 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:17:29.202 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:17:29.202 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:17:29.209 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:17:29.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:17:29.210 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:17:29.210 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:17:29.214 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:17:29.214 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:17:29.215 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:17:29.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:17:29.252 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:17:29.252 DEBUG:teuthology.parallel:result is None 2024-04-13T03:17:29.253 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:17:29.253 DEBUG:teuthology.parallel:result is None 2024-04-13T03:17:29.256 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:17:29.256 DEBUG:teuthology.parallel:result is None 2024-04-13T03:17:29.257 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:17:29.257 DEBUG:teuthology.parallel:result is None 2024-04-13T03:17:29.257 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:17:29.328 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:17:29.329 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:17:29.343 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:17:29.344 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:17:29.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.365+0000 7fd98fba3640 1 Processor -- start 2024-04-13T03:17:29.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 -- start start 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd98815ed90 0x7fd9880d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9880b3d60 0x7fd9880d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9880cfd20 con 0x7fd98815e9b0 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd988162af0 con 0x7fd98815ed90 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.366+0000 7fd98fba3640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd988162dd0 con 0x7fd9880b3d60 2024-04-13T03:17:29.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd98815ed90 0x7fd9880d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98e140640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9880b3d60 0x7fd9880d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd98815ed90 0x7fd9880d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:47054/0 (socket says 172.21.15.106:47054) 2024-04-13T03:17:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 -- 172.21.15.106:0/3759883418 learned_addr learned my addr 172.21.15.106:0/3759883418 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 -- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9880b3d60 msgr2=0x7fd9880d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 --2- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9880b3d60 0x7fd9880d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 -- 172.21.15.106:0/3759883418 >> v1:172.21.15.114:6789/0 conn(0x7fd98815e9b0 legacy=0x7fd9880d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.367+0000 7fd98d13e640 1 -- 172.21.15.106:0/3759883418 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881630b0 con 0x7fd98815ed90 2024-04-13T03:17:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.368+0000 7fd98d13e640 1 --2- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd98815ed90 0x7fd9880d2300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd978009850 tx=0x7fd97802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f02c106ef9295594 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:29.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.372+0000 7fd98c93d640 1 -- 172.21.15.106:0/3759883418 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978004230 con 0x7fd98815ed90 2024-04-13T03:17:29.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.372+0000 7fd98c93d640 1 -- 172.21.15.106:0/3759883418 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9780043d0 con 0x7fd98815ed90 2024-04-13T03:17:29.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.372+0000 7fd98c93d640 1 -- 172.21.15.106:0/3759883418 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978003810 con 0x7fd98815ed90 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd98815ed90 msgr2=0x7fd9880d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd98815ed90 0x7fd9880d2300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd978009850 tx=0x7fd97802b9f0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/3759883418 shutdown_connections 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd9880b3d60 0x7fd9880d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/3759883418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd98815ed90 0x7fd9880d2300 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/3759883418 >> 172.21.15.106:0/3759883418 conn(0x7fd98808bf10 msgr2=0x7fd9880b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/3759883418 shutdown_connections 2024-04-13T03:17:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/3759883418 wait complete. 2024-04-13T03:17:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.374+0000 7fd98fba3640 1 Processor -- start 2024-04-13T03:17:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.374+0000 7fd98fba3640 1 -- start start 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 0x7fd9880b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd988160970 0x7fd988160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd988162800 con 0x7fd98815e9b0 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd988162680 con 0x7fd9880b3d60 2024-04-13T03:17:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98fba3640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd9880d4d00 con 0x7fd988160970 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36602/0 (socket says 172.21.15.106:36602) 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 -- 172.21.15.106:0/579576783 learned_addr learned my addr 172.21.15.106:0/579576783 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 -- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd988160970 msgr2=0x7fd988160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd988160970 0x7fd988160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d93f640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 0x7fd9880b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 -- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 msgr2=0x7fd9880b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 0x7fd9880b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 -- 172.21.15.106:0/579576783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9880cb980 con 0x7fd98815e9b0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d93f640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 0x7fd9880b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd98d13e640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd978009970 tx=0x7fd9780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978047070 con 0x7fd98815e9b0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.375+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd978003610 con 0x7fd98815e9b0 2024-04-13T03:17:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.376+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9880cbc60 con 0x7fd98815e9b0 2024-04-13T03:17:29.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.376+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780039e0 con 0x7fd98815e9b0 2024-04-13T03:17:29.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.376+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9880cc220 con 0x7fd98815e9b0 2024-04-13T03:17:29.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.376+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd988068a10 con 0x7fd98815e9b0 2024-04-13T03:17:29.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.377+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd97804a070 con 0x7fd98815e9b0 2024-04-13T03:17:29.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.377+0000 7fd97e7fc640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 0x7fd96c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:29.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.377+0000 7fd98d93f640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 0x7fd96c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:29.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.377+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5507+0+0 (secure 0 0 0) 0x7fd9780ad040 con 0x7fd98815e9b0 2024-04-13T03:17:29.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.378+0000 7fd98d93f640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 0x7fd96c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd984001680 tx=0x7fd984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:29.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.382+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd978033070 con 0x7fd98815e9b0 2024-04-13T03:17:29.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:29.570+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd9880d3e40 con 0x7fd98815e9b0 2024-04-13T03:17:30.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.370+0000 7fd97e7fc640 1 -- 172.21.15.106:0/579576783 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) ==== 97+0+0 (secure 0 0 0) 0x7fd97807b4a0 con 0x7fd98815e9b0 2024-04-13T03:17:30.371 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 msgr2=0x7fd96c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 0x7fd96c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd984001680 tx=0x7fd984009040 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 msgr2=0x7fd9880b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd978009970 tx=0x7fd9780046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 shutdown_connections 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd96c06c8f0 0x7fd96c06edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd988160970 0x7fd988160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd98815e9b0 0x7fd9880b4140 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 --2- 172.21.15.106:0/579576783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9880b3d60 0x7fd9880b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 >> 172.21.15.106:0/579576783 conn(0x7fd98808bf10 msgr2=0x7fd98815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 shutdown_connections 2024-04-13T03:17:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.373+0000 7fd98fba3640 1 -- 172.21.15.106:0/579576783 wait complete. 2024-04-13T03:17:30.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:17:30.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.487+0000 7fcfbb680640 1 Processor -- start 2024-04-13T03:17:30.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.487+0000 7fcfbb680640 1 -- start start 2024-04-13T03:17:30.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfbb680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 0x7fcfb40ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfbb680640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcfb406cb10 0x7fcfb4072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfbb680640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb40ce870 con 0x7fcfb406b8e0 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfbb680640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb40998d0 con 0x7fcfb406cb10 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfbb680640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb4073020 con 0x7fcfb40cdf50 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb8c1b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcfb406cb10 0x7fcfb4072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb9c1d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcfb406b8e0 0x7fcfb406bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38628/0 (socket says 172.21.15.106:38628) 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb9c1d640 1 -- 172.21.15.106:0/1747285505 learned_addr learned my addr 172.21.15.106:0/1747285505 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/1747285505 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 msgr2=0x7fcfb40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb941c640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 0x7fcfb40ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfabfff640 1 -- 172.21.15.106:0/1747285505 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443877603 0 0) 0x7fcfb40ce870 con 0x7fcfb406b8e0 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb8c1b640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 0x7fcfb40ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.488+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/1747285505 >> v1:172.21.15.106:6789/0 conn(0x7fcfb406b8e0 legacy=0x7fcfb406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/1747285505 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb4073300 con 0x7fcfb406cb10 2024-04-13T03:17:30.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfb941c640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 0x7fcfb40ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfb8c1b640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb4072720 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097a0 tx=0x7fcfa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6768616ace075a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfabfff640 1 -- 172.21.15.106:0/1747285505 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4035020 con 0x7fcfb406cb10 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfabfff640 1 -- 172.21.15.106:0/1747285505 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa4031830 con 0x7fcfb406cb10 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.489+0000 7fcfabfff640 1 -- 172.21.15.106:0/1747285505 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4031c00 con 0x7fcfb406cb10 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 -- 172.21.15.106:0/1747285505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 msgr2=0x7fcfb4072720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb4072720 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097a0 tx=0x7fcfa402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 -- 172.21.15.106:0/1747285505 shutdown_connections 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb4072720 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 --2- 172.21.15.106:0/1747285505 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb40cdf50 0x7fcfb40ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 -- 172.21.15.106:0/1747285505 >> 172.21.15.106:0/1747285505 conn(0x7fcfb40b1f10 msgr2=0x7fcfb40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 -- 172.21.15.106:0/1747285505 shutdown_connections 2024-04-13T03:17:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.490+0000 7fcfbb680640 1 -- 172.21.15.106:0/1747285505 wait complete. 2024-04-13T03:17:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.491+0000 7fcfbb680640 1 Processor -- start 2024-04-13T03:17:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.491+0000 7fcfbb680640 1 -- start start 2024-04-13T03:17:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 0x7fcfb41f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 0x7fcfb40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4072c60 con 0x7fcfb406cb10 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb40738c0 con 0x7fcfb40cdf50 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfbb680640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb4072de0 con 0x7fcfb406b8e0 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36618/0 (socket says 172.21.15.106:36618) 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/2732830932 learned_addr learned my addr 172.21.15.106:0/2732830932 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb941c640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 0x7fcfb41f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb9c1d640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 0x7fcfb40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 msgr2=0x7fcfb41f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 0x7fcfb41f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 msgr2=0x7fcfb40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 0x7fcfb40cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb8c1b640 1 -- 172.21.15.106:0/2732830932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41f80c0 con 0x7fcfb406cb10 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb941c640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 0x7fcfb41f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.492+0000 7fcfb9c1d640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 0x7fcfb40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfb8c1b640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40098c0 tx=0x7fcfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4041070 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa403a990 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41f83a0 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4035040 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.493+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41f7d20 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.494+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb40cf3c0 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.494+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfa404c060 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.495+0000 7fcfa9ffb640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 0x7fcf9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.495+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7fcfa40ad2f0 con 0x7fcfb406cb10 2024-04-13T03:17:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.495+0000 7fcfb941c640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 0x7fcf9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.495+0000 7fcfb941c640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 0x7fcf9806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.499+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfb40cf3c0 con 0x7fcfb406cb10 2024-04-13T03:17:30.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.741+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcfb4068a10 con 0x7fcfb406cb10 2024-04-13T03:17:30.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.741+0000 7fcfa9ffb640 1 -- 172.21.15.106:0/2732830932 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7fcfa407b970 con 0x7fcfb406cb10 2024-04-13T03:17:30.742 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:30.742 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:17:30.743 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:17:30.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.745 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.746 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.747 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.748 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.749 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.750 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.751 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:17:30.752 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:17:30.753 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:17:30.754 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:17:30.755 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 msgr2=0x7fcf9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 0x7fcf9806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 msgr2=0x7fcfb415bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40098c0 tx=0x7fcfa40046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 shutdown_connections 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fcf9806c8f0 0x7fcf9806edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb40cdf50 0x7fcfb40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfb406cb10 0x7fcfb415bbf0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 --2- 172.21.15.106:0/2732830932 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcfb406b8e0 0x7fcfb41f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 >> 172.21.15.106:0/2732830932 conn(0x7fcfb40b1f10 msgr2=0x7fcfb40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:30.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 shutdown_connections 2024-04-13T03:17:30.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.743+0000 7fcfbb680640 1 -- 172.21.15.106:0/2732830932 wait complete. 2024-04-13T03:17:30.758 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:30.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.857+0000 7f575086d640 1 Processor -- start 2024-04-13T03:17:30.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.857+0000 7f575086d640 1 -- start start 2024-04-13T03:17:30.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f575086d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 0x7f574c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f575086d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f575086d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c0ce870 con 0x7f574c06b8e0 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f575086d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f574c0998d0 con 0x7f574c06cb10 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f575086d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f574c073020 con 0x7f574c0cdf50 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f5749d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f574ad9d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f574c06b8e0 0x7f574c06bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:47296/0 (socket says 172.21.15.106:47296) 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f574a59c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 0x7f574c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f5749d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:47082/0 (socket says 172.21.15.106:47082) 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.858+0000 7f574ad9d640 1 -- 172.21.15.106:0/3738150566 learned_addr learned my addr 172.21.15.106:0/3738150566 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574a59c640 1 -- 172.21.15.106:0/3738150566 >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 msgr2=0x7f574c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574a59c640 1 --2- 172.21.15.106:0/3738150566 >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574a59c640 1 -- 172.21.15.106:0/3738150566 >> v1:172.21.15.114:6789/0 conn(0x7f574c06b8e0 legacy=0x7f574c06bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574a59c640 1 -- 172.21.15.106:0/3738150566 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c073300 con 0x7f574c0cdf50 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574959a640 1 -- 172.21.15.106:0/3738150566 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2059480348 0 0) 0x7f574c0ce870 con 0x7f574c06b8e0 2024-04-13T03:17:30.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f5749d9b640 1 --2- 172.21.15.106:0/3738150566 >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.859+0000 7f574a59c640 1 --2- 172.21.15.106:0/3738150566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 0x7f574c0ce330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f573c009d90 tx=0x7f573c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184edf14e2f495a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f574959a640 1 -- 172.21.15.106:0/3738150566 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c039070 con 0x7f574c0cdf50 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f574959a640 1 -- 172.21.15.106:0/3738150566 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c02cc50 con 0x7f574c0cdf50 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f574959a640 1 -- 172.21.15.106:0/3738150566 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c034440 con 0x7f574c0cdf50 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 -- 172.21.15.106:0/3738150566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 msgr2=0x7f574c0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 --2- 172.21.15.106:0/3738150566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 0x7f574c0ce330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f573c009d90 tx=0x7f573c02bea0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 -- 172.21.15.106:0/3738150566 shutdown_connections 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 --2- 172.21.15.106:0/3738150566 >> v2:172.21.15.114:3300/0 conn(0x7f574c06cb10 0x7f574c072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 --2- 172.21.15.106:0/3738150566 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c0cdf50 0x7f574c0ce330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 -- 172.21.15.106:0/3738150566 >> 172.21.15.106:0/3738150566 conn(0x7f574c0b1f10 msgr2=0x7f574c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:30.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 -- 172.21.15.106:0/3738150566 shutdown_connections 2024-04-13T03:17:30.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.860+0000 7f575086d640 1 -- 172.21.15.106:0/3738150566 wait complete. 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.861+0000 7f575086d640 1 Processor -- start 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.861+0000 7f575086d640 1 -- start start 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c06b8e0 0x7f574c0c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f574c0cdf50 0x7f574c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f574c072c60 con 0x7f574c06cb10 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f574c0738c0 con 0x7f574c0cdf50 2024-04-13T03:17:30.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f575086d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f574c072de0 con 0x7f574c06b8e0 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f5749d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f5749d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36620/0 (socket says 172.21.15.106:36620) 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f5749d9b640 1 -- 172.21.15.106:0/3225297084 learned_addr learned my addr 172.21.15.106:0/3225297084 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f574a59c640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c06b8e0 0x7f574c0c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.862+0000 7f574ad9d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f574c0cdf50 0x7f574c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 -- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c06b8e0 msgr2=0x7f574c0c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c06b8e0 0x7f574c0c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 -- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f574c0cdf50 msgr2=0x7f574c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f574c0cdf50 0x7f574c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 -- 172.21.15.106:0/3225297084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c0c8a80 con 0x7f574c06cb10 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f5749d9b640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f574000e9b0 tx=0x7f574000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740017070 con 0x7f574c06cb10 2024-04-13T03:17:30.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c0c8d60 con 0x7f574c06cb10 2024-04-13T03:17:30.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574000ca00 con 0x7f574c06cb10 2024-04-13T03:17:30.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574000cdf0 con 0x7f574c06cb10 2024-04-13T03:17:30.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.863+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c0c6340 con 0x7f574c06cb10 2024-04-13T03:17:30.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.864+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c068a10 con 0x7f574c06cb10 2024-04-13T03:17:30.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.864+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f574001a070 con 0x7f574c06cb10 2024-04-13T03:17:30.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.865+0000 7f573affd640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:30.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.865+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7f574008b960 con 0x7f574c06cb10 2024-04-13T03:17:30.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.865+0000 7f574a59c640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:30.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.865+0000 7f574a59c640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 0x7f572c06edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f573c036000 tx=0x7f573c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:30.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:30.869+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5740059f60 con 0x7f574c06cb10 2024-04-13T03:17:31.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.046+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f574c0c9420 con 0x7f574c06cb10 2024-04-13T03:17:31.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.047+0000 7f573affd640 1 -- 172.21.15.106:0/3225297084 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+13202 (secure 0 0 0) 0x7f5740059860 con 0x7f574c06cb10 2024-04-13T03:17:31.048 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:31.048 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":103,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:30.366635+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 msgr2=0x7f572c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 0x7f572c06edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f573c036000 tx=0x7f573c030000 comp rx=0 tx=0).stop 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 msgr2=0x7f574c0ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f574000e9b0 tx=0x7f574000ee70 comp rx=0 tx=0).stop 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 shutdown_connections 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f574c0cdf50 0x7f574c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f574c06cb10 0x7f574c0ccb80 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 --2- 172.21.15.106:0/3225297084 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f574c06b8e0 0x7f574c0c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 >> 172.21.15.106:0/3225297084 conn(0x7f574c0b1f10 msgr2=0x7f574c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 shutdown_connections 2024-04-13T03:17:31.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.049+0000 7f575086d640 1 -- 172.21.15.106:0/3225297084 wait complete. 2024-04-13T03:17:31.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:31.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:31.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.162+0000 7f84dc66d640 1 Processor -- start 2024-04-13T03:17:31.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.162+0000 7f84dc66d640 1 -- start start 2024-04-13T03:17:31.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.163+0000 7f84dc66d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84d415ed90 0x7f84d40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.163+0000 7f84dc66d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d40b3d60 0x7f84d40d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.163+0000 7f84dc66d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d40cfd20 con 0x7f84d415e9b0 2024-04-13T03:17:31.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.163+0000 7f84dc66d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f84d4162af0 con 0x7f84d415ed90 2024-04-13T03:17:31.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.163+0000 7f84dc66d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84d4162dd0 con 0x7f84d40b3d60 2024-04-13T03:17:31.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9c08640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84d415ed90 0x7f84d40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84da409640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f84d415e9b0 0x7f84d40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:47310/0 (socket says 172.21.15.106:47310) 2024-04-13T03:17:31.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84dac0a640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d40b3d60 0x7f84d40d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84da409640 1 -- 172.21.15.106:0/649199512 learned_addr learned my addr 172.21.15.106:0/649199512 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9c08640 1 -- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d40b3d60 msgr2=0x7f84d40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9c08640 1 --2- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d40b3d60 0x7f84d40d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9c08640 1 -- 172.21.15.106:0/649199512 >> v1:172.21.15.114:6789/0 conn(0x7f84d415e9b0 legacy=0x7f84d40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9c08640 1 -- 172.21.15.106:0/649199512 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d41630b0 con 0x7f84d415ed90 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.164+0000 7f84d9407640 1 -- 172.21.15.106:0/649199512 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748594356 0 0) 0x7f84d40cfd20 con 0x7f84d415e9b0 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84d9c08640 1 --2- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d415ed90 0x7f84d40d2300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009790 tx=0x7f84c402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51568e134dba662b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84d9407640 1 -- 172.21.15.106:0/649199512 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4035020 con 0x7f84d415ed90 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84d9407640 1 -- 172.21.15.106:0/649199512 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c4031890 con 0x7f84d415ed90 2024-04-13T03:17:31.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84d9407640 1 -- 172.21.15.106:0/649199512 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031c60 con 0x7f84d415ed90 2024-04-13T03:17:31.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 -- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d415ed90 msgr2=0x7f84d40d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 --2- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d415ed90 0x7f84d40d2300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009790 tx=0x7f84c402b930 comp rx=0 tx=0).stop 2024-04-13T03:17:31.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 -- 172.21.15.106:0/649199512 shutdown_connections 2024-04-13T03:17:31.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 --2- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d40b3d60 0x7f84d40d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 --2- 172.21.15.106:0/649199512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d415ed90 0x7f84d40d2300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 -- 172.21.15.106:0/649199512 >> 172.21.15.106:0/649199512 conn(0x7f84d408bf10 msgr2=0x7f84d40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 -- 172.21.15.106:0/649199512 shutdown_connections 2024-04-13T03:17:31.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.165+0000 7f84dc66d640 1 -- 172.21.15.106:0/649199512 wait complete. 2024-04-13T03:17:31.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 Processor -- start 2024-04-13T03:17:31.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 -- start start 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d415e9b0 0x7f84d41e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 0x7f84d41e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4162800 con 0x7f84d40b3d60 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4162680 con 0x7f84d41601d0 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84dc66d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f84d40d4d00 con 0x7f84d415e9b0 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84da409640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84da409640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36626/0 (socket says 172.21.15.106:36626) 2024-04-13T03:17:31.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.167+0000 7f84da409640 1 -- 172.21.15.106:0/4032040391 learned_addr learned my addr 172.21.15.106:0/4032040391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84dac0a640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 0x7f84d41e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84d9c08640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d415e9b0 0x7f84d41e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 -- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d415e9b0 msgr2=0x7f84d41e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d415e9b0 0x7f84d41e8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 -- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 msgr2=0x7f84d41e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 0x7f84d41e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 -- 172.21.15.106:0/4032040391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d4160750 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84dac0a640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 0x7f84d41e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84da409640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c890 tx=0x7f84d000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0018070 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d000a640 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d4160a30 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d000aa30 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.168+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d4160ff0 con 0x7f84d40b3d60 2024-04-13T03:17:31.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.169+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4068a10 con 0x7f84d40b3d60 2024-04-13T03:17:31.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.169+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84d001b070 con 0x7f84d40b3d60 2024-04-13T03:17:31.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.170+0000 7f84caffd640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 0x7f84b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.170+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7f84d008afe0 con 0x7f84d40b3d60 2024-04-13T03:17:31.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.170+0000 7f84d9c08640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 0x7f84b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.170+0000 7f84d9c08640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 0x7f84b806ede0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.174+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84d0059660 con 0x7f84d40b3d60 2024-04-13T03:17:31.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.349+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84d40ca9d0 con 0x7f84d40b3d60 2024-04-13T03:17:31.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.350+0000 7f84caffd640 1 -- 172.21.15.106:0/4032040391 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f84d0058f60 con 0x7f84d40b3d60 2024-04-13T03:17:31.351 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 msgr2=0x7f84b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 0x7f84b806ede0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c4008030 comp rx=0 tx=0).stop 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 msgr2=0x7f84d41617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c890 tx=0x7f84d000cd50 comp rx=0 tx=0).stop 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 shutdown_connections 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84b806c920 0x7f84b806ede0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f84d415e9b0 0x7f84d41e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84d41601d0 0x7f84d41e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 --2- 172.21.15.106:0/4032040391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d40b3d60 0x7f84d41617f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 >> 172.21.15.106:0/4032040391 conn(0x7f84d408bf10 msgr2=0x7f84d40b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 shutdown_connections 2024-04-13T03:17:31.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.352+0000 7f84dc66d640 1 -- 172.21.15.106:0/4032040391 wait complete. 2024-04-13T03:17:31.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:17:31.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.463+0000 7fa7b4d5e640 1 Processor -- start 2024-04-13T03:17:31.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.463+0000 7fa7b4d5e640 1 -- start start 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.464+0000 7fa7b4d5e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7b015e9b0 0x7fa7b00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.464+0000 7fa7b4d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa7b00d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.464+0000 7fa7b4d5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b00cfd20 con 0x7fa7b015ed90 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.464+0000 7fa7b4d5e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b0162af0 con 0x7fa7b015e9b0 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.464+0000 7fa7b4d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b0162dd0 con 0x7fa7b00b3d60 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7add9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa7b015ed90 0x7fa7b00d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38634/0 (socket says 172.21.15.106:38634) 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7add9b640 1 -- 172.21.15.106:0/2654898263 learned_addr learned my addr 172.21.15.106:0/2654898263 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ae59c640 1 --2- 172.21.15.106:0/2654898263 >> v2:172.21.15.114:3300/0 conn(0x7fa7b015e9b0 0x7fa7b00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa7b00d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397437084 0 0) 0x7fa7b00cfd20 con 0x7fa7b015ed90 2024-04-13T03:17:31.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa798003520 con 0x7fa7b015ed90 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3231186186 0 0) 0x7fa798003520 con 0x7fa7b015ed90 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 msgr2=0x7fa7b00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa7b00d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 >> v2:172.21.15.114:3300/0 conn(0x7fa7b015e9b0 msgr2=0x7fa7b00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 --2- 172.21.15.106:0/2654898263 >> v2:172.21.15.114:3300/0 conn(0x7fa7b015e9b0 0x7fa7b00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01630b0 con 0x7fa7b015ed90 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa7a4001a40 con 0x7fa7b015ed90 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 >> v1:172.21.15.106:6789/0 conn(0x7fa7b015ed90 legacy=0x7fa7b00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7980039b0 0x7fa798003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0162af0 con 0x7fa798004d40 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0162dd0 con 0x7fa7980039b0 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa79800b9c0 con 0x7fa7b00b3d60 2024-04-13T03:17:31.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7ae59c640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7980039b0 0x7fa798003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7add9b640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 -- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 msgr2=0x7fa798004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b00b3d60 0x7fa798004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 -- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7980039b0 msgr2=0x7fa798003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7980039b0 0x7fa798003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.465+0000 7fa7aed9d640 1 -- 172.21.15.106:0/2654898263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa798003520 con 0x7fa798004d40 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.466+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7980051c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009a90 tx=0x7fa798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea521e9bc16a2df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.466+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0039070 con 0x7fa798004d40 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.466+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a002c680 con 0x7fa798004d40 2024-04-13T03:17:31.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.466+0000 7fa7ad59a640 1 -- 172.21.15.106:0/2654898263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0009690 con 0x7fa798004d40 2024-04-13T03:17:31.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/2654898263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 msgr2=0x7fa7980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/2654898263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7980051c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009a90 tx=0x7fa798005830 comp rx=0 tx=0).stop 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7aed9d640 1 -- 172.21.15.106:0/2654898263 reap_dead start 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/2654898263 shutdown_connections 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/2654898263 >> 172.21.15.106:0/2654898263 conn(0x7fa7b008bf10 msgr2=0x7fa7b00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/2654898263 shutdown_connections 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.467+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/2654898263 wait complete. 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.468+0000 7fa7b4d5e640 1 Processor -- start 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.468+0000 7fa7b4d5e640 1 -- start start 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b006d8b0 0x7fa7b01e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 0x7fa7b0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0162800 con 0x7fa798004d40 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0162680 con 0x7fa7b0155d10 2024-04-13T03:17:31.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7b4d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b00d4d00 con 0x7fa7b006d8b0 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36640/0 (socket says 172.21.15.106:36640) 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 -- 172.21.15.106:0/3931343673 learned_addr learned my addr 172.21.15.106:0/3931343673 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7add9b640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b006d8b0 0x7fa7b01e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 0x7fa7b0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 -- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b006d8b0 msgr2=0x7fa7b01e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b006d8b0 0x7fa7b01e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 -- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 msgr2=0x7fa7b0156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 0x7fa7b0156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7ae59c640 1 -- 172.21.15.106:0/3931343673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01566d0 con 0x7fa798004d40 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.469+0000 7fa7aed9d640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 0x7fa7b0156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:31.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa7ae59c640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8011960 tx=0x7fa7a8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800faf0 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a800fc90 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b00c8120 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8014440 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.470+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b006e0e0 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.471+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0068a10 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.471+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7a8012040 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.472+0000 7fa79effd640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 0x7fa79006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.472+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7fa7a808bfb0 con 0x7fa798004d40 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.472+0000 7fa7add9b640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 0x7fa79006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.472+0000 7fa7add9b640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 0x7fa79006edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000f80 tx=0x7fa7a4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.476+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7a8019030 con 0x7fa798004d40 2024-04-13T03:17:31.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.654+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa7b00d53a0 con 0x7fa798004d40 2024-04-13T03:17:31.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.655+0000 7fa79effd640 1 -- 172.21.15.106:0/3931343673 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) ==== 84+0+65 (secure 0 0 0) 0x7fa7a805a5a0 con 0x7fa798004d40 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:31.656 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 msgr2=0x7fa79006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 0x7fa79006edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000f80 tx=0x7fa7a4006040 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 msgr2=0x7fa7b01568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8011960 tx=0x7fa7a8011e20 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 shutdown_connections 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa79006c8f0 0x7fa79006edb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa7b006d8b0 0x7fa7b01e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7b0155d10 0x7fa7b0156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 --2- 172.21.15.106:0/3931343673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa798004d40 0x7fa7b01568c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 >> 172.21.15.106:0/3931343673 conn(0x7fa7b008bf10 msgr2=0x7fa7b00b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 shutdown_connections 2024-04-13T03:17:31.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.657+0000 7fa7b4d5e640 1 -- 172.21.15.106:0/3931343673 wait complete. 2024-04-13T03:17:31.669 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:17:31.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:17:31.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.772+0000 7f8ca37cc640 1 Processor -- start 2024-04-13T03:17:31.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.772+0000 7f8ca37cc640 1 -- start start 2024-04-13T03:17:31.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca37cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c9c15b620 0x7f8c9c06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca37cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c9c06f030 con 0x7f8c9c15c110 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca37cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c9c06f310 con 0x7f8c9c15c760 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca37cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8c9c0ce1c0 con 0x7f8c9c15b620 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca1d69640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8c9c15c110 0x7f8c9c0ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38650/0 (socket says 172.21.15.106:38650) 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca1d69640 1 -- 172.21.15.106:0/3857845516 learned_addr learned my addr 172.21.15.106:0/3857845516 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211293772 0 0) 0x7f8c9c06f030 con 0x7f8c9c15c110 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3857845516 >> v2:172.21.15.114:3300/0 conn(0x7f8c9c15b620 0x7f8c9c06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c88003520 con 0x7f8c9c15c110 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3386490918 0 0) 0x7f8c88003520 con 0x7f8c9c15c110 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 >> v2:172.21.15.114:3300/0 conn(0x7f8c9c15b620 msgr2=0x7f8c9c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 --2- 172.21.15.106:0/3857845516 >> v2:172.21.15.114:3300/0 conn(0x7f8c9c15b620 0x7f8c9c06e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 >> v1:172.21.15.114:6789/0 conn(0x7f8c9c15c760 legacy=0x7f8c9c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c06f5f0 con 0x7f8c9c15c110 2024-04-13T03:17:31.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f8c940019f0 con 0x7f8c9c15c110 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 >> v1:172.21.15.106:6789/0 conn(0x7f8c9c15c110 legacy=0x7f8c9c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880039b0 0x7f8c88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.773+0000 7f8c8ffff640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c88004800 0x7f8c88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8c8ffff640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 0x7f8c8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8800c030 con 0x7f8c880051c0 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8800c1e0 con 0x7f8c880039b0 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8800c3e0 con 0x7f8c88004800 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880039b0 0x7f8c88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 0x7f8c8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca0d67640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c88004800 0x7f8c88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 -- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c88004800 msgr2=0x7f8c88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c88004800 0x7f8c88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 -- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880039b0 msgr2=0x7f8c88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880039b0 0x7f8c88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.774+0000 7f8ca1d69640 1 -- 172.21.15.106:0/3857845516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c88003520 con 0x7f8c880051c0 2024-04-13T03:17:31.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca1d69640 1 -- 172.21.15.106:0/3857845516 reap_dead start 2024-04-13T03:17:31.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 0x7f8c8800b9c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400a850 tx=0x7f8c88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1c7ed04922c0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400a4a0 con 0x7f8c880051c0 2024-04-13T03:17:31.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c94034650 con 0x7f8c880051c0 2024-04-13T03:17:31.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8c8ffff640 1 -- 172.21.15.106:0/3857845516 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94034a40 con 0x7f8c880051c0 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 msgr2=0x7f8c8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 0x7f8c8800b9c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400a850 tx=0x7f8c88005830 comp rx=0 tx=0).stop 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3857845516 shutdown_connections 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3857845516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c880051c0 0x7f8c8800b9c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3857845516 >> 172.21.15.106:0/3857845516 conn(0x7f8c9c0d0370 msgr2=0x7f8c9c0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3857845516 shutdown_connections 2024-04-13T03:17:31.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.775+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3857845516 wait complete. 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 Processor -- start 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 -- start start 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 0x7f8c9c154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 0x7f8c9c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c15ba00 con 0x7f8c9c157020 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c06fbb0 con 0x7f8c880051c0 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca37cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c15bb80 con 0x7f8c9c0aed10 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca0d67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca0d67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36668/0 (socket says 172.21.15.106:36668) 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.777+0000 7f8ca0d67640 1 -- 172.21.15.106:0/3831509563 learned_addr learned my addr 172.21.15.106:0/3831509563 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 0x7f8c9c154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 0x7f8c9c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 -- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 msgr2=0x7f8c9c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 0x7f8c9c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 -- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 msgr2=0x7f8c9c154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 0x7f8c9c154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 -- 172.21.15.106:0/3831509563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c1569f0 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 0x7f8c9c154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca1d69640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 0x7f8c9c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca0d67640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800f5f0 tx=0x7f8c9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c980052e0 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c98018070 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98003810 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c0adf50 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.778+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c0ae510 con 0x7f8c9c157020 2024-04-13T03:17:31.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.779+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c068a10 con 0x7f8c9c157020 2024-04-13T03:17:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.779+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c980039b0 con 0x7f8c9c157020 2024-04-13T03:17:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.780+0000 7f8c8dffb640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 0x7f8c8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.780+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 0x7f8c8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.780+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7f8c9808b810 con 0x7f8c9c157020 2024-04-13T03:17:31.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.780+0000 7f8ca1568640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 0x7f8c8006ede0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009b40 tx=0x7f8c90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:31.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.784+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c98059d70 con 0x7f8c9c157020 2024-04-13T03:17:31.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:31.960+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8c9c072720 con 0x7f8c9c157020 2024-04-13T03:17:32.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.375+0000 7f8c8dffb640 1 -- 172.21.15.106:0/3831509563 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) ==== 138+0+0 (secure 0 0 0) 0x7f8c98059670 con 0x7f8c9c157020 2024-04-13T03:17:32.377 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 msgr2=0x7f8c8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 0x7f8c8006ede0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009b40 tx=0x7f8c90008040 comp rx=0 tx=0).stop 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 msgr2=0x7f8c9c157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800f5f0 tx=0x7f8c9800fab0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 shutdown_connections 2024-04-13T03:17:32.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8c8006c920 0x7f8c8006ede0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8c9c0aed10 0x7f8c9c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c9c157020 0x7f8c9c157480 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 --2- 172.21.15.106:0/3831509563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c880051c0 0x7f8c9c154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.378+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 >> 172.21.15.106:0/3831509563 conn(0x7f8c9c0d0370 msgr2=0x7f8c9c0b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.379+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 shutdown_connections 2024-04-13T03:17:32.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.379+0000 7f8ca37cc640 1 -- 172.21.15.106:0/3831509563 wait complete. 2024-04-13T03:17:32.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:17:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.502+0000 7fa6ac11e640 1 Processor -- start 2024-04-13T03:17:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.502+0000 7fa6ac11e640 1 -- start start 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.502+0000 7fa6ac11e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a415e930 0x7fa6a40b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6ac11e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6a406ed70 0x7fa6a40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6ac11e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6a40d0820 0x7fa6a40cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6ac11e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a40d0c90 con 0x7fa6a406ed70 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6ac11e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a406e280 con 0x7fa6a40d0820 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6ac11e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a406e560 con 0x7fa6a415e930 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6a406ed70 0x7fa6a40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6a406ed70 0x7fa6a40d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36682/0 (socket says 172.21.15.106:36682) 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 -- 172.21.15.106:0/3730890906 learned_addr learned my addr 172.21.15.106:0/3730890906 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a9eba640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a415e930 0x7fa6a40b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6aa6bb640 1 --2- 172.21.15.106:0/3730890906 >> v2:172.21.15.114:3300/0 conn(0x7fa6a40d0820 0x7fa6a40cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 -- 172.21.15.106:0/3730890906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a415e930 msgr2=0x7fa6a40b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a415e930 0x7fa6a40b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 -- 172.21.15.106:0/3730890906 >> v2:172.21.15.114:3300/0 conn(0x7fa6a40d0820 msgr2=0x7fa6a40cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 --2- 172.21.15.106:0/3730890906 >> v2:172.21.15.114:3300/0 conn(0x7fa6a40d0820 0x7fa6a40cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.503+0000 7fa6a96b9640 1 -- 172.21.15.106:0/3730890906 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a40cccf0 con 0x7fa6a406ed70 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6a96b9640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a406ed70 0x7fa6a40d02e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40cfc10 tx=0x7fa69402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fae8aae7982553 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6a8eb8640 1 -- 172.21.15.106:0/3730890906 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694035020 con 0x7fa6a406ed70 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6a8eb8640 1 -- 172.21.15.106:0/3730890906 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694031870 con 0x7fa6a406ed70 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6a8eb8640 1 -- 172.21.15.106:0/3730890906 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694031c40 con 0x7fa6a406ed70 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3730890906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a406ed70 msgr2=0x7fa6a40d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a406ed70 0x7fa6a40d02e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40cfc10 tx=0x7fa69402b910 comp rx=0 tx=0).stop 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3730890906 shutdown_connections 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3730890906 >> v2:172.21.15.114:3300/0 conn(0x7fa6a40d0820 0x7fa6a40cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a406ed70 0x7fa6a40d02e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3730890906 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a415e930 0x7fa6a40b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.504+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3730890906 >> 172.21.15.106:0/3730890906 conn(0x7fa6a40cef80 msgr2=0x7fa6a40cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.505+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3730890906 shutdown_connections 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.505+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3730890906 wait complete. 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 Processor -- start 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 -- start start 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a406ed70 0x7fa6a40ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6a40d0820 0x7fa6a4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a406f1d0 con 0x7fa6a415e930 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40cd2b0 con 0x7fa6a40d0820 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6ac11e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40ce830 con 0x7fa6a406ed70 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6aa6bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.506+0000 7fa6aa6bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36690/0 (socket says 172.21.15.106:36690) 2024-04-13T03:17:32.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 -- 172.21.15.106:0/3797974283 learned_addr learned my addr 172.21.15.106:0/3797974283 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6a96b9640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6a40d0820 0x7fa6a4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6a9eba640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a406ed70 0x7fa6a40ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 -- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a406ed70 msgr2=0x7fa6a40ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a406ed70 0x7fa6a40ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 -- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6a40d0820 msgr2=0x7fa6a4158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6a40d0820 0x7fa6a4158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 -- 172.21.15.106:0/3797974283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a41576c0 con 0x7fa6a415e930 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6aa6bb640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007e60 tx=0x7fa69c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c014620 con 0x7fa6a415e930 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c019070 con 0x7fa6a415e930 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a40c9820 con 0x7fa6a415e930 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c014a20 con 0x7fa6a415e930 2024-04-13T03:17:32.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.507+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a40c9de0 con 0x7fa6a415e930 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.508+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a406d750 con 0x7fa6a415e930 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.508+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa69c014bc0 con 0x7fa6a415e930 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.509+0000 7fa69a7fc640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 0x7fa68806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.509+0000 7fa6a9eba640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 0x7fa68806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.509+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5619+0+0 (secure 0 0 0) 0x7fa69c08c380 con 0x7fa6a415e930 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.509+0000 7fa6a9eba640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 0x7fa68806edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006fd0 tx=0x7fa6a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.514+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6a406d750 con 0x7fa6a415e930 2024-04-13T03:17:32.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:32.690+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa6a4068a10 con 0x7fa6a415e930 2024-04-13T03:17:33.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.381+0000 7fa69a7fc640 1 -- 172.21.15.106:0/3797974283 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) ==== 170+0+0 (secure 0 0 0) 0x7fa69c05a800 con 0x7fa6a415e930 2024-04-13T03:17:33.382 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:17:33.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.384+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 msgr2=0x7fa68806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.384+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 0x7fa68806edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006fd0 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.384+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 msgr2=0x7fa6a41f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007e60 tx=0x7fa69c00e4b0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 shutdown_connections 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68806c8f0 0x7fa68806edb0 secure :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006fd0 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6a415e930 0x7fa6a41f83c0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6a40d0820 0x7fa6a4158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 --2- 172.21.15.106:0/3797974283 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa6a406ed70 0x7fa6a40ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 >> 172.21.15.106:0/3797974283 conn(0x7fa6a40cef80 msgr2=0x7fa6a40b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 shutdown_connections 2024-04-13T03:17:33.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.385+0000 7fa6ac11e640 1 -- 172.21.15.106:0/3797974283 wait complete. 2024-04-13T03:17:33.412 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:17:33.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.515+0000 7f2ec515e640 1 Processor -- start 2024-04-13T03:17:33.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.515+0000 7f2ec515e640 1 -- start start 2024-04-13T03:17:33.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ec515e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ec015d400 0x7f2ec013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ec515e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ec515e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec01429b0 con 0x7f2ec00cf720 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ec515e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec006c0d0 con 0x7f2ec015d7e0 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ec515e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec006ba80 con 0x7f2ec015d400 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54984/0 (socket says 172.21.15.106:54984) 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 -- 172.21.15.106:0/3886802551 learned_addr learned my addr 172.21.15.106:0/3886802551 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2ebed9d640 1 --2- 172.21.15.106:0/3886802551 >> v2:172.21.15.114:3300/0 conn(0x7f2ec015d400 0x7f2ec013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 -- 172.21.15.106:0/3886802551 >> v2:172.21.15.114:3300/0 conn(0x7f2ec015d400 msgr2=0x7f2ec013fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 --2- 172.21.15.106:0/3886802551 >> v2:172.21.15.114:3300/0 conn(0x7f2ec015d400 0x7f2ec013fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 -- 172.21.15.106:0/3886802551 >> v1:172.21.15.114:6789/0 conn(0x7f2ec00cf720 legacy=0x7f2ec0141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.516+0000 7f2eb7fff640 1 -- 172.21.15.106:0/3886802551 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec006bd60 con 0x7f2ec015d7e0 2024-04-13T03:17:33.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2eb7fff640 1 --2- 172.21.15.106:0/3886802551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00b3090 tx=0x7f2ea802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37a5077d88c2483 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ebe59c640 1 -- 172.21.15.106:0/3886802551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea802e070 con 0x7f2ec015d7e0 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ebe59c640 1 -- 172.21.15.106:0/3886802551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea8037070 con 0x7f2ec015d7e0 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ebe59c640 1 -- 172.21.15.106:0/3886802551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8031a00 con 0x7f2ec015d7e0 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 -- 172.21.15.106:0/3886802551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 msgr2=0x7f2ec00cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3886802551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00b3090 tx=0x7f2ea802b880 comp rx=0 tx=0).stop 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 -- 172.21.15.106:0/3886802551 shutdown_connections 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3886802551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec00cf1e0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3886802551 >> v2:172.21.15.114:3300/0 conn(0x7f2ec015d400 0x7f2ec013fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 -- 172.21.15.106:0/3886802551 >> 172.21.15.106:0/3886802551 conn(0x7f2ec00b2600 msgr2=0x7f2ec00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 -- 172.21.15.106:0/3886802551 shutdown_connections 2024-04-13T03:17:33.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.517+0000 7f2ec515e640 1 -- 172.21.15.106:0/3886802551 wait complete. 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 Processor -- start 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 -- start start 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ec00cf720 0x7f2ec01f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 0x7f2ec00b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0142550 con 0x7f2ec015d7e0 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01423d0 con 0x7f2ec015d400 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.519+0000 7f2ec515e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec01426d0 con 0x7f2ec00cf720 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54988/0 (socket says 172.21.15.106:54988) 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 -- 172.21.15.106:0/3159664426 learned_addr learned my addr 172.21.15.106:0/3159664426 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2eb7fff640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 0x7f2ec00b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 -- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ec00cf720 msgr2=0x7f2ec01f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ec00cf720 0x7f2ec01f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 -- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 msgr2=0x7f2ec00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 0x7f2ec00b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 -- 172.21.15.106:0/3159664426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec00b4af0 con 0x7f2ec015d7e0 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2eb7fff640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 0x7f2ec00b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ebf59e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800b970 tx=0x7f2eb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:33.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8019070 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb800cb40 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec006ddd0 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8014440 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.520+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec006e390 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.521+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec0068a10 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.522+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2eb801c070 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.522+0000 7f2eb77fe640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 0x7f2ea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.522+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6001+0+0 (secure 0 0 0) 0x7f2eb808b8c0 con 0x7f2ec015d7e0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.522+0000 7f2ebed9d640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 0x7f2ea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:33.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.523+0000 7f2ebed9d640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 0x7f2ea006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006fd0 tx=0x7f2eb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:33.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.527+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2eb8059bc0 con 0x7f2ec015d7e0 2024-04-13T03:17:33.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:33.739+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2ec0161040 con 0x7f2ec015d7e0 2024-04-13T03:17:34.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.382+0000 7f2eb77fe640 1 -- 172.21.15.106:0/3159664426 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v68) ==== 174+0+0 (secure 0 0 0) 0x7f2eb80594c0 con 0x7f2ec015d7e0 2024-04-13T03:17:34.384 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 15 and data pool 16 2024-04-13T03:17:34.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.385+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 msgr2=0x7f2ea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.385+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 0x7f2ea006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006fd0 tx=0x7f2eb0008040 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.385+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 msgr2=0x7f2ec01f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.385+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800b970 tx=0x7f2eb800be30 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 shutdown_connections 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ea006c8f0 0x7f2ea006edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ec015d7e0 0x7f2ec01f4520 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec015d400 0x7f2ec00b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 --2- 172.21.15.106:0/3159664426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ec00cf720 0x7f2ec01f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.386+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 >> 172.21.15.106:0/3159664426 conn(0x7f2ec00b2600 msgr2=0x7f2ec01603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.390+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 shutdown_connections 2024-04-13T03:17:34.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.392+0000 7f2ec515e640 1 -- 172.21.15.106:0/3159664426 wait complete. 2024-04-13T03:17:34.420 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:34.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.523+0000 7f07b00b7640 1 Processor -- start 2024-04-13T03:17:34.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.523+0000 7f07b00b7640 1 -- start start 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07a80cd770 0x7f07a80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f07a80cf5c0 0x7f07a8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f07a806ecc0 con 0x7f07a80cd770 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f07a80707f0 con 0x7f07a80cf5c0 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07b00b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f07a815f420 con 0x7f07a806e8e0 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07a80cd770 0x7f07a80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07a80cd770 0x7f07a80cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55004/0 (socket says 172.21.15.106:55004) 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 -- 172.21.15.106:0/1567227468 learned_addr learned my addr 172.21.15.106:0/1567227468 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:34.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 -- 172.21.15.106:0/1567227468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 msgr2=0x7f07a815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:34.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 --2- 172.21.15.106:0/1567227468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 -- 172.21.15.106:0/1567227468 >> v2:172.21.15.114:3300/0 conn(0x7f07a80cf5c0 msgr2=0x7f07a8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:34.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 --2- 172.21.15.106:0/1567227468 >> v2:172.21.15.114:3300/0 conn(0x7f07a80cf5c0 0x7f07a8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.524+0000 7f07ad652640 1 -- 172.21.15.106:0/1567227468 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a815f700 con 0x7f07a80cd770 2024-04-13T03:17:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07ad652640 1 --2- 172.21.15.106:0/1567227468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a80cf080 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f07a80cb720 tx=0x7f079802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbf99aa11071617 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07ace51640 1 -- 172.21.15.106:0/1567227468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798035020 con 0x7f07a80cd770 2024-04-13T03:17:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07ace51640 1 -- 172.21.15.106:0/1567227468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0798031890 con 0x7f07a80cd770 2024-04-13T03:17:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07ace51640 1 -- 172.21.15.106:0/1567227468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798031c60 con 0x7f07a80cd770 2024-04-13T03:17:34.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 -- 172.21.15.106:0/1567227468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 msgr2=0x7f07a80cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 --2- 172.21.15.106:0/1567227468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a80cf080 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f07a80cb720 tx=0x7f079802b930 comp rx=0 tx=0).stop 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 -- 172.21.15.106:0/1567227468 shutdown_connections 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 --2- 172.21.15.106:0/1567227468 >> v2:172.21.15.114:3300/0 conn(0x7f07a80cf5c0 0x7f07a8070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 --2- 172.21.15.106:0/1567227468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a80cf080 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 --2- 172.21.15.106:0/1567227468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.525+0000 7f07b00b7640 1 -- 172.21.15.106:0/1567227468 >> 172.21.15.106:0/1567227468 conn(0x7f07a806f180 msgr2=0x7f07a806f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.526+0000 7f07b00b7640 1 -- 172.21.15.106:0/1567227468 shutdown_connections 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.526+0000 7f07b00b7640 1 -- 172.21.15.106:0/1567227468 wait complete. 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 Processor -- start 2024-04-13T03:17:34.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 -- start start 2024-04-13T03:17:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a80b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a80cf5c0 0x7f07a80b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80cdbd0 con 0x7f07a80cd770 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07a815fcc0 con 0x7f07a80cf5c0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07b00b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80711c0 con 0x7f07a806e8e0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.527+0000 7f07ad652640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55016/0 (socket says 172.21.15.106:55016) 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 -- 172.21.15.106:0/3530957391 learned_addr learned my addr 172.21.15.106:0/3530957391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ade53640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a80b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ae654640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a80cf5c0 0x7f07a80b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 -- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 msgr2=0x7f07a80b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a80b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 -- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a80cf5c0 msgr2=0x7f07a80b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a80cf5c0 0x7f07a80b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 -- 172.21.15.106:0/3530957391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a8157950 con 0x7f07a80cd770 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ade53640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a80b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07ad652640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0798000c00 tx=0x7f07980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079803ad60 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07980443f0 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980447e0 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a8157c30 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.528+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a81581f0 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.529+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a80b7710 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.530+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f079804b060 con 0x7f07a80cd770 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.530+0000 7f079e7fc640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 0x7f078c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.530+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f07980adf90 con 0x7f07a80cd770 2024-04-13T03:17:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.530+0000 7f07ade53640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 0x7f078c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.530+0000 7f07ade53640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 0x7f078c06edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006fd0 tx=0x7f07a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.534+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07a80b7710 con 0x7f07a80cd770 2024-04-13T03:17:34.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.711+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07a8068a10 con 0x7f07a80cd770 2024-04-13T03:17:34.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.712+0000 7f079e7fc640 1 -- 172.21.15.106:0/3530957391 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16933 (secure 0 0 0) 0x7f079803f070 con 0x7f07a80cd770 2024-04-13T03:17:34.713 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:34.713 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":106,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:33.740553+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 msgr2=0x7f078c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 0x7f078c06edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006fd0 tx=0x7f07a4008040 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 msgr2=0x7f07a81563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0798000c00 tx=0x7f07980046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 shutdown_connections 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078c06c8f0 0x7f078c06edb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a80cf5c0 0x7f07a80b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cd770 0x7f07a81563d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 --2- 172.21.15.106:0/3530957391 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07a806e8e0 0x7f07a80b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 >> 172.21.15.106:0/3530957391 conn(0x7f07a806f180 msgr2=0x7f07a80b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:34.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 shutdown_connections 2024-04-13T03:17:34.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.714+0000 7f07b00b7640 1 -- 172.21.15.106:0/3530957391 wait complete. 2024-04-13T03:17:34.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 Processor -- start 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 -- start start 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f33c40b3d60 0x7f33c40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c40cfd20 con 0x7f33c415e9b0 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c4162af0 con 0x7f33c415ed90 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c9620640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f33c4162dd0 con 0x7f33c40b3d60 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.827+0000 7f33c2ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f33c415e9b0 0x7f33c40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59168/0 (socket says 172.21.15.106:59168) 2024-04-13T03:17:34.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c2ffd640 1 -- 172.21.15.106:0/520952960 learned_addr learned my addr 172.21.15.106:0/520952960 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c37fe640 1 --2- 172.21.15.106:0/520952960 >> v2:172.21.15.114:3300/0 conn(0x7f33c40b3d60 0x7f33c40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216123550 0 0) 0x7f33c40cfd20 con 0x7f33c415e9b0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ac003520 con 0x7f33c415e9b0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4062314289 0 0) 0x7f33ac003520 con 0x7f33c415e9b0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 >> v2:172.21.15.114:3300/0 conn(0x7f33c40b3d60 msgr2=0x7f33c40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 --2- 172.21.15.106:0/520952960 >> v2:172.21.15.114:3300/0 conn(0x7f33c40b3d60 0x7f33c40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 >> v1:172.21.15.114:6789/0 conn(0x7f33c415ed90 legacy=0x7f33c40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c41630b0 con 0x7f33c415e9b0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f33bc0019f0 con 0x7f33c415e9b0 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 >> v1:172.21.15.106:6789/0 conn(0x7f33c415e9b0 legacy=0x7f33c40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0039b0 0x7f33ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33ac004800 0x7f33ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 0x7f33ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4162dd0 con 0x7f33ac0051c0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac00c070 con 0x7f33ac0039b0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac00c270 con 0x7f33ac004800 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c37fe640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 0x7f33ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c2ffd640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0039b0 0x7f33ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.828+0000 7f33c27fc640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33ac004800 0x7f33ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 -- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33ac004800 msgr2=0x7f33ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33ac004800 0x7f33ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 -- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0039b0 msgr2=0x7f33ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0039b0 0x7f33ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 -- 172.21.15.106:0/520952960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac003520 con 0x7f33ac0051c0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 -- 172.21.15.106:0/520952960 reap_dead start 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c37fe640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 0x7f33ac00b9c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009ac0 tx=0x7f33ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8367712f6ad416ea server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009690 con 0x7f33ac0051c0 2024-04-13T03:17:34.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b4004020 con 0x7f33ac0051c0 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.829+0000 7f33c1ffb640 1 -- 172.21.15.106:0/520952960 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4004410 con 0x7f33ac0051c0 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 -- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 msgr2=0x7f33ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 0x7f33ac00b9c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009ac0 tx=0x7f33ac005830 comp rx=0 tx=0).stop 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 -- 172.21.15.106:0/520952960 shutdown_connections 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 --2- 172.21.15.106:0/520952960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33ac0051c0 0x7f33ac00b9c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 -- 172.21.15.106:0/520952960 >> 172.21.15.106:0/520952960 conn(0x7f33c408bf10 msgr2=0x7f33c40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 -- 172.21.15.106:0/520952960 shutdown_connections 2024-04-13T03:17:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.830+0000 7f33c9620640 1 -- 172.21.15.106:0/520952960 wait complete. 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.831+0000 7f33c9620640 1 Processor -- start 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.831+0000 7f33c9620640 1 -- start start 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0051c0 0x7f33c4152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33c406b8e0 0x7f33c406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4162800 con 0x7f33c406c2a0 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4162680 con 0x7f33ac0051c0 2024-04-13T03:17:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c9620640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f33c406eb20 con 0x7f33c406b8e0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55020/0 (socket says 172.21.15.106:55020) 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 -- 172.21.15.106:0/3603575527 learned_addr learned my addr 172.21.15.106:0/3603575527 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c27fc640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33c406b8e0 0x7f33c406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c2ffd640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0051c0 0x7f33c4152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 -- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33c406b8e0 msgr2=0x7f33c406bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33c406b8e0 0x7f33c406bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 -- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0051c0 msgr2=0x7f33c4152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0051c0 0x7f33c4152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.832+0000 7f33c37fe640 1 -- 172.21.15.106:0/3603575527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c4160260 con 0x7f33c406c2a0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33c37fe640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f33b402c380 tx=0x7f33b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009690 con 0x7f33c406c2a0 2024-04-13T03:17:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b402cbd0 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4003610 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c406c720 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.833+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41604d0 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.834+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c4068a10 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.834+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33b40037b0 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.834+0000 7f33ab7fe640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 0x7f33b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.835+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f33b40ac940 con 0x7f33c406c2a0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.835+0000 7f33c2ffd640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 0x7f33b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.835+0000 7f33c2ffd640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 0x7f33b006edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004d00 tx=0x7f33bc006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:34.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:34.839+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33b407abe0 con 0x7f33c406c2a0 2024-04-13T03:17:35.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.014+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33c40d53a0 con 0x7f33c406c2a0 2024-04-13T03:17:35.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.015+0000 7f33ab7fe640 1 -- 172.21.15.106:0/3603575527 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16933 (secure 0 0 0) 0x7f33b407a4e0 con 0x7f33c406c2a0 2024-04-13T03:17:35.016 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:35.016 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":106,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:33.740553+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 msgr2=0x7f33b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 0x7f33b006edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004d00 tx=0x7f33bc006040 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 msgr2=0x7f33c4160cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f33b402c380 tx=0x7f33b4004730 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 shutdown_connections 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f33b006c8f0 0x7f33b006edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f33c406b8e0 0x7f33c406bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c406c2a0 0x7f33c4160cd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 --2- 172.21.15.106:0/3603575527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33ac0051c0 0x7f33c4152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 >> 172.21.15.106:0/3603575527 conn(0x7f33c408bf10 msgr2=0x7f33c40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:35.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 shutdown_connections 2024-04-13T03:17:35.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.017+0000 7f33c9620640 1 -- 172.21.15.106:0/3603575527 wait complete. 2024-04-13T03:17:35.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:17:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.132+0000 7f0cdb436640 1 Processor -- start 2024-04-13T03:17:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.132+0000 7f0cdb436640 1 -- start start 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cdb436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cd40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cdb436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd40d1ec0 con 0x7f0cd40704a0 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cdb436640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd40709a0 con 0x7f0cd415c970 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cdb436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd415cd50 con 0x7f0cd40cd4f0 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cd89d1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0cd40704a0 0x7f0cd40ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59182/0 (socket says 172.21.15.106:59182) 2024-04-13T03:17:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.133+0000 7f0cd89d1640 1 -- 172.21.15.106:0/1837509758 learned_addr learned my addr 172.21.15.106:0/1837509758 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93488998 0 0) 0x7f0cd40d1ec0 con 0x7f0cd40704a0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cc0003520 con 0x7f0cd40704a0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cd40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2361810466 0 0) 0x7f0cc0003520 con 0x7f0cd40704a0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 msgr2=0x7f0cd40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cd40b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 >> v1:172.21.15.114:6789/0 conn(0x7f0cd415c970 legacy=0x7f0cd40d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd4070c80 con 0x7f0cd40704a0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f0cc4001670 con 0x7f0cd40704a0 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 >> v1:172.21.15.106:6789/0 conn(0x7f0cd40704a0 legacy=0x7f0cd40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cc00039e0 0x7f0cc0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc0004d40 0x7f0cc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd415cd50 con 0x7f0cc0004d40 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc000baa0 con 0x7f0cc00039e0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc000bca0 con 0x7f0cd40cd4f0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc0004d40 0x7f0cc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd89d1640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd91d2640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cc00039e0 0x7f0cc0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 -- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 msgr2=0x7f0cc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cd40cd4f0 0x7f0cc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 -- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cc00039e0 msgr2=0x7f0cc0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cc00039e0 0x7f0cc0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.134+0000 7f0cd99d3640 1 -- 172.21.15.106:0/1837509758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc0003520 con 0x7f0cc0004d40 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.135+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc0004d40 0x7f0cc00051c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40738e0 tx=0x7f0cc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e4464e81f6d960 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.135+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc03d070 con 0x7f0cc0004d40 2024-04-13T03:17:35.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.135+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ccc030a10 con 0x7f0cc0004d40 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.135+0000 7f0ccbfff640 1 -- 172.21.15.106:0/1837509758 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc009690 con 0x7f0cc0004d40 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 -- 172.21.15.106:0/1837509758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc0004d40 msgr2=0x7f0cc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 --2- 172.21.15.106:0/1837509758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cc0004d40 0x7f0cc00051c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40738e0 tx=0x7f0cc0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cd99d3640 1 -- 172.21.15.106:0/1837509758 reap_dead start 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 -- 172.21.15.106:0/1837509758 shutdown_connections 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 -- 172.21.15.106:0/1837509758 >> 172.21.15.106:0/1837509758 conn(0x7f0cd40cf000 msgr2=0x7f0cd40cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 -- 172.21.15.106:0/1837509758 shutdown_connections 2024-04-13T03:17:35.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.136+0000 7f0cdb436640 1 -- 172.21.15.106:0/1837509758 wait complete. 2024-04-13T03:17:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.137+0000 7f0cdb436640 1 Processor -- start 2024-04-13T03:17:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.137+0000 7f0cdb436640 1 -- start start 2024-04-13T03:17:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cc0004d40 0x7f0cd40c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 0x7f0cd4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40d1290 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4071240 con 0x7f0cd40c6030 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cdb436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd40c5d30 con 0x7f0cc0004d40 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55030/0 (socket says 172.21.15.106:55030) 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 -- 172.21.15.106:0/2521262438 learned_addr learned my addr 172.21.15.106:0/2521262438 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd91d2640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cc0004d40 0x7f0cd40c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd89d1640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 0x7f0cd4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 -- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cc0004d40 msgr2=0x7f0cd40c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cc0004d40 0x7f0cd40c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 -- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 msgr2=0x7f0cd4151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 0x7f0cd4151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:35.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 -- 172.21.15.106:0/2521262438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd4152410 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd89d1640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 0x7f0cd4151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.138+0000 7f0cd99d3640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009620 tx=0x7f0ccc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.139+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc03d070 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.139+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ccc047070 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.139+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd40c67d0 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.139+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc038d20 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.139+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd40c6d90 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.140+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd4069120 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.140+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ccc042440 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.141+0000 7f0cc9ffb640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 0x7f0cb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.141+0000 7f0cd91d2640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 0x7f0cb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.141+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f0ccc0b12d0 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.141+0000 7f0cd91d2640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 0x7f0cb806edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001ee0 tx=0x7f0cd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:35.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.145+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ccc050020 con 0x7f0cd40cbdd0 2024-04-13T03:17:35.341 INFO:tasks.ceph.mds.b.smithi114.stderr:2024-04-13T03:17:35.340+0000 7f4e7dff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:17:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:35.359+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0cd41525f0 con 0x7f0cd40cbdd0 2024-04-13T03:17:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.023+0000 7f0cc9ffb640 1 -- 172.21.15.106:0/2521262438 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7f0cd4152410 con 0x7f0cd40cbdd0 2024-04-13T03:17:36.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 msgr2=0x7f0cb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 0x7f0cb806edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001ee0 tx=0x7f0cd0009040 comp rx=0 tx=0).stop 2024-04-13T03:17:36.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 msgr2=0x7f0cd40cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009620 tx=0x7f0ccc004730 comp rx=0 tx=0).stop 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 shutdown_connections 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0cb806c8f0 0x7f0cb806edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cd40cbdd0 0x7f0cd40cc230 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd40c6030 0x7f0cd4151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 --2- 172.21.15.106:0/2521262438 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0cc0004d40 0x7f0cd40c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 >> 172.21.15.106:0/2521262438 conn(0x7f0cd40cf000 msgr2=0x7f0cd40b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 shutdown_connections 2024-04-13T03:17:36.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.025+0000 7f0cdb436640 1 -- 172.21.15.106:0/2521262438 wait complete. 2024-04-13T03:17:36.038 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:17:36.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:17:36.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.141+0000 7fbaf87f7640 1 Processor -- start 2024-04-13T03:17:36.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.141+0000 7fbaf87f7640 1 -- start start 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf87f7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbaf00cd4c0 0x7fbaf0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf87f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbaf00d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf87f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf00b54c0 con 0x7fbaf015cb00 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf87f7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf0071320 con 0x7fbaf00cd4c0 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf87f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf0070d60 con 0x7fbaf00b81f0 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf6593640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbaf015cb00 0x7fbaf00ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59192/0 (socket says 172.21.15.106:59192) 2024-04-13T03:17:36.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.142+0000 7fbaf6593640 1 -- 172.21.15.106:0/364519083 learned_addr learned my addr 172.21.15.106:0/364519083 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458933427 0 0) 0x7fbaec001490 con 0x7fbaf015cb00 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbadc003520 con 0x7fbaf015cb00 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbaf00d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5d92640 1 --2- 172.21.15.106:0/364519083 >> v2:172.21.15.114:3300/0 conn(0x7fbaf00cd4c0 0x7fbaf0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617539810 0 0) 0x7fbadc003520 con 0x7fbaf015cb00 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 msgr2=0x7fbaf00d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbaf00d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 >> v2:172.21.15.114:3300/0 conn(0x7fbaf00cd4c0 msgr2=0x7fbaf0160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 --2- 172.21.15.106:0/364519083 >> v2:172.21.15.114:3300/0 conn(0x7fbaf00cd4c0 0x7fbaf0160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0071040 con 0x7fbaf015cb00 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fbaec001bc0 con 0x7fbaf015cb00 2024-04-13T03:17:36.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 >> v1:172.21.15.106:6789/0 conn(0x7fbaf015cb00 legacy=0x7fbaf00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbadc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc003f30 0x7fbadc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbadc004d40 0x7fbadc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0071320 con 0x7fbadc004d40 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0070d60 con 0x7fbadc003f30 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbadc00ba10 con 0x7fbaf00b81f0 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbadc004d40 0x7fbadc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6593640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbadc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf5d92640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc003f30 0x7fbadc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 -- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 msgr2=0x7fbadc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbadc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 -- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc003f30 msgr2=0x7fbadc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc003f30 0x7fbadc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6d94640 1 -- 172.21.15.106:0/364519083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc003520 con 0x7fbadc004d40 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.143+0000 7fbaf6593640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf00b81f0 0x7fbadc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.144+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbadc004d40 0x7fbadc0051c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009af0 tx=0x7fbadc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a0888c7c4ab84b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.144+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8039070 con 0x7fbadc004d40 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.144+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae802c6b0 con 0x7fbadc004d40 2024-04-13T03:17:36.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.144+0000 7fbaf5591640 1 -- 172.21.15.106:0/364519083 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8009690 con 0x7fbadc004d40 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 -- 172.21.15.106:0/364519083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbadc004d40 msgr2=0x7fbadc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/364519083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbadc004d40 0x7fbadc0051c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009af0 tx=0x7fbadc005830 comp rx=0 tx=0).stop 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf6d94640 1 -- 172.21.15.106:0/364519083 reap_dead start 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 -- 172.21.15.106:0/364519083 shutdown_connections 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 -- 172.21.15.106:0/364519083 >> 172.21.15.106:0/364519083 conn(0x7fbaf00d2530 msgr2=0x7fbaf00d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 -- 172.21.15.106:0/364519083 shutdown_connections 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.145+0000 7fbaf87f7640 1 -- 172.21.15.106:0/364519083 wait complete. 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.146+0000 7fbaf87f7640 1 Processor -- start 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.146+0000 7fbaf87f7640 1 -- start start 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc004d40 0x7fbaf01f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf01585e0 0x7fbaf015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf00d0cd0 con 0x7fbaf00abdc0 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf00d07b0 con 0x7fbadc004d40 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf00d0e50 con 0x7fbaf01585e0 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55050/0 (socket says 172.21.15.106:55050) 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 -- 172.21.15.106:0/2863849569 learned_addr learned my addr 172.21.15.106:0/2863849569 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6593640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc004d40 0x7fbaf01f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf5d92640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf01585e0 0x7fbaf015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 -- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf01585e0 msgr2=0x7fbaf015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf01585e0 0x7fbaf015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 -- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc004d40 msgr2=0x7fbaf01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:36.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc004d40 0x7fbaf01f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 -- 172.21.15.106:0/2863849569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0158a60 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf6d94640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009a90 tx=0x7fbae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8039070 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae8043070 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf015e6d0 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.147+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8009690 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.148+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf015ec90 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.148+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf0068a10 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.149+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbae8003850 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.149+0000 7fbae6ffd640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 0x7fbad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.149+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fbae80acd60 con 0x7fbaf00abdc0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.149+0000 7fbaf6593640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 0x7fbad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:36.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.150+0000 7fbaf6593640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 0x7fbad406edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbaec002230 tx=0x7fbaec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:36.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.154+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbae804c020 con 0x7fbaf00abdc0 2024-04-13T03:17:36.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:36.365+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbaf00d3120 con 0x7fbaf00abdc0 2024-04-13T03:17:37.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.024+0000 7fbae6ffd640 1 -- 172.21.15.106:0/2863849569 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7fbae807b000 con 0x7fbaf00abdc0 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 msgr2=0x7fbad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 0x7fbad406edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbaec002230 tx=0x7fbaec009040 comp rx=0 tx=0).stop 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 msgr2=0x7fbaf00ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009a90 tx=0x7fbae8004730 comp rx=0 tx=0).stop 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 shutdown_connections 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbad406c8f0 0x7fbad406edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbaf01585e0 0x7fbaf015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf00abdc0 0x7fbaf00ac240 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 --2- 172.21.15.106:0/2863849569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbadc004d40 0x7fbaf01f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 >> 172.21.15.106:0/2863849569 conn(0x7fbaf00d2530 msgr2=0x7fbaf0160620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 shutdown_connections 2024-04-13T03:17:37.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.028+0000 7fbaf87f7640 1 -- 172.21.15.106:0/2863849569 wait complete. 2024-04-13T03:17:37.041 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:17:37.041 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:37.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.144+0000 7f16f8888640 1 Processor -- start 2024-04-13T03:17:37.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.144+0000 7f16f8888640 1 -- start start 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f8888640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f16f40b3d60 0x7f16f40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f8888640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f40cfd20 con 0x7f16f415e9b0 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f8888640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f4162af0 con 0x7f16f415ed90 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f8888640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f16f4162dd0 con 0x7f16f40b3d60 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f16f415e9b0 0x7f16f40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59204/0 (socket says 172.21.15.106:59204) 2024-04-13T03:17:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f259c640 1 -- 172.21.15.106:0/835492461 learned_addr learned my addr 172.21.15.106:0/835492461 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.145+0000 7f16f2d9d640 1 --2- 172.21.15.106:0/835492461 >> v2:172.21.15.114:3300/0 conn(0x7f16f40b3d60 0x7f16f40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839112900 0 0) 0x7f16f40cfd20 con 0x7f16f415e9b0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16dc003520 con 0x7f16f415e9b0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2690841494 0 0) 0x7f16dc003520 con 0x7f16f415e9b0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 >> v2:172.21.15.114:3300/0 conn(0x7f16f40b3d60 msgr2=0x7f16f40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 --2- 172.21.15.106:0/835492461 >> v2:172.21.15.114:3300/0 conn(0x7f16f40b3d60 0x7f16f40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 >> v1:172.21.15.114:6789/0 conn(0x7f16f415ed90 legacy=0x7f16f40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f41630b0 con 0x7f16f415e9b0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f16e40019f0 con 0x7f16f415e9b0 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 >> v1:172.21.15.106:6789/0 conn(0x7f16f415e9b0 legacy=0x7f16f40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4162dd0 con 0x7f16dc004800 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc00c070 con 0x7f16dc0039b0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f16dc00c270 con 0x7f16dc0051c0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f259c640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f2d9d640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 -- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16dc0051c0 msgr2=0x7f16dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 -- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc0039b0 msgr2=0x7f16dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f1d9b640 1 -- 172.21.15.106:0/835492461 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc003520 con 0x7f16dc004800 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.146+0000 7f16f2d9d640 1 -- 172.21.15.106:0/835492461 reap_dead start 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.147+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001740 tx=0x7f16dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c387f44e1753e19a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.147+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8001360 con 0x7f16dc004800 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.147+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e80052d0 con 0x7f16dc004800 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.147+0000 7f16f159a640 1 -- 172.21.15.106:0/835492461 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8003830 con 0x7f16dc004800 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 -- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 msgr2=0x7f16dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001740 tx=0x7f16dc005830 comp rx=0 tx=0).stop 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 -- 172.21.15.106:0/835492461 shutdown_connections 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 --2- 172.21.15.106:0/835492461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 -- 172.21.15.106:0/835492461 >> 172.21.15.106:0/835492461 conn(0x7f16f408bf10 msgr2=0x7f16f40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 -- 172.21.15.106:0/835492461 shutdown_connections 2024-04-13T03:17:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.148+0000 7f16f8888640 1 -- 172.21.15.106:0/835492461 wait complete. 2024-04-13T03:17:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.149+0000 7f16f8888640 1 Processor -- start 2024-04-13T03:17:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.149+0000 7f16f8888640 1 -- start start 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 0x7f16f40cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16f40cd920 0x7f16f40cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4162800 con 0x7f16f4161230 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4162680 con 0x7f16dc004800 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f16f406eb20 con 0x7f16f40cd920 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55072/0 (socket says 172.21.15.106:55072) 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 -- 172.21.15.106:0/3547201261 learned_addr learned my addr 172.21.15.106:0/3547201261 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f259c640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 0x7f16f40cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f2d9d640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16f40cd920 0x7f16f40cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 -- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16f40cd920 msgr2=0x7f16f40cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16f40cd920 0x7f16f40cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 -- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 msgr2=0x7f16f40cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 0x7f16f40cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 -- 172.21.15.106:0/3547201261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f40b41e0 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f259c640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 0x7f16f40cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f1d9b640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f16e802bcd0 tx=0x7f16e8001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8041020 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e8005480 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f40b44c0 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.150+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8001360 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.151+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f40cc1e0 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.151+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f4068a10 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.152+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16e8044020 con 0x7f16f4161230 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.152+0000 7f16e2ffd640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.152+0000 7f16f259c640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.153+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f16e80ad5d0 con 0x7f16f4161230 2024-04-13T03:17:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.153+0000 7f16f259c640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f16e4004c20 tx=0x7f16e4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.157+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16e804b030 con 0x7f16f4161230 2024-04-13T03:17:37.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.370+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16f40d53a0 con 0x7f16f4161230 2024-04-13T03:17:37.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.370+0000 7f16e2ffd640 1 -- 172.21.15.106:0/3547201261 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5195 (secure 0 0 0) 0x7f16e807b870 con 0x7f16f4161230 2024-04-13T03:17:37.371 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:37.371 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:17:36.367064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8430":{"gid":8430,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/2645971931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2645971931},{"type":"v1","addr":"172.21.15.114:6835","nonce":2645971931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8430,"qdb_cluster":[8430]},"id":7}]} 2024-04-13T03:17:37.371 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 71 2024-04-13T03:17:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 msgr2=0x7f16d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f16e4004c20 tx=0x7f16e4006040 comp rx=0 tx=0).stop 2024-04-13T03:17:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 msgr2=0x7f16f41616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f16e802bcd0 tx=0x7f16e8001170 comp rx=0 tx=0).stop 2024-04-13T03:17:37.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 shutdown_connections 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f16f40cd920 0x7f16f40cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16f4161230 0x7f16f41616b0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 --2- 172.21.15.106:0/3547201261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16dc004800 0x7f16f40cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 >> 172.21.15.106:0/3547201261 conn(0x7f16f408bf10 msgr2=0x7f16f40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 shutdown_connections 2024-04-13T03:17:37.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.372+0000 7f16f8888640 1 -- 172.21.15.106:0/3547201261 wait complete. 2024-04-13T03:17:37.384 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 Processor -- start 2024-04-13T03:17:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 -- start start 2024-04-13T03:17:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b5815ed90 0x7f1b580d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b580b3d60 0x7f1b580d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b580cfd20 con 0x7f1b5815e9b0 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.484+0000 7f1b5d355640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1b58162af0 con 0x7f1b5815ed90 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b5d355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b58162dd0 con 0x7f1b580b3d60 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b5815ed90 0x7f1b580d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b5815ed90 0x7f1b580d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55078/0 (socket says 172.21.15.106:55078) 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 -- 172.21.15.106:0/295084238 learned_addr learned my addr 172.21.15.106:0/295084238 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b577fe640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b580b3d60 0x7f1b580d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 -- 172.21.15.106:0/295084238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b580b3d60 msgr2=0x7f1b580d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b580b3d60 0x7f1b580d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 -- 172.21.15.106:0/295084238 >> v1:172.21.15.114:6789/0 conn(0x7f1b5815e9b0 legacy=0x7f1b580d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.485+0000 7f1b567fc640 1 -- 172.21.15.106:0/295084238 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b581630b0 con 0x7f1b5815ed90 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b567fc640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815ed90 0x7f1b580d2300 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009850 tx=0x7f1b4c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ebb4e627b5af53 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b55ffb640 1 -- 172.21.15.106:0/295084238 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c004230 con 0x7f1b5815ed90 2024-04-13T03:17:37.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b55ffb640 1 -- 172.21.15.106:0/295084238 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c0043d0 con 0x7f1b5815ed90 2024-04-13T03:17:37.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b55ffb640 1 -- 172.21.15.106:0/295084238 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c003810 con 0x7f1b5815ed90 2024-04-13T03:17:37.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 -- 172.21.15.106:0/295084238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815ed90 msgr2=0x7f1b580d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815ed90 0x7f1b580d2300 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009850 tx=0x7f1b4c02b960 comp rx=0 tx=0).stop 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 -- 172.21.15.106:0/295084238 shutdown_connections 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b580b3d60 0x7f1b580d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 --2- 172.21.15.106:0/295084238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815ed90 0x7f1b580d2300 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.486+0000 7f1b5d355640 1 -- 172.21.15.106:0/295084238 >> 172.21.15.106:0/295084238 conn(0x7f1b5808bf10 msgr2=0x7f1b580b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.487+0000 7f1b5d355640 1 -- 172.21.15.106:0/295084238 shutdown_connections 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.487+0000 7f1b5d355640 1 -- 172.21.15.106:0/295084238 wait complete. 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 Processor -- start 2024-04-13T03:17:37.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 -- start start 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b580b3d60 0x7f1b58150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b58160ba0 0x7f1b580cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58162800 con 0x7f1b5815e9b0 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58162680 con 0x7f1b580b3d60 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.488+0000 7f1b5d355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1b580d4d00 con 0x7f1b58160ba0 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55082/0 (socket says 172.21.15.106:55082) 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 -- 172.21.15.106:0/1993332909 learned_addr learned my addr 172.21.15.106:0/1993332909 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 -- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b58160ba0 msgr2=0x7f1b580cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b58160ba0 0x7f1b580cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 -- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b580b3d60 msgr2=0x7f1b58150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b56ffd640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b580b3d60 0x7f1b58150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b580b3d60 0x7f1b58150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 -- 172.21.15.106:0/1993332909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b580ce6c0 con 0x7f1b5815e9b0 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b567fc640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009970 tx=0x7f1b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c047070 con 0x7f1b5815e9b0 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c003bd0 con 0x7f1b5815e9b0 2024-04-13T03:17:37.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b580cb8a0 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c006040 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.489+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b580cbe60 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.490+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58068a10 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.491+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b4c04a070 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.491+0000 7f1b3f7fe640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 0x7f1b4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.491+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f1b4c0ad850 con 0x7f1b5815e9b0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.491+0000 7f1b56ffd640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 0x7f1b4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.491+0000 7f1b56ffd640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 0x7f1b4406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1b480015d0 tx=0x7f1b48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.495+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b4c034020 con 0x7f1b5815e9b0 2024-04-13T03:17:37.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.672+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b580d53a0 con 0x7f1b5815e9b0 2024-04-13T03:17:37.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.673+0000 7f1b3f7fe640 1 -- 172.21.15.106:0/1993332909 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16920 (secure 0 0 0) 0x7f1b4c07baf0 con 0x7f1b5815e9b0 2024-04-13T03:17:37.674 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:37.674 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":107,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:35.383741+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:37.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 msgr2=0x7f1b4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 0x7f1b4406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1b480015d0 tx=0x7f1b48009040 comp rx=0 tx=0).stop 2024-04-13T03:17:37.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 msgr2=0x7f1b580ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009970 tx=0x7f1b4c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 shutdown_connections 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1b4406c920 0x7f1b4406ede0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1b58160ba0 0x7f1b580cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b5815e9b0 0x7f1b580ce180 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 --2- 172.21.15.106:0/1993332909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b580b3d60 0x7f1b58150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 >> 172.21.15.106:0/1993332909 conn(0x7f1b5808bf10 msgr2=0x7f1b580d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 shutdown_connections 2024-04-13T03:17:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.675+0000 7f1b5d355640 1 -- 172.21.15.106:0/1993332909 wait complete. 2024-04-13T03:17:37.687 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:17:37.687 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:17:37.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.786+0000 7f29f3788640 1 Processor -- start 2024-04-13T03:17:37.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 -- start start 2024-04-13T03:17:37.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 0x7f29ec072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29ec06e150 0x7f29ec06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ec0998d0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f29ec073020 con 0x7f29ec06e150 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.787+0000 7f29f3788640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f29ec073300 con 0x7f29ec06cb10 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1d25640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29ec06e150 0x7f29ec06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1524640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f29ec0cdf50 0x7f29ec0ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59214/0 (socket says 172.21.15.106:59214) 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1524640 1 -- 172.21.15.106:0/3319393593 learned_addr learned my addr 172.21.15.106:0/3319393593 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f0d23640 1 --2- 172.21.15.106:0/3319393593 >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 0x7f29ec072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29e3fff640 1 -- 172.21.15.106:0/3319393593 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435317301 0 0) 0x7f29ec0998d0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29e3fff640 1 -- 172.21.15.106:0/3319393593 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d8003520 con 0x7f29ec0cdf50 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1d25640 1 -- 172.21.15.106:0/3319393593 >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 msgr2=0x7f29ec072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1d25640 1 --2- 172.21.15.106:0/3319393593 >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 0x7f29ec072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1d25640 1 -- 172.21.15.106:0/3319393593 >> v1:172.21.15.106:6789/0 conn(0x7f29ec0cdf50 legacy=0x7f29ec0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f1d25640 1 -- 172.21.15.106:0/3319393593 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec0735e0 con 0x7f29ec06e150 2024-04-13T03:17:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.788+0000 7f29f0d23640 1 --2- 172.21.15.106:0/3319393593 >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 0x7f29ec072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f1d25640 1 --2- 172.21.15.106:0/3319393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec06e150 0x7f29ec06e600 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0b64f0 tx=0x7f29e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d0a3951c085760 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29e3fff640 1 -- 172.21.15.106:0/3319393593 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8035020 con 0x7f29ec06e150 2024-04-13T03:17:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29e3fff640 1 -- 172.21.15.106:0/3319393593 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8031830 con 0x7f29ec06e150 2024-04-13T03:17:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29e3fff640 1 -- 172.21.15.106:0/3319393593 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8031c00 con 0x7f29ec06e150 2024-04-13T03:17:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 -- 172.21.15.106:0/3319393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec06e150 msgr2=0x7f29ec06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 --2- 172.21.15.106:0/3319393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec06e150 0x7f29ec06e600 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0b64f0 tx=0x7f29e802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 -- 172.21.15.106:0/3319393593 shutdown_connections 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 --2- 172.21.15.106:0/3319393593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec06e150 0x7f29ec06e600 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 --2- 172.21.15.106:0/3319393593 >> v2:172.21.15.114:3300/0 conn(0x7f29ec06cb10 0x7f29ec072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 -- 172.21.15.106:0/3319393593 >> 172.21.15.106:0/3319393593 conn(0x7f29ec0b1f10 msgr2=0x7f29ec0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 -- 172.21.15.106:0/3319393593 shutdown_connections 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.789+0000 7f29f3788640 1 -- 172.21.15.106:0/3319393593 wait complete. 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.790+0000 7f29f3788640 1 Processor -- start 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.790+0000 7f29f3788640 1 -- start start 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 0x7f29ec09cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 0x7f29ec157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec072de0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec072c60 con 0x7f29ec06e150 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f3788640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f29ec06eb40 con 0x7f29ec06cb10 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f1d25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f1d25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55090/0 (socket says 172.21.15.106:55090) 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.791+0000 7f29f1d25640 1 -- 172.21.15.106:0/89457163 learned_addr learned my addr 172.21.15.106:0/89457163 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f0d23640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 0x7f29ec157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1524640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 0x7f29ec09cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 -- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 msgr2=0x7f29ec09cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 0x7f29ec09cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 -- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 msgr2=0x7f29ec157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 0x7f29ec157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 -- 172.21.15.106:0/89457163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec09afe0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f0d23640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 0x7f29ec157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1524640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 0x7f29ec09cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f1d25640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f29e802bee0 tx=0x7f29e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8048070 con 0x7f29ec0cdf50 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e803a630 con 0x7f29ec0cdf50 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ec09b2c0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e803aa00 con 0x7f29ec0cdf50 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.792+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ec09b880 con 0x7f29ec0cdf50 2024-04-13T03:17:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.793+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ec0ae4c0 con 0x7f29ec0cdf50 2024-04-13T03:17:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.793+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29e804b070 con 0x7f29ec0cdf50 2024-04-13T03:17:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.794+0000 7f29e1ffb640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.794+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f29e80ad420 con 0x7f29ec0cdf50 2024-04-13T03:17:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.794+0000 7f29f1524640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.794+0000 7f29f1524640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 0x7f29d006edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005c00 tx=0x7f29e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:37.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.798+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29e8031800 con 0x7f29ec0cdf50 2024-04-13T03:17:37.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.972+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29ec09ba80 con 0x7f29d006c8f0 2024-04-13T03:17:37.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.978+0000 7f29e1ffb640 1 -- 172.21.15.106:0/89457163 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240297 (secure 0 0 0) 0x7f29e400dc30 con 0x7f29d006c8f0 2024-04-13T03:17:37.981 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:37.982 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 msgr2=0x7f29d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 0x7f29d006edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005c00 tx=0x7f29e4009040 comp rx=0 tx=0).stop 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 msgr2=0x7f29ec0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f29e802bee0 tx=0x7f29e80046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 shutdown_connections 2024-04-13T03:17:37.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29ec0cdf50 0x7f29ec0ad980 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29ec06e150 0x7f29ec157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 --2- 172.21.15.106:0/89457163 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f29ec06cb10 0x7f29ec09cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 >> 172.21.15.106:0/89457163 conn(0x7f29ec0b1f10 msgr2=0x7f29ec0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 shutdown_connections 2024-04-13T03:17:37.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:37.981+0000 7f29f3788640 1 -- 172.21.15.106:0/89457163 wait complete. 2024-04-13T03:17:37.992 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-04-13T03:17:36.471754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260616,"kb_used_data":11120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714104,"statfs":{"total":773094113280,"available":772827242496,"internally_reserved":0,"allocated":11386880,"data_stored":5901560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.094677"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384935+0000","last_change":"2024-04-13T03:17:32.384935+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384935+0000","last_undegraded":"2024-04-13T03:17:32.384935+0000","last_fullsized":"2024-04-13T03:17:32.384935+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381472+0000","last_change":"2024-04-13T03:17:32.381472+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381472+0000","last_undegraded":"2024-04-13T03:17:32.381472+0000","last_fullsized":"2024-04-13T03:17:32.381472+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383567+0000","last_change":"2024-04-13T03:17:32.383567+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383567+0000","last_undegraded":"2024-04-13T03:17:32.383567+0000","last_fullsized":"2024-04-13T03:17:32.383567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.380225+0000","last_change":"2024-04-13T03:17:32.380225+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.380225+0000","last_undegraded":"2024-04-13T03:17:32.380225+0000","last_fullsized":"2024-04-13T03:17:32.380225+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385837+0000","last_change":"2024-04-13T03:17:32.385837+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385837+0000","last_undegraded":"2024-04-13T03:17:32.385837+0000","last_fullsized":"2024-04-13T03:17:32.385837+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383816+0000","last_change":"2024-04-13T03:17:32.383816+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383816+0000","last_undegraded":"2024-04-13T03:17:32.383816+0000","last_fullsized":"2024-04-13T03:17:32.383816+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383280+0000","last_change":"2024-04-13T03:17:32.383280+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383280+0000","last_undegraded":"2024-04-13T03:17:32.383280+0000","last_fullsized":"2024-04-13T03:17:32.383280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384076+0000","last_change":"2024-04-13T03:17:32.384076+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384076+0000","last_undegraded":"2024-04-13T03:17:32.384076+0000","last_fullsized":"2024-04-13T03:17:32.384076+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383565+0000","last_change":"2024-04-13T03:17:32.383565+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383565+0000","last_undegraded":"2024-04-13T03:17:32.383565+0000","last_fullsized":"2024-04-13T03:17:32.383565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384582+0000","last_change":"2024-04-13T03:17:32.384582+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384582+0000","last_undegraded":"2024-04-13T03:17:32.384582+0000","last_fullsized":"2024-04-13T03:17:32.384582+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385594+0000","last_change":"2024-04-13T03:17:32.385594+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385594+0000","last_undegraded":"2024-04-13T03:17:32.385594+0000","last_fullsized":"2024-04-13T03:17:32.385594+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382459+0000","last_change":"2024-04-13T03:17:32.382459+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382459+0000","last_undegraded":"2024-04-13T03:17:32.382459+0000","last_fullsized":"2024-04-13T03:17:32.382459+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379279+0000","last_change":"2024-04-13T03:17:32.379279+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379279+0000","last_undegraded":"2024-04-13T03:17:32.379279+0000","last_fullsized":"2024-04-13T03:17:32.379279+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383988+0000","last_change":"2024-04-13T03:17:32.383988+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383988+0000","last_undegraded":"2024-04-13T03:17:32.383988+0000","last_fullsized":"2024-04-13T03:17:32.383988+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381081+0000","last_change":"2024-04-13T03:17:32.381081+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381081+0000","last_undegraded":"2024-04-13T03:17:32.381081+0000","last_fullsized":"2024-04-13T03:17:32.381081+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382660+0000","last_change":"2024-04-13T03:17:32.382660+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382660+0000","last_undegraded":"2024-04-13T03:17:32.382660+0000","last_fullsized":"2024-04-13T03:17:32.382660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383004+0000","last_change":"2024-04-13T03:17:32.383004+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383004+0000","last_undegraded":"2024-04-13T03:17:32.383004+0000","last_fullsized":"2024-04-13T03:17:32.383004+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385088+0000","last_change":"2024-04-13T03:17:32.385088+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385088+0000","last_undegraded":"2024-04-13T03:17:32.385088+0000","last_fullsized":"2024-04-13T03:17:32.385088+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383810+0000","last_change":"2024-04-13T03:17:32.383810+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383810+0000","last_undegraded":"2024-04-13T03:17:32.383810+0000","last_fullsized":"2024-04-13T03:17:32.383810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381058+0000","last_change":"2024-04-13T03:17:32.381058+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381058+0000","last_undegraded":"2024-04-13T03:17:32.381058+0000","last_fullsized":"2024-04-13T03:17:32.381058+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381695+0000","last_change":"2024-04-13T03:17:32.381695+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381695+0000","last_undegraded":"2024-04-13T03:17:32.381695+0000","last_fullsized":"2024-04-13T03:17:32.381695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383708+0000","last_change":"2024-04-13T03:17:32.383708+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383708+0000","last_undegraded":"2024-04-13T03:17:32.383708+0000","last_fullsized":"2024-04-13T03:17:32.383708+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381676+0000","last_change":"2024-04-13T03:17:32.381676+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381676+0000","last_undegraded":"2024-04-13T03:17:32.381676+0000","last_fullsized":"2024-04-13T03:17:32.381676+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379287+0000","last_change":"2024-04-13T03:17:32.379287+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379287+0000","last_undegraded":"2024-04-13T03:17:32.379287+0000","last_fullsized":"2024-04-13T03:17:32.379287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384103+0000","last_change":"2024-04-13T03:17:32.384103+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384103+0000","last_undegraded":"2024-04-13T03:17:32.384103+0000","last_fullsized":"2024-04-13T03:17:32.384103+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382396+0000","last_change":"2024-04-13T03:17:32.382396+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382396+0000","last_undegraded":"2024-04-13T03:17:32.382396+0000","last_fullsized":"2024-04-13T03:17:32.382396+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379299+0000","last_change":"2024-04-13T03:17:32.379299+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379299+0000","last_undegraded":"2024-04-13T03:17:32.379299+0000","last_fullsized":"2024-04-13T03:17:32.379299+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383508+0000","last_change":"2024-04-13T03:17:32.383508+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383508+0000","last_undegraded":"2024-04-13T03:17:32.383508+0000","last_fullsized":"2024-04-13T03:17:32.383508+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381279+0000","last_change":"2024-04-13T03:17:32.381279+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381279+0000","last_undegraded":"2024-04-13T03:17:32.381279+0000","last_fullsized":"2024-04-13T03:17:32.381279+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382390+0000","last_change":"2024-04-13T03:17:32.382390+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382390+0000","last_undegraded":"2024-04-13T03:17:32.382390+0000","last_fullsized":"2024-04-13T03:17:32.382390+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381020+0000","last_change":"2024-04-13T03:17:32.381020+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381020+0000","last_undegraded":"2024-04-13T03:17:32.381020+0000","last_fullsized":"2024-04-13T03:17:32.381020+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381091+0000","last_change":"2024-04-13T03:17:32.381091+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381091+0000","last_undegraded":"2024-04-13T03:17:32.381091+0000","last_fullsized":"2024-04-13T03:17:32.381091+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":325,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-13T03:17:32.385816+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:17:32.385816+0000","last_peered":"2024-04-13T03:17:32.385816+0000","last_clean":"2024-04-13T03:17:32.385816+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:17:32.385816+0000","last_undegraded":"2024-04-13T03:17:32.385816+0000","last_fullsized":"2024-04-13T03:17:32.385816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1884160,"data_stored":1188026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1871872,"data_stored":1181817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:17:37.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:17:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.132+0000 7fef77311640 1 Processor -- start 2024-04-13T03:17:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.132+0000 7fef77311640 1 -- start start 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fef700b53a0 0x7fef700b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef7015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef7006ef90 0x7fef7015e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fef700b5cc0 con 0x7fef7006ef90 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fef7006f540 con 0x7fef700b53a0 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef77311640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fef7006f820 con 0x7fef700cdf50 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef758ae640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef7006ef90 0x7fef7015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef758ae640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef7006ef90 0x7fef7015e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55102/0 (socket says 172.21.15.106:55102) 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef758ae640 1 -- 172.21.15.106:0/1775904656 learned_addr learned my addr 172.21.15.106:0/1775904656 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef758ae640 1 -- 172.21.15.106:0/1775904656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 msgr2=0x7fef7015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef758ae640 1 --2- 172.21.15.106:0/1775904656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef7015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.133+0000 7fef750ad640 1 --2- 172.21.15.106:0/1775904656 >> v2:172.21.15.114:3300/0 conn(0x7fef700b53a0 0x7fef700b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef758ae640 1 -- 172.21.15.106:0/1775904656 >> v2:172.21.15.114:3300/0 conn(0x7fef700b53a0 msgr2=0x7fef700b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef758ae640 1 --2- 172.21.15.106:0/1775904656 >> v2:172.21.15.114:3300/0 conn(0x7fef700b53a0 0x7fef700b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef758ae640 1 -- 172.21.15.106:0/1775904656 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7015b440 con 0x7fef7006ef90 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef758ae640 1 --2- 172.21.15.106:0/1775904656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7006ef90 0x7fef7015e920 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fef700d1480 tx=0x7fef6802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd75c77f8924b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef67fff640 1 -- 172.21.15.106:0/1775904656 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6803b070 con 0x7fef7006ef90 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef67fff640 1 -- 172.21.15.106:0/1775904656 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68036440 con 0x7fef7006ef90 2024-04-13T03:17:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.134+0000 7fef67fff640 1 -- 172.21.15.106:0/1775904656 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68036810 con 0x7fef7006ef90 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 -- 172.21.15.106:0/1775904656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7006ef90 msgr2=0x7fef7015e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 --2- 172.21.15.106:0/1775904656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7006ef90 0x7fef7015e920 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fef700d1480 tx=0x7fef6802e0e0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 -- 172.21.15.106:0/1775904656 shutdown_connections 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 --2- 172.21.15.106:0/1775904656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7006ef90 0x7fef7015e920 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 --2- 172.21.15.106:0/1775904656 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef7015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 --2- 172.21.15.106:0/1775904656 >> v2:172.21.15.114:3300/0 conn(0x7fef700b53a0 0x7fef700b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 -- 172.21.15.106:0/1775904656 >> 172.21.15.106:0/1775904656 conn(0x7fef700b1f10 msgr2=0x7fef700b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 -- 172.21.15.106:0/1775904656 shutdown_connections 2024-04-13T03:17:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.135+0000 7fef77311640 1 -- 172.21.15.106:0/1775904656 wait complete. 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.136+0000 7fef77311640 1 Processor -- start 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.136+0000 7fef77311640 1 -- start start 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef7006ef90 0x7fef701f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef700cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef7015c180 con 0x7fef700b53a0 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef7006fb00 con 0x7fef7006ef90 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef77311640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fef70158bb0 con 0x7fef700cdf50 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55110/0 (socket says 172.21.15.106:55110) 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 -- 172.21.15.106:0/2734453221 learned_addr learned my addr 172.21.15.106:0/2734453221 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef750ad640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef7006ef90 0x7fef701f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef758ae640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef700cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 -- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 msgr2=0x7fef700cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef700cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 -- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef7006ef90 msgr2=0x7fef701f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef7006ef90 0x7fef701f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 -- 172.21.15.106:0/2734453221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef700b08e0 con 0x7fef700b53a0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.137+0000 7fef748ac640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fef6000b410 tx=0x7fef6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.138+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60015020 con 0x7fef700b53a0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.138+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60011850 con 0x7fef700b53a0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.138+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef700adcc0 con 0x7fef700b53a0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.138+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60011c40 con 0x7fef700b53a0 2024-04-13T03:17:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.138+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef700c66e0 con 0x7fef700b53a0 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.139+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7015e670 con 0x7fef700b53a0 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.139+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef60018020 con 0x7fef700b53a0 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.139+0000 7fef65ffb640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 0x7fef5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.140+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fef6008bc50 con 0x7fef700b53a0 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.140+0000 7fef750ad640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 0x7fef5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.140+0000 7fef750ad640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 0x7fef5406ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef6c008ef0 tx=0x7fef6c008440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.144+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef7015e670 con 0x7fef700b53a0 2024-04-13T03:17:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.314+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef701f4cd0 con 0x7fef5406c920 2024-04-13T03:17:38.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.319+0000 7fef65ffb640 1 -- 172.21.15.106:0/2734453221 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240297 (secure 0 0 0) 0x7fef701f4cd0 con 0x7fef5406c920 2024-04-13T03:17:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:38.324 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:17:38.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 msgr2=0x7fef5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 0x7fef5406ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef6c008ef0 tx=0x7fef6c008440 comp rx=0 tx=0).stop 2024-04-13T03:17:38.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 msgr2=0x7fef70157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fef6000b410 tx=0x7fef6000b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 shutdown_connections 2024-04-13T03:17:38.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fef5406c920 0x7fef5406ede0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fef700cdf50 0x7fef700cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef700b53a0 0x7fef70157640 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 --2- 172.21.15.106:0/2734453221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef7006ef90 0x7fef701f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 >> 172.21.15.106:0/2734453221 conn(0x7fef700b1f10 msgr2=0x7fef700b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 shutdown_connections 2024-04-13T03:17:38.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.322+0000 7fef77311640 1 -- 172.21.15.106:0/2734453221 wait complete. 2024-04-13T03:17:38.334 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-04-13T03:17:36.471754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260616,"kb_used_data":11120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714104,"statfs":{"total":773094113280,"available":772827242496,"internally_reserved":0,"allocated":11386880,"data_stored":5901560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.094677"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384935+0000","last_change":"2024-04-13T03:17:32.384935+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384935+0000","last_undegraded":"2024-04-13T03:17:32.384935+0000","last_fullsized":"2024-04-13T03:17:32.384935+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381472+0000","last_change":"2024-04-13T03:17:32.381472+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381472+0000","last_undegraded":"2024-04-13T03:17:32.381472+0000","last_fullsized":"2024-04-13T03:17:32.381472+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383567+0000","last_change":"2024-04-13T03:17:32.383567+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383567+0000","last_undegraded":"2024-04-13T03:17:32.383567+0000","last_fullsized":"2024-04-13T03:17:32.383567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.380225+0000","last_change":"2024-04-13T03:17:32.380225+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.380225+0000","last_undegraded":"2024-04-13T03:17:32.380225+0000","last_fullsized":"2024-04-13T03:17:32.380225+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385837+0000","last_change":"2024-04-13T03:17:32.385837+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385837+0000","last_undegraded":"2024-04-13T03:17:32.385837+0000","last_fullsized":"2024-04-13T03:17:32.385837+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383816+0000","last_change":"2024-04-13T03:17:32.383816+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383816+0000","last_undegraded":"2024-04-13T03:17:32.383816+0000","last_fullsized":"2024-04-13T03:17:32.383816+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383280+0000","last_change":"2024-04-13T03:17:32.383280+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383280+0000","last_undegraded":"2024-04-13T03:17:32.383280+0000","last_fullsized":"2024-04-13T03:17:32.383280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384076+0000","last_change":"2024-04-13T03:17:32.384076+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384076+0000","last_undegraded":"2024-04-13T03:17:32.384076+0000","last_fullsized":"2024-04-13T03:17:32.384076+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383565+0000","last_change":"2024-04-13T03:17:32.383565+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383565+0000","last_undegraded":"2024-04-13T03:17:32.383565+0000","last_fullsized":"2024-04-13T03:17:32.383565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384582+0000","last_change":"2024-04-13T03:17:32.384582+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384582+0000","last_undegraded":"2024-04-13T03:17:32.384582+0000","last_fullsized":"2024-04-13T03:17:32.384582+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385594+0000","last_change":"2024-04-13T03:17:32.385594+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385594+0000","last_undegraded":"2024-04-13T03:17:32.385594+0000","last_fullsized":"2024-04-13T03:17:32.385594+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382459+0000","last_change":"2024-04-13T03:17:32.382459+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382459+0000","last_undegraded":"2024-04-13T03:17:32.382459+0000","last_fullsized":"2024-04-13T03:17:32.382459+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379279+0000","last_change":"2024-04-13T03:17:32.379279+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379279+0000","last_undegraded":"2024-04-13T03:17:32.379279+0000","last_fullsized":"2024-04-13T03:17:32.379279+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383988+0000","last_change":"2024-04-13T03:17:32.383988+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383988+0000","last_undegraded":"2024-04-13T03:17:32.383988+0000","last_fullsized":"2024-04-13T03:17:32.383988+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381081+0000","last_change":"2024-04-13T03:17:32.381081+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381081+0000","last_undegraded":"2024-04-13T03:17:32.381081+0000","last_fullsized":"2024-04-13T03:17:32.381081+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382660+0000","last_change":"2024-04-13T03:17:32.382660+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382660+0000","last_undegraded":"2024-04-13T03:17:32.382660+0000","last_fullsized":"2024-04-13T03:17:32.382660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383004+0000","last_change":"2024-04-13T03:17:32.383004+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383004+0000","last_undegraded":"2024-04-13T03:17:32.383004+0000","last_fullsized":"2024-04-13T03:17:32.383004+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.385088+0000","last_change":"2024-04-13T03:17:32.385088+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.385088+0000","last_undegraded":"2024-04-13T03:17:32.385088+0000","last_fullsized":"2024-04-13T03:17:32.385088+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383810+0000","last_change":"2024-04-13T03:17:32.383810+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383810+0000","last_undegraded":"2024-04-13T03:17:32.383810+0000","last_fullsized":"2024-04-13T03:17:32.383810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381058+0000","last_change":"2024-04-13T03:17:32.381058+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381058+0000","last_undegraded":"2024-04-13T03:17:32.381058+0000","last_fullsized":"2024-04-13T03:17:32.381058+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381695+0000","last_change":"2024-04-13T03:17:32.381695+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381695+0000","last_undegraded":"2024-04-13T03:17:32.381695+0000","last_fullsized":"2024-04-13T03:17:32.381695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383708+0000","last_change":"2024-04-13T03:17:32.383708+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383708+0000","last_undegraded":"2024-04-13T03:17:32.383708+0000","last_fullsized":"2024-04-13T03:17:32.383708+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381676+0000","last_change":"2024-04-13T03:17:32.381676+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381676+0000","last_undegraded":"2024-04-13T03:17:32.381676+0000","last_fullsized":"2024-04-13T03:17:32.381676+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379287+0000","last_change":"2024-04-13T03:17:32.379287+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379287+0000","last_undegraded":"2024-04-13T03:17:32.379287+0000","last_fullsized":"2024-04-13T03:17:32.379287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.384103+0000","last_change":"2024-04-13T03:17:32.384103+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.384103+0000","last_undegraded":"2024-04-13T03:17:32.384103+0000","last_fullsized":"2024-04-13T03:17:32.384103+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382396+0000","last_change":"2024-04-13T03:17:32.382396+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382396+0000","last_undegraded":"2024-04-13T03:17:32.382396+0000","last_fullsized":"2024-04-13T03:17:32.382396+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.379299+0000","last_change":"2024-04-13T03:17:32.379299+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.379299+0000","last_undegraded":"2024-04-13T03:17:32.379299+0000","last_fullsized":"2024-04-13T03:17:32.379299+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.383508+0000","last_change":"2024-04-13T03:17:32.383508+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.383508+0000","last_undegraded":"2024-04-13T03:17:32.383508+0000","last_fullsized":"2024-04-13T03:17:32.383508+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381279+0000","last_change":"2024-04-13T03:17:32.381279+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381279+0000","last_undegraded":"2024-04-13T03:17:32.381279+0000","last_fullsized":"2024-04-13T03:17:32.381279+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.382390+0000","last_change":"2024-04-13T03:17:32.382390+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.382390+0000","last_undegraded":"2024-04-13T03:17:32.382390+0000","last_fullsized":"2024-04-13T03:17:32.382390+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381020+0000","last_change":"2024-04-13T03:17:32.381020+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381020+0000","last_undegraded":"2024-04-13T03:17:32.381020+0000","last_fullsized":"2024-04-13T03:17:32.381020+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:17:33.377189+0000","last_change":"2024-04-13T03:17:33.377189+0000","last_active":"2024-04-13T03:17:33.377189+0000","last_peered":"2024-04-13T03:17:33.377189+0000","last_clean":"2024-04-13T03:17:33.377189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:33.377189+0000","last_undegraded":"2024-04-13T03:17:33.377189+0000","last_fullsized":"2024-04-13T03:17:33.377189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-04-13T03:17:32.381091+0000","last_change":"2024-04-13T03:17:32.381091+0000","last_active":"2024-04-13T03:17:32.372106+0000","last_peered":"2024-04-13T03:17:32.372106+0000","last_clean":"2024-04-13T03:17:32.372106+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:17:32.381091+0000","last_undegraded":"2024-04-13T03:17:32.381091+0000","last_fullsized":"2024-04-13T03:17:32.381091+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":325,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-13T03:17:32.385816+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:17:32.385816+0000","last_peered":"2024-04-13T03:17:32.385816+0000","last_clean":"2024-04-13T03:17:32.385816+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:17:32.385816+0000","last_undegraded":"2024-04-13T03:17:32.385816+0000","last_fullsized":"2024-04-13T03:17:32.385816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1884160,"data_stored":1188026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":585790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1871872,"data_stored":1181817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:17:38.338 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:17:38.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.474+0000 7f4c2f7e0640 1 Processor -- start 2024-04-13T03:17:38.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.474+0000 7f4c2f7e0640 1 -- start start 2024-04-13T03:17:38.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2f7e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2f7e0640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4c280b5ab0 0x7f4c2806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2f7e0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c280aefc0 con 0x7f4c2815ce90 2024-04-13T03:17:38.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2f7e0640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2806e180 con 0x7f4c280b5ab0 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2f7e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280ad180 con 0x7f4c280aebe0 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2d57c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2dd7d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4c280b5ab0 0x7f4c2806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2d57c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:54166/0 (socket says 172.21.15.106:54166) 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.475+0000 7f4c2d57c640 1 -- 172.21.15.106:0/3154607129 learned_addr learned my addr 172.21.15.106:0/3154607129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.476+0000 7f4c2d57c640 1 -- 172.21.15.106:0/3154607129 >> v2:172.21.15.114:3300/0 conn(0x7f4c280b5ab0 msgr2=0x7f4c2806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.476+0000 7f4c2d57c640 1 --2- 172.21.15.106:0/3154607129 >> v2:172.21.15.114:3300/0 conn(0x7f4c280b5ab0 0x7f4c2806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.476+0000 7f4c2d57c640 1 -- 172.21.15.106:0/3154607129 >> v1:172.21.15.114:6789/0 conn(0x7f4c2815ce90 legacy=0x7f4c280b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.476+0000 7f4c2d57c640 1 -- 172.21.15.106:0/3154607129 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c28160ca0 con 0x7f4c280aebe0 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.476+0000 7f4c2d57c640 1 --2- 172.21.15.106:0/3154607129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a038ef957564a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c1ffff640 1 -- 172.21.15.106:0/3154607129 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20035020 con 0x7f4c280aebe0 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c1ffff640 1 -- 172.21.15.106:0/3154607129 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20031640 con 0x7f4c280aebe0 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c1ffff640 1 -- 172.21.15.106:0/3154607129 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2002d040 con 0x7f4c280aebe0 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/3154607129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 msgr2=0x7f4c281603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/3154607129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/3154607129 shutdown_connections 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/3154607129 >> v2:172.21.15.114:3300/0 conn(0x7f4c280b5ab0 0x7f4c2806e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/3154607129 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c280aebe0 0x7f4c281603a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/3154607129 >> 172.21.15.106:0/3154607129 conn(0x7f4c280b1f10 msgr2=0x7f4c280b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/3154607129 shutdown_connections 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.477+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/3154607129 wait complete. 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.478+0000 7f4c2f7e0640 1 Processor -- start 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.478+0000 7f4c2f7e0640 1 -- start start 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c280aebe0 0x7f4c28155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c2815ce90 0x7f4c280c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2806de10 con 0x7f4c280b5ab0 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28161260 con 0x7f4c280aebe0 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2f7e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4c281608e0 con 0x7f4c2815ce90 2024-04-13T03:17:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2cd7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2cd7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55120/0 (socket says 172.21.15.106:55120) 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2cd7b640 1 -- 172.21.15.106:0/1182706284 learned_addr learned my addr 172.21.15.106:0/1182706284 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2dd7d640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c2815ce90 0x7f4c280c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.479+0000 7f4c2d57c640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c280aebe0 0x7f4c28155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 -- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c2815ce90 msgr2=0x7f4c280c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c2815ce90 0x7f4c280c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 -- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c280aebe0 msgr2=0x7f4c28155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c280aebe0 0x7f4c28155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 -- 172.21.15.106:0/1182706284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280c7f00 con 0x7f4c280b5ab0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2cd7b640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c3f0 tx=0x7f4c1800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18017070 con 0x7f4c280b5ab0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c180052f0 con 0x7f4c280b5ab0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c280c81a0 con 0x7f4c280b5ab0 2024-04-13T03:17:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003830 con 0x7f4c280b5ab0 2024-04-13T03:17:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.480+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c280c8760 con 0x7f4c280b5ab0 2024-04-13T03:17:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.481+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c28075180 con 0x7f4c280b5ab0 2024-04-13T03:17:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.481+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c1801a070 con 0x7f4c280b5ab0 2024-04-13T03:17:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.482+0000 7f4c1dffb640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.482+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f4c1808b0b0 con 0x7f4c280b5ab0 2024-04-13T03:17:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.482+0000 7f4c2d57c640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.482+0000 7f4c2d57c640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4c20036000 tx=0x7f4c2003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.487+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c28075180 con 0x7f4c280b5ab0 2024-04-13T03:17:38.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.659+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c28068a10 con 0x7f4c0c06c8f0 2024-04-13T03:17:38.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.665+0000 7f4c1dffb640 1 -- 172.21.15.106:0/1182706284 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f4c28068a10 con 0x7f4c0c06c8f0 2024-04-13T03:17:38.669 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:38.670 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:17:38.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 msgr2=0x7f4c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4c20036000 tx=0x7f4c2003c040 comp rx=0 tx=0).stop 2024-04-13T03:17:38.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 msgr2=0x7f4c28072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c3f0 tx=0x7f4c1800c8b0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 shutdown_connections 2024-04-13T03:17:38.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c2815ce90 0x7f4c280c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c280b5ab0 0x7f4c28072a50 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 --2- 172.21.15.106:0/1182706284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c280aebe0 0x7f4c28155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 >> 172.21.15.106:0/1182706284 conn(0x7f4c280b1f10 msgr2=0x7f4c280af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 shutdown_connections 2024-04-13T03:17:38.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.668+0000 7f4c2f7e0640 1 -- 172.21.15.106:0/1182706284 wait complete. 2024-04-13T03:17:38.679 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-13T03:17:38.472210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261040,"kb_used_data":11592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713680,"statfs":{"total":773094113280,"available":772826808320,"internally_reserved":0,"allocated":11870208,"data_stored":6131028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001179"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395781+0000","last_change":"2024-04-13T03:17:34.397755+0000","last_active":"2024-04-13T03:17:35.395781+0000","last_peered":"2024-04-13T03:17:35.395781+0000","last_clean":"2024-04-13T03:17:35.395781+0000","last_became_active":"2024-04-13T03:17:34.397499+0000","last_became_peered":"2024-04-13T03:17:34.397499+0000","last_unstale":"2024-04-13T03:17:35.395781+0000","last_undegraded":"2024-04-13T03:17:35.395781+0000","last_fullsized":"2024-04-13T03:17:35.395781+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:52.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065784+0000","last_change":"2024-04-13T03:17:34.414082+0000","last_active":"2024-04-13T03:17:36.065784+0000","last_peered":"2024-04-13T03:17:36.065784+0000","last_clean":"2024-04-13T03:17:36.065784+0000","last_became_active":"2024-04-13T03:17:34.412064+0000","last_became_peered":"2024-04-13T03:17:34.412064+0000","last_unstale":"2024-04-13T03:17:36.065784+0000","last_undegraded":"2024-04-13T03:17:36.065784+0000","last_fullsized":"2024-04-13T03:17:36.065784+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395702+0000","last_change":"2024-04-13T03:17:34.406889+0000","last_active":"2024-04-13T03:17:35.395702+0000","last_peered":"2024-04-13T03:17:35.395702+0000","last_clean":"2024-04-13T03:17:35.395702+0000","last_became_active":"2024-04-13T03:17:34.406253+0000","last_became_peered":"2024-04-13T03:17:34.406253+0000","last_unstale":"2024-04-13T03:17:35.395702+0000","last_undegraded":"2024-04-13T03:17:35.395702+0000","last_fullsized":"2024-04-13T03:17:35.395702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:00:22.906340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480782+0000","last_change":"2024-04-13T03:17:34.396984+0000","last_active":"2024-04-13T03:17:35.480782+0000","last_peered":"2024-04-13T03:17:35.480782+0000","last_clean":"2024-04-13T03:17:35.480782+0000","last_became_active":"2024-04-13T03:17:34.396599+0000","last_became_peered":"2024-04-13T03:17:34.396599+0000","last_unstale":"2024-04-13T03:17:35.480782+0000","last_undegraded":"2024-04-13T03:17:35.480782+0000","last_fullsized":"2024-04-13T03:17:35.480782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:05:44.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910724+0000","last_change":"2024-04-13T03:17:34.399389+0000","last_active":"2024-04-13T03:17:35.910724+0000","last_peered":"2024-04-13T03:17:35.910724+0000","last_clean":"2024-04-13T03:17:35.910724+0000","last_became_active":"2024-04-13T03:17:34.399010+0000","last_became_peered":"2024-04-13T03:17:34.399010+0000","last_unstale":"2024-04-13T03:17:35.910724+0000","last_undegraded":"2024-04-13T03:17:35.910724+0000","last_fullsized":"2024-04-13T03:17:35.910724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066413+0000","last_change":"2024-04-13T03:17:34.412752+0000","last_active":"2024-04-13T03:17:36.066413+0000","last_peered":"2024-04-13T03:17:36.066413+0000","last_clean":"2024-04-13T03:17:36.066413+0000","last_became_active":"2024-04-13T03:17:34.412470+0000","last_became_peered":"2024-04-13T03:17:34.412470+0000","last_unstale":"2024-04-13T03:17:36.066413+0000","last_undegraded":"2024-04-13T03:17:36.066413+0000","last_fullsized":"2024-04-13T03:17:36.066413+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065869+0000","last_change":"2024-04-13T03:17:34.411655+0000","last_active":"2024-04-13T03:17:36.065869+0000","last_peered":"2024-04-13T03:17:36.065869+0000","last_clean":"2024-04-13T03:17:36.065869+0000","last_became_active":"2024-04-13T03:17:34.411211+0000","last_became_peered":"2024-04-13T03:17:34.411211+0000","last_unstale":"2024-04-13T03:17:36.065869+0000","last_undegraded":"2024-04-13T03:17:36.065869+0000","last_fullsized":"2024-04-13T03:17:36.065869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389760+0000","last_change":"2024-04-13T03:17:34.402872+0000","last_active":"2024-04-13T03:17:35.389760+0000","last_peered":"2024-04-13T03:17:35.389760+0000","last_clean":"2024-04-13T03:17:35.389760+0000","last_became_active":"2024-04-13T03:17:34.401106+0000","last_became_peered":"2024-04-13T03:17:34.401106+0000","last_unstale":"2024-04-13T03:17:35.389760+0000","last_undegraded":"2024-04-13T03:17:35.389760+0000","last_fullsized":"2024-04-13T03:17:35.389760+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:15:37.328883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065661+0000","last_change":"2024-04-13T03:17:34.410456+0000","last_active":"2024-04-13T03:17:36.065661+0000","last_peered":"2024-04-13T03:17:36.065661+0000","last_clean":"2024-04-13T03:17:36.065661+0000","last_became_active":"2024-04-13T03:17:34.410166+0000","last_became_peered":"2024-04-13T03:17:34.410166+0000","last_unstale":"2024-04-13T03:17:36.065661+0000","last_undegraded":"2024-04-13T03:17:36.065661+0000","last_fullsized":"2024-04-13T03:17:36.065661+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:35.141523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.911023+0000","last_change":"2024-04-13T03:17:34.410800+0000","last_active":"2024-04-13T03:17:35.911023+0000","last_peered":"2024-04-13T03:17:35.911023+0000","last_clean":"2024-04-13T03:17:35.911023+0000","last_became_active":"2024-04-13T03:17:34.410568+0000","last_became_peered":"2024-04-13T03:17:34.410568+0000","last_unstale":"2024-04-13T03:17:35.911023+0000","last_undegraded":"2024-04-13T03:17:35.911023+0000","last_fullsized":"2024-04-13T03:17:35.911023+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395658+0000","last_change":"2024-04-13T03:17:34.407143+0000","last_active":"2024-04-13T03:17:35.395658+0000","last_peered":"2024-04-13T03:17:35.395658+0000","last_clean":"2024-04-13T03:17:35.395658+0000","last_became_active":"2024-04-13T03:17:34.406821+0000","last_became_peered":"2024-04-13T03:17:34.406821+0000","last_unstale":"2024-04-13T03:17:35.395658+0000","last_undegraded":"2024-04-13T03:17:35.395658+0000","last_fullsized":"2024-04-13T03:17:35.395658+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:15:20.135164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066224+0000","last_change":"2024-04-13T03:17:34.392870+0000","last_active":"2024-04-13T03:17:36.066224+0000","last_peered":"2024-04-13T03:17:36.066224+0000","last_clean":"2024-04-13T03:17:36.066224+0000","last_became_active":"2024-04-13T03:17:34.392598+0000","last_became_peered":"2024-04-13T03:17:34.392598+0000","last_unstale":"2024-04-13T03:17:36.066224+0000","last_undegraded":"2024-04-13T03:17:36.066224+0000","last_fullsized":"2024-04-13T03:17:36.066224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:08.590896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480726+0000","last_change":"2024-04-13T03:17:34.406086+0000","last_active":"2024-04-13T03:17:35.480726+0000","last_peered":"2024-04-13T03:17:35.480726+0000","last_clean":"2024-04-13T03:17:35.480726+0000","last_became_active":"2024-04-13T03:17:34.405810+0000","last_became_peered":"2024-04-13T03:17:34.405810+0000","last_unstale":"2024-04-13T03:17:35.480726+0000","last_undegraded":"2024-04-13T03:17:35.480726+0000","last_fullsized":"2024-04-13T03:17:35.480726+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:02.642224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065362+0000","last_change":"2024-04-13T03:17:34.400371+0000","last_active":"2024-04-13T03:17:36.065362+0000","last_peered":"2024-04-13T03:17:36.065362+0000","last_clean":"2024-04-13T03:17:36.065362+0000","last_became_active":"2024-04-13T03:17:34.399998+0000","last_became_peered":"2024-04-13T03:17:34.399998+0000","last_unstale":"2024-04-13T03:17:36.065362+0000","last_undegraded":"2024-04-13T03:17:36.065362+0000","last_fullsized":"2024-04-13T03:17:36.065362+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065088+0000","last_change":"2024-04-13T03:17:34.407130+0000","last_active":"2024-04-13T03:17:36.065088+0000","last_peered":"2024-04-13T03:17:36.065088+0000","last_clean":"2024-04-13T03:17:36.065088+0000","last_became_active":"2024-04-13T03:17:34.406854+0000","last_became_peered":"2024-04-13T03:17:34.406854+0000","last_unstale":"2024-04-13T03:17:36.065088+0000","last_undegraded":"2024-04-13T03:17:36.065088+0000","last_fullsized":"2024-04-13T03:17:36.065088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065064+0000","last_change":"2024-04-13T03:17:34.401481+0000","last_active":"2024-04-13T03:17:36.065064+0000","last_peered":"2024-04-13T03:17:36.065064+0000","last_clean":"2024-04-13T03:17:36.065064+0000","last_became_active":"2024-04-13T03:17:34.401209+0000","last_became_peered":"2024-04-13T03:17:34.401209+0000","last_unstale":"2024-04-13T03:17:36.065064+0000","last_undegraded":"2024-04-13T03:17:36.065064+0000","last_fullsized":"2024-04-13T03:17:36.065064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:21.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395885+0000","last_change":"2024-04-13T03:17:34.405442+0000","last_active":"2024-04-13T03:17:35.395885+0000","last_peered":"2024-04-13T03:17:35.395885+0000","last_clean":"2024-04-13T03:17:35.395885+0000","last_became_active":"2024-04-13T03:17:34.404800+0000","last_became_peered":"2024-04-13T03:17:34.404800+0000","last_unstale":"2024-04-13T03:17:35.395885+0000","last_undegraded":"2024-04-13T03:17:35.395885+0000","last_fullsized":"2024-04-13T03:17:35.395885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:50:42.464825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065972+0000","last_change":"2024-04-13T03:17:34.410814+0000","last_active":"2024-04-13T03:17:36.065972+0000","last_peered":"2024-04-13T03:17:36.065972+0000","last_clean":"2024-04-13T03:17:36.065972+0000","last_became_active":"2024-04-13T03:17:34.410551+0000","last_became_peered":"2024-04-13T03:17:34.410551+0000","last_unstale":"2024-04-13T03:17:36.065972+0000","last_undegraded":"2024-04-13T03:17:36.065972+0000","last_fullsized":"2024-04-13T03:17:36.065972+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395709+0000","last_change":"2024-04-13T03:17:34.408576+0000","last_active":"2024-04-13T03:17:35.395709+0000","last_peered":"2024-04-13T03:17:35.395709+0000","last_clean":"2024-04-13T03:17:35.395709+0000","last_became_active":"2024-04-13T03:17:34.408330+0000","last_became_peered":"2024-04-13T03:17:34.408330+0000","last_unstale":"2024-04-13T03:17:35.395709+0000","last_undegraded":"2024-04-13T03:17:35.395709+0000","last_fullsized":"2024-04-13T03:17:35.395709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:15.555032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066073+0000","last_change":"2024-04-13T03:17:34.398173+0000","last_active":"2024-04-13T03:17:36.066073+0000","last_peered":"2024-04-13T03:17:36.066073+0000","last_clean":"2024-04-13T03:17:36.066073+0000","last_became_active":"2024-04-13T03:17:34.397874+0000","last_became_peered":"2024-04-13T03:17:34.397874+0000","last_unstale":"2024-04-13T03:17:36.066073+0000","last_undegraded":"2024-04-13T03:17:36.066073+0000","last_fullsized":"2024-04-13T03:17:36.066073+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390283+0000","last_change":"2024-04-13T03:17:34.411955+0000","last_active":"2024-04-13T03:17:35.390283+0000","last_peered":"2024-04-13T03:17:35.390283+0000","last_clean":"2024-04-13T03:17:35.390283+0000","last_became_active":"2024-04-13T03:17:34.411568+0000","last_became_peered":"2024-04-13T03:17:34.411568+0000","last_unstale":"2024-04-13T03:17:35.390283+0000","last_undegraded":"2024-04-13T03:17:35.390283+0000","last_fullsized":"2024-04-13T03:17:35.390283+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:28:52.418315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910831+0000","last_change":"2024-04-13T03:17:34.410118+0000","last_active":"2024-04-13T03:17:35.910831+0000","last_peered":"2024-04-13T03:17:35.910831+0000","last_clean":"2024-04-13T03:17:35.910831+0000","last_became_active":"2024-04-13T03:17:34.409793+0000","last_became_peered":"2024-04-13T03:17:34.409793+0000","last_unstale":"2024-04-13T03:17:35.910831+0000","last_undegraded":"2024-04-13T03:17:35.910831+0000","last_fullsized":"2024-04-13T03:17:35.910831+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395535+0000","last_change":"2024-04-13T03:17:34.402676+0000","last_active":"2024-04-13T03:17:35.395535+0000","last_peered":"2024-04-13T03:17:35.395535+0000","last_clean":"2024-04-13T03:17:35.395535+0000","last_became_active":"2024-04-13T03:17:34.402403+0000","last_became_peered":"2024-04-13T03:17:34.402403+0000","last_unstale":"2024-04-13T03:17:35.395535+0000","last_undegraded":"2024-04-13T03:17:35.395535+0000","last_fullsized":"2024-04-13T03:17:35.395535+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:27.486850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910608+0000","last_change":"2024-04-13T03:17:34.409245+0000","last_active":"2024-04-13T03:17:35.910608+0000","last_peered":"2024-04-13T03:17:35.910608+0000","last_clean":"2024-04-13T03:17:35.910608+0000","last_became_active":"2024-04-13T03:17:34.408969+0000","last_became_peered":"2024-04-13T03:17:34.408969+0000","last_unstale":"2024-04-13T03:17:35.910608+0000","last_undegraded":"2024-04-13T03:17:35.910608+0000","last_fullsized":"2024-04-13T03:17:35.910608+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065690+0000","last_change":"2024-04-13T03:17:34.405874+0000","last_active":"2024-04-13T03:17:36.065690+0000","last_peered":"2024-04-13T03:17:36.065690+0000","last_clean":"2024-04-13T03:17:36.065690+0000","last_became_active":"2024-04-13T03:17:34.405601+0000","last_became_peered":"2024-04-13T03:17:34.405601+0000","last_unstale":"2024-04-13T03:17:36.065690+0000","last_undegraded":"2024-04-13T03:17:36.065690+0000","last_fullsized":"2024-04-13T03:17:36.065690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065465+0000","last_change":"2024-04-13T03:17:34.407016+0000","last_active":"2024-04-13T03:17:36.065465+0000","last_peered":"2024-04-13T03:17:36.065465+0000","last_clean":"2024-04-13T03:17:36.065465+0000","last_became_active":"2024-04-13T03:17:34.406642+0000","last_became_peered":"2024-04-13T03:17:34.406642+0000","last_unstale":"2024-04-13T03:17:36.065465+0000","last_undegraded":"2024-04-13T03:17:36.065465+0000","last_fullsized":"2024-04-13T03:17:36.065465+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395683+0000","last_change":"2024-04-13T03:17:34.412860+0000","last_active":"2024-04-13T03:17:35.395683+0000","last_peered":"2024-04-13T03:17:35.395683+0000","last_clean":"2024-04-13T03:17:35.395683+0000","last_became_active":"2024-04-13T03:17:34.411893+0000","last_became_peered":"2024-04-13T03:17:34.411893+0000","last_unstale":"2024-04-13T03:17:35.395683+0000","last_undegraded":"2024-04-13T03:17:35.395683+0000","last_fullsized":"2024-04-13T03:17:35.395683+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:45:20.716257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065970+0000","last_change":"2024-04-13T03:17:34.414317+0000","last_active":"2024-04-13T03:17:36.065970+0000","last_peered":"2024-04-13T03:17:36.065970+0000","last_clean":"2024-04-13T03:17:36.065970+0000","last_became_active":"2024-04-13T03:17:34.414038+0000","last_became_peered":"2024-04-13T03:17:34.414038+0000","last_unstale":"2024-04-13T03:17:36.065970+0000","last_undegraded":"2024-04-13T03:17:36.065970+0000","last_fullsized":"2024-04-13T03:17:36.065970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390278+0000","last_change":"2024-04-13T03:17:34.403227+0000","last_active":"2024-04-13T03:17:35.390278+0000","last_peered":"2024-04-13T03:17:35.390278+0000","last_clean":"2024-04-13T03:17:35.390278+0000","last_became_active":"2024-04-13T03:17:34.402961+0000","last_became_peered":"2024-04-13T03:17:34.402961+0000","last_unstale":"2024-04-13T03:17:35.390278+0000","last_undegraded":"2024-04-13T03:17:35.390278+0000","last_fullsized":"2024-04-13T03:17:35.390278+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:53:56.840439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066167+0000","last_change":"2024-04-13T03:17:34.405766+0000","last_active":"2024-04-13T03:17:36.066167+0000","last_peered":"2024-04-13T03:17:36.066167+0000","last_clean":"2024-04-13T03:17:36.066167+0000","last_became_active":"2024-04-13T03:17:34.404793+0000","last_became_peered":"2024-04-13T03:17:34.404793+0000","last_unstale":"2024-04-13T03:17:36.066167+0000","last_undegraded":"2024-04-13T03:17:36.066167+0000","last_fullsized":"2024-04-13T03:17:36.066167+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:13:26.382175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.416600+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.416208+0000","last_became_peered":"2024-04-13T03:17:34.416208+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389814+0000","last_change":"2024-04-13T03:17:34.402382+0000","last_active":"2024-04-13T03:17:35.389814+0000","last_peered":"2024-04-13T03:17:35.389814+0000","last_clean":"2024-04-13T03:17:35.389814+0000","last_became_active":"2024-04-13T03:17:34.401181+0000","last_became_peered":"2024-04-13T03:17:34.401181+0000","last_unstale":"2024-04-13T03:17:35.389814+0000","last_undegraded":"2024-04-13T03:17:35.389814+0000","last_fullsized":"2024-04-13T03:17:35.389814+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:43.678954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395636+0000","last_change":"2024-04-13T03:17:34.408372+0000","last_active":"2024-04-13T03:17:35.395636+0000","last_peered":"2024-04-13T03:17:35.395636+0000","last_clean":"2024-04-13T03:17:35.395636+0000","last_became_active":"2024-04-13T03:17:34.408097+0000","last_became_peered":"2024-04-13T03:17:34.408097+0000","last_unstale":"2024-04-13T03:17:35.395636+0000","last_undegraded":"2024-04-13T03:17:35.395636+0000","last_fullsized":"2024-04-13T03:17:35.395636+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:14:09.892547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480883+0000","last_change":"2024-04-13T03:17:33.402709+0000","last_active":"2024-04-13T03:17:35.480883+0000","last_peered":"2024-04-13T03:17:35.480883+0000","last_clean":"2024-04-13T03:17:35.480883+0000","last_became_active":"2024-04-13T03:17:33.402409+0000","last_became_peered":"2024-04-13T03:17:33.402409+0000","last_unstale":"2024-04-13T03:17:35.480883+0000","last_undegraded":"2024-04-13T03:17:35.480883+0000","last_fullsized":"2024-04-13T03:17:35.480883+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:06.088410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389771+0000","last_change":"2024-04-13T03:17:34.403177+0000","last_active":"2024-04-13T03:17:35.389771+0000","last_peered":"2024-04-13T03:17:35.389771+0000","last_clean":"2024-04-13T03:17:35.389771+0000","last_became_active":"2024-04-13T03:17:34.402894+0000","last_became_peered":"2024-04-13T03:17:34.402894+0000","last_unstale":"2024-04-13T03:17:35.389771+0000","last_undegraded":"2024-04-13T03:17:35.389771+0000","last_fullsized":"2024-04-13T03:17:35.389771+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:34.744333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390131+0000","last_change":"2024-04-13T03:17:33.401391+0000","last_active":"2024-04-13T03:17:35.390131+0000","last_peered":"2024-04-13T03:17:35.390131+0000","last_clean":"2024-04-13T03:17:35.390131+0000","last_became_active":"2024-04-13T03:17:33.401115+0000","last_became_peered":"2024-04-13T03:17:33.401115+0000","last_unstale":"2024-04-13T03:17:35.390131+0000","last_undegraded":"2024-04-13T03:17:35.390131+0000","last_fullsized":"2024-04-13T03:17:35.390131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:34.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066244+0000","last_change":"2024-04-13T03:17:34.401443+0000","last_active":"2024-04-13T03:17:36.066244+0000","last_peered":"2024-04-13T03:17:36.066244+0000","last_clean":"2024-04-13T03:17:36.066244+0000","last_became_active":"2024-04-13T03:17:34.401088+0000","last_became_peered":"2024-04-13T03:17:34.401088+0000","last_unstale":"2024-04-13T03:17:36.066244+0000","last_undegraded":"2024-04-13T03:17:36.066244+0000","last_fullsized":"2024-04-13T03:17:36.066244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:05:39.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065122+0000","last_change":"2024-04-13T03:17:33.396742+0000","last_active":"2024-04-13T03:17:36.065122+0000","last_peered":"2024-04-13T03:17:36.065122+0000","last_clean":"2024-04-13T03:17:36.065122+0000","last_became_active":"2024-04-13T03:17:33.396322+0000","last_became_peered":"2024-04-13T03:17:33.396322+0000","last_unstale":"2024-04-13T03:17:36.065122+0000","last_undegraded":"2024-04-13T03:17:36.065122+0000","last_fullsized":"2024-04-13T03:17:36.065122+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:13.384064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395276+0000","last_change":"2024-04-13T03:17:34.404596+0000","last_active":"2024-04-13T03:17:35.395276+0000","last_peered":"2024-04-13T03:17:35.395276+0000","last_clean":"2024-04-13T03:17:35.395276+0000","last_became_active":"2024-04-13T03:17:34.403699+0000","last_became_peered":"2024-04-13T03:17:34.403699+0000","last_unstale":"2024-04-13T03:17:35.395276+0000","last_undegraded":"2024-04-13T03:17:35.395276+0000","last_fullsized":"2024-04-13T03:17:35.395276+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910957+0000","last_change":"2024-04-13T03:17:33.398782+0000","last_active":"2024-04-13T03:17:35.910957+0000","last_peered":"2024-04-13T03:17:35.910957+0000","last_clean":"2024-04-13T03:17:35.910957+0000","last_became_active":"2024-04-13T03:17:33.398493+0000","last_became_peered":"2024-04-13T03:17:33.398493+0000","last_unstale":"2024-04-13T03:17:35.910957+0000","last_undegraded":"2024-04-13T03:17:35.910957+0000","last_fullsized":"2024-04-13T03:17:35.910957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:06:27.097899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480837+0000","last_change":"2024-04-13T03:17:34.400995+0000","last_active":"2024-04-13T03:17:35.480837+0000","last_peered":"2024-04-13T03:17:35.480837+0000","last_clean":"2024-04-13T03:17:35.480837+0000","last_became_active":"2024-04-13T03:17:34.400701+0000","last_became_peered":"2024-04-13T03:17:34.400701+0000","last_unstale":"2024-04-13T03:17:35.480837+0000","last_undegraded":"2024-04-13T03:17:35.480837+0000","last_fullsized":"2024-04-13T03:17:35.480837+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:16:23.670640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065289+0000","last_change":"2024-04-13T03:17:33.401599+0000","last_active":"2024-04-13T03:17:36.065289+0000","last_peered":"2024-04-13T03:17:36.065289+0000","last_clean":"2024-04-13T03:17:36.065289+0000","last_became_active":"2024-04-13T03:17:33.401334+0000","last_became_peered":"2024-04-13T03:17:33.401334+0000","last_unstale":"2024-04-13T03:17:36.065289+0000","last_undegraded":"2024-04-13T03:17:36.065289+0000","last_fullsized":"2024-04-13T03:17:36.065289+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:37.518351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.408972+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.408008+0000","last_became_peered":"2024-04-13T03:17:34.408008+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:44:31.275566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395064+0000","last_change":"2024-04-13T03:17:33.400110+0000","last_active":"2024-04-13T03:17:35.395064+0000","last_peered":"2024-04-13T03:17:35.395064+0000","last_clean":"2024-04-13T03:17:35.395064+0000","last_became_active":"2024-04-13T03:17:33.399844+0000","last_became_peered":"2024-04-13T03:17:33.399844+0000","last_unstale":"2024-04-13T03:17:35.395064+0000","last_undegraded":"2024-04-13T03:17:35.395064+0000","last_fullsized":"2024-04-13T03:17:35.395064+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:15:33.561822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.414775+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.414524+0000","last_became_peered":"2024-04-13T03:17:34.414524+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066358+0000","last_change":"2024-04-13T03:17:33.396584+0000","last_active":"2024-04-13T03:17:36.066358+0000","last_peered":"2024-04-13T03:17:36.066358+0000","last_clean":"2024-04-13T03:17:36.066358+0000","last_became_active":"2024-04-13T03:17:33.395985+0000","last_became_peered":"2024-04-13T03:17:33.395985+0000","last_unstale":"2024-04-13T03:17:36.066358+0000","last_undegraded":"2024-04-13T03:17:36.066358+0000","last_fullsized":"2024-04-13T03:17:36.066358+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:05.909520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480922+0000","last_change":"2024-04-13T03:17:34.401457+0000","last_active":"2024-04-13T03:17:35.480922+0000","last_peered":"2024-04-13T03:17:35.480922+0000","last_clean":"2024-04-13T03:17:35.480922+0000","last_became_active":"2024-04-13T03:17:34.401101+0000","last_became_peered":"2024-04-13T03:17:34.401101+0000","last_unstale":"2024-04-13T03:17:35.480922+0000","last_undegraded":"2024-04-13T03:17:35.480922+0000","last_fullsized":"2024-04-13T03:17:35.480922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:03:45.552674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480567+0000","last_change":"2024-04-13T03:17:33.391995+0000","last_active":"2024-04-13T03:17:35.480567+0000","last_peered":"2024-04-13T03:17:35.480567+0000","last_clean":"2024-04-13T03:17:35.480567+0000","last_became_active":"2024-04-13T03:17:33.391716+0000","last_became_peered":"2024-04-13T03:17:33.391716+0000","last_unstale":"2024-04-13T03:17:35.480567+0000","last_undegraded":"2024-04-13T03:17:35.480567+0000","last_fullsized":"2024-04-13T03:17:35.480567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:01.494353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395497+0000","last_change":"2024-04-13T03:17:34.406950+0000","last_active":"2024-04-13T03:17:35.395497+0000","last_peered":"2024-04-13T03:17:35.395497+0000","last_clean":"2024-04-13T03:17:35.395497+0000","last_became_active":"2024-04-13T03:17:34.406487+0000","last_became_peered":"2024-04-13T03:17:34.406487+0000","last_unstale":"2024-04-13T03:17:35.395497+0000","last_undegraded":"2024-04-13T03:17:35.395497+0000","last_fullsized":"2024-04-13T03:17:35.395497+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:35.175581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065815+0000","last_change":"2024-04-13T03:17:33.397241+0000","last_active":"2024-04-13T03:17:36.065815+0000","last_peered":"2024-04-13T03:17:36.065815+0000","last_clean":"2024-04-13T03:17:36.065815+0000","last_became_active":"2024-04-13T03:17:33.396811+0000","last_became_peered":"2024-04-13T03:17:33.396811+0000","last_unstale":"2024-04-13T03:17:36.065815+0000","last_undegraded":"2024-04-13T03:17:36.065815+0000","last_fullsized":"2024-04-13T03:17:36.065815+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:14.055121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910747+0000","last_change":"2024-04-13T03:17:34.404666+0000","last_active":"2024-04-13T03:17:35.910747+0000","last_peered":"2024-04-13T03:17:35.910747+0000","last_clean":"2024-04-13T03:17:35.910747+0000","last_became_active":"2024-04-13T03:17:34.404419+0000","last_became_peered":"2024-04-13T03:17:34.404419+0000","last_unstale":"2024-04-13T03:17:35.910747+0000","last_undegraded":"2024-04-13T03:17:35.910747+0000","last_fullsized":"2024-04-13T03:17:35.910747+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395462+0000","last_change":"2024-04-13T03:17:33.398984+0000","last_active":"2024-04-13T03:17:35.395462+0000","last_peered":"2024-04-13T03:17:35.395462+0000","last_clean":"2024-04-13T03:17:35.395462+0000","last_became_active":"2024-04-13T03:17:33.398683+0000","last_became_peered":"2024-04-13T03:17:33.398683+0000","last_unstale":"2024-04-13T03:17:35.395462+0000","last_undegraded":"2024-04-13T03:17:35.395462+0000","last_fullsized":"2024-04-13T03:17:35.395462+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:19:37.874955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390013+0000","last_change":"2024-04-13T03:17:34.404347+0000","last_active":"2024-04-13T03:17:35.390013+0000","last_peered":"2024-04-13T03:17:35.390013+0000","last_clean":"2024-04-13T03:17:35.390013+0000","last_became_active":"2024-04-13T03:17:34.404070+0000","last_became_peered":"2024-04-13T03:17:34.404070+0000","last_unstale":"2024-04-13T03:17:35.390013+0000","last_undegraded":"2024-04-13T03:17:35.390013+0000","last_fullsized":"2024-04-13T03:17:35.390013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:27:04.357744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065277+0000","last_change":"2024-04-13T03:17:33.395592+0000","last_active":"2024-04-13T03:17:36.065277+0000","last_peered":"2024-04-13T03:17:36.065277+0000","last_clean":"2024-04-13T03:17:36.065277+0000","last_became_active":"2024-04-13T03:17:33.395290+0000","last_became_peered":"2024-04-13T03:17:33.395290+0000","last_unstale":"2024-04-13T03:17:36.065277+0000","last_undegraded":"2024-04-13T03:17:36.065277+0000","last_fullsized":"2024-04-13T03:17:36.065277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:01.476888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910892+0000","last_change":"2024-04-13T03:17:34.406520+0000","last_active":"2024-04-13T03:17:35.910892+0000","last_peered":"2024-04-13T03:17:35.910892+0000","last_clean":"2024-04-13T03:17:35.910892+0000","last_became_active":"2024-04-13T03:17:34.406214+0000","last_became_peered":"2024-04-13T03:17:34.406214+0000","last_unstale":"2024-04-13T03:17:35.910892+0000","last_undegraded":"2024-04-13T03:17:35.910892+0000","last_fullsized":"2024-04-13T03:17:35.910892+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066117+0000","last_change":"2024-04-13T03:17:33.401643+0000","last_active":"2024-04-13T03:17:36.066117+0000","last_peered":"2024-04-13T03:17:36.066117+0000","last_clean":"2024-04-13T03:17:36.066117+0000","last_became_active":"2024-04-13T03:17:33.401302+0000","last_became_peered":"2024-04-13T03:17:33.401302+0000","last_unstale":"2024-04-13T03:17:36.066117+0000","last_undegraded":"2024-04-13T03:17:36.066117+0000","last_fullsized":"2024-04-13T03:17:36.066117+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:14.155305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395482+0000","last_change":"2024-04-13T03:17:34.410826+0000","last_active":"2024-04-13T03:17:35.395482+0000","last_peered":"2024-04-13T03:17:35.395482+0000","last_clean":"2024-04-13T03:17:35.395482+0000","last_became_active":"2024-04-13T03:17:34.410554+0000","last_became_peered":"2024-04-13T03:17:34.410554+0000","last_unstale":"2024-04-13T03:17:35.395482+0000","last_undegraded":"2024-04-13T03:17:35.395482+0000","last_fullsized":"2024-04-13T03:17:35.395482+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:53:21.523097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910964+0000","last_change":"2024-04-13T03:17:33.399676+0000","last_active":"2024-04-13T03:17:35.910964+0000","last_peered":"2024-04-13T03:17:35.910964+0000","last_clean":"2024-04-13T03:17:35.910964+0000","last_became_active":"2024-04-13T03:17:33.399363+0000","last_became_peered":"2024-04-13T03:17:33.399363+0000","last_unstale":"2024-04-13T03:17:35.910964+0000","last_undegraded":"2024-04-13T03:17:35.910964+0000","last_fullsized":"2024-04-13T03:17:35.910964+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:12:48.773459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066350+0000","last_change":"2024-04-13T03:17:34.409673+0000","last_active":"2024-04-13T03:17:36.066350+0000","last_peered":"2024-04-13T03:17:36.066350+0000","last_clean":"2024-04-13T03:17:36.066350+0000","last_became_active":"2024-04-13T03:17:34.409382+0000","last_became_peered":"2024-04-13T03:17:34.409382+0000","last_unstale":"2024-04-13T03:17:36.066350+0000","last_undegraded":"2024-04-13T03:17:36.066350+0000","last_fullsized":"2024-04-13T03:17:36.066350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480628+0000","last_change":"2024-04-13T03:17:33.397638+0000","last_active":"2024-04-13T03:17:35.480628+0000","last_peered":"2024-04-13T03:17:35.480628+0000","last_clean":"2024-04-13T03:17:35.480628+0000","last_became_active":"2024-04-13T03:17:33.397347+0000","last_became_peered":"2024-04-13T03:17:33.397347+0000","last_unstale":"2024-04-13T03:17:35.480628+0000","last_undegraded":"2024-04-13T03:17:35.480628+0000","last_fullsized":"2024-04-13T03:17:35.480628+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:20.211899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390092+0000","last_change":"2024-04-13T03:17:34.404927+0000","last_active":"2024-04-13T03:17:35.390092+0000","last_peered":"2024-04-13T03:17:35.390092+0000","last_clean":"2024-04-13T03:17:35.390092+0000","last_became_active":"2024-04-13T03:17:34.404291+0000","last_became_peered":"2024-04-13T03:17:34.404291+0000","last_unstale":"2024-04-13T03:17:35.390092+0000","last_undegraded":"2024-04-13T03:17:35.390092+0000","last_fullsized":"2024-04-13T03:17:35.390092+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:24.854841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389785+0000","last_change":"2024-04-13T03:17:33.399225+0000","last_active":"2024-04-13T03:17:35.389785+0000","last_peered":"2024-04-13T03:17:35.389785+0000","last_clean":"2024-04-13T03:17:35.389785+0000","last_became_active":"2024-04-13T03:17:33.398968+0000","last_became_peered":"2024-04-13T03:17:33.398968+0000","last_unstale":"2024-04-13T03:17:35.389785+0000","last_undegraded":"2024-04-13T03:17:35.389785+0000","last_fullsized":"2024-04-13T03:17:35.389785+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:22.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395912+0000","last_change":"2024-04-13T03:17:34.405422+0000","last_active":"2024-04-13T03:17:35.395912+0000","last_peered":"2024-04-13T03:17:35.395912+0000","last_clean":"2024-04-13T03:17:35.395912+0000","last_became_active":"2024-04-13T03:17:34.403149+0000","last_became_peered":"2024-04-13T03:17:34.403149+0000","last_unstale":"2024-04-13T03:17:35.395912+0000","last_undegraded":"2024-04-13T03:17:35.395912+0000","last_fullsized":"2024-04-13T03:17:35.395912+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:44.808739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065035+0000","last_change":"2024-04-13T03:17:33.395530+0000","last_active":"2024-04-13T03:17:36.065035+0000","last_peered":"2024-04-13T03:17:36.065035+0000","last_clean":"2024-04-13T03:17:36.065035+0000","last_became_active":"2024-04-13T03:17:33.395261+0000","last_became_peered":"2024-04-13T03:17:33.395261+0000","last_unstale":"2024-04-13T03:17:36.065035+0000","last_undegraded":"2024-04-13T03:17:36.065035+0000","last_fullsized":"2024-04-13T03:17:36.065035+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:22.577449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390007+0000","last_change":"2024-04-13T03:17:34.401454+0000","last_active":"2024-04-13T03:17:35.390007+0000","last_peered":"2024-04-13T03:17:35.390007+0000","last_clean":"2024-04-13T03:17:35.390007+0000","last_became_active":"2024-04-13T03:17:34.400733+0000","last_became_peered":"2024-04-13T03:17:34.400733+0000","last_unstale":"2024-04-13T03:17:35.390007+0000","last_undegraded":"2024-04-13T03:17:35.390007+0000","last_fullsized":"2024-04-13T03:17:35.390007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:04:21.818219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065740+0000","last_change":"2024-04-13T03:17:33.399663+0000","last_active":"2024-04-13T03:17:36.065740+0000","last_peered":"2024-04-13T03:17:36.065740+0000","last_clean":"2024-04-13T03:17:36.065740+0000","last_became_active":"2024-04-13T03:17:33.399272+0000","last_became_peered":"2024-04-13T03:17:33.399272+0000","last_unstale":"2024-04-13T03:17:36.065740+0000","last_undegraded":"2024-04-13T03:17:36.065740+0000","last_fullsized":"2024-04-13T03:17:36.065740+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:23.537159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066474+0000","last_change":"2024-04-13T03:17:34.416507+0000","last_active":"2024-04-13T03:17:36.066474+0000","last_peered":"2024-04-13T03:17:36.066474+0000","last_clean":"2024-04-13T03:17:36.066474+0000","last_became_active":"2024-04-13T03:17:34.416182+0000","last_became_peered":"2024-04-13T03:17:34.416182+0000","last_unstale":"2024-04-13T03:17:36.066474+0000","last_undegraded":"2024-04-13T03:17:36.066474+0000","last_fullsized":"2024-04-13T03:17:36.066474+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065410+0000","last_change":"2024-04-13T03:17:33.398398+0000","last_active":"2024-04-13T03:17:36.065410+0000","last_peered":"2024-04-13T03:17:36.065410+0000","last_clean":"2024-04-13T03:17:36.065410+0000","last_became_active":"2024-04-13T03:17:33.398121+0000","last_became_peered":"2024-04-13T03:17:33.398121+0000","last_unstale":"2024-04-13T03:17:36.065410+0000","last_undegraded":"2024-04-13T03:17:36.065410+0000","last_fullsized":"2024-04-13T03:17:36.065410+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:08.566071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910885+0000","last_change":"2024-04-13T03:17:34.399615+0000","last_active":"2024-04-13T03:17:35.910885+0000","last_peered":"2024-04-13T03:17:35.910885+0000","last_clean":"2024-04-13T03:17:35.910885+0000","last_became_active":"2024-04-13T03:17:34.398730+0000","last_became_peered":"2024-04-13T03:17:34.398730+0000","last_unstale":"2024-04-13T03:17:35.910885+0000","last_undegraded":"2024-04-13T03:17:35.910885+0000","last_fullsized":"2024-04-13T03:17:35.910885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395409+0000","last_change":"2024-04-13T03:17:33.400583+0000","last_active":"2024-04-13T03:17:35.395409+0000","last_peered":"2024-04-13T03:17:35.395409+0000","last_clean":"2024-04-13T03:17:35.395409+0000","last_became_active":"2024-04-13T03:17:33.400299+0000","last_became_peered":"2024-04-13T03:17:33.400299+0000","last_unstale":"2024-04-13T03:17:35.395409+0000","last_undegraded":"2024-04-13T03:17:35.395409+0000","last_fullsized":"2024-04-13T03:17:35.395409+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:40:01.687395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395717+0000","last_change":"2024-04-13T03:17:34.402541+0000","last_active":"2024-04-13T03:17:35.395717+0000","last_peered":"2024-04-13T03:17:35.395717+0000","last_clean":"2024-04-13T03:17:35.395717+0000","last_became_active":"2024-04-13T03:17:34.402071+0000","last_became_peered":"2024-04-13T03:17:34.402071+0000","last_unstale":"2024-04-13T03:17:35.395717+0000","last_undegraded":"2024-04-13T03:17:35.395717+0000","last_fullsized":"2024-04-13T03:17:35.395717+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:57:32.774945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390306+0000","last_change":"2024-04-13T03:17:33.401851+0000","last_active":"2024-04-13T03:17:35.390306+0000","last_peered":"2024-04-13T03:17:35.390306+0000","last_clean":"2024-04-13T03:17:35.390306+0000","last_became_active":"2024-04-13T03:17:33.401591+0000","last_became_peered":"2024-04-13T03:17:33.401591+0000","last_unstale":"2024-04-13T03:17:35.390306+0000","last_undegraded":"2024-04-13T03:17:35.390306+0000","last_fullsized":"2024-04-13T03:17:35.390306+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:31.962483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.400737+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.400491+0000","last_became_peered":"2024-04-13T03:17:34.400491+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:43.760714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389789+0000","last_change":"2024-04-13T03:17:33.401061+0000","last_active":"2024-04-13T03:17:35.389789+0000","last_peered":"2024-04-13T03:17:35.389789+0000","last_clean":"2024-04-13T03:17:35.389789+0000","last_became_active":"2024-04-13T03:17:33.400750+0000","last_became_peered":"2024-04-13T03:17:33.400750+0000","last_unstale":"2024-04-13T03:17:35.389789+0000","last_undegraded":"2024-04-13T03:17:35.389789+0000","last_fullsized":"2024-04-13T03:17:35.389789+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:08:22.455340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480880+0000","last_change":"2024-04-13T03:17:34.407766+0000","last_active":"2024-04-13T03:17:35.480880+0000","last_peered":"2024-04-13T03:17:35.480880+0000","last_clean":"2024-04-13T03:17:35.480880+0000","last_became_active":"2024-04-13T03:17:34.407522+0000","last_became_peered":"2024-04-13T03:17:34.407522+0000","last_unstale":"2024-04-13T03:17:35.480880+0000","last_undegraded":"2024-04-13T03:17:35.480880+0000","last_fullsized":"2024-04-13T03:17:35.480880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:58:46.910362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065961+0000","last_change":"2024-04-13T03:17:33.406834+0000","last_active":"2024-04-13T03:17:36.065961+0000","last_peered":"2024-04-13T03:17:36.065961+0000","last_clean":"2024-04-13T03:17:36.065961+0000","last_became_active":"2024-04-13T03:17:33.406592+0000","last_became_peered":"2024-04-13T03:17:33.406592+0000","last_unstale":"2024-04-13T03:17:36.065961+0000","last_undegraded":"2024-04-13T03:17:36.065961+0000","last_fullsized":"2024-04-13T03:17:36.065961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:55:26.429151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390170+0000","last_change":"2024-04-13T03:17:34.411881+0000","last_active":"2024-04-13T03:17:35.390170+0000","last_peered":"2024-04-13T03:17:35.390170+0000","last_clean":"2024-04-13T03:17:35.390170+0000","last_became_active":"2024-04-13T03:17:34.411623+0000","last_became_peered":"2024-04-13T03:17:34.411623+0000","last_unstale":"2024-04-13T03:17:35.390170+0000","last_undegraded":"2024-04-13T03:17:35.390170+0000","last_fullsized":"2024-04-13T03:17:35.390170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:24:28.559176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390052+0000","last_change":"2024-04-13T03:17:33.400409+0000","last_active":"2024-04-13T03:17:35.390052+0000","last_peered":"2024-04-13T03:17:35.390052+0000","last_clean":"2024-04-13T03:17:35.390052+0000","last_became_active":"2024-04-13T03:17:33.400146+0000","last_became_peered":"2024-04-13T03:17:33.400146+0000","last_unstale":"2024-04-13T03:17:35.390052+0000","last_undegraded":"2024-04-13T03:17:35.390052+0000","last_fullsized":"2024-04-13T03:17:35.390052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:51.101678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395449+0000","last_change":"2024-04-13T03:17:34.396324+0000","last_active":"2024-04-13T03:17:35.395449+0000","last_peered":"2024-04-13T03:17:35.395449+0000","last_clean":"2024-04-13T03:17:35.395449+0000","last_became_active":"2024-04-13T03:17:34.395910+0000","last_became_peered":"2024-04-13T03:17:34.395910+0000","last_unstale":"2024-04-13T03:17:35.395449+0000","last_undegraded":"2024-04-13T03:17:35.395449+0000","last_fullsized":"2024-04-13T03:17:35.395449+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:03.411503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910959+0000","last_change":"2024-04-13T03:17:33.399899+0000","last_active":"2024-04-13T03:17:35.910959+0000","last_peered":"2024-04-13T03:17:35.910959+0000","last_clean":"2024-04-13T03:17:35.910959+0000","last_became_active":"2024-04-13T03:17:33.399573+0000","last_became_peered":"2024-04-13T03:17:33.399573+0000","last_unstale":"2024-04-13T03:17:35.910959+0000","last_undegraded":"2024-04-13T03:17:35.910959+0000","last_fullsized":"2024-04-13T03:17:35.910959+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:02:59.848302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066415+0000","last_change":"2024-04-13T03:17:34.392085+0000","last_active":"2024-04-13T03:17:36.066415+0000","last_peered":"2024-04-13T03:17:36.066415+0000","last_clean":"2024-04-13T03:17:36.066415+0000","last_became_active":"2024-04-13T03:17:34.391813+0000","last_became_peered":"2024-04-13T03:17:34.391813+0000","last_unstale":"2024-04-13T03:17:36.066415+0000","last_undegraded":"2024-04-13T03:17:36.066415+0000","last_fullsized":"2024-04-13T03:17:36.066415+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:21.107968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395659+0000","last_change":"2024-04-13T03:17:33.393982+0000","last_active":"2024-04-13T03:17:35.395659+0000","last_peered":"2024-04-13T03:17:35.395659+0000","last_clean":"2024-04-13T03:17:35.395659+0000","last_became_active":"2024-04-13T03:17:33.393615+0000","last_became_peered":"2024-04-13T03:17:33.393615+0000","last_unstale":"2024-04-13T03:17:35.395659+0000","last_undegraded":"2024-04-13T03:17:35.395659+0000","last_fullsized":"2024-04-13T03:17:35.395659+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:35:20.641845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480738+0000","last_change":"2024-04-13T03:17:34.406398+0000","last_active":"2024-04-13T03:17:35.480738+0000","last_peered":"2024-04-13T03:17:35.480738+0000","last_clean":"2024-04-13T03:17:35.480738+0000","last_became_active":"2024-04-13T03:17:34.406118+0000","last_became_peered":"2024-04-13T03:17:34.406118+0000","last_unstale":"2024-04-13T03:17:35.480738+0000","last_undegraded":"2024-04-13T03:17:35.480738+0000","last_fullsized":"2024-04-13T03:17:35.480738+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:49.097723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066129+0000","last_change":"2024-04-13T03:17:33.400700+0000","last_active":"2024-04-13T03:17:36.066129+0000","last_peered":"2024-04-13T03:17:36.066129+0000","last_clean":"2024-04-13T03:17:36.066129+0000","last_became_active":"2024-04-13T03:17:33.400443+0000","last_became_peered":"2024-04-13T03:17:33.400443+0000","last_unstale":"2024-04-13T03:17:36.066129+0000","last_undegraded":"2024-04-13T03:17:36.066129+0000","last_fullsized":"2024-04-13T03:17:36.066129+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:56.254824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395197+0000","last_change":"2024-04-13T03:17:34.416879+0000","last_active":"2024-04-13T03:17:35.395197+0000","last_peered":"2024-04-13T03:17:35.395197+0000","last_clean":"2024-04-13T03:17:35.395197+0000","last_became_active":"2024-04-13T03:17:34.416615+0000","last_became_peered":"2024-04-13T03:17:34.416615+0000","last_unstale":"2024-04-13T03:17:35.395197+0000","last_undegraded":"2024-04-13T03:17:35.395197+0000","last_fullsized":"2024-04-13T03:17:35.395197+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910943+0000","last_change":"2024-04-13T03:17:33.402668+0000","last_active":"2024-04-13T03:17:35.910943+0000","last_peered":"2024-04-13T03:17:35.910943+0000","last_clean":"2024-04-13T03:17:35.910943+0000","last_became_active":"2024-04-13T03:17:33.402363+0000","last_became_peered":"2024-04-13T03:17:33.402363+0000","last_unstale":"2024-04-13T03:17:35.910943+0000","last_undegraded":"2024-04-13T03:17:35.910943+0000","last_fullsized":"2024-04-13T03:17:35.910943+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:45.226601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390104+0000","last_change":"2024-04-13T03:17:34.403801+0000","last_active":"2024-04-13T03:17:35.390104+0000","last_peered":"2024-04-13T03:17:35.390104+0000","last_clean":"2024-04-13T03:17:35.390104+0000","last_became_active":"2024-04-13T03:17:34.403541+0000","last_became_peered":"2024-04-13T03:17:34.403541+0000","last_unstale":"2024-04-13T03:17:35.390104+0000","last_undegraded":"2024-04-13T03:17:35.390104+0000","last_fullsized":"2024-04-13T03:17:35.390104+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:51:28.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065320+0000","last_change":"2024-04-13T03:17:33.401998+0000","last_active":"2024-04-13T03:17:36.065320+0000","last_peered":"2024-04-13T03:17:36.065320+0000","last_clean":"2024-04-13T03:17:36.065320+0000","last_became_active":"2024-04-13T03:17:33.401724+0000","last_became_peered":"2024-04-13T03:17:33.401724+0000","last_unstale":"2024-04-13T03:17:36.065320+0000","last_undegraded":"2024-04-13T03:17:36.065320+0000","last_fullsized":"2024-04-13T03:17:36.065320+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:11.989180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390069+0000","last_change":"2024-04-13T03:17:34.411296+0000","last_active":"2024-04-13T03:17:35.390069+0000","last_peered":"2024-04-13T03:17:35.390069+0000","last_clean":"2024-04-13T03:17:35.390069+0000","last_became_active":"2024-04-13T03:17:34.411035+0000","last_became_peered":"2024-04-13T03:17:34.411035+0000","last_unstale":"2024-04-13T03:17:35.390069+0000","last_undegraded":"2024-04-13T03:17:35.390069+0000","last_fullsized":"2024-04-13T03:17:35.390069+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:47:34.772093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389916+0000","last_change":"2024-04-13T03:17:33.399760+0000","last_active":"2024-04-13T03:17:35.389916+0000","last_peered":"2024-04-13T03:17:35.389916+0000","last_clean":"2024-04-13T03:17:35.389916+0000","last_became_active":"2024-04-13T03:17:33.399508+0000","last_became_peered":"2024-04-13T03:17:33.399508+0000","last_unstale":"2024-04-13T03:17:35.389916+0000","last_undegraded":"2024-04-13T03:17:35.389916+0000","last_fullsized":"2024-04-13T03:17:35.389916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:14.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065695+0000","last_change":"2024-04-13T03:17:34.406246+0000","last_active":"2024-04-13T03:17:36.065695+0000","last_peered":"2024-04-13T03:17:36.065695+0000","last_clean":"2024-04-13T03:17:36.065695+0000","last_became_active":"2024-04-13T03:17:34.405992+0000","last_became_peered":"2024-04-13T03:17:34.405992+0000","last_unstale":"2024-04-13T03:17:36.065695+0000","last_undegraded":"2024-04-13T03:17:36.065695+0000","last_fullsized":"2024-04-13T03:17:36.065695+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066108+0000","last_change":"2024-04-13T03:17:33.400564+0000","last_active":"2024-04-13T03:17:36.066108+0000","last_peered":"2024-04-13T03:17:36.066108+0000","last_clean":"2024-04-13T03:17:36.066108+0000","last_became_active":"2024-04-13T03:17:33.400283+0000","last_became_peered":"2024-04-13T03:17:33.400283+0000","last_unstale":"2024-04-13T03:17:36.066108+0000","last_undegraded":"2024-04-13T03:17:36.066108+0000","last_fullsized":"2024-04-13T03:17:36.066108+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:33.869508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395860+0000","last_change":"2024-04-13T03:17:34.413234+0000","last_active":"2024-04-13T03:17:35.395860+0000","last_peered":"2024-04-13T03:17:35.395860+0000","last_clean":"2024-04-13T03:17:35.395860+0000","last_became_active":"2024-04-13T03:17:34.412967+0000","last_became_peered":"2024-04-13T03:17:34.412967+0000","last_unstale":"2024-04-13T03:17:35.395860+0000","last_undegraded":"2024-04-13T03:17:35.395860+0000","last_fullsized":"2024-04-13T03:17:35.395860+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:13.036448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390152+0000","last_change":"2024-04-13T03:17:33.399229+0000","last_active":"2024-04-13T03:17:35.390152+0000","last_peered":"2024-04-13T03:17:35.390152+0000","last_clean":"2024-04-13T03:17:35.390152+0000","last_became_active":"2024-04-13T03:17:33.398953+0000","last_became_peered":"2024-04-13T03:17:33.398953+0000","last_unstale":"2024-04-13T03:17:35.390152+0000","last_undegraded":"2024-04-13T03:17:35.390152+0000","last_fullsized":"2024-04-13T03:17:35.390152+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:04.844331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395150+0000","last_change":"2024-04-13T03:17:34.410479+0000","last_active":"2024-04-13T03:17:35.395150+0000","last_peered":"2024-04-13T03:17:35.395150+0000","last_clean":"2024-04-13T03:17:35.395150+0000","last_became_active":"2024-04-13T03:17:34.410102+0000","last_became_peered":"2024-04-13T03:17:34.410102+0000","last_unstale":"2024-04-13T03:17:35.395150+0000","last_undegraded":"2024-04-13T03:17:35.395150+0000","last_fullsized":"2024-04-13T03:17:35.395150+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390239+0000","last_change":"2024-04-13T03:17:33.401216+0000","last_active":"2024-04-13T03:17:35.390239+0000","last_peered":"2024-04-13T03:17:35.390239+0000","last_clean":"2024-04-13T03:17:35.390239+0000","last_became_active":"2024-04-13T03:17:33.400900+0000","last_became_peered":"2024-04-13T03:17:33.400900+0000","last_unstale":"2024-04-13T03:17:35.390239+0000","last_undegraded":"2024-04-13T03:17:35.390239+0000","last_fullsized":"2024-04-13T03:17:35.390239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:54:39.812442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":331,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065126+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:17:36.065126+0000","last_peered":"2024-04-13T03:17:36.065126+0000","last_clean":"2024-04-13T03:17:36.065126+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:17:36.065126+0000","last_undegraded":"2024-04-13T03:17:36.065126+0000","last_fullsized":"2024-04-13T03:17:36.065126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":1208712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":620400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":618184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":618276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1210254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:17:38.684 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-13T03:17:38.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.818+0000 7f7bd038c640 1 Processor -- start 2024-04-13T03:17:38.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.818+0000 7f7bd038c640 1 -- start start 2024-04-13T03:17:38.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bd038c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bc80b32d0 0x7f7bc815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bd038c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 0x7f7bc815b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bd038c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc8158c50 con 0x7f7bc8158870 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bd038c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc815ea90 con 0x7f7bc80b32d0 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bd038c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc80b3800 con 0x7f7bc815f460 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd927640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bc80b32d0 0x7f7bc815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bce128640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7bc8158870 0x7f7bc806faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59224/0 (socket says 172.21.15.106:59224) 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bce128640 1 -- 172.21.15.106:0/1836153098 learned_addr learned my addr 172.21.15.106:0/1836153098 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bce929640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 0x7f7bc815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd927640 1 -- 172.21.15.106:0/1836153098 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 msgr2=0x7f7bc815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd126640 1 -- 172.21.15.106:0/1836153098 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930177255 0 0) 0x7f7bc8158c50 con 0x7f7bc8158870 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd927640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 0x7f7bc815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd927640 1 -- 172.21.15.106:0/1836153098 >> v1:172.21.15.106:6789/0 conn(0x7f7bc8158870 legacy=0x7f7bc806faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bcd927640 1 -- 172.21.15.106:0/1836153098 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc80b3ae0 con 0x7f7bc80b32d0 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.819+0000 7f7bce929640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 0x7f7bc815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:38.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.820+0000 7f7bcd927640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ef20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800a940 tx=0x7f7bb802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63997360d06769cd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.820+0000 7f7bcd126640 1 -- 172.21.15.106:0/1836153098 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb803a070 con 0x7f7bc80b32d0 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.820+0000 7f7bcd126640 1 -- 172.21.15.106:0/1836153098 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb802db30 con 0x7f7bc80b32d0 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.820+0000 7f7bcd126640 1 -- 172.21.15.106:0/1836153098 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800b040 con 0x7f7bc80b32d0 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.820+0000 7f7bd038c640 1 -- 172.21.15.106:0/1836153098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 msgr2=0x7f7bc815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ef20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800a940 tx=0x7f7bb802cd80 comp rx=0 tx=0).stop 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 -- 172.21.15.106:0/1836153098 shutdown_connections 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc815f460 0x7f7bc815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 --2- 172.21.15.106:0/1836153098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ef20 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 -- 172.21.15.106:0/1836153098 >> 172.21.15.106:0/1836153098 conn(0x7f7bc80d0340 msgr2=0x7f7bc80d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 -- 172.21.15.106:0/1836153098 shutdown_connections 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.821+0000 7f7bd038c640 1 -- 172.21.15.106:0/1836153098 wait complete. 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 Processor -- start 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 -- start start 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 0x7f7bc815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 0x7f7bc80c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc815b930 con 0x7f7bc80b32d0 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc815b7b0 con 0x7f7bc815f460 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.822+0000 7f7bd038c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc815bab0 con 0x7f7bc8158870 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55138/0 (socket says 172.21.15.106:55138) 2024-04-13T03:17:38.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 -- 172.21.15.106:0/2208718500 learned_addr learned my addr 172.21.15.106:0/2208718500 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce929640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 0x7f7bc80c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bcd927640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 0x7f7bc815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 -- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 msgr2=0x7f7bc815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 0x7f7bc815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 -- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 msgr2=0x7f7bc80c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 0x7f7bc80c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 -- 172.21.15.106:0/2208718500 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc815dae0 con 0x7f7bc80b32d0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce929640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 0x7f7bc80c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bcd927640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 0x7f7bc815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bce128640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c4d0 tx=0x7f7bc400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4018070 con 0x7f7bc80b32d0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4007500 con 0x7f7bc80b32d0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc80ae0a0 con 0x7f7bc80b32d0 2024-04-13T03:17:38.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4004a20 con 0x7f7bc80b32d0 2024-04-13T03:17:38.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.823+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc80ae660 con 0x7f7bc80b32d0 2024-04-13T03:17:38.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.824+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8068a10 con 0x7f7bc80b32d0 2024-04-13T03:17:38.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.825+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bc4004bc0 con 0x7f7bc80b32d0 2024-04-13T03:17:38.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.825+0000 7f7bbeffd640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:38.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.825+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f7bc408b030 con 0x7f7bc80b32d0 2024-04-13T03:17:38.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.825+0000 7f7bcd927640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:38.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.826+0000 7f7bcd927640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8037000 tx=0x7f7bb800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:38.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:38.829+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bc4059250 con 0x7f7bc80b32d0 2024-04-13T03:17:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.073+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7bc806d1d0 con 0x7f7bc80b32d0 2024-04-13T03:17:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.073+0000 7f7bbeffd640 1 -- 172.21.15.106:0/2208718500 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1176 (secure 0 0 0) 0x7f7bc4058b50 con 0x7f7bc80b32d0 2024-04-13T03:17:39.074 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:39.074 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"d7ff0368-2781-4238-b477-50485b08a223","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":518,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":107,"num_osds":8,"num_up_osds":8,"osd_up_since":1712977751,"num_in_osds":8,"osd_in_since":1712977742,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":267304960,"bytes_avail":772826808320,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8430}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-13T03:17:02.464821+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 msgr2=0x7f7bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8037000 tx=0x7f7bb800c040 comp rx=0 tx=0).stop 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 msgr2=0x7f7bc815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c4d0 tx=0x7f7bc400c990 comp rx=0 tx=0).stop 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 shutdown_connections 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bc8158870 0x7f7bc815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc815f460 0x7f7bc80c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 --2- 172.21.15.106:0/2208718500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bc80b32d0 0x7f7bc815ce20 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 >> 172.21.15.106:0/2208718500 conn(0x7f7bc80d0340 msgr2=0x7f7bc8157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 shutdown_connections 2024-04-13T03:17:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.075+0000 7f7bd038c640 1 -- 172.21.15.106:0/2208718500 wait complete. 2024-04-13T03:17:39.087 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-13T03:17:39.087 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:17:39.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.187+0000 7fb904036640 1 Processor -- start 2024-04-13T03:17:39.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.187+0000 7fb904036640 1 -- start start 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb904036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb904036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc15ce90 0x7fb8fc0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb904036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8fc0aefc0 con 0x7fb8fc0b58e0 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb904036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb8fc0b5d50 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb904036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc06e180 con 0x7fb8fc15ce90 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55148/0 (socket says 172.21.15.106:55148) 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 -- 172.21.15.106:0/322301611 learned_addr learned my addr 172.21.15.106:0/322301611 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb9015d1640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc15ce90 0x7fb8fc0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 -- 172.21.15.106:0/322301611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc15ce90 msgr2=0x7fb8fc0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:39.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc15ce90 0x7fb8fc0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 -- 172.21.15.106:0/322301611 >> v1:172.21.15.114:6789/0 conn(0x7fb8fc0b58e0 legacy=0x7fb8fc06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:39.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.188+0000 7fb901dd2640 1 -- 172.21.15.106:0/322301611 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc0ad180 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb901dd2640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009870 tx=0x7fb8f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884f40ddf51cb189 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb900dd0640 1 -- 172.21.15.106:0/322301611 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80317c0 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb900dd0640 1 -- 172.21.15.106:0/322301611 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f8037070 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb900dd0640 1 -- 172.21.15.106:0/322301611 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8031b80 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 -- 172.21.15.106:0/322301611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 msgr2=0x7fb8fc1603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009870 tx=0x7fb8f802b860 comp rx=0 tx=0).stop 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 -- 172.21.15.106:0/322301611 shutdown_connections 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc15ce90 0x7fb8fc0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 --2- 172.21.15.106:0/322301611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1603a0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.189+0000 7fb904036640 1 -- 172.21.15.106:0/322301611 >> 172.21.15.106:0/322301611 conn(0x7fb8fc0b1f10 msgr2=0x7fb8fc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:39.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.190+0000 7fb904036640 1 -- 172.21.15.106:0/322301611 shutdown_connections 2024-04-13T03:17:39.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.190+0000 7fb904036640 1 -- 172.21.15.106:0/322301611 wait complete. 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 Processor -- start 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 -- start start 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 0x7fb8fc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8fc15ce90 0x7fb8fc0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc161260 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc160f80 con 0x7fb8fc15ce90 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb904036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc06de10 con 0x7fb8fc0b58e0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb901dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb901dd2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55154/0 (socket says 172.21.15.106:55154) 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.191+0000 7fb901dd2640 1 -- 172.21.15.106:0/3867111423 learned_addr learned my addr 172.21.15.106:0/3867111423 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb9015d1640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 0x7fb8fc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb9025d3640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8fc15ce90 0x7fb8fc0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 -- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 msgr2=0x7fb8fc0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 0x7fb8fc0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 -- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8fc15ce90 msgr2=0x7fb8fc0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8fc15ce90 0x7fb8fc0cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 -- 172.21.15.106:0/3867111423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc0c75f0 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb9015d1640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 0x7fb8fc0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:39.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb901dd2640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8000c00 tx=0x7fb8f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8032650 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f80327f0 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8032be0 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc0c78d0 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.192+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc0c7e90 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.193+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc068a10 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.193+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8f804c060 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.194+0000 7fb8f27fc640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 0x7fb8e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:39.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.194+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb8f80ad810 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.194+0000 7fb9015d1640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 0x7fb8e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:39.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.194+0000 7fb9015d1640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 0x7fb8e006ede0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:39.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.198+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8f8040070 con 0x7fb8fc0aebe0 2024-04-13T03:17:39.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.370+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8fc0af450 con 0x7fb8e006c920 2024-04-13T03:17:39.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.376+0000 7fb8f27fc640 1 -- 172.21.15.106:0/3867111423 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7fb8fc0af450 con 0x7fb8e006c920 2024-04-13T03:17:39.379 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:39.380 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 msgr2=0x7fb8e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 0x7fb8e006ede0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 msgr2=0x7fb8fc1f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8000c00 tx=0x7fb8f80046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 shutdown_connections 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb8e006c920 0x7fb8e006ede0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8fc0b58e0 0x7fb8fc0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8fc15ce90 0x7fb8fc0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 --2- 172.21.15.106:0/3867111423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8fc0aebe0 0x7fb8fc1f41a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 >> 172.21.15.106:0/3867111423 conn(0x7fb8fc0b1f10 msgr2=0x7fb8fc0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:39.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 shutdown_connections 2024-04-13T03:17:39.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:39.379+0000 7fb904036640 1 -- 172.21.15.106:0/3867111423 wait complete. 2024-04-13T03:17:39.390 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-13T03:17:38.472210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261040,"kb_used_data":11592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713680,"statfs":{"total":773094113280,"available":772826808320,"internally_reserved":0,"allocated":11870208,"data_stored":6131028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001179"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395781+0000","last_change":"2024-04-13T03:17:34.397755+0000","last_active":"2024-04-13T03:17:35.395781+0000","last_peered":"2024-04-13T03:17:35.395781+0000","last_clean":"2024-04-13T03:17:35.395781+0000","last_became_active":"2024-04-13T03:17:34.397499+0000","last_became_peered":"2024-04-13T03:17:34.397499+0000","last_unstale":"2024-04-13T03:17:35.395781+0000","last_undegraded":"2024-04-13T03:17:35.395781+0000","last_fullsized":"2024-04-13T03:17:35.395781+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:52.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065784+0000","last_change":"2024-04-13T03:17:34.414082+0000","last_active":"2024-04-13T03:17:36.065784+0000","last_peered":"2024-04-13T03:17:36.065784+0000","last_clean":"2024-04-13T03:17:36.065784+0000","last_became_active":"2024-04-13T03:17:34.412064+0000","last_became_peered":"2024-04-13T03:17:34.412064+0000","last_unstale":"2024-04-13T03:17:36.065784+0000","last_undegraded":"2024-04-13T03:17:36.065784+0000","last_fullsized":"2024-04-13T03:17:36.065784+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395702+0000","last_change":"2024-04-13T03:17:34.406889+0000","last_active":"2024-04-13T03:17:35.395702+0000","last_peered":"2024-04-13T03:17:35.395702+0000","last_clean":"2024-04-13T03:17:35.395702+0000","last_became_active":"2024-04-13T03:17:34.406253+0000","last_became_peered":"2024-04-13T03:17:34.406253+0000","last_unstale":"2024-04-13T03:17:35.395702+0000","last_undegraded":"2024-04-13T03:17:35.395702+0000","last_fullsized":"2024-04-13T03:17:35.395702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:00:22.906340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480782+0000","last_change":"2024-04-13T03:17:34.396984+0000","last_active":"2024-04-13T03:17:35.480782+0000","last_peered":"2024-04-13T03:17:35.480782+0000","last_clean":"2024-04-13T03:17:35.480782+0000","last_became_active":"2024-04-13T03:17:34.396599+0000","last_became_peered":"2024-04-13T03:17:34.396599+0000","last_unstale":"2024-04-13T03:17:35.480782+0000","last_undegraded":"2024-04-13T03:17:35.480782+0000","last_fullsized":"2024-04-13T03:17:35.480782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:05:44.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910724+0000","last_change":"2024-04-13T03:17:34.399389+0000","last_active":"2024-04-13T03:17:35.910724+0000","last_peered":"2024-04-13T03:17:35.910724+0000","last_clean":"2024-04-13T03:17:35.910724+0000","last_became_active":"2024-04-13T03:17:34.399010+0000","last_became_peered":"2024-04-13T03:17:34.399010+0000","last_unstale":"2024-04-13T03:17:35.910724+0000","last_undegraded":"2024-04-13T03:17:35.910724+0000","last_fullsized":"2024-04-13T03:17:35.910724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066413+0000","last_change":"2024-04-13T03:17:34.412752+0000","last_active":"2024-04-13T03:17:36.066413+0000","last_peered":"2024-04-13T03:17:36.066413+0000","last_clean":"2024-04-13T03:17:36.066413+0000","last_became_active":"2024-04-13T03:17:34.412470+0000","last_became_peered":"2024-04-13T03:17:34.412470+0000","last_unstale":"2024-04-13T03:17:36.066413+0000","last_undegraded":"2024-04-13T03:17:36.066413+0000","last_fullsized":"2024-04-13T03:17:36.066413+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065869+0000","last_change":"2024-04-13T03:17:34.411655+0000","last_active":"2024-04-13T03:17:36.065869+0000","last_peered":"2024-04-13T03:17:36.065869+0000","last_clean":"2024-04-13T03:17:36.065869+0000","last_became_active":"2024-04-13T03:17:34.411211+0000","last_became_peered":"2024-04-13T03:17:34.411211+0000","last_unstale":"2024-04-13T03:17:36.065869+0000","last_undegraded":"2024-04-13T03:17:36.065869+0000","last_fullsized":"2024-04-13T03:17:36.065869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389760+0000","last_change":"2024-04-13T03:17:34.402872+0000","last_active":"2024-04-13T03:17:35.389760+0000","last_peered":"2024-04-13T03:17:35.389760+0000","last_clean":"2024-04-13T03:17:35.389760+0000","last_became_active":"2024-04-13T03:17:34.401106+0000","last_became_peered":"2024-04-13T03:17:34.401106+0000","last_unstale":"2024-04-13T03:17:35.389760+0000","last_undegraded":"2024-04-13T03:17:35.389760+0000","last_fullsized":"2024-04-13T03:17:35.389760+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:15:37.328883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065661+0000","last_change":"2024-04-13T03:17:34.410456+0000","last_active":"2024-04-13T03:17:36.065661+0000","last_peered":"2024-04-13T03:17:36.065661+0000","last_clean":"2024-04-13T03:17:36.065661+0000","last_became_active":"2024-04-13T03:17:34.410166+0000","last_became_peered":"2024-04-13T03:17:34.410166+0000","last_unstale":"2024-04-13T03:17:36.065661+0000","last_undegraded":"2024-04-13T03:17:36.065661+0000","last_fullsized":"2024-04-13T03:17:36.065661+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:35.141523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.911023+0000","last_change":"2024-04-13T03:17:34.410800+0000","last_active":"2024-04-13T03:17:35.911023+0000","last_peered":"2024-04-13T03:17:35.911023+0000","last_clean":"2024-04-13T03:17:35.911023+0000","last_became_active":"2024-04-13T03:17:34.410568+0000","last_became_peered":"2024-04-13T03:17:34.410568+0000","last_unstale":"2024-04-13T03:17:35.911023+0000","last_undegraded":"2024-04-13T03:17:35.911023+0000","last_fullsized":"2024-04-13T03:17:35.911023+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395658+0000","last_change":"2024-04-13T03:17:34.407143+0000","last_active":"2024-04-13T03:17:35.395658+0000","last_peered":"2024-04-13T03:17:35.395658+0000","last_clean":"2024-04-13T03:17:35.395658+0000","last_became_active":"2024-04-13T03:17:34.406821+0000","last_became_peered":"2024-04-13T03:17:34.406821+0000","last_unstale":"2024-04-13T03:17:35.395658+0000","last_undegraded":"2024-04-13T03:17:35.395658+0000","last_fullsized":"2024-04-13T03:17:35.395658+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:15:20.135164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066224+0000","last_change":"2024-04-13T03:17:34.392870+0000","last_active":"2024-04-13T03:17:36.066224+0000","last_peered":"2024-04-13T03:17:36.066224+0000","last_clean":"2024-04-13T03:17:36.066224+0000","last_became_active":"2024-04-13T03:17:34.392598+0000","last_became_peered":"2024-04-13T03:17:34.392598+0000","last_unstale":"2024-04-13T03:17:36.066224+0000","last_undegraded":"2024-04-13T03:17:36.066224+0000","last_fullsized":"2024-04-13T03:17:36.066224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:08.590896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480726+0000","last_change":"2024-04-13T03:17:34.406086+0000","last_active":"2024-04-13T03:17:35.480726+0000","last_peered":"2024-04-13T03:17:35.480726+0000","last_clean":"2024-04-13T03:17:35.480726+0000","last_became_active":"2024-04-13T03:17:34.405810+0000","last_became_peered":"2024-04-13T03:17:34.405810+0000","last_unstale":"2024-04-13T03:17:35.480726+0000","last_undegraded":"2024-04-13T03:17:35.480726+0000","last_fullsized":"2024-04-13T03:17:35.480726+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:02.642224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065362+0000","last_change":"2024-04-13T03:17:34.400371+0000","last_active":"2024-04-13T03:17:36.065362+0000","last_peered":"2024-04-13T03:17:36.065362+0000","last_clean":"2024-04-13T03:17:36.065362+0000","last_became_active":"2024-04-13T03:17:34.399998+0000","last_became_peered":"2024-04-13T03:17:34.399998+0000","last_unstale":"2024-04-13T03:17:36.065362+0000","last_undegraded":"2024-04-13T03:17:36.065362+0000","last_fullsized":"2024-04-13T03:17:36.065362+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065088+0000","last_change":"2024-04-13T03:17:34.407130+0000","last_active":"2024-04-13T03:17:36.065088+0000","last_peered":"2024-04-13T03:17:36.065088+0000","last_clean":"2024-04-13T03:17:36.065088+0000","last_became_active":"2024-04-13T03:17:34.406854+0000","last_became_peered":"2024-04-13T03:17:34.406854+0000","last_unstale":"2024-04-13T03:17:36.065088+0000","last_undegraded":"2024-04-13T03:17:36.065088+0000","last_fullsized":"2024-04-13T03:17:36.065088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065064+0000","last_change":"2024-04-13T03:17:34.401481+0000","last_active":"2024-04-13T03:17:36.065064+0000","last_peered":"2024-04-13T03:17:36.065064+0000","last_clean":"2024-04-13T03:17:36.065064+0000","last_became_active":"2024-04-13T03:17:34.401209+0000","last_became_peered":"2024-04-13T03:17:34.401209+0000","last_unstale":"2024-04-13T03:17:36.065064+0000","last_undegraded":"2024-04-13T03:17:36.065064+0000","last_fullsized":"2024-04-13T03:17:36.065064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:21.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395885+0000","last_change":"2024-04-13T03:17:34.405442+0000","last_active":"2024-04-13T03:17:35.395885+0000","last_peered":"2024-04-13T03:17:35.395885+0000","last_clean":"2024-04-13T03:17:35.395885+0000","last_became_active":"2024-04-13T03:17:34.404800+0000","last_became_peered":"2024-04-13T03:17:34.404800+0000","last_unstale":"2024-04-13T03:17:35.395885+0000","last_undegraded":"2024-04-13T03:17:35.395885+0000","last_fullsized":"2024-04-13T03:17:35.395885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:50:42.464825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065972+0000","last_change":"2024-04-13T03:17:34.410814+0000","last_active":"2024-04-13T03:17:36.065972+0000","last_peered":"2024-04-13T03:17:36.065972+0000","last_clean":"2024-04-13T03:17:36.065972+0000","last_became_active":"2024-04-13T03:17:34.410551+0000","last_became_peered":"2024-04-13T03:17:34.410551+0000","last_unstale":"2024-04-13T03:17:36.065972+0000","last_undegraded":"2024-04-13T03:17:36.065972+0000","last_fullsized":"2024-04-13T03:17:36.065972+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395709+0000","last_change":"2024-04-13T03:17:34.408576+0000","last_active":"2024-04-13T03:17:35.395709+0000","last_peered":"2024-04-13T03:17:35.395709+0000","last_clean":"2024-04-13T03:17:35.395709+0000","last_became_active":"2024-04-13T03:17:34.408330+0000","last_became_peered":"2024-04-13T03:17:34.408330+0000","last_unstale":"2024-04-13T03:17:35.395709+0000","last_undegraded":"2024-04-13T03:17:35.395709+0000","last_fullsized":"2024-04-13T03:17:35.395709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:15.555032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066073+0000","last_change":"2024-04-13T03:17:34.398173+0000","last_active":"2024-04-13T03:17:36.066073+0000","last_peered":"2024-04-13T03:17:36.066073+0000","last_clean":"2024-04-13T03:17:36.066073+0000","last_became_active":"2024-04-13T03:17:34.397874+0000","last_became_peered":"2024-04-13T03:17:34.397874+0000","last_unstale":"2024-04-13T03:17:36.066073+0000","last_undegraded":"2024-04-13T03:17:36.066073+0000","last_fullsized":"2024-04-13T03:17:36.066073+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390283+0000","last_change":"2024-04-13T03:17:34.411955+0000","last_active":"2024-04-13T03:17:35.390283+0000","last_peered":"2024-04-13T03:17:35.390283+0000","last_clean":"2024-04-13T03:17:35.390283+0000","last_became_active":"2024-04-13T03:17:34.411568+0000","last_became_peered":"2024-04-13T03:17:34.411568+0000","last_unstale":"2024-04-13T03:17:35.390283+0000","last_undegraded":"2024-04-13T03:17:35.390283+0000","last_fullsized":"2024-04-13T03:17:35.390283+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:28:52.418315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910831+0000","last_change":"2024-04-13T03:17:34.410118+0000","last_active":"2024-04-13T03:17:35.910831+0000","last_peered":"2024-04-13T03:17:35.910831+0000","last_clean":"2024-04-13T03:17:35.910831+0000","last_became_active":"2024-04-13T03:17:34.409793+0000","last_became_peered":"2024-04-13T03:17:34.409793+0000","last_unstale":"2024-04-13T03:17:35.910831+0000","last_undegraded":"2024-04-13T03:17:35.910831+0000","last_fullsized":"2024-04-13T03:17:35.910831+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395535+0000","last_change":"2024-04-13T03:17:34.402676+0000","last_active":"2024-04-13T03:17:35.395535+0000","last_peered":"2024-04-13T03:17:35.395535+0000","last_clean":"2024-04-13T03:17:35.395535+0000","last_became_active":"2024-04-13T03:17:34.402403+0000","last_became_peered":"2024-04-13T03:17:34.402403+0000","last_unstale":"2024-04-13T03:17:35.395535+0000","last_undegraded":"2024-04-13T03:17:35.395535+0000","last_fullsized":"2024-04-13T03:17:35.395535+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:27.486850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910608+0000","last_change":"2024-04-13T03:17:34.409245+0000","last_active":"2024-04-13T03:17:35.910608+0000","last_peered":"2024-04-13T03:17:35.910608+0000","last_clean":"2024-04-13T03:17:35.910608+0000","last_became_active":"2024-04-13T03:17:34.408969+0000","last_became_peered":"2024-04-13T03:17:34.408969+0000","last_unstale":"2024-04-13T03:17:35.910608+0000","last_undegraded":"2024-04-13T03:17:35.910608+0000","last_fullsized":"2024-04-13T03:17:35.910608+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065690+0000","last_change":"2024-04-13T03:17:34.405874+0000","last_active":"2024-04-13T03:17:36.065690+0000","last_peered":"2024-04-13T03:17:36.065690+0000","last_clean":"2024-04-13T03:17:36.065690+0000","last_became_active":"2024-04-13T03:17:34.405601+0000","last_became_peered":"2024-04-13T03:17:34.405601+0000","last_unstale":"2024-04-13T03:17:36.065690+0000","last_undegraded":"2024-04-13T03:17:36.065690+0000","last_fullsized":"2024-04-13T03:17:36.065690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065465+0000","last_change":"2024-04-13T03:17:34.407016+0000","last_active":"2024-04-13T03:17:36.065465+0000","last_peered":"2024-04-13T03:17:36.065465+0000","last_clean":"2024-04-13T03:17:36.065465+0000","last_became_active":"2024-04-13T03:17:34.406642+0000","last_became_peered":"2024-04-13T03:17:34.406642+0000","last_unstale":"2024-04-13T03:17:36.065465+0000","last_undegraded":"2024-04-13T03:17:36.065465+0000","last_fullsized":"2024-04-13T03:17:36.065465+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395683+0000","last_change":"2024-04-13T03:17:34.412860+0000","last_active":"2024-04-13T03:17:35.395683+0000","last_peered":"2024-04-13T03:17:35.395683+0000","last_clean":"2024-04-13T03:17:35.395683+0000","last_became_active":"2024-04-13T03:17:34.411893+0000","last_became_peered":"2024-04-13T03:17:34.411893+0000","last_unstale":"2024-04-13T03:17:35.395683+0000","last_undegraded":"2024-04-13T03:17:35.395683+0000","last_fullsized":"2024-04-13T03:17:35.395683+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:45:20.716257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065970+0000","last_change":"2024-04-13T03:17:34.414317+0000","last_active":"2024-04-13T03:17:36.065970+0000","last_peered":"2024-04-13T03:17:36.065970+0000","last_clean":"2024-04-13T03:17:36.065970+0000","last_became_active":"2024-04-13T03:17:34.414038+0000","last_became_peered":"2024-04-13T03:17:34.414038+0000","last_unstale":"2024-04-13T03:17:36.065970+0000","last_undegraded":"2024-04-13T03:17:36.065970+0000","last_fullsized":"2024-04-13T03:17:36.065970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390278+0000","last_change":"2024-04-13T03:17:34.403227+0000","last_active":"2024-04-13T03:17:35.390278+0000","last_peered":"2024-04-13T03:17:35.390278+0000","last_clean":"2024-04-13T03:17:35.390278+0000","last_became_active":"2024-04-13T03:17:34.402961+0000","last_became_peered":"2024-04-13T03:17:34.402961+0000","last_unstale":"2024-04-13T03:17:35.390278+0000","last_undegraded":"2024-04-13T03:17:35.390278+0000","last_fullsized":"2024-04-13T03:17:35.390278+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:53:56.840439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066167+0000","last_change":"2024-04-13T03:17:34.405766+0000","last_active":"2024-04-13T03:17:36.066167+0000","last_peered":"2024-04-13T03:17:36.066167+0000","last_clean":"2024-04-13T03:17:36.066167+0000","last_became_active":"2024-04-13T03:17:34.404793+0000","last_became_peered":"2024-04-13T03:17:34.404793+0000","last_unstale":"2024-04-13T03:17:36.066167+0000","last_undegraded":"2024-04-13T03:17:36.066167+0000","last_fullsized":"2024-04-13T03:17:36.066167+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:13:26.382175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.416600+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.416208+0000","last_became_peered":"2024-04-13T03:17:34.416208+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389814+0000","last_change":"2024-04-13T03:17:34.402382+0000","last_active":"2024-04-13T03:17:35.389814+0000","last_peered":"2024-04-13T03:17:35.389814+0000","last_clean":"2024-04-13T03:17:35.389814+0000","last_became_active":"2024-04-13T03:17:34.401181+0000","last_became_peered":"2024-04-13T03:17:34.401181+0000","last_unstale":"2024-04-13T03:17:35.389814+0000","last_undegraded":"2024-04-13T03:17:35.389814+0000","last_fullsized":"2024-04-13T03:17:35.389814+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:43.678954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395636+0000","last_change":"2024-04-13T03:17:34.408372+0000","last_active":"2024-04-13T03:17:35.395636+0000","last_peered":"2024-04-13T03:17:35.395636+0000","last_clean":"2024-04-13T03:17:35.395636+0000","last_became_active":"2024-04-13T03:17:34.408097+0000","last_became_peered":"2024-04-13T03:17:34.408097+0000","last_unstale":"2024-04-13T03:17:35.395636+0000","last_undegraded":"2024-04-13T03:17:35.395636+0000","last_fullsized":"2024-04-13T03:17:35.395636+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:14:09.892547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480883+0000","last_change":"2024-04-13T03:17:33.402709+0000","last_active":"2024-04-13T03:17:35.480883+0000","last_peered":"2024-04-13T03:17:35.480883+0000","last_clean":"2024-04-13T03:17:35.480883+0000","last_became_active":"2024-04-13T03:17:33.402409+0000","last_became_peered":"2024-04-13T03:17:33.402409+0000","last_unstale":"2024-04-13T03:17:35.480883+0000","last_undegraded":"2024-04-13T03:17:35.480883+0000","last_fullsized":"2024-04-13T03:17:35.480883+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:06.088410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389771+0000","last_change":"2024-04-13T03:17:34.403177+0000","last_active":"2024-04-13T03:17:35.389771+0000","last_peered":"2024-04-13T03:17:35.389771+0000","last_clean":"2024-04-13T03:17:35.389771+0000","last_became_active":"2024-04-13T03:17:34.402894+0000","last_became_peered":"2024-04-13T03:17:34.402894+0000","last_unstale":"2024-04-13T03:17:35.389771+0000","last_undegraded":"2024-04-13T03:17:35.389771+0000","last_fullsized":"2024-04-13T03:17:35.389771+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:34.744333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390131+0000","last_change":"2024-04-13T03:17:33.401391+0000","last_active":"2024-04-13T03:17:35.390131+0000","last_peered":"2024-04-13T03:17:35.390131+0000","last_clean":"2024-04-13T03:17:35.390131+0000","last_became_active":"2024-04-13T03:17:33.401115+0000","last_became_peered":"2024-04-13T03:17:33.401115+0000","last_unstale":"2024-04-13T03:17:35.390131+0000","last_undegraded":"2024-04-13T03:17:35.390131+0000","last_fullsized":"2024-04-13T03:17:35.390131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:34.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066244+0000","last_change":"2024-04-13T03:17:34.401443+0000","last_active":"2024-04-13T03:17:36.066244+0000","last_peered":"2024-04-13T03:17:36.066244+0000","last_clean":"2024-04-13T03:17:36.066244+0000","last_became_active":"2024-04-13T03:17:34.401088+0000","last_became_peered":"2024-04-13T03:17:34.401088+0000","last_unstale":"2024-04-13T03:17:36.066244+0000","last_undegraded":"2024-04-13T03:17:36.066244+0000","last_fullsized":"2024-04-13T03:17:36.066244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:05:39.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065122+0000","last_change":"2024-04-13T03:17:33.396742+0000","last_active":"2024-04-13T03:17:36.065122+0000","last_peered":"2024-04-13T03:17:36.065122+0000","last_clean":"2024-04-13T03:17:36.065122+0000","last_became_active":"2024-04-13T03:17:33.396322+0000","last_became_peered":"2024-04-13T03:17:33.396322+0000","last_unstale":"2024-04-13T03:17:36.065122+0000","last_undegraded":"2024-04-13T03:17:36.065122+0000","last_fullsized":"2024-04-13T03:17:36.065122+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:13.384064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395276+0000","last_change":"2024-04-13T03:17:34.404596+0000","last_active":"2024-04-13T03:17:35.395276+0000","last_peered":"2024-04-13T03:17:35.395276+0000","last_clean":"2024-04-13T03:17:35.395276+0000","last_became_active":"2024-04-13T03:17:34.403699+0000","last_became_peered":"2024-04-13T03:17:34.403699+0000","last_unstale":"2024-04-13T03:17:35.395276+0000","last_undegraded":"2024-04-13T03:17:35.395276+0000","last_fullsized":"2024-04-13T03:17:35.395276+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910957+0000","last_change":"2024-04-13T03:17:33.398782+0000","last_active":"2024-04-13T03:17:35.910957+0000","last_peered":"2024-04-13T03:17:35.910957+0000","last_clean":"2024-04-13T03:17:35.910957+0000","last_became_active":"2024-04-13T03:17:33.398493+0000","last_became_peered":"2024-04-13T03:17:33.398493+0000","last_unstale":"2024-04-13T03:17:35.910957+0000","last_undegraded":"2024-04-13T03:17:35.910957+0000","last_fullsized":"2024-04-13T03:17:35.910957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:06:27.097899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480837+0000","last_change":"2024-04-13T03:17:34.400995+0000","last_active":"2024-04-13T03:17:35.480837+0000","last_peered":"2024-04-13T03:17:35.480837+0000","last_clean":"2024-04-13T03:17:35.480837+0000","last_became_active":"2024-04-13T03:17:34.400701+0000","last_became_peered":"2024-04-13T03:17:34.400701+0000","last_unstale":"2024-04-13T03:17:35.480837+0000","last_undegraded":"2024-04-13T03:17:35.480837+0000","last_fullsized":"2024-04-13T03:17:35.480837+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:16:23.670640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065289+0000","last_change":"2024-04-13T03:17:33.401599+0000","last_active":"2024-04-13T03:17:36.065289+0000","last_peered":"2024-04-13T03:17:36.065289+0000","last_clean":"2024-04-13T03:17:36.065289+0000","last_became_active":"2024-04-13T03:17:33.401334+0000","last_became_peered":"2024-04-13T03:17:33.401334+0000","last_unstale":"2024-04-13T03:17:36.065289+0000","last_undegraded":"2024-04-13T03:17:36.065289+0000","last_fullsized":"2024-04-13T03:17:36.065289+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:37.518351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.408972+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.408008+0000","last_became_peered":"2024-04-13T03:17:34.408008+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:44:31.275566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395064+0000","last_change":"2024-04-13T03:17:33.400110+0000","last_active":"2024-04-13T03:17:35.395064+0000","last_peered":"2024-04-13T03:17:35.395064+0000","last_clean":"2024-04-13T03:17:35.395064+0000","last_became_active":"2024-04-13T03:17:33.399844+0000","last_became_peered":"2024-04-13T03:17:33.399844+0000","last_unstale":"2024-04-13T03:17:35.395064+0000","last_undegraded":"2024-04-13T03:17:35.395064+0000","last_fullsized":"2024-04-13T03:17:35.395064+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:15:33.561822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.414775+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.414524+0000","last_became_peered":"2024-04-13T03:17:34.414524+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066358+0000","last_change":"2024-04-13T03:17:33.396584+0000","last_active":"2024-04-13T03:17:36.066358+0000","last_peered":"2024-04-13T03:17:36.066358+0000","last_clean":"2024-04-13T03:17:36.066358+0000","last_became_active":"2024-04-13T03:17:33.395985+0000","last_became_peered":"2024-04-13T03:17:33.395985+0000","last_unstale":"2024-04-13T03:17:36.066358+0000","last_undegraded":"2024-04-13T03:17:36.066358+0000","last_fullsized":"2024-04-13T03:17:36.066358+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:05.909520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480922+0000","last_change":"2024-04-13T03:17:34.401457+0000","last_active":"2024-04-13T03:17:35.480922+0000","last_peered":"2024-04-13T03:17:35.480922+0000","last_clean":"2024-04-13T03:17:35.480922+0000","last_became_active":"2024-04-13T03:17:34.401101+0000","last_became_peered":"2024-04-13T03:17:34.401101+0000","last_unstale":"2024-04-13T03:17:35.480922+0000","last_undegraded":"2024-04-13T03:17:35.480922+0000","last_fullsized":"2024-04-13T03:17:35.480922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:03:45.552674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480567+0000","last_change":"2024-04-13T03:17:33.391995+0000","last_active":"2024-04-13T03:17:35.480567+0000","last_peered":"2024-04-13T03:17:35.480567+0000","last_clean":"2024-04-13T03:17:35.480567+0000","last_became_active":"2024-04-13T03:17:33.391716+0000","last_became_peered":"2024-04-13T03:17:33.391716+0000","last_unstale":"2024-04-13T03:17:35.480567+0000","last_undegraded":"2024-04-13T03:17:35.480567+0000","last_fullsized":"2024-04-13T03:17:35.480567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:01.494353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395497+0000","last_change":"2024-04-13T03:17:34.406950+0000","last_active":"2024-04-13T03:17:35.395497+0000","last_peered":"2024-04-13T03:17:35.395497+0000","last_clean":"2024-04-13T03:17:35.395497+0000","last_became_active":"2024-04-13T03:17:34.406487+0000","last_became_peered":"2024-04-13T03:17:34.406487+0000","last_unstale":"2024-04-13T03:17:35.395497+0000","last_undegraded":"2024-04-13T03:17:35.395497+0000","last_fullsized":"2024-04-13T03:17:35.395497+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:35.175581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065815+0000","last_change":"2024-04-13T03:17:33.397241+0000","last_active":"2024-04-13T03:17:36.065815+0000","last_peered":"2024-04-13T03:17:36.065815+0000","last_clean":"2024-04-13T03:17:36.065815+0000","last_became_active":"2024-04-13T03:17:33.396811+0000","last_became_peered":"2024-04-13T03:17:33.396811+0000","last_unstale":"2024-04-13T03:17:36.065815+0000","last_undegraded":"2024-04-13T03:17:36.065815+0000","last_fullsized":"2024-04-13T03:17:36.065815+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:14.055121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910747+0000","last_change":"2024-04-13T03:17:34.404666+0000","last_active":"2024-04-13T03:17:35.910747+0000","last_peered":"2024-04-13T03:17:35.910747+0000","last_clean":"2024-04-13T03:17:35.910747+0000","last_became_active":"2024-04-13T03:17:34.404419+0000","last_became_peered":"2024-04-13T03:17:34.404419+0000","last_unstale":"2024-04-13T03:17:35.910747+0000","last_undegraded":"2024-04-13T03:17:35.910747+0000","last_fullsized":"2024-04-13T03:17:35.910747+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395462+0000","last_change":"2024-04-13T03:17:33.398984+0000","last_active":"2024-04-13T03:17:35.395462+0000","last_peered":"2024-04-13T03:17:35.395462+0000","last_clean":"2024-04-13T03:17:35.395462+0000","last_became_active":"2024-04-13T03:17:33.398683+0000","last_became_peered":"2024-04-13T03:17:33.398683+0000","last_unstale":"2024-04-13T03:17:35.395462+0000","last_undegraded":"2024-04-13T03:17:35.395462+0000","last_fullsized":"2024-04-13T03:17:35.395462+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:19:37.874955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390013+0000","last_change":"2024-04-13T03:17:34.404347+0000","last_active":"2024-04-13T03:17:35.390013+0000","last_peered":"2024-04-13T03:17:35.390013+0000","last_clean":"2024-04-13T03:17:35.390013+0000","last_became_active":"2024-04-13T03:17:34.404070+0000","last_became_peered":"2024-04-13T03:17:34.404070+0000","last_unstale":"2024-04-13T03:17:35.390013+0000","last_undegraded":"2024-04-13T03:17:35.390013+0000","last_fullsized":"2024-04-13T03:17:35.390013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:27:04.357744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065277+0000","last_change":"2024-04-13T03:17:33.395592+0000","last_active":"2024-04-13T03:17:36.065277+0000","last_peered":"2024-04-13T03:17:36.065277+0000","last_clean":"2024-04-13T03:17:36.065277+0000","last_became_active":"2024-04-13T03:17:33.395290+0000","last_became_peered":"2024-04-13T03:17:33.395290+0000","last_unstale":"2024-04-13T03:17:36.065277+0000","last_undegraded":"2024-04-13T03:17:36.065277+0000","last_fullsized":"2024-04-13T03:17:36.065277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:01.476888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910892+0000","last_change":"2024-04-13T03:17:34.406520+0000","last_active":"2024-04-13T03:17:35.910892+0000","last_peered":"2024-04-13T03:17:35.910892+0000","last_clean":"2024-04-13T03:17:35.910892+0000","last_became_active":"2024-04-13T03:17:34.406214+0000","last_became_peered":"2024-04-13T03:17:34.406214+0000","last_unstale":"2024-04-13T03:17:35.910892+0000","last_undegraded":"2024-04-13T03:17:35.910892+0000","last_fullsized":"2024-04-13T03:17:35.910892+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066117+0000","last_change":"2024-04-13T03:17:33.401643+0000","last_active":"2024-04-13T03:17:36.066117+0000","last_peered":"2024-04-13T03:17:36.066117+0000","last_clean":"2024-04-13T03:17:36.066117+0000","last_became_active":"2024-04-13T03:17:33.401302+0000","last_became_peered":"2024-04-13T03:17:33.401302+0000","last_unstale":"2024-04-13T03:17:36.066117+0000","last_undegraded":"2024-04-13T03:17:36.066117+0000","last_fullsized":"2024-04-13T03:17:36.066117+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:14.155305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395482+0000","last_change":"2024-04-13T03:17:34.410826+0000","last_active":"2024-04-13T03:17:35.395482+0000","last_peered":"2024-04-13T03:17:35.395482+0000","last_clean":"2024-04-13T03:17:35.395482+0000","last_became_active":"2024-04-13T03:17:34.410554+0000","last_became_peered":"2024-04-13T03:17:34.410554+0000","last_unstale":"2024-04-13T03:17:35.395482+0000","last_undegraded":"2024-04-13T03:17:35.395482+0000","last_fullsized":"2024-04-13T03:17:35.395482+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:53:21.523097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910964+0000","last_change":"2024-04-13T03:17:33.399676+0000","last_active":"2024-04-13T03:17:35.910964+0000","last_peered":"2024-04-13T03:17:35.910964+0000","last_clean":"2024-04-13T03:17:35.910964+0000","last_became_active":"2024-04-13T03:17:33.399363+0000","last_became_peered":"2024-04-13T03:17:33.399363+0000","last_unstale":"2024-04-13T03:17:35.910964+0000","last_undegraded":"2024-04-13T03:17:35.910964+0000","last_fullsized":"2024-04-13T03:17:35.910964+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:12:48.773459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066350+0000","last_change":"2024-04-13T03:17:34.409673+0000","last_active":"2024-04-13T03:17:36.066350+0000","last_peered":"2024-04-13T03:17:36.066350+0000","last_clean":"2024-04-13T03:17:36.066350+0000","last_became_active":"2024-04-13T03:17:34.409382+0000","last_became_peered":"2024-04-13T03:17:34.409382+0000","last_unstale":"2024-04-13T03:17:36.066350+0000","last_undegraded":"2024-04-13T03:17:36.066350+0000","last_fullsized":"2024-04-13T03:17:36.066350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480628+0000","last_change":"2024-04-13T03:17:33.397638+0000","last_active":"2024-04-13T03:17:35.480628+0000","last_peered":"2024-04-13T03:17:35.480628+0000","last_clean":"2024-04-13T03:17:35.480628+0000","last_became_active":"2024-04-13T03:17:33.397347+0000","last_became_peered":"2024-04-13T03:17:33.397347+0000","last_unstale":"2024-04-13T03:17:35.480628+0000","last_undegraded":"2024-04-13T03:17:35.480628+0000","last_fullsized":"2024-04-13T03:17:35.480628+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:20.211899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390092+0000","last_change":"2024-04-13T03:17:34.404927+0000","last_active":"2024-04-13T03:17:35.390092+0000","last_peered":"2024-04-13T03:17:35.390092+0000","last_clean":"2024-04-13T03:17:35.390092+0000","last_became_active":"2024-04-13T03:17:34.404291+0000","last_became_peered":"2024-04-13T03:17:34.404291+0000","last_unstale":"2024-04-13T03:17:35.390092+0000","last_undegraded":"2024-04-13T03:17:35.390092+0000","last_fullsized":"2024-04-13T03:17:35.390092+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:24.854841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389785+0000","last_change":"2024-04-13T03:17:33.399225+0000","last_active":"2024-04-13T03:17:35.389785+0000","last_peered":"2024-04-13T03:17:35.389785+0000","last_clean":"2024-04-13T03:17:35.389785+0000","last_became_active":"2024-04-13T03:17:33.398968+0000","last_became_peered":"2024-04-13T03:17:33.398968+0000","last_unstale":"2024-04-13T03:17:35.389785+0000","last_undegraded":"2024-04-13T03:17:35.389785+0000","last_fullsized":"2024-04-13T03:17:35.389785+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:22.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395912+0000","last_change":"2024-04-13T03:17:34.405422+0000","last_active":"2024-04-13T03:17:35.395912+0000","last_peered":"2024-04-13T03:17:35.395912+0000","last_clean":"2024-04-13T03:17:35.395912+0000","last_became_active":"2024-04-13T03:17:34.403149+0000","last_became_peered":"2024-04-13T03:17:34.403149+0000","last_unstale":"2024-04-13T03:17:35.395912+0000","last_undegraded":"2024-04-13T03:17:35.395912+0000","last_fullsized":"2024-04-13T03:17:35.395912+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:44.808739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065035+0000","last_change":"2024-04-13T03:17:33.395530+0000","last_active":"2024-04-13T03:17:36.065035+0000","last_peered":"2024-04-13T03:17:36.065035+0000","last_clean":"2024-04-13T03:17:36.065035+0000","last_became_active":"2024-04-13T03:17:33.395261+0000","last_became_peered":"2024-04-13T03:17:33.395261+0000","last_unstale":"2024-04-13T03:17:36.065035+0000","last_undegraded":"2024-04-13T03:17:36.065035+0000","last_fullsized":"2024-04-13T03:17:36.065035+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:22.577449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390007+0000","last_change":"2024-04-13T03:17:34.401454+0000","last_active":"2024-04-13T03:17:35.390007+0000","last_peered":"2024-04-13T03:17:35.390007+0000","last_clean":"2024-04-13T03:17:35.390007+0000","last_became_active":"2024-04-13T03:17:34.400733+0000","last_became_peered":"2024-04-13T03:17:34.400733+0000","last_unstale":"2024-04-13T03:17:35.390007+0000","last_undegraded":"2024-04-13T03:17:35.390007+0000","last_fullsized":"2024-04-13T03:17:35.390007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:04:21.818219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065740+0000","last_change":"2024-04-13T03:17:33.399663+0000","last_active":"2024-04-13T03:17:36.065740+0000","last_peered":"2024-04-13T03:17:36.065740+0000","last_clean":"2024-04-13T03:17:36.065740+0000","last_became_active":"2024-04-13T03:17:33.399272+0000","last_became_peered":"2024-04-13T03:17:33.399272+0000","last_unstale":"2024-04-13T03:17:36.065740+0000","last_undegraded":"2024-04-13T03:17:36.065740+0000","last_fullsized":"2024-04-13T03:17:36.065740+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:23.537159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066474+0000","last_change":"2024-04-13T03:17:34.416507+0000","last_active":"2024-04-13T03:17:36.066474+0000","last_peered":"2024-04-13T03:17:36.066474+0000","last_clean":"2024-04-13T03:17:36.066474+0000","last_became_active":"2024-04-13T03:17:34.416182+0000","last_became_peered":"2024-04-13T03:17:34.416182+0000","last_unstale":"2024-04-13T03:17:36.066474+0000","last_undegraded":"2024-04-13T03:17:36.066474+0000","last_fullsized":"2024-04-13T03:17:36.066474+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065410+0000","last_change":"2024-04-13T03:17:33.398398+0000","last_active":"2024-04-13T03:17:36.065410+0000","last_peered":"2024-04-13T03:17:36.065410+0000","last_clean":"2024-04-13T03:17:36.065410+0000","last_became_active":"2024-04-13T03:17:33.398121+0000","last_became_peered":"2024-04-13T03:17:33.398121+0000","last_unstale":"2024-04-13T03:17:36.065410+0000","last_undegraded":"2024-04-13T03:17:36.065410+0000","last_fullsized":"2024-04-13T03:17:36.065410+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:08.566071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910885+0000","last_change":"2024-04-13T03:17:34.399615+0000","last_active":"2024-04-13T03:17:35.910885+0000","last_peered":"2024-04-13T03:17:35.910885+0000","last_clean":"2024-04-13T03:17:35.910885+0000","last_became_active":"2024-04-13T03:17:34.398730+0000","last_became_peered":"2024-04-13T03:17:34.398730+0000","last_unstale":"2024-04-13T03:17:35.910885+0000","last_undegraded":"2024-04-13T03:17:35.910885+0000","last_fullsized":"2024-04-13T03:17:35.910885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395409+0000","last_change":"2024-04-13T03:17:33.400583+0000","last_active":"2024-04-13T03:17:35.395409+0000","last_peered":"2024-04-13T03:17:35.395409+0000","last_clean":"2024-04-13T03:17:35.395409+0000","last_became_active":"2024-04-13T03:17:33.400299+0000","last_became_peered":"2024-04-13T03:17:33.400299+0000","last_unstale":"2024-04-13T03:17:35.395409+0000","last_undegraded":"2024-04-13T03:17:35.395409+0000","last_fullsized":"2024-04-13T03:17:35.395409+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:40:01.687395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395717+0000","last_change":"2024-04-13T03:17:34.402541+0000","last_active":"2024-04-13T03:17:35.395717+0000","last_peered":"2024-04-13T03:17:35.395717+0000","last_clean":"2024-04-13T03:17:35.395717+0000","last_became_active":"2024-04-13T03:17:34.402071+0000","last_became_peered":"2024-04-13T03:17:34.402071+0000","last_unstale":"2024-04-13T03:17:35.395717+0000","last_undegraded":"2024-04-13T03:17:35.395717+0000","last_fullsized":"2024-04-13T03:17:35.395717+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:57:32.774945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390306+0000","last_change":"2024-04-13T03:17:33.401851+0000","last_active":"2024-04-13T03:17:35.390306+0000","last_peered":"2024-04-13T03:17:35.390306+0000","last_clean":"2024-04-13T03:17:35.390306+0000","last_became_active":"2024-04-13T03:17:33.401591+0000","last_became_peered":"2024-04-13T03:17:33.401591+0000","last_unstale":"2024-04-13T03:17:35.390306+0000","last_undegraded":"2024-04-13T03:17:35.390306+0000","last_fullsized":"2024-04-13T03:17:35.390306+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:31.962483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.400737+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.400491+0000","last_became_peered":"2024-04-13T03:17:34.400491+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:43.760714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389789+0000","last_change":"2024-04-13T03:17:33.401061+0000","last_active":"2024-04-13T03:17:35.389789+0000","last_peered":"2024-04-13T03:17:35.389789+0000","last_clean":"2024-04-13T03:17:35.389789+0000","last_became_active":"2024-04-13T03:17:33.400750+0000","last_became_peered":"2024-04-13T03:17:33.400750+0000","last_unstale":"2024-04-13T03:17:35.389789+0000","last_undegraded":"2024-04-13T03:17:35.389789+0000","last_fullsized":"2024-04-13T03:17:35.389789+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:08:22.455340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480880+0000","last_change":"2024-04-13T03:17:34.407766+0000","last_active":"2024-04-13T03:17:35.480880+0000","last_peered":"2024-04-13T03:17:35.480880+0000","last_clean":"2024-04-13T03:17:35.480880+0000","last_became_active":"2024-04-13T03:17:34.407522+0000","last_became_peered":"2024-04-13T03:17:34.407522+0000","last_unstale":"2024-04-13T03:17:35.480880+0000","last_undegraded":"2024-04-13T03:17:35.480880+0000","last_fullsized":"2024-04-13T03:17:35.480880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:58:46.910362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065961+0000","last_change":"2024-04-13T03:17:33.406834+0000","last_active":"2024-04-13T03:17:36.065961+0000","last_peered":"2024-04-13T03:17:36.065961+0000","last_clean":"2024-04-13T03:17:36.065961+0000","last_became_active":"2024-04-13T03:17:33.406592+0000","last_became_peered":"2024-04-13T03:17:33.406592+0000","last_unstale":"2024-04-13T03:17:36.065961+0000","last_undegraded":"2024-04-13T03:17:36.065961+0000","last_fullsized":"2024-04-13T03:17:36.065961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:55:26.429151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390170+0000","last_change":"2024-04-13T03:17:34.411881+0000","last_active":"2024-04-13T03:17:35.390170+0000","last_peered":"2024-04-13T03:17:35.390170+0000","last_clean":"2024-04-13T03:17:35.390170+0000","last_became_active":"2024-04-13T03:17:34.411623+0000","last_became_peered":"2024-04-13T03:17:34.411623+0000","last_unstale":"2024-04-13T03:17:35.390170+0000","last_undegraded":"2024-04-13T03:17:35.390170+0000","last_fullsized":"2024-04-13T03:17:35.390170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:24:28.559176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390052+0000","last_change":"2024-04-13T03:17:33.400409+0000","last_active":"2024-04-13T03:17:35.390052+0000","last_peered":"2024-04-13T03:17:35.390052+0000","last_clean":"2024-04-13T03:17:35.390052+0000","last_became_active":"2024-04-13T03:17:33.400146+0000","last_became_peered":"2024-04-13T03:17:33.400146+0000","last_unstale":"2024-04-13T03:17:35.390052+0000","last_undegraded":"2024-04-13T03:17:35.390052+0000","last_fullsized":"2024-04-13T03:17:35.390052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:51.101678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395449+0000","last_change":"2024-04-13T03:17:34.396324+0000","last_active":"2024-04-13T03:17:35.395449+0000","last_peered":"2024-04-13T03:17:35.395449+0000","last_clean":"2024-04-13T03:17:35.395449+0000","last_became_active":"2024-04-13T03:17:34.395910+0000","last_became_peered":"2024-04-13T03:17:34.395910+0000","last_unstale":"2024-04-13T03:17:35.395449+0000","last_undegraded":"2024-04-13T03:17:35.395449+0000","last_fullsized":"2024-04-13T03:17:35.395449+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:03.411503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910959+0000","last_change":"2024-04-13T03:17:33.399899+0000","last_active":"2024-04-13T03:17:35.910959+0000","last_peered":"2024-04-13T03:17:35.910959+0000","last_clean":"2024-04-13T03:17:35.910959+0000","last_became_active":"2024-04-13T03:17:33.399573+0000","last_became_peered":"2024-04-13T03:17:33.399573+0000","last_unstale":"2024-04-13T03:17:35.910959+0000","last_undegraded":"2024-04-13T03:17:35.910959+0000","last_fullsized":"2024-04-13T03:17:35.910959+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:02:59.848302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066415+0000","last_change":"2024-04-13T03:17:34.392085+0000","last_active":"2024-04-13T03:17:36.066415+0000","last_peered":"2024-04-13T03:17:36.066415+0000","last_clean":"2024-04-13T03:17:36.066415+0000","last_became_active":"2024-04-13T03:17:34.391813+0000","last_became_peered":"2024-04-13T03:17:34.391813+0000","last_unstale":"2024-04-13T03:17:36.066415+0000","last_undegraded":"2024-04-13T03:17:36.066415+0000","last_fullsized":"2024-04-13T03:17:36.066415+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:21.107968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395659+0000","last_change":"2024-04-13T03:17:33.393982+0000","last_active":"2024-04-13T03:17:35.395659+0000","last_peered":"2024-04-13T03:17:35.395659+0000","last_clean":"2024-04-13T03:17:35.395659+0000","last_became_active":"2024-04-13T03:17:33.393615+0000","last_became_peered":"2024-04-13T03:17:33.393615+0000","last_unstale":"2024-04-13T03:17:35.395659+0000","last_undegraded":"2024-04-13T03:17:35.395659+0000","last_fullsized":"2024-04-13T03:17:35.395659+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:35:20.641845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480738+0000","last_change":"2024-04-13T03:17:34.406398+0000","last_active":"2024-04-13T03:17:35.480738+0000","last_peered":"2024-04-13T03:17:35.480738+0000","last_clean":"2024-04-13T03:17:35.480738+0000","last_became_active":"2024-04-13T03:17:34.406118+0000","last_became_peered":"2024-04-13T03:17:34.406118+0000","last_unstale":"2024-04-13T03:17:35.480738+0000","last_undegraded":"2024-04-13T03:17:35.480738+0000","last_fullsized":"2024-04-13T03:17:35.480738+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:49.097723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066129+0000","last_change":"2024-04-13T03:17:33.400700+0000","last_active":"2024-04-13T03:17:36.066129+0000","last_peered":"2024-04-13T03:17:36.066129+0000","last_clean":"2024-04-13T03:17:36.066129+0000","last_became_active":"2024-04-13T03:17:33.400443+0000","last_became_peered":"2024-04-13T03:17:33.400443+0000","last_unstale":"2024-04-13T03:17:36.066129+0000","last_undegraded":"2024-04-13T03:17:36.066129+0000","last_fullsized":"2024-04-13T03:17:36.066129+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:56.254824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395197+0000","last_change":"2024-04-13T03:17:34.416879+0000","last_active":"2024-04-13T03:17:35.395197+0000","last_peered":"2024-04-13T03:17:35.395197+0000","last_clean":"2024-04-13T03:17:35.395197+0000","last_became_active":"2024-04-13T03:17:34.416615+0000","last_became_peered":"2024-04-13T03:17:34.416615+0000","last_unstale":"2024-04-13T03:17:35.395197+0000","last_undegraded":"2024-04-13T03:17:35.395197+0000","last_fullsized":"2024-04-13T03:17:35.395197+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910943+0000","last_change":"2024-04-13T03:17:33.402668+0000","last_active":"2024-04-13T03:17:35.910943+0000","last_peered":"2024-04-13T03:17:35.910943+0000","last_clean":"2024-04-13T03:17:35.910943+0000","last_became_active":"2024-04-13T03:17:33.402363+0000","last_became_peered":"2024-04-13T03:17:33.402363+0000","last_unstale":"2024-04-13T03:17:35.910943+0000","last_undegraded":"2024-04-13T03:17:35.910943+0000","last_fullsized":"2024-04-13T03:17:35.910943+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:45.226601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390104+0000","last_change":"2024-04-13T03:17:34.403801+0000","last_active":"2024-04-13T03:17:35.390104+0000","last_peered":"2024-04-13T03:17:35.390104+0000","last_clean":"2024-04-13T03:17:35.390104+0000","last_became_active":"2024-04-13T03:17:34.403541+0000","last_became_peered":"2024-04-13T03:17:34.403541+0000","last_unstale":"2024-04-13T03:17:35.390104+0000","last_undegraded":"2024-04-13T03:17:35.390104+0000","last_fullsized":"2024-04-13T03:17:35.390104+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:51:28.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065320+0000","last_change":"2024-04-13T03:17:33.401998+0000","last_active":"2024-04-13T03:17:36.065320+0000","last_peered":"2024-04-13T03:17:36.065320+0000","last_clean":"2024-04-13T03:17:36.065320+0000","last_became_active":"2024-04-13T03:17:33.401724+0000","last_became_peered":"2024-04-13T03:17:33.401724+0000","last_unstale":"2024-04-13T03:17:36.065320+0000","last_undegraded":"2024-04-13T03:17:36.065320+0000","last_fullsized":"2024-04-13T03:17:36.065320+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:11.989180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390069+0000","last_change":"2024-04-13T03:17:34.411296+0000","last_active":"2024-04-13T03:17:35.390069+0000","last_peered":"2024-04-13T03:17:35.390069+0000","last_clean":"2024-04-13T03:17:35.390069+0000","last_became_active":"2024-04-13T03:17:34.411035+0000","last_became_peered":"2024-04-13T03:17:34.411035+0000","last_unstale":"2024-04-13T03:17:35.390069+0000","last_undegraded":"2024-04-13T03:17:35.390069+0000","last_fullsized":"2024-04-13T03:17:35.390069+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:47:34.772093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389916+0000","last_change":"2024-04-13T03:17:33.399760+0000","last_active":"2024-04-13T03:17:35.389916+0000","last_peered":"2024-04-13T03:17:35.389916+0000","last_clean":"2024-04-13T03:17:35.389916+0000","last_became_active":"2024-04-13T03:17:33.399508+0000","last_became_peered":"2024-04-13T03:17:33.399508+0000","last_unstale":"2024-04-13T03:17:35.389916+0000","last_undegraded":"2024-04-13T03:17:35.389916+0000","last_fullsized":"2024-04-13T03:17:35.389916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:14.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065695+0000","last_change":"2024-04-13T03:17:34.406246+0000","last_active":"2024-04-13T03:17:36.065695+0000","last_peered":"2024-04-13T03:17:36.065695+0000","last_clean":"2024-04-13T03:17:36.065695+0000","last_became_active":"2024-04-13T03:17:34.405992+0000","last_became_peered":"2024-04-13T03:17:34.405992+0000","last_unstale":"2024-04-13T03:17:36.065695+0000","last_undegraded":"2024-04-13T03:17:36.065695+0000","last_fullsized":"2024-04-13T03:17:36.065695+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066108+0000","last_change":"2024-04-13T03:17:33.400564+0000","last_active":"2024-04-13T03:17:36.066108+0000","last_peered":"2024-04-13T03:17:36.066108+0000","last_clean":"2024-04-13T03:17:36.066108+0000","last_became_active":"2024-04-13T03:17:33.400283+0000","last_became_peered":"2024-04-13T03:17:33.400283+0000","last_unstale":"2024-04-13T03:17:36.066108+0000","last_undegraded":"2024-04-13T03:17:36.066108+0000","last_fullsized":"2024-04-13T03:17:36.066108+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:33.869508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395860+0000","last_change":"2024-04-13T03:17:34.413234+0000","last_active":"2024-04-13T03:17:35.395860+0000","last_peered":"2024-04-13T03:17:35.395860+0000","last_clean":"2024-04-13T03:17:35.395860+0000","last_became_active":"2024-04-13T03:17:34.412967+0000","last_became_peered":"2024-04-13T03:17:34.412967+0000","last_unstale":"2024-04-13T03:17:35.395860+0000","last_undegraded":"2024-04-13T03:17:35.395860+0000","last_fullsized":"2024-04-13T03:17:35.395860+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:13.036448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390152+0000","last_change":"2024-04-13T03:17:33.399229+0000","last_active":"2024-04-13T03:17:35.390152+0000","last_peered":"2024-04-13T03:17:35.390152+0000","last_clean":"2024-04-13T03:17:35.390152+0000","last_became_active":"2024-04-13T03:17:33.398953+0000","last_became_peered":"2024-04-13T03:17:33.398953+0000","last_unstale":"2024-04-13T03:17:35.390152+0000","last_undegraded":"2024-04-13T03:17:35.390152+0000","last_fullsized":"2024-04-13T03:17:35.390152+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:04.844331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395150+0000","last_change":"2024-04-13T03:17:34.410479+0000","last_active":"2024-04-13T03:17:35.395150+0000","last_peered":"2024-04-13T03:17:35.395150+0000","last_clean":"2024-04-13T03:17:35.395150+0000","last_became_active":"2024-04-13T03:17:34.410102+0000","last_became_peered":"2024-04-13T03:17:34.410102+0000","last_unstale":"2024-04-13T03:17:35.395150+0000","last_undegraded":"2024-04-13T03:17:35.395150+0000","last_fullsized":"2024-04-13T03:17:35.395150+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390239+0000","last_change":"2024-04-13T03:17:33.401216+0000","last_active":"2024-04-13T03:17:35.390239+0000","last_peered":"2024-04-13T03:17:35.390239+0000","last_clean":"2024-04-13T03:17:35.390239+0000","last_became_active":"2024-04-13T03:17:33.400900+0000","last_became_peered":"2024-04-13T03:17:33.400900+0000","last_unstale":"2024-04-13T03:17:35.390239+0000","last_undegraded":"2024-04-13T03:17:35.390239+0000","last_fullsized":"2024-04-13T03:17:35.390239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:54:39.812442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":331,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065126+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:17:36.065126+0000","last_peered":"2024-04-13T03:17:36.065126+0000","last_clean":"2024-04-13T03:17:36.065126+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:17:36.065126+0000","last_undegraded":"2024-04-13T03:17:36.065126+0000","last_fullsized":"2024-04-13T03:17:36.065126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":1208712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":620400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":618184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":618276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1210254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:17:42.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:17:42.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 Processor -- start 2024-04-13T03:17:42.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 -- start start 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9d415e9b0 0x7ff9d40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d40cfd20 con 0x7ff9d415ed90 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d4162af0 con 0x7ff9d40b3d60 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.508+0000 7ff9da405640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d4162dd0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9d415e9b0 0x7ff9d40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9d415e9b0 0x7ff9d40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55162/0 (socket says 172.21.15.106:55162) 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff9d415ed90 0x7ff9d40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59230/0 (socket says 172.21.15.106:59230) 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 -- 172.21.15.106:0/2324974173 learned_addr learned my addr 172.21.15.106:0/2324974173 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:42.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d2ffd640 1 -- 172.21.15.106:0/2324974173 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330640309 0 0) 0x7ff9d40cfd20 con 0x7ff9d415ed90 2024-04-13T03:17:42.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d2ffd640 1 -- 172.21.15.106:0/2324974173 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9bc003520 con 0x7ff9d415ed90 2024-04-13T03:17:42.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 -- 172.21.15.106:0/2324974173 >> v1:172.21.15.114:6789/0 conn(0x7ff9d40b3d60 legacy=0x7ff9d40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:42.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 -- 172.21.15.106:0/2324974173 >> v1:172.21.15.106:6789/0 conn(0x7ff9d415ed90 legacy=0x7ff9d40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.509+0000 7ff9d3fff640 1 -- 172.21.15.106:0/2324974173 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d41630b0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9d3fff640 1 --2- 172.21.15.106:0/2324974173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d2300 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009870 tx=0x7ff9c402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c0d496ac88db21 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9d2ffd640 1 -- 172.21.15.106:0/2324974173 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c402f840 con 0x7ff9d415e9b0 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9d2ffd640 1 -- 172.21.15.106:0/2324974173 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c402f9e0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9d2ffd640 1 -- 172.21.15.106:0/2324974173 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c402fdb0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9da405640 1 -- 172.21.15.106:0/2324974173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 msgr2=0x7ff9d40d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9da405640 1 --2- 172.21.15.106:0/2324974173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d2300 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009870 tx=0x7ff9c402ba20 comp rx=0 tx=0).stop 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9da405640 1 -- 172.21.15.106:0/2324974173 shutdown_connections 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9da405640 1 --2- 172.21.15.106:0/2324974173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d2300 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.510+0000 7ff9da405640 1 -- 172.21.15.106:0/2324974173 >> 172.21.15.106:0/2324974173 conn(0x7ff9d408bf10 msgr2=0x7ff9d40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.511+0000 7ff9da405640 1 -- 172.21.15.106:0/2324974173 shutdown_connections 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.511+0000 7ff9da405640 1 -- 172.21.15.106:0/2324974173 wait complete. 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.512+0000 7ff9da405640 1 Processor -- start 2024-04-13T03:17:42.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.512+0000 7ff9da405640 1 -- start start 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d40b3d60 0x7ff9d415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9d40d39f0 0x7ff9d406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4162800 con 0x7ff9d415e9b0 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4162680 con 0x7ff9d40b3d60 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d40d4d00 con 0x7ff9d40d39f0 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55164/0 (socket says 172.21.15.106:55164) 2024-04-13T03:17:42.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 -- 172.21.15.106:0/3904082237 learned_addr learned my addr 172.21.15.106:0/3904082237 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 -- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9d40d39f0 msgr2=0x7ff9d406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9d40d39f0 0x7ff9d406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 -- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d40b3d60 msgr2=0x7ff9d415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d40b3d60 0x7ff9d415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 -- 172.21.15.106:0/3904082237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d40ccb80 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d37fe640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c370 tx=0x7ff9c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8017070 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c80074e0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80059e0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d40cb800 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.513+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d40cbdc0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.514+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d40cc6b0 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.515+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9c801a070 con 0x7ff9d415e9b0 2024-04-13T03:17:42.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.515+0000 7ff9d0ff9640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 0x7ff9c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.515+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7ff9c808b090 con 0x7ff9d415e9b0 2024-04-13T03:17:42.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.515+0000 7ff9d3fff640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 0x7ff9c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.516+0000 7ff9d3fff640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 0x7ff9c006edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4036000 tx=0x7ff9c4039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.520+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9c8059330 con 0x7ff9d415e9b0 2024-04-13T03:17:42.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.693+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9d40d3e20 con 0x7ff9c006c8f0 2024-04-13T03:17:42.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.699+0000 7ff9d0ff9640 1 -- 172.21.15.106:0/3904082237 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252374 (secure 0 0 0) 0x7ff9c402f7c0 con 0x7ff9c006c8f0 2024-04-13T03:17:42.702 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:42.704 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:17:42.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 msgr2=0x7ff9c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 0x7ff9c006edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4036000 tx=0x7ff9c4039040 comp rx=0 tx=0).stop 2024-04-13T03:17:42.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 msgr2=0x7ff9d40d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c370 tx=0x7ff9c800c830 comp rx=0 tx=0).stop 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 shutdown_connections 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff9c006c8f0 0x7ff9c006edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff9d40d39f0 0x7ff9d406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9d415e9b0 0x7ff9d40d34b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 --2- 172.21.15.106:0/3904082237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d40b3d60 0x7ff9d415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 >> 172.21.15.106:0/3904082237 conn(0x7ff9d408bf10 msgr2=0x7ff9d40b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 shutdown_connections 2024-04-13T03:17:42.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.702+0000 7ff9da405640 1 -- 172.21.15.106:0/3904082237 wait complete. 2024-04-13T03:17:42.714 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-04-13T03:17:42.473053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261040,"kb_used_data":11592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713680,"statfs":{"total":773094113280,"available":772826808320,"internally_reserved":0,"allocated":11870208,"data_stored":6131028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.088478"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395781+0000","last_change":"2024-04-13T03:17:34.397755+0000","last_active":"2024-04-13T03:17:35.395781+0000","last_peered":"2024-04-13T03:17:35.395781+0000","last_clean":"2024-04-13T03:17:35.395781+0000","last_became_active":"2024-04-13T03:17:34.397499+0000","last_became_peered":"2024-04-13T03:17:34.397499+0000","last_unstale":"2024-04-13T03:17:35.395781+0000","last_undegraded":"2024-04-13T03:17:35.395781+0000","last_fullsized":"2024-04-13T03:17:35.395781+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:51:52.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065784+0000","last_change":"2024-04-13T03:17:34.414082+0000","last_active":"2024-04-13T03:17:36.065784+0000","last_peered":"2024-04-13T03:17:36.065784+0000","last_clean":"2024-04-13T03:17:36.065784+0000","last_became_active":"2024-04-13T03:17:34.412064+0000","last_became_peered":"2024-04-13T03:17:34.412064+0000","last_unstale":"2024-04-13T03:17:36.065784+0000","last_undegraded":"2024-04-13T03:17:36.065784+0000","last_fullsized":"2024-04-13T03:17:36.065784+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395702+0000","last_change":"2024-04-13T03:17:34.406889+0000","last_active":"2024-04-13T03:17:35.395702+0000","last_peered":"2024-04-13T03:17:35.395702+0000","last_clean":"2024-04-13T03:17:35.395702+0000","last_became_active":"2024-04-13T03:17:34.406253+0000","last_became_peered":"2024-04-13T03:17:34.406253+0000","last_unstale":"2024-04-13T03:17:35.395702+0000","last_undegraded":"2024-04-13T03:17:35.395702+0000","last_fullsized":"2024-04-13T03:17:35.395702+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:00:22.906340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480782+0000","last_change":"2024-04-13T03:17:34.396984+0000","last_active":"2024-04-13T03:17:35.480782+0000","last_peered":"2024-04-13T03:17:35.480782+0000","last_clean":"2024-04-13T03:17:35.480782+0000","last_became_active":"2024-04-13T03:17:34.396599+0000","last_became_peered":"2024-04-13T03:17:34.396599+0000","last_unstale":"2024-04-13T03:17:35.480782+0000","last_undegraded":"2024-04-13T03:17:35.480782+0000","last_fullsized":"2024-04-13T03:17:35.480782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:05:44.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910724+0000","last_change":"2024-04-13T03:17:34.399389+0000","last_active":"2024-04-13T03:17:35.910724+0000","last_peered":"2024-04-13T03:17:35.910724+0000","last_clean":"2024-04-13T03:17:35.910724+0000","last_became_active":"2024-04-13T03:17:34.399010+0000","last_became_peered":"2024-04-13T03:17:34.399010+0000","last_unstale":"2024-04-13T03:17:35.910724+0000","last_undegraded":"2024-04-13T03:17:35.910724+0000","last_fullsized":"2024-04-13T03:17:35.910724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066413+0000","last_change":"2024-04-13T03:17:34.412752+0000","last_active":"2024-04-13T03:17:36.066413+0000","last_peered":"2024-04-13T03:17:36.066413+0000","last_clean":"2024-04-13T03:17:36.066413+0000","last_became_active":"2024-04-13T03:17:34.412470+0000","last_became_peered":"2024-04-13T03:17:34.412470+0000","last_unstale":"2024-04-13T03:17:36.066413+0000","last_undegraded":"2024-04-13T03:17:36.066413+0000","last_fullsized":"2024-04-13T03:17:36.066413+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065869+0000","last_change":"2024-04-13T03:17:34.411655+0000","last_active":"2024-04-13T03:17:36.065869+0000","last_peered":"2024-04-13T03:17:36.065869+0000","last_clean":"2024-04-13T03:17:36.065869+0000","last_became_active":"2024-04-13T03:17:34.411211+0000","last_became_peered":"2024-04-13T03:17:34.411211+0000","last_unstale":"2024-04-13T03:17:36.065869+0000","last_undegraded":"2024-04-13T03:17:36.065869+0000","last_fullsized":"2024-04-13T03:17:36.065869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:12.994264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389760+0000","last_change":"2024-04-13T03:17:34.402872+0000","last_active":"2024-04-13T03:17:35.389760+0000","last_peered":"2024-04-13T03:17:35.389760+0000","last_clean":"2024-04-13T03:17:35.389760+0000","last_became_active":"2024-04-13T03:17:34.401106+0000","last_became_peered":"2024-04-13T03:17:34.401106+0000","last_unstale":"2024-04-13T03:17:35.389760+0000","last_undegraded":"2024-04-13T03:17:35.389760+0000","last_fullsized":"2024-04-13T03:17:35.389760+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:15:37.328883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065661+0000","last_change":"2024-04-13T03:17:34.410456+0000","last_active":"2024-04-13T03:17:36.065661+0000","last_peered":"2024-04-13T03:17:36.065661+0000","last_clean":"2024-04-13T03:17:36.065661+0000","last_became_active":"2024-04-13T03:17:34.410166+0000","last_became_peered":"2024-04-13T03:17:34.410166+0000","last_unstale":"2024-04-13T03:17:36.065661+0000","last_undegraded":"2024-04-13T03:17:36.065661+0000","last_fullsized":"2024-04-13T03:17:36.065661+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:35.141523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.911023+0000","last_change":"2024-04-13T03:17:34.410800+0000","last_active":"2024-04-13T03:17:35.911023+0000","last_peered":"2024-04-13T03:17:35.911023+0000","last_clean":"2024-04-13T03:17:35.911023+0000","last_became_active":"2024-04-13T03:17:34.410568+0000","last_became_peered":"2024-04-13T03:17:34.410568+0000","last_unstale":"2024-04-13T03:17:35.911023+0000","last_undegraded":"2024-04-13T03:17:35.911023+0000","last_fullsized":"2024-04-13T03:17:35.911023+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395658+0000","last_change":"2024-04-13T03:17:34.407143+0000","last_active":"2024-04-13T03:17:35.395658+0000","last_peered":"2024-04-13T03:17:35.395658+0000","last_clean":"2024-04-13T03:17:35.395658+0000","last_became_active":"2024-04-13T03:17:34.406821+0000","last_became_peered":"2024-04-13T03:17:34.406821+0000","last_unstale":"2024-04-13T03:17:35.395658+0000","last_undegraded":"2024-04-13T03:17:35.395658+0000","last_fullsized":"2024-04-13T03:17:35.395658+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:15:20.135164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066224+0000","last_change":"2024-04-13T03:17:34.392870+0000","last_active":"2024-04-13T03:17:36.066224+0000","last_peered":"2024-04-13T03:17:36.066224+0000","last_clean":"2024-04-13T03:17:36.066224+0000","last_became_active":"2024-04-13T03:17:34.392598+0000","last_became_peered":"2024-04-13T03:17:34.392598+0000","last_unstale":"2024-04-13T03:17:36.066224+0000","last_undegraded":"2024-04-13T03:17:36.066224+0000","last_fullsized":"2024-04-13T03:17:36.066224+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:08.590896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480726+0000","last_change":"2024-04-13T03:17:34.406086+0000","last_active":"2024-04-13T03:17:35.480726+0000","last_peered":"2024-04-13T03:17:35.480726+0000","last_clean":"2024-04-13T03:17:35.480726+0000","last_became_active":"2024-04-13T03:17:34.405810+0000","last_became_peered":"2024-04-13T03:17:34.405810+0000","last_unstale":"2024-04-13T03:17:35.480726+0000","last_undegraded":"2024-04-13T03:17:35.480726+0000","last_fullsized":"2024-04-13T03:17:35.480726+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:09:02.642224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065362+0000","last_change":"2024-04-13T03:17:34.400371+0000","last_active":"2024-04-13T03:17:36.065362+0000","last_peered":"2024-04-13T03:17:36.065362+0000","last_clean":"2024-04-13T03:17:36.065362+0000","last_became_active":"2024-04-13T03:17:34.399998+0000","last_became_peered":"2024-04-13T03:17:34.399998+0000","last_unstale":"2024-04-13T03:17:36.065362+0000","last_undegraded":"2024-04-13T03:17:36.065362+0000","last_fullsized":"2024-04-13T03:17:36.065362+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065088+0000","last_change":"2024-04-13T03:17:34.407130+0000","last_active":"2024-04-13T03:17:36.065088+0000","last_peered":"2024-04-13T03:17:36.065088+0000","last_clean":"2024-04-13T03:17:36.065088+0000","last_became_active":"2024-04-13T03:17:34.406854+0000","last_became_peered":"2024-04-13T03:17:34.406854+0000","last_unstale":"2024-04-13T03:17:36.065088+0000","last_undegraded":"2024-04-13T03:17:36.065088+0000","last_fullsized":"2024-04-13T03:17:36.065088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065064+0000","last_change":"2024-04-13T03:17:34.401481+0000","last_active":"2024-04-13T03:17:36.065064+0000","last_peered":"2024-04-13T03:17:36.065064+0000","last_clean":"2024-04-13T03:17:36.065064+0000","last_became_active":"2024-04-13T03:17:34.401209+0000","last_became_peered":"2024-04-13T03:17:34.401209+0000","last_unstale":"2024-04-13T03:17:36.065064+0000","last_undegraded":"2024-04-13T03:17:36.065064+0000","last_fullsized":"2024-04-13T03:17:36.065064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:21.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395885+0000","last_change":"2024-04-13T03:17:34.405442+0000","last_active":"2024-04-13T03:17:35.395885+0000","last_peered":"2024-04-13T03:17:35.395885+0000","last_clean":"2024-04-13T03:17:35.395885+0000","last_became_active":"2024-04-13T03:17:34.404800+0000","last_became_peered":"2024-04-13T03:17:34.404800+0000","last_unstale":"2024-04-13T03:17:35.395885+0000","last_undegraded":"2024-04-13T03:17:35.395885+0000","last_fullsized":"2024-04-13T03:17:35.395885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:50:42.464825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065972+0000","last_change":"2024-04-13T03:17:34.410814+0000","last_active":"2024-04-13T03:17:36.065972+0000","last_peered":"2024-04-13T03:17:36.065972+0000","last_clean":"2024-04-13T03:17:36.065972+0000","last_became_active":"2024-04-13T03:17:34.410551+0000","last_became_peered":"2024-04-13T03:17:34.410551+0000","last_unstale":"2024-04-13T03:17:36.065972+0000","last_undegraded":"2024-04-13T03:17:36.065972+0000","last_fullsized":"2024-04-13T03:17:36.065972+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395709+0000","last_change":"2024-04-13T03:17:34.408576+0000","last_active":"2024-04-13T03:17:35.395709+0000","last_peered":"2024-04-13T03:17:35.395709+0000","last_clean":"2024-04-13T03:17:35.395709+0000","last_became_active":"2024-04-13T03:17:34.408330+0000","last_became_peered":"2024-04-13T03:17:34.408330+0000","last_unstale":"2024-04-13T03:17:35.395709+0000","last_undegraded":"2024-04-13T03:17:35.395709+0000","last_fullsized":"2024-04-13T03:17:35.395709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:15.555032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066073+0000","last_change":"2024-04-13T03:17:34.398173+0000","last_active":"2024-04-13T03:17:36.066073+0000","last_peered":"2024-04-13T03:17:36.066073+0000","last_clean":"2024-04-13T03:17:36.066073+0000","last_became_active":"2024-04-13T03:17:34.397874+0000","last_became_peered":"2024-04-13T03:17:34.397874+0000","last_unstale":"2024-04-13T03:17:36.066073+0000","last_undegraded":"2024-04-13T03:17:36.066073+0000","last_fullsized":"2024-04-13T03:17:36.066073+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390283+0000","last_change":"2024-04-13T03:17:34.411955+0000","last_active":"2024-04-13T03:17:35.390283+0000","last_peered":"2024-04-13T03:17:35.390283+0000","last_clean":"2024-04-13T03:17:35.390283+0000","last_became_active":"2024-04-13T03:17:34.411568+0000","last_became_peered":"2024-04-13T03:17:34.411568+0000","last_unstale":"2024-04-13T03:17:35.390283+0000","last_undegraded":"2024-04-13T03:17:35.390283+0000","last_fullsized":"2024-04-13T03:17:35.390283+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:28:52.418315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910831+0000","last_change":"2024-04-13T03:17:34.410118+0000","last_active":"2024-04-13T03:17:35.910831+0000","last_peered":"2024-04-13T03:17:35.910831+0000","last_clean":"2024-04-13T03:17:35.910831+0000","last_became_active":"2024-04-13T03:17:34.409793+0000","last_became_peered":"2024-04-13T03:17:34.409793+0000","last_unstale":"2024-04-13T03:17:35.910831+0000","last_undegraded":"2024-04-13T03:17:35.910831+0000","last_fullsized":"2024-04-13T03:17:35.910831+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:06.914604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395535+0000","last_change":"2024-04-13T03:17:34.402676+0000","last_active":"2024-04-13T03:17:35.395535+0000","last_peered":"2024-04-13T03:17:35.395535+0000","last_clean":"2024-04-13T03:17:35.395535+0000","last_became_active":"2024-04-13T03:17:34.402403+0000","last_became_peered":"2024-04-13T03:17:34.402403+0000","last_unstale":"2024-04-13T03:17:35.395535+0000","last_undegraded":"2024-04-13T03:17:35.395535+0000","last_fullsized":"2024-04-13T03:17:35.395535+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:27.486850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910608+0000","last_change":"2024-04-13T03:17:34.409245+0000","last_active":"2024-04-13T03:17:35.910608+0000","last_peered":"2024-04-13T03:17:35.910608+0000","last_clean":"2024-04-13T03:17:35.910608+0000","last_became_active":"2024-04-13T03:17:34.408969+0000","last_became_peered":"2024-04-13T03:17:34.408969+0000","last_unstale":"2024-04-13T03:17:35.910608+0000","last_undegraded":"2024-04-13T03:17:35.910608+0000","last_fullsized":"2024-04-13T03:17:35.910608+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:58:35.934049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065690+0000","last_change":"2024-04-13T03:17:34.405874+0000","last_active":"2024-04-13T03:17:36.065690+0000","last_peered":"2024-04-13T03:17:36.065690+0000","last_clean":"2024-04-13T03:17:36.065690+0000","last_became_active":"2024-04-13T03:17:34.405601+0000","last_became_peered":"2024-04-13T03:17:34.405601+0000","last_unstale":"2024-04-13T03:17:36.065690+0000","last_undegraded":"2024-04-13T03:17:36.065690+0000","last_fullsized":"2024-04-13T03:17:36.065690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:07.093494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065465+0000","last_change":"2024-04-13T03:17:34.407016+0000","last_active":"2024-04-13T03:17:36.065465+0000","last_peered":"2024-04-13T03:17:36.065465+0000","last_clean":"2024-04-13T03:17:36.065465+0000","last_became_active":"2024-04-13T03:17:34.406642+0000","last_became_peered":"2024-04-13T03:17:34.406642+0000","last_unstale":"2024-04-13T03:17:36.065465+0000","last_undegraded":"2024-04-13T03:17:36.065465+0000","last_fullsized":"2024-04-13T03:17:36.065465+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:02.499437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395683+0000","last_change":"2024-04-13T03:17:34.412860+0000","last_active":"2024-04-13T03:17:35.395683+0000","last_peered":"2024-04-13T03:17:35.395683+0000","last_clean":"2024-04-13T03:17:35.395683+0000","last_became_active":"2024-04-13T03:17:34.411893+0000","last_became_peered":"2024-04-13T03:17:34.411893+0000","last_unstale":"2024-04-13T03:17:35.395683+0000","last_undegraded":"2024-04-13T03:17:35.395683+0000","last_fullsized":"2024-04-13T03:17:35.395683+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:45:20.716257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065970+0000","last_change":"2024-04-13T03:17:34.414317+0000","last_active":"2024-04-13T03:17:36.065970+0000","last_peered":"2024-04-13T03:17:36.065970+0000","last_clean":"2024-04-13T03:17:36.065970+0000","last_became_active":"2024-04-13T03:17:34.414038+0000","last_became_peered":"2024-04-13T03:17:34.414038+0000","last_unstale":"2024-04-13T03:17:36.065970+0000","last_undegraded":"2024-04-13T03:17:36.065970+0000","last_fullsized":"2024-04-13T03:17:36.065970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390278+0000","last_change":"2024-04-13T03:17:34.403227+0000","last_active":"2024-04-13T03:17:35.390278+0000","last_peered":"2024-04-13T03:17:35.390278+0000","last_clean":"2024-04-13T03:17:35.390278+0000","last_became_active":"2024-04-13T03:17:34.402961+0000","last_became_peered":"2024-04-13T03:17:34.402961+0000","last_unstale":"2024-04-13T03:17:35.390278+0000","last_undegraded":"2024-04-13T03:17:35.390278+0000","last_fullsized":"2024-04-13T03:17:35.390278+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:53:56.840439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066167+0000","last_change":"2024-04-13T03:17:34.405766+0000","last_active":"2024-04-13T03:17:36.066167+0000","last_peered":"2024-04-13T03:17:36.066167+0000","last_clean":"2024-04-13T03:17:36.066167+0000","last_became_active":"2024-04-13T03:17:34.404793+0000","last_became_peered":"2024-04-13T03:17:34.404793+0000","last_unstale":"2024-04-13T03:17:36.066167+0000","last_undegraded":"2024-04-13T03:17:36.066167+0000","last_fullsized":"2024-04-13T03:17:36.066167+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:13:26.382175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.416600+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.416208+0000","last_became_peered":"2024-04-13T03:17:34.416208+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389814+0000","last_change":"2024-04-13T03:17:34.402382+0000","last_active":"2024-04-13T03:17:35.389814+0000","last_peered":"2024-04-13T03:17:35.389814+0000","last_clean":"2024-04-13T03:17:35.389814+0000","last_became_active":"2024-04-13T03:17:34.401181+0000","last_became_peered":"2024-04-13T03:17:34.401181+0000","last_unstale":"2024-04-13T03:17:35.389814+0000","last_undegraded":"2024-04-13T03:17:35.389814+0000","last_fullsized":"2024-04-13T03:17:35.389814+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:26:43.678954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395636+0000","last_change":"2024-04-13T03:17:34.408372+0000","last_active":"2024-04-13T03:17:35.395636+0000","last_peered":"2024-04-13T03:17:35.395636+0000","last_clean":"2024-04-13T03:17:35.395636+0000","last_became_active":"2024-04-13T03:17:34.408097+0000","last_became_peered":"2024-04-13T03:17:34.408097+0000","last_unstale":"2024-04-13T03:17:35.395636+0000","last_undegraded":"2024-04-13T03:17:35.395636+0000","last_fullsized":"2024-04-13T03:17:35.395636+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:14:09.892547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480883+0000","last_change":"2024-04-13T03:17:33.402709+0000","last_active":"2024-04-13T03:17:35.480883+0000","last_peered":"2024-04-13T03:17:35.480883+0000","last_clean":"2024-04-13T03:17:35.480883+0000","last_became_active":"2024-04-13T03:17:33.402409+0000","last_became_peered":"2024-04-13T03:17:33.402409+0000","last_unstale":"2024-04-13T03:17:35.480883+0000","last_undegraded":"2024-04-13T03:17:35.480883+0000","last_fullsized":"2024-04-13T03:17:35.480883+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:06.088410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389771+0000","last_change":"2024-04-13T03:17:34.403177+0000","last_active":"2024-04-13T03:17:35.389771+0000","last_peered":"2024-04-13T03:17:35.389771+0000","last_clean":"2024-04-13T03:17:35.389771+0000","last_became_active":"2024-04-13T03:17:34.402894+0000","last_became_peered":"2024-04-13T03:17:34.402894+0000","last_unstale":"2024-04-13T03:17:35.389771+0000","last_undegraded":"2024-04-13T03:17:35.389771+0000","last_fullsized":"2024-04-13T03:17:35.389771+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:00:34.744333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390131+0000","last_change":"2024-04-13T03:17:33.401391+0000","last_active":"2024-04-13T03:17:35.390131+0000","last_peered":"2024-04-13T03:17:35.390131+0000","last_clean":"2024-04-13T03:17:35.390131+0000","last_became_active":"2024-04-13T03:17:33.401115+0000","last_became_peered":"2024-04-13T03:17:33.401115+0000","last_unstale":"2024-04-13T03:17:35.390131+0000","last_undegraded":"2024-04-13T03:17:35.390131+0000","last_fullsized":"2024-04-13T03:17:35.390131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:34.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066244+0000","last_change":"2024-04-13T03:17:34.401443+0000","last_active":"2024-04-13T03:17:36.066244+0000","last_peered":"2024-04-13T03:17:36.066244+0000","last_clean":"2024-04-13T03:17:36.066244+0000","last_became_active":"2024-04-13T03:17:34.401088+0000","last_became_peered":"2024-04-13T03:17:34.401088+0000","last_unstale":"2024-04-13T03:17:36.066244+0000","last_undegraded":"2024-04-13T03:17:36.066244+0000","last_fullsized":"2024-04-13T03:17:36.066244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:05:39.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065122+0000","last_change":"2024-04-13T03:17:33.396742+0000","last_active":"2024-04-13T03:17:36.065122+0000","last_peered":"2024-04-13T03:17:36.065122+0000","last_clean":"2024-04-13T03:17:36.065122+0000","last_became_active":"2024-04-13T03:17:33.396322+0000","last_became_peered":"2024-04-13T03:17:33.396322+0000","last_unstale":"2024-04-13T03:17:36.065122+0000","last_undegraded":"2024-04-13T03:17:36.065122+0000","last_fullsized":"2024-04-13T03:17:36.065122+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:13.384064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395276+0000","last_change":"2024-04-13T03:17:34.404596+0000","last_active":"2024-04-13T03:17:35.395276+0000","last_peered":"2024-04-13T03:17:35.395276+0000","last_clean":"2024-04-13T03:17:35.395276+0000","last_became_active":"2024-04-13T03:17:34.403699+0000","last_became_peered":"2024-04-13T03:17:34.403699+0000","last_unstale":"2024-04-13T03:17:35.395276+0000","last_undegraded":"2024-04-13T03:17:35.395276+0000","last_fullsized":"2024-04-13T03:17:35.395276+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910957+0000","last_change":"2024-04-13T03:17:33.398782+0000","last_active":"2024-04-13T03:17:35.910957+0000","last_peered":"2024-04-13T03:17:35.910957+0000","last_clean":"2024-04-13T03:17:35.910957+0000","last_became_active":"2024-04-13T03:17:33.398493+0000","last_became_peered":"2024-04-13T03:17:33.398493+0000","last_unstale":"2024-04-13T03:17:35.910957+0000","last_undegraded":"2024-04-13T03:17:35.910957+0000","last_fullsized":"2024-04-13T03:17:35.910957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:06:27.097899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480837+0000","last_change":"2024-04-13T03:17:34.400995+0000","last_active":"2024-04-13T03:17:35.480837+0000","last_peered":"2024-04-13T03:17:35.480837+0000","last_clean":"2024-04-13T03:17:35.480837+0000","last_became_active":"2024-04-13T03:17:34.400701+0000","last_became_peered":"2024-04-13T03:17:34.400701+0000","last_unstale":"2024-04-13T03:17:35.480837+0000","last_undegraded":"2024-04-13T03:17:35.480837+0000","last_fullsized":"2024-04-13T03:17:35.480837+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:16:23.670640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065289+0000","last_change":"2024-04-13T03:17:33.401599+0000","last_active":"2024-04-13T03:17:36.065289+0000","last_peered":"2024-04-13T03:17:36.065289+0000","last_clean":"2024-04-13T03:17:36.065289+0000","last_became_active":"2024-04-13T03:17:33.401334+0000","last_became_peered":"2024-04-13T03:17:33.401334+0000","last_unstale":"2024-04-13T03:17:36.065289+0000","last_undegraded":"2024-04-13T03:17:36.065289+0000","last_fullsized":"2024-04-13T03:17:36.065289+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:37.518351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.408972+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.408008+0000","last_became_peered":"2024-04-13T03:17:34.408008+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:44:31.275566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395064+0000","last_change":"2024-04-13T03:17:33.400110+0000","last_active":"2024-04-13T03:17:35.395064+0000","last_peered":"2024-04-13T03:17:35.395064+0000","last_clean":"2024-04-13T03:17:35.395064+0000","last_became_active":"2024-04-13T03:17:33.399844+0000","last_became_peered":"2024-04-13T03:17:33.399844+0000","last_unstale":"2024-04-13T03:17:35.395064+0000","last_undegraded":"2024-04-13T03:17:35.395064+0000","last_fullsized":"2024-04-13T03:17:35.395064+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:15:33.561822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066170+0000","last_change":"2024-04-13T03:17:34.414775+0000","last_active":"2024-04-13T03:17:36.066170+0000","last_peered":"2024-04-13T03:17:36.066170+0000","last_clean":"2024-04-13T03:17:36.066170+0000","last_became_active":"2024-04-13T03:17:34.414524+0000","last_became_peered":"2024-04-13T03:17:34.414524+0000","last_unstale":"2024-04-13T03:17:36.066170+0000","last_undegraded":"2024-04-13T03:17:36.066170+0000","last_fullsized":"2024-04-13T03:17:36.066170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066358+0000","last_change":"2024-04-13T03:17:33.396584+0000","last_active":"2024-04-13T03:17:36.066358+0000","last_peered":"2024-04-13T03:17:36.066358+0000","last_clean":"2024-04-13T03:17:36.066358+0000","last_became_active":"2024-04-13T03:17:33.395985+0000","last_became_peered":"2024-04-13T03:17:33.395985+0000","last_unstale":"2024-04-13T03:17:36.066358+0000","last_undegraded":"2024-04-13T03:17:36.066358+0000","last_fullsized":"2024-04-13T03:17:36.066358+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:08:05.909520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480922+0000","last_change":"2024-04-13T03:17:34.401457+0000","last_active":"2024-04-13T03:17:35.480922+0000","last_peered":"2024-04-13T03:17:35.480922+0000","last_clean":"2024-04-13T03:17:35.480922+0000","last_became_active":"2024-04-13T03:17:34.401101+0000","last_became_peered":"2024-04-13T03:17:34.401101+0000","last_unstale":"2024-04-13T03:17:35.480922+0000","last_undegraded":"2024-04-13T03:17:35.480922+0000","last_fullsized":"2024-04-13T03:17:35.480922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:03:45.552674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480567+0000","last_change":"2024-04-13T03:17:33.391995+0000","last_active":"2024-04-13T03:17:35.480567+0000","last_peered":"2024-04-13T03:17:35.480567+0000","last_clean":"2024-04-13T03:17:35.480567+0000","last_became_active":"2024-04-13T03:17:33.391716+0000","last_became_peered":"2024-04-13T03:17:33.391716+0000","last_unstale":"2024-04-13T03:17:35.480567+0000","last_undegraded":"2024-04-13T03:17:35.480567+0000","last_fullsized":"2024-04-13T03:17:35.480567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:50:01.494353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395497+0000","last_change":"2024-04-13T03:17:34.406950+0000","last_active":"2024-04-13T03:17:35.395497+0000","last_peered":"2024-04-13T03:17:35.395497+0000","last_clean":"2024-04-13T03:17:35.395497+0000","last_became_active":"2024-04-13T03:17:34.406487+0000","last_became_peered":"2024-04-13T03:17:34.406487+0000","last_unstale":"2024-04-13T03:17:35.395497+0000","last_undegraded":"2024-04-13T03:17:35.395497+0000","last_fullsized":"2024-04-13T03:17:35.395497+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:15:35.175581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065815+0000","last_change":"2024-04-13T03:17:33.397241+0000","last_active":"2024-04-13T03:17:36.065815+0000","last_peered":"2024-04-13T03:17:36.065815+0000","last_clean":"2024-04-13T03:17:36.065815+0000","last_became_active":"2024-04-13T03:17:33.396811+0000","last_became_peered":"2024-04-13T03:17:33.396811+0000","last_unstale":"2024-04-13T03:17:36.065815+0000","last_undegraded":"2024-04-13T03:17:36.065815+0000","last_fullsized":"2024-04-13T03:17:36.065815+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:14.055121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910747+0000","last_change":"2024-04-13T03:17:34.404666+0000","last_active":"2024-04-13T03:17:35.910747+0000","last_peered":"2024-04-13T03:17:35.910747+0000","last_clean":"2024-04-13T03:17:35.910747+0000","last_became_active":"2024-04-13T03:17:34.404419+0000","last_became_peered":"2024-04-13T03:17:34.404419+0000","last_unstale":"2024-04-13T03:17:35.910747+0000","last_undegraded":"2024-04-13T03:17:35.910747+0000","last_fullsized":"2024-04-13T03:17:35.910747+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:23.582532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395462+0000","last_change":"2024-04-13T03:17:33.398984+0000","last_active":"2024-04-13T03:17:35.395462+0000","last_peered":"2024-04-13T03:17:35.395462+0000","last_clean":"2024-04-13T03:17:35.395462+0000","last_became_active":"2024-04-13T03:17:33.398683+0000","last_became_peered":"2024-04-13T03:17:33.398683+0000","last_unstale":"2024-04-13T03:17:35.395462+0000","last_undegraded":"2024-04-13T03:17:35.395462+0000","last_fullsized":"2024-04-13T03:17:35.395462+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:19:37.874955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390013+0000","last_change":"2024-04-13T03:17:34.404347+0000","last_active":"2024-04-13T03:17:35.390013+0000","last_peered":"2024-04-13T03:17:35.390013+0000","last_clean":"2024-04-13T03:17:35.390013+0000","last_became_active":"2024-04-13T03:17:34.404070+0000","last_became_peered":"2024-04-13T03:17:34.404070+0000","last_unstale":"2024-04-13T03:17:35.390013+0000","last_undegraded":"2024-04-13T03:17:35.390013+0000","last_fullsized":"2024-04-13T03:17:35.390013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:27:04.357744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065277+0000","last_change":"2024-04-13T03:17:33.395592+0000","last_active":"2024-04-13T03:17:36.065277+0000","last_peered":"2024-04-13T03:17:36.065277+0000","last_clean":"2024-04-13T03:17:36.065277+0000","last_became_active":"2024-04-13T03:17:33.395290+0000","last_became_peered":"2024-04-13T03:17:33.395290+0000","last_unstale":"2024-04-13T03:17:36.065277+0000","last_undegraded":"2024-04-13T03:17:36.065277+0000","last_fullsized":"2024-04-13T03:17:36.065277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:01.476888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910892+0000","last_change":"2024-04-13T03:17:34.406520+0000","last_active":"2024-04-13T03:17:35.910892+0000","last_peered":"2024-04-13T03:17:35.910892+0000","last_clean":"2024-04-13T03:17:35.910892+0000","last_became_active":"2024-04-13T03:17:34.406214+0000","last_became_peered":"2024-04-13T03:17:34.406214+0000","last_unstale":"2024-04-13T03:17:35.910892+0000","last_undegraded":"2024-04-13T03:17:35.910892+0000","last_fullsized":"2024-04-13T03:17:35.910892+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:59:15.060205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066117+0000","last_change":"2024-04-13T03:17:33.401643+0000","last_active":"2024-04-13T03:17:36.066117+0000","last_peered":"2024-04-13T03:17:36.066117+0000","last_clean":"2024-04-13T03:17:36.066117+0000","last_became_active":"2024-04-13T03:17:33.401302+0000","last_became_peered":"2024-04-13T03:17:33.401302+0000","last_unstale":"2024-04-13T03:17:36.066117+0000","last_undegraded":"2024-04-13T03:17:36.066117+0000","last_fullsized":"2024-04-13T03:17:36.066117+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:22:14.155305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395482+0000","last_change":"2024-04-13T03:17:34.410826+0000","last_active":"2024-04-13T03:17:35.395482+0000","last_peered":"2024-04-13T03:17:35.395482+0000","last_clean":"2024-04-13T03:17:35.395482+0000","last_became_active":"2024-04-13T03:17:34.410554+0000","last_became_peered":"2024-04-13T03:17:34.410554+0000","last_unstale":"2024-04-13T03:17:35.395482+0000","last_undegraded":"2024-04-13T03:17:35.395482+0000","last_fullsized":"2024-04-13T03:17:35.395482+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:53:21.523097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910964+0000","last_change":"2024-04-13T03:17:33.399676+0000","last_active":"2024-04-13T03:17:35.910964+0000","last_peered":"2024-04-13T03:17:35.910964+0000","last_clean":"2024-04-13T03:17:35.910964+0000","last_became_active":"2024-04-13T03:17:33.399363+0000","last_became_peered":"2024-04-13T03:17:33.399363+0000","last_unstale":"2024-04-13T03:17:35.910964+0000","last_undegraded":"2024-04-13T03:17:35.910964+0000","last_fullsized":"2024-04-13T03:17:35.910964+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:12:48.773459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066350+0000","last_change":"2024-04-13T03:17:34.409673+0000","last_active":"2024-04-13T03:17:36.066350+0000","last_peered":"2024-04-13T03:17:36.066350+0000","last_clean":"2024-04-13T03:17:36.066350+0000","last_became_active":"2024-04-13T03:17:34.409382+0000","last_became_peered":"2024-04-13T03:17:34.409382+0000","last_unstale":"2024-04-13T03:17:36.066350+0000","last_undegraded":"2024-04-13T03:17:36.066350+0000","last_fullsized":"2024-04-13T03:17:36.066350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480628+0000","last_change":"2024-04-13T03:17:33.397638+0000","last_active":"2024-04-13T03:17:35.480628+0000","last_peered":"2024-04-13T03:17:35.480628+0000","last_clean":"2024-04-13T03:17:35.480628+0000","last_became_active":"2024-04-13T03:17:33.397347+0000","last_became_peered":"2024-04-13T03:17:33.397347+0000","last_unstale":"2024-04-13T03:17:35.480628+0000","last_undegraded":"2024-04-13T03:17:35.480628+0000","last_fullsized":"2024-04-13T03:17:35.480628+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:22:20.211899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390092+0000","last_change":"2024-04-13T03:17:34.404927+0000","last_active":"2024-04-13T03:17:35.390092+0000","last_peered":"2024-04-13T03:17:35.390092+0000","last_clean":"2024-04-13T03:17:35.390092+0000","last_became_active":"2024-04-13T03:17:34.404291+0000","last_became_peered":"2024-04-13T03:17:34.404291+0000","last_unstale":"2024-04-13T03:17:35.390092+0000","last_undegraded":"2024-04-13T03:17:35.390092+0000","last_fullsized":"2024-04-13T03:17:35.390092+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:55:24.854841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389785+0000","last_change":"2024-04-13T03:17:33.399225+0000","last_active":"2024-04-13T03:17:35.389785+0000","last_peered":"2024-04-13T03:17:35.389785+0000","last_clean":"2024-04-13T03:17:35.389785+0000","last_became_active":"2024-04-13T03:17:33.398968+0000","last_became_peered":"2024-04-13T03:17:33.398968+0000","last_unstale":"2024-04-13T03:17:35.389785+0000","last_undegraded":"2024-04-13T03:17:35.389785+0000","last_fullsized":"2024-04-13T03:17:35.389785+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:35:22.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395912+0000","last_change":"2024-04-13T03:17:34.405422+0000","last_active":"2024-04-13T03:17:35.395912+0000","last_peered":"2024-04-13T03:17:35.395912+0000","last_clean":"2024-04-13T03:17:35.395912+0000","last_became_active":"2024-04-13T03:17:34.403149+0000","last_became_peered":"2024-04-13T03:17:34.403149+0000","last_unstale":"2024-04-13T03:17:35.395912+0000","last_undegraded":"2024-04-13T03:17:35.395912+0000","last_fullsized":"2024-04-13T03:17:35.395912+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:44.808739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065035+0000","last_change":"2024-04-13T03:17:33.395530+0000","last_active":"2024-04-13T03:17:36.065035+0000","last_peered":"2024-04-13T03:17:36.065035+0000","last_clean":"2024-04-13T03:17:36.065035+0000","last_became_active":"2024-04-13T03:17:33.395261+0000","last_became_peered":"2024-04-13T03:17:33.395261+0000","last_unstale":"2024-04-13T03:17:36.065035+0000","last_undegraded":"2024-04-13T03:17:36.065035+0000","last_fullsized":"2024-04-13T03:17:36.065035+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:21:22.577449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390007+0000","last_change":"2024-04-13T03:17:34.401454+0000","last_active":"2024-04-13T03:17:35.390007+0000","last_peered":"2024-04-13T03:17:35.390007+0000","last_clean":"2024-04-13T03:17:35.390007+0000","last_became_active":"2024-04-13T03:17:34.400733+0000","last_became_peered":"2024-04-13T03:17:34.400733+0000","last_unstale":"2024-04-13T03:17:35.390007+0000","last_undegraded":"2024-04-13T03:17:35.390007+0000","last_fullsized":"2024-04-13T03:17:35.390007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:04:21.818219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065740+0000","last_change":"2024-04-13T03:17:33.399663+0000","last_active":"2024-04-13T03:17:36.065740+0000","last_peered":"2024-04-13T03:17:36.065740+0000","last_clean":"2024-04-13T03:17:36.065740+0000","last_became_active":"2024-04-13T03:17:33.399272+0000","last_became_peered":"2024-04-13T03:17:33.399272+0000","last_unstale":"2024-04-13T03:17:36.065740+0000","last_undegraded":"2024-04-13T03:17:36.065740+0000","last_fullsized":"2024-04-13T03:17:36.065740+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:15:23.537159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066474+0000","last_change":"2024-04-13T03:17:34.416507+0000","last_active":"2024-04-13T03:17:36.066474+0000","last_peered":"2024-04-13T03:17:36.066474+0000","last_clean":"2024-04-13T03:17:36.066474+0000","last_became_active":"2024-04-13T03:17:34.416182+0000","last_became_peered":"2024-04-13T03:17:34.416182+0000","last_unstale":"2024-04-13T03:17:36.066474+0000","last_undegraded":"2024-04-13T03:17:36.066474+0000","last_fullsized":"2024-04-13T03:17:36.066474+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:35:38.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065410+0000","last_change":"2024-04-13T03:17:33.398398+0000","last_active":"2024-04-13T03:17:36.065410+0000","last_peered":"2024-04-13T03:17:36.065410+0000","last_clean":"2024-04-13T03:17:36.065410+0000","last_became_active":"2024-04-13T03:17:33.398121+0000","last_became_peered":"2024-04-13T03:17:33.398121+0000","last_unstale":"2024-04-13T03:17:36.065410+0000","last_undegraded":"2024-04-13T03:17:36.065410+0000","last_fullsized":"2024-04-13T03:17:36.065410+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:08.566071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910885+0000","last_change":"2024-04-13T03:17:34.399615+0000","last_active":"2024-04-13T03:17:35.910885+0000","last_peered":"2024-04-13T03:17:35.910885+0000","last_clean":"2024-04-13T03:17:35.910885+0000","last_became_active":"2024-04-13T03:17:34.398730+0000","last_became_peered":"2024-04-13T03:17:34.398730+0000","last_unstale":"2024-04-13T03:17:35.910885+0000","last_undegraded":"2024-04-13T03:17:35.910885+0000","last_fullsized":"2024-04-13T03:17:35.910885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:14.389147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395409+0000","last_change":"2024-04-13T03:17:33.400583+0000","last_active":"2024-04-13T03:17:35.395409+0000","last_peered":"2024-04-13T03:17:35.395409+0000","last_clean":"2024-04-13T03:17:35.395409+0000","last_became_active":"2024-04-13T03:17:33.400299+0000","last_became_peered":"2024-04-13T03:17:33.400299+0000","last_unstale":"2024-04-13T03:17:35.395409+0000","last_undegraded":"2024-04-13T03:17:35.395409+0000","last_fullsized":"2024-04-13T03:17:35.395409+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:40:01.687395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395717+0000","last_change":"2024-04-13T03:17:34.402541+0000","last_active":"2024-04-13T03:17:35.395717+0000","last_peered":"2024-04-13T03:17:35.395717+0000","last_clean":"2024-04-13T03:17:35.395717+0000","last_became_active":"2024-04-13T03:17:34.402071+0000","last_became_peered":"2024-04-13T03:17:34.402071+0000","last_unstale":"2024-04-13T03:17:35.395717+0000","last_undegraded":"2024-04-13T03:17:35.395717+0000","last_fullsized":"2024-04-13T03:17:35.395717+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:57:32.774945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390306+0000","last_change":"2024-04-13T03:17:33.401851+0000","last_active":"2024-04-13T03:17:35.390306+0000","last_peered":"2024-04-13T03:17:35.390306+0000","last_clean":"2024-04-13T03:17:35.390306+0000","last_became_active":"2024-04-13T03:17:33.401591+0000","last_became_peered":"2024-04-13T03:17:33.401591+0000","last_unstale":"2024-04-13T03:17:35.390306+0000","last_undegraded":"2024-04-13T03:17:35.390306+0000","last_fullsized":"2024-04-13T03:17:35.390306+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:12:31.962483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395857+0000","last_change":"2024-04-13T03:17:34.400737+0000","last_active":"2024-04-13T03:17:35.395857+0000","last_peered":"2024-04-13T03:17:35.395857+0000","last_clean":"2024-04-13T03:17:35.395857+0000","last_became_active":"2024-04-13T03:17:34.400491+0000","last_became_peered":"2024-04-13T03:17:34.400491+0000","last_unstale":"2024-04-13T03:17:35.395857+0000","last_undegraded":"2024-04-13T03:17:35.395857+0000","last_fullsized":"2024-04-13T03:17:35.395857+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:43.760714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389789+0000","last_change":"2024-04-13T03:17:33.401061+0000","last_active":"2024-04-13T03:17:35.389789+0000","last_peered":"2024-04-13T03:17:35.389789+0000","last_clean":"2024-04-13T03:17:35.389789+0000","last_became_active":"2024-04-13T03:17:33.400750+0000","last_became_peered":"2024-04-13T03:17:33.400750+0000","last_unstale":"2024-04-13T03:17:35.389789+0000","last_undegraded":"2024-04-13T03:17:35.389789+0000","last_fullsized":"2024-04-13T03:17:35.389789+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:08:22.455340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480880+0000","last_change":"2024-04-13T03:17:34.407766+0000","last_active":"2024-04-13T03:17:35.480880+0000","last_peered":"2024-04-13T03:17:35.480880+0000","last_clean":"2024-04-13T03:17:35.480880+0000","last_became_active":"2024-04-13T03:17:34.407522+0000","last_became_peered":"2024-04-13T03:17:34.407522+0000","last_unstale":"2024-04-13T03:17:35.480880+0000","last_undegraded":"2024-04-13T03:17:35.480880+0000","last_fullsized":"2024-04-13T03:17:35.480880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:58:46.910362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065961+0000","last_change":"2024-04-13T03:17:33.406834+0000","last_active":"2024-04-13T03:17:36.065961+0000","last_peered":"2024-04-13T03:17:36.065961+0000","last_clean":"2024-04-13T03:17:36.065961+0000","last_became_active":"2024-04-13T03:17:33.406592+0000","last_became_peered":"2024-04-13T03:17:33.406592+0000","last_unstale":"2024-04-13T03:17:36.065961+0000","last_undegraded":"2024-04-13T03:17:36.065961+0000","last_fullsized":"2024-04-13T03:17:36.065961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:55:26.429151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390170+0000","last_change":"2024-04-13T03:17:34.411881+0000","last_active":"2024-04-13T03:17:35.390170+0000","last_peered":"2024-04-13T03:17:35.390170+0000","last_clean":"2024-04-13T03:17:35.390170+0000","last_became_active":"2024-04-13T03:17:34.411623+0000","last_became_peered":"2024-04-13T03:17:34.411623+0000","last_unstale":"2024-04-13T03:17:35.390170+0000","last_undegraded":"2024-04-13T03:17:35.390170+0000","last_fullsized":"2024-04-13T03:17:35.390170+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:24:28.559176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390052+0000","last_change":"2024-04-13T03:17:33.400409+0000","last_active":"2024-04-13T03:17:35.390052+0000","last_peered":"2024-04-13T03:17:35.390052+0000","last_clean":"2024-04-13T03:17:35.390052+0000","last_became_active":"2024-04-13T03:17:33.400146+0000","last_became_peered":"2024-04-13T03:17:33.400146+0000","last_unstale":"2024-04-13T03:17:35.390052+0000","last_undegraded":"2024-04-13T03:17:35.390052+0000","last_fullsized":"2024-04-13T03:17:35.390052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:51.101678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395449+0000","last_change":"2024-04-13T03:17:34.396324+0000","last_active":"2024-04-13T03:17:35.395449+0000","last_peered":"2024-04-13T03:17:35.395449+0000","last_clean":"2024-04-13T03:17:35.395449+0000","last_became_active":"2024-04-13T03:17:34.395910+0000","last_became_peered":"2024-04-13T03:17:34.395910+0000","last_unstale":"2024-04-13T03:17:35.395449+0000","last_undegraded":"2024-04-13T03:17:35.395449+0000","last_fullsized":"2024-04-13T03:17:35.395449+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:26:03.411503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910959+0000","last_change":"2024-04-13T03:17:33.399899+0000","last_active":"2024-04-13T03:17:35.910959+0000","last_peered":"2024-04-13T03:17:35.910959+0000","last_clean":"2024-04-13T03:17:35.910959+0000","last_became_active":"2024-04-13T03:17:33.399573+0000","last_became_peered":"2024-04-13T03:17:33.399573+0000","last_unstale":"2024-04-13T03:17:35.910959+0000","last_undegraded":"2024-04-13T03:17:35.910959+0000","last_fullsized":"2024-04-13T03:17:35.910959+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:02:59.848302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066415+0000","last_change":"2024-04-13T03:17:34.392085+0000","last_active":"2024-04-13T03:17:36.066415+0000","last_peered":"2024-04-13T03:17:36.066415+0000","last_clean":"2024-04-13T03:17:36.066415+0000","last_became_active":"2024-04-13T03:17:34.391813+0000","last_became_peered":"2024-04-13T03:17:34.391813+0000","last_unstale":"2024-04-13T03:17:36.066415+0000","last_undegraded":"2024-04-13T03:17:36.066415+0000","last_fullsized":"2024-04-13T03:17:36.066415+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:02:21.107968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395659+0000","last_change":"2024-04-13T03:17:33.393982+0000","last_active":"2024-04-13T03:17:35.395659+0000","last_peered":"2024-04-13T03:17:35.395659+0000","last_clean":"2024-04-13T03:17:35.395659+0000","last_became_active":"2024-04-13T03:17:33.393615+0000","last_became_peered":"2024-04-13T03:17:33.393615+0000","last_unstale":"2024-04-13T03:17:35.395659+0000","last_undegraded":"2024-04-13T03:17:35.395659+0000","last_fullsized":"2024-04-13T03:17:35.395659+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:35:20.641845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.480738+0000","last_change":"2024-04-13T03:17:34.406398+0000","last_active":"2024-04-13T03:17:35.480738+0000","last_peered":"2024-04-13T03:17:35.480738+0000","last_clean":"2024-04-13T03:17:35.480738+0000","last_became_active":"2024-04-13T03:17:34.406118+0000","last_became_peered":"2024-04-13T03:17:34.406118+0000","last_unstale":"2024-04-13T03:17:35.480738+0000","last_undegraded":"2024-04-13T03:17:35.480738+0000","last_fullsized":"2024-04-13T03:17:35.480738+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:39:49.097723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066129+0000","last_change":"2024-04-13T03:17:33.400700+0000","last_active":"2024-04-13T03:17:36.066129+0000","last_peered":"2024-04-13T03:17:36.066129+0000","last_clean":"2024-04-13T03:17:36.066129+0000","last_became_active":"2024-04-13T03:17:33.400443+0000","last_became_peered":"2024-04-13T03:17:33.400443+0000","last_unstale":"2024-04-13T03:17:36.066129+0000","last_undegraded":"2024-04-13T03:17:36.066129+0000","last_fullsized":"2024-04-13T03:17:36.066129+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:40:56.254824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395197+0000","last_change":"2024-04-13T03:17:34.416879+0000","last_active":"2024-04-13T03:17:35.395197+0000","last_peered":"2024-04-13T03:17:35.395197+0000","last_clean":"2024-04-13T03:17:35.395197+0000","last_became_active":"2024-04-13T03:17:34.416615+0000","last_became_peered":"2024-04-13T03:17:34.416615+0000","last_unstale":"2024-04-13T03:17:35.395197+0000","last_undegraded":"2024-04-13T03:17:35.395197+0000","last_fullsized":"2024-04-13T03:17:35.395197+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:09.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.910943+0000","last_change":"2024-04-13T03:17:33.402668+0000","last_active":"2024-04-13T03:17:35.910943+0000","last_peered":"2024-04-13T03:17:35.910943+0000","last_clean":"2024-04-13T03:17:35.910943+0000","last_became_active":"2024-04-13T03:17:33.402363+0000","last_became_peered":"2024-04-13T03:17:33.402363+0000","last_unstale":"2024-04-13T03:17:35.910943+0000","last_undegraded":"2024-04-13T03:17:35.910943+0000","last_fullsized":"2024-04-13T03:17:35.910943+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:45.226601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390104+0000","last_change":"2024-04-13T03:17:34.403801+0000","last_active":"2024-04-13T03:17:35.390104+0000","last_peered":"2024-04-13T03:17:35.390104+0000","last_clean":"2024-04-13T03:17:35.390104+0000","last_became_active":"2024-04-13T03:17:34.403541+0000","last_became_peered":"2024-04-13T03:17:34.403541+0000","last_unstale":"2024-04-13T03:17:35.390104+0000","last_undegraded":"2024-04-13T03:17:35.390104+0000","last_fullsized":"2024-04-13T03:17:35.390104+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:51:28.944577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065320+0000","last_change":"2024-04-13T03:17:33.401998+0000","last_active":"2024-04-13T03:17:36.065320+0000","last_peered":"2024-04-13T03:17:36.065320+0000","last_clean":"2024-04-13T03:17:36.065320+0000","last_became_active":"2024-04-13T03:17:33.401724+0000","last_became_peered":"2024-04-13T03:17:33.401724+0000","last_unstale":"2024-04-13T03:17:36.065320+0000","last_undegraded":"2024-04-13T03:17:36.065320+0000","last_fullsized":"2024-04-13T03:17:36.065320+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:11.989180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390069+0000","last_change":"2024-04-13T03:17:34.411296+0000","last_active":"2024-04-13T03:17:35.390069+0000","last_peered":"2024-04-13T03:17:35.390069+0000","last_clean":"2024-04-13T03:17:35.390069+0000","last_became_active":"2024-04-13T03:17:34.411035+0000","last_became_peered":"2024-04-13T03:17:34.411035+0000","last_unstale":"2024-04-13T03:17:35.390069+0000","last_undegraded":"2024-04-13T03:17:35.390069+0000","last_fullsized":"2024-04-13T03:17:35.390069+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:47:34.772093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.389916+0000","last_change":"2024-04-13T03:17:33.399760+0000","last_active":"2024-04-13T03:17:35.389916+0000","last_peered":"2024-04-13T03:17:35.389916+0000","last_clean":"2024-04-13T03:17:35.389916+0000","last_became_active":"2024-04-13T03:17:33.399508+0000","last_became_peered":"2024-04-13T03:17:33.399508+0000","last_unstale":"2024-04-13T03:17:35.389916+0000","last_undegraded":"2024-04-13T03:17:35.389916+0000","last_fullsized":"2024-04-13T03:17:35.389916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:14.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065695+0000","last_change":"2024-04-13T03:17:34.406246+0000","last_active":"2024-04-13T03:17:36.065695+0000","last_peered":"2024-04-13T03:17:36.065695+0000","last_clean":"2024-04-13T03:17:36.065695+0000","last_became_active":"2024-04-13T03:17:34.405992+0000","last_became_peered":"2024-04-13T03:17:34.405992+0000","last_unstale":"2024-04-13T03:17:36.065695+0000","last_undegraded":"2024-04-13T03:17:36.065695+0000","last_fullsized":"2024-04-13T03:17:36.065695+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:12.382037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.066108+0000","last_change":"2024-04-13T03:17:33.400564+0000","last_active":"2024-04-13T03:17:36.066108+0000","last_peered":"2024-04-13T03:17:36.066108+0000","last_clean":"2024-04-13T03:17:36.066108+0000","last_became_active":"2024-04-13T03:17:33.400283+0000","last_became_peered":"2024-04-13T03:17:33.400283+0000","last_unstale":"2024-04-13T03:17:36.066108+0000","last_undegraded":"2024-04-13T03:17:36.066108+0000","last_fullsized":"2024-04-13T03:17:36.066108+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:32:33.869508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395860+0000","last_change":"2024-04-13T03:17:34.413234+0000","last_active":"2024-04-13T03:17:35.395860+0000","last_peered":"2024-04-13T03:17:35.395860+0000","last_clean":"2024-04-13T03:17:35.395860+0000","last_became_active":"2024-04-13T03:17:34.412967+0000","last_became_peered":"2024-04-13T03:17:34.412967+0000","last_unstale":"2024-04-13T03:17:35.395860+0000","last_undegraded":"2024-04-13T03:17:35.395860+0000","last_fullsized":"2024-04-13T03:17:35.395860+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:13.036448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390152+0000","last_change":"2024-04-13T03:17:33.399229+0000","last_active":"2024-04-13T03:17:35.390152+0000","last_peered":"2024-04-13T03:17:35.390152+0000","last_clean":"2024-04-13T03:17:35.390152+0000","last_became_active":"2024-04-13T03:17:33.398953+0000","last_became_peered":"2024-04-13T03:17:33.398953+0000","last_unstale":"2024-04-13T03:17:35.390152+0000","last_undegraded":"2024-04-13T03:17:35.390152+0000","last_fullsized":"2024-04-13T03:17:35.390152+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:26:04.844331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.395150+0000","last_change":"2024-04-13T03:17:34.410479+0000","last_active":"2024-04-13T03:17:35.395150+0000","last_peered":"2024-04-13T03:17:35.395150+0000","last_clean":"2024-04-13T03:17:35.395150+0000","last_became_active":"2024-04-13T03:17:34.410102+0000","last_became_peered":"2024-04-13T03:17:34.410102+0000","last_unstale":"2024-04-13T03:17:35.395150+0000","last_undegraded":"2024-04-13T03:17:35.395150+0000","last_fullsized":"2024-04-13T03:17:35.395150+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:33.377189+0000","last_clean_scrub_stamp":"2024-04-13T03:17:33.377189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:42:02.481972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:35.390239+0000","last_change":"2024-04-13T03:17:33.401216+0000","last_active":"2024-04-13T03:17:35.390239+0000","last_peered":"2024-04-13T03:17:35.390239+0000","last_clean":"2024-04-13T03:17:35.390239+0000","last_became_active":"2024-04-13T03:17:33.400900+0000","last_became_peered":"2024-04-13T03:17:33.400900+0000","last_unstale":"2024-04-13T03:17:35.390239+0000","last_undegraded":"2024-04-13T03:17:35.390239+0000","last_fullsized":"2024-04-13T03:17:35.390239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:17:32.372106+0000","last_clean_scrub_stamp":"2024-04-13T03:17:32.372106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:54:39.812442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":331,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-13T03:17:36.065126+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:17:36.065126+0000","last_peered":"2024-04-13T03:17:36.065126+0000","last_clean":"2024-04-13T03:17:36.065126+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:17:36.065126+0000","last_undegraded":"2024-04-13T03:17:36.065126+0000","last_fullsized":"2024-04-13T03:17:36.065126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":1208712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":620400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":618184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":618276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":618218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1210254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:17:42.718 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:17:42.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:17:42.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.861+0000 7f66e9d1e640 1 Processor -- start 2024-04-13T03:17:42.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.861+0000 7f66e9d1e640 1 -- start start 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e9d1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 0x7f66e40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e9d1e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66e415eca0 0x7f66e40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e9d1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e40b5060 con 0x7f66e415e8c0 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e9d1e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f66e40b5340 con 0x7f66e415eca0 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e9d1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f66e406c500 con 0x7f66e4073520 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66e415eca0 0x7f66e40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66e415eca0 0x7f66e40b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35448/0 (socket says 172.21.15.106:35448) 2024-04-13T03:17:42.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 -- 172.21.15.106:0/4282473476 learned_addr learned my addr 172.21.15.106:0/4282473476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e2ffd640 1 --2- 172.21.15.106:0/4282473476 >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 0x7f66e40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 -- 172.21.15.106:0/4282473476 >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 msgr2=0x7f66e40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 --2- 172.21.15.106:0/4282473476 >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 0x7f66e40ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.862+0000 7f66e3fff640 1 -- 172.21.15.106:0/4282473476 >> v1:172.21.15.114:6789/0 conn(0x7f66e415e8c0 legacy=0x7f66e4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e3fff640 1 -- 172.21.15.106:0/4282473476 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e406c7e0 con 0x7f66e415eca0 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e2ffd640 1 --2- 172.21.15.106:0/4282473476 >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 0x7f66e40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e3fff640 1 --2- 172.21.15.106:0/4282473476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415eca0 0x7f66e40b4430 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f66e4072ae0 tx=0x7f66dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800165223729f0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e27fc640 1 -- 172.21.15.106:0/4282473476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc034070 con 0x7f66e415eca0 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e27fc640 1 -- 172.21.15.106:0/4282473476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc030860 con 0x7f66e415eca0 2024-04-13T03:17:42.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.863+0000 7f66e27fc640 1 -- 172.21.15.106:0/4282473476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc030c30 con 0x7f66e415eca0 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 -- 172.21.15.106:0/4282473476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415eca0 msgr2=0x7f66e40b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/4282473476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415eca0 0x7f66e40b4430 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f66e4072ae0 tx=0x7f66dc02b9a0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 -- 172.21.15.106:0/4282473476 shutdown_connections 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/4282473476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415eca0 0x7f66e40b4430 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/4282473476 >> v2:172.21.15.114:3300/0 conn(0x7f66e4073520 0x7f66e40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 -- 172.21.15.106:0/4282473476 >> 172.21.15.106:0/4282473476 conn(0x7f66e406f0d0 msgr2=0x7f66e406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 -- 172.21.15.106:0/4282473476 shutdown_connections 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.864+0000 7f66e9d1e640 1 -- 172.21.15.106:0/4282473476 wait complete. 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.865+0000 7f66e9d1e640 1 Processor -- start 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.865+0000 7f66e9d1e640 1 -- start start 2024-04-13T03:17:42.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66e4073520 0x7f66e41f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 0x7f66e40ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40b4d40 con 0x7f66e415e8c0 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40b4bc0 con 0x7f66e415eca0 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e9d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66e406cd00 con 0x7f66e4073520 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35462/0 (socket says 172.21.15.106:35462) 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 -- 172.21.15.106:0/2771508169 learned_addr learned my addr 172.21.15.106:0/2771508169 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e3fff640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 0x7f66e40ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e37fe640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66e4073520 0x7f66e41f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 -- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66e4073520 msgr2=0x7f66e41f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66e4073520 0x7f66e41f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 -- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 msgr2=0x7f66e40ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 0x7f66e40ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e2ffd640 1 -- 172.21.15.106:0/2771508169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e40cc180 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.866+0000 7f66e3fff640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 0x7f66e40ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66e2ffd640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c7f0 tx=0x7f66cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc015070 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc007a30 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e40cc460 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc007e00 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.867+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e40cca20 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.868+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e4068a10 con 0x7f66e415e8c0 2024-04-13T03:17:42.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.868+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66cc018070 con 0x7f66e415e8c0 2024-04-13T03:17:42.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.869+0000 7f66dbfff640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:42.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.869+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f66cc08bce0 con 0x7f66e415e8c0 2024-04-13T03:17:42.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.869+0000 7f66e37fe640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:42.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.869+0000 7f66e37fe640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 0x7f66c406edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005520 tx=0x7f66d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:42.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:42.873+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66cc059f90 con 0x7f66e415e8c0 2024-04-13T03:17:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.120+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f66e40cd710 con 0x7f66e415e8c0 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.122+0000 7f66dbfff640 1 -- 172.21.15.106:0/2771508169 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f66cc059890 con 0x7f66e415e8c0 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:17:43.123 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:17:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 msgr2=0x7f66c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 0x7f66c406edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005520 tx=0x7f66d4009040 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 msgr2=0x7f66e41562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c7f0 tx=0x7f66cc00cbf0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 shutdown_connections 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e415eca0 0x7f66e40ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66e415e8c0 0x7f66e41562a0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 --2- 172.21.15.106:0/2771508169 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66e4073520 0x7f66e41f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.124+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 >> 172.21.15.106:0/2771508169 conn(0x7f66e406f0d0 msgr2=0x7f66e406dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.125+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 shutdown_connections 2024-04-13T03:17:43.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.125+0000 7f66e9d1e640 1 -- 172.21.15.106:0/2771508169 wait complete. 2024-04-13T03:17:43.136 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:43.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.235+0000 7fb349ad5640 1 Processor -- start 2024-04-13T03:17:43.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 -- start start 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb34415c7d0 0x7fb344160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3440c60a0 0x7fb3440d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3440d2310 con 0x7fb3440ccba0 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440b1f30 con 0x7fb34415c7d0 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb349ad5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440b2210 con 0x7fb3440c60a0 2024-04-13T03:17:43.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb3437fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb34415c7d0 0x7fb344160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb3437fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb34415c7d0 0x7fb344160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35478/0 (socket says 172.21.15.106:35478) 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.236+0000 7fb3437fe640 1 -- 172.21.15.106:0/826662575 learned_addr learned my addr 172.21.15.106:0/826662575 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb3437fe640 1 -- 172.21.15.106:0/826662575 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3440c60a0 msgr2=0x7fb3440d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb3437fe640 1 --2- 172.21.15.106:0/826662575 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3440c60a0 0x7fb3440d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb3437fe640 1 -- 172.21.15.106:0/826662575 >> v1:172.21.15.114:6789/0 conn(0x7fb3440ccba0 legacy=0x7fb3440b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb3437fe640 1 -- 172.21.15.106:0/826662575 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440b24f0 con 0x7fb34415c7d0 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb3437fe640 1 --2- 172.21.15.106:0/826662575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34415c7d0 0x7fb344160dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb334009870 tx=0x7fb33402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97ec145bb8e1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.237+0000 7fb342ffd640 1 -- 172.21.15.106:0/826662575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334035020 con 0x7fb34415c7d0 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb342ffd640 1 -- 172.21.15.106:0/826662575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334031640 con 0x7fb34415c7d0 2024-04-13T03:17:43.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb342ffd640 1 -- 172.21.15.106:0/826662575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33402d040 con 0x7fb34415c7d0 2024-04-13T03:17:43.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 -- 172.21.15.106:0/826662575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34415c7d0 msgr2=0x7fb344160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 --2- 172.21.15.106:0/826662575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34415c7d0 0x7fb344160dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb334009870 tx=0x7fb33402b8f0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 -- 172.21.15.106:0/826662575 shutdown_connections 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 --2- 172.21.15.106:0/826662575 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3440c60a0 0x7fb3440d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 --2- 172.21.15.106:0/826662575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb34415c7d0 0x7fb344160dd0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 -- 172.21.15.106:0/826662575 >> 172.21.15.106:0/826662575 conn(0x7fb3440d2550 msgr2=0x7fb3440d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 -- 172.21.15.106:0/826662575 shutdown_connections 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.238+0000 7fb349ad5640 1 -- 172.21.15.106:0/826662575 wait complete. 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.239+0000 7fb349ad5640 1 Processor -- start 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.239+0000 7fb349ad5640 1 -- start start 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3440ccba0 0x7fb34415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb34415c7d0 0x7fb34415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb34415cbb0 con 0x7fb3440c60a0 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440cd010 con 0x7fb3440ccba0 2024-04-13T03:17:43.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb349ad5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440caf50 con 0x7fb34415c7d0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35486/0 (socket says 172.21.15.106:35486) 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb3437fe640 1 -- 172.21.15.106:0/1232323827 learned_addr learned my addr 172.21.15.106:0/1232323827 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb343fff640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb34415c7d0 0x7fb34415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb33affd640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3440ccba0 0x7fb34415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb3437fe640 1 -- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb34415c7d0 msgr2=0x7fb34415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.240+0000 7fb3437fe640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb34415c7d0 0x7fb34415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb3437fe640 1 -- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3440ccba0 msgr2=0x7fb34415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb3437fe640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3440ccba0 0x7fb34415f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb3437fe640 1 -- 172.21.15.106:0/1232323827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440cb1a0 con 0x7fb3440c60a0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb3437fe640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb33402b930 tx=0x7fb3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33403ac60 con 0x7fb3440c60a0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33403ae20 con 0x7fb3440c60a0 2024-04-13T03:17:43.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334044630 con 0x7fb3440c60a0 2024-04-13T03:17:43.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3440cb480 con 0x7fb3440c60a0 2024-04-13T03:17:43.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.241+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3440cba40 con 0x7fb3440c60a0 2024-04-13T03:17:43.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.242+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb344069120 con 0x7fb3440c60a0 2024-04-13T03:17:43.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.242+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb334038020 con 0x7fb3440c60a0 2024-04-13T03:17:43.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.243+0000 7fb340ff9640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 0x7fb32406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.243+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb3340ad830 con 0x7fb3440c60a0 2024-04-13T03:17:43.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.243+0000 7fb33affd640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 0x7fb32406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.243+0000 7fb33affd640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 0x7fb32406edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb3440b3ed0 tx=0x7fb32c009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.247+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb33403f070 con 0x7fb3440c60a0 2024-04-13T03:17:43.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.457+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3440cad30 con 0x7fb3440c60a0 2024-04-13T03:17:43.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.457+0000 7fb340ff9640 1 -- 172.21.15.106:0/1232323827 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5195 (secure 0 0 0) 0x7fb33407b630 con 0x7fb3440c60a0 2024-04-13T03:17:43.459 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:43.459 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:17:36.367064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8430":{"gid":8430,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/2645971931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2645971931},{"type":"v1","addr":"172.21.15.114:6835","nonce":2645971931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8430,"qdb_cluster":[8430]},"id":7}]} 2024-04-13T03:17:43.459 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 71 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 msgr2=0x7fb32406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 0x7fb32406edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb3440b3ed0 tx=0x7fb32c009110 comp rx=0 tx=0).stop 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 msgr2=0x7fb3440cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb33402b930 tx=0x7fb3340046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 shutdown_connections 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb32406c8f0 0x7fb32406edb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb34415c7d0 0x7fb34415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3440ccba0 0x7fb34415f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 --2- 172.21.15.106:0/1232323827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3440c60a0 0x7fb3440cc240 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 >> 172.21.15.106:0/1232323827 conn(0x7fb3440d2550 msgr2=0x7fb34406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:43.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 shutdown_connections 2024-04-13T03:17:43.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.459+0000 7fb349ad5640 1 -- 172.21.15.106:0/1232323827 wait complete. 2024-04-13T03:17:43.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:17:33.740516+0000', 'modified': '2024-04-13T03:17:36.367064+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8430}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8430': {'gid': 8430, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6835/2645971931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 2645971931}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 2645971931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8430, 'qdb_cluster': [8430]} 2024-04-13T03:17:43.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:17:43.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.977+0000 7f354c820640 1 Processor -- start 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.977+0000 7f354c820640 1 -- start start 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354c820640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f35481592b0 0x7f354815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354c820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f354815c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354c820640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354806d180 con 0x7f354815c5d0 2024-04-13T03:17:43.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354c820640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f35480ce180 con 0x7f35481592b0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354c820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f354806ec90 con 0x7f354806e740 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f3546d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f354815c5d0 0x7f354815f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58740/0 (socket says 172.21.15.106:58740) 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f3546d9d640 1 -- 172.21.15.106:0/469027358 learned_addr learned my addr 172.21.15.106:0/469027358 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f3545d9b640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f354815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354659c640 1 --2- 172.21.15.106:0/469027358 >> v2:172.21.15.114:3300/0 conn(0x7f35481592b0 0x7f354815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.978+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091872846 0 0) 0x7f354806d180 con 0x7f354815c5d0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3530005730 con 0x7f354815c5d0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1676263784 0 0) 0x7f3530005730 con 0x7f354815c5d0 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 msgr2=0x7f354815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f354815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 >> v2:172.21.15.114:3300/0 conn(0x7f35481592b0 msgr2=0x7f354815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 --2- 172.21.15.106:0/469027358 >> v2:172.21.15.114:3300/0 conn(0x7f35481592b0 0x7f354815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354806ef70 con 0x7f354815c5d0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f3540001670 con 0x7f354815c5d0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 >> v1:172.21.15.106:6789/0 conn(0x7f354815c5d0 legacy=0x7f354815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3530005bc0 0x7f3530005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f3530006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f35300073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35480ce180 con 0x7f3530006a10 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f354806ec90 con 0x7f3530005bc0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f353000dbd0 con 0x7f354806e740 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f3530006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f354659c640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3530005bc0 0x7f3530005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3546d9d640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f35300073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 -- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 msgr2=0x7f35300073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f354806e740 0x7f35300073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 -- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3530005bc0 msgr2=0x7f3530005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3530005bc0 0x7f3530005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.979+0000 7f3545d9b640 1 -- 172.21.15.106:0/469027358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3530005730 con 0x7f3530006a10 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.980+0000 7f3545d9b640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f3530006e90 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f353c009ac0 tx=0x7f3530007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45ffcdd2227eb46 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.980+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c039070 con 0x7f3530006a10 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.980+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c02c6b0 con 0x7f3530006a10 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.980+0000 7f354559a640 1 -- 172.21.15.106:0/469027358 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c009690 con 0x7f3530006a10 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 -- 172.21.15.106:0/469027358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 msgr2=0x7f3530006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 --2- 172.21.15.106:0/469027358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f3530006e90 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f353c009ac0 tx=0x7f3530007a40 comp rx=0 tx=0).stop 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f3546d9d640 1 -- 172.21.15.106:0/469027358 reap_dead start 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 -- 172.21.15.106:0/469027358 shutdown_connections 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 -- 172.21.15.106:0/469027358 >> 172.21.15.106:0/469027358 conn(0x7f35480b2600 msgr2=0x7f35480d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 -- 172.21.15.106:0/469027358 shutdown_connections 2024-04-13T03:17:43.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.981+0000 7f354c820640 1 -- 172.21.15.106:0/469027358 wait complete. 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.982+0000 7f354c820640 1 Processor -- start 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.982+0000 7f354c820640 1 -- start start 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35480ab0a0 0x7f354806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35480ab520 0x7f354806be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35480cf630 con 0x7f3530006a10 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f354806f530 con 0x7f35480ab0a0 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35480cf7b0 con 0x7f35480ab520 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35504/0 (socket says 172.21.15.106:35504) 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 -- 172.21.15.106:0/2489851164 learned_addr learned my addr 172.21.15.106:0/2489851164 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f3545d9b640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35480ab0a0 0x7f354806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 -- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35480ab520 msgr2=0x7f354806be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35480ab520 0x7f354806be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 -- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35480ab0a0 msgr2=0x7f354806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35480ab0a0 0x7f354806b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 -- 172.21.15.106:0/2489851164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354814b1a0 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354659c640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3538009f90 tx=0x7f353800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f3536ffd640 1 -- 172.21.15.106:0/2489851164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538015430 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f3536ffd640 1 -- 172.21.15.106:0/2489851164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35380155d0 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35480a9580 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f3536ffd640 1 -- 172.21.15.106:0/2489851164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35380159c0 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.983+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35480a9b40 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.985+0000 7f3536ffd640 1 -- 172.21.15.106:0/2489851164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3538015b60 con 0x7f3530006a10 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.985+0000 7f3536ffd640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 0x7f352806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.986+0000 7f3536ffd640 1 -- 172.21.15.106:0/2489851164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f353808ce00 con 0x7f3530006a10 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.986+0000 7f3545d9b640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 0x7f352806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.986+0000 7f3545d9b640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 0x7f352806ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f353c036000 tx=0x7f353c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.986+0000 7f354f608740 1 Processor -- start 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 -- start start 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c9960 0x55ccd81c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81813d0 0x55ccd81abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ccd80e6990 con 0x55ccd81a91a0 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ccd808d620 con 0x55ccd81c9960 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ccd80e2950 con 0x55ccd81813d0 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f3545d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f3545d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35516/0 (socket says 172.21.15.106:35516) 2024-04-13T03:17:43.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.987+0000 7f3545d9b640 1 -- 172.21.15.106:0/295119994 learned_addr learned my addr 172.21.15.106:0/295119994 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f354659c640 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c9960 0x55ccd81c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3546d9d640 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81813d0 0x55ccd81abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 -- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81813d0 msgr2=0x55ccd81abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81813d0 0x55ccd81abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 -- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c9960 msgr2=0x55ccd81c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c9960 0x55ccd81c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 -- 172.21.15.106:0/295119994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd7df92d0 con 0x55ccd81a91a0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3545d9b640 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f353c009bb0 tx=0x7f353c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da6ac19bc4128b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3527fff640 1 -- 172.21.15.106:0/295119994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c050070 con 0x55ccd81a91a0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3527fff640 1 -- 172.21.15.106:0/295119994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c02d840 con 0x55ccd81a91a0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.988+0000 7f3527fff640 1 -- 172.21.15.106:0/295119994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c039040 con 0x55ccd81a91a0 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- 172.21.15.106:0/295119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 msgr2=0x55ccd8180e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f353c009bb0 tx=0x7f353c02e000 comp rx=0 tx=0).stop 2024-04-13T03:17:43.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- 172.21.15.106:0/295119994 shutdown_connections 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81813d0 0x55ccd81abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd8180e90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 --2- 172.21.15.106:0/295119994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c9960 0x55ccd81c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- 172.21.15.106:0/295119994 >> 172.21.15.106:0/295119994 conn(0x55ccd817dff0 msgr2=0x55ccd81c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- 172.21.15.106:0/295119994 shutdown_connections 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- 172.21.15.106:0/295119994 wait complete. 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 -- start start 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.989+0000 7f354f608740 1 Processor -- start 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ccd81802f0 con 0x55ccd81a91a0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ccd80a09e0 con 0x55ccd81c27a0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ccd817c620 con 0x55ccd81c9960 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354f608740 10 client.0 _ll_register_callbacks cb 0x55ccd80dc220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35528/0 (socket says 172.21.15.106:35528) 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.990+0000 7f354659c640 1 -- 172.21.15.106:0/3476635027 learned_addr learned my addr 172.21.15.106:0/3476635027 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f3545d9b640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f3546d9d640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 -- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 msgr2=0x55ccd81bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81bec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 -- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 msgr2=0x55ccd81bf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81bf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 -- 172.21.15.106:0/3476635027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd7ec36b0 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354659c640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f353800fa80 tx=0x7f3538063640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548111e67989a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538059ea0 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f353805a040 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353c02d840 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f3538084b00 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 10 client.8574 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55ccd81c1fa0 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c02d840 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f3538077080 con 0x55ccd81a91a0 2024-04-13T03:17:44.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 10 client.8574 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 10 client.8574 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 20 client.8574 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd81c3b30 0x55ccd81e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 4 client.8574 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.991+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccd80f43b0 con 0x55ccd81c3b30 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.992+0000 7f3545d9b640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd81c3b30 0x55ccd81e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.992+0000 7f3545d9b640 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd81c3b30 0x55ccd81e90e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1aea5389b0a5ede server_cookie=ce76bca470de1952 in_seq=0 out_seq=0 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.992+0000 7f35257fa640 10 client.8574 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f35257fa640 1 -- 172.21.15.106:0/3476635027 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccd80f43b0 con 0x55ccd81c3b30 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f35257fa640 10 client.8574 handle_command_reply: tid=0 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 client.8574 shutdown 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 msgr2=0x55ccd81c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f353800fa80 tx=0x7f3538063640 comp rx=0 tx=0).stop 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 shutdown_connections 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd81c3b30 0x55ccd81e90e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81bec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81bf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 --2- 172.21.15.106:0/3476635027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c5fe0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f35257fa640 0 client.8574 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 >> 172.21.15.106:0/3476635027 conn(0x55ccd81813d0 msgr2=0x55ccd817ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 shutdown_connections 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 1 -- 172.21.15.106:0/3476635027 wait complete. 2024-04-13T03:17:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:43.993+0000 7f354f608740 20 client.8574 trim_cache size 0 max 16384 2024-04-13T03:17:44.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.011+0000 7f354f608740 1 Processor -- start 2024-04-13T03:17:44.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.011+0000 7f354f608740 1 -- start start 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81a91a0 0x55ccd817c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35481fc750 con 0x55ccd81c9960 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35480adf60 con 0x55ccd81c27a0 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35481fc0e0 con 0x55ccd81a91a0 2024-04-13T03:17:44.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35532/0 (socket says 172.21.15.106:35532) 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 -- 172.21.15.106:0/948405643 learned_addr learned my addr 172.21.15.106:0/948405643 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 -- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81a91a0 msgr2=0x55ccd817c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81a91a0 0x55ccd817c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 -- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 msgr2=0x55ccd81817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 -- 172.21.15.106:0/948405643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd81c6a90 con 0x55ccd81c9960 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f3546d9d640 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f3540001700 tx=0x7f3540004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e75c420f1a2e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f35277fe640 1 -- 172.21.15.106:0/948405643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540015440 con 0x55ccd81c9960 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f35277fe640 1 -- 172.21.15.106:0/948405643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35400155e0 con 0x55ccd81c9960 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.012+0000 7f35277fe640 1 -- 172.21.15.106:0/948405643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35400159b0 con 0x55ccd81c9960 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- 172.21.15.106:0/948405643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 msgr2=0x55ccd8181cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f3540001700 tx=0x7f3540004bb0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- 172.21.15.106:0/948405643 shutdown_connections 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81c9960 0x55ccd8181cf0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 --2- 172.21.15.106:0/948405643 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81a91a0 0x55ccd817c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- 172.21.15.106:0/948405643 >> 172.21.15.106:0/948405643 conn(0x55ccd81813d0 msgr2=0x55ccd817ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- 172.21.15.106:0/948405643 shutdown_connections 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- 172.21.15.106:0/948405643 wait complete. 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.013+0000 7f354f608740 1 -- start start 2024-04-13T03:17:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 Processor -- start 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ccd81c39b0 con 0x55ccd81a91a0 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ccd81c6630 con 0x55ccd81c27a0 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55ccd81c3f70 con 0x55ccd81c9960 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.014+0000 7f354f608740 10 client.0 _ll_register_callbacks cb 0x7f353805a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:44.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35534/0 (socket says 172.21.15.106:35534) 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 -- 172.21.15.106:0/298380229 learned_addr learned my addr 172.21.15.106:0/298380229 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f3545d9b640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f3546d9d640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 -- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 msgr2=0x55ccd81c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 -- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 msgr2=0x55ccd81c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 -- 172.21.15.106:0/298380229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd81edbb0 con 0x55ccd81a91a0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354659c640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3538009420 tx=0x7f3538014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ce1a853b28c1be server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538081030 con 0x55ccd81a91a0 2024-04-13T03:17:44.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f35380567a0 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35400155e0 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f353805a820 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 10 client.8580 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55ccd8246350 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35400155e0 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f3538025030 con 0x55ccd81a91a0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 10 client.8580 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 10 client.8580 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.015+0000 7f354f608740 20 client.8580 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd8245000 0x55ccd82453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 4 client.8580 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "status"}) -- 0x55ccd81c5920 con 0x55ccd8245000 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f3545d9b640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd8245000 0x55ccd82453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f35257fa640 10 client.8580 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f3545d9b640 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd8245000 0x55ccd82453e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c79343a60c3b954 server_cookie=d2939fb66969f6be in_seq=0 out_seq=0 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f35257fa640 1 -- 172.21.15.106:0/298380229 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ccd81c5920 con 0x55ccd8245000 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f35257fa640 10 client.8580 handle_command_reply: tid=0 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 client.8580 shutdown 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 msgr2=0x55ccd81c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3538009420 tx=0x7f3538014000 comp rx=0 tx=0).stop 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 shutdown_connections 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55ccd8245000 0x55ccd82453e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55ccd81c9960 0x55ccd81c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ccd81c27a0 0x55ccd81c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f35257fa640 0 client.8580 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 --2- 172.21.15.106:0/298380229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccd81a91a0 0x55ccd81c6c60 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.016+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 >> 172.21.15.106:0/298380229 conn(0x55ccd81813d0 msgr2=0x55ccd81773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 shutdown_connections 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354f608740 1 -- 172.21.15.106:0/298380229 wait complete. 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354f608740 20 client.8580 trim_cache size 0 max 16384 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 8430, 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:17:44.025 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 9.6340823639999993, 2024-04-13T03:17:44.026 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 71, 2024-04-13T03:17:44.026 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 107, 2024-04-13T03:17:44.026 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 107, 2024-04-13T03:17:44.026 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 14.671221849 2024-04-13T03:17:44.026 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 msgr2=0x7f352806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 0x7f352806ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f353c036000 tx=0x7f353c030000 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 msgr2=0x7f354814ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3538009f90 tx=0x7f353800f580 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 shutdown_connections 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f352806c920 0x7f352806ede0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35480ab520 0x7f354806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35480ab0a0 0x7f354806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 --2- 172.21.15.106:0/2489851164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3530006a10 0x7f354814ac60 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 >> 172.21.15.106:0/2489851164 conn(0x7f35480b2600 msgr2=0x7f35480b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 shutdown_connections 2024-04-13T03:17:44.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.017+0000 7f354c820640 1 -- 172.21.15.106:0/2489851164 wait complete. 2024-04-13T03:17:44.030 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:17:44.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.128+0000 7f5773c2d640 1 Processor -- start 2024-04-13T03:17:44.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.128+0000 7f5773c2d640 1 -- start start 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f5773c2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f576c0b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f5773c2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f576c0c9140 0x7f576c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f5773c2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576c0b61a0 con 0x7f576c0d1460 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f5773c2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f576c0d0fd0 con 0x7f576c0c9140 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f5773c2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f576c0b5ce0 con 0x7f576c0b53c0 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57721ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f576c0d1460 0x7f576c0ccb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58748/0 (socket says 172.21.15.106:58748) 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57721ca640 1 -- 172.21.15.106:0/2335995715 learned_addr learned my addr 172.21.15.106:0/2335995715 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312036611 0 0) 0x7f576c0b61a0 con 0x7f576c0d1460 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5758003520 con 0x7f576c0d1460 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57711c8640 1 --2- 172.21.15.106:0/2335995715 >> v2:172.21.15.114:3300/0 conn(0x7f576c0c9140 0x7f576c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57719c9640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f576c0b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1295927957 0 0) 0x7f5758003520 con 0x7f576c0d1460 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 msgr2=0x7f576c0b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f576c0b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 >> v2:172.21.15.114:3300/0 conn(0x7f576c0c9140 msgr2=0x7f576c0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 --2- 172.21.15.106:0/2335995715 >> v2:172.21.15.114:3300/0 conn(0x7f576c0c9140 0x7f576c0b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c0d0100 con 0x7f576c0d1460 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f57640019f0 con 0x7f576c0d1460 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 >> v1:172.21.15.106:6789/0 conn(0x7f576c0d1460 legacy=0x7f576c0ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57580039b0 0x7f5758003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5758004800 0x7f5758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f57580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0d0fd0 con 0x7f5758004800 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0b5ce0 con 0x7f57580039b0 2024-04-13T03:17:44.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.129+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f575800b9c0 con 0x7f576c0b53c0 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5758004800 0x7f5758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57719c9640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57580039b0 0x7f5758003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57721ca640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f57580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 -- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 msgr2=0x7f57580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c0b53c0 0x7f57580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 -- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57580039b0 msgr2=0x7f5758003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57580039b0 0x7f5758003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.130+0000 7f57711c8640 1 -- 172.21.15.106:0/2335995715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5758003520 con 0x7f5758004800 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f57711c8640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5758004800 0x7f5758004c80 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f575c009a90 tx=0x7f5758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e9590f5409c5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c039070 con 0x7f5758004800 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c02c6b0 con 0x7f5758004800 2024-04-13T03:17:44.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f57709c7640 1 -- 172.21.15.106:0/2335995715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c009690 con 0x7f5758004800 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 -- 172.21.15.106:0/2335995715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5758004800 msgr2=0x7f5758004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 --2- 172.21.15.106:0/2335995715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5758004800 0x7f5758004c80 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f575c009a90 tx=0x7f5758005830 comp rx=0 tx=0).stop 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f57721ca640 1 -- 172.21.15.106:0/2335995715 reap_dead start 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 -- 172.21.15.106:0/2335995715 shutdown_connections 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 -- 172.21.15.106:0/2335995715 >> 172.21.15.106:0/2335995715 conn(0x7f576c0d2530 msgr2=0x7f576c0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 -- 172.21.15.106:0/2335995715 shutdown_connections 2024-04-13T03:17:44.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.131+0000 7f5773c2d640 1 -- 172.21.15.106:0/2335995715 wait complete. 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.132+0000 7f5773c2d640 1 Processor -- start 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 -- start start 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5758004800 0x7f576c14ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c15c3e0 0x7f576c1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0d0980 con 0x7f576c0ca3e0 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0d06a0 con 0x7f5758004800 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f5773c2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f576c0d0c60 con 0x7f576c15c3e0 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f57721ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f57721ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35550/0 (socket says 172.21.15.106:35550) 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.133+0000 7f57721ca640 1 -- 172.21.15.106:0/3413542862 learned_addr learned my addr 172.21.15.106:0/3413542862 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57719c9640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5758004800 0x7f576c14ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 -- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c15c3e0 msgr2=0x7f576c1616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57711c8640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c15c3e0 0x7f576c1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c15c3e0 0x7f576c1616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 -- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5758004800 msgr2=0x7f576c14ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5758004800 0x7f576c14ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 -- 172.21.15.106:0/3413542862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c161c10 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57721ca640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f576c0b70c0 tx=0x7f576400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576400edc0 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5764018070 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764013660 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c161ed0 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.134+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c162490 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.135+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c068a10 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.135+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5764013800 con 0x7f576c0ca3e0 2024-04-13T03:17:44.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.136+0000 7f57627fc640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 0x7f575006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:44.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.136+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f576408c160 con 0x7f576c0ca3e0 2024-04-13T03:17:44.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.136+0000 7f57719c9640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 0x7f575006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:44.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.136+0000 7f57719c9640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 0x7f575006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f5768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:44.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.140+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f576405a400 con 0x7f576c0ca3e0 2024-04-13T03:17:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.351+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f576c0d2050 con 0x7f576c0ca3e0 2024-04-13T03:17:44.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.352+0000 7f57627fc640 1 -- 172.21.15.106:0/3413542862 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f5764059d00 con 0x7f576c0ca3e0 2024-04-13T03:17:44.354 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 msgr2=0x7f575006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 0x7f575006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f5768008040 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 msgr2=0x7f576c0ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f576c0b70c0 tx=0x7f576400e530 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 shutdown_connections 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f575006c8f0 0x7f575006edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f576c15c3e0 0x7f576c1616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f576c0ca3e0 0x7f576c0ca860 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 --2- 172.21.15.106:0/3413542862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5758004800 0x7f576c14ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 >> 172.21.15.106:0/3413542862 conn(0x7f576c0d2530 msgr2=0x7f576c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:44.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 shutdown_connections 2024-04-13T03:17:44.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44.354+0000 7f5773c2d640 1 -- 172.21.15.106:0/3413542862 wait complete. 2024-04-13T03:17:44.366 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:17:44.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:17:44.367 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:44.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.368 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.383 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42085sec preferred_lft 42085sec 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.384 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:17:44.385 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:44.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.386 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:17:44.449 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:17:44.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.450 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:17:44.508 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:17:44.509 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:17:44.509 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:17:44.509 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:17:44.509 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:17:44.509 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:17:44.509 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:17:44.509 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:17:44.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:17:44.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:17:44.510 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:17:44.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:17:44.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:44.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.644 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:17:44.658 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:17:44.658 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:17:44.658 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:44.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.659 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:17:44.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:17:44.725 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:44.780 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:44.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:17:44.781 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:44.835 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:17:44.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.835 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:44.920 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:44.920 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:17:44.927 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:44.927 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:17:44.927 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:17:44.927 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:17:44.927 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:17:44.928 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:17:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:17:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:17:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:17:44.928 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:17:44.929 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:17:44.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:44.930 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:44.956 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:17:45.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:17:45.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:45.061 INFO:teuthology.orchestra.run.smithi106.stdout:client8589 2024-04-13T03:17:45.061 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:45.061 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:45.078 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:45.078 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:45.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:45.133 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-13T03:17:45.233 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records in 2024-04-13T03:17:45.234 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records out 2024-04-13T03:17:45.234 INFO:teuthology.orchestra.run.smithi106.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0408372 s, 154 MB/s 2024-04-13T03:17:45.234 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:45.234 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:45.250 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:45.250 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:45.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:45.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.443+0000 7f1681fdc640 1 Processor -- start 2024-04-13T03:17:45.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.443+0000 7f1681fdc640 1 -- start start 2024-04-13T03:17:45.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f1681fdc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f167c15ebf0 0x7f167c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f1681fdc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167c06f380 con 0x7f167c0ce1f0 2024-04-13T03:17:45.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f1681fdc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167c0ce6f0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f1681fdc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f167c15fa70 con 0x7f167c15ebf0 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f167c15ebf0 0x7f167c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f167c0ce1f0 0x7f167c06e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58754/0 (socket says 172.21.15.106:58754) 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167affd640 1 -- 172.21.15.106:0/2732280540 learned_addr learned my addr 172.21.15.106:0/2732280540 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167b7fe640 1 -- 172.21.15.106:0/2732280540 >> v1:172.21.15.114:6789/0 conn(0x7f167c0b5bf0 legacy=0x7f167c06ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167b7fe640 1 -- 172.21.15.106:0/2732280540 >> v1:172.21.15.106:6789/0 conn(0x7f167c0ce1f0 legacy=0x7f167c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.444+0000 7f167b7fe640 1 -- 172.21.15.106:0/2732280540 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c15f420 con 0x7f167c15ebf0 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.445+0000 7f167b7fe640 1 --2- 172.21.15.106:0/2732280540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c15ebf0 0x7f167c06f780 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097a0 tx=0x7f166c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557ee676a69ea00c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.445+0000 7f167a7fc640 1 -- 172.21.15.106:0/2732280540 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0315a0 con 0x7f167c15ebf0 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.445+0000 7f167a7fc640 1 -- 172.21.15.106:0/2732280540 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c037070 con 0x7f167c15ebf0 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.445+0000 7f167a7fc640 1 -- 172.21.15.106:0/2732280540 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c031960 con 0x7f167c15ebf0 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 -- 172.21.15.106:0/2732280540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c15ebf0 msgr2=0x7f167c06f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 --2- 172.21.15.106:0/2732280540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c15ebf0 0x7f167c06f780 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097a0 tx=0x7f166c02b840 comp rx=0 tx=0).stop 2024-04-13T03:17:45.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 -- 172.21.15.106:0/2732280540 shutdown_connections 2024-04-13T03:17:45.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 --2- 172.21.15.106:0/2732280540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c15ebf0 0x7f167c06f780 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 -- 172.21.15.106:0/2732280540 >> 172.21.15.106:0/2732280540 conn(0x7f167c0cd6d0 msgr2=0x7f167c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:45.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 -- 172.21.15.106:0/2732280540 shutdown_connections 2024-04-13T03:17:45.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.446+0000 7f1681fdc640 1 -- 172.21.15.106:0/2732280540 wait complete. 2024-04-13T03:17:45.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.447+0000 7f1681fdc640 1 Processor -- start 2024-04-13T03:17:45.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.447+0000 7f1681fdc640 1 -- start start 2024-04-13T03:17:45.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f167c0ce1f0 0x7f167c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f167c15ebf0 0x7f167c1e8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0b04d0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f167c15ff00 con 0x7f167c15ebf0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f1681fdc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f167c0b0650 con 0x7f167c0ce1f0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35562/0 (socket says 172.21.15.106:35562) 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 -- 172.21.15.106:0/473063677 learned_addr learned my addr 172.21.15.106:0/473063677 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167affd640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f167c0ce1f0 0x7f167c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167bfff640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f167c15ebf0 0x7f167c1e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 -- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f167c0ce1f0 msgr2=0x7f167c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f167c0ce1f0 0x7f167c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 -- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f167c15ebf0 msgr2=0x7f167c1e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f167c15ebf0 0x7f167c1e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.448+0000 7f167b7fe640 1 -- 172.21.15.106:0/473063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c06bc30 con 0x7f167c0b5bf0 2024-04-13T03:17:45.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f167b7fe640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098c0 tx=0x7f166c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c032840 con 0x7f167c0b5bf0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c0329e0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c032dd0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c06bf10 con 0x7f167c0b5bf0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.449+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c06c4d0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.450+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167c068a10 con 0x7f167c0b5bf0 2024-04-13T03:17:45.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.450+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f166c04d3f0 con 0x7f167c0b5bf0 2024-04-13T03:17:45.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.450+0000 7f1663fff640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 0x7f166806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.451+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f166c02f030 con 0x7f167c0b5bf0 2024-04-13T03:17:45.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.451+0000 7f167affd640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 0x7f166806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.451+0000 7f167affd640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 0x7f166806ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f167c161460 tx=0x7f1670007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.455+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f166c040070 con 0x7f167c0b5bf0 2024-04-13T03:17:45.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.630+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f167c15b140 con 0x7f167c0b5bf0 2024-04-13T03:17:45.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.631+0000 7f1663fff640 1 -- 172.21.15.106:0/473063677 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16920 (secure 0 0 0) 0x7f166c07b380 con 0x7f167c0b5bf0 2024-04-13T03:17:45.633 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:45.634 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":107,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:35.383741+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:45.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 msgr2=0x7f166806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 0x7f166806ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f167c161460 tx=0x7f1670007e80 comp rx=0 tx=0).stop 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 msgr2=0x7f167c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098c0 tx=0x7f166c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 shutdown_connections 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f166806c920 0x7f166806ede0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f167c0ce1f0 0x7f167c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f167c15ebf0 0x7f167c1e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 --2- 172.21.15.106:0/473063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f167c0b5bf0 0x7f167c06d320 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.633+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 >> 172.21.15.106:0/473063677 conn(0x7f167c0cd6d0 msgr2=0x7f167c06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.634+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 shutdown_connections 2024-04-13T03:17:45.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.634+0000 7f1681fdc640 1 -- 172.21.15.106:0/473063677 wait complete. 2024-04-13T03:17:45.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:17:31.965063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:17:32.694523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:45.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.748+0000 7f0fdbc29640 1 Processor -- start 2024-04-13T03:17:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.748+0000 7f0fdbc29640 1 -- start start 2024-04-13T03:17:45.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fdbc29640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fd415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fdbc29640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd40b5cf0 con 0x7f0fd4070bb0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fdbc29640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd40b5fd0 con 0x7f0fd40b54a0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fdbc29640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd40b62b0 con 0x7f0fd40b1c30 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fda1c6640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0fd4070bb0 0x7f0fd406d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58758/0 (socket says 172.21.15.106:58758) 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fda1c6640 1 -- 172.21.15.106:0/2117731406 learned_addr learned my addr 172.21.15.106:0/2117731406 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.749+0000 7f0fd99c5640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fd415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27090103 0 0) 0x7f0fd40b5cf0 con 0x7f0fd4070bb0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc0003520 con 0x7f0fd4070bb0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3206082074 0 0) 0x7f0fc0003520 con 0x7f0fd4070bb0 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 msgr2=0x7f0fd415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fd415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 >> v1:172.21.15.114:6789/0 conn(0x7f0fd40b54a0 legacy=0x7f0fd40704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd40b6590 con 0x7f0fd4070bb0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f0fcc0019f0 con 0x7f0fd4070bb0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 >> v1:172.21.15.106:6789/0 conn(0x7f0fd4070bb0 legacy=0x7f0fd406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00039e0 0x7f0fc0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fc0004d40 0x7f0fc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd40b62b0 con 0x7f0fc0004d40 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc000b9c0 con 0x7f0fc00039e0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc000bbc0 con 0x7f0fd40b1c30 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fc0004d40 0x7f0fc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd99c5640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00039e0 0x7f0fc0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fd91c4640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 -- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 msgr2=0x7f0fc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd40b1c30 0x7f0fc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 -- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00039e0 msgr2=0x7f0fc0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00039e0 0x7f0fc0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.750+0000 7f0fda1c6640 1 -- 172.21.15.106:0/2117731406 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc0003520 con 0x7f0fc0004d40 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.751+0000 7f0fda1c6640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fc0004d40 0x7f0fc00051c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00a860 tx=0x7f0fc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8592b706b0138fef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.751+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00a480 con 0x7f0fc0004d40 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.751+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc034630 con 0x7f0fc0004d40 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.751+0000 7f0fd89c3640 1 -- 172.21.15.106:0/2117731406 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc034a40 con 0x7f0fc0004d40 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 -- 172.21.15.106:0/2117731406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fc0004d40 msgr2=0x7f0fc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/2117731406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fc0004d40 0x7f0fc00051c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00a860 tx=0x7f0fc0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fda1c6640 1 -- 172.21.15.106:0/2117731406 reap_dead start 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 -- 172.21.15.106:0/2117731406 shutdown_connections 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 -- 172.21.15.106:0/2117731406 >> 172.21.15.106:0/2117731406 conn(0x7f0fd4071870 msgr2=0x7f0fd4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 -- 172.21.15.106:0/2117731406 shutdown_connections 2024-04-13T03:17:45.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.752+0000 7f0fdbc29640 1 -- 172.21.15.106:0/2117731406 wait complete. 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.753+0000 7f0fdbc29640 1 Processor -- start 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.753+0000 7f0fdbc29640 1 -- start start 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd4150d80 0x7f0fd4151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd40cff00 0x7f0fd40d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd415efc0 con 0x7f0fd4157e40 2024-04-13T03:17:45.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd415ee40 con 0x7f0fd40cff00 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd415f140 con 0x7f0fd4150d80 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35582/0 (socket says 172.21.15.106:35582) 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 -- 172.21.15.106:0/1663647453 learned_addr learned my addr 172.21.15.106:0/1663647453 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd99c5640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd4150d80 0x7f0fd4151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 -- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd4150d80 msgr2=0x7f0fd4151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd4150d80 0x7f0fd4151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 -- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd40cff00 msgr2=0x7f0fd40d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd40cff00 0x7f0fd40d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 -- 172.21.15.106:0/1663647453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4158620 con 0x7f0fd4157e40 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fd91c4640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000f820 tx=0x7f0fd000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00052e0 con 0x7f0fd4157e40 2024-04-13T03:17:45.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd0018070 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000d890 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd4158340 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.754+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41531a0 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.755+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4068a10 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.756+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fd000da30 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.756+0000 7f0fca7fc640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 0x7f0fb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.756+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f0fd008bad0 con 0x7f0fd4157e40 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.756+0000 7f0fd99c5640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 0x7f0fb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:45.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.757+0000 7f0fd99c5640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 0x7f0fb806ede0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4151ad0 tx=0x7f0fc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:45.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.761+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fd001a310 con 0x7f0fd4157e40 2024-04-13T03:17:45.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.935+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0fd406eb90 con 0x7f0fd4157e40 2024-04-13T03:17:45.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.935+0000 7f0fca7fc640 1 -- 172.21.15.106:0/1663647453 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f0fd0059e00 con 0x7f0fd4157e40 2024-04-13T03:17:45.937 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:45.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 msgr2=0x7f0fb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 0x7f0fb806ede0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4151ad0 tx=0x7f0fc4008040 comp rx=0 tx=0).stop 2024-04-13T03:17:45.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 msgr2=0x7f0fd40cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:45.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000f820 tx=0x7f0fd000fce0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 shutdown_connections 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0fb806c920 0x7f0fb806ede0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd40cff00 0x7f0fd40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4157e40 0x7f0fd40cc5f0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 --2- 172.21.15.106:0/1663647453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0fd4150d80 0x7f0fd4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 >> 172.21.15.106:0/1663647453 conn(0x7f0fd4071870 msgr2=0x7f0fd40704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 shutdown_connections 2024-04-13T03:17:45.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:45.937+0000 7f0fdbc29640 1 -- 172.21.15.106:0/1663647453 wait complete. 2024-04-13T03:17:45.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:17:46.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.052+0000 7f7d0c01c640 1 Processor -- start 2024-04-13T03:17:46.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.052+0000 7f7d0c01c640 1 -- start start 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0c01c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7d040ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0c01c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d04160470 con 0x7f7d04070c90 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0c01c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d04160750 con 0x7f7d040cc750 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0c01c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0415a770 con 0x7f7d0415a390 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0a5b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7d04070c90 0x7f7d040c6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58768/0 (socket says 172.21.15.106:58768) 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.053+0000 7f7d0a5b9640 1 -- 172.21.15.106:0/36780791 learned_addr learned my addr 172.21.15.106:0/36780791 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d09db8640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7d040ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106244147 0 0) 0x7f7d04160470 con 0x7f7d04070c90 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf0003520 con 0x7f7d04070c90 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1800966480 0 0) 0x7f7cf0003520 con 0x7f7d04070c90 2024-04-13T03:17:46.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 msgr2=0x7f7d040ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7d040ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 >> v1:172.21.15.114:6789/0 conn(0x7f7d040cc750 legacy=0x7f7d04070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04160a30 con 0x7f7d04070c90 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f7cfc0019f0 con 0x7f7d04070c90 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 >> v1:172.21.15.106:6789/0 conn(0x7f7d04070c90 legacy=0x7f7d040c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00039e0 0x7f7cf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7cf0004800 0x7f7cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7cf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf000ba70 con 0x7f7cf0004800 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf000bbf0 con 0x7f7cf00039e0 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf000bdf0 con 0x7f7d0415a390 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7cf0004800 0x7f7cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d09db8640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00039e0 0x7f7cf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d0a5b9640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7cf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 -- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 msgr2=0x7f7cf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d0415a390 0x7f7cf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 -- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00039e0 msgr2=0x7f7cf0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00039e0 0x7f7cf0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.054+0000 7f7d095b7640 1 -- 172.21.15.106:0/36780791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf0003520 con 0x7f7cf0004800 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.055+0000 7f7d095b7640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7cf0004800 0x7f7cf0004c80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4001720 tx=0x7f7cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4431c6f2af32e625 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.055+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4001370 con 0x7f7cf0004800 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.055+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf40052d0 con 0x7f7cf0004800 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.055+0000 7f7d08db6640 1 -- 172.21.15.106:0/36780791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4003830 con 0x7f7cf0004800 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 -- 172.21.15.106:0/36780791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7cf0004800 msgr2=0x7f7cf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/36780791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7cf0004800 0x7f7cf0004c80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4001720 tx=0x7f7cf0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0a5b9640 1 -- 172.21.15.106:0/36780791 reap_dead start 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 -- 172.21.15.106:0/36780791 shutdown_connections 2024-04-13T03:17:46.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 -- 172.21.15.106:0/36780791 >> 172.21.15.106:0/36780791 conn(0x7f7d040d1650 msgr2=0x7f7d040d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 -- 172.21.15.106:0/36780791 shutdown_connections 2024-04-13T03:17:46.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.056+0000 7f7d0c01c640 1 -- 172.21.15.106:0/36780791 wait complete. 2024-04-13T03:17:46.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.057+0000 7f7d0c01c640 1 Processor -- start 2024-04-13T03:17:46.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.057+0000 7f7d0c01c640 1 -- start start 2024-04-13T03:17:46.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf0004800 0x7f7d0414a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 0x7f7d040b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d040ccc80 con 0x7f7d040ca030 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04161080 con 0x7f7cf0004800 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0c01c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7d040cce00 con 0x7f7d040b1f90 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35590/0 (socket says 172.21.15.106:35590) 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 -- 172.21.15.106:0/1457148953 learned_addr learned my addr 172.21.15.106:0/1457148953 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d09db8640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf0004800 0x7f7d0414a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0a5b9640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 0x7f7d040b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 -- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 msgr2=0x7f7d040b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 0x7f7d040b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 -- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf0004800 msgr2=0x7f7d0414a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf0004800 0x7f7d0414a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 -- 172.21.15.106:0/1457148953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0415b840 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d0a5b9640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 0x7f7d040b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.058+0000 7f7d095b7640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4008fd0 tx=0x7f7cf4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.059+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4041070 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.059+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4005480 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.059+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041e7450 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.059+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4007a30 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.059+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d040caba0 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.060+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d04069120 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.060+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cf4044070 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.061+0000 7f7cfa7fc640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 0x7f7ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.061+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f7cf40ad810 con 0x7f7d040ca030 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.061+0000 7f7d09db8640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 0x7f7ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.061+0000 7f7d09db8640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 0x7f7ce806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009bc0 tx=0x7f7d00009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.065+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cf407bab0 con 0x7f7d040ca030 2024-04-13T03:17:46.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.240+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d040c6980 con 0x7f7d040ca030 2024-04-13T03:17:46.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.240+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1457148953 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f7cf407b3b0 con 0x7f7d040ca030 2024-04-13T03:17:46.242 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 msgr2=0x7f7ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 0x7f7ce806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009bc0 tx=0x7f7d00009190 comp rx=0 tx=0).stop 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 msgr2=0x7f7d040ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4008fd0 tx=0x7f7cf4001120 comp rx=0 tx=0).stop 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 shutdown_connections 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7ce806c8f0 0x7f7ce806edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7d040b1f90 0x7f7d040b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d040ca030 0x7f7d040ca490 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 --2- 172.21.15.106:0/1457148953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf0004800 0x7f7d0414a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 >> 172.21.15.106:0/1457148953 conn(0x7f7d040d1650 msgr2=0x7f7d040c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 shutdown_connections 2024-04-13T03:17:46.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.242+0000 7f7d0c01c640 1 -- 172.21.15.106:0/1457148953 wait complete. 2024-04-13T03:17:46.255 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:17:46.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.356+0000 7f367b1c8640 1 Processor -- start 2024-04-13T03:17:46.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.356+0000 7f367b1c8640 1 -- start start 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.357+0000 7f367b1c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f367415b790 0x7f3674073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.357+0000 7f367b1c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 0x7f367406dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.357+0000 7f367b1c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367415bb70 con 0x7f36740704f0 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.357+0000 7f367b1c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f367406d280 con 0x7f367415b790 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.357+0000 7f367b1c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f36740cfea0 con 0x7f367406e890 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3673fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 0x7f367406dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3678f64640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f367415b790 0x7f3674073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3673fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 0x7f367406dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:57238/0 (socket says 172.21.15.106:57238) 2024-04-13T03:17:46.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3673fff640 1 -- 172.21.15.106:0/1898764834 learned_addr learned my addr 172.21.15.106:0/1898764834 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3679765640 1 --1- 172.21.15.106:0/1898764834 >> v1:172.21.15.114:6789/0 conn(0x7f36740704f0 0x7f36740b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:38764/0 (socket says 172.21.15.106:38764) 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3678f64640 1 -- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 msgr2=0x7f367406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3678f64640 1 --2- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 0x7f367406dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3678f64640 1 -- 172.21.15.106:0/1898764834 >> v1:172.21.15.114:6789/0 conn(0x7f36740704f0 legacy=0x7f36740b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f3678f64640 1 -- 172.21.15.106:0/1898764834 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36740d0180 con 0x7f367415b790 2024-04-13T03:17:46.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.358+0000 7f36737fe640 1 -- 172.21.15.106:0/1898764834 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934885302 0 0) 0x7f367415bb70 con 0x7f36740704f0 2024-04-13T03:17:46.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f3678f64640 1 --2- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674073750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f366400ba40 tx=0x7f366402e080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b34d84e67f7d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f36737fe640 1 -- 172.21.15.106:0/1898764834 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366403b070 con 0x7f367415b790 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f36737fe640 1 -- 172.21.15.106:0/1898764834 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366402ee30 con 0x7f367415b790 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f36737fe640 1 -- 172.21.15.106:0/1898764834 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664036640 con 0x7f367415b790 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f367b1c8640 1 -- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 msgr2=0x7f3674073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f367b1c8640 1 --2- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674073750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f366400ba40 tx=0x7f366402e080 comp rx=0 tx=0).stop 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f367b1c8640 1 -- 172.21.15.106:0/1898764834 shutdown_connections 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f367b1c8640 1 --2- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f367406e890 0x7f367406dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.359+0000 7f367b1c8640 1 --2- 172.21.15.106:0/1898764834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674073750 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.360+0000 7f367b1c8640 1 -- 172.21.15.106:0/1898764834 >> 172.21.15.106:0/1898764834 conn(0x7f36740d13f0 msgr2=0x7f36740d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.360+0000 7f367b1c8640 1 -- 172.21.15.106:0/1898764834 shutdown_connections 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.360+0000 7f367b1c8640 1 -- 172.21.15.106:0/1898764834 wait complete. 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.361+0000 7f367b1c8640 1 Processor -- start 2024-04-13T03:17:46.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.361+0000 7f367b1c8640 1 -- start start 2024-04-13T03:17:46.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f36740704f0 0x7f3674154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3674073c90 con 0x7f367415c7b0 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f367406d7b0 con 0x7f367415b790 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f367406e210 con 0x7f36740704f0 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35606/0 (socket says 172.21.15.106:35606) 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 -- 172.21.15.106:0/2935010751 learned_addr learned my addr 172.21.15.106:0/2935010751 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3678f64640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f36740704f0 0x7f3674154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3673fff640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 -- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f36740704f0 msgr2=0x7f3674154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f36740704f0 0x7f3674154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 -- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 msgr2=0x7f3674150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 -- 172.21.15.106:0/2935010751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367415cd70 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f3679765640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c7a0 tx=0x7f366c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c018070 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c00a640 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367415d050 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.362+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c00aa70 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.363+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367415d610 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.363+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674069120 con 0x7f367415c7b0 2024-04-13T03:17:46.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.364+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f366c01b070 con 0x7f367415c7b0 2024-04-13T03:17:46.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.364+0000 7f36717fa640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 0x7f366006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.364+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f366c08b160 con 0x7f367415c7b0 2024-04-13T03:17:46.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.364+0000 7f3678f64640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 0x7f366006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.365+0000 7f3678f64640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 0x7f366006ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3664038000 tx=0x7f3664032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.369+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f366c058e40 con 0x7f367415c7b0 2024-04-13T03:17:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.544+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f36740d03c0 con 0x7f367415c7b0 2024-04-13T03:17:46.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.544+0000 7f36717fa640 1 -- 172.21.15.106:0/2935010751 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f366c05d140 con 0x7f367415c7b0 2024-04-13T03:17:46.546 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 msgr2=0x7f366006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 0x7f366006ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3664038000 tx=0x7f3664032000 comp rx=0 tx=0).stop 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 msgr2=0x7f3674155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c7a0 tx=0x7f366c00cc60 comp rx=0 tx=0).stop 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 shutdown_connections 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f366006c920 0x7f366006ede0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f367415c7b0 0x7f3674155d20 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f367415b790 0x7f3674150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 --2- 172.21.15.106:0/2935010751 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f36740704f0 0x7f3674154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 >> 172.21.15.106:0/2935010751 conn(0x7f36740d13f0 msgr2=0x7f36740cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 shutdown_connections 2024-04-13T03:17:46.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.546+0000 7f367b1c8640 1 -- 172.21.15.106:0/2935010751 wait complete. 2024-04-13T03:17:46.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:17:46.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 Processor -- start 2024-04-13T03:17:46.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 -- start start 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7f640b7f40 0x7f7f6415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640d0640 con 0x7f7f6415d380 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640d0920 con 0x7f7f640ccd40 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.658+0000 7f7f69080640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7f640d0f70 con 0x7f7f640b7f40 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f62d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7f6415d380 0x7f7f6406f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58780/0 (socket says 172.21.15.106:58780) 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f62d9d640 1 -- 172.21.15.106:0/2195091411 learned_addr learned my addr 172.21.15.106:0/2195091411 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:46.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/2195091411 >> v2:172.21.15.114:3300/0 conn(0x7f7f640b7f40 0x7f7f6415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732814729 0 0) 0x7f7f640d0640 con 0x7f7f6415d380 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f50003520 con 0x7f7f6415d380 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 18331696 0 0) 0x7f7f50003520 con 0x7f7f6415d380 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 >> v2:172.21.15.114:3300/0 conn(0x7f7f640b7f40 msgr2=0x7f7f6415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 --2- 172.21.15.106:0/2195091411 >> v2:172.21.15.114:3300/0 conn(0x7f7f640b7f40 0x7f7f6415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 >> v1:172.21.15.114:6789/0 conn(0x7f7f640ccd40 legacy=0x7f7f640d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640d0c00 con 0x7f7f6415d380 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f7f54001670 con 0x7f7f6415d380 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 >> v1:172.21.15.106:6789/0 conn(0x7f7f6415d380 legacy=0x7f7f6406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640d0f70 con 0x7f7f50004800 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000c160 con 0x7f7f500039b0 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5000c360 con 0x7f7f500051c0 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f62d9d640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f6359e640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 -- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f500051c0 msgr2=0x7f7f5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 -- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f500039b0 msgr2=0x7f7f50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.659+0000 7f7f5bfff640 1 -- 172.21.15.106:0/2195091411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f50003520 con 0x7f7f50004800 2024-04-13T03:17:46.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.660+0000 7f7f6359e640 1 -- 172.21.15.106:0/2195091411 reap_dead start 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.660+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009ac0 tx=0x7f7f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734187cc5970a515 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.660+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c039070 con 0x7f7f50004800 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.660+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c02c6b0 con 0x7f7f50004800 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.660+0000 7f7f6259c640 1 -- 172.21.15.106:0/2195091411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c009690 con 0x7f7f50004800 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 -- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 msgr2=0x7f7f50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009ac0 tx=0x7f7f50005830 comp rx=0 tx=0).stop 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 -- 172.21.15.106:0/2195091411 shutdown_connections 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 --2- 172.21.15.106:0/2195091411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 -- 172.21.15.106:0/2195091411 >> 172.21.15.106:0/2195091411 conn(0x7f7f640d2530 msgr2=0x7f7f640d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 -- 172.21.15.106:0/2195091411 shutdown_connections 2024-04-13T03:17:46.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.661+0000 7f7f69080640 1 -- 172.21.15.106:0/2195091411 wait complete. 2024-04-13T03:17:46.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.662+0000 7f7f69080640 1 Processor -- start 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.662+0000 7f7f69080640 1 -- start start 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 0x7f7f640c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f641542a0 0x7f7f640c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6415d8e0 con 0x7f7f64153e20 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6415d760 con 0x7f7f50004800 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f69080640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6415da60 con 0x7f7f641542a0 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35628/0 (socket says 172.21.15.106:35628) 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 -- 172.21.15.106:0/1265703545 learned_addr learned my addr 172.21.15.106:0/1265703545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f6359e640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f641542a0 0x7f7f640c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f62d9d640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 0x7f7f640c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 -- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f641542a0 msgr2=0x7f7f640c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f641542a0 0x7f7f640c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 -- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 msgr2=0x7f7f640c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 0x7f7f640c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 -- 172.21.15.106:0/1265703545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640c6270 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f62d9d640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 0x7f7f640c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.663+0000 7f7f5bfff640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.664+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c042020 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.664+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c03a040 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.664+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c039070 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.664+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f640c6550 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.664+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f640c8ac0 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.665+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f64069120 con 0x7f7f64153e20 2024-04-13T03:17:46.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.665+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f4c045020 con 0x7f7f64153e20 2024-04-13T03:17:46.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.665+0000 7f7f5b7fe640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 0x7f7f4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:46.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.665+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f7f4c0ad0a0 con 0x7f7f64153e20 2024-04-13T03:17:46.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.665+0000 7f7f62d9d640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 0x7f7f4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:46.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.666+0000 7f7f62d9d640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 0x7f7f4406edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7f640cbc50 tx=0x7f7f54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:46.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.670+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f4c04c070 con 0x7f7f64153e20 2024-04-13T03:17:46.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.856+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f640d3190 con 0x7f7f64153e20 2024-04-13T03:17:46.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.856+0000 7f7f5b7fe640 1 -- 172.21.15.106:0/1265703545 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f7f4c07b3d0 con 0x7f7f64153e20 2024-04-13T03:17:46.858 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 msgr2=0x7f7f4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 0x7f7f4406edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7f640cbc50 tx=0x7f7f54009040 comp rx=0 tx=0).stop 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 msgr2=0x7f7f640c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c004730 comp rx=0 tx=0).stop 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 shutdown_connections 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7f4406c8f0 0x7f7f4406edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7f641542a0 0x7f7f640c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f64153e20 0x7f7f640c7010 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 --2- 172.21.15.106:0/1265703545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f50004800 0x7f7f640c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.858+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 >> 172.21.15.106:0/1265703545 conn(0x7f7f640d2530 msgr2=0x7f7f64152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.859+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 shutdown_connections 2024-04-13T03:17:46.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:46.859+0000 7f7f69080640 1 -- 172.21.15.106:0/1265703545 wait complete. 2024-04-13T03:17:46.871 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:17:46.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:46.871 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:46.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:17:46.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:46.904 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:17:46.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:47.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.250+0000 7fd6689dc640 1 Processor -- start 2024-04-13T03:17:47.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.250+0000 7fd6689dc640 1 -- start start 2024-04-13T03:17:47.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd6689dc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd66406cb10 0x7fd664072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd6689dc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd66406e150 0x7fd66406e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd6689dc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6640998d0 con 0x7fd6640cdf50 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd6689dc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd664073020 con 0x7fd66406e150 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd6689dc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd664073300 con 0x7fd66406cb10 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd66406e150 0x7fd66406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd66259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd6640cdf50 0x7fd6640ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58792/0 (socket says 172.21.15.106:58792) 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd66406e150 0x7fd66406e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35630/0 (socket says 172.21.15.106:35630) 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd66259c640 1 -- 172.21.15.106:0/1747166206 learned_addr learned my addr 172.21.15.106:0/1747166206 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd661d9b640 1 --2- 172.21.15.106:0/1747166206 >> v2:172.21.15.114:3300/0 conn(0x7fd66406cb10 0x7fd664072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 -- 172.21.15.106:0/1747166206 >> v2:172.21.15.114:3300/0 conn(0x7fd66406cb10 msgr2=0x7fd664072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 --2- 172.21.15.106:0/1747166206 >> v2:172.21.15.114:3300/0 conn(0x7fd66406cb10 0x7fd664072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 -- 172.21.15.106:0/1747166206 >> v1:172.21.15.106:6789/0 conn(0x7fd6640cdf50 legacy=0x7fd6640ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd66159a640 1 -- 172.21.15.106:0/1747166206 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766824235 0 0) 0x7fd6640998d0 con 0x7fd6640cdf50 2024-04-13T03:17:47.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.251+0000 7fd662d9d640 1 -- 172.21.15.106:0/1747166206 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6640735e0 con 0x7fd66406e150 2024-04-13T03:17:47.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.252+0000 7fd662d9d640 1 --2- 172.21.15.106:0/1747166206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406e150 0x7fd66406e600 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd6640b64f0 tx=0x7fd65c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68694fab92377d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.252+0000 7fd66159a640 1 -- 172.21.15.106:0/1747166206 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c039070 con 0x7fd66406e150 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.252+0000 7fd66159a640 1 -- 172.21.15.106:0/1747166206 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c02cc80 con 0x7fd66406e150 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.252+0000 7fd66159a640 1 -- 172.21.15.106:0/1747166206 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c034660 con 0x7fd66406e150 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 -- 172.21.15.106:0/1747166206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406e150 msgr2=0x7fd66406e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 --2- 172.21.15.106:0/1747166206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406e150 0x7fd66406e600 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd6640b64f0 tx=0x7fd65c02bed0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 -- 172.21.15.106:0/1747166206 shutdown_connections 2024-04-13T03:17:47.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 --2- 172.21.15.106:0/1747166206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406e150 0x7fd66406e600 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 --2- 172.21.15.106:0/1747166206 >> v2:172.21.15.114:3300/0 conn(0x7fd66406cb10 0x7fd664072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 -- 172.21.15.106:0/1747166206 >> 172.21.15.106:0/1747166206 conn(0x7fd6640b1f10 msgr2=0x7fd6640b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 -- 172.21.15.106:0/1747166206 shutdown_connections 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.253+0000 7fd6689dc640 1 -- 172.21.15.106:0/1747166206 wait complete. 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.254+0000 7fd6689dc640 1 Processor -- start 2024-04-13T03:17:47.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.254+0000 7fd6689dc640 1 -- start start 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd66406e150 0x7fd66415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6640cdf50 0x7fd66415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd664072de0 con 0x7fd66406cb10 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd664072c60 con 0x7fd6640cdf50 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd6689dc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd66406eb40 con 0x7fd66406e150 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd66259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd66259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35646/0 (socket says 172.21.15.106:35646) 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd66259c640 1 -- 172.21.15.106:0/3176712137 learned_addr learned my addr 172.21.15.106:0/3176712137 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd662d9d640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6640cdf50 0x7fd66415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.255+0000 7fd661d9b640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd66406e150 0x7fd66415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 -- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd66406e150 msgr2=0x7fd66415bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd66406e150 0x7fd66415bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 -- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6640cdf50 msgr2=0x7fd66415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6640cdf50 0x7fd66415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 -- 172.21.15.106:0/3176712137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66415c7e0 con 0x7fd66406cb10 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd66259c640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c460 tx=0x7fd65400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654018070 con 0x7fd66406cb10 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd654007500 con 0x7fd66406cb10 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66415cac0 con 0x7fd66406cb10 2024-04-13T03:17:47.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654004a20 con 0x7fd66406cb10 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.256+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66415d080 con 0x7fd66406cb10 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.257+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd664068a10 con 0x7fd66406cb10 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.257+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd65401b070 con 0x7fd66406cb10 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.258+0000 7fd652ffd640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 0x7fd64406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.258+0000 7fd661d9b640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 0x7fd64406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.258+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fd65408ae00 con 0x7fd66406cb10 2024-04-13T03:17:47.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.258+0000 7fd661d9b640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 0x7fd64406edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6640729a0 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.262+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd654058ae0 con 0x7fd66406cb10 2024-04-13T03:17:47.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.473+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd66406bdd0 con 0x7fd66406cb10 2024-04-13T03:17:47.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.474+0000 7fd652ffd640 1 -- 172.21.15.106:0/3176712137 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5195 (secure 0 0 0) 0x7fd65405cde0 con 0x7fd66406cb10 2024-04-13T03:17:47.476 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:47.476 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:17:36.367064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8430":{"gid":8430,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/2645971931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2645971931},{"type":"v1","addr":"172.21.15.114:6835","nonce":2645971931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8430,"qdb_cluster":[8430]},"id":7}]} 2024-04-13T03:17:47.476 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 71 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 msgr2=0x7fd64406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 0x7fd64406edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6640729a0 tx=0x7fd658008040 comp rx=0 tx=0).stop 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 msgr2=0x7fd66415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c460 tx=0x7fd65400c920 comp rx=0 tx=0).stop 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 shutdown_connections 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd64406c8f0 0x7fd64406edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd66406e150 0x7fd66415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6640cdf50 0x7fd66415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 --2- 172.21.15.106:0/3176712137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd66406cb10 0x7fd66415e130 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 >> 172.21.15.106:0/3176712137 conn(0x7fd6640b1f10 msgr2=0x7fd6640b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:47.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 shutdown_connections 2024-04-13T03:17:47.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.476+0000 7fd6689dc640 1 -- 172.21.15.106:0/3176712137 wait complete. 2024-04-13T03:17:47.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-13T03:17:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:17:47.623 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:17:47.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:47.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.722+0000 7fb5d666d640 1 Processor -- start 2024-04-13T03:17:47.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.722+0000 7fb5d666d640 1 -- start start 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 0x7fb5d00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015ed90 0x7fb5d00d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb5d00b3d60 0x7fb5d00d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d00d5070 con 0x7fb5d00b3d60 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d00cfd20 con 0x7fb5d015e9b0 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d666d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d0162af0 con 0x7fb5d015ed90 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb5d00b3d60 0x7fb5d00d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb5d00b3d60 0x7fb5d00d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35652/0 (socket says 172.21.15.106:35652) 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 -- 172.21.15.106:0/3781272473 learned_addr learned my addr 172.21.15.106:0/3781272473 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5cffff640 1 --2- 172.21.15.106:0/3781272473 >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 0x7fb5d00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5cf7fe640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015ed90 0x7fb5d00d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 -- 172.21.15.106:0/3781272473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015ed90 msgr2=0x7fb5d00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015ed90 0x7fb5d00d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 -- 172.21.15.106:0/3781272473 >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 msgr2=0x7fb5d00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 --2- 172.21.15.106:0/3781272473 >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 0x7fb5d00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5d4c0a640 1 -- 172.21.15.106:0/3781272473 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d0162dd0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.723+0000 7fb5cffff640 1 --2- 172.21.15.106:0/3781272473 >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 0x7fb5d00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.724+0000 7fb5d4c0a640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d00d4b30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0162d00 tx=0x7fb5c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0b7cc3fa2c16ce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.724+0000 7fb5ceffd640 1 -- 172.21.15.106:0/3781272473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c802caf0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.724+0000 7fb5ceffd640 1 -- 172.21.15.106:0/3781272473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c802cc90 con 0x7fb5d00b3d60 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.724+0000 7fb5ceffd640 1 -- 172.21.15.106:0/3781272473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8034610 con 0x7fb5d00b3d60 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 -- 172.21.15.106:0/3781272473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 msgr2=0x7fb5d00d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d00d4b30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0162d00 tx=0x7fb5c802bf50 comp rx=0 tx=0).stop 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 -- 172.21.15.106:0/3781272473 shutdown_connections 2024-04-13T03:17:47.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d00d4b30 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 --2- 172.21.15.106:0/3781272473 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015ed90 0x7fb5d00d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 --2- 172.21.15.106:0/3781272473 >> v2:172.21.15.114:3300/0 conn(0x7fb5d015e9b0 0x7fb5d00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 -- 172.21.15.106:0/3781272473 >> 172.21.15.106:0/3781272473 conn(0x7fb5d008bf10 msgr2=0x7fb5d00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:47.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 -- 172.21.15.106:0/3781272473 shutdown_connections 2024-04-13T03:17:47.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.725+0000 7fb5d666d640 1 -- 172.21.15.106:0/3781272473 wait complete. 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.726+0000 7fb5d666d640 1 Processor -- start 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.726+0000 7fb5d666d640 1 -- start start 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 0x7fb5d006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d006be20 0x7fb5d01601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0162680 con 0x7fb5d00b3d60 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0163390 con 0x7fb5d006be20 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d0162800 con 0x7fb5d015e9b0 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35662/0 (socket says 172.21.15.106:35662) 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 -- 172.21.15.106:0/528181749 learned_addr learned my addr 172.21.15.106:0/528181749 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d4c0a640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d006be20 0x7fb5d01601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cf7fe640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 0x7fb5d006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 -- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 msgr2=0x7fb5d006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 0x7fb5d006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 -- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d006be20 msgr2=0x7fb5d01601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d006be20 0x7fb5d01601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 -- 172.21.15.106:0/528181749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d0160e20 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cf7fe640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 0x7fb5d006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5cffff640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e760 tx=0x7fb5c000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000f040 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c0004230 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d0161100 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000ee70 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.727+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01616c0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.728+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d0068a10 con 0x7fb5d00b3d60 2024-04-13T03:17:47.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.729+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5c000c6d0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.729+0000 7fb5ccff9640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 0x7fb5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:47.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.729+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb5c008ba40 con 0x7fb5d00b3d60 2024-04-13T03:17:47.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.729+0000 7fb5cf7fe640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 0x7fb5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:47.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.730+0000 7fb5cf7fe640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 0x7fb5bc06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00cb280 tx=0x7fb5c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:47.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.733+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5c0017020 con 0x7fb5d00b3d60 2024-04-13T03:17:47.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.947+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5d00d53a0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.947+0000 7fb5ccff9640 1 -- 172.21.15.106:0/528181749 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5195 (secure 0 0 0) 0x7fb5c0059ce0 con 0x7fb5d00b3d60 2024-04-13T03:17:47.949 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:47.949 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:17:36.367064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8430":{"gid":8430,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/2645971931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2645971931},{"type":"v1","addr":"172.21.15.114:6835","nonce":2645971931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8430,"qdb_cluster":[8430]},"id":7}]} 2024-04-13T03:17:47.950 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 71 2024-04-13T03:17:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 msgr2=0x7fb5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 0x7fb5bc06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00cb280 tx=0x7fb5c4008040 comp rx=0 tx=0).stop 2024-04-13T03:17:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 msgr2=0x7fb5d015b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:47.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e760 tx=0x7fb5c000ec20 comp rx=0 tx=0).stop 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 shutdown_connections 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb5bc06c8f0 0x7fb5bc06edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb5d015e9b0 0x7fb5d006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d006be20 0x7fb5d01601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 --2- 172.21.15.106:0/528181749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d00b3d60 0x7fb5d015b510 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 >> 172.21.15.106:0/528181749 conn(0x7fb5d008bf10 msgr2=0x7fb5d00b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 shutdown_connections 2024-04-13T03:17:47.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:47.949+0000 7fb5d666d640 1 -- 172.21.15.106:0/528181749 wait complete. 2024-04-13T03:17:47.962 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-13T03:17:48.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 Processor -- start 2024-04-13T03:17:48.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 -- start start 2024-04-13T03:17:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e69aec20 0x55a0e6a23540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e6a20a00 0x55a0e6a36c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0e6965450 con 0x55a0e6964c30 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0e690dd00 con 0x55a0e6a20a00 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a0e69656f0 con 0x55a0e69aec20 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f37fa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f37fa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35676/0 (socket says 172.21.15.106:35676) 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.003+0000 7f8f37fa3640 1 -- 172.21.15.106:0/175205316 learned_addr learned my addr 172.21.15.106:0/175205316 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 -- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e69aec20 msgr2=0x55a0e6a23540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e69aec20 0x55a0e6a23540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 -- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e6a20a00 msgr2=0x55a0e6a36c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e6a20a00 0x55a0e6a36c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 -- 172.21.15.106:0/175205316 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e6a220a0 con 0x55a0e6964c30 2024-04-13T03:17:48.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009710 tx=0x7f8f28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b77432bf7e660a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f367a0640 1 -- 172.21.15.106:0/175205316 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28014070 con 0x55a0e6964c30 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f367a0640 1 -- 172.21.15.106:0/175205316 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2800b8d0 con 0x55a0e6964c30 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f367a0640 1 -- 172.21.15.106:0/175205316 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800bc80 con 0x55a0e6964c30 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/175205316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 msgr2=0x55a0e6a23000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009710 tx=0x7f8f28009bd0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/175205316 shutdown_connections 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e69aec20 0x55a0e6a23540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e6a20a00 0x55a0e6a36c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/175205316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a23000 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/175205316 >> 172.21.15.106:0/175205316 conn(0x55a0e6990ae0 msgr2=0x55a0e6a1f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.004+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/175205316 shutdown_connections 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.005+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/175205316 wait complete. 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.005+0000 7f8f3ae3ff40 1 Processor -- start 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.005+0000 7f8f3ae3ff40 1 -- start start 2024-04-13T03:17:48.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a39430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e69aec20 0x55a0e6a36880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e6a20a00 0x55a0e6a19d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0e6a3cfb0 con 0x55a0e6964c30 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0e6a3cd10 con 0x55a0e69aec20 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55a0e6a22340 con 0x55a0e6a20a00 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a39430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35688/0 (socket says 172.21.15.106:35688) 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 -- 172.21.15.106:0/2704266125 learned_addr learned my addr 172.21.15.106:0/2704266125 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f377a2640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e69aec20 0x55a0e6a36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f387a4640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e6a20a00 0x55a0e6a19d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 -- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e6a20a00 msgr2=0x55a0e6a19d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e6a20a00 0x55a0e6a19d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 -- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e69aec20 msgr2=0x55a0e6a36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0e69aec20 0x55a0e6a36880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 -- 172.21.15.106:0/2704266125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e6a1c620 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f387a4640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55a0e6a20a00 0x55a0e6a19d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f37fa3640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a39430 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800a010 tx=0x7f8f280033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3579e640 1 -- 172.21.15.106:0/2704266125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802a070 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3579e640 1 -- 172.21.15.106:0/2704266125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2800b6d0 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a0e6a1c900 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.006+0000 7f8f3579e640 1 -- 172.21.15.106:0/2704266125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28014040 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.007+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e6a1cec0 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f3579e640 1 -- 172.21.15.106:0/2704266125 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f2802d070 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f3579e640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8f2406b2d0 0x7f8f2406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f3579e640 1 -- 172.21.15.106:0/2704266125 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f8f2808c030 con 0x55a0e6964c30 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f377a2640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8f2406b2d0 0x7f8f2406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55a0e6a36dc0 0x55a0e6ad20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107) -- 0x55a0e6a1aa60 con 0x55a0e6a36dc0 2024-04-13T03:17:48.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.008+0000 7f8f387a4640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55a0e6a36dc0 0x55a0e6ad20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f377a2640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8f2406b2d0 0x7f8f2406d790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c057000 tx=0x7f8f2c051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f387a4640 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55a0e6a36dc0 0x55a0e6ad20e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f387a4640 1 -- 172.21.15.106:0/2704266125 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f8f3000d440 con 0x55a0e6a36dc0 2024-04-13T03:17:48.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55a0e6a36dc0 msgr2=0x55a0e6ad20e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55a0e6a36dc0 0x55a0e6ad20e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8f2406b2d0 msgr2=0x7f8f2406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8f2406b2d0 0x7f8f2406d790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c057000 tx=0x7f8f2c051000 comp rx=0 tx=0).stop 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 msgr2=0x55a0e6a39430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 --2- 172.21.15.106:0/2704266125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0e6964c30 0x55a0e6a39430 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800a010 tx=0x7f8f280033a0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f387a4640 1 -- 172.21.15.106:0/2704266125 reap_dead start 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 shutdown_connections 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 >> 172.21.15.106:0/2704266125 conn(0x55a0e6990ae0 msgr2=0x55a0e6a1f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 shutdown_connections 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.009+0000 7f8f3ae3ff40 1 -- 172.21.15.106:0/2704266125 wait complete. 2024-04-13T03:17:48.017 INFO:teuthology.orchestra.run.smithi106.stdout:����� 2024-04-13T03:17:48.017 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:17:48.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.155+0000 7f1a6582d640 1 Processor -- start 2024-04-13T03:17:48.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.155+0000 7f1a6582d640 1 -- start start 2024-04-13T03:17:48.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a6582d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a600d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a6582d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a601635b0 con 0x7f1a600d3b10 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a6582d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a601627f0 con 0x7f1a600b6ac0 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a6582d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1a600d3f70 con 0x7f1a6015ea80 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a57fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1a600d3b10 0x7f1a600b63b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58808/0 (socket says 172.21.15.106:58808) 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a57fff640 1 -- 172.21.15.106:0/2183764464 learned_addr learned my addr 172.21.15.106:0/2183764464 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394598559 0 0) 0x7f1a601635b0 con 0x7f1a600d3b10 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a4c003520 con 0x7f1a600d3b10 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5effd640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a600d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1620847872 0 0) 0x7f1a4c003520 con 0x7f1a600d3b10 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 msgr2=0x7f1a600d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a600d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 >> v1:172.21.15.114:6789/0 conn(0x7f1a600b6ac0 legacy=0x7f1a601689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a60162ad0 con 0x7f1a600d3b10 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f1a48001670 con 0x7f1a600d3b10 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5effd640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a600d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 >> v1:172.21.15.106:6789/0 conn(0x7f1a600d3b10 legacy=0x7f1a600b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c00bb50 con 0x7f1a4c004d40 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c00bcd0 con 0x7f1a4c0039e0 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.156+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c00bed0 con 0x7f1a6015ea80 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5effd640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a57fff640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 -- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 msgr2=0x7f1a4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015ea80 0x7f1a4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 -- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a4c0039e0 msgr2=0x7f1a4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.157+0000 7f1a5f7fe640 1 -- 172.21.15.106:0/2183764464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c003520 con 0x7f1a4c004d40 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a5f7fe640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1a58001e50 tx=0x7f1a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a050e4d1bca346f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58001a70 con 0x7f1a4c004d40 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a58003b70 con 0x7f1a4c004d40 2024-04-13T03:17:48.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a5e7fc640 1 -- 172.21.15.106:0/2183764464 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a580050d0 con 0x7f1a4c004d40 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 -- 172.21.15.106:0/2183764464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c004d40 msgr2=0x7f1a4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 --2- 172.21.15.106:0/2183764464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1a58001e50 tx=0x7f1a4c005830 comp rx=0 tx=0).stop 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a5f7fe640 1 -- 172.21.15.106:0/2183764464 reap_dead start 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 -- 172.21.15.106:0/2183764464 shutdown_connections 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 -- 172.21.15.106:0/2183764464 >> 172.21.15.106:0/2183764464 conn(0x7f1a600b97a0 msgr2=0x7f1a600b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 -- 172.21.15.106:0/2183764464 shutdown_connections 2024-04-13T03:17:48.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.158+0000 7f1a6582d640 1 -- 172.21.15.106:0/2183764464 wait complete. 2024-04-13T03:17:48.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 Processor -- start 2024-04-13T03:17:48.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 -- start start 2024-04-13T03:17:48.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 0x7f1a6006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015faa0 0x7f1a6015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60163090 con 0x7f1a6006d760 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600b7100 con 0x7f1a601546a0 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a6582d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6015ee60 con 0x7f1a6015faa0 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35702/0 (socket says 172.21.15.106:35702) 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.160+0000 7f1a57fff640 1 -- 172.21.15.106:0/181900899 learned_addr learned my addr 172.21.15.106:0/181900899 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5effd640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 0x7f1a6006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5f7fe640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015faa0 0x7f1a6015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 -- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015faa0 msgr2=0x7f1a6015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015faa0 0x7f1a6015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 -- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 msgr2=0x7f1a6006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 0x7f1a6006d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 -- 172.21.15.106:0/181900899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6006df80 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5effd640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 0x7f1a6006d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a57fff640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f1a480072c0 tx=0x7f1a48007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800f960 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a48018070 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800fd60 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6006e260 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.161+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6006e820 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.162+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6006ea20 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.162+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a4801d460 con 0x7f1a6006d760 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.163+0000 7f1a5cff9640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 0x7f1a4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.163+0000 7f1a5effd640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 0x7f1a4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:48.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.163+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f1a4808be40 con 0x7f1a6006d760 2024-04-13T03:17:48.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.163+0000 7f1a5effd640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 0x7f1a4006ede0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:48.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.167+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a6006ea20 con 0x7f1a6006d760 2024-04-13T03:17:48.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.378+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1a60068a10 con 0x7f1a6006d760 2024-04-13T03:17:48.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.378+0000 7f1a5cff9640 1 -- 172.21.15.106:0/181900899 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f1a4805a0e0 con 0x7f1a6006d760 2024-04-13T03:17:48.380 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:17:48.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 msgr2=0x7f1a4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 0x7f1a4006ede0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 msgr2=0x7f1a6015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f1a480072c0 tx=0x7f1a48007780 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 shutdown_connections 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1a4006c920 0x7f1a4006ede0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1a6015faa0 0x7f1a6015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a6006d760 0x7f1a6015f560 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 --2- 172.21.15.106:0/181900899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a601546a0 0x7f1a6006d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 >> 172.21.15.106:0/181900899 conn(0x7f1a600b97a0 msgr2=0x7f1a600b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 shutdown_connections 2024-04-13T03:17:48.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48.380+0000 7f1a6582d640 1 -- 172.21.15.106:0/181900899 wait complete. 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:17:48.393 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.394 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:48.409 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.410 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:17:48.411 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:17:48.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.412 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:17:48.475 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:17:48.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.476 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:17:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:17:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:17:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:17:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:17:48.533 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:17:48.533 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:17:48.534 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:17:48.534 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:17:48.534 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:17:48.534 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:17:48.534 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:17:48.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:17:48.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:48.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.667 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:17:48.682 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:17:48.682 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:17:48.682 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:48.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.682 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:17:48.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:17:48.749 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:48.805 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:17:48.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:17:48.805 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:48.860 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:17:48.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.860 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:48.945 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:48.945 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:17:48.952 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:17:48.953 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:17:48.953 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:17:48.953 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:17:48.954 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:17:48.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:48.954 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:48.980 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:17:49.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:17:49.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:49.077 INFO:teuthology.orchestra.run.smithi106.stdout:client6653 2024-04-13T03:17:49.077 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:49.077 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:49.092 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:49.092 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:49.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:49.148 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-13T03:17:49.243 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records in 2024-04-13T03:17:49.243 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records out 2024-04-13T03:17:49.243 INFO:teuthology.orchestra.run.smithi106.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0367692 s, 171 MB/s 2024-04-13T03:17:49.244 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:49.244 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:49.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:49.260 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-13T03:17:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records in 2024-04-13T03:17:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:6+0 records out 2024-04-13T03:17:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0358901 s, 175 MB/s 2024-04-13T03:17:49.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:49.352 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-13T03:17:49.368 INFO:teuthology.orchestra.run.smithi106.stdout:./ 2024-04-13T03:17:49.368 INFO:teuthology.orchestra.run.smithi106.stdout:./file2_sixmegs 2024-04-13T03:17:49.368 INFO:teuthology.orchestra.run.smithi106.stdout:./file3_sixmegs 2024-04-13T03:17:49.368 INFO:teuthology.orchestra.run.smithi106.stdout:./file1_sixmegs 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-13T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:17:49.474 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:17:49.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:49.482 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:17:49.483 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:17:49.483 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:17:49.483 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:17:49.483 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-13T03:17:49.483 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:17:49.587 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:17:49.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-13T03:17:49.596 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:17:49.700 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2024-04-13T03:17:49.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:49.709 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:17:49.709 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:17:49.709 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:17:49.709 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:17:49.709 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-13T03:17:49.710 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:17:49.814 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:17:49.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:17:49.823 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:49.823 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:49.878 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:49.879 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:49.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:17:50.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.073+0000 7f25283e0640 1 Processor -- start 2024-04-13T03:17:50.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.073+0000 7f25283e0640 1 -- start start 2024-04-13T03:17:50.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f25283e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200ceb70 0x7f25200d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f25283e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25200b5ad0 0x7f25200d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f25283e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25200b6120 con 0x7f252015ec20 2024-04-13T03:17:50.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f25283e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f25200cde50 con 0x7f25200b5ad0 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f25283e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25200cd240 con 0x7f25200ceb70 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f252697d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f25200b5ad0 0x7f25200d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f252617c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f252015ec20 0x7f25200b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58818/0 (socket says 172.21.15.106:58818) 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.074+0000 7f252617c640 1 -- 172.21.15.106:0/3680932440 learned_addr learned my addr 172.21.15.106:0/3680932440 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252697d640 1 -- 172.21.15.106:0/3680932440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200ceb70 msgr2=0x7f25200d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252697d640 1 --2- 172.21.15.106:0/3680932440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200ceb70 0x7f25200d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252697d640 1 -- 172.21.15.106:0/3680932440 >> v1:172.21.15.106:6789/0 conn(0x7f252015ec20 legacy=0x7f25200b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252697d640 1 -- 172.21.15.106:0/3680932440 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25200cd520 con 0x7f25200b5ad0 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252517a640 1 -- 172.21.15.106:0/3680932440 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545262849 0 0) 0x7f25200b6120 con 0x7f252015ec20 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.075+0000 7f252697d640 1 --2- 172.21.15.106:0/3680932440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25200b5ad0 0x7f25200d0a00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b2e6515994a0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f252517a640 1 -- 172.21.15.106:0/3680932440 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c039070 con 0x7f25200b5ad0 2024-04-13T03:17:50.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f252517a640 1 -- 172.21.15.106:0/3680932440 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251c02cc50 con 0x7f25200b5ad0 2024-04-13T03:17:50.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f252517a640 1 -- 172.21.15.106:0/3680932440 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c034440 con 0x7f25200b5ad0 2024-04-13T03:17:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 -- 172.21.15.106:0/3680932440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25200b5ad0 msgr2=0x7f25200d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 --2- 172.21.15.106:0/3680932440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25200b5ad0 0x7f25200d0a00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02bea0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 -- 172.21.15.106:0/3680932440 shutdown_connections 2024-04-13T03:17:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 --2- 172.21.15.106:0/3680932440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25200b5ad0 0x7f25200d0a00 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 --2- 172.21.15.106:0/3680932440 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200ceb70 0x7f25200d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 -- 172.21.15.106:0/3680932440 >> 172.21.15.106:0/3680932440 conn(0x7f25200d2530 msgr2=0x7f25200d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 -- 172.21.15.106:0/3680932440 shutdown_connections 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.076+0000 7f25283e0640 1 -- 172.21.15.106:0/3680932440 wait complete. 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.077+0000 7f25283e0640 1 Processor -- start 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.077+0000 7f25283e0640 1 -- start start 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 0x7f2520149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 0x7f252014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25200cced0 con 0x7f2520147890 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25200cdae0 con 0x7f252015ec20 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f25283e0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25201626b0 con 0x7f25200b5ad0 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35716/0 (socket says 172.21.15.106:35716) 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 -- 172.21.15.106:0/2752920549 learned_addr learned my addr 172.21.15.106:0/2752920549 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252597b640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 0x7f252014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252617c640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 0x7f2520149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 -- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 msgr2=0x7f2520149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 0x7f2520149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 -- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 msgr2=0x7f252014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 0x7f252014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252697d640 1 -- 172.21.15.106:0/2752920549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25200c97c0 con 0x7f2520147890 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252597b640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 0x7f252014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.078+0000 7f252617c640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 0x7f2520149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f252697d640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f251c037040 tx=0x7f251c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c044070 con 0x7f2520147890 2024-04-13T03:17:50.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251c02ca50 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c039040 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25200c9aa0 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.079+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25200ca060 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.080+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2520068a10 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.080+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f251c003bb0 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.080+0000 7f2516ffd640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 0x7f250406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.081+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f251c0ae4a0 con 0x7f2520147890 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.081+0000 7f252617c640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 0x7f250406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.081+0000 7f252617c640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 0x7f250406ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2518000f60 tx=0x7f2518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.085+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f251c04e020 con 0x7f2520147890 2024-04-13T03:17:50.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.259+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25200d3120 con 0x7f2520147890 2024-04-13T03:17:50.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.260+0000 7f2516ffd640 1 -- 172.21.15.106:0/2752920549 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16920 (secure 0 0 0) 0x7f251c02cc20 con 0x7f2520147890 2024-04-13T03:17:50.261 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:50.261 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":107,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:17:35.383741+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:17:50.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 msgr2=0x7f250406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 0x7f250406ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2518000f60 tx=0x7f2518009040 comp rx=0 tx=0).stop 2024-04-13T03:17:50.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 msgr2=0x7f25200ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f251c037040 tx=0x7f251c004a40 comp rx=0 tx=0).stop 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 shutdown_connections 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f250406c920 0x7f250406ede0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2520147890 0x7f25200ca860 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f252015ec20 0x7f252014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 --2- 172.21.15.106:0/2752920549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25200b5ad0 0x7f2520149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 >> 172.21.15.106:0/2752920549 conn(0x7f25200d2530 msgr2=0x7f25200c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 shutdown_connections 2024-04-13T03:17:50.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.262+0000 7f25283e0640 1 -- 172.21.15.106:0/2752920549 wait complete. 2024-04-13T03:17:50.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:17:31.965063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:17:32.694523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:17:50.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:17:50.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.377+0000 7f7ed7a2d640 1 Processor -- start 2024-04-13T03:17:50.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 -- start start 2024-04-13T03:17:50.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 0x7f7ed015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ed0070bb0 0x7f7ed006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed00b5cf0 con 0x7f7ed00b54a0 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed00b5fd0 con 0x7f7ed0070bb0 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed7a2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed00b62b0 con 0x7f7ed00b1c30 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.378+0000 7f7ed5fca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ed0070bb0 0x7f7ed006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7ed0070bb0 0x7f7ed006d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35720/0 (socket says 172.21.15.106:35720) 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 -- 172.21.15.106:0/194571111 learned_addr learned my addr 172.21.15.106:0/194571111 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed57c9640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 0x7f7ed015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 -- 172.21.15.106:0/194571111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 msgr2=0x7f7ed015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 0x7f7ed015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 -- 172.21.15.106:0/194571111 >> v1:172.21.15.114:6789/0 conn(0x7f7ed00b54a0 legacy=0x7f7ed00704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 -- 172.21.15.106:0/194571111 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00b6590 con 0x7f7ed0070bb0 2024-04-13T03:17:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed57c9640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 0x7f7ed015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.379+0000 7f7ed5fca640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed0070bb0 0x7f7ed006d660 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009e80 tx=0x7f7ec802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bd5a7c0e0cf72a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ec7fff640 1 -- 172.21.15.106:0/194571111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec802ca50 con 0x7f7ed0070bb0 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ec7fff640 1 -- 172.21.15.106:0/194571111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec802cbf0 con 0x7f7ed0070bb0 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ec7fff640 1 -- 172.21.15.106:0/194571111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80343f0 con 0x7f7ed0070bb0 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/194571111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed0070bb0 msgr2=0x7f7ed006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed0070bb0 0x7f7ed006d660 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009e80 tx=0x7f7ec802beb0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/194571111 shutdown_connections 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed0070bb0 0x7f7ed006d660 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/194571111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b1c30 0x7f7ed015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/194571111 >> 172.21.15.106:0/194571111 conn(0x7f7ed0071870 msgr2=0x7f7ed0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/194571111 shutdown_connections 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.380+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/194571111 wait complete. 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.381+0000 7f7ed7a2d640 1 Processor -- start 2024-04-13T03:17:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.381+0000 7f7ed7a2d640 1 -- start start 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed0070bb0 0x7f7ed0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b54a0 0x7f7ed01f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed006dd20 con 0x7f7ed00b1c30 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed006dba0 con 0x7f7ed0070bb0 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed7a2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed006dea0 con 0x7f7ed00b54a0 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35730/0 (socket says 172.21.15.106:35730) 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 -- 172.21.15.106:0/416893988 learned_addr learned my addr 172.21.15.106:0/416893988 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed57c9640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed0070bb0 0x7f7ed0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed5fca640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b54a0 0x7f7ed01f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 -- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b54a0 msgr2=0x7f7ed01f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b54a0 0x7f7ed01f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 -- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed0070bb0 msgr2=0x7f7ed0150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed0070bb0 0x7f7ed0150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.382+0000 7f7ed4fc8640 1 -- 172.21.15.106:0/416893988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01626f0 con 0x7f7ed00b1c30 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ed4fc8640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000b420 tx=0x7f7ec000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0011830 con 0x7f7ed00b1c30 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec0016070 con 0x7f7ed00b1c30 2024-04-13T03:17:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01629d0 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0011c10 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.383+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed0162f90 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.384+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0068a10 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.384+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec001b460 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.385+0000 7f7ec5ffb640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.385+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f7ec008b680 con 0x7f7ed00b1c30 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.385+0000 7f7ed57c9640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.385+0000 7f7ed57c9640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00cf510 tx=0x7f7ecc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.389+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ec0020020 con 0x7f7ed00b1c30 2024-04-13T03:17:50.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.568+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ed00711f0 con 0x7f7ed00b1c30 2024-04-13T03:17:50.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.568+0000 7f7ec5ffb640 1 -- 172.21.15.106:0/416893988 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f7ec00592d0 con 0x7f7ed00b1c30 2024-04-13T03:17:50.569 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:17:50.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 msgr2=0x7f7eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00cf510 tx=0x7f7ecc008040 comp rx=0 tx=0).stop 2024-04-13T03:17:50.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 msgr2=0x7f7ed015ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000b420 tx=0x7f7ec000b8e0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 shutdown_connections 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7ed00b54a0 0x7f7ed01f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ed00b1c30 0x7f7ed015ff70 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 --2- 172.21.15.106:0/416893988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed0070bb0 0x7f7ed0150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 >> 172.21.15.106:0/416893988 conn(0x7f7ed0071870 msgr2=0x7f7ed015b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 shutdown_connections 2024-04-13T03:17:50.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.570+0000 7f7ed7a2d640 1 -- 172.21.15.106:0/416893988 wait complete. 2024-04-13T03:17:50.582 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:17:50.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.684+0000 7f381095e640 1 Processor -- start 2024-04-13T03:17:50.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 -- start start 2024-04-13T03:17:50.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f380c0704f0 0x7f380c0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c0cf2f0 0x7f380c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380c070e10 con 0x7f380c15f080 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f380c162f40 con 0x7f380c0704f0 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.685+0000 7f381095e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f380c15fbf0 con 0x7f380c0cf2f0 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f380c0704f0 0x7f380c0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f3809d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f380c15f080 0x7f380c15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58820/0 (socket says 172.21.15.106:58820) 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f380c0704f0 0x7f380c0708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35732/0 (socket says 172.21.15.106:35732) 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f3809d9b640 1 -- 172.21.15.106:0/1932422816 learned_addr learned my addr 172.21.15.106:0/1932422816 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 -- 172.21.15.106:0/1932422816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c0cf2f0 msgr2=0x7f380c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 --2- 172.21.15.106:0/1932422816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c0cf2f0 0x7f380c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 -- 172.21.15.106:0/1932422816 >> v1:172.21.15.106:6789/0 conn(0x7f380c15f080 legacy=0x7f380c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380959a640 1 -- 172.21.15.106:0/1932422816 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577649498 0 0) 0x7f380c070e10 con 0x7f380c15f080 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.686+0000 7f380a59c640 1 -- 172.21.15.106:0/1932422816 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c15fe40 con 0x7f380c0704f0 2024-04-13T03:17:50.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f380a59c640 1 --2- 172.21.15.106:0/1932422816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c0708d0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009d90 tx=0x7f37fc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a06e3b26c9c9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f380959a640 1 -- 172.21.15.106:0/1932422816 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc039070 con 0x7f380c0704f0 2024-04-13T03:17:50.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f380959a640 1 -- 172.21.15.106:0/1932422816 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc02cc50 con 0x7f380c0704f0 2024-04-13T03:17:50.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f380959a640 1 -- 172.21.15.106:0/1932422816 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc034440 con 0x7f380c0704f0 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 -- 172.21.15.106:0/1932422816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 msgr2=0x7f380c0708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 --2- 172.21.15.106:0/1932422816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c0708d0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009d90 tx=0x7f37fc02bea0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 -- 172.21.15.106:0/1932422816 shutdown_connections 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 --2- 172.21.15.106:0/1932422816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c0cf2f0 0x7f380c0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 --2- 172.21.15.106:0/1932422816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c0708d0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 -- 172.21.15.106:0/1932422816 >> 172.21.15.106:0/1932422816 conn(0x7f380c0d13f0 msgr2=0x7f380c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 -- 172.21.15.106:0/1932422816 shutdown_connections 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.687+0000 7f381095e640 1 -- 172.21.15.106:0/1932422816 wait complete. 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 Processor -- start 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 -- start start 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 0x7f380c1397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 0x7f380c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1628f0 con 0x7f380c0704f0 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.689+0000 7f381095e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f380c162610 con 0x7f380c0cf2f0 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f381095e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f380c162bd0 con 0x7f380c15f080 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35746/0 (socket says 172.21.15.106:35746) 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 -- 172.21.15.106:0/2505326531 learned_addr learned my addr 172.21.15.106:0/2505326531 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380ad9d640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 0x7f380c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f3809d9b640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 0x7f380c1397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 -- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 msgr2=0x7f380c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 0x7f380c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 -- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 msgr2=0x7f380c1397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 0x7f380c1397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 -- 172.21.15.106:0/2505326531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1618e0 con 0x7f380c0704f0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380ad9d640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 0x7f380c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f3809d9b640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 0x7f380c1397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f380a59c640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f37fc037040 tx=0x7f37fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc044020 con 0x7f380c0704f0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc03f470 con 0x7f380c0704f0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c161600 con 0x7f380c0704f0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc039040 con 0x7f380c0704f0 2024-04-13T03:17:50.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.690+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c15cf00 con 0x7f380c0704f0 2024-04-13T03:17:50.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.691+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c069120 con 0x7f380c0704f0 2024-04-13T03:17:50.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.692+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37fc047020 con 0x7f380c0704f0 2024-04-13T03:17:50.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.692+0000 7f37faffd640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.692+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f37fc0ae690 con 0x7f380c0704f0 2024-04-13T03:17:50.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.692+0000 7f3809d9b640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.693+0000 7f3809d9b640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 0x7f37ec06edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f380c13a7b0 tx=0x7f3800009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.697+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37fc04e030 con 0x7f380c0704f0 2024-04-13T03:17:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.870+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f380c071330 con 0x7f380c0704f0 2024-04-13T03:17:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.871+0000 7f37faffd640 1 -- 172.21.15.106:0/2505326531 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f37fc07c9c0 con 0x7f380c0704f0 2024-04-13T03:17:50.872 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.872+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 msgr2=0x7f37ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 0x7f37ec06edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f380c13a7b0 tx=0x7f3800009040 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 msgr2=0x7f380c06be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f37fc037040 tx=0x7f37fc004a40 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 shutdown_connections 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f380c15f080 0x7f380c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c0cf2f0 0x7f380c1397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 --2- 172.21.15.106:0/2505326531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f380c0704f0 0x7f380c06be70 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 >> 172.21.15.106:0/2505326531 conn(0x7f380c0d13f0 msgr2=0x7f380c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 shutdown_connections 2024-04-13T03:17:50.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.873+0000 7f381095e640 1 -- 172.21.15.106:0/2505326531 wait complete. 2024-04-13T03:17:50.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:17:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.989+0000 7fce6b355640 1 Processor -- start 2024-04-13T03:17:50.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 -- start start 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415ed90 0x7fce640d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce640b3d60 0x7fce640d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce640cfd20 con 0x7fce6415e9b0 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fce64162af0 con 0x7fce640b3d60 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.990+0000 7fce6b355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fce64162dd0 con 0x7fce6415ed90 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce640b3d60 0x7fce640d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce640b3d60 0x7fce640d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35760/0 (socket says 172.21.15.106:35760) 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 -- 172.21.15.106:0/361672190 learned_addr learned my addr 172.21.15.106:0/361672190 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce688f0640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415ed90 0x7fce640d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 -- 172.21.15.106:0/361672190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415ed90 msgr2=0x7fce640d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415ed90 0x7fce640d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 -- 172.21.15.106:0/361672190 >> v1:172.21.15.114:6789/0 conn(0x7fce6415e9b0 legacy=0x7fce640d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.991+0000 7fce698f2640 1 -- 172.21.15.106:0/361672190 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641630b0 con 0x7fce640b3d60 2024-04-13T03:17:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce698f2640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce640b3d60 0x7fce640d0200 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009e50 tx=0x7fce5c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7f84e431decf7e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce5bfff640 1 -- 172.21.15.106:0/361672190 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c02cb90 con 0x7fce640b3d60 2024-04-13T03:17:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce5bfff640 1 -- 172.21.15.106:0/361672190 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5c02cd30 con 0x7fce640b3d60 2024-04-13T03:17:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce5bfff640 1 -- 172.21.15.106:0/361672190 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c034610 con 0x7fce640b3d60 2024-04-13T03:17:50.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 -- 172.21.15.106:0/361672190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce640b3d60 msgr2=0x7fce640d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce640b3d60 0x7fce640d0200 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009e50 tx=0x7fce5c02bff0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 -- 172.21.15.106:0/361672190 shutdown_connections 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce640b3d60 0x7fce640d0200 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 --2- 172.21.15.106:0/361672190 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415ed90 0x7fce640d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 -- 172.21.15.106:0/361672190 >> 172.21.15.106:0/361672190 conn(0x7fce6408bf10 msgr2=0x7fce640b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 -- 172.21.15.106:0/361672190 shutdown_connections 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.992+0000 7fce6b355640 1 -- 172.21.15.106:0/361672190 wait complete. 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 Processor -- start 2024-04-13T03:17:50.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 -- start start 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 0x7fce640b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415bed0 0x7fce640ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce64162800 con 0x7fce6415e9b0 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fce64162680 con 0x7fce640b3d60 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce6b355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fce640d4d00 con 0x7fce6415bed0 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce688f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce688f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35776/0 (socket says 172.21.15.106:35776) 2024-04-13T03:17:50.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.994+0000 7fce688f0640 1 -- 172.21.15.106:0/1929144598 learned_addr learned my addr 172.21.15.106:0/1929144598 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce698f2640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415bed0 0x7fce640ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce690f1640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 0x7fce640b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 -- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415bed0 msgr2=0x7fce640ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415bed0 0x7fce640ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 -- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 msgr2=0x7fce640b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 0x7fce640b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 -- 172.21.15.106:0/1929144598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641f56b0 con 0x7fce6415e9b0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce690f1640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 0x7fce640b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce688f0640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d7d0 tx=0x7fce5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54018020 con 0x7fce6415e9b0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce540096c0 con 0x7fce6415e9b0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641f5990 con 0x7fce6415e9b0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54009ab0 con 0x7fce6415e9b0 2024-04-13T03:17:50.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.995+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6406cbb0 con 0x7fce6415e9b0 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.996+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6406c9a0 con 0x7fce6415e9b0 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.997+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce5401b020 con 0x7fce6415e9b0 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.997+0000 7fce59ffb640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 0x7fce4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.997+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fce5408c2a0 con 0x7fce6415e9b0 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.997+0000 7fce690f1640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 0x7fce4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:50.997+0000 7fce690f1640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 0x7fce4806edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fce640b8630 tx=0x7fce60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.001+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce54022020 con 0x7fce6415e9b0 2024-04-13T03:17:51.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.176+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce64068a10 con 0x7fce6415e9b0 2024-04-13T03:17:51.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.176+0000 7fce59ffb640 1 -- 172.21.15.106:0/1929144598 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7fce5405a540 con 0x7fce6415e9b0 2024-04-13T03:17:51.177 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 msgr2=0x7fce4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 0x7fce4806edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fce640b8630 tx=0x7fce60009040 comp rx=0 tx=0).stop 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 msgr2=0x7fce641f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d7d0 tx=0x7fce5400dc90 comp rx=0 tx=0).stop 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 shutdown_connections 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce4806c8f0 0x7fce4806edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce6415bed0 0x7fce640ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce6415e9b0 0x7fce641f4e80 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 --2- 172.21.15.106:0/1929144598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce640b3d60 0x7fce640b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 >> 172.21.15.106:0/1929144598 conn(0x7fce6408bf10 msgr2=0x7fce6415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:51.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 shutdown_connections 2024-04-13T03:17:51.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.178+0000 7fce6b355640 1 -- 172.21.15.106:0/1929144598 wait complete. 2024-04-13T03:17:51.189 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:17:51.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.290+0000 7fae3ea88640 1 Processor -- start 2024-04-13T03:17:51.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.290+0000 7fae3ea88640 1 -- start start 2024-04-13T03:17:51.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3ea88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae380d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3ea88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae380cfd20 con 0x7fae3815e9b0 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3ea88640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae38162af0 con 0x7fae3815ed90 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3ea88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fae38162dd0 con 0x7fae380b3d60 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3c824640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fae3815e9b0 0x7fae380d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58836/0 (socket says 172.21.15.106:58836) 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.291+0000 7fae3c824640 1 -- 172.21.15.106:0/1701907901 learned_addr learned my addr 172.21.15.106:0/1701907901 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae380d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2193583722 0 0) 0x7fae380cfd20 con 0x7fae3815e9b0 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae20003520 con 0x7fae3815e9b0 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174798730 0 0) 0x7fae38162af0 con 0x7fae3815ed90 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae380cfd20 con 0x7fae3815ed90 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 949386350 0 0) 0x7fae20003520 con 0x7fae3815e9b0 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 msgr2=0x7fae380d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae380d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 >> v1:172.21.15.114:6789/0 conn(0x7fae3815ed90 legacy=0x7fae380d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae381630b0 con 0x7fae3815e9b0 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fae28001670 con 0x7fae3815e9b0 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae380d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 >> v1:172.21.15.106:6789/0 conn(0x7fae3815e9b0 legacy=0x7fae380d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae20003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae200040b0 0x7fae20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae20004d40 0x7fae200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fae38162dd0 con 0x7fae20004d40 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae2000ba70 con 0x7fae200040b0 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fae2000bc70 con 0x7fae380b3d60 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae20004d40 0x7fae200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3c824640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae20003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae37fff640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae200040b0 0x7fae20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 -- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 msgr2=0x7fae20003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae20003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 -- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae200040b0 msgr2=0x7fae20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae200040b0 0x7fae20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3d025640 1 -- 172.21.15.106:0/1701907901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae20003520 con 0x7fae20004d40 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.292+0000 7fae3c824640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae380b3d60 0x7fae20003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.293+0000 7fae3d025640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae20004d40 0x7fae200051c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e350 tx=0x7fae20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abac8280e6b131a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.293+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30009dc0 con 0x7fae20004d40 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.293+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3002cd90 con 0x7fae20004d40 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.293+0000 7fae377fe640 1 -- 172.21.15.106:0/1701907901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30003b90 con 0x7fae20004d40 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 -- 172.21.15.106:0/1701907901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae20004d40 msgr2=0x7fae200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 --2- 172.21.15.106:0/1701907901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae20004d40 0x7fae200051c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e350 tx=0x7fae20005830 comp rx=0 tx=0).stop 2024-04-13T03:17:51.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3d025640 1 -- 172.21.15.106:0/1701907901 reap_dead start 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 -- 172.21.15.106:0/1701907901 shutdown_connections 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 -- 172.21.15.106:0/1701907901 >> 172.21.15.106:0/1701907901 conn(0x7fae3808bf10 msgr2=0x7fae380b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 -- 172.21.15.106:0/1701907901 shutdown_connections 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.294+0000 7fae3ea88640 1 -- 172.21.15.106:0/1701907901 wait complete. 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.295+0000 7fae3ea88640 1 Processor -- start 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.295+0000 7fae3ea88640 1 -- start start 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae20004d40 0x7fae3806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae380ca6c0 0x7fae380cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fae38162800 con 0x7fae380b6be0 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae38162680 con 0x7fae380ca6c0 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3ea88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fae380d4d00 con 0x7fae20004d40 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35790/0 (socket says 172.21.15.106:35790) 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 -- 172.21.15.106:0/3419095610 learned_addr learned my addr 172.21.15.106:0/3419095610 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3c824640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae20004d40 0x7fae3806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae3d025640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae380ca6c0 0x7fae380cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 -- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae20004d40 msgr2=0x7fae3806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae20004d40 0x7fae3806c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 -- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae380ca6c0 msgr2=0x7fae380cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae380ca6c0 0x7fae380cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 -- 172.21.15.106:0/3419095610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae380cb080 con 0x7fae380b6be0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae37fff640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e580 tx=0x7fae2c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0052e0 con 0x7fae380b6be0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.296+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c018070 con 0x7fae380b6be0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.297+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c003640 con 0x7fae380b6be0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.297+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae380b7060 con 0x7fae380b6be0 2024-04-13T03:17:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.297+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3806d310 con 0x7fae380b6be0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.298+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae38068a10 con 0x7fae380b6be0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.298+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae2c0037e0 con 0x7fae380b6be0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.298+0000 7fae357fa640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 0x7fae2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.298+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fae2c08bb10 con 0x7fae380b6be0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.298+0000 7fae3c824640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 0x7fae2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.299+0000 7fae3c824640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 0x7fae2406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fae28001eb0 tx=0x7fae28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.303+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae2c059e80 con 0x7fae380b6be0 2024-04-13T03:17:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.488+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae380d1820 con 0x7fae380b6be0 2024-04-13T03:17:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.488+0000 7fae357fa640 1 -- 172.21.15.106:0/3419095610 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7fae2c059780 con 0x7fae380b6be0 2024-04-13T03:17:51.489 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:17:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 msgr2=0x7fae2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 0x7fae2406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fae28001eb0 tx=0x7fae28009040 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 msgr2=0x7fae380b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e580 tx=0x7fae2c00ea40 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 shutdown_connections 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fae2406c920 0x7fae2406ede0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae380ca6c0 0x7fae380cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae380b6be0 0x7fae380b43b0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 --2- 172.21.15.106:0/3419095610 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fae20004d40 0x7fae3806c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.490+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 >> 172.21.15.106:0/3419095610 conn(0x7fae3808bf10 msgr2=0x7fae380b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.491+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 shutdown_connections 2024-04-13T03:17:51.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.491+0000 7fae3ea88640 1 -- 172.21.15.106:0/3419095610 wait complete. 2024-04-13T03:17:51.502 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:17:51.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:51.502 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:17:51.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:17:51.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:51.560 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:17:51.582 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:17:51.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.722+0000 7fc9bc555640 1 Processor -- start 2024-04-13T03:17:51.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.722+0000 7fc9bc555640 1 -- start start 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9bc555640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9b415e9b0 0x7fc9b40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9bc555640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b40cfd20 con 0x7fc9b415ed90 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9bc555640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b4162af0 con 0x7fc9b40b3d60 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9bc555640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc9b4162dd0 con 0x7fc9b415e9b0 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9b9af0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc9b415ed90 0x7fc9b40d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58850/0 (socket says 172.21.15.106:58850) 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.723+0000 7fc9b9af0640 1 -- 172.21.15.106:0/1715759902 learned_addr learned my addr 172.21.15.106:0/1715759902 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9ba2f1640 1 --2- 172.21.15.106:0/1715759902 >> v2:172.21.15.114:3300/0 conn(0x7fc9b415e9b0 0x7fc9b40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642172994 0 0) 0x7fc9b40cfd20 con 0x7fc9b415ed90 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a0003520 con 0x7fc9b415ed90 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2783300429 0 0) 0x7fc9a0003520 con 0x7fc9b415ed90 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 >> v2:172.21.15.114:3300/0 conn(0x7fc9b415e9b0 msgr2=0x7fc9b40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 --2- 172.21.15.106:0/1715759902 >> v2:172.21.15.114:3300/0 conn(0x7fc9b415e9b0 0x7fc9b40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 >> v1:172.21.15.114:6789/0 conn(0x7fc9b40b3d60 legacy=0x7fc9b40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b41630b0 con 0x7fc9b415ed90 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc9a40016c0 con 0x7fc9b415ed90 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 >> v1:172.21.15.106:6789/0 conn(0x7fc9b415ed90 legacy=0x7fc9b40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00039b0 0x7fc9a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9a0004800 0x7fc9a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 0x7fc9a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4162dd0 con 0x7fc9a00051c0 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a000c070 con 0x7fc9a0004800 2024-04-13T03:17:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a000c270 con 0x7fc9a00039b0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9baaf2640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 0x7fc9a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9ba2f1640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00039b0 0x7fc9a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9b9af0640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9a0004800 0x7fc9a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9baaf2640 1 -- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00039b0 msgr2=0x7fc9a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9baaf2640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00039b0 0x7fc9a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9baaf2640 1 -- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9a0004800 msgr2=0x7fc9a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.724+0000 7fc9baaf2640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9a0004800 0x7fc9a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9baaf2640 1 -- 172.21.15.106:0/1715759902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a0003520 con 0x7fc9a00051c0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9baaf2640 1 -- 172.21.15.106:0/1715759902 reap_dead start 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9baaf2640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 0x7fc9a000b9c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac001750 tx=0x7fc9a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3c50f6e70ab97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac001370 con 0x7fc9a00051c0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac005310 con 0x7fc9a00051c0 2024-04-13T03:17:51.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.725+0000 7fc9b92ef640 1 -- 172.21.15.106:0/1715759902 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac003830 con 0x7fc9a00051c0 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 -- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 msgr2=0x7fc9a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 0x7fc9a000b9c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac001750 tx=0x7fc9a0005830 comp rx=0 tx=0).stop 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 -- 172.21.15.106:0/1715759902 shutdown_connections 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 --2- 172.21.15.106:0/1715759902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a00051c0 0x7fc9a000b9c0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 -- 172.21.15.106:0/1715759902 >> 172.21.15.106:0/1715759902 conn(0x7fc9b408bf10 msgr2=0x7fc9b40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 -- 172.21.15.106:0/1715759902 shutdown_connections 2024-04-13T03:17:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.726+0000 7fc9bc555640 1 -- 172.21.15.106:0/1715759902 wait complete. 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.727+0000 7fc9bc555640 1 Processor -- start 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.727+0000 7fc9bc555640 1 -- start start 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00051c0 0x7fc9b40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b41f7da0 0x7fc9b41f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4162800 con 0x7fc9b41f8960 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4162680 con 0x7fc9b41f7da0 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9bc555640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b406eb20 con 0x7fc9a00051c0 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35810/0 (socket says 172.21.15.106:35810) 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 -- 172.21.15.106:0/2777153067 learned_addr learned my addr 172.21.15.106:0/2777153067 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 -- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00051c0 msgr2=0x7fc9b40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00051c0 0x7fc9b40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 -- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b41f7da0 msgr2=0x7fc9b41f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b41f7da0 0x7fc9b41f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 -- 172.21.15.106:0/2777153067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b40c8a20 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9b9af0640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40073a0 tx=0x7fc9a4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.728+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a401a070 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.729+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a400f9c0 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.729+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b40b6d90 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.729+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400fd90 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.729+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b406bcf0 con 0x7fc9b41f8960 2024-04-13T03:17:51.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.729+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b4068a10 con 0x7fc9b41f8960 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.730+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9a401d070 con 0x7fc9b41f8960 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.730+0000 7fc9aaffd640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 0x7fc99806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.730+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fc9a408b520 con 0x7fc9b41f8960 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.730+0000 7fc9ba2f1640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 0x7fc99806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.731+0000 7fc9ba2f1640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 0x7fc99806ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009f40 tx=0x7fc9b0009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.735+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9a4059850 con 0x7fc9b41f8960 2024-04-13T03:17:51.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.945+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9b40c8bf0 con 0x7fc9b41f8960 2024-04-13T03:17:51.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.946+0000 7fc9aaffd640 1 -- 172.21.15.106:0/2777153067 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5195 (secure 0 0 0) 0x7fc9a4059150 con 0x7fc9b41f8960 2024-04-13T03:17:51.947 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:51.947 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:17:36.367064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8430":{"gid":8430,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6835/2645971931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2645971931},{"type":"v1","addr":"172.21.15.114:6835","nonce":2645971931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8430,"qdb_cluster":[8430]},"id":7}]} 2024-04-13T03:17:51.947 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 71 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 msgr2=0x7fc99806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 0x7fc99806ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009f40 tx=0x7fc9b0009750 comp rx=0 tx=0).stop 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 msgr2=0x7fc9b4151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40073a0 tx=0x7fc9a4007860 comp rx=0 tx=0).stop 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 shutdown_connections 2024-04-13T03:17:51.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc99806c920 0x7fc99806ede0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b41f7da0 0x7fc9b41f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9b41f8960 0x7fc9b4151a00 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 --2- 172.21.15.106:0/2777153067 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9a00051c0 0x7fc9b40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 >> 172.21.15.106:0/2777153067 conn(0x7fc9b408bf10 msgr2=0x7fc9b415df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 shutdown_connections 2024-04-13T03:17:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:51.948+0000 7fc9bc555640 1 -- 172.21.15.106:0/2777153067 wait complete. 2024-04-13T03:17:51.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-13T03:17:52.093 INFO:teuthology.orchestra.run.smithi114.stdout:{"message":"","return_code":0} 2024-04-13T03:17:52.094 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-13T03:17:52.094 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:17:52.094 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:17:52.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:17:52.112 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:17:52.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:52.112 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:17:52.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:17:52.166 INFO:teuthology.orchestra.run.smithi106.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-13T03:17:52.167 DEBUG:teuthology.orchestra.run.smithi106:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-13T03:17:52.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:52.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-13T03:17:52.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.314+0000 7f983231e640 1 Processor -- start 2024-04-13T03:17:52.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.314+0000 7f983231e640 1 -- start start 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f983231e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f982c0b3920 0x7f982c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f983231e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c15c870 con 0x7f982c0d2370 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f983231e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c15ee70 con 0x7f982c0d3490 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f983231e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f982c0b1b00 con 0x7f982c0b3920 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f98308bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f982c0b3920 0x7f982c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f982b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f982c0d2370 0x7f982c0b13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58862/0 (socket says 172.21.15.106:58862) 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f982b7fe640 1 -- 172.21.15.106:0/2841860771 learned_addr learned my addr 172.21.15.106:0/2841860771 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f98308bb640 1 -- 172.21.15.106:0/2841860771 >> v1:172.21.15.114:6789/0 conn(0x7f982c0d3490 legacy=0x7f982c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f98308bb640 1 -- 172.21.15.106:0/2841860771 >> v1:172.21.15.106:6789/0 conn(0x7f982c0d2370 legacy=0x7f982c0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f982affd640 1 -- 172.21.15.106:0/2841860771 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169374269 0 0) 0x7f982c15c870 con 0x7f982c0d2370 2024-04-13T03:17:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.315+0000 7f98308bb640 1 -- 172.21.15.106:0/2841860771 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0d2800 con 0x7f982c0b3920 2024-04-13T03:17:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.316+0000 7f98308bb640 1 --2- 172.21.15.106:0/2841860771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0b3920 0x7f982c15f360 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9824009d90 tx=0x7f982402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cd06cb05bf93a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.316+0000 7f982affd640 1 -- 172.21.15.106:0/2841860771 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824039070 con 0x7f982c0b3920 2024-04-13T03:17:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.316+0000 7f982affd640 1 -- 172.21.15.106:0/2841860771 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982402cbd0 con 0x7f982c0b3920 2024-04-13T03:17:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.316+0000 7f982affd640 1 -- 172.21.15.106:0/2841860771 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824034440 con 0x7f982c0b3920 2024-04-13T03:17:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 -- 172.21.15.106:0/2841860771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0b3920 msgr2=0x7f982c15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 --2- 172.21.15.106:0/2841860771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0b3920 0x7f982c15f360 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9824009d90 tx=0x7f982402be20 comp rx=0 tx=0).stop 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 -- 172.21.15.106:0/2841860771 shutdown_connections 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 --2- 172.21.15.106:0/2841860771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0b3920 0x7f982c15f360 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 -- 172.21.15.106:0/2841860771 >> 172.21.15.106:0/2841860771 conn(0x7f982c0b9760 msgr2=0x7f982c0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 -- 172.21.15.106:0/2841860771 shutdown_connections 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.317+0000 7f983231e640 1 -- 172.21.15.106:0/2841860771 wait complete. 2024-04-13T03:17:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.318+0000 7f983231e640 1 Processor -- start 2024-04-13T03:17:52.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.318+0000 7f983231e640 1 -- start start 2024-04-13T03:17:52.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f982c0b3920 0x7f982c06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f982c159d40 0x7f982c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0d2dc0 con 0x7f982c0d3490 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0d5420 con 0x7f982c0b3920 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f983231e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0d30a0 con 0x7f982c159d40 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35828/0 (socket says 172.21.15.106:35828) 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 -- 172.21.15.106:0/1989441804 learned_addr learned my addr 172.21.15.106:0/1989441804 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982bfff640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f982c0b3920 0x7f982c06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f98308bb640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f982c159d40 0x7f982c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 -- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f982c159d40 msgr2=0x7f982c158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f982c159d40 0x7f982c158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 -- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f982c0b3920 msgr2=0x7f982c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f982c0b3920 0x7f982c06d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 -- 172.21.15.106:0/1989441804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0b4ab0 con 0x7f982c0d3490 2024-04-13T03:17:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f982000c4d0 tx=0x7f982000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f9828ff9640 1 -- 172.21.15.106:0/1989441804 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820007480 con 0x7f982c0d3490 2024-04-13T03:17:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.319+0000 7f9828ff9640 1 -- 172.21.15.106:0/1989441804 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98200057a0 con 0x7f982c0d3490 2024-04-13T03:17:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.320+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c0b4d90 con 0x7f982c0d3490 2024-04-13T03:17:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.320+0000 7f9828ff9640 1 -- 172.21.15.106:0/1989441804 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820007620 con 0x7f982c0d3490 2024-04-13T03:17:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.320+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c0b5350 con 0x7f982c0d3490 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.321+0000 7f9828ff9640 1 -- 172.21.15.106:0/1989441804 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9820005b50 con 0x7f982c0d3490 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.321+0000 7f9828ff9640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.321+0000 7f9828ff9640 1 -- 172.21.15.106:0/1989441804 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f982008b820 con 0x7f982c0d3490 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.321+0000 7f982bfff640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.322+0000 7f982bfff640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 0x7f981806edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f981c0017f0 tx=0x7f981c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.323+0000 7f98351e6740 1 Processor -- start 2024-04-13T03:17:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.323+0000 7f98351e6740 1 -- start start 2024-04-13T03:17:52.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d1796e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d1796bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d1796240a0 con 0x55d1796e6db0 2024-04-13T03:17:52.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d1795ca930 con 0x55d1796bfd40 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d179620200 con 0x55d1796c0120 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35842/0 (socket says 172.21.15.106:35842) 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 -- 172.21.15.106:0/203252777 learned_addr learned my addr 172.21.15.106:0/203252777 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982bfff640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d1796e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98308bb640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d1796bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 -- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 msgr2=0x55d1796bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d1796bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 -- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 msgr2=0x55d1796e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d1796e6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 -- 172.21.15.106:0/203252777 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1793fe7e0 con 0x55d1796e6db0 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f98308bb640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d1796bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:52.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f982b7fe640 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f982000ce90 tx=0x7f98200592f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56339bed53b6fa2c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f9811ffb640 1 -- 172.21.15.106:0/203252777 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820057770 con 0x55d1796e6db0 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f9811ffb640 1 -- 172.21.15.106:0/203252777 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9820055ae0 con 0x55d1796e6db0 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.325+0000 7f9811ffb640 1 -- 172.21.15.106:0/203252777 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820061d30 con 0x55d1796e6db0 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 -- 172.21.15.106:0/203252777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 msgr2=0x55d1796bed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f982000ce90 tx=0x7f98200592f0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 -- 172.21.15.106:0/203252777 shutdown_connections 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d1796bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796e6db0 0x55d1796bed80 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 --2- 172.21.15.106:0/203252777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d1796e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 -- 172.21.15.106:0/203252777 >> 172.21.15.106:0/203252777 conn(0x55d1796866e0 msgr2=0x55d179704a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 -- 172.21.15.106:0/203252777 shutdown_connections 2024-04-13T03:17:52.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.326+0000 7f98351e6740 1 -- 172.21.15.106:0/203252777 wait complete. 2024-04-13T03:17:52.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.327+0000 7f98351e6740 1 -- start start 2024-04-13T03:17:52.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.327+0000 7f98351e6740 1 Processor -- start 2024-04-13T03:17:52.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.328+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d179714a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d179708400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d1796e63e0 con 0x55d1796fcd30 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d1795ddcf0 con 0x55d1796bfd40 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d1796df160 con 0x55d1796c0120 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 10 client.0 _ll_register_callbacks cb 0x55d1795dd6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35846/0 (socket says 172.21.15.106:35846) 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 -- 172.21.15.106:0/3164630118 learned_addr learned my addr 172.21.15.106:0/3164630118 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f982b7fe640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d179708400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f982bfff640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d179714a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 -- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 msgr2=0x55d179708400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d179708400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 -- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 msgr2=0x55d179714a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d179714a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 -- 172.21.15.106:0/3164630118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17945ef70 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f982bfff640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d179714a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98308bb640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9824037040 tx=0x7f9824004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc31744ed4d36dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f980affd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824039040 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f9824034c30 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9820055ae0 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.329+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f9824052050 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 10 client.8709 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d1792f6170 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9820055ae0 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f9824055020 con 0x55d1796fcd30 2024-04-13T03:17:52.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 10 client.8709 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 10 client.8709 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 20 client.8709 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179706b10 0x55d1796fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 4 client.8709 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d179631ac0 con 0x55d179706b10 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f982bfff640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179706b10 0x55d1796fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f980affd640 10 client.8709 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.330+0000 7f982bfff640 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179706b10 0x55d1796fbfe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572aa0968280cf03 server_cookie=61fbfbe444be28a8 in_seq=0 out_seq=0 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f980affd640 1 -- 172.21.15.106:0/3164630118 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d179631ac0 con 0x55d179706b10 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f980affd640 10 client.8709 handle_command_reply: tid=0 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 client.8709 shutdown 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 msgr2=0x55d179708940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9824037040 tx=0x7f9824004ab0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 shutdown_connections 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179706b10 0x55d1796fbfe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796c0120 0x55d179708400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796fcd30 0x55d179708940 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f980affd640 0 client.8709 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 --2- 172.21.15.106:0/3164630118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796bfd40 0x55d179714a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 >> 172.21.15.106:0/3164630118 conn(0x55d1796866e0 msgr2=0x55d1796e5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 shutdown_connections 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 1 -- 172.21.15.106:0/3164630118 wait complete. 2024-04-13T03:17:52.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.331+0000 7f98351e6740 20 client.8709 trim_cache size 0 max 16384 2024-04-13T03:17:52.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.335+0000 7f68e7280640 1 Processor -- start 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 -- start start 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e00d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f68e0160350 0x7f68e015de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e01607d0 con 0x7f68e015e3b0 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f68e0160a80 con 0x7f68e0160350 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.336+0000 7f68e7280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68e0160d60 con 0x7f68e0146220 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f68e0160350 0x7f68e015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e501c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e00d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e581d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f68e015e3b0 0x7f68e00704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:38836/0 (socket says 172.21.15.106:38836) 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e581d640 1 -- 172.21.15.106:0/110932088 learned_addr learned my addr 172.21.15.106:0/110932088 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f68e0160350 0x7f68e015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:40822/0 (socket says 172.21.15.106:40822) 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 -- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 msgr2=0x7f68e00d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e00d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 -- 172.21.15.106:0/110932088 >> v1:172.21.15.114:6789/0 conn(0x7f68e015e3b0 legacy=0x7f68e00704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68d7fff640 1 -- 172.21.15.106:0/110932088 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512751417 0 0) 0x7f68e01607d0 con 0x7f68e015e3b0 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e481b640 1 -- 172.21.15.106:0/110932088 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e0161040 con 0x7f68e0160350 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.337+0000 7f68e501c640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e00d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.338+0000 7f68e481b640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e0160350 0x7f68e015de70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f68d00097a0 tx=0x7f68d002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68eb70d2cf04f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.338+0000 7f68d7fff640 1 -- 172.21.15.106:0/110932088 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0035020 con 0x7f68e0160350 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.338+0000 7f68d7fff640 1 -- 172.21.15.106:0/110932088 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d0031830 con 0x7f68e0160350 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.338+0000 7f68d7fff640 1 -- 172.21.15.106:0/110932088 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0031c00 con 0x7f68e0160350 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 -- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e0160350 msgr2=0x7f68e015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e0160350 0x7f68e015de70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f68d00097a0 tx=0x7f68d002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 -- 172.21.15.106:0/110932088 shutdown_connections 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e0160350 0x7f68e015de70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 --2- 172.21.15.106:0/110932088 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e00d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 -- 172.21.15.106:0/110932088 >> 172.21.15.106:0/110932088 conn(0x7f68e0071900 msgr2=0x7f68e0071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 -- 172.21.15.106:0/110932088 shutdown_connections 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.339+0000 7f68e7280640 1 -- 172.21.15.106:0/110932088 wait complete. 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.340+0000 7f68e7280640 1 Processor -- start 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.340+0000 7f68e7280640 1 -- start start 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e015e3b0 0x7f68e01e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 0x7f68e006f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0070bb0 con 0x7f68e0160350 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68e00cf1c0 con 0x7f68e015e3b0 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e7280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f68e0070d30 con 0x7f68e0146220 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e581d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 0x7f68e006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e581d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 0x7f68e006f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35862/0 (socket says 172.21.15.106:35862) 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e581d640 1 -- 172.21.15.106:0/728861241 learned_addr learned my addr 172.21.15.106:0/728861241 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e481b640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e015e3b0 0x7f68e01e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 -- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e015e3b0 msgr2=0x7f68e01e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e015e3b0 0x7f68e01e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 -- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 msgr2=0x7f68e006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 0x7f68e006f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e501c640 1 -- 172.21.15.106:0/728861241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e00c7690 con 0x7f68e0146220 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e481b640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e015e3b0 0x7f68e01e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.341+0000 7f68e581d640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68e0160350 0x7f68e006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68e501c640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f68e0146220 0x7f68e015c1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00e930 tx=0x7f68dc00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc017070 con 0x7f68e0146220 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc00c940 con 0x7f68e0146220 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68e7280640 1 -- 172.21.15.106:0/728861241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e00c7970 con 0x7f68e0146220 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00cd10 con 0x7f68e0146220 2024-04-13T03:17:52.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.342+0000 7f68e7280640 1 -- 172.21.15.106:0/728861241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01417f0 con 0x7f68e0146220 2024-04-13T03:17:52.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.343+0000 7f68e7280640 1 -- 172.21.15.106:0/728861241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f68e00c7b90 con 0x7f68e0146220 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.343+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68dc01a070 con 0x7f68e0146220 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68d5ffb640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f68dc08bda0 con 0x7f68e0146220 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68e481b640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f68dc0599e0 con 0x7f68e0146220 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68e7280640 1 -- 172.21.15.106:0/728861241 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({log-info=0+}) -- 0x7f68e0069120 con 0x7f68e0146220 2024-04-13T03:17:52.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.344+0000 7f68e481b640 1 --2- 172.21.15.106:0/728861241 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68c406c8f0 0x7f68c406edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f68d0036000 tx=0x7f68d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.351+0000 7f98351e6740 1 Processor -- start 2024-04-13T03:17:52.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.351+0000 7f98351e6740 1 -- start start 2024-04-13T03:17:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d1796e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1796e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d1796fd7d0 con 0x55d1796bfd40 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f982c201390 con 0x55d1796c0120 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f982c2003d0 con 0x55d1796fcd30 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35872/0 (socket says 172.21.15.106:35872) 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 -- 172.21.15.106:0/873378453 learned_addr learned my addr 172.21.15.106:0/873378453 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f98308bb640 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1796e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982b7fe640 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d1796e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 -- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 msgr2=0x55d1796e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1796e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 -- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 msgr2=0x55d1796e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d1796e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 -- 172.21.15.106:0/873378453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d179708f30 con 0x55d1796bfd40 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.352+0000 7f982bfff640 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f981c001eb0 tx=0x7f981c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab8523aab5f774f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98117fa640 1 -- 172.21.15.106:0/873378453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0160d0 con 0x55d1796bfd40 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98117fa640 1 -- 172.21.15.106:0/873378453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c01b040 con 0x55d1796bfd40 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98117fa640 1 -- 172.21.15.106:0/873378453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c016a60 con 0x55d1796bfd40 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 -- 172.21.15.106:0/873378453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 msgr2=0x55d1796edb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f981c001eb0 tx=0x7f981c012000 comp rx=0 tx=0).stop 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 -- 172.21.15.106:0/873378453 shutdown_connections 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1796e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d1796e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 --2- 172.21.15.106:0/873378453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d1796edb80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 -- 172.21.15.106:0/873378453 >> 172.21.15.106:0/873378453 conn(0x55d1796866e0 msgr2=0x55d1797081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 -- 172.21.15.106:0/873378453 shutdown_connections 2024-04-13T03:17:52.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.353+0000 7f98351e6740 1 -- 172.21.15.106:0/873378453 wait complete. 2024-04-13T03:17:52.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.354+0000 7f98351e6740 1 -- start start 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.354+0000 7f98351e6740 1 Processor -- start 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d179708140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1797010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0ce1f0 con 0x55d1796bfd40 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0cdba0 con 0x55d1796c0120 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55d1796fbfe0 con 0x55d1796fcd30 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98351e6740 10 client.0 _ll_register_callbacks cb 0x7f98240344f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35882/0 (socket says 172.21.15.106:35882) 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 -- 172.21.15.106:0/1859836899 learned_addr learned my addr 172.21.15.106:0/1859836899 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d179708140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f98308bb640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1797010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 -- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 msgr2=0x55d1797010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1797010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 -- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 msgr2=0x55d179708140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d179708140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 -- 172.21.15.106:0/1859836899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1797144a0 con 0x55d1796bfd40 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d179708140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f982bfff640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f981c025040 tx=0x7f981c001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6efa5cb8320dec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.355+0000 7f980affd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c037060 con 0x55d1796bfd40 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f981c0281d0 con 0x55d1796bfd40 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f981c01b040 con 0x55d1796bfd40 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f981c016e70 con 0x55d1796bfd40 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 10 client.8718 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:52.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d179702470 con 0x55d1796bfd40 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c01b040 con 0x55d1796bfd40 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f981c028410 con 0x55d1796bfd40 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 10 client.8718 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 10 client.8718 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 20 client.8718 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179782100 0x55d1797015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 4 client.8718 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55d17962b930 con 0x55d179782100 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.356+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179782100 0x55d1797015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f982b7fe640 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179782100 0x55d1797015e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af8a3b307a4480f server_cookie=a7a67bdda784dfaf in_seq=0 out_seq=0 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f980affd640 10 client.8718 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f980affd640 1 -- 172.21.15.106:0/1859836899 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d17962b930 con 0x55d179782100 2024-04-13T03:17:52.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f980affd640 10 client.8718 handle_command_reply: tid=0 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 client.8718 shutdown 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 msgr2=0x55d17977e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f981c025040 tx=0x7f981c001020 comp rx=0 tx=0).stop 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 shutdown_connections 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55d179782100 0x55d1797015e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55d1796fcd30 0x55d1797010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1796c0120 0x55d179708140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 --2- 172.21.15.106:0/1859836899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1796bfd40 0x55d17977e630 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f980affd640 0 client.8718 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 >> 172.21.15.106:0/1859836899 conn(0x55d1796866e0 msgr2=0x55d1796c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 shutdown_connections 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 1 -- 172.21.15.106:0/1859836899 wait complete. 2024-04-13T03:17:52.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.357+0000 7f98351e6740 20 client.8718 trim_cache size 0 max 16384 2024-04-13T03:17:52.368 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:17:52.368 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:17:52.368 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "272453ff-073f-4c1d-9896-9d9da95e590c", 2024-04-13T03:17:52.368 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:17:52.368 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:17:52.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 msgr2=0x7f981806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 0x7f981806edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f981c0017f0 tx=0x7f981c009040 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 msgr2=0x7f982c1587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f982000c4d0 tx=0x7f982000c990 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 shutdown_connections 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f982c159d40 0x7f982c158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f982c0d3490 0x7f982c1587e0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 --2- 172.21.15.106:0/1989441804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f982c0b3920 0x7f982c06d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 >> 172.21.15.106:0/1989441804 conn(0x7f982c0b9760 msgr2=0x7f982c0b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 shutdown_connections 2024-04-13T03:17:52.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.358+0000 7f983231e640 1 -- 172.21.15.106:0/1989441804 wait complete. 2024-04-13T03:17:52.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:17:52.371 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-13T03:17:52.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.474+0000 7f61a6236640 1 Processor -- start 2024-04-13T03:17:52.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.474+0000 7f61a6236640 1 -- start start 2024-04-13T03:17:52.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f61a6236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f61a006bff0 0x7f61a006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f61a6236640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f61a00d07c0 0x7f61a006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f61a6236640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a00cef80 con 0x7f61a015fb50 2024-04-13T03:17:52.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f61a6236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f61a00cf260 con 0x7f61a006bff0 2024-04-13T03:17:52.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f61a6236640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f61a00cf540 con 0x7f61a00d07c0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f619effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f61a006bff0 0x7f61a006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f619effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f61a006bff0 0x7f61a006e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35892/0 (socket says 172.21.15.106:35892) 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f619effd640 1 -- 172.21.15.106:0/3204152936 learned_addr learned my addr 172.21.15.106:0/3204152936 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.475+0000 7f619ffff640 1 --2- 172.21.15.106:0/3204152936 >> v2:172.21.15.114:3300/0 conn(0x7f61a00d07c0 0x7f61a006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619effd640 1 -- 172.21.15.106:0/3204152936 >> v2:172.21.15.114:3300/0 conn(0x7f61a00d07c0 msgr2=0x7f61a006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619effd640 1 --2- 172.21.15.106:0/3204152936 >> v2:172.21.15.114:3300/0 conn(0x7f61a00d07c0 0x7f61a006ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619effd640 1 -- 172.21.15.106:0/3204152936 >> v1:172.21.15.114:6789/0 conn(0x7f61a015fb50 legacy=0x7f61a006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619effd640 1 -- 172.21.15.106:0/3204152936 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a00cf820 con 0x7f61a006bff0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619effd640 1 --2- 172.21.15.106:0/3204152936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a006e6b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f61a01594f0 tx=0x7f618802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce02454fb6231b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619e7fc640 1 -- 172.21.15.106:0/3204152936 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188035020 con 0x7f61a006bff0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619e7fc640 1 -- 172.21.15.106:0/3204152936 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188031870 con 0x7f61a006bff0 2024-04-13T03:17:52.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.476+0000 7f619e7fc640 1 -- 172.21.15.106:0/3204152936 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188031c40 con 0x7f61a006bff0 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 -- 172.21.15.106:0/3204152936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 msgr2=0x7f61a006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 --2- 172.21.15.106:0/3204152936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a006e6b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f61a01594f0 tx=0x7f618802b910 comp rx=0 tx=0).stop 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 -- 172.21.15.106:0/3204152936 shutdown_connections 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 --2- 172.21.15.106:0/3204152936 >> v2:172.21.15.114:3300/0 conn(0x7f61a00d07c0 0x7f61a006ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 --2- 172.21.15.106:0/3204152936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a006e6b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 -- 172.21.15.106:0/3204152936 >> 172.21.15.106:0/3204152936 conn(0x7f61a00cd6a0 msgr2=0x7f61a00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 -- 172.21.15.106:0/3204152936 shutdown_connections 2024-04-13T03:17:52.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.477+0000 7f61a6236640 1 -- 172.21.15.106:0/3204152936 wait complete. 2024-04-13T03:17:52.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.478+0000 7f61a6236640 1 Processor -- start 2024-04-13T03:17:52.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.478+0000 7f61a6236640 1 -- start start 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00d07c0 0x7f61a00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a015fb50 0x7f61a00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61a00ced20 con 0x7f61a006bff0 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61a00ceba0 con 0x7f61a00d07c0 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f61a6236640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f61a015ef10 con 0x7f61a015fb50 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35908/0 (socket says 172.21.15.106:35908) 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 -- 172.21.15.106:0/2650771887 learned_addr learned my addr 172.21.15.106:0/2650771887 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619effd640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00d07c0 0x7f61a00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619ffff640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a015fb50 0x7f61a00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 -- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a015fb50 msgr2=0x7f61a00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a015fb50 0x7f61a00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 -- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00d07c0 msgr2=0x7f61a00aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.479+0000 7f619f7fe640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00d07c0 0x7f61a00aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f619f7fe640 1 -- 172.21.15.106:0/2650771887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a01f7eb0 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f619f7fe640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f619000c3c0 tx=0x7f619000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f6197fff640 1 -- 172.21.15.106:0/2650771887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61900052d0 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f6197fff640 1 -- 172.21.15.106:0/2650771887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6190005490 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01f8130 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f6197fff640 1 -- 172.21.15.106:0/2650771887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190007260 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.480+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a01f86f0 con 0x7f61a006bff0 2024-04-13T03:17:52.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.481+0000 7f6197fff640 1 -- 172.21.15.106:0/2650771887 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6190007400 con 0x7f61a006bff0 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.482+0000 7f6197fff640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.482+0000 7f6197fff640 1 -- 172.21.15.106:0/2650771887 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f619008b0a0 con 0x7f61a006bff0 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.482+0000 7f619effd640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.482+0000 7f619effd640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 0x7f618006edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6188036000 tx=0x7f618803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.483+0000 7f61a9049740 1 Processor -- start 2024-04-13T03:17:52.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.483+0000 7f61a9049740 1 -- start start 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b597ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b597f4d0 0x55b8b59af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8b58ea170 con 0x55b8b59ac3d0 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b8b58907f0 con 0x55b8b59abff0 2024-04-13T03:17:52.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b8b58e6130 con 0x55b8b597f4d0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35922/0 (socket says 172.21.15.106:35922) 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 -- 172.21.15.106:0/2991518168 learned_addr learned my addr 172.21.15.106:0/2991518168 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 -- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b597f4d0 msgr2=0x55b8b59af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b597f4d0 0x55b8b59af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 -- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 msgr2=0x55b8b597ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619f7fe640 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b597ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b597ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 -- 172.21.15.106:0/2991518168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b56c67d0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619f7fe640 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b597ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f619effd640 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6188009960 tx=0x7f6188030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d379138d1604372a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61957fa640 1 -- 172.21.15.106:0/2991518168 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188053070 con 0x55b8b59ac3d0 2024-04-13T03:17:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61957fa640 1 -- 172.21.15.106:0/2991518168 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188044990 con 0x55b8b59ac3d0 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.484+0000 7f61957fa640 1 -- 172.21.15.106:0/2991518168 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188035040 con 0x55b8b59ac3d0 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 -- 172.21.15.106:0/2991518168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 msgr2=0x55b8b59aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6188009960 tx=0x7f6188030000 comp rx=0 tx=0).stop 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 -- 172.21.15.106:0/2991518168 shutdown_connections 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b597f4d0 0x55b8b59af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b59aed80 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 --2- 172.21.15.106:0/2991518168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b597ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 -- 172.21.15.106:0/2991518168 >> 172.21.15.106:0/2991518168 conn(0x55b8b5981200 msgr2=0x55b8b59c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 -- 172.21.15.106:0/2991518168 shutdown_connections 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.485+0000 7f61a9049740 1 -- 172.21.15.106:0/2991518168 wait complete. 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 -- start start 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 Processor -- start 2024-04-13T03:17:52.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b59991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 0x55b8b59dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8b59838a0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b8b58a3d00 con 0x55b8b59abff0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.486+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b8b59860e0 con 0x55b8b59c95c0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f61a9049740 10 client.0 _ll_register_callbacks cb 0x55b8b58a36f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35926/0 (socket says 172.21.15.106:35926) 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 -- 172.21.15.106:0/873066766 learned_addr learned my addr 172.21.15.106:0/873066766 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619f7fe640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b59991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619ffff640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 0x55b8b59dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 -- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 msgr2=0x55b8b59dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 0x55b8b59dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 -- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 msgr2=0x55b8b59991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b59991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 -- 172.21.15.106:0/873066766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b55fc3f0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619ffff640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 0x55b8b59dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f619effd640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f61880090b0 tx=0x7f6188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfb6d72a9b2ffd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f617e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188053040 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f618804eaa0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6188044990 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f618804eca0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f61a9049740 10 client.8730 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55b8b55bc0f0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.487+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188044990 con 0x55b8b59ac3d0 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f6188056030 con 0x55b8b59ac3d0 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 10 client.8730 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 10 client.8730 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 20 client.8730 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5999810 0x55b8b5999c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 4 client.8730 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8b58f7b90 con 0x55b8b5999810 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f619f7fe640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5999810 0x55b8b5999c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f617e7fc640 10 client.8730 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.488+0000 7f619f7fe640 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5999810 0x55b8b5999c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8522908dabf88582 server_cookie=143a7377c1fc1f17 in_seq=0 out_seq=0 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f617e7fc640 1 -- 172.21.15.106:0/873066766 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8b58f7b90 con 0x55b8b5999810 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f617e7fc640 10 client.8730 handle_command_reply: tid=0 2024-04-13T03:17:52.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 client.8730 shutdown 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 msgr2=0x55b8b599c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f61880090b0 tx=0x7f6188004730 comp rx=0 tx=0).stop 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 shutdown_connections 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5999810 0x55b8b5999c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59c95c0 0x55b8b59dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59ac3d0 0x55b8b599c320 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f617e7fc640 0 client.8730 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 --2- 172.21.15.106:0/873066766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59abff0 0x55b8b59991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 >> 172.21.15.106:0/873066766 conn(0x55b8b597f4d0 msgr2=0x55b8b5983010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 shutdown_connections 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 1 -- 172.21.15.106:0/873066766 wait complete. 2024-04-13T03:17:52.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.489+0000 7f61a9049740 20 client.8730 trim_cache size 0 max 16384 2024-04-13T03:17:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.506+0000 7f61a9049740 1 Processor -- start 2024-04-13T03:17:52.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.506+0000 7f61a9049740 1 -- start start 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59abff0 0x55b8b59dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59ac3d0 0x55b8b59ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61a02014c0 con 0x55b8b59c95c0 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0200560 con 0x55b8b59ac3d0 2024-04-13T03:17:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f61a0152120 con 0x55b8b59abff0 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35938/0 (socket says 172.21.15.106:35938) 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 -- 172.21.15.106:0/3623075177 learned_addr learned my addr 172.21.15.106:0/3623075177 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 -- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59abff0 msgr2=0x55b8b59dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59abff0 0x55b8b59dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 -- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59ac3d0 msgr2=0x55b8b59ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59ac3d0 0x55b8b59ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.507+0000 7f619ffff640 1 -- 172.21.15.106:0/3623075177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b58ec610 con 0x55b8b59c95c0 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f619ffff640 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6198007680 tx=0x7f6198007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31421a7990a88972 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f6194ff9640 1 -- 172.21.15.106:0/3623075177 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619801c020 con 0x55b8b59c95c0 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f6194ff9640 1 -- 172.21.15.106:0/3623075177 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6198003b90 con 0x55b8b59c95c0 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f6194ff9640 1 -- 172.21.15.106:0/3623075177 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198013790 con 0x55b8b59c95c0 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 -- 172.21.15.106:0/3623075177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 msgr2=0x55b8b59cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6198007680 tx=0x7f6198007b40 comp rx=0 tx=0).stop 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 -- 172.21.15.106:0/3623075177 shutdown_connections 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59c95c0 0x55b8b59cd490 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59ac3d0 0x55b8b59ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 --2- 172.21.15.106:0/3623075177 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59abff0 0x55b8b59dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 -- 172.21.15.106:0/3623075177 >> 172.21.15.106:0/3623075177 conn(0x55b8b597f4d0 msgr2=0x55b8b59c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 -- 172.21.15.106:0/3623075177 shutdown_connections 2024-04-13T03:17:52.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.508+0000 7f61a9049740 1 -- 172.21.15.106:0/3623075177 wait complete. 2024-04-13T03:17:52.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.509+0000 7f61a9049740 1 -- start start 2024-04-13T03:17:52.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.509+0000 7f61a9049740 1 Processor -- start 2024-04-13T03:17:52.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59ac3d0 0x55b8b59cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59c95c0 0x55b8b5a428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8b5986170 con 0x55b8b59abff0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01fff10 con 0x55b8b59c95c0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b8b59802d0 con 0x55b8b59ac3d0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f61a9049740 10 client.0 _ll_register_callbacks cb 0x7f6188044e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35954/0 (socket says 172.21.15.106:35954) 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 -- 172.21.15.106:0/878469978 learned_addr learned my addr 172.21.15.106:0/878469978 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619ffff640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59c95c0 0x55b8b5a428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619effd640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59ac3d0 0x55b8b59cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 -- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59ac3d0 msgr2=0x55b8b59cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59ac3d0 0x55b8b59cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 -- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59c95c0 msgr2=0x55b8b5a428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59c95c0 0x55b8b5a428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 -- 172.21.15.106:0/878469978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b599c230 con 0x55b8b59abff0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f619f7fe640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f6190007810 tx=0x7f61900589e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea91c14c6412dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f617e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190087060 con 0x55b8b59abff0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.510+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f6190071100 con 0x55b8b59abff0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6198003b90 con 0x55b8b59abff0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6190076050 con 0x55b8b59abff0 2024-04-13T03:17:52.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 10 client.8736 fetch_fsmap learned FSMap version 71 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55b8b59986d0 con 0x55b8b59abff0 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6198003b90 con 0x55b8b59abff0 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f61900581d0 con 0x55b8b59abff0 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 10 client.8736 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 10 client.8736 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 20 client.8736 populate_metadata read hostname 'smithi106' 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5a42ef0 0x55b8b597de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 4 client.8736 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8b58f1a00 con 0x55b8b5a42ef0 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f619effd640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5a42ef0 0x55b8b597de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f617e7fc640 10 client.8736 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.511+0000 7f619effd640 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5a42ef0 0x55b8b597de80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a459668541506737 server_cookie=b58d349b3cbf8ede in_seq=0 out_seq=0 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f617e7fc640 1 -- 172.21.15.106:0/878469978 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55b8b58f1a00 con 0x55b8b5a42ef0 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f617e7fc640 10 client.8736 handle_command_reply: tid=0 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 client.8736 shutdown 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 msgr2=0x55b8b59cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f6190007810 tx=0x7f61900589e0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 shutdown_connections 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55b8b5a42ef0 0x55b8b597de80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b8b59ac3d0 0x55b8b59cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b8b59c95c0 0x55b8b5a428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 --2- 172.21.15.106:0/878469978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8b59abff0 0x55b8b59cd350 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f617e7fc640 0 client.8736 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:17:52.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 >> 172.21.15.106:0/878469978 conn(0x55b8b597f4d0 msgr2=0x55b8b59a99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 shutdown_connections 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 1 -- 172.21.15.106:0/878469978 wait complete. 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a9049740 20 client.8736 trim_cache size 0 max 16384 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:17:52.522 INFO:teuthology.orchestra.run.smithi106.stdout: "272453ff-073f-4c1d-9896-9d9da95e590c": { 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/", 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "272453ff-073f-4c1d-9896-9d9da95e590c", 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:17:52.523 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 msgr2=0x7f618006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 0x7f618006edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6188036000 tx=0x7f618803c040 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 msgr2=0x7f61a01f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f619000c3c0 tx=0x7f619000c880 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 shutdown_connections 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a015fb50 0x7f61a00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00d07c0 0x7f61a00aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.512+0000 7f61a6236640 1 --2- 172.21.15.106:0/2650771887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a006bff0 0x7f61a01f7970 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:17:52.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.513+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 >> 172.21.15.106:0/2650771887 conn(0x7f61a00cd6a0 msgr2=0x7f61a00b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:17:52.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.513+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 shutdown_connections 2024-04-13T03:17:52.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:52.513+0000 7f61a6236640 1 -- 172.21.15.106:0/2650771887 wait complete. 2024-04-13T03:17:52.526 INFO:tasks.cephfs.filesystem:scrub status for tag:272453ff-073f-4c1d-9896-9d9da95e590c - {'path': '/', 'tag': '272453ff-073f-4c1d-9896-9d9da95e590c', 'options': 'recursive,repair'} 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:54.078+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 7 ==== log(4 entries from seq 1 at 2024-04-13T03:17:52.358141+0000) ==== 695+0+0 (secure 0 0 0) 0x7f68dc057a80 con 0x7f68e0146220 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stdout: cluster: 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stdout: id: d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stdout: health: HEALTH_OK 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:54.079 INFO:teuthology.orchestra.run.smithi106.stdout: services: 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: mgr: y(active, since 8m), standbys: x 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: mds: 1/1 daemons up, 3 standby 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: data: 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: volumes: 1/1 healthy 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: pools: 3 pools, 97 pgs 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: objects: 26 objects, 6.6 MiB 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: usage: 267 MiB used, 720 GiB / 720 GiB avail 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: pgs: 97 active+clean 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: io: 2024-04-13T03:17:54.080 INFO:teuthology.orchestra.run.smithi106.stdout: client: 513 KiB/s wr, 0 op/s rd, 0 op/s wr 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:17:52.358141+0000 mds.b [INF] scrub queued for path: / 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:17:52.358144+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:17:52.358148+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-13T03:17:54.081 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:17:52.359336+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:17:55.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:17:55.083+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 8 ==== log(1 entries from seq 5 at 2024-04-13T03:17:54.349770+0000) ==== 189+0+0 (secure 0 0 0) 0x7f68dc057ce0 con 0x7f68e0146220 2024-04-13T03:17:55.085 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:17:54.349770+0000 mds.b [INF] scrub summary: idle 2024-04-13T03:17:56.233 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:17:56.236 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:18:07.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:07.102+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 9 ==== log(1 entries from seq 1172 at 2024-04-13T03:18:06.752250+0000) ==== 323+0+0 (secure 0 0 0) 0x7f68dc08a1b0 con 0x7f68e0146220 2024-04-13T03:18:08.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:08.107+0000 7f68d5ffb640 1 -- 172.21.15.106:0/728861241 <== mon.2 v2:172.21.15.114:3301/0 10 ==== log(1 entries from seq 1173 at 2024-04-13T03:18:07.106038+0000) ==== 325+0+0 (secure 0 0 0) 0x7f68dc009280 con 0x7f68e0146220 2024-04-13T03:18:22.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:18:22.527 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-13T03:18:22.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.638+0000 7f72a7bed640 1 Processor -- start 2024-04-13T03:18:22.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.639+0000 7f72a7bed640 1 -- start start 2024-04-13T03:18:22.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.639+0000 7f72a7bed640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f72a006f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.639+0000 7f72a7bed640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a00b5090 con 0x7f72a006b8e0 2024-04-13T03:18:22.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.639+0000 7f72a7bed640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a00b3ff0 con 0x7f72a00af800 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.639+0000 7f72a7bed640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f72a006fba0 con 0x7f72a006f200 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a618a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f72a006b8e0 0x7f72a006bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57320/0 (socket says 172.21.15.106:57320) 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a618a640 1 -- 172.21.15.106:0/358178964 learned_addr learned my addr 172.21.15.106:0/358178964 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565447674 0 0) 0x7f72a00b5090 con 0x7f72a006b8e0 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a5188640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f72a006f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f728c004630 con 0x7f72a006b8e0 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 13833531 0 0) 0x7f728c004630 con 0x7f72a006b8e0 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 msgr2=0x7f72a006f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f72a006f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 >> v1:172.21.15.114:6789/0 conn(0x7f72a00af800 legacy=0x7f72a015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a00b42d0 con 0x7f72a006b8e0 2024-04-13T03:18:22.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f72980019f0 con 0x7f72a006b8e0 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 >> v1:172.21.15.106:6789/0 conn(0x7f72a006b8e0 legacy=0x7f72a006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f728c004af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f728c005070 0x7f728c005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f728c005e50 0x7f728c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f728c00cb80 con 0x7f728c005e50 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f728c00cd30 con 0x7f728c005070 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f728c00cf30 con 0x7f72a006f200 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a618a640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f728c005e50 0x7f728c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a5989640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f728c004af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.640+0000 7f72a5188640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f728c005070 0x7f728c005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 -- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 msgr2=0x7f728c004af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a006f200 0x7f728c004af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 -- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f728c005070 msgr2=0x7f728c005910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f728c005070 0x7f728c005910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 -- 172.21.15.106:0/358178964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c004630 con 0x7f728c005e50 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a618a640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f728c005e50 0x7f728c0062d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f729800a840 tx=0x7f728c006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3b774b833f4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.641+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729800a460 con 0x7f728c005e50 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298034650 con 0x7f728c005e50 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a4987640 1 -- 172.21.15.106:0/358178964 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298034a40 con 0x7f728c005e50 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 -- 172.21.15.106:0/358178964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f728c005e50 msgr2=0x7f728c0062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 --2- 172.21.15.106:0/358178964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f728c005e50 0x7f728c0062d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f729800a840 tx=0x7f728c006940 comp rx=0 tx=0).stop 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a618a640 1 -- 172.21.15.106:0/358178964 reap_dead start 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 -- 172.21.15.106:0/358178964 shutdown_connections 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 -- 172.21.15.106:0/358178964 >> 172.21.15.106:0/358178964 conn(0x7f72a00b2600 msgr2=0x7f72a00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 -- 172.21.15.106:0/358178964 shutdown_connections 2024-04-13T03:18:22.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.642+0000 7f72a7bed640 1 -- 172.21.15.106:0/358178964 wait complete. 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.643+0000 7f72a7bed640 1 Processor -- start 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.643+0000 7f72a7bed640 1 -- start start 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 0x7f72a0151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a0151fc0 0x7f72a01f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00b4b70 con 0x7f72a0145470 2024-04-13T03:18:22.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00b4890 con 0x7f72a014a720 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a7bed640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f72a00b3cc0 con 0x7f72a0151fc0 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47502/0 (socket says 172.21.15.106:47502) 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 -- 172.21.15.106:0/2327653957 learned_addr learned my addr 172.21.15.106:0/2327653957 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5188640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 0x7f72a0151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a618a640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a0151fc0 0x7f72a01f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 -- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a0151fc0 msgr2=0x7f72a01f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a0151fc0 0x7f72a01f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 -- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 msgr2=0x7f72a0151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 0x7f72a0151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5989640 1 -- 172.21.15.106:0/2327653957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a014cff0 con 0x7f72a0145470 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.644+0000 7f72a5188640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 0x7f72a0151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72a5989640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f729400f820 tx=0x7f729400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72927fc640 1 -- 172.21.15.106:0/2327653957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940050d0 con 0x7f72a0145470 2024-04-13T03:18:22.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72927fc640 1 -- 172.21.15.106:0/2327653957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294018070 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a014b730 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72927fc640 1 -- 172.21.15.106:0/2327653957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294005490 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.645+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a014ae90 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.646+0000 7f72927fc640 1 -- 172.21.15.106:0/2327653957 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f729400d730 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.647+0000 7f72927fc640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 0x7f728406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.647+0000 7f72927fc640 1 -- 172.21.15.106:0/2327653957 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6130+0+0 (secure 0 0 0) 0x7f729408bb90 con 0x7f72a0145470 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.647+0000 7f72a5188640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 0x7f728406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.647+0000 7f72a5188640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 0x7f728406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f72a0153900 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.648+0000 7f72aa9d6740 1 Processor -- start 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.648+0000 7f72aa9d6740 1 -- start start 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fca1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c02fc0d170 con 0x55c02fca24d0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c02fbb37f0 con 0x55c02fcceff0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c02fc09130 con 0x55c02fccf3d0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47516/0 (socket says 172.21.15.106:47516) 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 -- 172.21.15.106:0/2019860112 learned_addr learned my addr 172.21.15.106:0/2019860112 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a5989640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fca1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a5188640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 -- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 msgr2=0x55c02fcd1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 -- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 msgr2=0x55c02fca1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fca1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 -- 172.21.15.106:0/2019860112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02f9e97d0 con 0x55c02fca24d0 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a5188640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:22.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72a618a640 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f729802cc50 tx=0x7f7298007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb807f1437e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72837fe640 1 -- 172.21.15.106:0/2019860112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298001fb0 con 0x55c02fca24d0 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72837fe640 1 -- 172.21.15.106:0/2019860112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298046070 con 0x55c02fca24d0 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.649+0000 7f72837fe640 1 -- 172.21.15.106:0/2019860112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980017f0 con 0x55c02fca24d0 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 -- 172.21.15.106:0/2019860112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 msgr2=0x55c02fcd22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f729802cc50 tx=0x7f7298007000 comp rx=0 tx=0).stop 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 -- 172.21.15.106:0/2019860112 shutdown_connections 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fca24d0 0x55c02fcd22c0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/2019860112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fca1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 -- 172.21.15.106:0/2019860112 >> 172.21.15.106:0/2019860112 conn(0x55c02fca4200 msgr2=0x55c02fcecd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 -- 172.21.15.106:0/2019860112 shutdown_connections 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.650+0000 7f72aa9d6740 1 -- 172.21.15.106:0/2019860112 wait complete. 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.651+0000 7f72aa9d6740 1 -- start start 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.651+0000 7f72aa9d6740 1 Processor -- start 2024-04-13T03:18:22.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcc2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c02fca68a0 con 0x55c02fcec260 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c02fbc6d00 con 0x55c02fcceff0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c02fca90e0 con 0x55c02fccf3d0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 10 client.0 _ll_register_callbacks cb 0x55c02fbc66f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47530/0 (socket says 172.21.15.106:47530) 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 -- 172.21.15.106:0/966023903 learned_addr learned my addr 172.21.15.106:0/966023903 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a5188640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a5989640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcc2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 -- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 msgr2=0x55c02fcea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 -- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 msgr2=0x55c02fcc2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcc2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 -- 172.21.15.106:0/966023903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02f91f3f0 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72a618a640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7298042000 tx=0x7f72a00b41c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924dc39215909fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f7280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980017b0 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6130+0+0 (secure 0 0 0) 0x7f7298001950 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.652+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7298046070 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f729805a050 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 10 client.8748 fetch_fsmap learned FSMap version 71 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55c02f8df0f0 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298046070 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f729805d020 con 0x55c02fcec260 2024-04-13T03:18:22.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 10 client.8748 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 10 client.8748 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 20 client.8748 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fcd33f0 0x55c02fcd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 4 client.8748 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c02fc1ab90 con 0x55c02fcd33f0 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72a5989640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fcd33f0 0x55c02fcd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f72a5989640 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fcd33f0 0x55c02fcd3800 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c9866004090a3 server_cookie=7393c5c786ee00f7 in_seq=0 out_seq=0 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.653+0000 7f7280ff9640 10 client.8748 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f7280ff9640 1 -- 172.21.15.106:0/966023903 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c02fc1ab90 con 0x55c02fcd33f0 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f7280ff9640 10 client.8748 handle_command_reply: tid=0 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 client.8748 shutdown 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 msgr2=0x55c02fcec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7298042000 tx=0x7f72a00b41c0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 shutdown_connections 2024-04-13T03:18:22.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fcd33f0 0x55c02fcd3800 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fcec6e0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/966023903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcc2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f7280ff9640 0 client.8748 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 >> 172.21.15.106:0/966023903 conn(0x55c02fca24d0 msgr2=0x55c02fcf0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 shutdown_connections 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.654+0000 7f72aa9d6740 1 -- 172.21.15.106:0/966023903 wait complete. 2024-04-13T03:18:22.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.655+0000 7f72aa9d6740 20 client.8748 trim_cache size 0 max 16384 2024-04-13T03:18:22.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.672+0000 7f72aa9d6740 1 Processor -- start 2024-04-13T03:18:22.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.672+0000 7f72aa9d6740 1 -- start start 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcd37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c02fca3050 con 0x55c02fcec260 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01fdbb0 con 0x55c02fcceff0 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f72a01fcc30 con 0x55c02fccf3d0 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52748/0 (socket says 172.21.15.106:52748) 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 -- 172.21.15.106:0/4215956824 learned_addr learned my addr 172.21.15.106:0/4215956824 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a5989640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcd37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a5188640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 -- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 msgr2=0x55c02fcd1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 -- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 msgr2=0x55c02fcd37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcd37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 -- 172.21.15.106:0/4215956824 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02fca2e80 con 0x55c02fcec260 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a5188640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.673+0000 7f72a618a640 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7298001ca0 tx=0x7f729804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d7f448886f1415 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f7282ffd640 1 -- 172.21.15.106:0/4215956824 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298063070 con 0x55c02fcec260 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f7282ffd640 1 -- 172.21.15.106:0/4215956824 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298046070 con 0x55c02fcec260 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f7282ffd640 1 -- 172.21.15.106:0/4215956824 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298001930 con 0x55c02fcec260 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 -- 172.21.15.106:0/4215956824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 msgr2=0x55c02fd11ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7298001ca0 tx=0x7f729804b000 comp rx=0 tx=0).stop 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 -- 172.21.15.106:0/4215956824 shutdown_connections 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fccf3d0 0x55c02fcd1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fcec260 0x55c02fd11ef0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/4215956824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcceff0 0x55c02fcd37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 -- 172.21.15.106:0/4215956824 >> 172.21.15.106:0/4215956824 conn(0x55c02fca24d0 msgr2=0x55c02fcc2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 -- 172.21.15.106:0/4215956824 shutdown_connections 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.674+0000 7f72aa9d6740 1 -- 172.21.15.106:0/4215956824 wait complete. 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.675+0000 7f72aa9d6740 1 -- start start 2024-04-13T03:18:22.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.675+0000 7f72aa9d6740 1 Processor -- start 2024-04-13T03:18:22.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 0x55c02fcf0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 0x55c02fca0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01fc5e0 con 0x55c02fccf3d0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01fbf90 con 0x55c02fcec260 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c02fcc33b0 con 0x55c02fcceff0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 10 client.0 _ll_register_callbacks cb 0x7f729800a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52756/0 (socket says 172.21.15.106:52756) 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 -- 172.21.15.106:0/722515957 learned_addr learned my addr 172.21.15.106:0/722515957 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5989640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 0x55c02fcf0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a618a640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 0x55c02fca0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 -- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 msgr2=0x55c02fcf0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 0x55c02fcf0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 -- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 msgr2=0x55c02fca0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 0x55c02fca0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 -- 172.21.15.106:0/722515957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02fcd1dc0 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a618a640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 0x55c02fca0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5989640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 0x55c02fcf0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72a5188640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x55c02fca8d20 tx=0x7f729c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ba01750a86b897 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f7280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c016bd0 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6130+0+0 (secure 0 0 0) 0x7f729c00f040 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.676+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7298046070 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f729c036050 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 10 client.8754 fetch_fsmap learned FSMap version 71 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55c02fce8e80 con 0x55c02fccf3d0 2024-04-13T03:18:22.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298046070 con 0x55c02fccf3d0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f729c039020 con 0x55c02fccf3d0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 10 client.8754 fetch_fsmap finished waiting for FSMap version 71 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 10 client.8754 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 20 client.8754 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fd6b7a0 0x55c02fca13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 4 client.8754 mds_command: new command op to 8430 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 --> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c02fcc4eb0 con 0x55c02fd6b7a0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72a5989640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fd6b7a0 0x55c02fca13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f72a5989640 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fd6b7a0 0x55c02fca13c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3252ae77db2e8d9b server_cookie=7be10de5f92c7dcc in_seq=0 out_seq=0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.677+0000 7f7280ff9640 10 client.8754 ms_handle_connect on v2:172.21.15.114:6834/2645971931 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f7280ff9640 1 -- 172.21.15.106:0/722515957 <== mds.0 v2:172.21.15.114:6834/2645971931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c02fcc4eb0 con 0x55c02fd6b7a0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f7280ff9640 10 client.8754 handle_command_reply: tid=0 2024-04-13T03:18:22.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 client.8754 shutdown 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 msgr2=0x55c02fcf0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x55c02fca8d20 tx=0x7f729c004730 comp rx=0 tx=0).stop 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 shutdown_connections 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:6834/2645971931,v1:172.21.15.114:6835/2645971931] conn(0x55c02fd6b7a0 0x55c02fca13c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c02fcec260 0x55c02fca0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c02fccf3d0 0x55c02fcf0660 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 --2- 172.21.15.106:0/722515957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c02fcceff0 0x55c02fcf0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f7280ff9640 0 client.8754 ms_handle_reset on v2:172.21.15.114:6834/2645971931 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 >> 172.21.15.106:0/722515957 conn(0x55c02fca24d0 msgr2=0x55c02fceef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 shutdown_connections 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 1 -- 172.21.15.106:0/722515957 wait complete. 2024-04-13T03:18:22.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72aa9d6740 20 client.8754 trim_cache size 0 max 16384 2024-04-13T03:18:22.687 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:22.687 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:18:22.687 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:18:22.687 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:22.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 msgr2=0x7f728406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 0x7f728406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f72a0153900 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-04-13T03:18:22.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 msgr2=0x7f72a015d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f729400f820 tx=0x7f729400fce0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 shutdown_connections 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f728406c8f0 0x7f728406edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72a0151fc0 0x7f72a01f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72a014a720 0x7f72a0151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 --2- 172.21.15.106:0/2327653957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a0145470 0x7f72a015d110 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.678+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 >> 172.21.15.106:0/2327653957 conn(0x7f72a00b2600 msgr2=0x7f72a0149540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.679+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 shutdown_connections 2024-04-13T03:18:22.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.679+0000 7f72a7bed640 1 -- 172.21.15.106:0/2327653957 wait complete. 2024-04-13T03:18:22.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:18:22.792 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:18:22.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.896+0000 7f4ea25bb640 1 Processor -- start 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.896+0000 7f4ea25bb640 1 -- start start 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4ea25bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e9c15e930 0x7f4e9c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4ea25bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9c06f200 con 0x7f4e9c06ed70 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4ea25bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9c06e280 con 0x7f4e9c0d09f0 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4ea25bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c06e560 con 0x7f4e9c15e930 2024-04-13T03:18:22.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4e9c15e930 0x7f4e9c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4e9c06ed70 0x7f4e9c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59692/0 (socket says 172.21.15.106:59692) 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9b7fe640 1 -- 172.21.15.106:0/3113513476 learned_addr learned my addr 172.21.15.106:0/3113513476 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9bfff640 1 -- 172.21.15.106:0/3113513476 >> v1:172.21.15.114:6789/0 conn(0x7f4e9c0d09f0 legacy=0x7f4e9c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9bfff640 1 -- 172.21.15.106:0/3113513476 >> v1:172.21.15.106:6789/0 conn(0x7f4e9c06ed70 legacy=0x7f4e9c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9bfff640 1 -- 172.21.15.106:0/3113513476 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c0cccf0 con 0x7f4e9c15e930 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.897+0000 7f4e9affd640 1 -- 172.21.15.106:0/3113513476 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175433189 0 0) 0x7f4e9c06f200 con 0x7f4e9c06ed70 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.898+0000 7f4e9bfff640 1 --2- 172.21.15.106:0/3113513476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c15e930 0x7f4e9c0b5350 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097a0 tx=0x7f4e8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac55d7e3be51f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.898+0000 7f4e9affd640 1 -- 172.21.15.106:0/3113513476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c035020 con 0x7f4e9c15e930 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.898+0000 7f4e9affd640 1 -- 172.21.15.106:0/3113513476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c031830 con 0x7f4e9c15e930 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.898+0000 7f4e9affd640 1 -- 172.21.15.106:0/3113513476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c031c00 con 0x7f4e9c15e930 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 -- 172.21.15.106:0/3113513476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c15e930 msgr2=0x7f4e9c0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/3113513476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c15e930 0x7f4e9c0b5350 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097a0 tx=0x7f4e8c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 -- 172.21.15.106:0/3113513476 shutdown_connections 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/3113513476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c15e930 0x7f4e9c0b5350 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 -- 172.21.15.106:0/3113513476 >> 172.21.15.106:0/3113513476 conn(0x7f4e9c0cef80 msgr2=0x7f4e9c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:22.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 -- 172.21.15.106:0/3113513476 shutdown_connections 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.899+0000 7f4ea25bb640 1 -- 172.21.15.106:0/3113513476 wait complete. 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.900+0000 7f4ea25bb640 1 Processor -- start 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.900+0000 7f4ea25bb640 1 -- start start 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e9c0d09f0 0x7f4e9c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e9c15e930 0x7f4e9c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c0ce830 con 0x7f4e9c06ed70 2024-04-13T03:18:22.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c0cd2b0 con 0x7f4e9c15e930 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c0ce9b0 con 0x7f4e9c0d09f0 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52782/0 (socket says 172.21.15.106:52782) 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 -- 172.21.15.106:0/1889569380 learned_addr learned my addr 172.21.15.106:0/1889569380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9b7fe640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e9c0d09f0 0x7f4e9c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea0b58640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e9c15e930 0x7f4e9c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 -- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e9c0d09f0 msgr2=0x7f4e9c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e9c0d09f0 0x7f4e9c0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 -- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e9c15e930 msgr2=0x7f4e9c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:22.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e9c15e930 0x7f4e9c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:22.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 -- 172.21.15.106:0/1889569380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c0cc4e0 con 0x7f4e9c06ed70 2024-04-13T03:18:22.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e9bfff640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097a0 tx=0x7f4e8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004230 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c0043f0 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c0c6190 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c003850 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.901+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c0c6750 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.902+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c068a10 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.903+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e8c04c060 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.903+0000 7f4e98ff9640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.903+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6130+0+0 (secure 0 0 0) 0x7f4e8c0ad850 con 0x7f4e9c06ed70 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.903+0000 7f4e9b7fe640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:22.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.904+0000 7f4e9b7fe640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 0x7f4e8806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0b3cb0 tx=0x7f4e90007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:22.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:22.908+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e8c040070 con 0x7f4e9c06ed70 2024-04-13T03:18:23.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.120+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4e9c0c6e50 con 0x7f4e9c06ed70 2024-04-13T03:18:23.121 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:18:23.121+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:18:23.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.122+0000 7f4e98ff9640 1 -- 172.21.15.106:0/1889569380 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) ==== 161+0+0 (secure 0 0 0) 0x7f4e8c07ba50 con 0x7f4e9c06ed70 2024-04-13T03:18:23.123 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:18:23.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 msgr2=0x7f4e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 0x7f4e8806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0b3cb0 tx=0x7f4e90007b50 comp rx=0 tx=0).stop 2024-04-13T03:18:23.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 msgr2=0x7f4e9c0c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097a0 tx=0x7f4e8c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 shutdown_connections 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4e8806c8f0 0x7f4e8806edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4e9c0d09f0 0x7f4e9c0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e9c15e930 0x7f4e9c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 --2- 172.21.15.106:0/1889569380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e9c06ed70 0x7f4e9c0c5bb0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 >> 172.21.15.106:0/1889569380 conn(0x7f4e9c0cef80 msgr2=0x7f4e9c0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 shutdown_connections 2024-04-13T03:18:23.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.124+0000 7f4ea25bb640 1 -- 172.21.15.106:0/1889569380 wait complete. 2024-04-13T03:18:23.130 INFO:tasks.ceph.mds.b.smithi114.stderr: -7363> 2024-04-13T03:17:35.340+0000 7f4e7dff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:18:23.137 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-13T03:18:23.163 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:18:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.179+0000 7f4f4d2cb280 1 -- start start 2024-04-13T03:18:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.179+0000 7f4f4d2cb280 1 Processor -- start 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5585153e8c00 0x5585161b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585153e9800 0x5585161b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585161cc180 con 0x5585153e9c00 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585161cc300 con 0x5585153e9800 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5585161cc480 con 0x5585153e8c00 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52796/0 (socket says 172.21.15.106:52796) 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 -- 172.21.15.106:0/1733133324 learned_addr learned my addr 172.21.15.106:0/1733133324 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4affb640 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585153e9800 0x5585161b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5585153e8c00 0x5585161b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 -- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5585153e8c00 msgr2=0x5585161b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5585153e8c00 0x5585161b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 -- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585153e9800 msgr2=0x5585161b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585153e9800 0x5585161b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 -- 172.21.15.106:0/1733133324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558515420780 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4bffd640 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x558516212ab0 tx=0x55851617c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64fa04172089fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f48ff7640 1 -- 172.21.15.106:0/1733133324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e9c0 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f48ff7640 1 -- 172.21.15.106:0/1733133324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558515414fc0 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f48ff7640 1 -- 172.21.15.106:0/1733133324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e4e0 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585154214a0 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558515420000 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.180+0000 7f4f4d2cb280 4 waiting for MDS map... 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f48ff7640 1 -- 172.21.15.106:0/1733133324 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x5585162bc200 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f48ff7640 1 -- 172.21.15.106:0/1733133324 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 72) ==== 2885+0+0 (secure 0 0 0) 0x558515445180 con 0x5585153e9c00 2024-04-13T03:18:23.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f4d2cb280 4 Got MDS map 72 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f4d2cb280 10 main: JournalTool::main 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f4d2cb280 4 main: JournalTool: connecting to RADOS... 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f4d2cb280 1 Processor -- start 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.181+0000 7f4f4d2cb280 1 -- start start 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585161cc600 con 0x55851631c400 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585161cc780 con 0x55851631c800 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5585161cc900 con 0x55851631c000 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52798/0 (socket says 172.21.15.106:52798) 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/4075594487 learned_addr learned my addr 172.21.15.106:0/4075594487 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4affb640 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4affb640 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 msgr2=0x5585161b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c800 msgr2=0x5585161b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/4075594487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585154203c0 con 0x55851631c400 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.182+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x5585162c8900 tx=0x5585162d1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94123906a07532ce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f46ff3640 1 -- 172.21.15.106:0/4075594487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e9c0 con 0x55851631c400 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f46ff3640 1 -- 172.21.15.106:0/4075594487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558515414c40 con 0x55851631c400 2024-04-13T03:18:23.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f46ff3640 1 -- 172.21.15.106:0/4075594487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e4e0 con 0x55851631c400 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/4075594487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 msgr2=0x5585161b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x5585162c8900 tx=0x5585162d1800 comp rx=0 tx=0).stop 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/4075594487 shutdown_connections 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/4075594487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/4075594487 >> 172.21.15.106:0/4075594487 conn(0x5585161d7c00 msgr2=0x5585154d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/4075594487 shutdown_connections 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.183+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/4075594487 wait complete. 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 Processor -- start 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 -- start start 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585161cdc80 con 0x55851631c800 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585161cdb00 con 0x55851631c400 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4d2cb280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5585161cd980 con 0x55851631c000 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4affb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4affb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 0x5585161b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52814/0 (socket says 172.21.15.106:52814) 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 learned_addr learned my addr 172.21.15.106:0/113720675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.184+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 msgr2=0x5585161b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55851631c000 0x5585161b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c400 msgr2=0x5585161b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851631c400 0x5585161b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558515420960 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 0x5585161b5b80 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x5585162c95c0 tx=0x5585162d1aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f457f0640 1 -- 172.21.15.106:0/113720675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e9c0 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f457f0640 1 -- 172.21.15.106:0/113720675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558515414a80 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558515421680 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f457f0640 1 -- 172.21.15.106:0/113720675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55851541e4e0 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558515421860 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.185+0000 7f4f4d2cb280 4 main: JournalTool: resolving pool 15 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f457f0640 1 -- 172.21.15.106:0/113720675 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55851632cc00 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f457f0640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55851633cc00 0x5585161b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f457f0640 1 -- 172.21.15.106:0/113720675 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x55851632b000 con 0x55851631c800 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4d2cb280 4 main: JournalTool: creating IoCtx.. 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55851633cc00 0x5585161b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4d2cb280 4 main: Executing for rank 0 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4d2cb280 4 Filtering by inode '1099511627777' 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55851631cc00 0x5585161b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.186+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55851631d000 con 0x55851631cc00 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55851631cc00 0x5585161b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4b7fc640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55851633cc00 0x5585161b6100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5585162c8900 tx=0x55851617c9c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4affb640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55851631cc00 0x5585161b6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5585154ea000 con 0x55851631cc00 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4d2cb280 4 Pointer 400.00000000 is readable 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4d2cb280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.187+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55851633d000 con 0x55851631cc00 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5585154ea000 con 0x55851631cc00 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Header 0x400000 0x407d21 0x40809d 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Starting journal scan from offset 0x407d21 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55851633d400 con 0x55851631cc00 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4affb640 1 -- 172.21.15.106:0/113720675 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5585154ea000 con 0x55851631cc00 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-13T03:18:23.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Parsing data, 0x37c bytes available 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Attempting decode at 0x407d21 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Consumed 0x37c bytes 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 10 Valid entry at 0x407d21 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55851633d800 0x5585161b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55851633dc00 con 0x55851633d800 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.188+0000 7f4f4bffd640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55851633d800 0x5585161b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4bffd640 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55851633d800 0x5585161b6c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4bffd640 1 -- 172.21.15.106:0/113720675 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585154ea500 con 0x55851633d800 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 Reached end of journal objects 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 Scanned objects, 0 missing, 1 valid 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 Events scanned, 0 gaps 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 Found 1 valid events 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 Selected 0 events events for processing 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stdout:Events by type: 2024-04-13T03:18:23.199 INFO:teuthology.orchestra.run.smithi106.stdout:Errors: 0 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 4 0 events 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55851631cc00 msgr2=0x5585161b6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55851631cc00 0x5585161b6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55851633d800 msgr2=0x5585161b6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55851633d800 0x5585161b6c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55851633cc00 msgr2=0x5585161b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55851633cc00 0x5585161b6100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5585162c8900 tx=0x55851617c9c0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 msgr2=0x5585161b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/113720675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55851631c800 0x5585161b5b80 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x5585162c95c0 tx=0x5585162d1aa0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/113720675 reap_dead start 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4b7fc640 1 -- 172.21.15.106:0/113720675 reap_dead start 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 shutdown_connections 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 >> 172.21.15.106:0/113720675 conn(0x5585161d7c00 msgr2=0x5585154d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 shutdown_connections 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.189+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/113720675 wait complete. 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 msgr2=0x5585161b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x558516212ab0 tx=0x55851617c660 comp rx=0 tx=0).stop 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 shutdown_connections 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585153e9c00 0x5585161b4b00 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585153e9800 0x5585161b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 --2- 172.21.15.106:0/1733133324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5585153e8c00 0x5585161b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 >> 172.21.15.106:0/1733133324 conn(0x5585153e9000 msgr2=0x5585154d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 shutdown_connections 2024-04-13T03:18:23.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.190+0000 7f4f4d2cb280 1 -- 172.21.15.106:0/1733133324 wait complete. 2024-04-13T03:18:23.204 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.067699, result: Events by type: Errors: 0 2024-04-13T03:18:23.204 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-13T03:18:23.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 -- start start 2024-04-13T03:18:23.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 Processor -- start 2024-04-13T03:18:23.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55852666ec00 0x5585267dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852666f800 0x5585267dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558527442180 con 0x55852666fc00 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558527442300 con 0x55852666f800 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.284+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558527442480 con 0x55852666ec00 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52822/0 (socket says 172.21.15.106:52822) 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 -- 172.21.15.106:0/1417976943 learned_addr learned my addr 172.21.15.106:0/1417976943 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55852666ec00 0x5585267dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852666f800 0x5585267dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 -- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55852666ec00 msgr2=0x5585267dc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55852666ec00 0x5585267dc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 -- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852666f800 msgr2=0x5585267dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852666f800 0x5585267dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 -- 172.21.15.106:0/1417976943 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585266a6780 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64dd5fd640 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x5585274873b0 tx=0x558527404240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd60be6b5f6b70bb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64da5f7640 1 -- 172.21.15.106:0/1417976943 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a44e0 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64da5f7640 1 -- 172.21.15.106:0/1417976943 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55852669afc0 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64da5f7640 1 -- 172.21.15.106:0/1417976943 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a4b60 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585266a63c0 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5585266a7680 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64df042280 4 waiting for MDS map... 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64da5f7640 1 -- 172.21.15.106:0/1417976943 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x558527534a00 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64da5f7640 1 -- 172.21.15.106:0/1417976943 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 72) ==== 2885+0+0 (secure 0 0 0) 0x5585266cb180 con 0x55852666fc00 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64df042280 4 Got MDS map 72 2024-04-13T03:18:23.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.285+0000 7f64df042280 10 main: JournalTool::main 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.286+0000 7f64df042280 4 main: JournalTool: connecting to RADOS... 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.286+0000 7f64df042280 1 Processor -- start 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.286+0000 7f64df042280 1 -- start start 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558527442600 con 0x558527537800 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558527442780 con 0x558527537400 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558527442900 con 0x558527537c00 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52828/0 (socket says 172.21.15.106:52828) 2024-04-13T03:18:23.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1666067930 learned_addr learned my addr 172.21.15.106:0/1666067930 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 msgr2=0x5585267ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 msgr2=0x5585267dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1666067930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585266a74a0 con 0x558527537800 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x558527486a80 tx=0x558527402a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ac5df3e1e76dda server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64d85f3640 1 -- 172.21.15.106:0/1666067930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a49c0 con 0x558527537800 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64d85f3640 1 -- 172.21.15.106:0/1666067930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55852669ae00 con 0x558527537800 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.287+0000 7f64d85f3640 1 -- 172.21.15.106:0/1666067930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a44e0 con 0x558527537800 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- 172.21.15.106:0/1666067930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 msgr2=0x5585267dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x558527486a80 tx=0x558527402a20 comp rx=0 tx=0).stop 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- 172.21.15.106:0/1666067930 shutdown_connections 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 --2- 172.21.15.106:0/1666067930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- 172.21.15.106:0/1666067930 >> 172.21.15.106:0/1666067930 conn(0x558527537000 msgr2=0x55852675b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- 172.21.15.106:0/1666067930 shutdown_connections 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- 172.21.15.106:0/1666067930 wait complete. 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 Processor -- start 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.288+0000 7f64df042280 1 -- start start 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558527443b00 con 0x558527537800 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558527443980 con 0x558527537400 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558527443800 con 0x558527537c00 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52830/0 (socket says 172.21.15.106:52830) 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1004995635 learned_addr learned my addr 172.21.15.106:0/1004995635 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 msgr2=0x5585267ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x558527537c00 0x5585267ddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 msgr2=0x5585267dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558527537400 0x5585267dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585266a6780 con 0x558527537800 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64dcdfc640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x55852754e810 tx=0x5585274024e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64d6df0640 1 -- 172.21.15.106:0/1004995635 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a4b60 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64d6df0640 1 -- 172.21.15.106:0/1004995635 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55852669ac40 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64d6df0640 1 -- 172.21.15.106:0/1004995635 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585266a4000 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.289+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585266a63c0 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.290+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585266a6000 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.290+0000 7f64df042280 4 main: JournalTool: resolving pool 15 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64d6df0640 1 -- 172.21.15.106:0/1004995635 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x558527528c00 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64d6df0640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5585269e8000 0x5585267de100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64d6df0640 1 -- 172.21.15.106:0/1004995635 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x5585275be600 con 0x558527537800 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5585269e8000 0x5585267de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64df042280 4 main: JournalTool: creating IoCtx.. 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64df042280 4 main: Executing for rank 0 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64df042280 4 Filtering by inode '1099511627778' 2024-04-13T03:18:23.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5585269e8400 0x5585267de680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5585269e8800 con 0x5585269e8400 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5585269e8400 0x5585267de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.291+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5585269e8000 0x5585267de100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55852754f350 tx=0x558527402780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64dc5fb640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5585269e8400 0x5585267de680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64dc5fb640 1 -- 172.21.15.106:0/1004995635 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x558526770000 con 0x5585269e8400 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 4 Pointer 400.00000000 is readable 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5585269e8c00 con 0x5585269e8400 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64dc5fb640 1 -- 172.21.15.106:0/1004995635 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x558526770000 con 0x5585269e8400 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 10 Header 0x400000 0x407d21 0x40809d 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 10 Starting journal scan from offset 0x407d21 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.292+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5585269e9000 con 0x5585269e8400 2024-04-13T03:18:23.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64dc5fb640 1 -- 172.21.15.106:0/1004995635 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x558526770000 con 0x5585269e8400 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 10 Parsing data, 0x37c bytes available 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 10 Attempting decode at 0x407d21 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 10 Consumed 0x37c bytes 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 10 Valid entry at 0x407d21 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5585269e9400 0x5585267dec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5585269f6000 con 0x5585269e9400 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64dd5fd640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5585269e9400 0x5585267dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64dd5fd640 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5585269e9400 0x5585267dec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64dd5fd640 1 -- 172.21.15.106:0/1004995635 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558526770500 con 0x5585269e9400 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Reached end of journal objects 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Scanned objects, 0 missing, 1 valid 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Events scanned, 0 gaps 2024-04-13T03:18:23.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Found 1 valid events 2024-04-13T03:18:23.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 Selected 0 events events for processing 2024-04-13T03:18:23.304 INFO:teuthology.orchestra.run.smithi106.stdout:Events by type: 2024-04-13T03:18:23.304 INFO:teuthology.orchestra.run.smithi106.stdout:Errors: 0 2024-04-13T03:18:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 4 0 events 2024-04-13T03:18:23.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5585269e8400 msgr2=0x5585267de680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5585269e8400 0x5585267de680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5585269e9400 msgr2=0x5585267dec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5585269e9400 0x5585267dec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5585269e8000 msgr2=0x5585267de100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.293+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5585269e8000 0x5585267de100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55852754f350 tx=0x558527402780 comp rx=0 tx=0).stop 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 msgr2=0x5585267dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64dcdfc640 1 -- 172.21.15.106:0/1004995635 reap_dead start 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x55852754e810 tx=0x5585274024e0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 shutdown_connections 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1004995635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558527537800 0x5585267dd600 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 >> 172.21.15.106:0/1004995635 conn(0x558527537000 msgr2=0x55852675b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 shutdown_connections 2024-04-13T03:18:23.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1004995635 wait complete. 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 msgr2=0x5585267dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x5585274873b0 tx=0x558527404240 comp rx=0 tx=0).stop 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 shutdown_connections 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852666fc00 0x5585267dcb00 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852666f800 0x5585267dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 --2- 172.21.15.106:0/1417976943 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55852666ec00 0x5585267dc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 >> 172.21.15.106:0/1417976943 conn(0x55852666f000 msgr2=0x55852675a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 shutdown_connections 2024-04-13T03:18:23.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.294+0000 7f64df042280 1 -- 172.21.15.106:0/1417976943 wait complete. 2024-04-13T03:18:23.308 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.104065, result: Events by type: Errors: 0 2024-04-13T03:18:23.309 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-13T03:18:23.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 Processor -- start 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 -- start start 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5f97b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa51900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558d5f9bf050 con 0x558d5fa35870 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d5f922d00 con 0x558d5fa37830 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd706c3df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558d5f9bf1d0 con 0x7fd6ec002850 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd70444c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd70444c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52848/0 (socket says 172.21.15.106:52848) 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.390+0000 7fd70444c640 1 -- 172.21.15.106:0/3088753395 learned_addr learned my addr 172.21.15.106:0/3088753395 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 -- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 msgr2=0x558d5f97b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5f97b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 -- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 msgr2=0x558d5fa51900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa51900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 -- 172.21.15.106:0/3088753395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5fa37c10 con 0x558d5fa35870 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd70444c640 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0096e0 tx=0x7fd6fc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2212f3880c19a390 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd702448640 1 -- 172.21.15.106:0/3088753395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc01b070 con 0x558d5fa35870 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd702448640 1 -- 172.21.15.106:0/3088753395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc009e50 con 0x558d5fa35870 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd702448640 1 -- 172.21.15.106:0/3088753395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc007a30 con 0x558d5fa35870 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 -- 172.21.15.106:0/3088753395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 msgr2=0x558d5fa35c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0096e0 tx=0x7fd6fc005a00 comp rx=0 tx=0).stop 2024-04-13T03:18:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 -- 172.21.15.106:0/3088753395 shutdown_connections 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5fa35c50 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa51900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 --2- 172.21.15.106:0/3088753395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5f97b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 -- 172.21.15.106:0/3088753395 >> 172.21.15.106:0/3088753395 conn(0x558d5f9a5ae0 msgr2=0x558d5fa52fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 -- 172.21.15.106:0/3088753395 shutdown_connections 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.391+0000 7fd706c3df40 1 -- 172.21.15.106:0/3088753395 wait complete. 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.392+0000 7fd706c3df40 1 Processor -- start 2024-04-13T03:18:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.392+0000 7fd706c3df40 1 -- start start 2024-04-13T03:18:23.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5faeae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa50680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5fa34630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558d5fa373d0 con 0x558d5fa35870 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d5fa52120 con 0x558d5fa37830 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x558d5f9a8d70 con 0x7fd6ec002850 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5faeae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5faeae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52854/0 (socket says 172.21.15.106:52854) 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 -- 172.21.15.106:0/1880976419 learned_addr learned my addr 172.21.15.106:0/1880976419 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd70344a640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd70444c640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5fa34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 -- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 msgr2=0x558d5fa34630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd6ec002850 0x558d5fa34630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 -- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 msgr2=0x558d5fa50680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa50680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 -- 172.21.15.106:0/1880976419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5fa34b70 con 0x558d5fa35870 2024-04-13T03:18:23.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd70344a640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d5fa37830 0x558d5fa50680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:18:23.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd703c4b640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5faeae40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40036b0 tx=0x7fd6f4003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd701446640 1 -- 172.21.15.106:0/1880976419 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4010580 con 0x558d5fa35870 2024-04-13T03:18:23.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd701446640 1 -- 172.21.15.106:0/1880976419 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4017070 con 0x558d5fa35870 2024-04-13T03:18:23.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d5f9a49b0 con 0x558d5fa35870 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd701446640 1 -- 172.21.15.106:0/1880976419 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4010940 con 0x558d5fa35870 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.393+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5faea4f0 con 0x558d5fa35870 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd701446640 1 -- 172.21.15.106:0/1880976419 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6f4010ae0 con 0x558d5fa35870 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd701446640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd6f006b2d0 0x7fd6f006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd70344a640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd6f006b2d0 0x7fd6f006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd701446640 1 -- 172.21.15.106:0/1880976419 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fd6f408aba0 con 0x558d5fa35870 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd706c3df40 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x558d5fa35c50 0x558d5f97ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.395+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x558d5fa51900 con 0x558d5fa35c50 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.396+0000 7fd70444c640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x558d5fa35c50 0x558d5f97ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.396+0000 7fd70344a640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd6f006b2d0 0x7fd6f006d790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009810 tx=0x7fd6f8002fb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.396+0000 7fd70444c640 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x558d5fa35c50 0x558d5f97ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd70444c640 1 -- 172.21.15.106:0/1880976419 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd6fc007a50 con 0x558d5fa35c50 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x558d5fa35c50 msgr2=0x558d5f97ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x558d5fa35c50 0x558d5f97ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd6f006b2d0 msgr2=0x7fd6f006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd6f006b2d0 0x7fd6f006d790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009810 tx=0x7fd6f8002fb0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 msgr2=0x558d5faeae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 --2- 172.21.15.106:0/1880976419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5fa35870 0x558d5faeae40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40036b0 tx=0x7fd6f4003ab0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd70444c640 1 -- 172.21.15.106:0/1880976419 reap_dead start 2024-04-13T03:18:23.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 shutdown_connections 2024-04-13T03:18:23.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 >> 172.21.15.106:0/1880976419 conn(0x558d5f9a5ae0 msgr2=0x558d5fa52fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 shutdown_connections 2024-04-13T03:18:23.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.397+0000 7fd706c3df40 1 -- 172.21.15.106:0/1880976419 wait complete. 2024-04-13T03:18:23.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:18:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.546+0000 7f66e06de640 1 Processor -- start 2024-04-13T03:18:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.546+0000 7f66e06de640 1 -- start start 2024-04-13T03:18:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66e06de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f66d80b1c30 0x7f66d815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66e06de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66d80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66e06de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d80b5cf0 con 0x7f66d80709e0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66e06de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f66d80b5fd0 con 0x7f66d80b1c30 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66e06de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66d80b62b0 con 0x7f66d80b54a0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dec7b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f66d80709e0 0x7f66d8162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59710/0 (socket says 172.21.15.106:59710) 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dec7b640 1 -- 172.21.15.106:0/3801680211 learned_addr learned my addr 172.21.15.106:0/3801680211 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66de47a640 1 --2- 172.21.15.106:0/3801680211 >> v2:172.21.15.114:3300/0 conn(0x7f66d80b1c30 0x7f66d815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786345151 0 0) 0x7f66d80b5cf0 con 0x7f66d80709e0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66d80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66c4003520 con 0x7f66d80709e0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2136619910 0 0) 0x7f66c4003520 con 0x7f66d80709e0 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 msgr2=0x7f66d80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66d80704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 >> v2:172.21.15.114:3300/0 conn(0x7f66d80b1c30 msgr2=0x7f66d815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 --2- 172.21.15.106:0/3801680211 >> v2:172.21.15.114:3300/0 conn(0x7f66d80b1c30 0x7f66d815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d80b6590 con 0x7f66d80709e0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f66d00019f0 con 0x7f66d80709e0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66de47a640 1 --2- 172.21.15.106:0/3801680211 >> v2:172.21.15.114:3300/0 conn(0x7f66d80b1c30 0x7f66d815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.547+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 >> v1:172.21.15.106:6789/0 conn(0x7f66d80709e0 legacy=0x7f66d8162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66c4003f30 0x7f66c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4004d40 0x7f66c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80b5fd0 con 0x7f66c4003f30 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80b62b0 con 0x7f66c4004d40 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66c400b9c0 con 0x7f66d80b54a0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66c4003f30 0x7f66c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66de47a640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66dec7b640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4004d40 0x7f66c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 -- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 msgr2=0x7f66c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80b54a0 0x7f66c40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 -- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4004d40 msgr2=0x7f66c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4004d40 0x7f66c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.548+0000 7f66ddc79640 1 -- 172.21.15.106:0/3801680211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c4003520 con 0x7f66c4003f30 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66c4003f30 0x7f66c4004800 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009ac0 tx=0x7f66c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30eb31dbfe80b312 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8039070 con 0x7f66c4003f30 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c802c6b0 con 0x7f66c4003f30 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66dd478640 1 -- 172.21.15.106:0/3801680211 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8009690 con 0x7f66c4003f30 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 -- 172.21.15.106:0/3801680211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66c4003f30 msgr2=0x7f66c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 --2- 172.21.15.106:0/3801680211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66c4003f30 0x7f66c4004800 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009ac0 tx=0x7f66c4005830 comp rx=0 tx=0).stop 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66dec7b640 1 -- 172.21.15.106:0/3801680211 reap_dead start 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 -- 172.21.15.106:0/3801680211 shutdown_connections 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 -- 172.21.15.106:0/3801680211 >> 172.21.15.106:0/3801680211 conn(0x7f66d8071870 msgr2=0x7f66d8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 -- 172.21.15.106:0/3801680211 shutdown_connections 2024-04-13T03:18:23.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.549+0000 7f66e06de640 1 -- 172.21.15.106:0/3801680211 wait complete. 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.550+0000 7f66e06de640 1 Processor -- start 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 -- start start 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4003f30 0x7f66d8161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80d2030 0x7f66d815fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66d806e7f0 con 0x7f66d81f80d0 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66d806e670 con 0x7f66c4003f30 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66e06de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f66d806e970 con 0x7f66d80d2030 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66ddc79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66ddc79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52878/0 (socket says 172.21.15.106:52878) 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.551+0000 7f66ddc79640 1 -- 172.21.15.106:0/3586929756 learned_addr learned my addr 172.21.15.106:0/3586929756 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 -- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80d2030 msgr2=0x7f66d815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80d2030 0x7f66d815fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 -- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4003f30 msgr2=0x7f66d8161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4003f30 0x7f66d8161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 -- 172.21.15.106:0/3586929756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81408c0 con 0x7f66d81f80d0 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ddc79640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000c00 tx=0x7f66c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8042020 con 0x7f66d81f80d0 2024-04-13T03:18:23.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c803a040 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d8162360 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8039070 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.552+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d8162920 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.553+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d8068a10 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.553+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66c8045020 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.554+0000 7f66ceffd640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.554+0000 7f66de47a640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.554+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f66c80ad070 con 0x7f66d81f80d0 2024-04-13T03:18:23.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.554+0000 7f66de47a640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 0x7f66bc06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009cc0 tx=0x7f66d4009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:23.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.558+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66c804c070 con 0x7f66d81f80d0 2024-04-13T03:18:23.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:23.770+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f66d80711f0 con 0x7f66d81f80d0 2024-04-13T03:18:24.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.142+0000 7f66ceffd640 1 -- 172.21.15.106:0/3586929756 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v73) ==== 162+0+0 (secure 0 0 0) 0x7f66c807b330 con 0x7f66d81f80d0 2024-04-13T03:18:24.143 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 msgr2=0x7f66bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 0x7f66bc06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009cc0 tx=0x7f66d4009780 comp rx=0 tx=0).stop 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 msgr2=0x7f66d80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000c00 tx=0x7f66c8004730 comp rx=0 tx=0).stop 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 shutdown_connections 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f66d80d2030 0x7f66d815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66d81f80d0 0x7f66d80d1af0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 --2- 172.21.15.106:0/3586929756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66c4003f30 0x7f66d8161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 >> 172.21.15.106:0/3586929756 conn(0x7f66d8071870 msgr2=0x7f66d815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:24.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 shutdown_connections 2024-04-13T03:18:24.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.144+0000 7f66e06de640 1 -- 172.21.15.106:0/3586929756 wait complete. 2024-04-13T03:18:24.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:18:24.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.256+0000 7fb403b8c640 1 Processor -- start 2024-04-13T03:18:24.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 -- start start 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 0x7fb3fc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3fc15ed90 0x7fb3fc0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3fc0cfd20 con 0x7fb3fc0b3d60 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb3fc162af0 con 0x7fb3fc15ed90 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb403b8c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb3fc162dd0 con 0x7fb3fc15e9b0 2024-04-13T03:18:24.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb401127640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3fc15ed90 0x7fb3fc0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb402129640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb3fc0b3d60 0x7fb3fc0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59716/0 (socket says 172.21.15.106:59716) 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.257+0000 7fb402129640 1 -- 172.21.15.106:0/1448384141 learned_addr learned my addr 172.21.15.106:0/1448384141 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401928640 1 --2- 172.21.15.106:0/1448384141 >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 0x7fb3fc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401127640 1 -- 172.21.15.106:0/1448384141 >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 msgr2=0x7fb3fc0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401127640 1 --2- 172.21.15.106:0/1448384141 >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 0x7fb3fc0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401127640 1 -- 172.21.15.106:0/1448384141 >> v1:172.21.15.106:6789/0 conn(0x7fb3fc0b3d60 legacy=0x7fb3fc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401127640 1 -- 172.21.15.106:0/1448384141 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc1630b0 con 0x7fb3fc15ed90 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb401928640 1 --2- 172.21.15.106:0/1448384141 >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 0x7fb3fc0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.258+0000 7fb400926640 1 -- 172.21.15.106:0/1448384141 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1722205496 0 0) 0x7fb3fc0cfd20 con 0x7fb3fc0b3d60 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb401127640 1 --2- 172.21.15.106:0/1448384141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc15ed90 0x7fb3fc0d45f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc08e4e0 tx=0x7fb3ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73396e17b1f197ba server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb400926640 1 -- 172.21.15.106:0/1448384141 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec034070 con 0x7fb3fc15ed90 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb400926640 1 -- 172.21.15.106:0/1448384141 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec030610 con 0x7fb3fc15ed90 2024-04-13T03:18:24.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb400926640 1 -- 172.21.15.106:0/1448384141 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0309e0 con 0x7fb3fc15ed90 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 -- 172.21.15.106:0/1448384141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc15ed90 msgr2=0x7fb3fc0d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 --2- 172.21.15.106:0/1448384141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc15ed90 0x7fb3fc0d45f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc08e4e0 tx=0x7fb3ec02b960 comp rx=0 tx=0).stop 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 -- 172.21.15.106:0/1448384141 shutdown_connections 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 --2- 172.21.15.106:0/1448384141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc15ed90 0x7fb3fc0d45f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 --2- 172.21.15.106:0/1448384141 >> v2:172.21.15.114:3300/0 conn(0x7fb3fc15e9b0 0x7fb3fc0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 -- 172.21.15.106:0/1448384141 >> 172.21.15.106:0/1448384141 conn(0x7fb3fc08bf10 msgr2=0x7fb3fc0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 -- 172.21.15.106:0/1448384141 shutdown_connections 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.259+0000 7fb403b8c640 1 -- 172.21.15.106:0/1448384141 wait complete. 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.260+0000 7fb403b8c640 1 Processor -- start 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.260+0000 7fb403b8c640 1 -- start start 2024-04-13T03:18:24.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3fc0b3d60 0x7fb3fc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3fc15e9b0 0x7fb3fc15fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc162800 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc162680 con 0x7fb3fc15e9b0 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb403b8c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc06eb20 con 0x7fb3fc0b3d60 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb402129640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb402129640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52888/0 (socket says 172.21.15.106:52888) 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.261+0000 7fb402129640 1 -- 172.21.15.106:0/549037419 learned_addr learned my addr 172.21.15.106:0/549037419 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb401127640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3fc15e9b0 0x7fb3fc15fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb401928640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3fc0b3d60 0x7fb3fc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 -- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3fc0b3d60 msgr2=0x7fb3fc1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3fc0b3d60 0x7fb3fc1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 -- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3fc15e9b0 msgr2=0x7fb3fc15fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3fc15e9b0 0x7fb3fc15fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 -- 172.21.15.106:0/549037419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc1615a0 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb402129640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c480 tx=0x7fb3f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4018070 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4007500 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc160500 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004a20 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.262+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc160ac0 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.263+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3fc068a10 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.263+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3f401b070 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.264+0000 7fb3f27fc640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.264+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fb3f408afb0 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.264+0000 7fb401928640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.264+0000 7fb401928640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 0x7fb3e006edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80075b0 tx=0x7fb3f8007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:24.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.268+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3f4059270 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.478+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3fc0d3370 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.478+0000 7fb3f27fc640 1 -- 172.21.15.106:0/549037419 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5190 (secure 0 0 0) 0x7fb3f4058b70 con 0x7fb3fc0b67a0 2024-04-13T03:18:24.479 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:24.479 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:24.145154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"d","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:18:24.480 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 74 2024-04-13T03:18:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 msgr2=0x7fb3e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 0x7fb3e006edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80075b0 tx=0x7fb3f8007540 comp rx=0 tx=0).stop 2024-04-13T03:18:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 msgr2=0x7fb3fc15cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c480 tx=0x7fb3f400c940 comp rx=0 tx=0).stop 2024-04-13T03:18:24.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 shutdown_connections 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3fc0b67a0 0x7fb3fc15cb10 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3fc15e9b0 0x7fb3fc15fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 --2- 172.21.15.106:0/549037419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb3fc0b3d60 0x7fb3fc1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 >> 172.21.15.106:0/549037419 conn(0x7fb3fc08bf10 msgr2=0x7fb3fc0d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 shutdown_connections 2024-04-13T03:18:24.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:24.480+0000 7fb403b8c640 1 -- 172.21.15.106:0/549037419 wait complete. 2024-04-13T03:18:24.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:17:33.740516+0000', 'modified': '2024-04-13T03:18:24.145154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'd', 'rank': 0, 'incarnation': 74, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.114:6833/3141313499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 3141313499}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 3141313499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:18:24.895 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8427:up:replay 2024-04-13T03:18:25.896 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:18:26.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 Processor -- start 2024-04-13T03:18:26.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 -- start start 2024-04-13T03:18:26.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f211c15c940 0x7f211c06e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f211c06d0e0 0x7f211c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211c15cd20 con 0x7f211c06e780 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f211c06ebe0 con 0x7f211c06d0e0 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.006+0000 7f212195e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f211c0c6330 con 0x7f211c15c940 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f211c06d0e0 0x7f211c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f211c06e780 0x7f211c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59730/0 (socket says 172.21.15.106:59730) 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211a7fc640 1 -- 172.21.15.106:0/2598632487 learned_addr learned my addr 172.21.15.106:0/2598632487 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211b7fe640 1 -- 172.21.15.106:0/2598632487 >> v2:172.21.15.114:3300/0 conn(0x7f211c15c940 msgr2=0x7f211c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211b7fe640 1 --2- 172.21.15.106:0/2598632487 >> v2:172.21.15.114:3300/0 conn(0x7f211c15c940 0x7f211c06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211b7fe640 1 -- 172.21.15.106:0/2598632487 >> v1:172.21.15.106:6789/0 conn(0x7f211c06e780 legacy=0x7f211c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f211b7fe640 1 -- 172.21.15.106:0/2598632487 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0cc700 con 0x7f211c06d0e0 2024-04-13T03:18:26.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.007+0000 7f2119ffb640 1 -- 172.21.15.106:0/2598632487 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361750781 0 0) 0x7f211c15cd20 con 0x7f211c06e780 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f211b7fe640 1 --2- 172.21.15.106:0/2598632487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c06d0e0 0x7f211c0c6aa0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f211c0b4110 tx=0x7f210c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ba69e98a410d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f2119ffb640 1 -- 172.21.15.106:0/2598632487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c035020 con 0x7f211c06d0e0 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f2119ffb640 1 -- 172.21.15.106:0/2598632487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c031830 con 0x7f211c06d0e0 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f2119ffb640 1 -- 172.21.15.106:0/2598632487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c031c00 con 0x7f211c06d0e0 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 -- 172.21.15.106:0/2598632487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c06d0e0 msgr2=0x7f211c0c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 --2- 172.21.15.106:0/2598632487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c06d0e0 0x7f211c0c6aa0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f211c0b4110 tx=0x7f210c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 -- 172.21.15.106:0/2598632487 shutdown_connections 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 --2- 172.21.15.106:0/2598632487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c06d0e0 0x7f211c0c6aa0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 --2- 172.21.15.106:0/2598632487 >> v2:172.21.15.114:3300/0 conn(0x7f211c15c940 0x7f211c06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 -- 172.21.15.106:0/2598632487 >> 172.21.15.106:0/2598632487 conn(0x7f211c0cdef0 msgr2=0x7f211c0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:26.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 -- 172.21.15.106:0/2598632487 shutdown_connections 2024-04-13T03:18:26.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.008+0000 7f212195e640 1 -- 172.21.15.106:0/2598632487 wait complete. 2024-04-13T03:18:26.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.009+0000 7f212195e640 1 Processor -- start 2024-04-13T03:18:26.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 -- start start 2024-04-13T03:18:26.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c06d0e0 0x7f211c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c06e780 0x7f211c0b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f211c06dc70 con 0x7f211c15c940 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f211c06d990 con 0x7f211c06d0e0 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.010+0000 7f212195e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f211c0c6580 con 0x7f211c06e780 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52916/0 (socket says 172.21.15.106:52916) 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 -- 172.21.15.106:0/203806615 learned_addr learned my addr 172.21.15.106:0/203806615 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211affd640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c06d0e0 0x7f211c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211a7fc640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c06e780 0x7f211c0b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 -- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c06e780 msgr2=0x7f211c0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c06e780 0x7f211c0b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 -- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c06d0e0 msgr2=0x7f211c0c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c06d0e0 0x7f211c0c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 -- 172.21.15.106:0/203806615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0b6f70 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f211b7fe640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f210c02bee0 tx=0x7f210c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c004230 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c0043d0 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c03c990 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c0ca580 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.011+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c0b0030 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.012+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c069120 con 0x7f211c15c940 2024-04-13T03:18:26.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.013+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f210c038020 con 0x7f211c15c940 2024-04-13T03:18:26.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.013+0000 7f2103fff640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:26.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.013+0000 7f211affd640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:26.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.013+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f210c0ae1d0 con 0x7f211c15c940 2024-04-13T03:18:26.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.014+0000 7f211affd640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f211c1f7e70 tx=0x7f21140094f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:26.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.018+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f210c041070 con 0x7f211c15c940 2024-04-13T03:18:26.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.233+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f211c0b2ea0 con 0x7f211c15c940 2024-04-13T03:18:26.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.233+0000 7f2103fff640 1 -- 172.21.15.106:0/203806615 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5191 (secure 0 0 0) 0x7f211c0cc700 con 0x7f211c15c940 2024-04-13T03:18:26.235 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:26.235 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:25.159307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"d","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:18:26.236 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 76 2024-04-13T03:18:26.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 msgr2=0x7f210806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f211c1f7e70 tx=0x7f21140094f0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 msgr2=0x7f211c0c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f210c02bee0 tx=0x7f210c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 shutdown_connections 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f211c06e780 0x7f211c0b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f211c15c940 0x7f211c0c76e0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 --2- 172.21.15.106:0/203806615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f211c06d0e0 0x7f211c0c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 >> 172.21.15.106:0/203806615 conn(0x7f211c0cdef0 msgr2=0x7f211c157130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 shutdown_connections 2024-04-13T03:18:26.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:26.235+0000 7f212195e640 1 -- 172.21.15.106:0/203806615 wait complete. 2024-04-13T03:18:26.370 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:18:26.373 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:18:26.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:17:33.740516+0000', 'modified': '2024-04-13T03:18:25.159307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'd', 'rank': 0, 'incarnation': 74, 'state': 'up:rejoin', 'state_seq': 16, 'addr': '172.21.15.114:6833/3141313499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 3141313499}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 3141313499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:18:26.651 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8427:up:rejoin 2024-04-13T03:18:27.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:18:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.763+0000 7fc725184640 1 Processor -- start 2024-04-13T03:18:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.763+0000 7fc725184640 1 -- start start 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc725184640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7200b53a0 0x7fc7200b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc725184640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7200cdf50 0x7fc72015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc725184640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7200b5cc0 con 0x7fc72006ef90 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc725184640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc72006f540 con 0x7fc7200b53a0 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc725184640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc72006f820 con 0x7fc7200cdf50 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7200b53a0 0x7fc7200b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc7200b53a0 0x7fc7200b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52924/0 (socket says 172.21.15.106:52924) 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 -- 172.21.15.106:0/73357428 learned_addr learned my addr 172.21.15.106:0/73357428 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 -- 172.21.15.106:0/73357428 >> v2:172.21.15.114:3300/0 conn(0x7fc7200cdf50 msgr2=0x7fc72015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 --2- 172.21.15.106:0/73357428 >> v2:172.21.15.114:3300/0 conn(0x7fc7200cdf50 0x7fc72015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 -- 172.21.15.106:0/73357428 >> v1:172.21.15.114:6789/0 conn(0x7fc72006ef90 legacy=0x7fc72015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.764+0000 7fc71ed9d640 1 -- 172.21.15.106:0/73357428 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72015b440 con 0x7fc7200b53a0 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc71ed9d640 1 --2- 172.21.15.106:0/73357428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200b53a0 0x7fc7200b5780 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc7200d1410 tx=0x7fc71002c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72ca417cae3869d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc71dd9b640 1 -- 172.21.15.106:0/73357428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710035070 con 0x7fc7200b53a0 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc71dd9b640 1 -- 172.21.15.106:0/73357428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc710031850 con 0x7fc7200b53a0 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc71dd9b640 1 -- 172.21.15.106:0/73357428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710031c20 con 0x7fc7200b53a0 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc725184640 1 -- 172.21.15.106:0/73357428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200b53a0 msgr2=0x7fc7200b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.765+0000 7fc725184640 1 --2- 172.21.15.106:0/73357428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200b53a0 0x7fc7200b5780 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc7200d1410 tx=0x7fc71002c990 comp rx=0 tx=0).stop 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 -- 172.21.15.106:0/73357428 shutdown_connections 2024-04-13T03:18:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 --2- 172.21.15.106:0/73357428 >> v2:172.21.15.114:3300/0 conn(0x7fc7200cdf50 0x7fc72015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 --2- 172.21.15.106:0/73357428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200b53a0 0x7fc7200b5780 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 -- 172.21.15.106:0/73357428 >> 172.21.15.106:0/73357428 conn(0x7fc7200b1f10 msgr2=0x7fc7200b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 -- 172.21.15.106:0/73357428 shutdown_connections 2024-04-13T03:18:27.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.766+0000 7fc725184640 1 -- 172.21.15.106:0/73357428 wait complete. 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.767+0000 7fc725184640 1 Processor -- start 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.767+0000 7fc725184640 1 -- start start 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc72006ef90 0x7fc7201583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7200b53a0 0x7fc7200ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc72015c180 con 0x7fc7200cdf50 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc72006fb00 con 0x7fc72006ef90 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc725184640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc720158bb0 con 0x7fc7200b53a0 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52938/0 (socket says 172.21.15.106:52938) 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 -- 172.21.15.106:0/3190531534 learned_addr learned my addr 172.21.15.106:0/3190531534 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71ed9d640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc72006ef90 0x7fc7201583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71e59c640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7200b53a0 0x7fc7200ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 -- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7200b53a0 msgr2=0x7fc7200ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7200b53a0 0x7fc7200ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 -- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc72006ef90 msgr2=0x7fc7201583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc72006ef90 0x7fc7201583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.768+0000 7fc71f59e640 1 -- 172.21.15.106:0/3190531534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7200c94c0 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc71f59e640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b990 tx=0x7fc71800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718019070 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71800caa0 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718014440 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7200c97a0 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.769+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7200ad6b0 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.770+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc720068a10 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.770+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71801c070 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.770+0000 7fc7077fe640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.771+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fc71808b9a0 con 0x7fc7200cdf50 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.771+0000 7fc71ed9d640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.771+0000 7fc71ed9d640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc710037000 tx=0x7fc71003e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:27.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.775+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc718059590 con 0x7fc7200cdf50 2024-04-13T03:18:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.986+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc72015a3d0 con 0x7fc7200cdf50 2024-04-13T03:18:27.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.986+0000 7fc7077fe640 1 -- 172.21.15.106:0/3190531534 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5198 (secure 0 0 0) 0x7fc71805d890 con 0x7fc7200cdf50 2024-04-13T03:18:27.988 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:27.988 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:27.162282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"d","rank":0,"incarnation":74,"state":"up:active","state_seq":17,"addr":"172.21.15.114:6833/3141313499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3141313499},{"type":"v1","addr":"172.21.15.114:6833","nonce":3141313499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":7}]} 2024-04-13T03:18:27.989 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 77 2024-04-13T03:18:27.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 msgr2=0x7fc70c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:27.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc710037000 tx=0x7fc71003e040 comp rx=0 tx=0).stop 2024-04-13T03:18:27.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 msgr2=0x7fc7200cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:27.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b990 tx=0x7fc71800bd90 comp rx=0 tx=0).stop 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 shutdown_connections 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc70c06c8f0 0x7fc70c06edb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc7200b53a0 0x7fc7200ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7200cdf50 0x7fc7200cb760 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.988+0000 7fc725184640 1 --2- 172.21.15.106:0/3190531534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc72006ef90 0x7fc7201583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.989+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 >> 172.21.15.106:0/3190531534 conn(0x7fc7200b1f10 msgr2=0x7fc72006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.989+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 shutdown_connections 2024-04-13T03:18:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:27.989+0000 7fc725184640 1 -- 172.21.15.106:0/3190531534 wait complete. 2024-04-13T03:18:28.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:17:33.740516+0000', 'modified': '2024-04-13T03:18:27.162282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'd', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.114:6833/3141313499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 3141313499}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 3141313499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8427, 'qdb_cluster': [8427]} 2024-04-13T03:18:28.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:18:28.404 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-13T03:18:28.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 Processor -- start 2024-04-13T03:18:28.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 -- start start 2024-04-13T03:18:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111815ddd0 con 0x7f11180704a0 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f111815e100 con 0x7f11180d1c30 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.507+0000 7f111e4b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f111815d4d0 con 0x7f11180cfea0 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f111ca54640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f11177fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f11180704a0 0x7f11180d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:33376/0 (socket says 172.21.15.106:33376) 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f111ca54640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:49540/0 (socket says 172.21.15.106:49540) 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:53890/0 (socket says 172.21.15.106:53890) 2024-04-13T03:18:28.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f11177fe640 1 -- 172.21.15.106:0/1115950359 learned_addr learned my addr 172.21.15.106:0/1115950359 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 -- 172.21.15.106:0/1115950359 >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 msgr2=0x7f111815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 --2- 172.21.15.106:0/1115950359 >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 -- 172.21.15.106:0/1115950359 >> v1:172.21.15.114:6789/0 conn(0x7f11180704a0 legacy=0x7f11180d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f1117fff640 1 -- 172.21.15.106:0/1115950359 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111815d7b0 con 0x7f11180cfea0 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.508+0000 7f111ca54640 1 --2- 172.21.15.106:0/1115950359 >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.509+0000 7f1117fff640 1 --2- 172.21.15.106:0/1115950359 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11080097a0 tx=0x7f110802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd0f816015e5e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.509+0000 7f1116ffd640 1 -- 172.21.15.106:0/1115950359 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080315a0 con 0x7f11180cfea0 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.509+0000 7f1116ffd640 1 -- 172.21.15.106:0/1115950359 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108037070 con 0x7f11180cfea0 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.509+0000 7f1116ffd640 1 -- 172.21.15.106:0/1115950359 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108031960 con 0x7f11180cfea0 2024-04-13T03:18:28.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 -- 172.21.15.106:0/1115950359 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 msgr2=0x7f111815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 --2- 172.21.15.106:0/1115950359 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11080097a0 tx=0x7f110802b840 comp rx=0 tx=0).stop 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 -- 172.21.15.106:0/1115950359 shutdown_connections 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 --2- 172.21.15.106:0/1115950359 >> v2:172.21.15.114:3300/0 conn(0x7f11180d1c30 0x7f111815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 --2- 172.21.15.106:0/1115950359 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180cfea0 0x7f111815cc10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 -- 172.21.15.106:0/1115950359 >> 172.21.15.106:0/1115950359 conn(0x7f1118071870 msgr2=0x7f1118071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 -- 172.21.15.106:0/1115950359 shutdown_connections 2024-04-13T03:18:28.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.510+0000 7f111e4b7640 1 -- 172.21.15.106:0/1115950359 wait complete. 2024-04-13T03:18:28.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.511+0000 7f111e4b7640 1 Processor -- start 2024-04-13T03:18:28.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.511+0000 7f111e4b7640 1 -- start start 2024-04-13T03:18:28.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180704a0 0x7f111813d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180d1c30 0x7f111815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f111815ead0 con 0x7f11180cfea0 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f111815d150 con 0x7f11180704a0 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111e4b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f111815ec50 con 0x7f11180d1c30 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52942/0 (socket says 172.21.15.106:52942) 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 -- 172.21.15.106:0/3276793983 learned_addr learned my addr 172.21.15.106:0/3276793983 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f1117fff640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180704a0 0x7f111813d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f111ca54640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180d1c30 0x7f111815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 -- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180d1c30 msgr2=0x7f111815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180d1c30 0x7f111815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 -- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180704a0 msgr2=0x7f111813d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180704a0 0x7f111813d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 -- 172.21.15.106:0/3276793983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11181e6430 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f11177fe640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c400 tx=0x7f110c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.512+0000 7f1114ff9640 1 -- 172.21.15.106:0/3276793983 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c017070 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.513+0000 7f1114ff9640 1 -- 172.21.15.106:0/3276793983 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c005ce0 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.513+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11181e6710 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.513+0000 7f1114ff9640 1 -- 172.21.15.106:0/3276793983 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0072f0 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.513+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181e6cd0 con 0x7f11180cfea0 2024-04-13T03:18:28.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.514+0000 7f1114ff9640 1 -- 172.21.15.106:0/3276793983 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f110c01a070 con 0x7f11180cfea0 2024-04-13T03:18:28.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.514+0000 7f1114ff9640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.514+0000 7f1114ff9640 1 -- 172.21.15.106:0/3276793983 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f110c08b0d0 con 0x7f11180cfea0 2024-04-13T03:18:28.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.514+0000 7f1117fff640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.515+0000 7f1117fff640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1108036000 tx=0x7f1108008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 Processor -- start 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 -- start start 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dace6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 0x55b1dace70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b1dac21990 con 0x55b1dace3de0 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1dabc8620 con 0x55b1dacb72c0 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b1dac1d950 con 0x55b1dace41c0 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f1117fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f1117fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52954/0 (socket says 172.21.15.106:52954) 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.516+0000 7f1117fff640 1 -- 172.21.15.106:0/3429975024 learned_addr learned my addr 172.21.15.106:0/3429975024 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f11177fe640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dace6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f111ca54640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 0x55b1dace70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 -- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 msgr2=0x55b1dace6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dace6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 -- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 msgr2=0x55b1dace70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 0x55b1dace70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 -- 172.21.15.106:0/3429975024 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1da9342d0 con 0x55b1dace3de0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f11177fe640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dace6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f111ca54640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 0x55b1dace70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f1117fff640 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f110802be50 tx=0x7f1108030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaa5e09cd9b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f10fdffb640 1 -- 172.21.15.106:0/3429975024 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108031ed0 con 0x55b1dace3de0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f10fdffb640 1 -- 172.21.15.106:0/3429975024 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108037040 con 0x55b1dace3de0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f10fdffb640 1 -- 172.21.15.106:0/3429975024 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108050830 con 0x55b1dace3de0 2024-04-13T03:18:28.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f112137c740 1 -- 172.21.15.106:0/3429975024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 msgr2=0x55b1dacb6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.517+0000 7f112137c740 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f110802be50 tx=0x7f1108030000 comp rx=0 tx=0).stop 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 -- 172.21.15.106:0/3429975024 shutdown_connections 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dace6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dacb72c0 0x55b1dace70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 --2- 172.21.15.106:0/3429975024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace3de0 0x55b1dacb6d80 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 -- 172.21.15.106:0/3429975024 >> 172.21.15.106:0/3429975024 conn(0x55b1dacb8ff0 msgr2=0x55b1dad01b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 -- 172.21.15.106:0/3429975024 shutdown_connections 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 -- 172.21.15.106:0/3429975024 wait complete. 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 -- start start 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.518+0000 7f112137c740 1 Processor -- start 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacfa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b1dacbb590 con 0x55b1dace41c0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1dabdb9e0 con 0x55b1dace3de0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b1dacba980 con 0x55b1dacf9a90 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f112137c740 10 client.0 _ll_register_callbacks cb 0x55b1dac17220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52970/0 (socket says 172.21.15.106:52970) 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 -- 172.21.15.106:0/1329778102 learned_addr learned my addr 172.21.15.106:0/1329778102 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f1117fff640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f111ca54640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacfa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 -- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 msgr2=0x55b1dacfa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacfa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 -- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 msgr2=0x55b1dacea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.519+0000 7f11177fe640 1 -- 172.21.15.106:0/1329778102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1da9fe6b0 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f11177fe640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f110c001eb0 tx=0x7f110c058d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228ff1e0a6627325 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c071170 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f110c007490 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1108037040 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f110c089050 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 10 client.8844 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b1dace8b30 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108037040 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f110c0580e0 con 0x55b1dace41c0 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 10 client.8844 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 10 client.8844 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 20 client.8844 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacf9040 0x55b1dad121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 4 client.8844 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.520+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1dac2f3b0 con 0x55b1dacf9040 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.521+0000 7f1117fff640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacf9040 0x55b1dad121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.521+0000 7f10f6ffd640 10 client.8844 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.521+0000 7f1117fff640 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacf9040 0x55b1dad121a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e29498238d733 server_cookie=ff920cac9cb8094c in_seq=0 out_seq=0 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f10f6ffd640 1 -- 172.21.15.106:0/1329778102 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1dac2f3b0 con 0x55b1dacf9040 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f10f6ffd640 10 client.8844 handle_command_reply: tid=0 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 client.8844 shutdown 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 msgr2=0x55b1dace1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f110c001eb0 tx=0x7f110c058d50 comp rx=0 tx=0).stop 2024-04-13T03:18:28.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 shutdown_connections 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacf9040 0x55b1dad121a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacfa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dace1bf0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 --2- 172.21.15.106:0/1329778102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f10f6ffd640 0 client.8844 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 >> 172.21.15.106:0/1329778102 conn(0x55b1dacb72c0 msgr2=0x55b1dad03fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 shutdown_connections 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 1 -- 172.21.15.106:0/1329778102 wait complete. 2024-04-13T03:18:28.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.522+0000 7f112137c740 20 client.8844 trim_cache size 0 max 16384 2024-04-13T03:18:28.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.540+0000 7f112137c740 1 Processor -- start 2024-04-13T03:18:28.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.540+0000 7f112137c740 1 -- start start 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacfa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11181fe630 con 0x55b1dace41c0 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1118151d70 con 0x55b1dace3de0 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f11181523c0 con 0x55b1dacf9a90 2024-04-13T03:18:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52976/0 (socket says 172.21.15.106:52976) 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 -- 172.21.15.106:0/2024634258 learned_addr learned my addr 172.21.15.106:0/2024634258 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 -- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 msgr2=0x55b1dacba980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 -- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 msgr2=0x55b1dacfa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacfa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 -- 172.21.15.106:0/2024634258 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1dacf9760 con 0x55b1dace41c0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f11177fe640 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f110c000fb0 tx=0x7f110c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1180f9f98f33423a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f10fd7fa640 1 -- 172.21.15.106:0/2024634258 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0580e0 con 0x55b1dace41c0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f10fd7fa640 1 -- 172.21.15.106:0/2024634258 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c007490 con 0x55b1dace41c0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.541+0000 7f10fd7fa640 1 -- 172.21.15.106:0/2024634258 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c05f240 con 0x55b1dace41c0 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- 172.21.15.106:0/2024634258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 msgr2=0x55b1dad021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f110c000fb0 tx=0x7f110c01e070 comp rx=0 tx=0).stop 2024-04-13T03:18:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- 172.21.15.106:0/2024634258 shutdown_connections 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dace41c0 0x55b1dad021f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 --2- 172.21.15.106:0/2024634258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dacfa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- 172.21.15.106:0/2024634258 >> 172.21.15.106:0/2024634258 conn(0x55b1dacb72c0 msgr2=0x55b1dad126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- 172.21.15.106:0/2024634258 shutdown_connections 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- 172.21.15.106:0/2024634258 wait complete. 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.542+0000 7f112137c740 1 -- start start 2024-04-13T03:18:28.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 Processor -- start 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dace24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dad7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b1dacf8ec0 con 0x55b1dacf9a90 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1dace8e10 con 0x55b1dace3de0 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55b1dacf9510 con 0x55b1dace41c0 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f112137c740 10 client.0 _ll_register_callbacks cb 0x7f110c058ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f111ca54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f111ca54640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52992/0 (socket says 172.21.15.106:52992) 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.543+0000 7f111ca54640 1 -- 172.21.15.106:0/3337092480 learned_addr learned my addr 172.21.15.106:0/3337092480 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f11177fe640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dad7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 -- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 msgr2=0x55b1dad7b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dad7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f1117fff640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dace24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 -- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 msgr2=0x55b1dace24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dace24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 -- 172.21.15.106:0/3337092480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dad12e50 con 0x55b1dacf9a90 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f111ca54640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x55b1dacb9800 tx=0x7f1110004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fc8df1ffa49cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111001b070 con 0x55b1dacf9a90 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f1110013990 con 0x55b1dacf9a90 2024-04-13T03:18:28.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c007490 con 0x55b1dacf9a90 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1110026050 con 0x55b1dacf9a90 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 10 client.8850 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b1dacfa7b0 con 0x55b1dacf9a90 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c007490 con 0x55b1dacf9a90 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f1110029020 con 0x55b1dacf9a90 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 10 client.8850 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 10 client.8850 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 20 client.8850 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacbc1f0 0x55b1dacbaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 4 client.8850 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.544+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "status"}) -- 0x55b1dac29220 con 0x55b1dacbc1f0 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f1117fff640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacbc1f0 0x55b1dacbaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f10f6ffd640 10 client.8850 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f1117fff640 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacbc1f0 0x55b1dacbaec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685d0cd4c8782638 server_cookie=480a76bf2dcb60b4 in_seq=0 out_seq=0 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f10f6ffd640 1 -- 172.21.15.106:0/3337092480 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b1dac29220 con 0x55b1dacbc1f0 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f10f6ffd640 10 client.8850 handle_command_reply: tid=0 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 client.8850 shutdown 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 msgr2=0x55b1dacba980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x55b1dacb9800 tx=0x7f1110004750 comp rx=0 tx=0).stop 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 shutdown_connections 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x55b1dacbc1f0 0x55b1dacbaec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55b1dace41c0 0x55b1dad7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b1dacf9a90 0x55b1dacba980 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f10f6ffd640 0 client.8850 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 --2- 172.21.15.106:0/3337092480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1dace3de0 0x55b1dace24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 >> 172.21.15.106:0/3337092480 conn(0x55b1dacb72c0 msgr2=0x55b1dad25520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 shutdown_connections 2024-04-13T03:18:28.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.545+0000 7f112137c740 1 -- 172.21.15.106:0/3337092480 wait complete. 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f112137c740 20 client.8850 trim_cache size 0 max 16384 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 8427, 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:18:28.554 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.3970435319999996, 2024-04-13T03:18:28.555 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 77, 2024-04-13T03:18:28.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 110, 2024-04-13T03:18:28.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 110, 2024-04-13T03:18:28.555 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 59.201747496999999 2024-04-13T03:18:28.555 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 msgr2=0x7f110406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1108036000 tx=0x7f1108008030 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 msgr2=0x7f11181e5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c400 tx=0x7f110c00c8c0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 shutdown_connections 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11180d1c30 0x7f111815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11180cfea0 0x7f11181e5970 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 --2- 172.21.15.106:0/3276793983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180704a0 0x7f111813d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 >> 172.21.15.106:0/3276793983 conn(0x7f1118071870 msgr2=0x7f11180b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 shutdown_connections 2024-04-13T03:18:28.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.546+0000 7f111e4b7640 1 -- 172.21.15.106:0/3276793983 wait complete. 2024-04-13T03:18:28.562 DEBUG:teuthology.orchestra.run.smithi106:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-13T03:18:28.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:18:28.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-13T03:18:28.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.669+0000 7fb2cf188640 1 Processor -- start 2024-04-13T03:18:28.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.669+0000 7fb2cf188640 1 -- start start 2024-04-13T03:18:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cf188640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 0x7fb2c80b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cf188640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb2c80d1b90 0x7fb2c80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cf188640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c80b2020 con 0x7fb2c80d1ff0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cf188640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c80b2c30 con 0x7fb2c80d1b90 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cf188640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c80b2300 con 0x7fb2c815c6f0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 0x7fb2c80b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cd725640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb2c80d1ff0 0x7fb2c80b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59736/0 (socket says 172.21.15.106:59736) 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2cd725640 1 -- 172.21.15.106:0/3929921261 learned_addr learned my addr 172.21.15.106:0/3929921261 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/3929921261 >> v2:172.21.15.114:3300/0 conn(0x7fb2c80d1b90 0x7fb2c80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2c77fe640 1 -- 172.21.15.106:0/3929921261 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907684713 0 0) 0x7fb2c80b2020 con 0x7fb2c80d1ff0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2c77fe640 1 -- 172.21.15.106:0/3929921261 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b8003520 con 0x7fb2c80d1ff0 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2ccf24640 1 -- 172.21.15.106:0/3929921261 >> v2:172.21.15.114:3300/0 conn(0x7fb2c80d1b90 msgr2=0x7fb2c80ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/3929921261 >> v2:172.21.15.114:3300/0 conn(0x7fb2c80d1b90 0x7fb2c80ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2ccf24640 1 -- 172.21.15.106:0/3929921261 >> v1:172.21.15.106:6789/0 conn(0x7fb2c80d1ff0 legacy=0x7fb2c80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.670+0000 7fb2ccf24640 1 -- 172.21.15.106:0/3929921261 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80b25e0 con 0x7fb2c815c6f0 2024-04-13T03:18:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.671+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/3929921261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 0x7fb2c80b53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80b2db0 tx=0x7fb2bc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ed1c9d44309b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.671+0000 7fb2c77fe640 1 -- 172.21.15.106:0/3929921261 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc035020 con 0x7fb2c815c6f0 2024-04-13T03:18:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.671+0000 7fb2c77fe640 1 -- 172.21.15.106:0/3929921261 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc031800 con 0x7fb2c815c6f0 2024-04-13T03:18:28.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.671+0000 7fb2c77fe640 1 -- 172.21.15.106:0/3929921261 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc031bb0 con 0x7fb2c815c6f0 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 -- 172.21.15.106:0/3929921261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 msgr2=0x7fb2c80b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 --2- 172.21.15.106:0/3929921261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 0x7fb2c80b53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80b2db0 tx=0x7fb2bc02b8a0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 -- 172.21.15.106:0/3929921261 shutdown_connections 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 --2- 172.21.15.106:0/3929921261 >> v2:172.21.15.114:3300/0 conn(0x7fb2c80d1b90 0x7fb2c80ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 --2- 172.21.15.106:0/3929921261 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c815c6f0 0x7fb2c80b53c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 -- 172.21.15.106:0/3929921261 >> 172.21.15.106:0/3929921261 conn(0x7fb2c80b4060 msgr2=0x7fb2c80b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 -- 172.21.15.106:0/3929921261 shutdown_connections 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.672+0000 7fb2cf188640 1 -- 172.21.15.106:0/3929921261 wait complete. 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.673+0000 7fb2cf188640 1 Processor -- start 2024-04-13T03:18:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.673+0000 7fb2cf188640 1 -- start start 2024-04-13T03:18:28.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c80d1b90 0x7fb2c815e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c80d1ff0 0x7fb2c814b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c815cad0 con 0x7fb2c815c6f0 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80b5900 con 0x7fb2c80d1b90 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cf188640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c81f46d0 con 0x7fb2c80d1ff0 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cd725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cd725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52994/0 (socket says 172.21.15.106:52994) 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.674+0000 7fb2cd725640 1 -- 172.21.15.106:0/2807529340 learned_addr learned my addr 172.21.15.106:0/2807529340 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c80d1ff0 0x7fb2c814b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 -- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c80d1ff0 msgr2=0x7fb2c814b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c80d1ff0 0x7fb2c814b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 -- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c80d1b90 msgr2=0x7fb2c815e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c80d1b90 0x7fb2c815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c80d1b90 0x7fb2c815e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 -- 172.21.15.106:0/2807529340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81575e0 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000cb40 tx=0x7fb2c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2c57fa640 1 -- 172.21.15.106:0/2807529340 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0017070 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2c57fa640 1 -- 172.21.15.106:0/2807529340 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c000aca0 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c8157020 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2c57fa640 1 -- 172.21.15.106:0/2807529340 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012620 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.675+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c806d320 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.676+0000 7fb2c57fa640 1 -- 172.21.15.106:0/2807529340 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2c001a070 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.677+0000 7fb2c57fa640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.677+0000 7fb2c57fa640 1 -- 172.21.15.106:0/2807529340 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fb2c008bc00 con 0x7fb2c815c6f0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.677+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.677+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc036000 tx=0x7fb2bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.678+0000 7fb2d1f69740 1 Processor -- start 2024-04-13T03:18:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.678+0000 7fb2d1f69740 1 -- start start 2024-04-13T03:18:28.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.678+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 0x560250b4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560250ab00a0 con 0x560250b4bd40 2024-04-13T03:18:28.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560250a56930 con 0x560250b4c120 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560250aac200 con 0x560250b72db0 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53008/0 (socket says 172.21.15.106:53008) 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 -- 172.21.15.106:0/1186457104 learned_addr learned my addr 172.21.15.106:0/1186457104 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 0x560250b4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2cd725640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 -- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 msgr2=0x560250b4ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 0x560250b4ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 -- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 msgr2=0x560250b4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 -- 172.21.15.106:0/1186457104 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56025088a7e0 con 0x560250b4bd40 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 0x560250b4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009860 tx=0x7fb2bc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07bb309d063a7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2b27fc640 1 -- 172.21.15.106:0/1186457104 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc04b070 con 0x560250b4bd40 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2b27fc640 1 -- 172.21.15.106:0/1186457104 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc004230 con 0x560250b4bd40 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.679+0000 7fb2b27fc640 1 -- 172.21.15.106:0/1186457104 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc035040 con 0x560250b4bd40 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 -- 172.21.15.106:0/1186457104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 msgr2=0x560250b72870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009860 tx=0x7fb2bc030000 comp rx=0 tx=0).stop 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 -- 172.21.15.106:0/1186457104 shutdown_connections 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b72db0 0x560250b4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/1186457104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b72870 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 -- 172.21.15.106:0/1186457104 >> 172.21.15.106:0/1186457104 conn(0x560250b126e0 msgr2=0x560250b90a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 -- 172.21.15.106:0/1186457104 shutdown_connections 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.680+0000 7fb2d1f69740 1 -- 172.21.15.106:0/1186457104 wait complete. 2024-04-13T03:18:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.681+0000 7fb2d1f69740 1 -- start start 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.681+0000 7fb2d1f69740 1 Processor -- start 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b79310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b79850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560250b723e0 con 0x560250b4bd40 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560250a69cf0 con 0x560250b4c120 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560250b6b160 con 0x560250ba3bf0 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2d1f69740 10 client.0 _ll_register_callbacks cb 0x560250a696e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53014/0 (socket says 172.21.15.106:53014) 2024-04-13T03:18:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 -- 172.21.15.106:0/16795552 learned_addr learned my addr 172.21.15.106:0/16795552 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b79310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2cd725640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b79850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 -- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 msgr2=0x560250b79850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b79850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 -- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 msgr2=0x560250b79310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b79310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 -- 172.21.15.106:0/16795552 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602508eaf70 con 0x560250b4bd40 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b79310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc03b800 tx=0x7fb2bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02407181a0ebaed server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.682+0000 7fb2a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc04b030 con 0x560250b4bd40 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fb2bc059d20 con 0x560250b4bd40 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc004230 con 0x560250b4bd40 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb2bc035040 con 0x560250b4bd40 2024-04-13T03:18:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 10 client.8862 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x560250782170 con 0x560250b4bd40 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc004230 con 0x560250b4bd40 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7fb2bc003a20 con 0x560250b4bd40 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 10 client.8862 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 10 client.8862 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 20 client.8862 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250b78100 0x560250b8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 4 client.8862 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560250abdac0 con 0x560250b78100 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.683+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250b78100 0x560250b8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250b78100 0x560250b8e350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5022c52cf51bdcb server_cookie=530e8ee575561aa2 in_seq=0 out_seq=0 2024-04-13T03:18:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2a77fe640 10 client.8862 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2a77fe640 1 -- 172.21.15.106:0/16795552 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560250abdac0 con 0x560250b78100 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2a77fe640 10 client.8862 handle_command_reply: tid=0 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 client.8862 shutdown 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 msgr2=0x560250b8ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc03b800 tx=0x7fb2bc004730 comp rx=0 tx=0).stop 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 shutdown_connections 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250b78100 0x560250b8e350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b79850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250b79310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2a77fe640 0 client.8862 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.684+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/16795552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b8ff70 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.685+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 >> 172.21.15.106:0/16795552 conn(0x560250b126e0 msgr2=0x560250b71b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.685+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 shutdown_connections 2024-04-13T03:18:28.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.685+0000 7fb2d1f69740 1 -- 172.21.15.106:0/16795552 wait complete. 2024-04-13T03:18:28.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.685+0000 7fb2d1f69740 20 client.8862 trim_cache size 0 max 16384 2024-04-13T03:18:28.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.689+0000 7f3c6b37b640 1 Processor -- start 2024-04-13T03:18:28.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.689+0000 7f3c6b37b640 1 -- start start 2024-04-13T03:18:28.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c6b37b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3c6415ac90 0x7f3c6406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c6b37b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 0x7f3c640cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c6b37b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c6406e340 con 0x7f3c640b4e60 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c6b37b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6406e620 con 0x7f3c6415ac90 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c6b37b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6406e900 con 0x7f3c6406c880 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c68916640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3c640b4e60 0x7f3c6406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59750/0 (socket says 172.21.15.106:59750) 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c68916640 1 -- 172.21.15.106:0/3005411718 learned_addr learned my addr 172.21.15.106:0/3005411718 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69117640 1 --2- 172.21.15.106:0/3005411718 >> v2:172.21.15.114:3300/0 conn(0x7f3c6415ac90 0x7f3c6406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69918640 1 --2- 172.21.15.106:0/3005411718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 0x7f3c640cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886686821 0 0) 0x7f3c6406e340 con 0x7f3c640b4e60 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c50004630 con 0x7f3c640b4e60 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2658843586 0 0) 0x7f3c50004630 con 0x7f3c640b4e60 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69918640 1 -- 172.21.15.106:0/3005411718 >> v2:172.21.15.114:3300/0 conn(0x7f3c6415ac90 msgr2=0x7f3c6406f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69918640 1 --2- 172.21.15.106:0/3005411718 >> v2:172.21.15.114:3300/0 conn(0x7f3c6415ac90 0x7f3c6406f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69918640 1 -- 172.21.15.106:0/3005411718 >> v1:172.21.15.106:6789/0 conn(0x7f3c640b4e60 legacy=0x7f3c6406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.690+0000 7f3c69918640 1 -- 172.21.15.106:0/3005411718 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6406ebe0 con 0x7f3c6406c880 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c69918640 1 --2- 172.21.15.106:0/3005411718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 0x7f3c640cd910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c02bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5f1aee5249389e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c039070 con 0x7f3c6406c880 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c02c8a0 con 0x7f3c6406c880 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c5bfff640 1 -- 172.21.15.106:0/3005411718 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54001670 con 0x7f3c6406c880 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 -- 172.21.15.106:0/3005411718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 msgr2=0x7f3c640cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 --2- 172.21.15.106:0/3005411718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 0x7f3c640cd910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c02bd70 comp rx=0 tx=0).stop 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 -- 172.21.15.106:0/3005411718 shutdown_connections 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 --2- 172.21.15.106:0/3005411718 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6406c880 0x7f3c640cd910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 --2- 172.21.15.106:0/3005411718 >> v2:172.21.15.114:3300/0 conn(0x7f3c6415ac90 0x7f3c6406f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 -- 172.21.15.106:0/3005411718 >> 172.21.15.106:0/3005411718 conn(0x7f3c640d0380 msgr2=0x7f3c640d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 -- 172.21.15.106:0/3005411718 shutdown_connections 2024-04-13T03:18:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.692+0000 7f3c6b37b640 1 -- 172.21.15.106:0/3005411718 wait complete. 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.693+0000 7f3c6b37b640 1 Processor -- start 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 -- start start 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c6406c880 0x7f3c6415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6415ac90 0x7f3c64148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c64151d10 0x7f3c64154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6415b070 con 0x7f3c64151d10 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6406eec0 con 0x7f3c6406c880 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c6b37b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6406f920 con 0x7f3c6415ac90 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c69918640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c64151d10 0x7f3c64154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c69918640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c64151d10 0x7f3c64154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53020/0 (socket says 172.21.15.106:53020) 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c69918640 1 -- 172.21.15.106:0/2140842985 learned_addr learned my addr 172.21.15.106:0/2140842985 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c69117640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c6406c880 0x7f3c6415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.694+0000 7f3c68916640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6415ac90 0x7f3c64148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 -- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6415ac90 msgr2=0x7f3c64148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3c6415ac90 0x7f3c64148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 -- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c6406c880 msgr2=0x7f3c6415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c6406c880 0x7f3c6415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 -- 172.21.15.106:0/2140842985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c64152200 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c69918640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c64151d10 0x7f3c64154a50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009eb0 tx=0x7f3c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c039050 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c045070 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c6b37b640 1 -- 172.21.15.106:0/2140842985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6415c940 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c009450 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.695+0000 7f3c6b37b640 1 -- 172.21.15.106:0/2140842985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6415cf00 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.696+0000 7f3c6b37b640 1 -- 172.21.15.106:0/2140842985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3c64068a10 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.697+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c5c040690 con 0x7f3c64151d10 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.697+0000 7f3c59ffb640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c4806c8f0 0x7f3c4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f3c5c0aeff0 con 0x7f3c64151d10 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c69117640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c4806c8f0 0x7f3c4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+443 (secure 0 0 0) 0x7f3c5c0af4a0 con 0x7f3c64151d10 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c6b37b640 1 -- 172.21.15.106:0/2140842985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f3c6406d180 con 0x7f3c64151d10 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c69117640 1 --2- 172.21.15.106:0/2140842985 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3c4806c8f0 0x7f3c4806edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009aa0 tx=0x7f3c60009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.698+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 7 ==== log(2 entries from seq 1197 at 2024-04-13T03:18:27.162335+0000) ==== 389+0+0 (secure 0 0 0) 0x7f3c5c0ad430 con 0x7f3c64151d10 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stdout: cluster: 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stdout: id: d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stdout: health: HEALTH_OK 2024-04-13T03:18:28.705 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: services: 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: mgr: y(active, since 9m), standbys: x 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: mds: 1/1 daemons up, 3 standby 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: data: 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: volumes: 1/1 healthy 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: pools: 3 pools, 97 pgs 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: objects: 30 objects, 19 MiB 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-13T03:18:28.706 INFO:teuthology.orchestra.run.smithi106.stdout: pgs: 97 active+clean 2024-04-13T03:18:28.707 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:28.707 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:28.707 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:27.162335+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-13T03:18:28.707 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:27.162358+0000 mon.a [INF] Cluster is now healthy 2024-04-13T03:18:28.713 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:18:28.711+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-13T03:18:28.713 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:18:28.711+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-13T03:18:28.714 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:18:28.712+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-13T03:18:28.714 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:18:28.712+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 Processor -- start 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 -- start start 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bfcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560250b91430 con 0x560250b4bd40 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c82013a0 con 0x560250b4c120 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c8200380 con 0x560250ba3bf0 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2ccf24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53034/0 (socket says 172.21.15.106:53034) 2024-04-13T03:18:28.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.705+0000 7fb2ccf24640 1 -- 172.21.15.106:0/4180659967 learned_addr learned my addr 172.21.15.106:0/4180659967 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2c7fff640 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bfcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2cd725640 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 -- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 msgr2=0x560250b792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 -- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 msgr2=0x560250bfcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bfcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 -- 172.21.15.106:0/4180659967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560250a6e870 con 0x560250b4bd40 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc003900 tx=0x7fb2bc051070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712cb798990c699 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2b1ffb640 1 -- 172.21.15.106:0/4180659967 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc04b040 con 0x560250b4bd40 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2b1ffb640 1 -- 172.21.15.106:0/4180659967 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc053070 con 0x560250b4bd40 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.706+0000 7fb2b1ffb640 1 -- 172.21.15.106:0/4180659967 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0041e0 con 0x560250b4bd40 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- 172.21.15.106:0/4180659967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 msgr2=0x560250b44a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc003900 tx=0x7fb2bc051070 comp rx=0 tx=0).stop 2024-04-13T03:18:28.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- 172.21.15.106:0/4180659967 shutdown_connections 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bfcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/4180659967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250b4bd40 0x560250b44a90 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- 172.21.15.106:0/4180659967 >> 172.21.15.106:0/4180659967 conn(0x560250b126e0 msgr2=0x560250bb46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- 172.21.15.106:0/4180659967 shutdown_connections 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- 172.21.15.106:0/4180659967 wait complete. 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.707+0000 7fb2d1f69740 1 -- start start 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 Processor -- start 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b4bd40 0x560250bfc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bf5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81ffe90 con 0x560250ba3bf0 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c806e6c0 con 0x560250b4c120 2024-04-13T03:18:28.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560250ba42e0 con 0x560250b4bd40 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2d1f69740 10 client.0 _ll_register_callbacks cb 0x7fb2bc03ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2cd725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2cd725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53040/0 (socket says 172.21.15.106:53040) 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.708+0000 7fb2cd725640 1 -- 172.21.15.106:0/2332373437 learned_addr learned my addr 172.21.15.106:0/2332373437 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 -- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b4bd40 msgr2=0x560250bfc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b4bd40 0x560250bfc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 -- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 msgr2=0x560250bf5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bf5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 -- 172.21.15.106:0/2332373437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560250baad00 con 0x560250ba3bf0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2cd725640 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0089f30 tx=0x7fb2c00597f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cef1222b36593a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0061e80 con 0x560250ba3bf0 2024-04-13T03:18:28.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fb2c0062020 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc053070 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb2c0078050 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 10 client.8874 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x560250bf7930 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc053070 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7fb2c0004b60 con 0x560250ba3bf0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 10 client.8874 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 10 client.8874 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 20 client.8874 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250bf6370 0x560250bf6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 4 client.8874 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.709+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x560250b90300 con 0x560250bf6370 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.710+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250bf6370 0x560250bf6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.710+0000 7fb2ccf24640 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250bf6370 0x560250bf6780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b21c1217dc053a server_cookie=da1b8c9924eca116 in_seq=0 out_seq=0 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.710+0000 7fb2a77fe640 10 client.8874 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2a77fe640 1 -- 172.21.15.106:0/2332373437 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560250b90300 con 0x560250bf6370 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2a77fe640 10 client.8874 handle_command_reply: tid=0 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 client.8874 shutdown 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 msgr2=0x560250b792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0089f30 tx=0x7fb2c00597f0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 shutdown_connections 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x560250bf6370 0x560250bf6780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560250ba3bf0 0x560250b792f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2a77fe640 0 client.8874 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560250b4c120 0x560250bf5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 --2- 172.21.15.106:0/2332373437 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560250b4bd40 0x560250bfc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 >> 172.21.15.106:0/2332373437 conn(0x560250b126e0 msgr2=0x560250b863f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 shutdown_connections 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 1 -- 172.21.15.106:0/2332373437 wait complete. 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.711+0000 7fb2d1f69740 20 client.8874 trim_cache size 0 max 16384 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0, 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_tag": "01d70c5a-1b7c-40f1-a3b6-a6e0d8f38f27", 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": "asynchronous" 2024-04-13T03:18:28.721 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 msgr2=0x7fb2ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc036000 tx=0x7fb2bc008030 comp rx=0 tx=0).stop 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 msgr2=0x7fb2c814bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000cb40 tx=0x7fb2c000a060 comp rx=0 tx=0).stop 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 shutdown_connections 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb2c80d1ff0 0x7fb2c814b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2c815c6f0 0x7fb2c814bc00 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 --2- 172.21.15.106:0/2807529340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c80d1b90 0x7fb2c815e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 >> 172.21.15.106:0/2807529340 conn(0x7fb2c80b4060 msgr2=0x7fb2c815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 shutdown_connections 2024-04-13T03:18:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.712+0000 7fb2cf188640 1 -- 172.21.15.106:0/2807529340 wait complete. 2024-04-13T03:18:28.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:18:28.725 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-13T03:18:28.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.824+0000 7f89ca488640 1 Processor -- start 2024-04-13T03:18:28.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.824+0000 7f89ca488640 1 -- start start 2024-04-13T03:18:28.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89ca488640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f89c406e740 0x7f89c415c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89ca488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89c40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89ca488640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c406d180 con 0x7f89c41592b0 2024-04-13T03:18:28.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89ca488640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f89c40ce180 con 0x7f89c406e740 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89ca488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f89c406ec90 con 0x7f89c415c7a0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f89c41592b0 0x7f89c415bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59762/0 (socket says 172.21.15.106:59762) 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c3fff640 1 -- 172.21.15.106:0/3430000178 learned_addr learned my addr 172.21.15.106:0/3430000178 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1482798857 0 0) 0x7f89c406d180 con 0x7f89c41592b0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c8a25640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89c40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3430000178 >> v2:172.21.15.114:3300/0 conn(0x7f89c406e740 0x7f89c415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89ac003520 con 0x7f89c41592b0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1816449548 0 0) 0x7f89ac003520 con 0x7f89c41592b0 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 msgr2=0x7f89c40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89c40ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 >> v2:172.21.15.114:3300/0 conn(0x7f89c406e740 msgr2=0x7f89c415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 --2- 172.21.15.106:0/3430000178 >> v2:172.21.15.114:3300/0 conn(0x7f89c406e740 0x7f89c415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c406ef70 con 0x7f89c41592b0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f89b8001670 con 0x7f89c41592b0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 >> v1:172.21.15.106:6789/0 conn(0x7f89c41592b0 legacy=0x7f89c415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89ac004800 0x7f89ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89c40ce180 con 0x7f89ac004800 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89c406ec90 con 0x7f89ac0039b0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.825+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac00dbd0 con 0x7f89c415c7a0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89ac004800 0x7f89ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c3fff640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c8a25640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 -- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 msgr2=0x7f89ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89c415c7a0 0x7f89ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 -- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ac0039b0 msgr2=0x7f89ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.826+0000 7f89c37fe640 1 -- 172.21.15.106:0/3430000178 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac003520 con 0x7f89ac004800 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89ac004800 0x7f89ac004c80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ac0 tx=0x7f89ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74939d483c295048 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4039070 con 0x7f89ac004800 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b402c710 con 0x7f89ac004800 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89c2ffd640 1 -- 172.21.15.106:0/3430000178 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4009690 con 0x7f89ac004800 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 -- 172.21.15.106:0/3430000178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89ac004800 msgr2=0x7f89ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 --2- 172.21.15.106:0/3430000178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89ac004800 0x7f89ac004c80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ac0 tx=0x7f89ac005830 comp rx=0 tx=0).stop 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89c8a25640 1 -- 172.21.15.106:0/3430000178 reap_dead start 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 -- 172.21.15.106:0/3430000178 shutdown_connections 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 -- 172.21.15.106:0/3430000178 >> 172.21.15.106:0/3430000178 conn(0x7f89c40b2600 msgr2=0x7f89c40d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 -- 172.21.15.106:0/3430000178 shutdown_connections 2024-04-13T03:18:28.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.827+0000 7f89ca488640 1 -- 172.21.15.106:0/3430000178 wait complete. 2024-04-13T03:18:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.828+0000 7f89ca488640 1 Processor -- start 2024-04-13T03:18:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 -- start start 2024-04-13T03:18:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 0x7f89c4144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89c415e650 0x7f89c40b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89c406b8e0 con 0x7f89c41536a0 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89c406f530 con 0x7f89c415e650 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89ca488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f89c406ba60 con 0x7f89ac004800 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53060/0 (socket says 172.21.15.106:53060) 2024-04-13T03:18:28.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.829+0000 7f89c37fe640 1 -- 172.21.15.106:0/676570686 learned_addr learned my addr 172.21.15.106:0/676570686 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c3fff640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 0x7f89c4144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c8a25640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89c415e650 0x7f89c40b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 -- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 msgr2=0x7f89c4144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 0x7f89c4144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 -- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89c415e650 msgr2=0x7f89c40b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89c415e650 0x7f89c40b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 -- 172.21.15.106:0/676570686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c4145290 con 0x7f89c41536a0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c3fff640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 0x7f89c4144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c37fe640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f89b4037040 tx=0x7f89b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c0ff9640 1 -- 172.21.15.106:0/676570686 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4042020 con 0x7f89c41536a0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c0ff9640 1 -- 172.21.15.106:0/676570686 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b403a040 con 0x7f89c41536a0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89c0ff9640 1 -- 172.21.15.106:0/676570686 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4039070 con 0x7f89c41536a0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c40b3e60 con 0x7f89c41536a0 2024-04-13T03:18:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.830+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c4152ea0 con 0x7f89c41536a0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.831+0000 7f89c0ff9640 1 -- 172.21.15.106:0/676570686 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89b4045020 con 0x7f89c41536a0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.832+0000 7f89c0ff9640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.832+0000 7f89c0ff9640 1 -- 172.21.15.106:0/676570686 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f89b40ad0f0 con 0x7f89c41536a0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.832+0000 7f89c3fff640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.832+0000 7f89c3fff640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 0x7f89b006ede0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89b8001eb0 tx=0x7f89b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.833+0000 7f89cd271740 1 Processor -- start 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.833+0000 7f89cd271740 1 -- start start 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.833+0000 7f89cd271740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ba3d0 0x557e181bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e1818d4d0 0x557e181bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89cd271740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557e180f8170 con 0x557e181b9ff0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557e1809e7f0 con 0x557e181ba3d0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557e180f4130 con 0x557e1818d4d0 2024-04-13T03:18:28.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53062/0 (socket says 172.21.15.106:53062) 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 -- 172.21.15.106:0/64404836 learned_addr learned my addr 172.21.15.106:0/64404836 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 -- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e1818d4d0 msgr2=0x557e181bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e1818d4d0 0x557e181bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 -- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ba3d0 msgr2=0x557e181bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ba3d0 0x557e181bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 -- 172.21.15.106:0/64404836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e17ed47d0 con 0x557e181b9ff0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89c3fff640 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f89b8007760 tx=0x7f89b8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae3657fa2ad3ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89a9ffb640 1 -- 172.21.15.106:0/64404836 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8026070 con 0x557e181b9ff0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89a9ffb640 1 -- 172.21.15.106:0/64404836 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b801c050 con 0x557e181b9ff0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.834+0000 7f89a9ffb640 1 -- 172.21.15.106:0/64404836 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8021410 con 0x557e181b9ff0 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- 172.21.15.106:0/64404836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 msgr2=0x557e1818cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f89b8007760 tx=0x7f89b8013000 comp rx=0 tx=0).stop 2024-04-13T03:18:28.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- 172.21.15.106:0/64404836 shutdown_connections 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e1818d4d0 0x557e181bd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ba3d0 0x557e181bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 --2- 172.21.15.106:0/64404836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181b9ff0 0x557e1818cf90 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- 172.21.15.106:0/64404836 >> 172.21.15.106:0/64404836 conn(0x557e1818f200 msgr2=0x557e181d7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- 172.21.15.106:0/64404836 shutdown_connections 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- 172.21.15.106:0/64404836 wait complete. 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.835+0000 7f89cd271740 1 -- start start 2024-04-13T03:18:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 Processor -- start 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 0x557e181af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 0x557e181d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557e181918a0 con 0x557e181ba3d0 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557e180b1d00 con 0x557e181ebe00 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557e181940e0 con 0x557e181b9ff0 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89cd271740 10 client.0 _ll_register_callbacks cb 0x557e180b16f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.836+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53068/0 (socket says 172.21.15.106:53068) 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 -- 172.21.15.106:0/2220131064 learned_addr learned my addr 172.21.15.106:0/2220131064 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 -- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 msgr2=0x557e181af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c8a25640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 0x557e181d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c3fff640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 0x557e181af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 0x557e181af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 -- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 msgr2=0x557e181d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 0x557e181d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 -- 172.21.15.106:0/2220131064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e17e0a3f0 con 0x557e181ba3d0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c3fff640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 0x557e181af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c8a25640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 0x557e181d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89c37fe640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f89b4077110 tx=0x7f89b4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93103107cc0159a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b407b3b0 con 0x557e181ba3d0 2024-04-13T03:18:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f89b4083400 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b801c050 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f89b40ab050 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 10 client.8886 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557e17dca0f0 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b801c050 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f89b407a100 con 0x557e181ba3d0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 10 client.8886 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 10 client.8886 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 20 client.8886 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181be3f0 0x557e181be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 4 client.8886 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.837+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e18105b90 con 0x557e181be3f0 2024-04-13T03:18:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.838+0000 7f89c3fff640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181be3f0 0x557e181be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.838+0000 7f89c3fff640 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181be3f0 0x557e181be810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78f5b25cb174267 server_cookie=bdefd3b6a0aca7eb in_seq=0 out_seq=0 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.838+0000 7f89a2ffd640 10 client.8886 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89a2ffd640 1 -- 172.21.15.106:0/2220131064 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e18105b90 con 0x557e181be3f0 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89a2ffd640 10 client.8886 handle_command_reply: tid=0 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 client.8886 shutdown 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 msgr2=0x557e181d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f89b4077110 tx=0x7f89b4009830 comp rx=0 tx=0).stop 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 shutdown_connections 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181be3f0 0x557e181be810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181ebe00 0x557e181d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7650 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 --2- 172.21.15.106:0/2220131064 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181b9ff0 0x557e181af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89a2ffd640 0 client.8886 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 >> 172.21.15.106:0/2220131064 conn(0x557e1818d4d0 msgr2=0x557e181bc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 shutdown_connections 2024-04-13T03:18:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 1 -- 172.21.15.106:0/2220131064 wait complete. 2024-04-13T03:18:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.839+0000 7f89cd271740 20 client.8886 trim_cache size 0 max 16384 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.856+0000 7f89cd271740 1 Processor -- start 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.856+0000 7f89cd271740 1 -- start start 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 0x557e181d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41fdb50 con 0x557e181ba3d0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41fcb80 con 0x557e181b9ff0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f89c40b54b0 con 0x557e181ebe00 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53078/0 (socket says 172.21.15.106:53078) 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 -- 172.21.15.106:0/1373712390 learned_addr learned my addr 172.21.15.106:0/1373712390 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c3fff640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c8a25640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 0x557e181d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 -- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 msgr2=0x557e181d7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 0x557e181d7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 -- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 msgr2=0x557e181bfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181bfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 -- 172.21.15.106:0/1373712390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e181c03d0 con 0x557e181ba3d0 2024-04-13T03:18:28.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c3fff640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181bfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c8a25640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 0x557e181d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89c37fe640 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f89b4083360 tx=0x7f89b4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb907c8fc7980cf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89a97fa640 1 -- 172.21.15.106:0/1373712390 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4083890 con 0x557e181ba3d0 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89a97fa640 1 -- 172.21.15.106:0/1373712390 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b407acb0 con 0x557e181ba3d0 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.857+0000 7f89a97fa640 1 -- 172.21.15.106:0/1373712390 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b407f060 con 0x557e181ba3d0 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- 172.21.15.106:0/1373712390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 msgr2=0x557e181d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f89b4083360 tx=0x7f89b4040000 comp rx=0 tx=0).stop 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- 172.21.15.106:0/1373712390 shutdown_connections 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ebe00 0x557e181d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ba3d0 0x557e181d7590 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 --2- 172.21.15.106:0/1373712390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181bfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- 172.21.15.106:0/1373712390 >> 172.21.15.106:0/1373712390 conn(0x557e1818d4d0 msgr2=0x557e181c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- 172.21.15.106:0/1373712390 shutdown_connections 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- 172.21.15.106:0/1373712390 wait complete. 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.858+0000 7f89cd271740 1 -- start start 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 Processor -- start 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ba3d0 0x557e182517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557e181fca90 con 0x557e181ebe00 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41fc530 con 0x557e181b9ff0 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x557e1818d320 con 0x557e181ba3d0 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89cd271740 10 client.0 _ll_register_callbacks cb 0x7f89b407ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.859+0000 7f89c8a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53092/0 (socket says 172.21.15.106:53092) 2024-04-13T03:18:28.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 -- 172.21.15.106:0/3827767583 learned_addr learned my addr 172.21.15.106:0/3827767583 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c3fff640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c37fe640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ba3d0 0x557e182517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 -- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ba3d0 msgr2=0x557e182517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ba3d0 0x557e182517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 -- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 msgr2=0x557e181cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 -- 172.21.15.106:0/3827767583 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e181b7300 con 0x557e181ebe00 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89c8a25640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b3090 tx=0x7f89bc008630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b071efbf4eb4cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc01c020 con 0x557e181ebe00 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f89bc017440 con 0x557e181ebe00 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b407acb0 con 0x557e181ebe00 2024-04-13T03:18:28.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f89bc026050 con 0x557e181ebe00 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89cd271740 10 client.8892 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557e18190700 con 0x557e181ebe00 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.860+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b407acb0 con 0x557e181ebe00 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f89bc029020 con 0x557e181ebe00 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 10 client.8892 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 10 client.8892 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 20 client.8892 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181b87f0 0x557e181fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 4 client.8892 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e180ffa00 con 0x557e181b87f0 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89c3fff640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181b87f0 0x557e181fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89c3fff640 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181b87f0 0x557e181fb4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d49e47727474bde server_cookie=ecbd89a5ec3d80f5 in_seq=0 out_seq=0 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89a2ffd640 10 client.8892 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89a2ffd640 1 -- 172.21.15.106:0/3827767583 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x557e180ffa00 con 0x557e181b87f0 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89a2ffd640 10 client.8892 handle_command_reply: tid=0 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 client.8892 shutdown 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 msgr2=0x557e181db690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b3090 tx=0x7f89bc008630 comp rx=0 tx=0).stop 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 shutdown_connections 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x557e181b87f0 0x557e181fb4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.861+0000 7f89a2ffd640 0 client.8892 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x557e181ba3d0 0x557e182517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e181ebe00 0x557e181db690 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 --2- 172.21.15.106:0/3827767583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e181b9ff0 0x557e181cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 >> 172.21.15.106:0/3827767583 conn(0x557e1818d4d0 msgr2=0x557e181d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 shutdown_connections 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 1 -- 172.21.15.106:0/3827767583 wait complete. 2024-04-13T03:18:28.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89cd271740 20 client.8892 trim_cache size 0 max 16384 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": { 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "01d70c5a-1b7c-40f1-a3b6-a6e0d8f38f27": { 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/", 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "tag": "01d70c5a-1b7c-40f1-a3b6-a6e0d8f38f27", 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: "options": "recursive,repair" 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:28.872 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:28.873 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 msgr2=0x7f89b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 0x7f89b006ede0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89b8001eb0 tx=0x7f89b8009040 comp rx=0 tx=0).stop 2024-04-13T03:18:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 msgr2=0x7f89c415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f89b4037040 tx=0x7f89b4004730 comp rx=0 tx=0).stop 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 shutdown_connections 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89c415e650 0x7f89c40b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89c41536a0 0x7f89c415e110 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 --2- 172.21.15.106:0/676570686 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f89ac004800 0x7f89c4144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 >> 172.21.15.106:0/676570686 conn(0x7f89c40b2600 msgr2=0x7f89c40b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 shutdown_connections 2024-04-13T03:18:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:28.862+0000 7f89ca488640 1 -- 172.21.15.106:0/676570686 wait complete. 2024-04-13T03:18:28.875 INFO:tasks.cephfs.filesystem:scrub status for tag:01d70c5a-1b7c-40f1-a3b6-a6e0d8f38f27 - {'path': '/', 'tag': '01d70c5a-1b7c-40f1-a3b6-a6e0d8f38f27', 'options': 'recursive,repair'} 2024-04-13T03:18:30.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:30.170+0000 7f3c59ffb640 1 -- 172.21.15.106:0/2140842985 <== mon.0 v2:172.21.15.106:3300/0 8 ==== log(9 entries from seq 1 at 2024-04-13T03:18:28.712619+0000) ==== 1558+0+0 (secure 0 0 0) 0x7f3c5c07abf0 con 0x7f3c64151d10 2024-04-13T03:18:30.172 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.712619+0000 mds.d [INF] scrub queued for path: / 2024-04-13T03:18:30.172 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.712623+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.712630+0000 mds.d [INF] scrub summary: active paths [/] 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.713267+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.713276+0000 mds.d [ERR] inode table repaired for inode: 0x10000000001 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.713462+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.713467+0000 mds.d [ERR] inode table repaired for inode: 0x10000000002 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:28.713845+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-13T03:18:30.173 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:18:29.350187+0000 mds.d [INF] scrub summary: idle 2024-04-13T03:18:30.343 INFO:tasks.ceph.mds.d.smithi114.stderr:2024-04-13T03:18:30.342+0000 7ffad0df1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:18:56.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:18:56.479 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:18:58.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:18:58.877 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-13T03:18:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.990+0000 7f57be715640 1 Processor -- start 2024-04-13T03:18:58.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.990+0000 7f57be715640 1 -- start start 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57be715640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57be715640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f57b806e740 0x7f57b815c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57be715640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b806d180 con 0x7f57b815c5d0 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57be715640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f57b80ce180 con 0x7f57b806e740 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57be715640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57b806ec90 con 0x7f57b81592b0 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f57b806e740 0x7f57b815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:58.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:47258/0 (socket says 172.21.15.106:47258) 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57bccb2640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f57b815c5d0 0x7f57b815f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:45060/0 (socket says 172.21.15.106:45060) 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b7fff640 1 -- 172.21.15.106:0/1434408532 learned_addr learned my addr 172.21.15.106:0/1434408532 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f57b806e740 0x7f57b815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:56754/0 (socket says 172.21.15.106:56754) 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 -- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 msgr2=0x7f57b815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 -- 172.21.15.106:0/1434408532 >> v1:172.21.15.114:6789/0 conn(0x7f57b815c5d0 legacy=0x7f57b815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b77fe640 1 -- 172.21.15.106:0/1434408532 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b806ef70 con 0x7f57b806e740 2024-04-13T03:18:58.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.991+0000 7f57b7fff640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:18:58.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.992+0000 7f57b77fe640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b806e740 0x7f57b815c090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47841c6546b9b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.992+0000 7f57b6ffd640 1 -- 172.21.15.106:0/1434408532 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac035020 con 0x7f57b806e740 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.992+0000 7f57b6ffd640 1 -- 172.21.15.106:0/1434408532 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ac031830 con 0x7f57b806e740 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.992+0000 7f57b6ffd640 1 -- 172.21.15.106:0/1434408532 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac031c00 con 0x7f57b806e740 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 -- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b806e740 msgr2=0x7f57b815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b806e740 0x7f57b815c090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 -- 172.21.15.106:0/1434408532 shutdown_connections 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b806e740 0x7f57b815c090 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 --2- 172.21.15.106:0/1434408532 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b81592b0 0x7f57b815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 -- 172.21.15.106:0/1434408532 >> 172.21.15.106:0/1434408532 conn(0x7f57b80b2600 msgr2=0x7f57b80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 -- 172.21.15.106:0/1434408532 shutdown_connections 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.993+0000 7f57be715640 1 -- 172.21.15.106:0/1434408532 wait complete. 2024-04-13T03:18:58.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.994+0000 7f57be715640 1 Processor -- start 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.994+0000 7f57be715640 1 -- start start 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b81592b0 0x7f57b80ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b815c5d0 0x7f57b80c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57b80cf630 con 0x7f57b806e740 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57b806f530 con 0x7f57b81592b0 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57be715640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57b80cf7b0 con 0x7f57b815c5d0 2024-04-13T03:18:58.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35112/0 (socket says 172.21.15.106:35112) 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57b7fff640 1 -- 172.21.15.106:0/3585237019 learned_addr learned my addr 172.21.15.106:0/3585237019 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57bccb2640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b815c5d0 0x7f57b80c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.995+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b81592b0 0x7f57b80ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 -- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b815c5d0 msgr2=0x7f57b80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b815c5d0 0x7f57b80c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 -- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b81592b0 msgr2=0x7f57b80ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b81592b0 0x7f57b80ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 -- 172.21.15.106:0/3585237019 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b80c5840 con 0x7f57b806e740 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ed40 tx=0x7f57a800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b4ff9640 1 -- 172.21.15.106:0/3585237019 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010850 con 0x7f57b806e740 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b4ff9640 1 -- 172.21.15.106:0/3585237019 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a80109f0 con 0x7f57b806e740 2024-04-13T03:18:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b80c5b20 con 0x7f57b806e740 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57b4ff9640 1 -- 172.21.15.106:0/3585237019 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010da0 con 0x7f57b806e740 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.996+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b80b4d90 con 0x7f57b806e740 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.998+0000 7f57b4ff9640 1 -- 172.21.15.106:0/3585237019 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57a80223f0 con 0x7f57b806e740 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.998+0000 7f57b4ff9640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 0x7f57a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.998+0000 7f57b4ff9640 1 -- 172.21.15.106:0/3585237019 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f57a808c7d0 con 0x7f57b806e740 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.998+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 0x7f57a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.998+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 0x7f57a406edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.999+0000 7f57c15d2740 1 Processor -- start 2024-04-13T03:18:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:58.999+0000 7f57c15d2740 1 -- start start 2024-04-13T03:18:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed71bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5610ed687170 con 0x5610ed71c4d0 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610ed62d7f0 con 0x5610ed748ff0 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed683130 con 0x5610ed7493d0 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35128/0 (socket says 172.21.15.106:35128) 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 -- 172.21.15.106:0/1801991486 learned_addr learned my addr 172.21.15.106:0/1801991486 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57b77fe640 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57b7fff640 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed71bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 -- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 msgr2=0x5610ed74bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 -- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 msgr2=0x5610ed71bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed71bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 -- 172.21.15.106:0/1801991486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ed4637d0 con 0x5610ed71c4d0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f57bccb2640 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f57b0001ca0 tx=0x7f57b0002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1183be0cfa55ea60 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f579dffb640 1 -- 172.21.15.106:0/1801991486 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b00078b0 con 0x5610ed71c4d0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f579dffb640 1 -- 172.21.15.106:0/1801991486 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b0018070 con 0x5610ed71c4d0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.000+0000 7f579dffb640 1 -- 172.21.15.106:0/1801991486 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0007a50 con 0x5610ed71c4d0 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 -- 172.21.15.106:0/1801991486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 msgr2=0x5610ed74c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f57b0001ca0 tx=0x7f57b0002160 comp rx=0 tx=0).stop 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 -- 172.21.15.106:0/1801991486 shutdown_connections 2024-04-13T03:18:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed71c4d0 0x5610ed74c2c0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 --2- 172.21.15.106:0/1801991486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed71bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 -- 172.21.15.106:0/1801991486 >> 172.21.15.106:0/1801991486 conn(0x5610ed71e200 msgr2=0x5610ed766d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 -- 172.21.15.106:0/1801991486 shutdown_connections 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.001+0000 7f57c15d2740 1 -- 172.21.15.106:0/1801991486 wait complete. 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.002+0000 7f57c15d2740 1 -- start start 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.002+0000 7f57c15d2740 1 Processor -- start 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed748ff0 0x5610ed74bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed73e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5610ed7208a0 con 0x5610ed7493d0 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610ed640d00 con 0x5610ed73dd60 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed7230e0 con 0x5610ed748ff0 2024-04-13T03:18:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 10 client.0 _ll_register_callbacks cb 0x5610ed6406f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35134/0 (socket says 172.21.15.106:35134) 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 -- 172.21.15.106:0/177166229 learned_addr learned my addr 172.21.15.106:0/177166229 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57bccb2640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed73e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b7fff640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed748ff0 0x5610ed74bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 -- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed748ff0 msgr2=0x5610ed74bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed748ff0 0x5610ed74bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 -- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 msgr2=0x5610ed73e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed73e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 -- 172.21.15.106:0/177166229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ed3993f0 con 0x5610ed7493d0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57b77fe640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0098c0 tx=0x7f57ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1017291480f09d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f5796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac053060 con 0x5610ed7493d0 2024-04-13T03:18:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f57ac044600 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.003+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b0018070 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f57ac059050 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 10 client.8901 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5610ed3590f0 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b0018070 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f57ac05c020 con 0x5610ed7493d0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 10 client.8901 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 10 client.8901 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 20 client.8901 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed749c80 0x5610ed74a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 4 client.8901 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ed694b90 con 0x5610ed749c80 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57b7fff640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed749c80 0x5610ed74a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f5796ffd640 10 client.8901 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.004+0000 7f57b7fff640 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed749c80 0x5610ed74a0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a8a3f75aa91cfd server_cookie=5b9da6ae27dede84 in_seq=0 out_seq=0 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f5796ffd640 1 -- 172.21.15.106:0/177166229 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610ed694b90 con 0x5610ed749c80 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f5796ffd640 10 client.8901 handle_command_reply: tid=0 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 client.8901 shutdown 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 msgr2=0x5610ed769080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0098c0 tx=0x7f57ac004730 comp rx=0 tx=0).stop 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 shutdown_connections 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed749c80 0x5610ed74a0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed73e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed7493d0 0x5610ed769080 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 --2- 172.21.15.106:0/177166229 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed748ff0 0x5610ed74bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 >> 172.21.15.106:0/177166229 conn(0x5610ed71c4d0 msgr2=0x5610ed71fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f5796ffd640 0 client.8901 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:59.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 shutdown_connections 2024-04-13T03:18:59.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.005+0000 7f57c15d2740 1 -- 172.21.15.106:0/177166229 wait complete. 2024-04-13T03:18:59.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.006+0000 7f57c15d2740 20 client.8901 trim_cache size 0 max 16384 2024-04-13T03:18:59.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.023+0000 7f57c15d2740 1 Processor -- start 2024-04-13T03:18:59.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.023+0000 7f57c15d2740 1 -- start start 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed7748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8201750 con 0x5610ed73dd60 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8200730 con 0x5610ed748ff0 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f57b81f8d30 con 0x5610ed7493d0 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35144/0 (socket says 172.21.15.106:35144) 2024-04-13T03:18:59.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 -- 172.21.15.106:0/25752858 learned_addr learned my addr 172.21.15.106:0/25752858 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 -- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 msgr2=0x5610ed74b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 -- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 msgr2=0x5610ed7748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed7748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 -- 172.21.15.106:0/25752858 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ed645720 con 0x5610ed73dd60 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f57b7fff640 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f57a80568a0 tx=0x7f57a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7552b9473ebae5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f579d7fa640 1 -- 172.21.15.106:0/25752858 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8077070 con 0x5610ed73dd60 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f579d7fa640 1 -- 172.21.15.106:0/25752858 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a8072070 con 0x5610ed73dd60 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.024+0000 7f579d7fa640 1 -- 172.21.15.106:0/25752858 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8062f70 con 0x5610ed73dd60 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- 172.21.15.106:0/25752858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 msgr2=0x5610ed7da3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f57a80568a0 tx=0x7f57a8000bc0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- 172.21.15.106:0/25752858 shutdown_connections 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed74b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed748ff0 0x5610ed7748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 --2- 172.21.15.106:0/25752858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed73dd60 0x5610ed7da3a0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- 172.21.15.106:0/25752858 >> 172.21.15.106:0/25752858 conn(0x5610ed71c4d0 msgr2=0x5610ed74a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- 172.21.15.106:0/25752858 shutdown_connections 2024-04-13T03:18:59.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- 172.21.15.106:0/25752858 wait complete. 2024-04-13T03:18:59.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.025+0000 7f57c15d2740 1 -- start start 2024-04-13T03:18:59.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 Processor -- start 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed7d7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed769ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5610ed73e4d0 con 0x5610ed748ff0 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57b80ae440 con 0x5610ed73dd60 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5610ed74c290 con 0x5610ed7493d0 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57c15d2740 10 client.0 _ll_register_callbacks cb 0x7f57ac031980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35146/0 (socket says 172.21.15.106:35146) 2024-04-13T03:18:59.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.026+0000 7f57b77fe640 1 -- 172.21.15.106:0/3898809085 learned_addr learned my addr 172.21.15.106:0/3898809085 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed7d7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57bccb2640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed769ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 -- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 msgr2=0x5610ed769ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed769ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 -- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 msgr2=0x5610ed7d7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed7d7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 -- 172.21.15.106:0/3898809085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ed789e40 con 0x5610ed748ff0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57b77fe640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006a90 tx=0x7f57ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63737b35576f6a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac062060 con 0x5610ed748ff0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7f57ac04e800 con 0x5610ed748ff0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a8072070 con 0x5610ed748ff0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f57ac068050 con 0x5610ed748ff0 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 10 client.8907 fetch_fsmap learned FSMap version 77 2024-04-13T03:18:59.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5610ed717b40 con 0x5610ed748ff0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a8072070 con 0x5610ed748ff0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f57ac06b020 con 0x5610ed748ff0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 10 client.8907 fetch_fsmap finished waiting for FSMap version 77 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 10 client.8907 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 20 client.8907 populate_metadata read hostname 'smithi106' 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed7d72b0 0x5610ed76a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 4 client.8907 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.027+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 --> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610ed68ea00 con 0x5610ed7d72b0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed7d72b0 0x5610ed76a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f5796ffd640 10 client.8907 ms_handle_connect on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57b7fff640 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed7d72b0 0x5610ed76a020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14292d646621e2e2 server_cookie=b20615b792e1f157 in_seq=0 out_seq=0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f5796ffd640 1 -- 172.21.15.106:0/3898809085 <== mds.0 v2:172.21.15.114:6832/3141313499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610ed68ea00 con 0x5610ed7d72b0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f5796ffd640 10 client.8907 handle_command_reply: tid=0 2024-04-13T03:18:59.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 client.8907 shutdown 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 msgr2=0x5610ed7d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006a90 tx=0x7f57ac0046c0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 shutdown_connections 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:6832/3141313499,v1:172.21.15.114:6833/3141313499] conn(0x5610ed7d72b0 0x5610ed76a020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5610ed7493d0 0x5610ed769ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f5796ffd640 0 client.8907 ms_handle_reset on v2:172.21.15.114:6832/3141313499 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5610ed748ff0 0x5610ed7d84b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 --2- 172.21.15.106:0/3898809085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610ed73dd60 0x5610ed7d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 >> 172.21.15.106:0/3898809085 conn(0x5610ed71c4d0 msgr2=0x5610ed747740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 shutdown_connections 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.028+0000 7f57c15d2740 1 -- 172.21.15.106:0/3898809085 wait complete. 2024-04-13T03:18:59.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57c15d2740 20 client.8907 trim_cache size 0 max 16384 2024-04-13T03:18:59.036 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:59.036 INFO:teuthology.orchestra.run.smithi106.stdout: "status": "no active scrubs running", 2024-04-13T03:18:59.036 INFO:teuthology.orchestra.run.smithi106.stdout: "scrubs": {} 2024-04-13T03:18:59.036 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 msgr2=0x7f57a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 0x7f57a406edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac03c040 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 msgr2=0x7f57b80b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ed40 tx=0x7f57a800d720 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 shutdown_connections 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f57a406c8f0 0x7f57a406edb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f57b815c5d0 0x7f57b80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57b81592b0 0x7f57b80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 --2- 172.21.15.106:0/3585237019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57b806e740 0x7f57b80b4850 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 >> 172.21.15.106:0/3585237019 conn(0x7f57b80b2600 msgr2=0x7f57b80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 shutdown_connections 2024-04-13T03:18:59.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.029+0000 7f57be715640 1 -- 172.21.15.106:0/3585237019 wait complete. 2024-04-13T03:18:59.041 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T03:18:59.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:18:59.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.242+0000 7fd2dbfaa640 1 Processor -- start 2024-04-13T03:18:59.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 -- start start 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd2d415ec20 0x7fd2d40b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d40ceb70 0x7fd2d40b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d40b6120 con 0x7fd2d41626b0 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40cde50 con 0x7fd2d40ceb70 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2dbfaa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40cd240 con 0x7fd2d415ec20 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.243+0000 7fd2d9545640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d40ceb70 0x7fd2d40b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d40ceb70 0x7fd2d40b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35160/0 (socket says 172.21.15.106:35160) 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 -- 172.21.15.106:0/2703844488 learned_addr learned my addr 172.21.15.106:0/2703844488 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9d46640 1 --2- 172.21.15.106:0/2703844488 >> v2:172.21.15.114:3300/0 conn(0x7fd2d415ec20 0x7fd2d40b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 -- 172.21.15.106:0/2703844488 >> v2:172.21.15.114:3300/0 conn(0x7fd2d415ec20 msgr2=0x7fd2d40b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 --2- 172.21.15.106:0/2703844488 >> v2:172.21.15.114:3300/0 conn(0x7fd2d415ec20 0x7fd2d40b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 -- 172.21.15.106:0/2703844488 >> v1:172.21.15.114:6789/0 conn(0x7fd2d41626b0 legacy=0x7fd2d40b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 -- 172.21.15.106:0/2703844488 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40cd520 con 0x7fd2d40ceb70 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.244+0000 7fd2d9545640 1 --2- 172.21.15.106:0/2703844488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d40ceb70 0x7fd2d40b5900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40d0080 tx=0x7fd2c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a2e0660dea5f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2d8d44640 1 -- 172.21.15.106:0/2703844488 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4035020 con 0x7fd2d40ceb70 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2d8d44640 1 -- 172.21.15.106:0/2703844488 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4031870 con 0x7fd2d40ceb70 2024-04-13T03:18:59.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2d8d44640 1 -- 172.21.15.106:0/2703844488 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4031c40 con 0x7fd2d40ceb70 2024-04-13T03:18:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/2703844488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d40ceb70 msgr2=0x7fd2d40b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/2703844488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d40ceb70 0x7fd2d40b5900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40d0080 tx=0x7fd2c402b910 comp rx=0 tx=0).stop 2024-04-13T03:18:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/2703844488 shutdown_connections 2024-04-13T03:18:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/2703844488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d40ceb70 0x7fd2d40b5900 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/2703844488 >> v2:172.21.15.114:3300/0 conn(0x7fd2d415ec20 0x7fd2d40b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/2703844488 >> 172.21.15.106:0/2703844488 conn(0x7fd2d40d2530 msgr2=0x7fd2d40d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/2703844488 shutdown_connections 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.245+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/2703844488 wait complete. 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.246+0000 7fd2dbfaa640 1 Processor -- start 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.246+0000 7fd2dbfaa640 1 -- start start 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2d41626b0 0x7fd2d40ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2d40c8650 0x7fd2d41e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40cced0 con 0x7fd2d415ec20 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40cdae0 con 0x7fd2d40c8650 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2dbfaa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40d14d0 con 0x7fd2d41626b0 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2d9d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2d9d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35162/0 (socket says 172.21.15.106:35162) 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2d9d46640 1 -- 172.21.15.106:0/3876508221 learned_addr learned my addr 172.21.15.106:0/3876508221 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2d9545640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2d41626b0 0x7fd2d40ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.247+0000 7fd2da547640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2d40c8650 0x7fd2d41e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 -- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2d41626b0 msgr2=0x7fd2d40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2d41626b0 0x7fd2d40ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 -- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2d40c8650 msgr2=0x7fd2d41e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2d40c8650 0x7fd2d41e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 -- 172.21.15.106:0/3876508221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40cadd0 con 0x7fd2d415ec20 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2d9d46640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0007ef0 tx=0x7fd2d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0014620 con 0x7fd2d415ec20 2024-04-13T03:18:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d0019070 con 0x7fd2d415ec20 2024-04-13T03:18:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0014a00 con 0x7fd2d415ec20 2024-04-13T03:18:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d40cb0b0 con 0x7fd2d415ec20 2024-04-13T03:18:59.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.248+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d40cb670 con 0x7fd2d415ec20 2024-04-13T03:18:59.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.249+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d4068a10 con 0x7fd2d415ec20 2024-04-13T03:18:59.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.249+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2d0014ba0 con 0x7fd2d415ec20 2024-04-13T03:18:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.250+0000 7fd2ca7fc640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 0x7fd2b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.250+0000 7fd2d9545640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 0x7fd2b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.250+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6216+0+0 (secure 0 0 0) 0x7fd2d008c550 con 0x7fd2d415ec20 2024-04-13T03:18:59.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.250+0000 7fd2d9545640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 0x7fd2b806edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4036000 tx=0x7fd2c403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.254+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2d005a130 con 0x7fd2d415ec20 2024-04-13T03:18:59.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.466+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2d40d3120 con 0x7fd2d415ec20 2024-04-13T03:18:59.467 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:18:59.466+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:18:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.468+0000 7fd2ca7fc640 1 -- 172.21.15.106:0/3876508221 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7fd2d005e430 con 0x7fd2d415ec20 2024-04-13T03:18:59.469 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 msgr2=0x7fd2b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 0x7fd2b806edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4036000 tx=0x7fd2c403c040 comp rx=0 tx=0).stop 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 msgr2=0x7fd2d4156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0007ef0 tx=0x7fd2d000e4b0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 shutdown_connections 2024-04-13T03:18:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd2b806c8f0 0x7fd2b806edb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd2d41626b0 0x7fd2d40ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2d40c8650 0x7fd2d41e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 --2- 172.21.15.106:0/3876508221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d415ec20 0x7fd2d4156390 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 >> 172.21.15.106:0/3876508221 conn(0x7fd2d40d2530 msgr2=0x7fd2d4162ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 shutdown_connections 2024-04-13T03:18:59.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.470+0000 7fd2dbfaa640 1 -- 172.21.15.106:0/3876508221 wait complete. 2024-04-13T03:18:59.475 INFO:tasks.ceph.mds.d.smithi114.stderr: -1155> 2024-04-13T03:18:28.711+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-13T03:18:59.475 INFO:tasks.ceph.mds.d.smithi114.stderr: -1152> 2024-04-13T03:18:28.711+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-13T03:18:59.475 INFO:tasks.ceph.mds.d.smithi114.stderr: -1113> 2024-04-13T03:18:28.712+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-13T03:18:59.475 INFO:tasks.ceph.mds.d.smithi114.stderr: -1110> 2024-04-13T03:18:28.712+0000 7ffaccde9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-13T03:18:59.475 INFO:tasks.ceph.mds.d.smithi114.stderr: -917> 2024-04-13T03:18:30.342+0000 7ffad0df1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:18:59.482 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-13T03:18:59.501 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:18:59.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 -- start start 2024-04-13T03:18:59.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 Processor -- start 2024-04-13T03:18:59.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 0x5592a67d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a5a0bc00 0x5592a67d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa780 con 0x5592a5a0ac00 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa600 con 0x5592a5a0bc00 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5592a67fa480 con 0x5592a5a0b800 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b31fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b31fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35178/0 (socket says 172.21.15.106:35178) 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.521+0000 7f56b31fc640 1 -- 172.21.15.106:0/3558596141 learned_addr learned my addr 172.21.15.106:0/3558596141 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b39fd640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a5a0bc00 0x5592a67d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 0x5592a67d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 -- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 msgr2=0x5592a67d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 0x5592a67d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 -- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a5a0bc00 msgr2=0x5592a67d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a5a0bc00 0x5592a67d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 -- 172.21.15.106:0/3558596141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a5a42780 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 0x5592a67d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x5592a683d0b0 tx=0x5592a679e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7decac0c0275a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b09f7640 1 -- 172.21.15.106:0/3558596141 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a409c0 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b09f7640 1 -- 172.21.15.106:0/3558596141 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592a5a36700 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b09f7640 1 -- 172.21.15.106:0/3558596141 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a404e0 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a5a423c0 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5592a5a43680 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b543b280 4 waiting for MDS map... 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b09f7640 1 -- 172.21.15.106:0/3558596141 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x5592a6770200 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b09f7640 1 -- 172.21.15.106:0/3558596141 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 78) ==== 2885+0+0 (secure 0 0 0) 0x5592a5a67180 con 0x5592a5a0ac00 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b543b280 4 Got MDS map 78 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.522+0000 7f56b543b280 10 main: main 2024-04-13T03:18:59.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 4 main: connecting to RADOS... 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 1 Processor -- start 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 1 -- start start 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.523+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b543b280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa300 con 0x5592a692a000 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa000 con 0x5592a692a400 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5592a5a30a80 con 0x5592a692a800 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35188/0 (socket says 172.21.15.106:35188) 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 -- 172.21.15.106:0/2466760695 learned_addr learned my addr 172.21.15.106:0/2466760695 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b29fb640 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b31fc640 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 -- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a800 msgr2=0x5592a67d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 -- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a400 msgr2=0x5592a67d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 -- 172.21.15.106:0/2466760695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a5a434a0 con 0x5592a692a000 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56b39fd640 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x5592a676e1b0 tx=0x5592a67f7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0194007d346a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56ae9f3640 1 -- 172.21.15.106:0/2466760695 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a409c0 con 0x5592a692a000 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56ae9f3640 1 -- 172.21.15.106:0/2466760695 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592a5a36e00 con 0x5592a692a000 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.524+0000 7f56ae9f3640 1 -- 172.21.15.106:0/2466760695 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a404e0 con 0x5592a692a000 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- 172.21.15.106:0/2466760695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 msgr2=0x5592a67d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x5592a676e1b0 tx=0x5592a67f7800 comp rx=0 tx=0).stop 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- 172.21.15.106:0/2466760695 shutdown_connections 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 --2- 172.21.15.106:0/2466760695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- 172.21.15.106:0/2466760695 >> 172.21.15.106:0/2466760695 conn(0x5592a67fdc00 msgr2=0x5592a5af7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- 172.21.15.106:0/2466760695 shutdown_connections 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- 172.21.15.106:0/2466760695 wait complete. 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 Processor -- start 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.525+0000 7f56b543b280 1 -- start start 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa300 con 0x5592a692a000 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5592a67fa000 con 0x5592a692a800 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5592a67fb500 con 0x5592a692a400 2024-04-13T03:18:59.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35190/0 (socket says 172.21.15.106:35190) 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 -- 172.21.15.106:0/3512143214 learned_addr learned my addr 172.21.15.106:0/3512143214 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 -- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a400 msgr2=0x5592a67d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a692a400 0x5592a67d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 -- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a800 msgr2=0x5592a67d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 -- 172.21.15.106:0/3512143214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a5a42000 con 0x5592a692a000 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a692a800 0x5592a67d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b39fd640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x5592a676f350 tx=0x5592a679e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56ad1f0640 1 -- 172.21.15.106:0/3512143214 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a40000 con 0x5592a692a000 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56ad1f0640 1 -- 172.21.15.106:0/3512143214 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592a5a36c40 con 0x5592a692a000 2024-04-13T03:18:59.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5592a5a42960 con 0x5592a692a000 2024-04-13T03:18:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56ad1f0640 1 -- 172.21.15.106:0/3512143214 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592a5a409c0 con 0x5592a692a000 2024-04-13T03:18:59.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a5a43860 con 0x5592a692a000 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.526+0000 7f56b543b280 4 main: resolving pool 15 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56ad1f0640 1 -- 172.21.15.106:0/3512143214 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5592a6944c00 con 0x5592a692a000 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56ad1f0640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5592a5d72000 0x5592a67d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56ad1f0640 1 -- 172.21.15.106:0/3512143214 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x5592a6931400 con 0x5592a692a000 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b543b280 4 main: creating IoCtx.. 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5592a5d72000 0x5592a67d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b543b280 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5592a692ac00 0x5592a67d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5592a692b000 con 0x5592a692ac00 2024-04-13T03:18:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5592a692ac00 0x5592a67d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b31fc640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5592a5d72000 0x5592a67d6100 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5592a683da10 tx=0x5592a679e780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.528+0000 7f56b29fb640 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5592a692ac00 0x5592a67d6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.530+0000 7f56b29fb640 1 -- 172.21.15.106:0/3512143214 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x5592a6788000 con 0x5592a692ac00 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "0": { 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "data": { 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "version": 5, 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "inotable": { 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "projected_free": [ 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "start": 1099511627779, 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: "len": 1099511627773 2024-04-13T03:18:59.539 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: "free": [ 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: "start": 1099511627779, 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: "len": 1099511627773 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: "result": 0 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:18:59.540 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:18:59.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.530+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5592a692ac00 msgr2=0x5592a67d6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.530+0000 7f56b543b280 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5592a692ac00 0x5592a67d6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5592a5d72000 msgr2=0x5592a67d6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5592a5d72000 0x5592a67d6100 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5592a683da10 tx=0x5592a679e780 comp rx=0 tx=0).stop 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 msgr2=0x5592a67d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3512143214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a692a000 0x5592a67d5080 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x5592a676f350 tx=0x5592a679e4e0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b29fb640 1 -- 172.21.15.106:0/3512143214 reap_dead start 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 shutdown_connections 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 >> 172.21.15.106:0/3512143214 conn(0x5592a67fdc00 msgr2=0x5592a5af7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 shutdown_connections 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3512143214 wait complete. 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 msgr2=0x5592a67d4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x5592a683d0b0 tx=0x5592a679e720 comp rx=0 tx=0).stop 2024-04-13T03:18:59.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 shutdown_connections 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5592a5a0b800 0x5592a67d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592a5a0bc00 0x5592a67d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 --2- 172.21.15.106:0/3558596141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592a5a0ac00 0x5592a67d4000 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 >> 172.21.15.106:0/3558596141 conn(0x5592a5a0b000 msgr2=0x5592a5af6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 shutdown_connections 2024-04-13T03:18:59.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.531+0000 7f56b543b280 1 -- 172.21.15.106:0/3558596141 wait complete. 2024-04-13T03:18:59.544 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.061988, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-13T03:18:59.544 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:18:59.545 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:18:59.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:18:59.688 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:18:59.688 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:18:59.688 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:18:59.688 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:18:59.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:18:59.704 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:18:59.704 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:18:59.705 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:18:59.721 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:18:59.721 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:18:59.721 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:18:59.721 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:18:59.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:18:59.761 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:18:59.761 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:18:59.761 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:18:59.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:18:59.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:18:59.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.900+0000 7f723b836640 1 Processor -- start 2024-04-13T03:18:59.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.900+0000 7f723b836640 1 -- start start 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f723b836640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 0x7f72340d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f723b836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72340b5b10 0x7f723415e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f723b836640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340d0b40 con 0x7f72340d06c0 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f723b836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f72340d1be0 con 0x7f72340b5b10 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f723b836640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f72340d0e20 con 0x7f72340ce580 2024-04-13T03:18:59.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f7239dd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72340b5b10 0x7f723415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f7239dd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72340b5b10 0x7f723415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35204/0 (socket says 172.21.15.106:35204) 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.901+0000 7f7239dd3640 1 -- 172.21.15.106:0/3410469498 learned_addr learned my addr 172.21.15.106:0/3410469498 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f7239dd3640 1 -- 172.21.15.106:0/3410469498 >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 msgr2=0x7f72340d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f72395d2640 1 --2- 172.21.15.106:0/3410469498 >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 0x7f72340d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f7239dd3640 1 --2- 172.21.15.106:0/3410469498 >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 0x7f72340d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f7239dd3640 1 -- 172.21.15.106:0/3410469498 >> v1:172.21.15.114:6789/0 conn(0x7f72340d06c0 legacy=0x7f72340b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f7239dd3640 1 -- 172.21.15.106:0/3410469498 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340d1100 con 0x7f72340b5b10 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f72395d2640 1 --2- 172.21.15.106:0/3410469498 >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 0x7f72340d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.902+0000 7f7239dd3640 1 --2- 172.21.15.106:0/3410469498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f723415e220 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f72340d0080 tx=0x7f722c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e005bd14b16ddd5a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f722bfff640 1 -- 172.21.15.106:0/3410469498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c02ca80 con 0x7f72340b5b10 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f722bfff640 1 -- 172.21.15.106:0/3410469498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c02cc20 con 0x7f72340b5b10 2024-04-13T03:18:59.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f722bfff640 1 -- 172.21.15.106:0/3410469498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c0343f0 con 0x7f72340b5b10 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 -- 172.21.15.106:0/3410469498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 msgr2=0x7f723415e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 --2- 172.21.15.106:0/3410469498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f723415e220 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f72340d0080 tx=0x7f722c02bee0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 -- 172.21.15.106:0/3410469498 shutdown_connections 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 --2- 172.21.15.106:0/3410469498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f723415e220 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 --2- 172.21.15.106:0/3410469498 >> v2:172.21.15.114:3300/0 conn(0x7f72340ce580 0x7f72340d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 -- 172.21.15.106:0/3410469498 >> 172.21.15.106:0/3410469498 conn(0x7f72340d25b0 msgr2=0x7f72340d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 -- 172.21.15.106:0/3410469498 shutdown_connections 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.903+0000 7f723b836640 1 -- 172.21.15.106:0/3410469498 wait complete. 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.904+0000 7f723b836640 1 Processor -- start 2024-04-13T03:18:59.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.904+0000 7f723b836640 1 -- start start 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72340ce580 0x7f72340b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72340d06c0 0x7f72340c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72340ce960 con 0x7f72340b5b10 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72340d16c0 con 0x7f72340d06c0 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f723b836640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f72340b7890 con 0x7f72340ce580 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35208/0 (socket says 172.21.15.106:35208) 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 -- 172.21.15.106:0/3288493592 learned_addr learned my addr 172.21.15.106:0/3288493592 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f7238dd1640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72340ce580 0x7f72340b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f7239dd3640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72340d06c0 0x7f72340c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 -- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72340ce580 msgr2=0x7f72340b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72340ce580 0x7f72340b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 -- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72340d06c0 msgr2=0x7f72340c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72340d06c0 0x7f72340c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.905+0000 7f72395d2640 1 -- 172.21.15.106:0/3288493592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7234157cb0 con 0x7f72340b5b10 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f72395d2640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f723000c820 tx=0x7f723000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230015070 con 0x7f72340b5b10 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7230007a30 con 0x7f72340b5b10 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230007e00 con 0x7f72340b5b10 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341f4b70 con 0x7f72340b5b10 2024-04-13T03:18:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.906+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341f50f0 con 0x7f72340b5b10 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.907+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7234074750 con 0x7f72340b5b10 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.907+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7230018070 con 0x7f72340b5b10 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.908+0000 7f7229ffb640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 0x7f721806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.908+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f723008bcf0 con 0x7f72340b5b10 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.908+0000 7f7238dd1640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 0x7f721806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:18:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.908+0000 7f7238dd1640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 0x7f721806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:18:59.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:18:59.912+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7230090070 con 0x7f72340b5b10 2024-04-13T03:19:00.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.125+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72340c57f0 con 0x7f72340b5b10 2024-04-13T03:19:00.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.126+0000 7f7229ffb640 1 -- 172.21.15.106:0/3288493592 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+4354 (secure 0 0 0) 0x7f7234157cb0 con 0x7f72340b5b10 2024-04-13T03:19:00.127 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:00.127 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":78,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:59.467638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:19:00.127 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 78 2024-04-13T03:19:00.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 msgr2=0x7f721806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 0x7f721806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224008040 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 msgr2=0x7f72340b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f723000c820 tx=0x7f723000cc20 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 shutdown_connections 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f721806c8f0 0x7f721806edb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f72340ce580 0x7f72340b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72340d06c0 0x7f72340c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 --2- 172.21.15.106:0/3288493592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72340b5b10 0x7f72340b25b0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 >> 172.21.15.106:0/3288493592 conn(0x7f72340d25b0 msgr2=0x7f72340c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 shutdown_connections 2024-04-13T03:19:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.128+0000 7f723b836640 1 -- 172.21.15.106:0/3288493592 wait complete. 2024-04-13T03:19:00.139 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:00.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.239+0000 7f9b3b0b7640 1 Processor -- start 2024-04-13T03:19:00.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b3b0b7640 1 -- start start 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b3b0b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b34160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b3b0b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340d2310 con 0x7f9b340cd2b0 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b3b0b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340b1f30 con 0x7f9b3415c7d0 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b3b0b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9b340b2210 con 0x7f9b340c60a0 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b33fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9b340cd2b0 0x7f9b340b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33676/0 (socket says 172.21.15.106:33676) 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.240+0000 7f9b33fff640 1 -- 172.21.15.106:0/3933178854 learned_addr learned my addr 172.21.15.106:0/3933178854 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b34160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002890368 0 0) 0x7f9b340d2310 con 0x7f9b340cd2b0 2024-04-13T03:19:00.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b24003520 con 0x7f9b340cd2b0 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 916327827 0 0) 0x7f9b24003520 con 0x7f9b340cd2b0 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 msgr2=0x7f9b34160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b34160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 >> v1:172.21.15.114:6789/0 conn(0x7f9b3415c7d0 legacy=0x7f9b340ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b340b24f0 con 0x7f9b340cd2b0 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9b20001670 con 0x7f9b340cd2b0 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 >> v1:172.21.15.106:6789/0 conn(0x7f9b340cd2b0 legacy=0x7f9b340b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24003f60 0x7f9b24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b24004d40 0x7f9b240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b340b2210 con 0x7f9b24004d40 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2400baa0 con 0x7f9b24003f60 2024-04-13T03:19:00.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2400bca0 con 0x7f9b340c60a0 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b24004d40 0x7f9b240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b38e53640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b33fff640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24003f60 0x7f9b24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 -- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 msgr2=0x7f9b240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 -- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24003f60 msgr2=0x7f9b24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24003f60 0x7f9b24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b39654640 1 -- 172.21.15.106:0/3933178854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b24003520 con 0x7f9b24004d40 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.241+0000 7f9b38e53640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b340c60a0 0x7f9b240039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.242+0000 7f9b39654640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b24004d40 0x7f9b240051c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009fc0 tx=0x7f9b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eb879b847b2848 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.242+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c039070 con 0x7f9b24004d40 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.242+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2c02cc20 con 0x7f9b24004d40 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.242+0000 7f9b337fe640 1 -- 172.21.15.106:0/3933178854 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c009c10 con 0x7f9b24004d40 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/3933178854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b24004d40 msgr2=0x7f9b240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/3933178854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b24004d40 0x7f9b240051c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009fc0 tx=0x7f9b24005830 comp rx=0 tx=0).stop 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b39654640 1 -- 172.21.15.106:0/3933178854 reap_dead start 2024-04-13T03:19:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/3933178854 shutdown_connections 2024-04-13T03:19:00.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/3933178854 >> 172.21.15.106:0/3933178854 conn(0x7f9b340d2550 msgr2=0x7f9b340d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/3933178854 shutdown_connections 2024-04-13T03:19:00.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.243+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/3933178854 wait complete. 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.244+0000 7f9b3b0b7640 1 Processor -- start 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.244+0000 7f9b3b0b7640 1 -- start start 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24004d40 0x7f9b3415db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b341f28d0 0x7f9b341553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3415cbb0 con 0x7f9b341558e0 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b34160dd0 con 0x7f9b24004d40 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b3b0b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9b341f30b0 con 0x7f9b341f28d0 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35224/0 (socket says 172.21.15.106:35224) 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 -- 172.21.15.106:0/4065791545 learned_addr learned my addr 172.21.15.106:0/4065791545 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b38e53640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24004d40 0x7f9b3415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b33fff640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b341f28d0 0x7f9b341553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 -- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b341f28d0 msgr2=0x7f9b341553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b341f28d0 0x7f9b341553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 -- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24004d40 msgr2=0x7f9b3415db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24004d40 0x7f9b3415db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 -- 172.21.15.106:0/4065791545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b341f3390 con 0x7f9b341558e0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.245+0000 7f9b39654640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c002230 tx=0x7f9b2c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.246+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c039030 con 0x7f9b341558e0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.246+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2c043070 con 0x7f9b341558e0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.246+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c009c10 con 0x7f9b341558e0 2024-04-13T03:19:00.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.246+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b341f3670 con 0x7f9b341558e0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.246+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b341f3c30 con 0x7f9b341558e0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.247+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b341f3e30 con 0x7f9b341558e0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.247+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b2c005d10 con 0x7f9b341558e0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.247+0000 7f9b317fa640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.248+0000 7f9b38e53640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.248+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f9b2c0adf10 con 0x7f9b341558e0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.248+0000 7f9b38e53640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 0x7f9b1806edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b28007fd0 tx=0x7f9b28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.252+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b341f3e30 con 0x7f9b341558e0 2024-04-13T03:19:00.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.429+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b34069120 con 0x7f9b341558e0 2024-04-13T03:19:00.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.430+0000 7f9b317fa640 1 -- 172.21.15.106:0/4065791545 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17233 (secure 0 0 0) 0x7f9b2c07baa0 con 0x7f9b341558e0 2024-04-13T03:19:00.431 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:00.431 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":111,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:18:59.467477+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6833/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6832/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6835/2645971931":"2024-04-14T03:18:23.121696+0000","172.21.15.114:6834/2645971931":"2024-04-14T03:18:23.121696+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:00.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 msgr2=0x7f9b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 0x7f9b1806edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b28007fd0 tx=0x7f9b28009040 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 msgr2=0x7f9b341f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c002230 tx=0x7f9b2c009750 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 shutdown_connections 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9b1806c8f0 0x7f9b1806edb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b341f28d0 0x7f9b341553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b341558e0 0x7f9b341f4150 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 --2- 172.21.15.106:0/4065791545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b24004d40 0x7f9b3415db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 >> 172.21.15.106:0/4065791545 conn(0x7f9b340d2550 msgr2=0x7f9b340c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 shutdown_connections 2024-04-13T03:19:00.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.432+0000 7f9b3b0b7640 1 -- 172.21.15.106:0/4065791545 wait complete. 2024-04-13T03:19:00.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:17:31.965063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:17:32.694523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:00.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.549+0000 7fc796436640 1 Processor -- start 2024-04-13T03:19:00.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.549+0000 7fc796436640 1 -- start start 2024-04-13T03:19:00.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc796436640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7901592b0 0x7fc79015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc796436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7900ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc796436640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79006d180 con 0x7fc79006e740 2024-04-13T03:19:00.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc796436640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7900ce180 con 0x7fc7901592b0 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc796436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc79006ec90 con 0x7fc79015c7a0 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc79006e740 0x7fc79015c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33692/0 (socket says 172.21.15.106:33692) 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78f7fe640 1 -- 172.21.15.106:0/836509125 learned_addr learned my addr 172.21.15.106:0/836509125 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758235401 0 0) 0x7fc79006d180 con 0x7fc79006e740 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc778003520 con 0x7fc79006e740 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78ffff640 1 --2- 172.21.15.106:0/836509125 >> v2:172.21.15.114:3300/0 conn(0x7fc7901592b0 0x7fc79015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc7949d3640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7900ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2136037232 0 0) 0x7fc778003520 con 0x7fc79006e740 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 msgr2=0x7fc7900ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7900ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 >> v2:172.21.15.114:3300/0 conn(0x7fc7901592b0 msgr2=0x7fc79015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 --2- 172.21.15.106:0/836509125 >> v2:172.21.15.114:3300/0 conn(0x7fc7901592b0 0x7fc79015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79006ef70 con 0x7fc79006e740 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc784001670 con 0x7fc79006e740 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 >> v1:172.21.15.106:6789/0 conn(0x7fc79006e740 legacy=0x7fc79015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7900ce180 con 0x7fc7780039b0 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc79006ec90 con 0x7fc778004800 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc77800baa0 con 0x7fc79015c7a0 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78ffff640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc78f7fe640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.550+0000 7fc7949d3640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 -- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 msgr2=0x7fc7780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc79015c7a0 0x7fc7780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 -- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc778004800 msgr2=0x7fc778004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 -- 172.21.15.106:0/836509125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc778003520 con 0x7fc7780039b0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.551+0000 7fc78ffff640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b5c0 tx=0x7fc778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e926b1f83408b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78003a070 con 0x7fc7780039b0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780034d20 con 0x7fc7780039b0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc78effd640 1 -- 172.21.15.106:0/836509125 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000b600 con 0x7fc7780039b0 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 -- 172.21.15.106:0/836509125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 msgr2=0x7fc778003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 --2- 172.21.15.106:0/836509125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b5c0 tx=0x7fc778005830 comp rx=0 tx=0).stop 2024-04-13T03:19:00.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc7949d3640 1 -- 172.21.15.106:0/836509125 reap_dead start 2024-04-13T03:19:00.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 -- 172.21.15.106:0/836509125 shutdown_connections 2024-04-13T03:19:00.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 -- 172.21.15.106:0/836509125 >> 172.21.15.106:0/836509125 conn(0x7fc7900b2600 msgr2=0x7fc7900d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 -- 172.21.15.106:0/836509125 shutdown_connections 2024-04-13T03:19:00.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.552+0000 7fc796436640 1 -- 172.21.15.106:0/836509125 wait complete. 2024-04-13T03:19:00.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.553+0000 7fc796436640 1 Processor -- start 2024-04-13T03:19:00.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.553+0000 7fc796436640 1 -- start start 2024-04-13T03:19:00.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc790157390 0x7fc7900b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc79014a980 0x7fc79014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc79006b8e0 con 0x7fc7780039b0 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc79006f530 con 0x7fc79014a980 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc796436640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc79006ba60 con 0x7fc790157390 2024-04-13T03:19:00.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc78ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc78ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35234/0 (socket says 172.21.15.106:35234) 2024-04-13T03:19:00.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc78ffff640 1 -- 172.21.15.106:0/3490676837 learned_addr learned my addr 172.21.15.106:0/3490676837 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:00.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc78f7fe640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc790157390 0x7fc7900b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.554+0000 7fc7949d3640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc79014a980 0x7fc79014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 -- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc790157390 msgr2=0x7fc7900b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc790157390 0x7fc7900b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 -- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc79014a980 msgr2=0x7fc79014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc79014a980 0x7fc79014ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 -- 172.21.15.106:0/3490676837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900a9560 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78ffff640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc78000baf0 tx=0x7fc780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78004d030 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780034b20 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7900a9840 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000b600 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.555+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7900a9e00 con 0x7fc7780039b0 2024-04-13T03:19:00.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.556+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7900aa240 con 0x7fc7780039b0 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.556+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc780050070 con 0x7fc7780039b0 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.557+0000 7fc78cff9640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 0x7fc77c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.557+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7fc7800b66c0 con 0x7fc7780039b0 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.557+0000 7fc78f7fe640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 0x7fc77c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.557+0000 7fc78f7fe640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 0x7fc77c06edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc7900b1960 tx=0x7fc784007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.561+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7800841c0 con 0x7fc7780039b0 2024-04-13T03:19:00.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.734+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc790068a10 con 0x7fc7780039b0 2024-04-13T03:19:00.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.735+0000 7fc78cff9640 1 -- 172.21.15.106:0/3490676837 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fc7800884c0 con 0x7fc7780039b0 2024-04-13T03:19:00.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 msgr2=0x7fc77c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 0x7fc77c06edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc7900b1960 tx=0x7fc784007970 comp rx=0 tx=0).stop 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 msgr2=0x7fc7901584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc78000baf0 tx=0x7fc780004730 comp rx=0 tx=0).stop 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 shutdown_connections 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc77c06c8f0 0x7fc77c06edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc790157390 0x7fc7900b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc79014a980 0x7fc79014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 --2- 172.21.15.106:0/3490676837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7780039b0 0x7fc7901584d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 >> 172.21.15.106:0/3490676837 conn(0x7fc7900b2600 msgr2=0x7fc7900b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 shutdown_connections 2024-04-13T03:19:00.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.737+0000 7fc796436640 1 -- 172.21.15.106:0/3490676837 wait complete. 2024-04-13T03:19:00.748 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:00.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.850+0000 7fdba011e640 1 Processor -- start 2024-04-13T03:19:00.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.850+0000 7fdba011e640 1 -- start start 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdba011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9806bff0 0x7fdb9806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdba011e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb980d07c0 0x7fdb9806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdba011e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb980cef80 con 0x7fdb9815fb50 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdba011e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdb980cf260 con 0x7fdb980d07c0 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdba011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980cf540 con 0x7fdb9806bff0 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9e6bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb980d07c0 0x7fdb9806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9deba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdb9815fb50 0x7fdb9806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33706/0 (socket says 172.21.15.106:33706) 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9deba640 1 -- 172.21.15.106:0/3781770331 learned_addr learned my addr 172.21.15.106:0/3781770331 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9d6b9640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9806bff0 0x7fdb9806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9e6bb640 1 -- 172.21.15.106:0/3781770331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9806bff0 msgr2=0x7fdb9806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:00.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9e6bb640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9806bff0 0x7fdb9806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9e6bb640 1 -- 172.21.15.106:0/3781770331 >> v1:172.21.15.106:6789/0 conn(0x7fdb9815fb50 legacy=0x7fdb9806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9ceb8640 1 -- 172.21.15.106:0/3781770331 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988044659 0 0) 0x7fdb980cef80 con 0x7fdb9815fb50 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.851+0000 7fdb9e6bb640 1 -- 172.21.15.106:0/3781770331 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980cf820 con 0x7fdb980d07c0 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.852+0000 7fdb9e6bb640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb980d07c0 0x7fdb9806ebf0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009d90 tx=0x7fdb9002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df16bedd2314e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.852+0000 7fdb9ceb8640 1 -- 172.21.15.106:0/3781770331 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90039070 con 0x7fdb980d07c0 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.852+0000 7fdb9ceb8640 1 -- 172.21.15.106:0/3781770331 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9002cbc0 con 0x7fdb980d07c0 2024-04-13T03:19:00.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.852+0000 7fdb9ceb8640 1 -- 172.21.15.106:0/3781770331 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90034440 con 0x7fdb980d07c0 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 -- 172.21.15.106:0/3781770331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb980d07c0 msgr2=0x7fdb9806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb980d07c0 0x7fdb9806ebf0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009d90 tx=0x7fdb9002be10 comp rx=0 tx=0).stop 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 -- 172.21.15.106:0/3781770331 shutdown_connections 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb980d07c0 0x7fdb9806ebf0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 --2- 172.21.15.106:0/3781770331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9806bff0 0x7fdb9806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 -- 172.21.15.106:0/3781770331 >> 172.21.15.106:0/3781770331 conn(0x7fdb980cd6a0 msgr2=0x7fdb980cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 -- 172.21.15.106:0/3781770331 shutdown_connections 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.853+0000 7fdba011e640 1 -- 172.21.15.106:0/3781770331 wait complete. 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.854+0000 7fdba011e640 1 Processor -- start 2024-04-13T03:19:00.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.854+0000 7fdba011e640 1 -- start start 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb980d07c0 0x7fdb98157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9815fb50 0x7fdb980b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980ced20 con 0x7fdb9806bff0 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980ceba0 con 0x7fdb980d07c0 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9815ef10 con 0x7fdb9815fb50 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35252/0 (socket says 172.21.15.106:35252) 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 -- 172.21.15.106:0/2965606048 learned_addr learned my addr 172.21.15.106:0/2965606048 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9e6bb640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9815fb50 0x7fdb980b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9d6b9640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb980d07c0 0x7fdb98157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 -- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9815fb50 msgr2=0x7fdb980b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9815fb50 0x7fdb980b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 -- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb980d07c0 msgr2=0x7fdb98157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb980d07c0 0x7fdb98157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 -- 172.21.15.106:0/2965606048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb98156ba0 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb9deba640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c430 tx=0x7fdb9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940074e0 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940047e0 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb98156e80 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94004bd0 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.855+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb980b5d20 con 0x7fdb9806bff0 2024-04-13T03:19:00.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.856+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb980b5870 con 0x7fdb9806bff0 2024-04-13T03:19:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.857+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb94011070 con 0x7fdb9806bff0 2024-04-13T03:19:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.857+0000 7fdb8e7fc640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 0x7fdb7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.857+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7fdb9408ba80 con 0x7fdb9806bff0 2024-04-13T03:19:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.857+0000 7fdb9d6b9640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 0x7fdb7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:00.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.858+0000 7fdb9d6b9640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 0x7fdb7c06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdb9815eee0 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:00.862+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb94059c60 con 0x7fdb9806bff0 2024-04-13T03:19:01.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.036+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb98068a10 con 0x7fdb9806bff0 2024-04-13T03:19:01.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.036+0000 7fdb8e7fc640 1 -- 172.21.15.106:0/2965606048 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) ==== 104+0+11 (secure 0 0 0) 0x7fdb94059560 con 0x7fdb9806bff0 2024-04-13T03:19:01.037 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 msgr2=0x7fdb7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 0x7fdb7c06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdb9815eee0 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 msgr2=0x7fdb98153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c430 tx=0x7fdb9400c8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 shutdown_connections 2024-04-13T03:19:01.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb7c06c8f0 0x7fdb7c06edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb9815fb50 0x7fdb980b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb980d07c0 0x7fdb98157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 --2- 172.21.15.106:0/2965606048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb9806bff0 0x7fdb98153ae0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 >> 172.21.15.106:0/2965606048 conn(0x7fdb980cd6a0 msgr2=0x7fdb980b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 shutdown_connections 2024-04-13T03:19:01.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.038+0000 7fdba011e640 1 -- 172.21.15.106:0/2965606048 wait complete. 2024-04-13T03:19:01.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:01.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.152+0000 7f2b7a420640 1 Processor -- start 2024-04-13T03:19:01.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.152+0000 7f2b7a420640 1 -- start start 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b7a420640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2b740b5390 0x7f2b7415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b7a420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b7406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b7a420640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b740b3690 con 0x7f2b7415b4e0 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b7a420640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740b3970 con 0x7f2b740b5390 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b7a420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740b3c50 con 0x7f2b740b4730 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b73fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2b7415b4e0 0x7f2b740b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33716/0 (socket says 172.21.15.106:33716) 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b73fff640 1 -- 172.21.15.106:0/1108570453 learned_addr learned my addr 172.21.15.106:0/1108570453 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209191385 0 0) 0x7f2b740b3690 con 0x7f2b7415b4e0 2024-04-13T03:19:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b5c003520 con 0x7f2b7415b4e0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b789bd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b7406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.153+0000 7f2b737fe640 1 --2- 172.21.15.106:0/1108570453 >> v2:172.21.15.114:3300/0 conn(0x7f2b740b5390 0x7f2b7415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2199816115 0 0) 0x7f2b5c003520 con 0x7f2b7415b4e0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 msgr2=0x7f2b7406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b7406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 >> v2:172.21.15.114:3300/0 conn(0x7f2b740b5390 msgr2=0x7f2b7415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 --2- 172.21.15.106:0/1108570453 >> v2:172.21.15.114:3300/0 conn(0x7f2b740b5390 0x7f2b7415fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740b3f30 con 0x7f2b7415b4e0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2b64001670 con 0x7f2b7415b4e0 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 >> v1:172.21.15.106:6789/0 conn(0x7f2b7415b4e0 legacy=0x7f2b740b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b5c0039b0 0x7f2b5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740b3970 con 0x7f2b5c004d40 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740b3c50 con 0x7f2b5c0039b0 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5c00b9c0 con 0x7f2b740b4730 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b73fff640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b5c0039b0 0x7f2b5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b737fe640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 -- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 msgr2=0x7f2b5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740b4730 0x7f2b5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 -- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b5c0039b0 msgr2=0x7f2b5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b5c0039b0 0x7f2b5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.154+0000 7f2b789bd640 1 -- 172.21.15.106:0/1108570453 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c003520 con 0x7f2b5c004d40 2024-04-13T03:19:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.155+0000 7f2b789bd640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e170 tx=0x7f2b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c15bcbe5df3c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.155+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c039070 con 0x7f2b5c004d40 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.155+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c02cbf0 con 0x7f2b5c004d40 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.155+0000 7f2b72ffd640 1 -- 172.21.15.106:0/1108570453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c009c10 con 0x7f2b5c004d40 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 -- 172.21.15.106:0/1108570453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c004d40 msgr2=0x7f2b5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 --2- 172.21.15.106:0/1108570453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e170 tx=0x7f2b5c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b789bd640 1 -- 172.21.15.106:0/1108570453 reap_dead start 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 -- 172.21.15.106:0/1108570453 shutdown_connections 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 -- 172.21.15.106:0/1108570453 >> 172.21.15.106:0/1108570453 conn(0x7f2b740b2600 msgr2=0x7f2b740d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 -- 172.21.15.106:0/1108570453 shutdown_connections 2024-04-13T03:19:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.156+0000 7f2b7a420640 1 -- 172.21.15.106:0/1108570453 wait complete. 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.157+0000 7f2b7a420640 1 Processor -- start 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.157+0000 7f2b7a420640 1 -- start start 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740cc850 0x7f2b74145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b740ccc30 0x7f2b7415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7406d0c0 con 0x7f2b74153ae0 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740b32d0 con 0x7f2b740ccc30 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7415be00 con 0x7f2b740cc850 2024-04-13T03:19:01.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35270/0 (socket says 172.21.15.106:35270) 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 -- 172.21.15.106:0/4253905499 learned_addr learned my addr 172.21.15.106:0/4253905499 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b73fff640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740cc850 0x7f2b74145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 -- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740cc850 msgr2=0x7f2b74145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740cc850 0x7f2b74145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b737fe640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b740ccc30 0x7f2b7415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 -- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b740ccc30 msgr2=0x7f2b7415b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b740ccc30 0x7f2b7415b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 -- 172.21.15.106:0/4253905499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7415c030 con 0x7f2b74153ae0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b789bd640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e170 tx=0x7f2b6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c044070 con 0x7f2b74153ae0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c03a040 con 0x7f2b74153ae0 2024-04-13T03:19:01.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c039030 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.158+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7415c310 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.159+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7415c8d0 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.160+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b74068a10 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.160+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b6c047070 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.160+0000 7f2b70ff9640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.161+0000 7f2b73fff640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.161+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f2b6c0ad730 con 0x7f2b74153ae0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.161+0000 7f2b73fff640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 0x7f2b6006edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2b64004840 tx=0x7f2b64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.165+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b6c07b2c0 con 0x7f2b74153ae0 2024-04-13T03:19:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.339+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b740af9c0 con 0x7f2b74153ae0 2024-04-13T03:19:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.340+0000 7f2b70ff9640 1 -- 172.21.15.106:0/4253905499 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) ==== 100+0+11 (secure 0 0 0) 0x7f2b6c07f5c0 con 0x7f2b74153ae0 2024-04-13T03:19:01.341 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 msgr2=0x7f2b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 0x7f2b6006edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2b64004840 tx=0x7f2b64009040 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 msgr2=0x7f2b74153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e170 tx=0x7f2b6c009750 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 shutdown_connections 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b74153ae0 0x7f2b74153f60 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b740ccc30 0x7f2b7415b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 --2- 172.21.15.106:0/4253905499 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2b740cc850 0x7f2b74145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 >> 172.21.15.106:0/4253905499 conn(0x7f2b740b2600 msgr2=0x7f2b740b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 shutdown_connections 2024-04-13T03:19:01.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.342+0000 7f2b7a420640 1 -- 172.21.15.106:0/4253905499 wait complete. 2024-04-13T03:19:01.353 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:01.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.454+0000 7f317535e640 1 Processor -- start 2024-04-13T03:19:01.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.454+0000 7f317535e640 1 -- start start 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f317535e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f317006fe90 0x7f31701626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f317535e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700ce6f0 con 0x7f317015ebf0 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f317535e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317015f420 con 0x7f31700ce1f0 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f317535e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f317015fa70 con 0x7f317006fe90 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f3167fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f317006fe90 0x7f31701626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f316effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f317015ebf0 0x7f317006f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33726/0 (socket says 172.21.15.106:33726) 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f316effd640 1 -- 172.21.15.106:0/2887000352 learned_addr learned my addr 172.21.15.106:0/2887000352 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f3167fff640 1 -- 172.21.15.106:0/2887000352 >> v1:172.21.15.114:6789/0 conn(0x7f31700ce1f0 legacy=0x7f3170162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f3167fff640 1 -- 172.21.15.106:0/2887000352 >> v1:172.21.15.106:6789/0 conn(0x7f317015ebf0 legacy=0x7f317006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f3167fff640 1 -- 172.21.15.106:0/2887000352 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317015f700 con 0x7f317006fe90 2024-04-13T03:19:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.455+0000 7f316e7fc640 1 -- 172.21.15.106:0/2887000352 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602429561 0 0) 0x7f31700ce6f0 con 0x7f317015ebf0 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f3167fff640 1 --2- 172.21.15.106:0/2887000352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317006fe90 0x7f31701626a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f31580097a0 tx=0x7f315802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f77020bfe752a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f316e7fc640 1 -- 172.21.15.106:0/2887000352 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158035020 con 0x7f317006fe90 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f316e7fc640 1 -- 172.21.15.106:0/2887000352 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3158031830 con 0x7f317006fe90 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f316e7fc640 1 -- 172.21.15.106:0/2887000352 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158031c00 con 0x7f317006fe90 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f317535e640 1 -- 172.21.15.106:0/2887000352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317006fe90 msgr2=0x7f31701626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.456+0000 7f317535e640 1 --2- 172.21.15.106:0/2887000352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317006fe90 0x7f31701626a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f31580097a0 tx=0x7f315802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.457+0000 7f317535e640 1 -- 172.21.15.106:0/2887000352 shutdown_connections 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.457+0000 7f317535e640 1 --2- 172.21.15.106:0/2887000352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317006fe90 0x7f31701626a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.457+0000 7f317535e640 1 -- 172.21.15.106:0/2887000352 >> 172.21.15.106:0/2887000352 conn(0x7f31700cd6d0 msgr2=0x7f31700cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.457+0000 7f317535e640 1 -- 172.21.15.106:0/2887000352 shutdown_connections 2024-04-13T03:19:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.457+0000 7f317535e640 1 -- 172.21.15.106:0/2887000352 wait complete. 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 Processor -- start 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 -- start start 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f317006fe90 0x7f31701f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f317015ebf0 0x7f31700adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31700b0650 con 0x7f31700ce1f0 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f31700b04d0 con 0x7f317015ebf0 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f317535e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f317015efd0 con 0x7f317006fe90 2024-04-13T03:19:01.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.458+0000 7f3167fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35298/0 (socket says 172.21.15.106:35298) 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 -- 172.21.15.106:0/1227630135 learned_addr learned my addr 172.21.15.106:0/1227630135 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f316effd640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f317006fe90 0x7f31701f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f316f7fe640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f317015ebf0 0x7f31700adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 -- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f317006fe90 msgr2=0x7f31701f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f317006fe90 0x7f31701f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 -- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f317015ebf0 msgr2=0x7f31700adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f317015ebf0 0x7f31700adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 -- 172.21.15.106:0/1227630135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31700b1310 con 0x7f31700ce1f0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f3167fff640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f31580098c0 tx=0x7f31580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315803ba80 con 0x7f31700ce1f0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315803bc20 con 0x7f31700ce1f0 2024-04-13T03:19:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31700b15f0 con 0x7f31700ce1f0 2024-04-13T03:19:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31580453f0 con 0x7f31700ce1f0 2024-04-13T03:19:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.459+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3170160380 con 0x7f31700ce1f0 2024-04-13T03:19:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.460+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31700b1b10 con 0x7f31700ce1f0 2024-04-13T03:19:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.461+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f315804c060 con 0x7f31700ce1f0 2024-04-13T03:19:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.461+0000 7f31677fe640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 0x7f315006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.461+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f31580adad0 con 0x7f31700ce1f0 2024-04-13T03:19:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.461+0000 7f316effd640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 0x7f315006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.461+0000 7f316effd640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 0x7f315006edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3160000f60 tx=0x7f3160007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.465+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31700b1b10 con 0x7f31700ce1f0 2024-04-13T03:19:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.678+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3170068a10 con 0x7f31700ce1f0 2024-04-13T03:19:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.679+0000 7f31677fe640 1 -- 172.21.15.106:0/1227630135 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5182 (secure 0 0 0) 0x7f3158040070 con 0x7f31700ce1f0 2024-04-13T03:19:01.680 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:01.680 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3179085184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3179085184},{"type":"v1","addr":"172.21.15.114:6833","nonce":3179085184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":78,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:59.467638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:19:01.680 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 79 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 msgr2=0x7f315006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 0x7f315006edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3160000f60 tx=0x7f3160007c00 comp rx=0 tx=0).stop 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 msgr2=0x7f31701585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f31580098c0 tx=0x7f31580046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 shutdown_connections 2024-04-13T03:19:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f315006c8f0 0x7f315006edb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f317015ebf0 0x7f31700adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31700ce1f0 0x7f31701585c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 --2- 172.21.15.106:0/1227630135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f317006fe90 0x7f31701f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 >> 172.21.15.106:0/1227630135 conn(0x7f31700cd6d0 msgr2=0x7f3170159210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 shutdown_connections 2024-04-13T03:19:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.681+0000 7f317535e640 1 -- 172.21.15.106:0/1227630135 wait complete. 2024-04-13T03:19:01.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:01.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 Processor -- start 2024-04-13T03:19:01.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 -- start start 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24c80d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80d3530 con 0x7f24c81622d0 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80d0b60 con 0x7f24c80b8080 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.791+0000 7f24ceb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24c81626b0 con 0x7f24c80d1c00 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24cc8ba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f24c81622d0 0x7f24c80b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33736/0 (socket says 172.21.15.106:33736) 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24cc8ba640 1 -- 172.21.15.106:0/3316529840 learned_addr learned my addr 172.21.15.106:0/3316529840 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c7fff640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24c80d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906509546 0 0) 0x7f24c80d3530 con 0x7f24c81622d0 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24b8003520 con 0x7f24c81622d0 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398482784 0 0) 0x7f24c80d0b60 con 0x7f24c80b8080 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24c80d3530 con 0x7f24c80b8080 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4069421990 0 0) 0x7f24b8003520 con 0x7f24c81622d0 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 msgr2=0x7f24c80d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24c80d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 >> v1:172.21.15.114:6789/0 conn(0x7f24c80b8080 legacy=0x7f24c81688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80d0e40 con 0x7f24c81622d0 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f24bc001670 con 0x7f24c81622d0 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 >> v1:172.21.15.106:6789/0 conn(0x7f24c81622d0 legacy=0x7f24c80b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24b8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8004d40 0x7f24b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24b800bad0 con 0x7f24b8004120 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24b800bc80 con 0x7f24b8004d40 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24b800be80 con 0x7f24c80d1c00 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c7fff640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24cc8ba640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24b8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24cd0bb640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8004d40 0x7f24b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c7fff640 1 -- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 msgr2=0x7f24b8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.792+0000 7f24c7fff640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c80d1c00 0x7f24b8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c7fff640 1 -- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8004d40 msgr2=0x7f24b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c7fff640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8004d40 0x7f24b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c7fff640 1 -- 172.21.15.106:0/3316529840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8003520 con 0x7f24b8004120 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c7fff640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24b8004800 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ac0 tx=0x7f24b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968c136b3270058b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4039070 con 0x7f24b8004120 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b402c600 con 0x7f24b8004120 2024-04-13T03:19:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.793+0000 7f24c77fe640 1 -- 172.21.15.106:0/3316529840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4009690 con 0x7f24b8004120 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 -- 172.21.15.106:0/3316529840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 msgr2=0x7f24b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/3316529840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24b8004800 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ac0 tx=0x7f24b8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24cd0bb640 1 -- 172.21.15.106:0/3316529840 reap_dead start 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 -- 172.21.15.106:0/3316529840 shutdown_connections 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 -- 172.21.15.106:0/3316529840 >> 172.21.15.106:0/3316529840 conn(0x7f24c80d25d0 msgr2=0x7f24c80d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 -- 172.21.15.106:0/3316529840 shutdown_connections 2024-04-13T03:19:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.794+0000 7f24ceb1e640 1 -- 172.21.15.106:0/3316529840 wait complete. 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.795+0000 7f24ceb1e640 1 Processor -- start 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.795+0000 7f24ceb1e640 1 -- start start 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c8158b80 0x7f24c8159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c8145a50 0x7f24c8145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d16e0 con 0x7f24b8004120 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d1400 con 0x7f24c8158b80 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24ceb1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80cc6f0 con 0x7f24c8145a50 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35312/0 (socket says 172.21.15.106:35312) 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 -- 172.21.15.106:0/119885502 learned_addr learned my addr 172.21.15.106:0/119885502 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24c7fff640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c8158b80 0x7f24c8159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cd0bb640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c8145a50 0x7f24c8145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 -- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c8145a50 msgr2=0x7f24c8145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c8145a50 0x7f24c8145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 -- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c8158b80 msgr2=0x7f24c8159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c8158b80 0x7f24c8159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 -- 172.21.15.106:0/119885502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c81f4640 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.796+0000 7f24cc8ba640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007320 tx=0x7f24bc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.797+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc01a070 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.797+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc00f8e0 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.797+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc00fc90 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.797+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80c9a80 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.797+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80ca040 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.798+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80692f0 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.798+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24bc01d360 con 0x7f24b8004120 2024-04-13T03:19:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.799+0000 7f24c57fa640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 0x7f24ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.799+0000 7f24c7fff640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 0x7f24ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.799+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f24bc08b5e0 con 0x7f24b8004120 2024-04-13T03:19:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.799+0000 7f24c7fff640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 0x7f24ac06ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24b4036000 tx=0x7f24b4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:01.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.803+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24c80692f0 con 0x7f24b8004120 2024-04-13T03:19:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.978+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24c8068a10 con 0x7f24b8004120 2024-04-13T03:19:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.979+0000 7f24c57fa640 1 -- 172.21.15.106:0/119885502 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17233 (secure 0 0 0) 0x7f24bc0597c0 con 0x7f24b8004120 2024-04-13T03:19:01.980 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:01.980 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":111,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:18:59.467477+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6833/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6832/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6835/2645971931":"2024-04-14T03:18:23.121696+0000","172.21.15.114:6834/2645971931":"2024-04-14T03:18:23.121696+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 msgr2=0x7f24ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 0x7f24ac06ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24b4036000 tx=0x7f24b4030000 comp rx=0 tx=0).stop 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 msgr2=0x7f24c80cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007320 tx=0x7f24bc0077e0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 shutdown_connections 2024-04-13T03:19:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f24ac06c920 0x7f24ac06ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f24c8145a50 0x7f24c8145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24c8158b80 0x7f24c8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 --2- 172.21.15.106:0/119885502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24b8004120 0x7f24c80cab20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 >> 172.21.15.106:0/119885502 conn(0x7f24c80d25d0 msgr2=0x7f24c80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 shutdown_connections 2024-04-13T03:19:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:01.981+0000 7f24ceb1e640 1 -- 172.21.15.106:0/119885502 wait complete. 2024-04-13T03:19:01.993 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:19:01.994 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.092+0000 7fe4693bb640 1 Processor -- start 2024-04-13T03:19:02.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.092+0000 7fe4693bb640 1 -- start start 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4693bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe46415e9c0 0x7fe4640ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4693bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4640b68d0 0x7fe4640d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4693bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46415eda0 con 0x7fe4641624b0 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4693bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe4640d27f0 con 0x7fe46415e9c0 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4693bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe464162c20 con 0x7fe4640b68d0 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe46415e9c0 0x7fe4640ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe46415e9c0 0x7fe4640ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35320/0 (socket says 172.21.15.106:35320) 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 -- 172.21.15.106:0/502039305 learned_addr learned my addr 172.21.15.106:0/502039305 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe4637fe640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4640b68d0 0x7fe4640d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 -- 172.21.15.106:0/502039305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4640b68d0 msgr2=0x7fe4640d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4640b68d0 0x7fe4640d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 -- 172.21.15.106:0/502039305 >> v1:172.21.15.114:6789/0 conn(0x7fe4641624b0 legacy=0x7fe4640b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.093+0000 7fe462ffd640 1 -- 172.21.15.106:0/502039305 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe464162f00 con 0x7fe46415e9c0 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.094+0000 7fe462ffd640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ba290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe454009870 tx=0x7fe45402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f5c6c30bc493c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.094+0000 7fe4627fc640 1 -- 172.21.15.106:0/502039305 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454035020 con 0x7fe46415e9c0 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.094+0000 7fe4627fc640 1 -- 172.21.15.106:0/502039305 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe454031640 con 0x7fe46415e9c0 2024-04-13T03:19:02.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.094+0000 7fe4627fc640 1 -- 172.21.15.106:0/502039305 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45402d040 con 0x7fe46415e9c0 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 -- 172.21.15.106:0/502039305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 msgr2=0x7fe4640ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ba290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe454009870 tx=0x7fe45402b8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 -- 172.21.15.106:0/502039305 shutdown_connections 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4640b68d0 0x7fe4640d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 --2- 172.21.15.106:0/502039305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ba290 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 -- 172.21.15.106:0/502039305 >> 172.21.15.106:0/502039305 conn(0x7fe4640d11c0 msgr2=0x7fe4640d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 -- 172.21.15.106:0/502039305 shutdown_connections 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.095+0000 7fe4693bb640 1 -- 172.21.15.106:0/502039305 wait complete. 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.096+0000 7fe4693bb640 1 Processor -- start 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.096+0000 7fe4693bb640 1 -- start start 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4640b68d0 0x7fe4640c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4641624b0 0x7fe464152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe464162910 con 0x7fe46415e9c0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe4640d2a40 con 0x7fe4640b68d0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe4640d34a0 con 0x7fe4641624b0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35330/0 (socket says 172.21.15.106:35330) 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 -- 172.21.15.106:0/543404284 learned_addr learned my addr 172.21.15.106:0/543404284 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4637fe640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4641624b0 0x7fe464152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe462ffd640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4640b68d0 0x7fe4640c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 -- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4641624b0 msgr2=0x7fe464152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4641624b0 0x7fe464152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 -- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4640b68d0 msgr2=0x7fe4640c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4640b68d0 0x7fe4640c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 -- 172.21.15.106:0/543404284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4640c88e0 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45bfff640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b3f0 tx=0x7fe44c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c015020 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c011830 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4640c8bc0 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c011c20 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.097+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4640c9180 con 0x7fe46415e9c0 2024-04-13T03:19:02.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.098+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe464068a10 con 0x7fe46415e9c0 2024-04-13T03:19:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.099+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe44c018020 con 0x7fe46415e9c0 2024-04-13T03:19:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.099+0000 7fe45b7fe640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.099+0000 7fe462ffd640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.099+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7fe44c08b9c0 con 0x7fe46415e9c0 2024-04-13T03:19:02.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.100+0000 7fe462ffd640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 0x7fe44406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe454036000 tx=0x7fe45403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.104+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe44c08de20 con 0x7fe46415e9c0 2024-04-13T03:19:02.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.316+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4640cf3e0 con 0x7fe46415e9c0 2024-04-13T03:19:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.317+0000 7fe45b7fe640 1 -- 172.21.15.106:0/543404284 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5182 (secure 0 0 0) 0x7fe44c01f030 con 0x7fe46415e9c0 2024-04-13T03:19:02.318 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:02.318 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3179085184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3179085184},{"type":"v1","addr":"172.21.15.114:6833","nonce":3179085184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":78,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:59.467638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:19:02.318 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 79 2024-04-13T03:19:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 msgr2=0x7fe44406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 0x7fe44406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe454036000 tx=0x7fe45403c040 comp rx=0 tx=0).stop 2024-04-13T03:19:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 msgr2=0x7fe4640ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b3f0 tx=0x7fe44c00b8b0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 shutdown_connections 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe4641624b0 0x7fe464152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe46415e9c0 0x7fe4640ce150 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 --2- 172.21.15.106:0/543404284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4640b68d0 0x7fe4640c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 >> 172.21.15.106:0/543404284 conn(0x7fe4640d11c0 msgr2=0x7fe4640d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 shutdown_connections 2024-04-13T03:19:02.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.319+0000 7fe4693bb640 1 -- 172.21.15.106:0/543404284 wait complete. 2024-04-13T03:19:02.330 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.432+0000 7f814a915640 1 Processor -- start 2024-04-13T03:19:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.432+0000 7f814a915640 1 -- start start 2024-04-13T03:19:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.432+0000 7f814a915640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f814415ebf0 0x7f814406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f814a915640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814406f380 con 0x7f81440ce1f0 2024-04-13T03:19:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f814a915640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440ce6f0 con 0x7f81440b5bf0 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f814a915640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f814415fa70 con 0x7f814415ebf0 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f81437fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f81440ce1f0 0x7f814406e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:33740/0 (socket says 172.21.15.106:33740) 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f81437fe640 1 -- 172.21.15.106:0/4155482692 learned_addr learned my addr 172.21.15.106:0/4155482692 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8143fff640 1 --2- 172.21.15.106:0/4155482692 >> v2:172.21.15.114:3300/0 conn(0x7f814415ebf0 0x7f814406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441172645 0 0) 0x7f814406f380 con 0x7f81440ce1f0 2024-04-13T03:19:02.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f812c003520 con 0x7f81440ce1f0 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3691514011 0 0) 0x7f812c003520 con 0x7f81440ce1f0 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 >> v2:172.21.15.114:3300/0 conn(0x7f814415ebf0 msgr2=0x7f814406f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 --2- 172.21.15.106:0/4155482692 >> v2:172.21.15.114:3300/0 conn(0x7f814415ebf0 0x7f814406f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 >> v1:172.21.15.114:6789/0 conn(0x7f81440b5bf0 legacy=0x7f814406ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814415f420 con 0x7f81440ce1f0 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f8138001670 con 0x7f81440ce1f0 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 >> v1:172.21.15.106:6789/0 conn(0x7f81440ce1f0 legacy=0x7f814406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f814415fa70 con 0x7f812c0039b0 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f812c00c160 con 0x7f812c004800 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f812c00c360 con 0x7f812c0051c0 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8143fff640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f81437fe640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.433+0000 7f8148eb2640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 -- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f812c0051c0 msgr2=0x7f812c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 -- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c004800 msgr2=0x7f812c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 -- 172.21.15.106:0/4155482692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c003520 con 0x7f812c0039b0 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8148eb2640 1 -- 172.21.15.106:0/4155482692 reap_dead start 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8143fff640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8134009ac0 tx=0x7f812c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d069bb7c5ce233 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134039070 con 0x7f812c0039b0 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813402c6d0 con 0x7f812c0039b0 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.434+0000 7f8142ffd640 1 -- 172.21.15.106:0/4155482692 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134009690 con 0x7f812c0039b0 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 -- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 msgr2=0x7f812c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8134009ac0 tx=0x7f812c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 -- 172.21.15.106:0/4155482692 shutdown_connections 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 --2- 172.21.15.106:0/4155482692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 -- 172.21.15.106:0/4155482692 >> 172.21.15.106:0/4155482692 conn(0x7f81440cd6d0 msgr2=0x7f81440cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 -- 172.21.15.106:0/4155482692 shutdown_connections 2024-04-13T03:19:02.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.435+0000 7f814a915640 1 -- 172.21.15.106:0/4155482692 wait complete. 2024-04-13T03:19:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.436+0000 7f814a915640 1 Processor -- start 2024-04-13T03:19:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.436+0000 7f814a915640 1 -- start start 2024-04-13T03:19:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c0039b0 0x7f8144148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 0x7f814415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b04d0 con 0x7f8144152ce0 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f814415ff00 con 0x7f812c0039b0 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f814a915640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f81440b0650 con 0x7f81440af440 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35350/0 (socket says 172.21.15.106:35350) 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 -- 172.21.15.106:0/65421353 learned_addr learned my addr 172.21.15.106:0/65421353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f8143fff640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c0039b0 0x7f8144148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f8148eb2640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 0x7f814415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 -- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 msgr2=0x7f814415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 0x7f814415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 -- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c0039b0 msgr2=0x7f8144148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c0039b0 0x7f8144148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f81437fe640 1 -- 172.21.15.106:0/65421353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8144162070 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.437+0000 7f8148eb2640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 0x7f814415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f81437fe640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8138007340 tx=0x7f8138007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800f8e0 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8138018070 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81440ade60 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800fcc0 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.438+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81440ae420 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.439+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8144068a10 con 0x7f8144152ce0 2024-04-13T03:19:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.439+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f813801d460 con 0x7f8144152ce0 2024-04-13T03:19:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.440+0000 7f8140ff9640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 0x7f813006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.440+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f813808bd80 con 0x7f8144152ce0 2024-04-13T03:19:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.440+0000 7f8143fff640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 0x7f813006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.440+0000 7f8143fff640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 0x7f813006ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8134036000 tx=0x7f8134030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.444+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f813800fe60 con 0x7f8144152ce0 2024-04-13T03:19:02.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.656+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81440cd140 con 0x7f8144152ce0 2024-04-13T03:19:02.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.656+0000 7f8140ff9640 1 -- 172.21.15.106:0/65421353 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5182 (secure 0 0 0) 0x7f8138059f60 con 0x7f8144152ce0 2024-04-13T03:19:02.657 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:02.657 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3179085184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3179085184},{"type":"v1","addr":"172.21.15.114:6833","nonce":3179085184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":78,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:17:33.740516+0000","modified":"2024-04-13T03:18:59.467638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-13T03:19:02.657 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 79 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 msgr2=0x7f813006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 0x7f813006ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8134036000 tx=0x7f8134030000 comp rx=0 tx=0).stop 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 msgr2=0x7f81440aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8138007340 tx=0x7f8138007800 comp rx=0 tx=0).stop 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 shutdown_connections 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f813006c920 0x7f813006ede0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81440af440 0x7f814415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144152ce0 0x7f81440aef00 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 --2- 172.21.15.106:0/65421353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c0039b0 0x7f8144148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 >> 172.21.15.106:0/65421353 conn(0x7f81440cd6d0 msgr2=0x7f81440cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 shutdown_connections 2024-04-13T03:19:02.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.658+0000 7f814a915640 1 -- 172.21.15.106:0/65421353 wait complete. 2024-04-13T03:19:02.670 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:02.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.769+0000 7f5b92faa640 1 Processor -- start 2024-04-13T03:19:02.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 -- start start 2024-04-13T03:19:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5b8c06ebb0 0x7f5b8c0d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c166ae0 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c166d80 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b92faa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c167060 con 0x7f5b8c166330 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b91547640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b91547640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56930/0 (socket says 172.21.15.106:56930) 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.770+0000 7f5b91547640 1 -- 172.21.15.106:0/2320879023 learned_addr learned my addr 172.21.15.106:0/2320879023 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b90d46640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b8bfff640 1 --2- 172.21.15.106:0/2320879023 >> v2:172.21.15.114:3300/0 conn(0x7f5b8c06ebb0 0x7f5b8c0d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 -- 172.21.15.106:0/2320879023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 msgr2=0x7f5b8c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 -- 172.21.15.106:0/2320879023 >> v2:172.21.15.114:3300/0 conn(0x7f5b8c06ebb0 msgr2=0x7f5b8c0d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 --2- 172.21.15.106:0/2320879023 >> v2:172.21.15.114:3300/0 conn(0x7f5b8c06ebb0 0x7f5b8c0d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 -- 172.21.15.106:0/2320879023 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c167340 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.771+0000 7f5b91547640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c08dde0 tx=0x7f5b8402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45371f2036ddacd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b8b7fe640 1 -- 172.21.15.106:0/2320879023 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8402c9e0 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b8b7fe640 1 -- 172.21.15.106:0/2320879023 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8402cb80 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b8b7fe640 1 -- 172.21.15.106:0/2320879023 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840343f0 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 -- 172.21.15.106:0/2320879023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c0d4af0 msgr2=0x7f5b8c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c08dde0 tx=0x7f5b8402be40 comp rx=0 tx=0).stop 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 -- 172.21.15.106:0/2320879023 shutdown_connections 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c0b61b0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 --2- 172.21.15.106:0/2320879023 >> v2:172.21.15.114:3300/0 conn(0x7f5b8c06ebb0 0x7f5b8c0d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 --2- 172.21.15.106:0/2320879023 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 -- 172.21.15.106:0/2320879023 >> 172.21.15.106:0/2320879023 conn(0x7f5b8c08bf10 msgr2=0x7f5b8c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 -- 172.21.15.106:0/2320879023 shutdown_connections 2024-04-13T03:19:02.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.772+0000 7f5b92faa640 1 -- 172.21.15.106:0/2320879023 wait complete. 2024-04-13T03:19:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.773+0000 7f5b92faa640 1 Processor -- start 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.773+0000 7f5b92faa640 1 -- start start 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c06f030 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0d4f80 con 0x7f5b8c0d4af0 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b92faa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c06f1b0 con 0x7f5b8c166330 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b90d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b90d46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56940/0 (socket says 172.21.15.106:56940) 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b90d46640 1 -- 172.21.15.106:0/3088936815 learned_addr learned my addr 172.21.15.106:0/3088936815 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b91547640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c06daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.774+0000 7f5b8bfff640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 -- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 msgr2=0x7f5b8c06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c06daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 -- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b8c0d4af0 msgr2=0x7f5b8c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 -- 172.21.15.106:0/3088936815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c1fa020 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b91547640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b90d46640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00d5f0 tx=0x7f5b7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0125f0 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c019070 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c1eca80 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0129d0 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.775+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1ed040 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.776+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c068a10 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.776+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7c012b70 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.777+0000 7f5b897fa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 0x7f5b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.777+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f5b7c08c100 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.777+0000 7f5b8bfff640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 0x7f5b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.777+0000 7f5b8bfff640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 0x7f5b7806edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0b64f0 tx=0x7f5b80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:02.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.781+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7c05a370 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.955+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b8c0b36c0 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.956+0000 7f5b897fa640 1 -- 172.21.15.106:0/3088936815 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17233 (secure 0 0 0) 0x7f5b7c060030 con 0x7f5b8c06ebb0 2024-04-13T03:19:02.957 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:02.957 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":111,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:18:59.467477+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:17:31.965063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-13T03:17:32.694523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6833/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6832/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6835/2645971931":"2024-04-14T03:18:23.121696+0000","172.21.15.114:6834/2645971931":"2024-04-14T03:18:23.121696+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:02.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 msgr2=0x7f5b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 0x7f5b7806edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0b64f0 tx=0x7f5b80008040 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 msgr2=0x7f5b8c161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00d5f0 tx=0x7f5b7c00dab0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 shutdown_connections 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b7806c8f0 0x7f5b7806edb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5b8c166330 0x7f5b8c06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b8c0d4af0 0x7f5b8c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.958+0000 7f5b92faa640 1 --2- 172.21.15.106:0/3088936815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b8c06ebb0 0x7f5b8c161370 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.959+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 >> 172.21.15.106:0/3088936815 conn(0x7f5b8c08bf10 msgr2=0x7f5b8c0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:02.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.959+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 shutdown_connections 2024-04-13T03:19:02.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:02.959+0000 7f5b92faa640 1 -- 172.21.15.106:0/3088936815 wait complete. 2024-04-13T03:19:02.972 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:19:03.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.072+0000 7f2f358d5640 1 Processor -- start 2024-04-13T03:19:03.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.072+0000 7f2f358d5640 1 -- start start 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f358d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2f30070c00 0x7f2f300cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f358d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f30072d40 con 0x7f2f3015ec10 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f358d5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f30073020 con 0x7f2f300b42e0 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f358d5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2f30073300 con 0x7f2f30070c00 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f2effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f3015ec10 0x7f2f300704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49758/0 (socket says 172.21.15.106:49758) 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.073+0000 7f2f2effd640 1 -- 172.21.15.106:0/192699731 learned_addr learned my addr 172.21.15.106:0/192699731 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2e7fc640 1 --2- 172.21.15.106:0/192699731 >> v2:172.21.15.114:3300/0 conn(0x7f2f30070c00 0x7f2f300cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257474976 0 0) 0x7f2f30072d40 con 0x7f2f3015ec10 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f18003520 con 0x7f2f3015ec10 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3247155108 0 0) 0x7f2f18003520 con 0x7f2f3015ec10 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 >> v2:172.21.15.114:3300/0 conn(0x7f2f30070c00 msgr2=0x7f2f300cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 --2- 172.21.15.106:0/192699731 >> v2:172.21.15.114:3300/0 conn(0x7f2f30070c00 0x7f2f300cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 >> v1:172.21.15.114:6789/0 conn(0x7f2f300b42e0 legacy=0x7f2f300cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f300735e0 con 0x7f2f3015ec10 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2f200019f0 con 0x7f2f3015ec10 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 >> v1:172.21.15.106:6789/0 conn(0x7f2f3015ec10 legacy=0x7f2f300704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180039b0 0x7f2f18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f18004800 0x7f2f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 0x7f2f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30073300 con 0x7f2f180051c0 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1800c060 con 0x7f2f18004800 2024-04-13T03:19:03.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1800c260 con 0x7f2f180039b0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 0x7f2f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2e7fc640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f18004800 0x7f2f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2effd640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180039b0 0x7f2f18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180039b0 msgr2=0x7f2f18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180039b0 0x7f2f18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f18004800 msgr2=0x7f2f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f18004800 0x7f2f18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.074+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/192699731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f18003520 con 0x7f2f180051c0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.075+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/192699731 reap_dead start 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.075+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 0x7f2f1800b9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2f28001750 tx=0x7f2f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2e7d0e9b6f9251 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.075+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28001370 con 0x7f2f180051c0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.075+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f28005310 con 0x7f2f180051c0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.075+0000 7f2f2dffb640 1 -- 172.21.15.106:0/192699731 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003830 con 0x7f2f180051c0 2024-04-13T03:19:03.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 -- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 msgr2=0x7f2f1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 0x7f2f1800b9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2f28001750 tx=0x7f2f18005830 comp rx=0 tx=0).stop 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 -- 172.21.15.106:0/192699731 shutdown_connections 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 --2- 172.21.15.106:0/192699731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f180051c0 0x7f2f1800b9c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 -- 172.21.15.106:0/192699731 >> 172.21.15.106:0/192699731 conn(0x7f2f300d13f0 msgr2=0x7f2f300d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 -- 172.21.15.106:0/192699731 shutdown_connections 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.076+0000 7f2f358d5640 1 -- 172.21.15.106:0/192699731 wait complete. 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.077+0000 7f2f358d5640 1 Processor -- start 2024-04-13T03:19:03.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.077+0000 7f2f358d5640 1 -- start start 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180051c0 0x7f2f301562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f30159fb0 0x7f2f3014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3006e450 con 0x7f2f300af420 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2f300cdc20 con 0x7f2f30159fb0 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f358d5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3006e5d0 con 0x7f2f180051c0 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56950/0 (socket says 172.21.15.106:56950) 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/1756514047 learned_addr learned my addr 172.21.15.106:0/1756514047 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2effd640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180051c0 0x7f2f301562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2e7fc640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f30159fb0 0x7f2f3014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180051c0 msgr2=0x7f2f301562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180051c0 0x7f2f301562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f30159fb0 msgr2=0x7f2f3014d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f30159fb0 0x7f2f3014d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 -- 172.21.15.106:0/1756514047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3015a470 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.078+0000 7f2f2f7fe640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2f28001870 tx=0x7f2f28001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.079+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28001370 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.079+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f280050d0 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.079+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3014d6a0 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.079+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28007a30 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.079+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f301569a0 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.080+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f300cec70 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.080+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f28005310 con 0x7f2f300af420 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.080+0000 7f2f17fff640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 0x7f2f1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:03.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.081+0000 7f2f2effd640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 0x7f2f1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:03.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.081+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f2f280ac3c0 con 0x7f2f300af420 2024-04-13T03:19:03.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.081+0000 7f2f2effd640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 0x7f2f1c06edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2f20004d00 tx=0x7f2f20006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:03.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.085+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f300cec70 con 0x7f2f300af420 2024-04-13T03:19:03.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:03.297+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2f30069120 con 0x7f2f300af420 2024-04-13T03:19:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.224+0000 7f2f17fff640 1 -- 172.21.15.106:0/1756514047 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f2f3014d6a0 con 0x7f2f300af420 2024-04-13T03:19:04.225 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 msgr2=0x7f2f1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 0x7f2f1c06edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2f20004d00 tx=0x7f2f20006040 comp rx=0 tx=0).stop 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 msgr2=0x7f2f300af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2f28001870 tx=0x7f2f28001120 comp rx=0 tx=0).stop 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 shutdown_connections 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2f1c06c8f0 0x7f2f1c06edb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f300af420 0x7f2f300af8a0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f30159fb0 0x7f2f3014d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 --2- 172.21.15.106:0/1756514047 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2f180051c0 0x7f2f301562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 >> 172.21.15.106:0/1756514047 conn(0x7f2f300d13f0 msgr2=0x7f2f3006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:04.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 shutdown_connections 2024-04-13T03:19:04.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.227+0000 7f2f358d5640 1 -- 172.21.15.106:0/1756514047 wait complete. 2024-04-13T03:19:04.242 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:19:04.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.350+0000 7f01696d1640 1 Processor -- start 2024-04-13T03:19:04.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.350+0000 7f01696d1640 1 -- start start 2024-04-13T03:19:04.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01696d1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f016415ed90 0x7f01640d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01696d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01640cfd20 con 0x7f01640b3d60 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01696d1640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0164162af0 con 0x7f016415e9b0 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01696d1640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0164162dd0 con 0x7f016415ed90 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01637fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f01640b3d60 0x7f01640d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49768/0 (socket says 172.21.15.106:49768) 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01637fe640 1 -- 172.21.15.106:0/1216715411 learned_addr learned my addr 172.21.15.106:0/1216715411 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3968816583 0 0) 0x7f01640cfd20 con 0x7f01640b3d60 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0150003520 con 0x7f01640b3d60 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f015bfff640 1 --2- 172.21.15.106:0/1216715411 >> v2:172.21.15.114:3300/0 conn(0x7f016415ed90 0x7f01640d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 875362031 0 0) 0x7f0150003520 con 0x7f01640b3d60 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 >> v2:172.21.15.114:3300/0 conn(0x7f016415ed90 msgr2=0x7f01640d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 --2- 172.21.15.106:0/1216715411 >> v2:172.21.15.114:3300/0 conn(0x7f016415ed90 0x7f01640d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 >> v1:172.21.15.114:6789/0 conn(0x7f016415e9b0 legacy=0x7f01640d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01641630b0 con 0x7f01640b3d60 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f015c0019f0 con 0x7f01640b3d60 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 >> v1:172.21.15.106:6789/0 conn(0x7f01640b3d60 legacy=0x7f01640d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.351+0000 7f01627fc640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0150004800 0x7f0150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01627fc640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500051c0 0x7f015000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0164162dd0 con 0x7f01500039b0 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f015000c070 con 0x7f0150004800 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f015000c270 con 0x7f01500051c0 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f015bfff640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0150004800 0x7f0150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01637fe640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500051c0 0x7f015000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 -- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500051c0 msgr2=0x7f015000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:04.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500051c0 0x7f015000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 -- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0150004800 msgr2=0x7f0150004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0150004800 0x7f0150004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f0162ffd640 1 -- 172.21.15.106:0/1216715411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0150003520 con 0x7f01500039b0 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.352+0000 7f01637fe640 1 -- 172.21.15.106:0/1216715411 reap_dead start 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f0162ffd640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0154001750 tx=0x7f0150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff43c17ef59dc3af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154001370 con 0x7f01500039b0 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01540052d0 con 0x7f01500039b0 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01627fc640 1 -- 172.21.15.106:0/1216715411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154003830 con 0x7f01500039b0 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 -- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 msgr2=0x7f0150003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0154001750 tx=0x7f0150005830 comp rx=0 tx=0).stop 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 -- 172.21.15.106:0/1216715411 shutdown_connections 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 --2- 172.21.15.106:0/1216715411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 -- 172.21.15.106:0/1216715411 >> 172.21.15.106:0/1216715411 conn(0x7f016408bf10 msgr2=0x7f01640b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:04.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 -- 172.21.15.106:0/1216715411 shutdown_connections 2024-04-13T03:19:04.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.353+0000 7f01696d1640 1 -- 172.21.15.106:0/1216715411 wait complete. 2024-04-13T03:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 Processor -- start 2024-04-13T03:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 -- start start 2024-04-13T03:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500039b0 0x7f016406d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016415d210 0x7f01641601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0164162800 con 0x7f0164160710 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0164162680 con 0x7f016415d210 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01696d1640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f016406eb20 con 0x7f01500039b0 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.355+0000 7f01637fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56958/0 (socket says 172.21.15.106:56958) 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 -- 172.21.15.106:0/3896045251 learned_addr learned my addr 172.21.15.106:0/3896045251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f0162ffd640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500039b0 0x7f016406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f015bfff640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016415d210 0x7f01641601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 -- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500039b0 msgr2=0x7f016406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500039b0 0x7f016406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 -- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016415d210 msgr2=0x7f01641601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016415d210 0x7f01641601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:04.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 -- 172.21.15.106:0/3896045251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016406c4f0 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01637fe640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f016408e0d0 tx=0x7f015c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00e980 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c018070 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00ed60 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016406c7d0 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.356+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016406cd90 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.357+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0164160b90 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.358+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f015c01d460 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.358+0000 7f015b7fe640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 0x7f014406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.358+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7f015c08be70 con 0x7f0164160710 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.358+0000 7f0162ffd640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 0x7f014406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:04.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.358+0000 7f0162ffd640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 0x7f014406ede0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0154033000 tx=0x7f0154009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:04.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.362+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f015c05a050 con 0x7f0164160710 2024-04-13T03:19:04.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:04.576+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0164068a10 con 0x7f0164160710 2024-04-13T03:19:05.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.237+0000 7f015b7fe640 1 -- 172.21.15.106:0/3896045251 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7f015c059950 con 0x7f0164160710 2024-04-13T03:19:05.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 msgr2=0x7f014406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 0x7f014406ede0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0154033000 tx=0x7f0154009030 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 msgr2=0x7f016406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f016408e0d0 tx=0x7f015c007b00 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 shutdown_connections 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f014406c920 0x7f014406ede0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0164160710 0x7f016406b8e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016415d210 0x7f01641601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 --2- 172.21.15.106:0/3896045251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f01500039b0 0x7f016406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 >> 172.21.15.106:0/3896045251 conn(0x7f016408bf10 msgr2=0x7f01640b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 shutdown_connections 2024-04-13T03:19:05.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.240+0000 7f01696d1640 1 -- 172.21.15.106:0/3896045251 wait complete. 2024-04-13T03:19:05.254 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:19:05.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.363+0000 7fe23215e640 1 Processor -- start 2024-04-13T03:19:05.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.363+0000 7fe23215e640 1 -- start start 2024-04-13T03:19:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe23215e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe23215e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe22c0709e0 0x7fe22c1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe23215e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22c15ece0 con 0x7fe22c15e900 2024-04-13T03:19:05.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe23215e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe22c160040 con 0x7fe22c0709e0 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe23215e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c06da90 con 0x7fe22c15fb20 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe22b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe22c15e900 0x7fe22c06d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49782/0 (socket says 172.21.15.106:49782) 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe22b7fe640 1 -- 172.21.15.106:0/76110307 learned_addr learned my addr 172.21.15.106:0/76110307 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe22bfff640 1 --2- 172.21.15.106:0/76110307 >> v2:172.21.15.106:3300/0 conn(0x7fe22c0709e0 0x7fe22c1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.364+0000 7fe22affd640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22a7fc640 1 -- 172.21.15.106:0/76110307 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4089475311 0 0) 0x7fe22c15ece0 con 0x7fe22c15e900 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22a7fc640 1 -- 172.21.15.106:0/76110307 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe214003520 con 0x7fe22c15e900 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22bfff640 1 -- 172.21.15.106:0/76110307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 msgr2=0x7fe22c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22bfff640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22bfff640 1 -- 172.21.15.106:0/76110307 >> v1:172.21.15.106:6789/0 conn(0x7fe22c15e900 legacy=0x7fe22c06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22bfff640 1 -- 172.21.15.106:0/76110307 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c06dd70 con 0x7fe22c0709e0 2024-04-13T03:19:05.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22bfff640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1607b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe224009d90 tx=0x7fe22402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1664ff38a7cc4a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.365+0000 7fe22a7fc640 1 -- 172.21.15.106:0/76110307 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224039070 con 0x7fe22c0709e0 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe22a7fc640 1 -- 172.21.15.106:0/76110307 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22402cb30 con 0x7fe22c0709e0 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe22a7fc640 1 -- 172.21.15.106:0/76110307 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224034440 con 0x7fe22c0709e0 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 -- 172.21.15.106:0/76110307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 msgr2=0x7fe22c1607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1607b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe224009d90 tx=0x7fe22402bd80 comp rx=0 tx=0).stop 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 -- 172.21.15.106:0/76110307 shutdown_connections 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1607b0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 --2- 172.21.15.106:0/76110307 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 -- 172.21.15.106:0/76110307 >> 172.21.15.106:0/76110307 conn(0x7fe22c0717e0 msgr2=0x7fe22c071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 -- 172.21.15.106:0/76110307 shutdown_connections 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.366+0000 7fe23215e640 1 -- 172.21.15.106:0/76110307 wait complete. 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.367+0000 7fe23215e640 1 Processor -- start 2024-04-13T03:19:05.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.367+0000 7fe23215e640 1 -- start start 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c15e900 0x7fe22c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c160e70 con 0x7fe22c0709e0 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c160cf0 con 0x7fe22c15e900 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe23215e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c160ff0 con 0x7fe22c15fb20 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56970/0 (socket says 172.21.15.106:56970) 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 -- 172.21.15.106:0/3870578711 learned_addr learned my addr 172.21.15.106:0/3870578711 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 -- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 msgr2=0x7fe22c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.368+0000 7fe22b7fe640 1 -- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c15e900 msgr2=0x7fe22c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:05.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe22b7fe640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c15e900 0x7fe22c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe22b7fe640 1 -- 172.21.15.106:0/3870578711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c0ce590 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe22b7fe640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c430 tx=0x7fe21c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c017070 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c005ce0 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0072d0 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c0ccab0 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.369+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1f4fb0 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.370+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c068a10 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.370+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.371+0000 7fe213fff640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 0x7fe21806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.371+0000 7fe22affd640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 0x7fe21806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.371+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6302+0+0 (secure 0 0 0) 0x7fe21c08b110 con 0x7fe22c0709e0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.371+0000 7fe22affd640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 0x7fe21806edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe220006a90 tx=0x7fe220006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:05.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.375+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe21c058ca0 con 0x7fe22c0709e0 2024-04-13T03:19:05.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:05.551+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe22c0b5ff0 con 0x7fe22c0709e0 2024-04-13T03:19:06.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.239+0000 7fe213fff640 1 -- 172.21.15.106:0/3870578711 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7fe21c05cfa0 con 0x7fe22c0709e0 2024-04-13T03:19:06.240 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:19:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 msgr2=0x7fe21806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:06.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 0x7fe21806edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe220006a90 tx=0x7fe220006a20 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 msgr2=0x7fe22c1f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c430 tx=0x7fe21c00c8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 shutdown_connections 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe21806c8f0 0x7fe21806edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe22c15fb20 0x7fe22c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c15e900 0x7fe22c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 --2- 172.21.15.106:0/3870578711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe22c0709e0 0x7fe22c1f57b0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.242+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 >> 172.21.15.106:0/3870578711 conn(0x7fe22c0717e0 msgr2=0x7fe22c0b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.243+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 shutdown_connections 2024-04-13T03:19:06.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.243+0000 7fe23215e640 1 -- 172.21.15.106:0/3870578711 wait complete. 2024-04-13T03:19:06.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:19:06.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.370+0000 7fd775355640 1 Processor -- start 2024-04-13T03:19:06.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.370+0000 7fd775355640 1 -- start start 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd775355640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 0x7fd77006d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd775355640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd77015fb20 0x7fd7700704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd775355640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd770070e00 con 0x7fd7700709e0 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd775355640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd770160040 con 0x7fd77015fb20 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd775355640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd77015ece0 con 0x7fd77015e900 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd77015fb20 0x7fd7700704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd77015fb20 0x7fd7700704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56972/0 (socket says 172.21.15.106:56972) 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 -- 172.21.15.106:0/3103323381 learned_addr learned my addr 172.21.15.106:0/3103323381 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76effd640 1 --2- 172.21.15.106:0/3103323381 >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 0x7fd77006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 -- 172.21.15.106:0/3103323381 >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 msgr2=0x7fd77006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 --2- 172.21.15.106:0/3103323381 >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 0x7fd77006d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 -- 172.21.15.106:0/3103323381 >> v1:172.21.15.114:6789/0 conn(0x7fd7700709e0 legacy=0x7fd770162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76e7fc640 1 -- 172.21.15.106:0/3103323381 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77006da90 con 0x7fd77015fb20 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.371+0000 7fd76effd640 1 --2- 172.21.15.106:0/3103323381 >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 0x7fd77006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.372+0000 7fd76e7fc640 1 --2- 172.21.15.106:0/3103323381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015fb20 0x7fd7700704a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd7700b86b0 tx=0x7fd76402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8427300940ead0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.372+0000 7fd76dffb640 1 -- 172.21.15.106:0/3103323381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764038070 con 0x7fd77015fb20 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.372+0000 7fd76dffb640 1 -- 172.21.15.106:0/3103323381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76402f050 con 0x7fd77015fb20 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.372+0000 7fd76dffb640 1 -- 172.21.15.106:0/3103323381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764031970 con 0x7fd77015fb20 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 -- 172.21.15.106:0/3103323381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015fb20 msgr2=0x7fd7700704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 --2- 172.21.15.106:0/3103323381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015fb20 0x7fd7700704a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd7700b86b0 tx=0x7fd76402b7f0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 -- 172.21.15.106:0/3103323381 shutdown_connections 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 --2- 172.21.15.106:0/3103323381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015fb20 0x7fd7700704a0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 --2- 172.21.15.106:0/3103323381 >> v2:172.21.15.114:3300/0 conn(0x7fd77015e900 0x7fd77006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 -- 172.21.15.106:0/3103323381 >> 172.21.15.106:0/3103323381 conn(0x7fd7700717e0 msgr2=0x7fd770071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:06.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 -- 172.21.15.106:0/3103323381 shutdown_connections 2024-04-13T03:19:06.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.373+0000 7fd775355640 1 -- 172.21.15.106:0/3103323381 wait complete. 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.374+0000 7fd775355640 1 Processor -- start 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.374+0000 7fd775355640 1 -- start start 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7700709e0 0x7fd770154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd77015fb20 0x7fd770148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd77006d550 con 0x7fd77015e900 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd77006e050 con 0x7fd7700709e0 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd775355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fd77006d6d0 con 0x7fd77015fb20 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56984/0 (socket says 172.21.15.106:56984) 2024-04-13T03:19:06.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 -- 172.21.15.106:0/1952210584 learned_addr learned my addr 172.21.15.106:0/1952210584 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76effd640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7700709e0 0x7fd770154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76f7fe640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd77015fb20 0x7fd770148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 -- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd77015fb20 msgr2=0x7fd770148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd77015fb20 0x7fd770148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 -- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7700709e0 msgr2=0x7fd770154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7700709e0 0x7fd770154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 -- 172.21.15.106:0/1952210584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd770153de0 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd76e7fc640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd764000c00 tx=0x7fd7640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.375+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764042070 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.376+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76403d890 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.376+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7701540c0 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.376+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764038040 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.376+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd770154680 con 0x7fd77015e900 2024-04-13T03:19:06.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.376+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770068a10 con 0x7fd77015e900 2024-04-13T03:19:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.377+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd764045070 con 0x7fd77015e900 2024-04-13T03:19:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.377+0000 7fd7577fe640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 0x7fd75c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.377+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5864+0+0 (secure 0 0 0) 0x7fd7640ad940 con 0x7fd77015e900 2024-04-13T03:19:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.377+0000 7fd76effd640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 0x7fd75c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:06.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.378+0000 7fd76effd640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 0x7fd75c06ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd76000f500 tx=0x7fd760007770 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:06.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.382+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd76407b7a0 con 0x7fd77015e900 2024-04-13T03:19:06.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:06.558+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd770160850 con 0x7fd77015e900 2024-04-13T03:19:07.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.255+0000 7fd7577fe640 1 -- 172.21.15.106:0/1952210584 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7fd76407faa0 con 0x7fd77015e900 2024-04-13T03:19:07.256 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:19:07.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.257+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 msgr2=0x7fd75c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.257+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 0x7fd75c06ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd76000f500 tx=0x7fd760007770 comp rx=0 tx=0).stop 2024-04-13T03:19:07.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.257+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 msgr2=0x7fd770157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.257+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd764000c00 tx=0x7fd7640046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 shutdown_connections 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fd75c06c920 0x7fd75c06ede0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fd77015fb20 0x7fd770148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd77015e900 0x7fd770157990 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 --2- 172.21.15.106:0/1952210584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7700709e0 0x7fd770154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.258+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 >> 172.21.15.106:0/1952210584 conn(0x7fd7700717e0 msgr2=0x7fd770162d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:07.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.260+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 shutdown_connections 2024-04-13T03:19:07.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.262+0000 7fd775355640 1 -- 172.21.15.106:0/1952210584 wait complete. 2024-04-13T03:19:07.279 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:19:07.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.384+0000 7f5a661bb640 1 Processor -- start 2024-04-13T03:19:07.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.384+0000 7f5a661bb640 1 -- start start 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a661bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5a600b4c20 0x7f5a600cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a661bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6006d220 con 0x7f5a600b47e0 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a661bb640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a600b3740 con 0x7f5a60145d20 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a661bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5a600b3a20 con 0x7f5a600b4c20 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5a600b4c20 0x7f5a600cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5a600b47e0 0x7f5a6015cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49790/0 (socket says 172.21.15.106:49790) 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5effd640 1 -- 172.21.15.106:0/1485212491 learned_addr learned my addr 172.21.15.106:0/1485212491 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:07.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5ffff640 1 -- 172.21.15.106:0/1485212491 >> v1:172.21.15.114:6789/0 conn(0x7f5a60145d20 legacy=0x7f5a6006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:07.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5ffff640 1 -- 172.21.15.106:0/1485212491 >> v1:172.21.15.106:6789/0 conn(0x7f5a600b47e0 legacy=0x7f5a6015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5e7fc640 1 -- 172.21.15.106:0/1485212491 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679570396 0 0) 0x7f5a6006d220 con 0x7f5a600b47e0 2024-04-13T03:19:07.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.385+0000 7f5a5ffff640 1 -- 172.21.15.106:0/1485212491 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a600b3d00 con 0x7f5a600b4c20 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.386+0000 7f5a5ffff640 1 --2- 172.21.15.106:0/1485212491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a600cf1a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009d90 tx=0x7f5a5802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558808d8d5daf80f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.386+0000 7f5a5e7fc640 1 -- 172.21.15.106:0/1485212491 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58039070 con 0x7f5a600b4c20 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.386+0000 7f5a5e7fc640 1 -- 172.21.15.106:0/1485212491 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5802cbd0 con 0x7f5a600b4c20 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.386+0000 7f5a5e7fc640 1 -- 172.21.15.106:0/1485212491 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58034440 con 0x7f5a600b4c20 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 -- 172.21.15.106:0/1485212491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 msgr2=0x7f5a600cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 --2- 172.21.15.106:0/1485212491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a600cf1a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009d90 tx=0x7f5a5802be20 comp rx=0 tx=0).stop 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 -- 172.21.15.106:0/1485212491 shutdown_connections 2024-04-13T03:19:07.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 --2- 172.21.15.106:0/1485212491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a600cf1a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 -- 172.21.15.106:0/1485212491 >> 172.21.15.106:0/1485212491 conn(0x7f5a600b2600 msgr2=0x7f5a600d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:07.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 -- 172.21.15.106:0/1485212491 shutdown_connections 2024-04-13T03:19:07.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.387+0000 7f5a661bb640 1 -- 172.21.15.106:0/1485212491 wait complete. 2024-04-13T03:19:07.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.388+0000 7f5a661bb640 1 Processor -- start 2024-04-13T03:19:07.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.388+0000 7f5a661bb640 1 -- start start 2024-04-13T03:19:07.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 0x7f5a6006f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a60145d20 0x7f5a601f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60146100 con 0x7f5a600b4c20 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600b42c0 con 0x7f5a600b47e0 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5a60146280 con 0x7f5a60145d20 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56998/0 (socket says 172.21.15.106:56998) 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 -- 172.21.15.106:0/2880580709 learned_addr learned my addr 172.21.15.106:0/2880580709 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5f7fe640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 0x7f5a6006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5ffff640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a60145d20 0x7f5a601f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 -- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a60145d20 msgr2=0x7f5a601f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a60145d20 0x7f5a601f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 -- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 msgr2=0x7f5a6006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 0x7f5a6006f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 -- 172.21.15.106:0/2880580709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6013c940 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5f7fe640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 0x7f5a6006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a5effd640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c420 tx=0x7f5a5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a540074e0 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a540047e0 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54004c10 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6013cec0 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.389+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a600adb70 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.390+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6013ccf0 con 0x7f5a600b4c20 2024-04-13T03:19:07.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.391+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a54011070 con 0x7f5a600b4c20 2024-04-13T03:19:07.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.391+0000 7f5a47fff640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 0x7f5a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.391+0000 7f5a5f7fe640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 0x7f5a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.391+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f5a5408b600 con 0x7f5a600b4c20 2024-04-13T03:19:07.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.392+0000 7f5a5f7fe640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 0x7f5a4c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a50007fd0 tx=0x7f5a50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.396+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a5400e280 con 0x7f5a600b4c20 2024-04-13T03:19:07.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.572+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5a60143d70 con 0x7f5a600b4c20 2024-04-13T03:19:07.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.576+0000 7f5a47fff640 1 -- 172.21.15.106:0/2880580709 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f5a54059be0 con 0x7f5a600b4c20 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 msgr2=0x7f5a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 0x7f5a4c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a50007fd0 tx=0x7f5a50009040 comp rx=0 tx=0).stop 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 msgr2=0x7f5a6013d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c420 tx=0x7f5a5400c8e0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 shutdown_connections 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a4c06c8f0 0x7f5a4c06edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a60145d20 0x7f5a601f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a600b4c20 0x7f5a6013d220 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 --2- 172.21.15.106:0/2880580709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a600b47e0 0x7f5a6006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 >> 172.21.15.106:0/2880580709 conn(0x7f5a600b2600 msgr2=0x7f5a600cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:07.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 shutdown_connections 2024-04-13T03:19:07.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.578+0000 7f5a661bb640 1 -- 172.21.15.106:0/2880580709 wait complete. 2024-04-13T03:19:07.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-13T03:19:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.689+0000 7f2ba6cd5640 1 Processor -- start 2024-04-13T03:19:07.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.689+0000 7f2ba6cd5640 1 -- start start 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba6cd5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ba00cdf50 0x7f2ba00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba6cd5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006e150 0x7f2ba006e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba6cd5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba00ce870 con 0x7f2ba006cb10 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba6cd5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba00998d0 con 0x7f2ba00cdf50 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba6cd5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba0073020 con 0x7f2ba006e150 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba4a71640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2ba00cdf50 0x7f2ba00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2b9ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2ba006cb10 0x7f2ba0072720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49806/0 (socket says 172.21.15.106:49806) 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2b9ffff640 1 -- 172.21.15.106:0/1930670945 learned_addr learned my addr 172.21.15.106:0/1930670945 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:07.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba4a71640 1 -- 172.21.15.106:0/1930670945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006e150 msgr2=0x7f2ba006e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:07.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1930670945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006e150 0x7f2ba006e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba4a71640 1 -- 172.21.15.106:0/1930670945 >> v1:172.21.15.106:6789/0 conn(0x7f2ba006cb10 legacy=0x7f2ba0072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2ba4a71640 1 -- 172.21.15.106:0/1930670945 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba0073300 con 0x7f2ba00cdf50 2024-04-13T03:19:07.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.690+0000 7f2b9f7fe640 1 -- 172.21.15.106:0/1930670945 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867537913 0 0) 0x7f2ba00ce870 con 0x7f2ba006cb10 2024-04-13T03:19:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1930670945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00ce330 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2b940097a0 tx=0x7f2b9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e005b913af70185 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2b9f7fe640 1 -- 172.21.15.106:0/1930670945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94035020 con 0x7f2ba00cdf50 2024-04-13T03:19:07.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2b9f7fe640 1 -- 172.21.15.106:0/1930670945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b94031830 con 0x7f2ba00cdf50 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2b9f7fe640 1 -- 172.21.15.106:0/1930670945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94031c00 con 0x7f2ba00cdf50 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1930670945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba00cdf50 msgr2=0x7f2ba00ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.691+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1930670945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00ce330 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2b940097a0 tx=0x7f2b9402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1930670945 shutdown_connections 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1930670945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006e150 0x7f2ba006e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1930670945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00ce330 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1930670945 >> 172.21.15.106:0/1930670945 conn(0x7f2ba00b1f10 msgr2=0x7f2ba00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:07.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1930670945 shutdown_connections 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.692+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1930670945 wait complete. 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.693+0000 7f2ba6cd5640 1 Processor -- start 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.693+0000 7f2ba6cd5640 1 -- start start 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 0x7f2ba01e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba006eb40 con 0x7f2ba006e150 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba00738c0 con 0x7f2ba00cdf50 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba006ecc0 con 0x7f2ba006cb10 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57004/0 (socket says 172.21.15.106:57004) 2024-04-13T03:19:07.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 -- 172.21.15.106:0/1918263511 learned_addr learned my addr 172.21.15.106:0/1918263511 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 0x7f2ba01e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 -- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 msgr2=0x7f2ba01e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba5272640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 0x7f2ba01e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 -- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 msgr2=0x7f2ba00c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 -- 172.21.15.106:0/1918263511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba01f7da0 con 0x7f2ba006e150 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 0x7f2ba01e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba5272640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9ffff640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c4b0 tx=0x7f2b8c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c018070 con 0x7f2ba006e150 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b8c007500 con 0x7f2ba006e150 2024-04-13T03:19:07.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba01f8040 con 0x7f2ba006e150 2024-04-13T03:19:07.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c004a20 con 0x7f2ba006e150 2024-04-13T03:19:07.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.694+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01f88e0 con 0x7f2ba006e150 2024-04-13T03:19:07.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.695+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba01f8430 con 0x7f2ba006e150 2024-04-13T03:19:07.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.696+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b8c01b070 con 0x7f2ba006e150 2024-04-13T03:19:07.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.696+0000 7f2b9d7fa640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 0x7f2b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:07.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.696+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f2b8c08ad60 con 0x7f2ba006e150 2024-04-13T03:19:07.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.696+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 0x7f2b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:07.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.697+0000 7f2ba4a71640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 0x7f2b8406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036000 tx=0x7f2b9403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:07.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.701+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b8c059370 con 0x7f2ba006e150 2024-04-13T03:19:07.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:07.943+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f2ba0068a10 con 0x7f2ba006e150 2024-04-13T03:19:08.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.255+0000 7f2b9d7fa640 1 -- 172.21.15.106:0/1918263511 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v454) ==== 146+0+0 (secure 0 0 0) 0x7f2b8c05f020 con 0x7f2ba006e150 2024-04-13T03:19:08.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 msgr2=0x7f2b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 0x7f2b8406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036000 tx=0x7f2b9403c040 comp rx=0 tx=0).stop 2024-04-13T03:19:08.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 msgr2=0x7f2ba00c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c4b0 tx=0x7f2b8c00c970 comp rx=0 tx=0).stop 2024-04-13T03:19:08.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 shutdown_connections 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2b8406c8f0 0x7f2b8406edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ba00cdf50 0x7f2ba00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ba006e150 0x7f2ba00c5a60 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 --2- 172.21.15.106:0/1918263511 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2ba006cb10 0x7f2ba01e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 >> 172.21.15.106:0/1918263511 conn(0x7f2ba00b1f10 msgr2=0x7f2ba006f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 shutdown_connections 2024-04-13T03:19:08.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.257+0000 7f2ba6cd5640 1 -- 172.21.15.106:0/1918263511 wait complete. 2024-04-13T03:19:08.270 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-13T03:19:08.271 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-13T03:19:08.271 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.370+0000 7fb16911e640 1 Processor -- start 2024-04-13T03:19:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.370+0000 7fb16911e640 1 -- start start 2024-04-13T03:19:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16911e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1640caf30 0x7fb1641630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16911e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1640cd700 con 0x7fb164144f10 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16911e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1640cd9e0 con 0x7fb1640d0bc0 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16911e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1640cd4d0 con 0x7fb1640caf30 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1640caf30 0x7fb1641630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1640caf30 0x7fb1641630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57014/0 (socket says 172.21.15.106:57014) 2024-04-13T03:19:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb162d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb164144f10 0x7fb1640ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49822/0 (socket says 172.21.15.106:49822) 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 -- 172.21.15.106:0/2773080534 learned_addr learned my addr 172.21.15.106:0/2773080534 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 -- 172.21.15.106:0/2773080534 >> v1:172.21.15.114:6789/0 conn(0x7fb1640d0bc0 legacy=0x7fb1640cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 -- 172.21.15.106:0/2773080534 >> v1:172.21.15.106:6789/0 conn(0x7fb164144f10 legacy=0x7fb1640ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb161d9b640 1 -- 172.21.15.106:0/2773080534 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979065924 0 0) 0x7fb1640cd700 con 0x7fb164144f10 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.371+0000 7fb16259c640 1 -- 172.21.15.106:0/2773080534 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640cdcc0 con 0x7fb1640caf30 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.372+0000 7fb16259c640 1 --2- 172.21.15.106:0/2773080534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb1641630c0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f3e87f1e7dff1b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.372+0000 7fb161d9b640 1 -- 172.21.15.106:0/2773080534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15802f840 con 0x7fb1640caf30 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.372+0000 7fb161d9b640 1 -- 172.21.15.106:0/2773080534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15802f9e0 con 0x7fb1640caf30 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.372+0000 7fb161d9b640 1 -- 172.21.15.106:0/2773080534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15802fdb0 con 0x7fb1640caf30 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 -- 172.21.15.106:0/2773080534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 msgr2=0x7fb1641630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 --2- 172.21.15.106:0/2773080534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb1641630c0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802ba20 comp rx=0 tx=0).stop 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 -- 172.21.15.106:0/2773080534 shutdown_connections 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 --2- 172.21.15.106:0/2773080534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb1641630c0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 -- 172.21.15.106:0/2773080534 >> 172.21.15.106:0/2773080534 conn(0x7fb16406f180 msgr2=0x7fb16406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 -- 172.21.15.106:0/2773080534 shutdown_connections 2024-04-13T03:19:08.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.373+0000 7fb16911e640 1 -- 172.21.15.106:0/2773080534 wait complete. 2024-04-13T03:19:08.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.374+0000 7fb16911e640 1 Processor -- start 2024-04-13T03:19:08.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.374+0000 7fb16911e640 1 -- start start 2024-04-13T03:19:08.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 0x7fb16415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb164144f10 0x7fb16413fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb16406c150 con 0x7fb1640caf30 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640d0fd0 con 0x7fb1640d0bc0 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16911e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb16406c2d0 con 0x7fb164144f10 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57026/0 (socket says 172.21.15.106:57026) 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 -- 172.21.15.106:0/212662796 learned_addr learned my addr 172.21.15.106:0/212662796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16259c640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 0x7fb16415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 -- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb164144f10 msgr2=0x7fb16413fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16359e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb164144f10 0x7fb16413fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb164144f10 0x7fb16413fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 -- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 msgr2=0x7fb16415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 0x7fb16415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 -- 172.21.15.106:0/212662796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16415f780 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb16259c640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 0x7fb16415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.375+0000 7fb162d9d640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c7c0 tx=0x7fb15400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.376+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154018070 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.376+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15400a640 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.376+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16415fa60 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.376+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400aa30 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.376+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb164160020 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.377+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb164068a10 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.377+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb15401b070 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.377+0000 7fb14b7fe640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 0x7fb15006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.377+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7fb15408aff0 con 0x7fb1640caf30 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.377+0000 7fb16259c640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 0x7fb15006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.378+0000 7fb16259c640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 0x7fb15006edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb158036000 tx=0x7fb158039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.382+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1540595d0 con 0x7fb1640caf30 2024-04-13T03:19:08.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.560+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1640cb720 con 0x7fb1640caf30 2024-04-13T03:19:08.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.561+0000 7fb14b7fe640 1 -- 172.21.15.106:0/212662796 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+13473 (secure 0 0 0) 0x7fb154058ed0 con 0x7fb1640caf30 2024-04-13T03:19:08.568 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:08.568 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":113,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:07.249122+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6833/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6832/3141313499":"2024-04-14T03:18:59.467453+0000","172.21.15.114:6835/2645971931":"2024-04-14T03:18:23.121696+0000","172.21.15.114:6834/2645971931":"2024-04-14T03:18:23.121696+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 msgr2=0x7fb15006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 0x7fb15006edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb158036000 tx=0x7fb158039040 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 msgr2=0x7fb16415f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c7c0 tx=0x7fb15400cc80 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 shutdown_connections 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb15006c8f0 0x7fb15006edb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb164144f10 0x7fb16413fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1640d0bc0 0x7fb16415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 --2- 172.21.15.106:0/212662796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1640caf30 0x7fb16415f0f0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 >> 172.21.15.106:0/212662796 conn(0x7fb16406f180 msgr2=0x7fb1641434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 shutdown_connections 2024-04-13T03:19:08.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.563+0000 7fb16911e640 1 -- 172.21.15.106:0/212662796 wait complete. 2024-04-13T03:19:08.576 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:08.576 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.679+0000 7fc004b5e640 1 Processor -- start 2024-04-13T03:19:08.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.679+0000 7fc004b5e640 1 -- start start 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fc004b5e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc000160400 0x7fc00006c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fc004b5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 0x7fc0000704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fc004b5e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc000160890 con 0x7fc00013f510 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fc004b5e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc000160b70 con 0x7fc000160400 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fc004b5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc000160e50 con 0x7fc00013f8f0 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fbffe59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc00013f510 0x7fc00015cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49824/0 (socket says 172.21.15.106:49824) 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.680+0000 7fbffe59c640 1 -- 172.21.15.106:0/2622765363 learned_addr learned my addr 172.21.15.106:0/2622765363 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffed9d640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 0x7fc0000704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/2622765363 >> v2:172.21.15.114:3300/0 conn(0x7fc000160400 0x7fc00006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 329730117 0 0) 0x7fc000160890 con 0x7fc00013f510 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe8003520 con 0x7fc00013f510 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1435960887 0 0) 0x7fbfe8003520 con 0x7fc00013f510 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffdd9b640 1 -- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 msgr2=0x7fc0000704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 0x7fc0000704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffdd9b640 1 -- 172.21.15.106:0/2622765363 >> v1:172.21.15.106:6789/0 conn(0x7fc00013f510 legacy=0x7fc00015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffdd9b640 1 -- 172.21.15.106:0/2622765363 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000161130 con 0x7fc000160400 2024-04-13T03:19:08.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.681+0000 7fbffed9d640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 0x7fc0000704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:08.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.682+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000160400 0x7fc00006c290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9c3c4cc20597c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.682+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037070 con 0x7fc000160400 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.682+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff4030510 con 0x7fc000160400 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.682+0000 7fbffd59a640 1 -- 172.21.15.106:0/2622765363 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0001670 con 0x7fc000160400 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.682+0000 7fc004b5e640 1 -- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000160400 msgr2=0x7fc00006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000160400 0x7fc00006c290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b790 comp rx=0 tx=0).stop 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 -- 172.21.15.106:0/2622765363 shutdown_connections 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f8f0 0x7fc0000704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 --2- 172.21.15.106:0/2622765363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000160400 0x7fc00006c290 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 -- 172.21.15.106:0/2622765363 >> 172.21.15.106:0/2622765363 conn(0x7fc0000d13f0 msgr2=0x7fc0000d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 -- 172.21.15.106:0/2622765363 shutdown_connections 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.683+0000 7fc004b5e640 1 -- 172.21.15.106:0/2622765363 wait complete. 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.684+0000 7fc004b5e640 1 Processor -- start 2024-04-13T03:19:08.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.684+0000 7fc004b5e640 1 -- start start 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 0x7fc000140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000140860 0x7fc00015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc000070bb0 con 0x7fc00013f8f0 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc000070a30 con 0x7fc000140860 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fc004b5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc000070d30 con 0x7fc00013f510 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57028/0 (socket says 172.21.15.106:57028) 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 -- 172.21.15.106:0/3903069673 learned_addr learned my addr 172.21.15.106:0/3903069673 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffed9d640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000140860 0x7fc00015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffe59c640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 0x7fc000140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 -- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 msgr2=0x7fc000140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 0x7fc000140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 -- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000140860 msgr2=0x7fc00015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000140860 0x7fc00015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 -- 172.21.15.106:0/3903069673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000132b30 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffe59c640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 0x7fc000140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbffdd9b640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbff402b7d0 tx=0x7fbff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.685+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4042020 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.686+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff40044c0 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.686+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc000132e10 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.686+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037040 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.686+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001333d0 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.687+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc000069120 con 0x7fc00013f8f0 2024-04-13T03:19:08.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.687+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbff4045020 con 0x7fc00013f8f0 2024-04-13T03:19:08.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.687+0000 7fbfeeffd640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.688+0000 7fbffe59c640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.688+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7fbff40ad5c0 con 0x7fc00013f8f0 2024-04-13T03:19:08.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.688+0000 7fbffe59c640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbff00011e0 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.692+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbff404c030 con 0x7fc00013f8f0 2024-04-13T03:19:08.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.865+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc000071160 con 0x7fc00013f8f0 2024-04-13T03:19:08.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.865+0000 7fbfeeffd640 1 -- 172.21.15.106:0/3903069673 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fbff407bba0 con 0x7fc00013f8f0 2024-04-13T03:19:08.867 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 msgr2=0x7fbfe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbff00011e0 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 msgr2=0x7fc00015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbff402b7d0 tx=0x7fbff40046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 shutdown_connections 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000140860 0x7fc00015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc00013f8f0 0x7fc00015b8e0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 --2- 172.21.15.106:0/3903069673 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc00013f510 0x7fc000140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 >> 172.21.15.106:0/3903069673 conn(0x7fc0000d13f0 msgr2=0x7fc0000d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 shutdown_connections 2024-04-13T03:19:08.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.867+0000 7fc004b5e640 1 -- 172.21.15.106:0/3903069673 wait complete. 2024-04-13T03:19:08.880 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-13T03:19:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.978+0000 7f7bb3e88640 1 Processor -- start 2024-04-13T03:19:08.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.978+0000 7f7bb3e88640 1 -- start start 2024-04-13T03:19:08.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb3e88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac1622d0 0x7f7bac0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb3e88640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bac0b8080 0x7f7bac1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb3e88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bac0d3530 con 0x7f7bac0d1c00 2024-04-13T03:19:08.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb3e88640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7bac1626b0 con 0x7f7bac0b8080 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb3e88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac0d0b60 con 0x7f7bac1622d0 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb2425640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bac0b8080 0x7f7bac1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb1423640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7bac0d1c00 0x7f7bac0b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49834/0 (socket says 172.21.15.106:49834) 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb1423640 1 -- 172.21.15.106:0/2304476987 learned_addr learned my addr 172.21.15.106:0/2304476987 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb1c24640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac1622d0 0x7f7bac0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb2425640 1 -- 172.21.15.106:0/2304476987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac1622d0 msgr2=0x7f7bac0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb2425640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac1622d0 0x7f7bac0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb2425640 1 -- 172.21.15.106:0/2304476987 >> v1:172.21.15.106:6789/0 conn(0x7f7bac0d1c00 legacy=0x7f7bac0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.979+0000 7f7bb2425640 1 -- 172.21.15.106:0/2304476987 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac0d0e40 con 0x7f7bac0b8080 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb2425640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0b8080 0x7f7bac1688e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0097a0 tx=0x7f7b9c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed74416323e843 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb0c22640 1 -- 172.21.15.106:0/2304476987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c037070 con 0x7f7bac0b8080 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb0c22640 1 -- 172.21.15.106:0/2304476987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c030710 con 0x7f7bac0b8080 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb0c22640 1 -- 172.21.15.106:0/2304476987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c030ae0 con 0x7f7bac0b8080 2024-04-13T03:19:08.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 -- 172.21.15.106:0/2304476987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0b8080 msgr2=0x7f7bac1688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0b8080 0x7f7bac1688e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0097a0 tx=0x7f7b9c02b710 comp rx=0 tx=0).stop 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 -- 172.21.15.106:0/2304476987 shutdown_connections 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0b8080 0x7f7bac1688e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/2304476987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac1622d0 0x7f7bac0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.980+0000 7f7bb3e88640 1 -- 172.21.15.106:0/2304476987 >> 172.21.15.106:0/2304476987 conn(0x7f7bac0d25d0 msgr2=0x7f7bac0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.981+0000 7f7bb3e88640 1 -- 172.21.15.106:0/2304476987 shutdown_connections 2024-04-13T03:19:08.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.981+0000 7f7bb3e88640 1 -- 172.21.15.106:0/2304476987 wait complete. 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 Processor -- start 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 -- start start 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 0x7f7bac14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bac1622d0 0x7f7bac161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac0d16e0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac0d1400 con 0x7f7bac1622d0 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb3e88640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac0cc6f0 con 0x7f7bac0b8080 2024-04-13T03:19:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.982+0000 7f7bb1423640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57058/0 (socket says 172.21.15.106:57058) 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 -- 172.21.15.106:0/3692689335 learned_addr learned my addr 172.21.15.106:0/3692689335 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1c24640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 0x7f7bac14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb2425640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bac1622d0 0x7f7bac161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 -- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 msgr2=0x7f7bac14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 0x7f7bac14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 -- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bac1622d0 msgr2=0x7f7bac161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bac1622d0 0x7f7bac161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 -- 172.21.15.106:0/3692689335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac14c0f0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1c24640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 0x7f7bac14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb1423640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c780 tx=0x7f7ba000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0018070 con 0x7f7bac0d1c00 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00072d0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00047e0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac14c3d0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.983+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac14c990 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.984+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac068a10 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.984+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ba001b070 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.985+0000 7f7baa7fc640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 0x7f7b9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.985+0000 7f7bb1c24640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 0x7f7b9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.985+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f7ba008aea0 con 0x7f7bac0d1c00 2024-04-13T03:19:08.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.985+0000 7f7bb1c24640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 0x7f7b9806edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009a80 tx=0x7f7ba40039d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:08.989+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ba0059480 con 0x7f7bac0d1c00 2024-04-13T03:19:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.233+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f7bac0b5570 con 0x7f7bac0d1c00 2024-04-13T03:19:09.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.262+0000 7f7baa7fc640 1 -- 172.21.15.106:0/3692689335 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v455) ==== 147+0+0 (secure 0 0 0) 0x7f7ba0058d80 con 0x7f7bac0d1c00 2024-04-13T03:19:09.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 msgr2=0x7f7b9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 0x7f7b9806edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009a80 tx=0x7f7ba40039d0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 msgr2=0x7f7bac14b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c780 tx=0x7f7ba000cc40 comp rx=0 tx=0).stop 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 shutdown_connections 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b9806c8f0 0x7f7b9806edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bac1622d0 0x7f7bac161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac0d1c00 0x7f7bac14b7f0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 --2- 172.21.15.106:0/3692689335 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7bac0b8080 0x7f7bac14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 >> 172.21.15.106:0/3692689335 conn(0x7f7bac0d25d0 msgr2=0x7f7bac0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 shutdown_connections 2024-04-13T03:19:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.264+0000 7f7bb3e88640 1 -- 172.21.15.106:0/3692689335 wait complete. 2024-04-13T03:19:09.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-13T03:19:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.380+0000 7f7b885bb640 1 Processor -- start 2024-04-13T03:19:09.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.380+0000 7f7b885bb640 1 -- start start 2024-04-13T03:19:09.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b885bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7b80160400 0x7f7b8006c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b885bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b8013f8f0 0x7f7b800704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b885bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b80160890 con 0x7f7b8013f510 2024-04-13T03:19:09.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b885bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7b80160b70 con 0x7f7b8013f8f0 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b885bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7b80160e50 con 0x7f7b80160400 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b86b58640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7b8013f8f0 0x7f7b800704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b86357640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7b8013f510 0x7f7b8015cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49836/0 (socket says 172.21.15.106:49836) 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.381+0000 7f7b86357640 1 -- 172.21.15.106:0/1104739872 learned_addr learned my addr 172.21.15.106:0/1104739872 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b85b56640 1 --2- 172.21.15.106:0/1104739872 >> v2:172.21.15.114:3300/0 conn(0x7f7b80160400 0x7f7b8006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b86b58640 1 -- 172.21.15.106:0/1104739872 >> v2:172.21.15.114:3300/0 conn(0x7f7b80160400 msgr2=0x7f7b8006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b86b58640 1 --2- 172.21.15.106:0/1104739872 >> v2:172.21.15.114:3300/0 conn(0x7f7b80160400 0x7f7b8006c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b86b58640 1 -- 172.21.15.106:0/1104739872 >> v1:172.21.15.106:6789/0 conn(0x7f7b8013f510 legacy=0x7f7b8015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b86b58640 1 -- 172.21.15.106:0/1104739872 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80161130 con 0x7f7b8013f8f0 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.382+0000 7f7b85355640 1 -- 172.21.15.106:0/1104739872 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699269124 0 0) 0x7f7b80160890 con 0x7f7b8013f510 2024-04-13T03:19:09.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b86b58640 1 --2- 172.21.15.106:0/1104739872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8013f8f0 0x7f7b800704f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7b800cde40 tx=0x7f7b7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77b679f0fb7dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b85355640 1 -- 172.21.15.106:0/1104739872 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78035020 con 0x7f7b8013f8f0 2024-04-13T03:19:09.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b85355640 1 -- 172.21.15.106:0/1104739872 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b78031830 con 0x7f7b8013f8f0 2024-04-13T03:19:09.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b85355640 1 -- 172.21.15.106:0/1104739872 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78031c00 con 0x7f7b8013f8f0 2024-04-13T03:19:09.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 -- 172.21.15.106:0/1104739872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8013f8f0 msgr2=0x7f7b800704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 --2- 172.21.15.106:0/1104739872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8013f8f0 0x7f7b800704f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7b800cde40 tx=0x7f7b7802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 -- 172.21.15.106:0/1104739872 shutdown_connections 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 --2- 172.21.15.106:0/1104739872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8013f8f0 0x7f7b800704f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 --2- 172.21.15.106:0/1104739872 >> v2:172.21.15.114:3300/0 conn(0x7f7b80160400 0x7f7b8006c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 -- 172.21.15.106:0/1104739872 >> 172.21.15.106:0/1104739872 conn(0x7f7b800d13f0 msgr2=0x7f7b800d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 -- 172.21.15.106:0/1104739872 shutdown_connections 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.383+0000 7f7b885bb640 1 -- 172.21.15.106:0/1104739872 wait complete. 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.384+0000 7f7b885bb640 1 Processor -- start 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 -- start start 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b8013f510 0x7f7b800731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 0x7f7b8015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b80070bb0 con 0x7f7b8015bfd0 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b80070a30 con 0x7f7b8013f8f0 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b885bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f7b80070d30 con 0x7f7b8013f510 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b86b58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b86b58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57086/0 (socket says 172.21.15.106:57086) 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.385+0000 7f7b86b58640 1 -- 172.21.15.106:0/790717546 learned_addr learned my addr 172.21.15.106:0/790717546 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b85b56640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 0x7f7b8015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 -- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b8013f510 msgr2=0x7f7b800731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b8013f510 0x7f7b800731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 -- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 msgr2=0x7f7b8015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 0x7f7b8015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 -- 172.21.15.106:0/790717546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b801486d0 con 0x7f7b8015bfd0 2024-04-13T03:19:09.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b85b56640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 0x7f7b8015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b86b58640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009860 tx=0x7f7b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78041070 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7803a8f0 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78035040 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b801489b0 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.386+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b80073700 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.387+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b80069120 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.387+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b78038020 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.388+0000 7f7b76ffd640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 0x7f7b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.388+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f7b780ad3b0 con 0x7f7b8015bfd0 2024-04-13T03:19:09.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.388+0000 7f7b86357640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 0x7f7b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.388+0000 7f7b86357640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 0x7f7b6406edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c007fd0 tx=0x7f7b7c007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.392+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b7807b990 con 0x7f7b8015bfd0 2024-04-13T03:19:09.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.569+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7b80071160 con 0x7f7b8015bfd0 2024-04-13T03:19:09.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.572+0000 7f7b76ffd640 1 -- 172.21.15.106:0/790717546 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f7b780abe20 con 0x7f7b8015bfd0 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.574+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 msgr2=0x7f7b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.574+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 0x7f7b6406edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c007fd0 tx=0x7f7b7c007ce0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.574+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 msgr2=0x7f7b801f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.574+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009860 tx=0x7f7b780046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 shutdown_connections 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7b6406c8f0 0x7f7b6406edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b8015bfd0 0x7f7b801f26e0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b8013f8f0 0x7f7b8015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 --2- 172.21.15.106:0/790717546 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7b8013f510 0x7f7b800731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 >> 172.21.15.106:0/790717546 conn(0x7f7b800d13f0 msgr2=0x7f7b800b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 shutdown_connections 2024-04-13T03:19:09.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.575+0000 7f7b885bb640 1 -- 172.21.15.106:0/790717546 wait complete. 2024-04-13T03:19:09.587 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:19:09.587 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:19:09.638 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:19:09.682 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:19:09.682 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:09.682 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:09.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:09.699 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:09.699 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:09.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:09.739 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:09.739 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:09.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:09.754 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:09.754 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:09.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:09.796 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-13T03:19:09.796 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:09.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.894+0000 7f1630d5e640 1 Processor -- start 2024-04-13T03:19:09.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.894+0000 7f1630d5e640 1 -- start start 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.895+0000 7f1630d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 0x7f162c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.895+0000 7f1630d5e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.895+0000 7f1630d5e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c161050 con 0x7f162c156b50 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.895+0000 7f1630d5e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f162c15d690 con 0x7f162c06e2b0 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.895+0000 7f1630d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f162c15d970 con 0x7f162c160c10 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f1629d9b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 0x7f162c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162ad9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162ad9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:34192/0 (socket says 172.21.15.106:34192) 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162a59c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f162c156b50 0x7f162c15cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:39630/0 (socket says 172.21.15.106:39630) 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162ad9d640 1 -- 172.21.15.106:0/3816383604 learned_addr learned my addr 172.21.15.106:0/3816383604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162959a640 1 -- 172.21.15.106:0/3816383604 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850780788 0 0) 0x7f161c001490 con 0x7f162c156b50 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162959a640 1 -- 172.21.15.106:0/3816383604 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1614003520 con 0x7f162c156b50 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f1629d9b640 1 -- 172.21.15.106:0/3816383604 >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 msgr2=0x7f162c06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f1629d9b640 1 --2- 172.21.15.106:0/3816383604 >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f1629d9b640 1 -- 172.21.15.106:0/3816383604 >> v1:172.21.15.114:6789/0 conn(0x7f162c156b50 legacy=0x7f162c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f1629d9b640 1 -- 172.21.15.106:0/3816383604 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c160410 con 0x7f162c160c10 2024-04-13T03:19:09.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.896+0000 7f162ad9d640 1 --2- 172.21.15.106:0/3816383604 >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.897+0000 7f1629d9b640 1 --2- 172.21.15.106:0/3816383604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 0x7f162c06dd70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16200097a0 tx=0x7f162002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4081bf130a874c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.897+0000 7f162959a640 1 -- 172.21.15.106:0/3816383604 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620035020 con 0x7f162c160c10 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.897+0000 7f162959a640 1 -- 172.21.15.106:0/3816383604 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620031830 con 0x7f162c160c10 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.897+0000 7f162959a640 1 -- 172.21.15.106:0/3816383604 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620031be0 con 0x7f162c160c10 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 -- 172.21.15.106:0/3816383604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 msgr2=0x7f162c06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 --2- 172.21.15.106:0/3816383604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 0x7f162c06dd70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16200097a0 tx=0x7f162002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 -- 172.21.15.106:0/3816383604 shutdown_connections 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 --2- 172.21.15.106:0/3816383604 >> v2:172.21.15.114:3300/0 conn(0x7f162c06e2b0 0x7f162c06bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 --2- 172.21.15.106:0/3816383604 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c160c10 0x7f162c06dd70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 -- 172.21.15.106:0/3816383604 >> 172.21.15.106:0/3816383604 conn(0x7f162c0b1f10 msgr2=0x7f162c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 -- 172.21.15.106:0/3816383604 shutdown_connections 2024-04-13T03:19:09.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.898+0000 7f1630d5e640 1 -- 172.21.15.106:0/3816383604 wait complete. 2024-04-13T03:19:09.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.899+0000 7f1630d5e640 1 Processor -- start 2024-04-13T03:19:09.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.899+0000 7f1630d5e640 1 -- start start 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c06e2b0 0x7f162c149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f162c156b50 0x7f162c1e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f162c156f30 con 0x7f162c160c10 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f162c15d320 con 0x7f162c156b50 2024-04-13T03:19:09.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0c8480 con 0x7f162c06e2b0 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57090/0 (socket says 172.21.15.106:57090) 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 -- 172.21.15.106:0/1402303027 learned_addr learned my addr 172.21.15.106:0/1402303027 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162a59c640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c06e2b0 0x7f162c149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1629d9b640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f162c156b50 0x7f162c1e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 -- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c06e2b0 msgr2=0x7f162c149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c06e2b0 0x7f162c149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 -- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f162c156b50 msgr2=0x7f162c1e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f162c156b50 0x7f162c1e6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 -- 172.21.15.106:0/1402303027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c0c8760 con 0x7f162c160c10 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f162ad9d640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f162400e890 tx=0x7f162400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624017070 con 0x7f162c160c10 2024-04-13T03:19:09.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1624004020 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.900+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c0c8a40 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.901+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240043f0 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.901+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c0c9000 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.902+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c068a10 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.902+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f162400c8e0 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.902+0000 7f161affd640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 0x7f160c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.902+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f162408bab0 con 0x7f162c160c10 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.902+0000 7f162a59c640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 0x7f160c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.903+0000 7f162a59c640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 0x7f160c06edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f161c001630 tx=0x7f161c007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:09.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:09.907+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16240599b0 con 0x7f162c160c10 2024-04-13T03:19:10.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.118+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c06ffd0 con 0x7f162c160c10 2024-04-13T03:19:10.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.118+0000 7f161affd640 1 -- 172.21.15.106:0/1402303027 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+3881 (secure 0 0 0) 0x7f162405dcb0 con 0x7f162c160c10 2024-04-13T03:19:10.120 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:10.121 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4135742568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4135742568},{"type":"v1","addr":"172.21.15.114:6835","nonce":4135742568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/3179085184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3179085184},{"type":"v1","addr":"172.21.15.114:6833","nonce":3179085184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2755848132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2755848132},{"type":"v1","addr":"172.21.15.106:6835","nonce":2755848132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2368128023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2368128023},{"type":"v1","addr":"172.21.15.106:6837","nonce":2368128023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[]} 2024-04-13T03:19:10.121 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 81 2024-04-13T03:19:10.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 msgr2=0x7f160c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 0x7f160c06edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f161c001630 tx=0x7f161c007eb0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 msgr2=0x7f162c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f162400e890 tx=0x7f162400ed50 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 shutdown_connections 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f160c06c8f0 0x7f160c06edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f162c160c10 0x7f162c072d50 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f162c156b50 0x7f162c1e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 --2- 172.21.15.106:0/1402303027 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f162c06e2b0 0x7f162c149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 >> 172.21.15.106:0/1402303027 conn(0x7f162c0b1f10 msgr2=0x7f162c15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 shutdown_connections 2024-04-13T03:19:10.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.120+0000 7f1630d5e640 1 -- 172.21.15.106:0/1402303027 wait complete. 2024-04-13T03:19:10.134 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-13T03:19:10.135 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-13T03:19:10.135 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:19:10.135 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:19:10.135 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-13T03:19:10.135 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-13T03:19:10.135 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:19:10.135 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:19:10.136 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-13T03:19:10.136 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-13T03:19:10.136 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:19:10.136 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:19:10.136 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-13T03:19:10.136 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-13T03:19:10.136 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:19:10.137 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:19:10.144 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:19:10.144 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-13T03:19:10.145 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:19:10.145 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-13T03:19:10.147 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:19:10.147 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-13T03:19:10.148 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:19:10.148 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-13T03:19:10.187 INFO:tasks.ceph.mds.a:Started 2024-04-13T03:19:10.187 DEBUG:teuthology.parallel:result is None 2024-04-13T03:19:10.188 INFO:tasks.ceph.mds.c:Started 2024-04-13T03:19:10.188 DEBUG:teuthology.parallel:result is None 2024-04-13T03:19:10.191 INFO:tasks.ceph.mds.b:Started 2024-04-13T03:19:10.191 DEBUG:teuthology.parallel:result is None 2024-04-13T03:19:10.192 INFO:tasks.ceph.mds.d:Started 2024-04-13T03:19:10.192 DEBUG:teuthology.parallel:result is None 2024-04-13T03:19:10.192 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-13T03:19:10.263 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-13T03:19:10.264 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:19:10.274 INFO:tasks.ceph.mds.b.smithi114.stdout:starting mds.b at 2024-04-13T03:19:10.274 INFO:tasks.ceph.mds.d.smithi114.stdout:starting mds.d at 2024-04-13T03:19:10.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.295+0000 7f5bb331e640 1 Processor -- start 2024-04-13T03:19:10.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.295+0000 7f5bb331e640 1 -- start start 2024-04-13T03:19:10.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb331e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bac0d0b40 0x7f5bac0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:10.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb331e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bac1635a0 con 0x7f5bac0b53c0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb331e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bac0cccb0 con 0x7f5bac15fbb0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb331e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5bac0b5d60 con 0x7f5bac0d0b40 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb08b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5bac0b53c0 0x7f5bac0d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49862/0 (socket says 172.21.15.106:49862) 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb08b9640 1 -- 172.21.15.106:0/2497059114 learned_addr learned my addr 172.21.15.106:0/2497059114 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5bb18bb640 1 --2- 172.21.15.106:0/2497059114 >> v2:172.21.15.114:3300/0 conn(0x7f5bac0d0b40 0x7f5bac0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544976800 0 0) 0x7f5bac1635a0 con 0x7f5bac0b53c0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.296+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b98003520 con 0x7f5bac0b53c0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596446165 0 0) 0x7f5bac0cccb0 con 0x7f5bac15fbb0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bac1635a0 con 0x7f5bac15fbb0 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5bb18bb640 1 -- 172.21.15.106:0/2497059114 >> v1:172.21.15.114:6789/0 conn(0x7f5bac15fbb0 legacy=0x7f5bac0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5bb18bb640 1 -- 172.21.15.106:0/2497059114 >> v1:172.21.15.106:6789/0 conn(0x7f5bac0b53c0 legacy=0x7f5bac0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5bb18bb640 1 -- 172.21.15.106:0/2497059114 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac1632c0 con 0x7f5bac0d0b40 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.297+0000 7f5bb18bb640 1 --2- 172.21.15.106:0/2497059114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0d0b40 0x7f5bac0b5820 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009d90 tx=0x7f5ba402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f793906c1450d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4039070 con 0x7f5bac0d0b40 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba402cc60 con 0x7f5bac0d0b40 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5b9ffff640 1 -- 172.21.15.106:0/2497059114 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4034660 con 0x7f5bac0d0b40 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 -- 172.21.15.106:0/2497059114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0d0b40 msgr2=0x7f5bac0b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 --2- 172.21.15.106:0/2497059114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0d0b40 0x7f5bac0b5820 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009d90 tx=0x7f5ba402beb0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 -- 172.21.15.106:0/2497059114 shutdown_connections 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 --2- 172.21.15.106:0/2497059114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0d0b40 0x7f5bac0b5820 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 -- 172.21.15.106:0/2497059114 >> 172.21.15.106:0/2497059114 conn(0x7f5bac0d2530 msgr2=0x7f5bac0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 -- 172.21.15.106:0/2497059114 shutdown_connections 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.298+0000 7f5bb331e640 1 -- 172.21.15.106:0/2497059114 wait complete. 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.299+0000 7f5bb331e640 1 Processor -- start 2024-04-13T03:19:10.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.299+0000 7f5bb331e640 1 -- start start 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0b53c0 0x7f5bac1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5bac15fbb0 0x7f5bac06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac0cde60 con 0x7f5bac0d0b40 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac0cd2c0 con 0x7f5bac0b53c0 2024-04-13T03:19:10.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb331e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5bac15ff90 con 0x7f5bac15fbb0 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57114/0 (socket says 172.21.15.106:57114) 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 -- 172.21.15.106:0/1780586861 learned_addr learned my addr 172.21.15.106:0/1780586861 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb10ba640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0b53c0 0x7f5bac1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb18bb640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5bac15fbb0 0x7f5bac06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 -- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5bac15fbb0 msgr2=0x7f5bac06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5bac15fbb0 0x7f5bac06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 -- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0b53c0 msgr2=0x7f5bac1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0b53c0 0x7f5bac1f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.300+0000 7f5bb08b9640 1 -- 172.21.15.106:0/1780586861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac06ddf0 con 0x7f5bac0d0b40 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5bb08b9640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c410 tx=0x7f5ba000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0017070 con 0x7f5bac0d0b40 2024-04-13T03:19:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0005ce0 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac06e0d0 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0007310 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.301+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac06e690 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.302+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac069120 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.302+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ba001a070 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.303+0000 7f5b9dffb640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 0x7f5b9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.303+0000 7f5bb10ba640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 0x7f5b9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.303+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f5ba008ace0 con 0x7f5bac0d0b40 2024-04-13T03:19:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.303+0000 7f5bb10ba640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 0x7f5b9006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8007fd0 tx=0x7f5ba8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.308+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ba0058d30 con 0x7f5bac0d0b40 2024-04-13T03:19:10.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:10.498+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5bac0d0430 con 0x7f5bac0d0b40 2024-04-13T03:19:11.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.272+0000 7f5b9dffb640 1 -- 172.21.15.106:0/1780586861 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v115) ==== 97+0+0 (secure 0 0 0) 0x7f5ba005d030 con 0x7f5bac0d0b40 2024-04-13T03:19:11.274 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 msgr2=0x7f5b9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 0x7f5b9006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8007fd0 tx=0x7f5ba8009040 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 msgr2=0x7f5bac06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c410 tx=0x7f5ba000c8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 shutdown_connections 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5b9006c8f0 0x7f5b9006edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5bac15fbb0 0x7f5bac06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5bac0d0b40 0x7f5bac06d280 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 --2- 172.21.15.106:0/1780586861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bac0b53c0 0x7f5bac1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 >> 172.21.15.106:0/1780586861 conn(0x7f5bac0d2530 msgr2=0x7f5bac161690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 shutdown_connections 2024-04-13T03:19:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.276+0000 7f5bb331e640 1 -- 172.21.15.106:0/1780586861 wait complete. 2024-04-13T03:19:11.291 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-13T03:19:11.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.391+0000 7fc8d7d1e640 1 Processor -- start 2024-04-13T03:19:11.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.391+0000 7fc8d7d1e640 1 -- start start 2024-04-13T03:19:11.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d7d1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc8d01622d0 0x7fc8d00d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d7d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8d01688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d7d1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d00d3530 con 0x7fc8d00d1c00 2024-04-13T03:19:11.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d7d1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d01626b0 con 0x7fc8d01622d0 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d7d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00d0b60 con 0x7fc8d00b8080 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d52b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc8d00d1c00 0x7fc8d00b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49872/0 (socket says 172.21.15.106:49872) 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d52b9640 1 -- 172.21.15.106:0/375494824 learned_addr learned my addr 172.21.15.106:0/375494824 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8d01688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/375494824 >> v2:172.21.15.114:3300/0 conn(0x7fc8d01622d0 0x7fc8d00d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867590798 0 0) 0x7fc8d00d3530 con 0x7fc8d00d1c00 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.392+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8bc003520 con 0x7fc8d00d1c00 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119885608 0 0) 0x7fc8bc003520 con 0x7fc8d00d1c00 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 msgr2=0x7fc8d01688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8d01688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 >> v2:172.21.15.114:3300/0 conn(0x7fc8d01622d0 msgr2=0x7fc8d00d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 --2- 172.21.15.106:0/375494824 >> v2:172.21.15.114:3300/0 conn(0x7fc8d01622d0 0x7fc8d00d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d00d0e40 con 0x7fc8d00d1c00 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fc8c0001670 con 0x7fc8d00d1c00 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 >> v1:172.21.15.106:6789/0 conn(0x7fc8d00d1c00 legacy=0x7fc8d00b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8bc0039b0 0x7fc8bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc004800 0x7fc8bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00d0b60 con 0x7fc8bc0039b0 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc00b9c0 con 0x7fc8bc004800 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc00bb40 con 0x7fc8d00b8080 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8bc0039b0 0x7fc8bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d52b9640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc004800 0x7fc8bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 -- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 msgr2=0x7fc8bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 -- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc004800 msgr2=0x7fc8bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc004800 0x7fc8bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d5aba640 1 -- 172.21.15.106:0/375494824 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc003520 con 0x7fc8bc0039b0 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d00b8080 0x7fc8bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.393+0000 7fc8d52b9640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc004800 0x7fc8bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8bc0039b0 0x7fc8bc003dd0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009ac0 tx=0x7fc8bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4714dd25b9336cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc039070 con 0x7fc8bc0039b0 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8cc02c710 con 0x7fc8bc0039b0 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d4ab8640 1 -- 172.21.15.106:0/375494824 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc009690 con 0x7fc8bc0039b0 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/375494824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8bc0039b0 msgr2=0x7fc8bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/375494824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8bc0039b0 0x7fc8bc003dd0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009ac0 tx=0x7fc8bc005830 comp rx=0 tx=0).stop 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d62bb640 1 -- 172.21.15.106:0/375494824 reap_dead start 2024-04-13T03:19:11.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/375494824 shutdown_connections 2024-04-13T03:19:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/375494824 >> 172.21.15.106:0/375494824 conn(0x7fc8d00d25d0 msgr2=0x7fc8d00d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.394+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/375494824 shutdown_connections 2024-04-13T03:19:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.395+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/375494824 wait complete. 2024-04-13T03:19:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 Processor -- start 2024-04-13T03:19:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 -- start start 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc0039b0 0x7fc8d0161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d015eb30 0x7fc8d015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00d16e0 con 0x7fc8d00b2820 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00d1400 con 0x7fc8bc0039b0 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d7d1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00cc6f0 con 0x7fc8d015eb30 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d62bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d62bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57140/0 (socket says 172.21.15.106:57140) 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.396+0000 7fc8d62bb640 1 -- 172.21.15.106:0/2061319426 learned_addr learned my addr 172.21.15.106:0/2061319426 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d52b9640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d015eb30 0x7fc8d015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc0039b0 0x7fc8d0161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 -- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d015eb30 msgr2=0x7fc8d015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d015eb30 0x7fc8d015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 -- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc0039b0 msgr2=0x7fc8d0161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc0039b0 0x7fc8d0161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 -- 172.21.15.106:0/2061319426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d015f2a0 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d62bb640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1eb0 tx=0x7fc8c8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c801a030 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c8003990 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d015f580 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8003d80 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.397+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d015fb40 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.398+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d0068a10 con 0x7fc8d00b2820 2024-04-13T03:19:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.398+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8c801d070 con 0x7fc8d00b2820 2024-04-13T03:19:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.399+0000 7fc8c67fc640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 0x7fc8b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.399+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7fc8c808c4d0 con 0x7fc8d00b2820 2024-04-13T03:19:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.399+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 0x7fc8b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.399+0000 7fc8d5aba640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 0x7fc8b406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036000 tx=0x7fc8cc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.403+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8c805ab50 con 0x7fc8d00b2820 2024-04-13T03:19:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.645+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc8d00b1c90 con 0x7fc8d00b2820 2024-04-13T03:19:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.646+0000 7fc8c67fc640 1 -- 172.21.15.106:0/2061319426 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fc8c805a450 con 0x7fc8d00b2820 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlm+RJ1GhAA8iviLUcnk+VSX+hu6Lakjw==", 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.648 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlml5UJJxAAccIM3hYiuyBHmUpb5P6Cqg==", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2024-04-13T03:19:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmt6oBIhAA0fNp0aLP+7wKNSur0VXTFA==", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmugJvLhAAd9K5zQMV40T6XFRW+PVLKw==", 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.650 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAu9xlmsZZzFhAARiitv/NBovfuaz9iMk0Cvg==", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.651 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAx9xlmlj91FBAAdptwjLK5SYdOzggcbFQp7A==", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA09xlmwGlCEhAA4EbRvnp4DvFieEIYREdzMg==", 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.652 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.653 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.653 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.653 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.653 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA39xlmQ356EBAAJC8P9kwI7EHF3lpUDt2suQ==", 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2024-04-13T03:19:11.654 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQA99xlmEO99FhAAw4l7TYk1obLT6EcbA62Irw==", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBA9xlmlHQWFRAAssRoBSHdn3deaduJyZK3HA==", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.655 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBD9xlmAgVdExAAPOs8SLOu449zJwoOMgZ9iw==", 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2024-04-13T03:19:11.656 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQBG9xlmIOzYERAASQoXpwfeOMID/OvNenmDQw==", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw==", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.657 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmr0emABAA5x9VoxAMaUphFc2xR3puNA==", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2024-04-13T03:19:11.658 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmEKiAOBAA+m/E/I/D36JqwgP0/GYliQ==", 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.659 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAr9xlmT1N8BhAAjIJ5E3C/m1Qt2zbkpXFkTQ==", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAp9xlmk8grOBAAfsMnER6KUSDuUW4PHyLMlw==", 2024-04-13T03:19:11.660 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmwvoDFxAAetmXiTHdh7gBU6fw06qEIQ==", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.661 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQAq9xlmC5TbERAALG6hzyg2frKfZrrEeWpaXA==", 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-13T03:19:11.662 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.647+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 msgr2=0x7fc8b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.647+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 0x7fc8b406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036000 tx=0x7fc8cc030000 comp rx=0 tx=0).stop 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.647+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 msgr2=0x7fc8d00b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.647+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1eb0 tx=0x7fc8c8010590 comp rx=0 tx=0).stop 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.647+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 shutdown_connections 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc8b406c8f0 0x7fc8b406edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc8d015eb30 0x7fc8d015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8d00b2820 0x7fc8d00b2ca0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 --2- 172.21.15.106:0/2061319426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8bc0039b0 0x7fc8d0161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 >> 172.21.15.106:0/2061319426 conn(0x7fc8d00d25d0 msgr2=0x7fc8d00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:11.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 shutdown_connections 2024-04-13T03:19:11.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.648+0000 7fc8d7d1e640 1 -- 172.21.15.106:0/2061319426 wait complete. 2024-04-13T03:19:11.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:11.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 Processor -- start 2024-04-13T03:19:11.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 -- start start 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7facd0160c10 0x7facd006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facd006e2b0 0x7facd006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd015d690 con 0x7facd0156b50 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7facd006e700 con 0x7facd006e2b0 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.764+0000 7facd65bb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7facd0161080 con 0x7facd0160c10 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7faccffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7facd0156b50 0x7facd015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49886/0 (socket says 172.21.15.106:49886) 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facd4b58640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7facd006e2b0 0x7facd006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7faccffff640 1 -- 172.21.15.106:0/257075650 learned_addr learned my addr 172.21.15.106:0/257075650 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7faccf7fe640 1 --2- 172.21.15.106:0/257075650 >> v2:172.21.15.114:3300/0 conn(0x7facd0160c10 0x7facd006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facd4b58640 1 -- 172.21.15.106:0/257075650 >> v2:172.21.15.114:3300/0 conn(0x7facd0160c10 msgr2=0x7facd006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facd4b58640 1 --2- 172.21.15.106:0/257075650 >> v2:172.21.15.114:3300/0 conn(0x7facd0160c10 0x7facd006dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facd4b58640 1 -- 172.21.15.106:0/257075650 >> v1:172.21.15.106:6789/0 conn(0x7facd0156b50 legacy=0x7facd015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facd4b58640 1 -- 172.21.15.106:0/257075650 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd015d970 con 0x7facd006e2b0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.765+0000 7facceffd640 1 -- 172.21.15.106:0/257075650 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465054610 0 0) 0x7facd015d690 con 0x7facd0156b50 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd4b58640 1 --2- 172.21.15.106:0/257075650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd006e2b0 0x7facd006bcb0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7facd006d700 tx=0x7facc802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0cd0197e32fd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facceffd640 1 -- 172.21.15.106:0/257075650 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8039070 con 0x7facd006e2b0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facceffd640 1 -- 172.21.15.106:0/257075650 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc802cc80 con 0x7facd006e2b0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facceffd640 1 -- 172.21.15.106:0/257075650 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8034660 con 0x7facd006e2b0 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 -- 172.21.15.106:0/257075650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd006e2b0 msgr2=0x7facd006bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 --2- 172.21.15.106:0/257075650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd006e2b0 0x7facd006bcb0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7facd006d700 tx=0x7facc802bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 -- 172.21.15.106:0/257075650 shutdown_connections 2024-04-13T03:19:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 --2- 172.21.15.106:0/257075650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd006e2b0 0x7facd006bcb0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 --2- 172.21.15.106:0/257075650 >> v2:172.21.15.114:3300/0 conn(0x7facd0160c10 0x7facd006dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 -- 172.21.15.106:0/257075650 >> 172.21.15.106:0/257075650 conn(0x7facd00b1f10 msgr2=0x7facd00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 -- 172.21.15.106:0/257075650 shutdown_connections 2024-04-13T03:19:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.766+0000 7facd65bb640 1 -- 172.21.15.106:0/257075650 wait complete. 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.767+0000 7facd65bb640 1 Processor -- start 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 -- start start 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd006e2b0 0x7facd0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7facd0156b50 0x7facd00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7facd015d320 con 0x7facd0160c10 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7facd01606f0 con 0x7facd006e2b0 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd65bb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7facd0156f30 con 0x7facd0156b50 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd4b58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd4b58640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57172/0 (socket says 172.21.15.106:57172) 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.768+0000 7facd4b58640 1 -- 172.21.15.106:0/1125020318 learned_addr learned my addr 172.21.15.106:0/1125020318 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 -- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7facd0156b50 msgr2=0x7facd00b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7facd0156b50 0x7facd00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 -- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd006e2b0 msgr2=0x7facd0156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7faccffff640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd006e2b0 0x7facd0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd006e2b0 0x7facd0156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 -- 172.21.15.106:0/1125020318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd015f690 con 0x7facd0160c10 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd4b58640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7facc802c4e0 tx=0x7facc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8039050 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8045070 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc803e830 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd015f970 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.769+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd015ff30 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.770+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd0160170 con 0x7facd0160c10 2024-04-13T03:19:11.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.770+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facc803e9d0 con 0x7facd0160c10 2024-04-13T03:19:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.771+0000 7facccff9640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 0x7facbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.771+0000 7faccffff640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 0x7facbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.771+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7facc80ae5a0 con 0x7facd0160c10 2024-04-13T03:19:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.771+0000 7faccffff640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 0x7facbc06ede0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7facc4000f60 tx=0x7facc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.775+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facc807cc20 con 0x7facd0160c10 2024-04-13T03:19:11.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.951+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facd0068a10 con 0x7facd0160c10 2024-04-13T03:19:11.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.952+0000 7facccff9640 1 -- 172.21.15.106:0/1125020318 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+13210 (secure 0 0 0) 0x7facc807c520 con 0x7facd0160c10 2024-04-13T03:19:11.953 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:11.953 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":117,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:11.282636+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 msgr2=0x7facbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 0x7facbc06ede0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7facc4000f60 tx=0x7facc4009040 comp rx=0 tx=0).stop 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 msgr2=0x7facd00b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7facc802c4e0 tx=0x7facc8004a40 comp rx=0 tx=0).stop 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 shutdown_connections 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7facbc06c920 0x7facbc06ede0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7facd0156b50 0x7facd00b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7facd0160c10 0x7facd00b06e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 --2- 172.21.15.106:0/1125020318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd006e2b0 0x7facd0156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:11.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 >> 172.21.15.106:0/1125020318 conn(0x7facd00b1f10 msgr2=0x7facd0072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:11.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 shutdown_connections 2024-04-13T03:19:11.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:11.954+0000 7facd65bb640 1 -- 172.21.15.106:0/1125020318 wait complete. 2024-04-13T03:19:11.967 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:11.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:12.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.068+0000 7f811a19c640 1 Processor -- start 2024-04-13T03:19:12.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.068+0000 7f811a19c640 1 -- start start 2024-04-13T03:19:12.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f811a19c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f811415e8c0 0x7f81140ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f811a19c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 0x7f8114072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f811a19c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81140b5060 con 0x7f8114073350 2024-04-13T03:19:12.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f811a19c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8114073780 con 0x7f811415e8c0 2024-04-13T03:19:12.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f811a19c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f81140b5340 con 0x7f811415eca0 2024-04-13T03:19:12.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81137fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f811415e8c0 0x7f81140ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f8113fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8114073350 0x7f81140b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49902/0 (socket says 172.21.15.106:49902) 2024-04-13T03:19:12.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f8113fff640 1 -- 172.21.15.106:0/2835269342 learned_addr learned my addr 172.21.15.106:0/2835269342 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:12.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f8112ffd640 1 --2- 172.21.15.106:0/2835269342 >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 0x7f8114072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81137fe640 1 -- 172.21.15.106:0/2835269342 >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 msgr2=0x7f8114072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81137fe640 1 --2- 172.21.15.106:0/2835269342 >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 0x7f8114072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81137fe640 1 -- 172.21.15.106:0/2835269342 >> v1:172.21.15.106:6789/0 conn(0x7f8114073350 legacy=0x7f81140b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81127fc640 1 -- 172.21.15.106:0/2835269342 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534528953 0 0) 0x7f81140b5060 con 0x7f8114073350 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f81137fe640 1 -- 172.21.15.106:0/2835269342 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811406c500 con 0x7f811415e8c0 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.069+0000 7f8112ffd640 1 --2- 172.21.15.106:0/2835269342 >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 0x7f8114072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.070+0000 7f81137fe640 1 --2- 172.21.15.106:0/2835269342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f81140ce250 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8114072ae0 tx=0x7f810402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3dd87fedabdfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.070+0000 7f81127fc640 1 -- 172.21.15.106:0/2835269342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104034070 con 0x7f811415e8c0 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.070+0000 7f81127fc640 1 -- 172.21.15.106:0/2835269342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104030610 con 0x7f811415e8c0 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.070+0000 7f81127fc640 1 -- 172.21.15.106:0/2835269342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040309e0 con 0x7f811415e8c0 2024-04-13T03:19:12.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 -- 172.21.15.106:0/2835269342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 msgr2=0x7f81140ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 --2- 172.21.15.106:0/2835269342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f81140ce250 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8114072ae0 tx=0x7f810402b960 comp rx=0 tx=0).stop 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 -- 172.21.15.106:0/2835269342 shutdown_connections 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 --2- 172.21.15.106:0/2835269342 >> v2:172.21.15.114:3300/0 conn(0x7f811415eca0 0x7f8114072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 --2- 172.21.15.106:0/2835269342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f81140ce250 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 -- 172.21.15.106:0/2835269342 >> 172.21.15.106:0/2835269342 conn(0x7f811406f0d0 msgr2=0x7f811406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 -- 172.21.15.106:0/2835269342 shutdown_connections 2024-04-13T03:19:12.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.071+0000 7f811a19c640 1 -- 172.21.15.106:0/2835269342 wait complete. 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.072+0000 7f811a19c640 1 Processor -- start 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.072+0000 7f811a19c640 1 -- start start 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8114073350 0x7f8114156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f811415eca0 0x7f81140cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81140b4cc0 con 0x7f811415e8c0 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f81140b4b40 con 0x7f811415eca0 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f811a19c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f81140b4e40 con 0x7f8114073350 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57196/0 (socket says 172.21.15.106:57196) 2024-04-13T03:19:12.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 -- 172.21.15.106:0/248155982 learned_addr learned my addr 172.21.15.106:0/248155982 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f81137fe640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8114073350 0x7f8114156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8113fff640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f811415eca0 0x7f81140cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 -- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8114073350 msgr2=0x7f8114156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8114073350 0x7f8114156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 -- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f811415eca0 msgr2=0x7f81140cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f811415eca0 0x7f81140cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.073+0000 7f8112ffd640 1 -- 172.21.15.106:0/248155982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81140cabb0 con 0x7f811415e8c0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f8112ffd640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f810800c7f0 tx=0x7f810800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108015070 con 0x7f811415e8c0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108007820 con 0x7f811415e8c0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108007bd0 con 0x7f811415e8c0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81140cd350 con 0x7f811415e8c0 2024-04-13T03:19:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.074+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81140ca7f0 con 0x7f811415e8c0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.075+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81140cd140 con 0x7f811415e8c0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.075+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8108018070 con 0x7f811415e8c0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.075+0000 7f80fbfff640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 0x7f810006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.076+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f810808bb00 con 0x7f811415e8c0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.076+0000 7f81137fe640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 0x7f810006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.076+0000 7f81137fe640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 0x7f810006edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8104036000 tx=0x7f8104008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.080+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f810805a0f0 con 0x7f811415e8c0 2024-04-13T03:19:12.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.257+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8114068a10 con 0x7f811415e8c0 2024-04-13T03:19:12.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.258+0000 7f80fbfff640 1 -- 172.21.15.106:0/248155982 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) ==== 93+0+10 (secure 0 0 0) 0x7f81080599f0 con 0x7f811415e8c0 2024-04-13T03:19:12.259 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:12.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 msgr2=0x7f810006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 0x7f810006edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8104036000 tx=0x7f8104008030 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 msgr2=0x7f8114155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f810800c7f0 tx=0x7f810800ccb0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 shutdown_connections 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f810006c8f0 0x7f810006edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f811415eca0 0x7f81140cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.259+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f811415e8c0 0x7f8114155d90 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.260+0000 7f811a19c640 1 --2- 172.21.15.106:0/248155982 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8114073350 0x7f8114156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.260+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 >> 172.21.15.106:0/248155982 conn(0x7f811406f0d0 msgr2=0x7f81140cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.260+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 shutdown_connections 2024-04-13T03:19:12.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.260+0000 7f811a19c640 1 -- 172.21.15.106:0/248155982 wait complete. 2024-04-13T03:19:12.272 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T03:19:12.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.370+0000 7f61ae9cc640 1 Processor -- start 2024-04-13T03:19:12.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.370+0000 7f61ae9cc640 1 -- start start 2024-04-13T03:19:12.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61ae9cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f61a80d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61ae9cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a80d0880 con 0x7f61a815b6a0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61ae9cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a80d0b60 con 0x7f61a80734c0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61ae9cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f61a806d280 con 0x7f61a8073e10 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61a7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f61a815b6a0 0x7f61a8072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49906/0 (socket says 172.21.15.106:49906) 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.371+0000 7f61a7fff640 1 -- 172.21.15.106:0/2380507324 learned_addr learned my addr 172.21.15.106:0/2380507324 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934694553 0 0) 0x7f61a80d0880 con 0x7f61a815b6a0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6190003520 con 0x7f61a815b6a0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a77fe640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f61a80d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1879150308 0 0) 0x7f6190003520 con 0x7f61a815b6a0 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 msgr2=0x7f61a80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:12.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f61a80d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 >> v1:172.21.15.114:6789/0 conn(0x7f61a80734c0 legacy=0x7f61a806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a80d0e40 con 0x7f61a815b6a0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6198001670 con 0x7f61a815b6a0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 >> v1:172.21.15.106:6789/0 conn(0x7f61a815b6a0 legacy=0x7f61a8072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61900039e0 0x7f6190003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f6190004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6190004d40 0x7f61900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f619000bb50 con 0x7f61900039e0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f619000bcd0 con 0x7f6190004d40 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f619000bea0 con 0x7f61a8073e10 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61900039e0 0x7f6190003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a77fe640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f6190004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61acf69640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6190004d40 0x7f61900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 -- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 msgr2=0x7f6190004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:12.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61a8073e10 0x7f6190004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 -- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6190004d40 msgr2=0x7f61900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6190004d40 0x7f61900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61a7fff640 1 -- 172.21.15.106:0/2380507324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190003520 con 0x7f61900039e0 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.372+0000 7f61acf69640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6190004d40 0x7f61900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.373+0000 7f61a7fff640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61900039e0 0x7f6190003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f619800c270 tx=0x7f6190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05f06267ba0f144 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.373+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198007cf0 con 0x7f61900039e0 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.373+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6198030070 con 0x7f61900039e0 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.373+0000 7f61a6ffd640 1 -- 172.21.15.106:0/2380507324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198005d30 con 0x7f61900039e0 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 -- 172.21.15.106:0/2380507324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61900039e0 msgr2=0x7f6190003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/2380507324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61900039e0 0x7f6190003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f619800c270 tx=0x7f6190005830 comp rx=0 tx=0).stop 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61acf69640 1 -- 172.21.15.106:0/2380507324 reap_dead start 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 -- 172.21.15.106:0/2380507324 shutdown_connections 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 -- 172.21.15.106:0/2380507324 >> 172.21.15.106:0/2380507324 conn(0x7f61a80cdef0 msgr2=0x7f61a80ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:12.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 -- 172.21.15.106:0/2380507324 shutdown_connections 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.374+0000 7f61ae9cc640 1 -- 172.21.15.106:0/2380507324 wait complete. 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.375+0000 7f61ae9cc640 1 Processor -- start 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.375+0000 7f61ae9cc640 1 -- start start 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61900039e0 0x7f61a8155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a81584e0 0x7f61a8158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80cd760 con 0x7f61a8149b10 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80cd5e0 con 0x7f61a81584e0 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f61a80cd8e0 con 0x7f61900039e0 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57206/0 (socket says 172.21.15.106:57206) 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 -- 172.21.15.106:0/4257607807 learned_addr learned my addr 172.21.15.106:0/4257607807 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a7fff640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61900039e0 0x7f61a8155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61acf69640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a81584e0 0x7f61a8158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 -- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61900039e0 msgr2=0x7f61a8155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61900039e0 0x7f61a8155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 -- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a81584e0 msgr2=0x7f61a8158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a81584e0 0x7f61a8158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 -- 172.21.15.106:0/4257607807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a80ad970 con 0x7f61a8149b10 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a77fe640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f619c010ab0 tx=0x7f619c010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c01a470 con 0x7f61a8149b10 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f619c015070 con 0x7f61a8149b10 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a814a4d0 con 0x7f61a8149b10 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c01a830 con 0x7f61a8149b10 2024-04-13T03:19:12.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.376+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a80adf10 con 0x7f61a8149b10 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.377+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a8069120 con 0x7f61a8149b10 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.378+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f619c01a9d0 con 0x7f61a8149b10 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.378+0000 7f61a4ff9640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.378+0000 7f61a7fff640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.378+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f619c08c460 con 0x7f61a8149b10 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.379+0000 7f61a7fff640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6198034000 tx=0x7f6198007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.383+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f619c05aa50 con 0x7f61a8149b10 2024-04-13T03:19:12.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.561+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f61a81579c0 con 0x7f61a8149b10 2024-04-13T03:19:12.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.562+0000 7f61a4ff9640 1 -- 172.21.15.106:0/4257607807 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) ==== 84+0+65 (secure 0 0 0) 0x7f619c05a350 con 0x7f61a8149b10 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-13T03:19:12.564 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-13T03:19:12.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 msgr2=0x7f619406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6198034000 tx=0x7f6198007000 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 msgr2=0x7f61a8149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f619c010ab0 tx=0x7f619c010f70 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 shutdown_connections 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.563+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a81584e0 0x7f61a8158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.564+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a8149b10 0x7f61a8149f90 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.564+0000 7f61ae9cc640 1 --2- 172.21.15.106:0/4257607807 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f61900039e0 0x7f61a8155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.564+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 >> 172.21.15.106:0/4257607807 conn(0x7f61a80cdef0 msgr2=0x7f61a8156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.564+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 shutdown_connections 2024-04-13T03:19:12.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.564+0000 7f61ae9cc640 1 -- 172.21.15.106:0/4257607807 wait complete. 2024-04-13T03:19:12.576 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T03:19:12.577 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T03:19:12.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.679+0000 7efdc1e60640 1 Processor -- start 2024-04-13T03:19:12.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.679+0000 7efdc1e60640 1 -- start start 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdc1e60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 0x7efdbc0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdc1e60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efdbc0b3920 0x7efdbc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdc1e60640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc15c870 con 0x7efdbc0d3490 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdc1e60640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc0d5420 con 0x7efdbc0b3920 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdc1e60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc15ee70 con 0x7efdbc0d2370 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdbbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efdbc0b3920 0x7efdbc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdbb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efdbc0d3490 0x7efdbc0d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54820/0 (socket says 172.21.15.106:54820) 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.680+0000 7efdbb7fe640 1 -- 172.21.15.106:0/2019715133 learned_addr learned my addr 172.21.15.106:0/2019715133 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbbfff640 1 -- 172.21.15.106:0/2019715133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 msgr2=0x7efdbc0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbaffd640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 0x7efdbc0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdba7fc640 1 -- 172.21.15.106:0/2019715133 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938430413 0 0) 0x7efdbc15c870 con 0x7efdbc0d3490 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbbfff640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 0x7efdbc0b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbbfff640 1 -- 172.21.15.106:0/2019715133 >> v1:172.21.15.106:6789/0 conn(0x7efdbc0d3490 legacy=0x7efdbc0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbbfff640 1 -- 172.21.15.106:0/2019715133 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc0d2800 con 0x7efdbc0b3920 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbaffd640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 0x7efdbc0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.681+0000 7efdbbfff640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0b3920 0x7efdbc15f360 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009d90 tx=0x7efdb402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b974b12653faa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdba7fc640 1 -- 172.21.15.106:0/2019715133 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4039070 con 0x7efdbc0b3920 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdba7fc640 1 -- 172.21.15.106:0/2019715133 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb402cbc0 con 0x7efdbc0b3920 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdba7fc640 1 -- 172.21.15.106:0/2019715133 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4034440 con 0x7efdbc0b3920 2024-04-13T03:19:12.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 -- 172.21.15.106:0/2019715133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0b3920 msgr2=0x7efdbc15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0b3920 0x7efdbc15f360 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009d90 tx=0x7efdb402be10 comp rx=0 tx=0).stop 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 -- 172.21.15.106:0/2019715133 shutdown_connections 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0b3920 0x7efdbc15f360 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 --2- 172.21.15.106:0/2019715133 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0d2370 0x7efdbc0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 -- 172.21.15.106:0/2019715133 >> 172.21.15.106:0/2019715133 conn(0x7efdbc0b9760 msgr2=0x7efdbc0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 -- 172.21.15.106:0/2019715133 shutdown_connections 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.682+0000 7efdc1e60640 1 -- 172.21.15.106:0/2019715133 wait complete. 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.683+0000 7efdc1e60640 1 Processor -- start 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 -- start start 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 0x7efdbc156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0ce2a0 0x7efdbc0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d30a0 con 0x7efdbc0d3490 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d2dc0 con 0x7efdbc0b3920 2024-04-13T03:19:12.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdc1e60640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc0b1930 con 0x7efdbc0ce2a0 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdbaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdbaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39940/0 (socket says 172.21.15.106:39940) 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdbaffd640 1 -- 172.21.15.106:0/1124018994 learned_addr learned my addr 172.21.15.106:0/1124018994 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.684+0000 7efdbbfff640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0ce2a0 0x7efdbc0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbb7fe640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 0x7efdbc156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 -- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0ce2a0 msgr2=0x7efdbc0b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0ce2a0 0x7efdbc0b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 -- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 msgr2=0x7efdbc156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 0x7efdbc156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 -- 172.21.15.106:0/1124018994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc0b6940 con 0x7efdbc0d3490 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbb7fe640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 0x7efdbc156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdbaffd640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c7f0 tx=0x7efdb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004220 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb00043c0 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc0b60a0 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0005bb0 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.685+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc0b6660 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.686+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc068a10 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.686+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb0006030 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.687+0000 7efda3fff640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 0x7efda806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.687+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7efdb008be60 con 0x7efdbc0d3490 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.687+0000 7efdbb7fe640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 0x7efda806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:12.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.687+0000 7efdbb7fe640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 0x7efda806edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7efdac000f60 tx=0x7efdac007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:12.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.691+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdb001e020 con 0x7efdbc0d3490 2024-04-13T03:19:12.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:12.865+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efdbc0cbb20 con 0x7efdbc0d3490 2024-04-13T03:19:13.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.278+0000 7efda3fff640 1 -- 172.21.15.106:0/1124018994 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) ==== 138+0+0 (secure 0 0 0) 0x7efdb0015030 con 0x7efdbc0d3490 2024-04-13T03:19:13.281 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 msgr2=0x7efda806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 0x7efda806edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7efdac000f60 tx=0x7efdac007ca0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 msgr2=0x7efdbc0cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c7f0 tx=0x7efdb000ccb0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 shutdown_connections 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efda806c8f0 0x7efda806edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efdbc0ce2a0 0x7efdbc0b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efdbc0d3490 0x7efdbc0cdd60 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 --2- 172.21.15.106:0/1124018994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdbc0b3920 0x7efdbc156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 >> 172.21.15.106:0/1124018994 conn(0x7efdbc0b9760 msgr2=0x7efdbc0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:13.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 shutdown_connections 2024-04-13T03:19:13.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.281+0000 7efdc1e60640 1 -- 172.21.15.106:0/1124018994 wait complete. 2024-04-13T03:19:13.297 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T03:19:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.402+0000 7f3952a36640 1 Processor -- start 2024-04-13T03:19:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.402+0000 7f3952a36640 1 -- start start 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.403+0000 7f3952a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.403+0000 7f3952a36640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 0x7f394c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.403+0000 7f3952a36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394c0cfd20 con 0x7f394c15ed90 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.403+0000 7f3952a36640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f394c162af0 con 0x7f394c0b3d60 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.403+0000 7f3952a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f394c162dd0 con 0x7f394c15e9b0 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f3950fd3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 0x7f394c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:44896/0 (socket says 172.21.15.106:44896) 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 -- 172.21.15.106:0/349701850 learned_addr learned my addr 172.21.15.106:0/349701850 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 -- 172.21.15.106:0/349701850 >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 msgr2=0x7f394c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 --2- 172.21.15.106:0/349701850 >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 0x7f394c0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 -- 172.21.15.106:0/349701850 >> v1:172.21.15.114:6789/0 conn(0x7f394c15ed90 legacy=0x7f394c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f394bfff640 1 -- 172.21.15.106:0/349701850 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c1630b0 con 0x7f394c15e9b0 2024-04-13T03:19:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.404+0000 7f3950fd3640 1 --2- 172.21.15.106:0/349701850 >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 0x7f394c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f394bfff640 1 --2- 172.21.15.106:0/349701850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b7de9b0613199e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f394affd640 1 -- 172.21.15.106:0/349701850 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c02f840 con 0x7f394c15e9b0 2024-04-13T03:19:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f394affd640 1 -- 172.21.15.106:0/349701850 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c02f9e0 con 0x7f394c15e9b0 2024-04-13T03:19:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f394affd640 1 -- 172.21.15.106:0/349701850 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c02fdb0 con 0x7f394c15e9b0 2024-04-13T03:19:13.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 -- 172.21.15.106:0/349701850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 msgr2=0x7f394c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 --2- 172.21.15.106:0/349701850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c02ba20 comp rx=0 tx=0).stop 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 -- 172.21.15.106:0/349701850 shutdown_connections 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 --2- 172.21.15.106:0/349701850 >> v2:172.21.15.114:3300/0 conn(0x7f394c0b3d60 0x7f394c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 --2- 172.21.15.106:0/349701850 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15e9b0 0x7f394c0d2300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 -- 172.21.15.106:0/349701850 >> 172.21.15.106:0/349701850 conn(0x7f394c08bf10 msgr2=0x7f394c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 -- 172.21.15.106:0/349701850 shutdown_connections 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.405+0000 7f3952a36640 1 -- 172.21.15.106:0/349701850 wait complete. 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 Processor -- start 2024-04-13T03:19:13.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 -- start start 2024-04-13T03:19:13.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f394c0b3d60 0x7f394c159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15f6c0 0x7f394c160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f394c162800 con 0x7f394c15e9b0 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f394c162680 con 0x7f394c0b3d60 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.407+0000 7f3952a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f394c0d4d00 con 0x7f394c15f6c0 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39942/0 (socket says 172.21.15.106:39942) 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 -- 172.21.15.106:0/3189863541 learned_addr learned my addr 172.21.15.106:0/3189863541 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394bfff640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f394c0b3d60 0x7f394c159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3950fd3640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15f6c0 0x7f394c160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 -- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15f6c0 msgr2=0x7f394c160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15f6c0 0x7f394c160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 -- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f394c0b3d60 msgr2=0x7f394c159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f394c0b3d60 0x7f394c159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 -- 172.21.15.106:0/3189863541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c159aa0 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f394b7fe640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f394000c3c0 tx=0x7f3940001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940017070 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39400052d0 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c15fc40 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940003830 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.408+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c160200 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.409+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c068a10 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.410+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f394001a070 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.410+0000 7f3948ff9640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 0x7f393806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.410+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5619+0+0 (secure 0 0 0) 0x7f394008ae00 con 0x7f394c15e9b0 2024-04-13T03:19:13.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.410+0000 7f394bfff640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 0x7f393806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:13.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.411+0000 7f394bfff640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 0x7f393806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f393c036000 tx=0x7f393c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:13.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.416+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3940059280 con 0x7f394c15e9b0 2024-04-13T03:19:13.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:13.590+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f394c0d0960 con 0x7f394c15e9b0 2024-04-13T03:19:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.282+0000 7f3948ff9640 1 -- 172.21.15.106:0/3189863541 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) ==== 170+0+0 (secure 0 0 0) 0x7f3940058b80 con 0x7f394c15e9b0 2024-04-13T03:19:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-13T03:19:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 msgr2=0x7f393806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 0x7f393806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f393c036000 tx=0x7f393c008030 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 msgr2=0x7f394c15f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f394000c3c0 tx=0x7f3940001e70 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 shutdown_connections 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f393806c8f0 0x7f393806edb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.285+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f394c15f6c0 0x7f394c160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.286+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f394c15e9b0 0x7f394c15f180 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.286+0000 7f3952a36640 1 --2- 172.21.15.106:0/3189863541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f394c0b3d60 0x7f394c159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.286+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 >> 172.21.15.106:0/3189863541 conn(0x7f394c08bf10 msgr2=0x7f394c15cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.286+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 shutdown_connections 2024-04-13T03:19:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.286+0000 7f3952a36640 1 -- 172.21.15.106:0/3189863541 wait complete. 2024-04-13T03:19:14.312 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T03:19:14.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.419+0000 7fdcf16de640 1 Processor -- start 2024-04-13T03:19:14.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.419+0000 7fdcf16de640 1 -- start start 2024-04-13T03:19:14.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcf16de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdcec15e9b0 0x7fdcec0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcf16de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec15ed90 0x7fdcec0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcf16de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcec0cfd20 con 0x7fdcec0b3d60 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcf16de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec162af0 con 0x7fdcec15e9b0 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcf16de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec162dd0 con 0x7fdcec15ed90 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdceb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdcec0b3d60 0x7fdcec0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54832/0 (socket says 172.21.15.106:54832) 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdceb7fe640 1 -- 172.21.15.106:0/3508962503 learned_addr learned my addr 172.21.15.106:0/3508962503 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdce3fff640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec15ed90 0x7fdcec0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdceaffd640 1 --2- 172.21.15.106:0/3508962503 >> v2:172.21.15.114:3300/0 conn(0x7fdcec15e9b0 0x7fdcec0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778427320 0 0) 0x7fdcec0cfd20 con 0x7fdcec0b3d60 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcd8003520 con 0x7fdcec0b3d60 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3249766622 0 0) 0x7fdcd8003520 con 0x7fdcec0b3d60 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec15ed90 msgr2=0x7fdcec0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec15ed90 0x7fdcec0d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 >> v2:172.21.15.114:3300/0 conn(0x7fdcec15e9b0 msgr2=0x7fdcec0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 --2- 172.21.15.106:0/3508962503 >> v2:172.21.15.114:3300/0 conn(0x7fdcec15e9b0 0x7fdcec0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec1630b0 con 0x7fdcec0b3d60 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fdce4001a10 con 0x7fdcec0b3d60 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 >> v1:172.21.15.106:6789/0 conn(0x7fdcec0b3d60 legacy=0x7fdcec0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd80039b0 0x7fdcd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd8004800 0x7fdcd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcd80051c0 0x7fdcd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec162af0 con 0x7fdcd80039b0 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec162dd0 con 0x7fdcd8004800 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.420+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd800bf00 con 0x7fdcd80051c0 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd80039b0 0x7fdcd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdce3fff640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd8004800 0x7fdcd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceb7fe640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcd80051c0 0x7fdcd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 -- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcd80051c0 msgr2=0x7fdcd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcd80051c0 0x7fdcd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 -- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd8004800 msgr2=0x7fdcd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd8004800 0x7fdcd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.421+0000 7fdceaffd640 1 -- 172.21.15.106:0/3508962503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd8003520 con 0x7fdcd80039b0 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdceaffd640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd80039b0 0x7fdcd8003dd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ac0 tx=0x7fdcd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49286e24d63fd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc039070 con 0x7fdcd80039b0 2024-04-13T03:19:14.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc02c710 con 0x7fdcd80039b0 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcea7fc640 1 -- 172.21.15.106:0/3508962503 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc009690 con 0x7fdcd80039b0 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 -- 172.21.15.106:0/3508962503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd80039b0 msgr2=0x7fdcd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 --2- 172.21.15.106:0/3508962503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcd80039b0 0x7fdcd8003dd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ac0 tx=0x7fdcd8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdceb7fe640 1 -- 172.21.15.106:0/3508962503 reap_dead start 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 -- 172.21.15.106:0/3508962503 shutdown_connections 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 -- 172.21.15.106:0/3508962503 >> 172.21.15.106:0/3508962503 conn(0x7fdcec08bf10 msgr2=0x7fdcec0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 -- 172.21.15.106:0/3508962503 shutdown_connections 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.422+0000 7fdcf16de640 1 -- 172.21.15.106:0/3508962503 wait complete. 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.423+0000 7fdcf16de640 1 Processor -- start 2024-04-13T03:19:14.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.423+0000 7fdcf16de640 1 -- start start 2024-04-13T03:19:14.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 0x7fdcec14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 0x7fdcec0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec162800 con 0x7fdcec150ec0 2024-04-13T03:19:14.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec162680 con 0x7fdcd80039b0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdcf16de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec06eb20 con 0x7fdcec1f76c0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39960/0 (socket says 172.21.15.106:39960) 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdce3fff640 1 -- 172.21.15.106:0/1298717945 learned_addr learned my addr 172.21.15.106:0/1298717945 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdceaffd640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 0x7fdcec14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.424+0000 7fdceb7fe640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 0x7fdcec0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 -- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 msgr2=0x7fdcec0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 0x7fdcec0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 -- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 msgr2=0x7fdcec14dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 0x7fdcec14dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 -- 172.21.15.106:0/1298717945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec1f8010 con 0x7fdcec150ec0 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdceaffd640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 0x7fdcec14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdceb7fe640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 0x7fdcec0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:14.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce3fff640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40118e0 tx=0x7fdcd4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4005c60 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd4005e00 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400fa60 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1f82f0 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.425+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec14e150 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.426+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec068a10 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.427+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcd4012040 con 0x7fdcec150ec0 2024-04-13T03:19:14.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.427+0000 7fdce37fe640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 0x7fdccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:14.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.427+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6001+0+0 (secure 0 0 0) 0x7fdcd408c340 con 0x7fdcec150ec0 2024-04-13T03:19:14.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.427+0000 7fdceaffd640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 0x7fdccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:14.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.427+0000 7fdceaffd640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 0x7fdccc06ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:14.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.432+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcd4018030 con 0x7fdcec150ec0 2024-04-13T03:19:14.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:14.645+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdcec15d360 con 0x7fdcec150ec0 2024-04-13T03:19:15.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.308+0000 7fdce37fe640 1 -- 172.21.15.106:0/1298717945 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v84) ==== 174+0+0 (secure 0 0 0) 0x7fdcd401a300 con 0x7fdcec150ec0 2024-04-13T03:19:15.309 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 17 and data pool 18 2024-04-13T03:19:15.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 msgr2=0x7fdccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 0x7fdccc06ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc030000 comp rx=0 tx=0).stop 2024-04-13T03:19:15.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 msgr2=0x7fdcec1f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40118e0 tx=0x7fdcd4011da0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 shutdown_connections 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdccc06c920 0x7fdccc06ede0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdcec1f76c0 0x7fdcec0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcec150ec0 0x7fdcec1f3960 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 --2- 172.21.15.106:0/1298717945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd80039b0 0x7fdcec14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 >> 172.21.15.106:0/1298717945 conn(0x7fdcec08bf10 msgr2=0x7fdcec15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 shutdown_connections 2024-04-13T03:19:15.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.310+0000 7fdcf16de640 1 -- 172.21.15.106:0/1298717945 wait complete. 2024-04-13T03:19:15.327 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:15.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.428+0000 7f2ae51cc640 1 Processor -- start 2024-04-13T03:19:15.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.428+0000 7f2ae51cc640 1 -- start start 2024-04-13T03:19:15.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ae51cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ae006e8e0 0x7f2ae015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ae51cc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae015f420 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ae51cc640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae015f700 con 0x7f2ae0070c80 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ae51cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae006ecc0 con 0x7f2ae006e8e0 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ad7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2ae00cd770 0x7f2ae00cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54844/0 (socket says 172.21.15.106:54844) 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ad7fff640 1 -- 172.21.15.106:0/3131352414 learned_addr learned my addr 172.21.15.106:0/3131352414 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031850866 0 0) 0x7f2ae015f420 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2acc003520 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3131352414 >> v2:172.21.15.114:3300/0 conn(0x7f2ae006e8e0 0x7f2ae015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404068002 0 0) 0x7f2acc003520 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 >> v2:172.21.15.114:3300/0 conn(0x7f2ae006e8e0 msgr2=0x7f2ae015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 --2- 172.21.15.106:0/3131352414 >> v2:172.21.15.114:3300/0 conn(0x7f2ae006e8e0 0x7f2ae015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 >> v1:172.21.15.114:6789/0 conn(0x7f2ae0070c80 legacy=0x7f2ae00ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae015f9e0 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2ac8001670 con 0x7f2ae00cd770 2024-04-13T03:19:15.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 >> v1:172.21.15.106:6789/0 conn(0x7f2ae00cd770 legacy=0x7f2ae00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 0x7f2acc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc004800 0x7f2acc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2acc0051c0 0x7f2acc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.429+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc00c030 con 0x7f2acc0039b0 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc00c1e0 con 0x7f2acc0051c0 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc00c3e0 con 0x7f2acc004800 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 0x7f2acc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2ad7fff640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc004800 0x7f2acc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2adf59e640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2acc0051c0 0x7f2acc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 -- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc004800 msgr2=0x7f2acc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc004800 0x7f2acc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 -- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2acc0051c0 msgr2=0x7f2acc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2acc0051c0 0x7f2acc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 -- 172.21.15.106:0/3131352414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc003520 con 0x7f2acc0039b0 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2adf59e640 1 -- 172.21.15.106:0/3131352414 reap_dead start 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.430+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 0x7f2acc003dd0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009ac0 tx=0x7f2acc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587c4fecb0d6c2df server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0039070 con 0x7f2acc0039b0 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad002c6b0 con 0x7f2acc0039b0 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ade59c640 1 -- 172.21.15.106:0/3131352414 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0009690 con 0x7f2acc0039b0 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 msgr2=0x7f2acc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 0x7f2acc003dd0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009ac0 tx=0x7f2acc005830 comp rx=0 tx=0).stop 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3131352414 shutdown_connections 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3131352414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2acc0039b0 0x7f2acc003dd0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3131352414 >> 172.21.15.106:0/3131352414 conn(0x7f2ae006f180 msgr2=0x7f2ae006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3131352414 shutdown_connections 2024-04-13T03:19:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.431+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3131352414 wait complete. 2024-04-13T03:19:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 Processor -- start 2024-04-13T03:19:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 -- start start 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc0039b0 0x7f2ae00b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae00b5b80 0x7f2ae0146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00cdbd0 con 0x7f2ae0146b80 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00707f0 con 0x7f2ae00b5b80 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2ae51cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae015ed80 con 0x7f2acc0039b0 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2adf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2adf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39978/0 (socket says 172.21.15.106:39978) 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.433+0000 7f2adf59e640 1 -- 172.21.15.106:0/3429488273 learned_addr learned my addr 172.21.15.106:0/3429488273 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc0039b0 0x7f2ae00b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ad7fff640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae00b5b80 0x7f2ae0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 -- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc0039b0 msgr2=0x7f2ae00b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc0039b0 0x7f2ae00b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 -- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae00b5b80 msgr2=0x7f2ae0146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae00b5b80 0x7f2ae0146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 -- 172.21.15.106:0/3429488273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae00b6310 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2adf59e640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8007fd0 tx=0x7f2ad800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8019070 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8006d60 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae00b65f0 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8003b90 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.434+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae00b6bb0 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.435+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae0068a10 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.436+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ad801c070 con 0x7f2ae0146b80 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.436+0000 7f2ad77fe640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.436+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.436+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6101+0+0 (secure 0 0 0) 0x7f2ad808b290 con 0x7f2ae0146b80 2024-04-13T03:19:15.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.436+0000 7f2aded9d640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0036000 tx=0x7f2ad0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.440+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ad8059530 con 0x7f2ae0146b80 2024-04-13T03:19:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.617+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ae00b2bd0 con 0x7f2ae0146b80 2024-04-13T03:19:15.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.618+0000 7f2ad77fe640 1 -- 172.21.15.106:0/3429488273 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17001 (secure 0 0 0) 0x7f2ad8058e30 con 0x7f2ae0146b80 2024-04-13T03:19:15.619 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:15.619 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":120,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:14.646527+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 msgr2=0x7f2ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0036000 tx=0x7f2ad0030000 comp rx=0 tx=0).stop 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 msgr2=0x7f2ae0147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8007fd0 tx=0x7f2ad800e960 comp rx=0 tx=0).stop 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 shutdown_connections 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2ae0146b80 0x7f2ae0147100 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae00b5b80 0x7f2ae0146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 --2- 172.21.15.106:0/3429488273 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2acc0039b0 0x7f2ae00b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 >> 172.21.15.106:0/3429488273 conn(0x7f2ae006f180 msgr2=0x7f2ae015d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 shutdown_connections 2024-04-13T03:19:15.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.620+0000 7f2ae51cc640 1 -- 172.21.15.106:0/3429488273 wait complete. 2024-04-13T03:19:15.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:15.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 Processor -- start 2024-04-13T03:19:15.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 -- start start 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb03415b6a0 0x7fb0340d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb034073e10 0x7fb0340cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0340d0880 con 0x7fb0340cd5e0 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb03406d280 con 0x7fb03415b6a0 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.732+0000 7fb039fa5640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb0340d0b60 con 0x7fb034073e10 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb03415b6a0 0x7fb0340d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb03415b6a0 0x7fb0340d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39988/0 (socket says 172.21.15.106:39988) 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 -- 172.21.15.106:0/1908808640 learned_addr learned my addr 172.21.15.106:0/1908808640 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 -- 172.21.15.106:0/1908808640 >> v2:172.21.15.114:3300/0 conn(0x7fb034073e10 msgr2=0x7fb0340cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 --2- 172.21.15.106:0/1908808640 >> v2:172.21.15.114:3300/0 conn(0x7fb034073e10 0x7fb0340cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 -- 172.21.15.106:0/1908808640 >> v1:172.21.15.114:6789/0 conn(0x7fb0340cd5e0 legacy=0x7fb034072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.733+0000 7fb0337fe640 1 -- 172.21.15.106:0/1908808640 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0340d0e40 con 0x7fb03415b6a0 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb0337fe640 1 --2- 172.21.15.106:0/1908808640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb03415b6a0 0x7fb0340d02a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb03406eb00 tx=0x7fb02402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d758f4bcc5001e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb032ffd640 1 -- 172.21.15.106:0/1908808640 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024035020 con 0x7fb03415b6a0 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb032ffd640 1 -- 172.21.15.106:0/1908808640 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb024031870 con 0x7fb03415b6a0 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb032ffd640 1 -- 172.21.15.106:0/1908808640 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024031c40 con 0x7fb03415b6a0 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 -- 172.21.15.106:0/1908808640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb03415b6a0 msgr2=0x7fb0340d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1908808640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb03415b6a0 0x7fb0340d02a0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb03406eb00 tx=0x7fb02402b910 comp rx=0 tx=0).stop 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 -- 172.21.15.106:0/1908808640 shutdown_connections 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1908808640 >> v2:172.21.15.114:3300/0 conn(0x7fb034073e10 0x7fb0340cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1908808640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb03415b6a0 0x7fb0340d02a0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 -- 172.21.15.106:0/1908808640 >> 172.21.15.106:0/1908808640 conn(0x7fb0340cdef0 msgr2=0x7fb0340ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 -- 172.21.15.106:0/1908808640 shutdown_connections 2024-04-13T03:19:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.734+0000 7fb039fa5640 1 -- 172.21.15.106:0/1908808640 wait complete. 2024-04-13T03:19:15.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.735+0000 7fb039fa5640 1 Processor -- start 2024-04-13T03:19:15.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 -- start start 2024-04-13T03:19:15.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 0x7fb0341e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 0x7fb0341e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb034073640 con 0x7fb034073e10 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0340734c0 con 0x7fb0340cd5e0 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb039fa5640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb03406d710 con 0x7fb03415b6a0 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb0337fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb0337fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40002/0 (socket says 172.21.15.106:40002) 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.736+0000 7fb0337fe640 1 -- 172.21.15.106:0/1470331432 learned_addr learned my addr 172.21.15.106:0/1470331432 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb033fff640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 0x7fb0341e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb02affd640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 0x7fb0341e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 -- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 msgr2=0x7fb0341e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 0x7fb0341e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 -- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 msgr2=0x7fb0341e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 0x7fb0341e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 -- 172.21.15.106:0/1470331432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0341f5590 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb033fff640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 0x7fb0341e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb02affd640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 0x7fb0341e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb0337fe640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb02402b950 tx=0x7fb0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02403ad60 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0240443f0 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240447e0 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0341f5870 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.737+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0340caee0 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.738+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb034069120 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.738+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb024038020 con 0x7fb034073e10 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.739+0000 7fb030ff9640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 0x7fb01406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.739+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6101+0+0 (secure 0 0 0) 0x7fb0240adf70 con 0x7fb034073e10 2024-04-13T03:19:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.739+0000 7fb02affd640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 0x7fb01406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:15.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.739+0000 7fb02affd640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 0x7fb01406edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb01c006fd0 tx=0x7fb01c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:15.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.743+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb02403f070 con 0x7fb034073e10 2024-04-13T03:19:15.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.917+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb03406ccb0 con 0x7fb034073e10 2024-04-13T03:19:15.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.918+0000 7fb030ff9640 1 -- 172.21.15.106:0/1470331432 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17001 (secure 0 0 0) 0x7fb02407c210 con 0x7fb034073e10 2024-04-13T03:19:15.919 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:15.919 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":120,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:14.646527+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 msgr2=0x7fb01406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 0x7fb01406edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb01c006fd0 tx=0x7fb01c008040 comp rx=0 tx=0).stop 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 msgr2=0x7fb034146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb02402b950 tx=0x7fb0240046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 shutdown_connections 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb01406c8f0 0x7fb01406edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb03415b6a0 0x7fb0341e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0340cd5e0 0x7fb0341e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 --2- 172.21.15.106:0/1470331432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034073e10 0x7fb034146870 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:15.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 >> 172.21.15.106:0/1470331432 conn(0x7fb0340cdef0 msgr2=0x7fb0340b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:15.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 shutdown_connections 2024-04-13T03:19:15.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:15.920+0000 7fb039fa5640 1 -- 172.21.15.106:0/1470331432 wait complete. 2024-04-13T03:19:15.932 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T03:19:16.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.034+0000 7f0b4871e640 1 Processor -- start 2024-04-13T03:19:16.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.034+0000 7f0b4871e640 1 -- start start 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 0x7f0b400b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b400ce9e0 0x7f0b400b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b4015ec00 0x7f0b400b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0b400cd240 con 0x7f0b400ce9e0 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b400cd520 con 0x7f0b4015e820 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b4871e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b400cd800 con 0x7f0b4015ec00 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b45cb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b400ce9e0 0x7f0b400b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b45cb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b400ce9e0 0x7f0b400b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40008/0 (socket says 172.21.15.106:40008) 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.035+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3052273967 learned_addr learned my addr 172.21.15.106:0/3052273967 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3052273967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b4015ec00 msgr2=0x7f0b400b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3052273967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b4015ec00 0x7f0b400b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3052273967 >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 msgr2=0x7f0b400b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:16.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b464ba640 1 --2- 172.21.15.106:0/3052273967 >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 0x7f0b400b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3052273967 >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 0x7f0b400b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3052273967 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b400cdae0 con 0x7f0b400ce9e0 2024-04-13T03:19:16.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b464ba640 1 --2- 172.21.15.106:0/3052273967 >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 0x7f0b400b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:16.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.036+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3052273967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b400ce9e0 0x7f0b400b53c0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009870 tx=0x7f0b3402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2464a70806bda13 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b454b8640 1 -- 172.21.15.106:0/3052273967 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3402e070 con 0x7f0b400ce9e0 2024-04-13T03:19:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b454b8640 1 -- 172.21.15.106:0/3052273967 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34037070 con 0x7f0b400ce9e0 2024-04-13T03:19:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b454b8640 1 -- 172.21.15.106:0/3052273967 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34031a00 con 0x7f0b400ce9e0 2024-04-13T03:19:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 -- 172.21.15.106:0/3052273967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b400ce9e0 msgr2=0x7f0b400b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3052273967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b400ce9e0 0x7f0b400b53c0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009870 tx=0x7f0b3402b880 comp rx=0 tx=0).stop 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 -- 172.21.15.106:0/3052273967 shutdown_connections 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3052273967 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b4015ec00 0x7f0b400b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3052273967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b400ce9e0 0x7f0b400b53c0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3052273967 >> v2:172.21.15.114:3300/0 conn(0x7f0b4015e820 0x7f0b400b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 -- 172.21.15.106:0/3052273967 >> 172.21.15.106:0/3052273967 conn(0x7f0b400d2510 msgr2=0x7f0b400d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 -- 172.21.15.106:0/3052273967 shutdown_connections 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.037+0000 7f0b4871e640 1 -- 172.21.15.106:0/3052273967 wait complete. 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.038+0000 7f0b4871e640 1 Processor -- start 2024-04-13T03:19:16.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.038+0000 7f0b4871e640 1 -- start start 2024-04-13T03:19:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b400ce9e0 0x7f0b40156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4015ec00 0x7f0b400c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400ccf50 con 0x7f0b4015e820 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400ccdd0 con 0x7f0b4015ec00 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b4871e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b40162600 con 0x7f0b400ce9e0 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b45cb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b45cb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40018/0 (socket says 172.21.15.106:40018) 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3616518611 learned_addr learned my addr 172.21.15.106:0/3616518611 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b46cbb640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4015ec00 0x7f0b400c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.039+0000 7f0b464ba640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b400ce9e0 0x7f0b40156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b400ce9e0 msgr2=0x7f0b40156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b400ce9e0 0x7f0b40156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4015ec00 msgr2=0x7f0b400c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4015ec00 0x7f0b400c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 -- 172.21.15.106:0/3616518611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b400c77f0 con 0x7f0b4015e820 2024-04-13T03:19:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.040+0000 7f0b45cb9640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.045+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032880 con 0x7f0b4015e820 2024-04-13T03:19:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.045+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34032a20 con 0x7f0b4015e820 2024-04-13T03:19:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.045+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032df0 con 0x7f0b4015e820 2024-04-13T03:19:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.045+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b400c7ad0 con 0x7f0b4015e820 2024-04-13T03:19:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.045+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b400c8090 con 0x7f0b4015e820 2024-04-13T03:19:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.046+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b40069120 con 0x7f0b4015e820 2024-04-13T03:19:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.046+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b3404c060 con 0x7f0b4015e820 2024-04-13T03:19:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.047+0000 7f0b3effd640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.047+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0b340ad5c0 con 0x7f0b4015e820 2024-04-13T03:19:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.047+0000 7f0b464ba640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.047+0000 7f0b464ba640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b300075b0 tx=0x7f0b30007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.051+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b34040070 con 0x7f0b4015e820 2024-04-13T03:19:16.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.262+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0b40073fa0 con 0x7f0b4015e820 2024-04-13T03:19:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.314+0000 7f0b3effd640 1 -- 172.21.15.106:0/3616518611 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7f0b3407b860 con 0x7f0b4015e820 2024-04-13T03:19:16.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 msgr2=0x7f0b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b300075b0 tx=0x7f0b30007540 comp rx=0 tx=0).stop 2024-04-13T03:19:16.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 msgr2=0x7f0b400c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b340046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 shutdown_connections 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.316+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4015ec00 0x7f0b400c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b4015e820 0x7f0b400c70a0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 --2- 172.21.15.106:0/3616518611 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b400ce9e0 0x7f0b40156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 >> 172.21.15.106:0/3616518611 conn(0x7f0b400d2510 msgr2=0x7f0b400ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 shutdown_connections 2024-04-13T03:19:16.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.317+0000 7f0b4871e640 1 -- 172.21.15.106:0/3616518611 wait complete. 2024-04-13T03:19:16.328 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-13T03:19:16.328 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-13T03:19:16.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 Processor -- start 2024-04-13T03:19:16.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 -- start start 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f23b815e9b0 0x7f23b80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b80b3d60 0x7f23b80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23b80cfd20 con 0x7f23b815ed90 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f23b8162af0 con 0x7f23b815e9b0 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23bea36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f23b8162dd0 con 0x7f23b80b3d60 2024-04-13T03:19:16.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.430+0000 7f23b7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f23b815e9b0 0x7f23b80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f23b815e9b0 0x7f23b80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40030/0 (socket says 172.21.15.106:40030) 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 -- 172.21.15.106:0/2856811858 learned_addr learned my addr 172.21.15.106:0/2856811858 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23bcfd3640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b80b3d60 0x7f23b80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 -- 172.21.15.106:0/2856811858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b80b3d60 msgr2=0x7f23b80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b80b3d60 0x7f23b80d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 -- 172.21.15.106:0/2856811858 >> v1:172.21.15.114:6789/0 conn(0x7f23b815ed90 legacy=0x7f23b80d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.431+0000 7f23b7fff640 1 -- 172.21.15.106:0/2856811858 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81630b0 con 0x7f23b815e9b0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23b7fff640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b815e9b0 0x7f23b80d2300 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009870 tx=0x7f23ac02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc73e0cc45b5841 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23b6ffd640 1 -- 172.21.15.106:0/2856811858 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac02f840 con 0x7f23b815e9b0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23b6ffd640 1 -- 172.21.15.106:0/2856811858 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ac02f9e0 con 0x7f23b815e9b0 2024-04-13T03:19:16.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23b6ffd640 1 -- 172.21.15.106:0/2856811858 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac02fdb0 con 0x7f23b815e9b0 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 -- 172.21.15.106:0/2856811858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b815e9b0 msgr2=0x7f23b80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b815e9b0 0x7f23b80d2300 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009870 tx=0x7f23ac02ba20 comp rx=0 tx=0).stop 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 -- 172.21.15.106:0/2856811858 shutdown_connections 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b80b3d60 0x7f23b80d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 --2- 172.21.15.106:0/2856811858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b815e9b0 0x7f23b80d2300 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 -- 172.21.15.106:0/2856811858 >> 172.21.15.106:0/2856811858 conn(0x7f23b808bf10 msgr2=0x7f23b80b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 -- 172.21.15.106:0/2856811858 shutdown_connections 2024-04-13T03:19:16.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.432+0000 7f23bea36640 1 -- 172.21.15.106:0/2856811858 wait complete. 2024-04-13T03:19:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.433+0000 7f23bea36640 1 Processor -- start 2024-04-13T03:19:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 -- start start 2024-04-13T03:19:16.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b80b3d60 0x7f23b80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b815e9b0 0x7f23b815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8162800 con 0x7f23b80cda70 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8162680 con 0x7f23b80b3d60 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bea36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f23b80d4d00 con 0x7f23b815e9b0 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.434+0000 7f23bcfd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40034/0 (socket says 172.21.15.106:40034) 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 -- 172.21.15.106:0/2416873144 learned_addr learned my addr 172.21.15.106:0/2416873144 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23b77fe640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b815e9b0 0x7f23b815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23b7fff640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b80b3d60 0x7f23b80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 -- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b815e9b0 msgr2=0x7f23b815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b815e9b0 0x7f23b815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 -- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b80b3d60 msgr2=0x7f23b80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b80b3d60 0x7f23b80cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 -- 172.21.15.106:0/2416873144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b80cc9d0 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bcfd3640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f23b0007e30 tx=0x7f23b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0014620 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0019070 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b80cccb0 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0014a00 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.435+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b80cd270 con 0x7f23b80cda70 2024-04-13T03:19:16.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.436+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b8068a10 con 0x7f23b80cda70 2024-04-13T03:19:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.437+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23b0014ba0 con 0x7f23b80cda70 2024-04-13T03:19:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.437+0000 7f23b4ff9640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 0x7f23a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.437+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f23b008c4e0 con 0x7f23b80cda70 2024-04-13T03:19:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.437+0000 7f23b7fff640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 0x7f23a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:16.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.437+0000 7f23b7fff640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 0x7f23a406edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23ac036000 tx=0x7f23ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:16.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.441+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23b005a0b0 con 0x7f23b80cda70 2024-04-13T03:19:16.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:16.656+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f23b80b83c0 con 0x7f23b80cda70 2024-04-13T03:19:17.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.321+0000 7f23b4ff9640 1 -- 172.21.15.106:0/2416873144 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f23b005e3b0 con 0x7f23b80cda70 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 msgr2=0x7f23a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 0x7f23a406edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23ac036000 tx=0x7f23ac008030 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 msgr2=0x7f23b815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f23b0007e30 tx=0x7f23b000e4b0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 shutdown_connections 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f23a406c8f0 0x7f23a406edb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f23b815e9b0 0x7f23b815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23b80cda70 0x7f23b815bd40 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 --2- 172.21.15.106:0/2416873144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b80b3d60 0x7f23b80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 >> 172.21.15.106:0/2416873144 conn(0x7f23b808bf10 msgr2=0x7f23b80b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 shutdown_connections 2024-04-13T03:19:17.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.324+0000 7f23bea36640 1 -- 172.21.15.106:0/2416873144 wait complete. 2024-04-13T03:19:17.337 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-13T03:19:17.337 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:17.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.441+0000 7f079b680640 1 Processor -- start 2024-04-13T03:19:17.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 -- start start 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07940cd550 0x7f079406d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f079415fe40 con 0x7f079415ed60 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f07941633f0 con 0x7f07940cd550 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f079b680640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f079415f2c0 con 0x7f079415e980 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f0799c1d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07940cd550 0x7f079406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f0798c1b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f079415ed60 0x7f07940cce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54850/0 (socket says 172.21.15.106:54850) 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.442+0000 7f0798c1b640 1 -- 172.21.15.106:0/4123331066 learned_addr learned my addr 172.21.15.106:0/4123331066 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.443+0000 7f079941c640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.443+0000 7f0799c1d640 1 -- 172.21.15.106:0/4123331066 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 msgr2=0x7f0794162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:17.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.443+0000 7f0799c1d640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.443+0000 7f0799c1d640 1 -- 172.21.15.106:0/4123331066 >> v1:172.21.15.106:6789/0 conn(0x7f079415ed60 legacy=0x7f07940cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.443+0000 7f0799c1d640 1 -- 172.21.15.106:0/4123331066 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079415f5a0 con 0x7f07940cd550 2024-04-13T03:19:17.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f0799c1d640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079406d370 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097a0 tx=0x7f078c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccafeed3b214a615 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f0793fff640 1 -- 172.21.15.106:0/4123331066 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c034070 con 0x7f07940cd550 2024-04-13T03:19:17.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f0793fff640 1 -- 172.21.15.106:0/4123331066 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c030800 con 0x7f07940cd550 2024-04-13T03:19:17.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f0793fff640 1 -- 172.21.15.106:0/4123331066 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c030bd0 con 0x7f07940cd550 2024-04-13T03:19:17.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 -- 172.21.15.106:0/4123331066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 msgr2=0x7f079406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079406d370 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097a0 tx=0x7f078c02b940 comp rx=0 tx=0).stop 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 -- 172.21.15.106:0/4123331066 shutdown_connections 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079406d370 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 --2- 172.21.15.106:0/4123331066 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.444+0000 7f079b680640 1 -- 172.21.15.106:0/4123331066 >> 172.21.15.106:0/4123331066 conn(0x7f07940d2550 msgr2=0x7f07940d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.445+0000 7f079b680640 1 -- 172.21.15.106:0/4123331066 shutdown_connections 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.445+0000 7f079b680640 1 -- 172.21.15.106:0/4123331066 wait complete. 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 Processor -- start 2024-04-13T03:19:17.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 -- start start 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079415ed60 0x7f07940c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07940d1e20 con 0x7f07940cd550 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f079415fb60 con 0x7f079415ed60 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079b680640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f07940d1fa0 con 0x7f079415e980 2024-04-13T03:19:17.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079941c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079941c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40046/0 (socket says 172.21.15.106:40046) 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.446+0000 7f079941c640 1 -- 172.21.15.106:0/2904434198 learned_addr learned my addr 172.21.15.106:0/2904434198 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 -- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 msgr2=0x7f0794150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 -- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079415ed60 msgr2=0x7f07940c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079415ed60 0x7f07940c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 -- 172.21.15.106:0/2904434198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07940cab30 con 0x7f07940cd550 2024-04-13T03:19:17.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079941c640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f078400ce10 tx=0x7f078400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784003b90 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0784003d30 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07940cae10 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784004240 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.447+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07940cb3d0 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.448+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0794069120 con 0x7f07940cd550 2024-04-13T03:19:17.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.448+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07840043e0 con 0x7f07940cd550 2024-04-13T03:19:17.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.449+0000 7f0791ffb640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.449+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f078408c810 con 0x7f07940cd550 2024-04-13T03:19:17.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.449+0000 7f0798c1b640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.449+0000 7f0798c1b640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0788000f60 tx=0x7f0788007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.453+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f078405aab0 con 0x7f07940cd550 2024-04-13T03:19:17.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.664+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f079406f250 con 0x7f07940cd550 2024-04-13T03:19:17.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.664+0000 7f0791ffb640 1 -- 172.21.15.106:0/2904434198 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5195 (secure 0 0 0) 0x7f078405a3b0 con 0x7f07940cd550 2024-04-13T03:19:17.665 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:17.666 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:16.657710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9084},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9084":{"gid":9084,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1427655446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1427655446},{"type":"v1","addr":"172.21.15.106:6837","nonce":1427655446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9084,"qdb_cluster":[9084]},"id":8}]} 2024-04-13T03:19:17.666 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 86 2024-04-13T03:19:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 msgr2=0x7f078006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0788000f60 tx=0x7f0788007a80 comp rx=0 tx=0).stop 2024-04-13T03:19:17.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 msgr2=0x7f079415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f078400ce10 tx=0x7f078400a660 comp rx=0 tx=0).stop 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 shutdown_connections 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f079415e980 0x7f0794150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079415ed60 0x7f07940c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 --2- 172.21.15.106:0/2904434198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07940cd550 0x7f079415b1c0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 >> 172.21.15.106:0/2904434198 conn(0x7f07940d2550 msgr2=0x7f07941608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 shutdown_connections 2024-04-13T03:19:17.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.666+0000 7f079b680640 1 -- 172.21.15.106:0/2904434198 wait complete. 2024-04-13T03:19:17.678 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:17.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 Processor -- start 2024-04-13T03:19:17.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 -- start start 2024-04-13T03:19:17.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbbbc15e9b0 0x7fbbbc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbbc0cfd20 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbbc162af0 con 0x7fbbbc15ed90 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.778+0000 7fbbc27e9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbbbc162dd0 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbbbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbbbc15e9b0 0x7fbbbc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbc0d86640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbbbc0b3d60 0x7fbbbc0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54866/0 (socket says 172.21.15.106:54866) 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbc0d86640 1 -- 172.21.15.106:0/1824828228 learned_addr learned my addr 172.21.15.106:0/1824828228 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbbbfff640 1 -- 172.21.15.106:0/1824828228 >> v1:172.21.15.114:6789/0 conn(0x7fbbbc15ed90 legacy=0x7fbbbc0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbbbfff640 1 -- 172.21.15.106:0/1824828228 >> v1:172.21.15.106:6789/0 conn(0x7fbbbc0b3d60 legacy=0x7fbbbc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbbbfff640 1 -- 172.21.15.106:0/1824828228 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc1630b0 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.779+0000 7fbbbaffd640 1 -- 172.21.15.106:0/1824828228 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548370013 0 0) 0x7fbbbc0cfd20 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbbbfff640 1 --2- 172.21.15.106:0/1824828228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc15e9b0 0x7fbbbc0d2300 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ecee01ea4a474b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbbaffd640 1 -- 172.21.15.106:0/1824828228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac02f810 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbbaffd640 1 -- 172.21.15.106:0/1824828228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac02f9b0 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbbaffd640 1 -- 172.21.15.106:0/1824828228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac02fd80 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 -- 172.21.15.106:0/1824828228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc15e9b0 msgr2=0x7fbbbc0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/1824828228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc15e9b0 0x7fbbbc0d2300 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02b9f0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 -- 172.21.15.106:0/1824828228 shutdown_connections 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/1824828228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc15e9b0 0x7fbbbc0d2300 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 -- 172.21.15.106:0/1824828228 >> 172.21.15.106:0/1824828228 conn(0x7fbbbc08bf10 msgr2=0x7fbbbc0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 -- 172.21.15.106:0/1824828228 shutdown_connections 2024-04-13T03:19:17.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.780+0000 7fbbc27e9640 1 -- 172.21.15.106:0/1824828228 wait complete. 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.781+0000 7fbbc27e9640 1 Processor -- start 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 -- start start 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbbbc15e9b0 0x7fbbbc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbbbc0cbd40 0x7fbbbc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc162800 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc162680 con 0x7fbbbc0cbd40 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fbbbc0d4d00 con 0x7fbbbc15e9b0 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40066/0 (socket says 172.21.15.106:40066) 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.782+0000 7fbbbbfff640 1 -- 172.21.15.106:0/3510920216 learned_addr learned my addr 172.21.15.106:0/3510920216 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 -- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbbbc15e9b0 msgr2=0x7fbbbc0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbbbc15e9b0 0x7fbbbc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 -- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbbbc0cbd40 msgr2=0x7fbbbc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbbbc0cbd40 0x7fbbbc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 -- 172.21.15.106:0/3510920216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc0a1f70 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbbbfff640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac046070 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac037c40 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbbc06e120 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac02fe60 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.783+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbbc0b4e40 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.784+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbbc068a10 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.784+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbac049070 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.785+0000 7fbbb8ff9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 0x7fbba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.785+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fbbac0ad620 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.785+0000 7fbbbb7fe640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 0x7fbba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:17.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.785+0000 7fbbbb7fe640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 0x7fbba806edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0007fd0 tx=0x7fbbb00079a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:17.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.789+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbac03e070 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.963+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbbc0d46e0 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.965+0000 7fbbb8ff9640 1 -- 172.21.15.106:0/3510920216 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16988 (secure 0 0 0) 0x7fbbac033030 con 0x7fbbbc0b3d60 2024-04-13T03:19:17.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:17.966 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":121,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:16.039307+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 msgr2=0x7fbba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 0x7fbba806edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0007fd0 tx=0x7fbbb00079a0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 msgr2=0x7fbbbc0a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac0046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 shutdown_connections 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbba806c8f0 0x7fbba806edb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbbbc15e9b0 0x7fbbbc0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbbbc0cbd40 0x7fbbbc06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 --2- 172.21.15.106:0/3510920216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbbc0b3d60 0x7fbbbc0a1a30 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 >> 172.21.15.106:0/3510920216 conn(0x7fbbbc08bf10 msgr2=0x7fbbbc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 shutdown_connections 2024-04-13T03:19:17.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:17.967+0000 7fbbc27e9640 1 -- 172.21.15.106:0/3510920216 wait complete. 2024-04-13T03:19:17.979 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T03:19:17.979 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:19:18.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.080+0000 7ff8a1a08640 1 Processor -- start 2024-04-13T03:19:18.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.080+0000 7ff8a1a08640 1 -- start start 2024-04-13T03:19:18.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff8a1a08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c15e9b0 0x7ff89c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff8a1a08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff89c15ed90 0x7ff89c0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff8a1a08640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89c0cfd20 con 0x7ff89c0b3d60 2024-04-13T03:19:18.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff8a1a08640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff89c162af0 con 0x7ff89c15ed90 2024-04-13T03:19:18.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff8a1a08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff89c162dd0 con 0x7ff89c15e9b0 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff89c15ed90 0x7ff89c0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff89c15ed90 0x7ff89c0d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40068/0 (socket says 172.21.15.106:40068) 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 -- 172.21.15.106:0/401938207 learned_addr learned my addr 172.21.15.106:0/401938207 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff89affd640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c15e9b0 0x7ff89c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 -- 172.21.15.106:0/401938207 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c15e9b0 msgr2=0x7ff89c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c15e9b0 0x7ff89c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 -- 172.21.15.106:0/401938207 >> v1:172.21.15.114:6789/0 conn(0x7ff89c0b3d60 legacy=0x7ff89c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.081+0000 7ff893fff640 1 -- 172.21.15.106:0/401938207 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1630b0 con 0x7ff89c15ed90 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.082+0000 7ff893fff640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c15ed90 0x7ff89c0d45f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff89c08dde0 tx=0x7ff88402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a650be066496b785 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.082+0000 7ff89a7fc640 1 -- 172.21.15.106:0/401938207 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884004230 con 0x7ff89c15ed90 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.082+0000 7ff89a7fc640 1 -- 172.21.15.106:0/401938207 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8840043d0 con 0x7ff89c15ed90 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.082+0000 7ff89a7fc640 1 -- 172.21.15.106:0/401938207 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884003810 con 0x7ff89c15ed90 2024-04-13T03:19:18.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 -- 172.21.15.106:0/401938207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c15ed90 msgr2=0x7ff89c0d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c15ed90 0x7ff89c0d45f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff89c08dde0 tx=0x7ff88402b960 comp rx=0 tx=0).stop 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 -- 172.21.15.106:0/401938207 shutdown_connections 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c15ed90 0x7ff89c0d45f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/401938207 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c15e9b0 0x7ff89c0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 -- 172.21.15.106:0/401938207 >> 172.21.15.106:0/401938207 conn(0x7ff89c08bf10 msgr2=0x7ff89c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 -- 172.21.15.106:0/401938207 shutdown_connections 2024-04-13T03:19:18.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.083+0000 7ff8a1a08640 1 -- 172.21.15.106:0/401938207 wait complete. 2024-04-13T03:19:18.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 Processor -- start 2024-04-13T03:19:18.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 -- start start 2024-04-13T03:19:18.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c0b3d60 0x7ff89c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff89c15e9b0 0x7ff89c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c162800 con 0x7ff89c0cae00 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c162680 con 0x7ff89c15e9b0 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.084+0000 7ff8a1a08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff89c06eb20 con 0x7ff89c0b3d60 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40076/0 (socket says 172.21.15.106:40076) 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 -- 172.21.15.106:0/1914521006 learned_addr learned my addr 172.21.15.106:0/1914521006 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff893fff640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff89c15e9b0 0x7ff89c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89affd640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c0b3d60 0x7ff89c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 -- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c0b3d60 msgr2=0x7ff89c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c0b3d60 0x7ff89c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 -- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff89c15e9b0 msgr2=0x7ff89c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff89c15e9b0 0x7ff89c0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 -- 172.21.15.106:0/1914521006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c159fc0 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff89b7fe640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c730 tx=0x7ff89400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894016020 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89400aa50 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89400ce40 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c06baf0 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.085+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c06c0b0 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.086+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c068a10 con 0x7ff89c0cae00 2024-04-13T03:19:18.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.087+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff894019020 con 0x7ff89c0cae00 2024-04-13T03:19:18.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.087+0000 7ff898ff9640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 0x7ff87c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.087+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7ff89408bc60 con 0x7ff89c0cae00 2024-04-13T03:19:18.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.087+0000 7ff89affd640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 0x7ff87c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.088+0000 7ff89affd640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 0x7ff87c06ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006fd0 tx=0x7ff88c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.092+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff894020030 con 0x7ff89c0cae00 2024-04-13T03:19:18.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.263+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff89c0b2630 con 0x7ff87c06c920 2024-04-13T03:19:18.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.269+0000 7ff898ff9640 1 -- 172.21.15.106:0/1914521006 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237281 (secure 0 0 0) 0x7ff89c0b2630 con 0x7ff87c06c920 2024-04-13T03:19:18.274 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:18.276 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:19:18.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 msgr2=0x7ff87c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 0x7ff87c06ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006fd0 tx=0x7ff88c008040 comp rx=0 tx=0).stop 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 msgr2=0x7ff89c0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c730 tx=0x7ff89400cbf0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 shutdown_connections 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff87c06c920 0x7ff87c06ede0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff89c0cae00 0x7ff89c0cd270 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff89c15e9b0 0x7ff89c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 --2- 172.21.15.106:0/1914521006 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff89c0b3d60 0x7ff89c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 >> 172.21.15.106:0/1914521006 conn(0x7ff89c08bf10 msgr2=0x7ff89c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 shutdown_connections 2024-04-13T03:19:18.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.272+0000 7ff8a1a08640 1 -- 172.21.15.106:0/1914521006 wait complete. 2024-04-13T03:19:18.284 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-13T03:19:16.495282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262040,"kb_used_data":12552,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712680,"statfs":{"total":773094113280,"available":772825784320,"internally_reserved":0,"allocated":12853248,"data_stored":6609928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000599"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:14.280820+0000","last_change":"2024-04-13T03:19:14.280820+0000","last_active":"2024-04-13T03:19:14.280820+0000","last_peered":"2024-04-13T03:19:14.280820+0000","last_clean":"2024-04-13T03:19:14.280820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:14.280820+0000","last_undegraded":"2024-04-13T03:19:14.280820+0000","last_fullsized":"2024-04-13T03:19:14.280820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-13T03:19:13.276570+0000","last_change":"2024-04-13T03:19:13.276570+0000","last_active":"2024-04-13T03:19:13.276570+0000","last_peered":"2024-04-13T03:19:13.276570+0000","last_clean":"2024-04-13T03:19:13.276570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-13T03:19:13.276570+0000","last_undegraded":"2024-04-13T03:19:13.276570+0000","last_fullsized":"2024-04-13T03:19:13.276570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":349,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-13T03:19:11.380227+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:11.380227+0000","last_peered":"2024-04-13T03:19:11.380227+0000","last_clean":"2024-04-13T03:19:11.380227+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:11.380227+0000","last_undegraded":"2024-04-13T03:19:11.380227+0000","last_fullsized":"2024-04-13T03:19:11.380227+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2052096,"data_stored":1269017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32616,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":678649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2052096,"data_stored":1269017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:19:18.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:19:18.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.436+0000 7ff85fea2640 1 Processor -- start 2024-04-13T03:19:18.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.437+0000 7ff85fea2640 1 -- start start 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.437+0000 7ff85fea2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8580c60a0 0x7ff858160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.437+0000 7ff85fea2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8580d2310 con 0x7ff85815c7d0 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.437+0000 7ff85fea2640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8580b1f30 con 0x7ff8580cd2b0 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.437+0000 7ff85fea2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff8580b2210 con 0x7ff8580c60a0 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85e43f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8580c60a0 0x7ff858160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85dc3e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff85815c7d0 0x7ff8580ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54870/0 (socket says 172.21.15.106:54870) 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85dc3e640 1 -- 172.21.15.106:0/4211892225 learned_addr learned my addr 172.21.15.106:0/4211892225 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85e43f640 1 -- 172.21.15.106:0/4211892225 >> v1:172.21.15.114:6789/0 conn(0x7ff8580cd2b0 legacy=0x7ff8580b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85e43f640 1 -- 172.21.15.106:0/4211892225 >> v1:172.21.15.106:6789/0 conn(0x7ff85815c7d0 legacy=0x7ff8580ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.438+0000 7ff85e43f640 1 -- 172.21.15.106:0/4211892225 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8580b24f0 con 0x7ff8580c60a0 2024-04-13T03:19:18.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85e43f640 1 --2- 172.21.15.106:0/4211892225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580c60a0 0x7ff858160dd0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85002bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c524c092cb4f114 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85cc3c640 1 -- 172.21.15.106:0/4211892225 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850039070 con 0x7ff8580c60a0 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85cc3c640 1 -- 172.21.15.106:0/4211892225 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85002ca20 con 0x7ff8580c60a0 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85cc3c640 1 -- 172.21.15.106:0/4211892225 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85002cdf0 con 0x7ff8580c60a0 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 -- 172.21.15.106:0/4211892225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580c60a0 msgr2=0x7ff858160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 --2- 172.21.15.106:0/4211892225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580c60a0 0x7ff858160dd0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85002bcf0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 -- 172.21.15.106:0/4211892225 shutdown_connections 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 --2- 172.21.15.106:0/4211892225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580c60a0 0x7ff858160dd0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 -- 172.21.15.106:0/4211892225 >> 172.21.15.106:0/4211892225 conn(0x7ff8580d2550 msgr2=0x7ff8580d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 -- 172.21.15.106:0/4211892225 shutdown_connections 2024-04-13T03:19:18.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.439+0000 7ff85fea2640 1 -- 172.21.15.106:0/4211892225 wait complete. 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.440+0000 7ff85fea2640 1 Processor -- start 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.440+0000 7ff85fea2640 1 -- start start 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8580c60a0 0x7ff85815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff85815c7d0 0x7ff858145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff85815cbb0 con 0x7ff8580cd2b0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8580d1c10 con 0x7ff85815c7d0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85fea2640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff858154ed0 con 0x7ff8580c60a0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85d43d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85d43d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40098/0 (socket says 172.21.15.106:40098) 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85d43d640 1 -- 172.21.15.106:0/1502076970 learned_addr learned my addr 172.21.15.106:0/1502076970 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85d43d640 1 -- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8580c60a0 msgr2=0x7ff85815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.441+0000 7ff85d43d640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8580c60a0 0x7ff85815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85d43d640 1 -- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff85815c7d0 msgr2=0x7ff858145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85e43f640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff85815c7d0 0x7ff858145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85d43d640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff85815c7d0 0x7ff858145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85d43d640 1 -- 172.21.15.106:0/1502076970 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8580b56c0 con 0x7ff8580cd2b0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85d43d640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c410 tx=0x7ff84800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848017070 con 0x7ff8580cd2b0 2024-04-13T03:19:18.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8480035f0 con 0x7ff8580cd2b0 2024-04-13T03:19:18.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85815eac0 con 0x7ff8580cd2b0 2024-04-13T03:19:18.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848003a20 con 0x7ff8580cd2b0 2024-04-13T03:19:18.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.442+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85815f080 con 0x7ff8580cd2b0 2024-04-13T03:19:18.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.443+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff858069120 con 0x7ff8580cd2b0 2024-04-13T03:19:18.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.443+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff84801a070 con 0x7ff8580cd2b0 2024-04-13T03:19:18.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.444+0000 7ff84e7fc640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.444+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7ff84808b250 con 0x7ff8580cd2b0 2024-04-13T03:19:18.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.444+0000 7ff85dc3e640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.444+0000 7ff85dc3e640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 0x7ff83c06ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff854007fd0 tx=0x7ff8540079a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.448+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff848059460 con 0x7ff8580cd2b0 2024-04-13T03:19:18.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.622+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff85815e8b0 con 0x7ff83c06c920 2024-04-13T03:19:18.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.628+0000 7ff84e7fc640 1 -- 172.21.15.106:0/1502076970 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7ff85815e8b0 con 0x7ff83c06c920 2024-04-13T03:19:18.631 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:18.632 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 msgr2=0x7ff83c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 0x7ff83c06ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff854007fd0 tx=0x7ff8540079a0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 msgr2=0x7ff8580cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c410 tx=0x7ff84800c8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 shutdown_connections 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff83c06c920 0x7ff83c06ede0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff85815c7d0 0x7ff858145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8580cd2b0 0x7ff8580cbe80 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 --2- 172.21.15.106:0/1502076970 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff8580c60a0 0x7ff85815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 >> 172.21.15.106:0/1502076970 conn(0x7ff8580d2550 msgr2=0x7ff8580c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:18.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 shutdown_connections 2024-04-13T03:19:18.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.631+0000 7ff85fea2640 1 -- 172.21.15.106:0/1502076970 wait complete. 2024-04-13T03:19:18.643 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-04-13T03:19:18.495683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262436,"kb_used_data":12952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712284,"statfs":{"total":773094113280,"available":772825378816,"internally_reserved":0,"allocated":13262848,"data_stored":6801860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.214489"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.396275+0000","last_change":"2024-04-13T03:19:15.057117+0000","last_active":"2024-04-13T03:19:17.396275+0000","last_peered":"2024-04-13T03:19:17.396275+0000","last_clean":"2024-04-13T03:19:17.396275+0000","last_became_active":"2024-04-13T03:19:15.056802+0000","last_became_peered":"2024-04-13T03:19:15.056802+0000","last_unstale":"2024-04-13T03:19:17.396275+0000","last_undegraded":"2024-04-13T03:19:17.396275+0000","last_fullsized":"2024-04-13T03:19:17.396275+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:13:26.030621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894535+0000","last_change":"2024-04-13T03:19:15.060613+0000","last_active":"2024-04-13T03:19:16.894535+0000","last_peered":"2024-04-13T03:19:16.894535+0000","last_clean":"2024-04-13T03:19:16.894535+0000","last_became_active":"2024-04-13T03:19:15.059553+0000","last_became_peered":"2024-04-13T03:19:15.059553+0000","last_unstale":"2024-04-13T03:19:16.894535+0000","last_undegraded":"2024-04-13T03:19:16.894535+0000","last_fullsized":"2024-04-13T03:19:16.894535+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:39:11.981544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894487+0000","last_change":"2024-04-13T03:19:15.059896+0000","last_active":"2024-04-13T03:19:16.894487+0000","last_peered":"2024-04-13T03:19:16.894487+0000","last_clean":"2024-04-13T03:19:16.894487+0000","last_became_active":"2024-04-13T03:19:15.058641+0000","last_became_peered":"2024-04-13T03:19:15.058641+0000","last_unstale":"2024-04-13T03:19:16.894487+0000","last_undegraded":"2024-04-13T03:19:16.894487+0000","last_fullsized":"2024-04-13T03:19:16.894487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:33.704488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395517+0000","last_change":"2024-04-13T03:19:15.060255+0000","last_active":"2024-04-13T03:19:17.395517+0000","last_peered":"2024-04-13T03:19:17.395517+0000","last_clean":"2024-04-13T03:19:17.395517+0000","last_became_active":"2024-04-13T03:19:15.059959+0000","last_became_peered":"2024-04-13T03:19:15.059959+0000","last_unstale":"2024-04-13T03:19:17.395517+0000","last_undegraded":"2024-04-13T03:19:17.395517+0000","last_fullsized":"2024-04-13T03:19:17.395517+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:04:38.872483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395222+0000","last_change":"2024-04-13T03:19:15.058232+0000","last_active":"2024-04-13T03:19:17.395222+0000","last_peered":"2024-04-13T03:19:17.395222+0000","last_clean":"2024-04-13T03:19:17.395222+0000","last_became_active":"2024-04-13T03:19:15.057934+0000","last_became_peered":"2024-04-13T03:19:15.057934+0000","last_unstale":"2024-04-13T03:19:17.395222+0000","last_undegraded":"2024-04-13T03:19:17.395222+0000","last_fullsized":"2024-04-13T03:19:17.395222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:39.841893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045865+0000","last_change":"2024-04-13T03:19:15.056940+0000","last_active":"2024-04-13T03:19:16.045865+0000","last_peered":"2024-04-13T03:19:16.045865+0000","last_clean":"2024-04-13T03:19:16.045865+0000","last_became_active":"2024-04-13T03:19:15.056637+0000","last_became_peered":"2024-04-13T03:19:15.056637+0000","last_unstale":"2024-04-13T03:19:16.045865+0000","last_undegraded":"2024-04-13T03:19:16.045865+0000","last_fullsized":"2024-04-13T03:19:16.045865+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:53:37.507863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045848+0000","last_change":"2024-04-13T03:19:15.055956+0000","last_active":"2024-04-13T03:19:16.045848+0000","last_peered":"2024-04-13T03:19:16.045848+0000","last_clean":"2024-04-13T03:19:16.045848+0000","last_became_active":"2024-04-13T03:19:15.055624+0000","last_became_peered":"2024-04-13T03:19:15.055624+0000","last_unstale":"2024-04-13T03:19:16.045848+0000","last_undegraded":"2024-04-13T03:19:16.045848+0000","last_fullsized":"2024-04-13T03:19:16.045848+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:03:03.772181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894099+0000","last_change":"2024-04-13T03:19:15.059925+0000","last_active":"2024-04-13T03:19:16.894099+0000","last_peered":"2024-04-13T03:19:16.894099+0000","last_clean":"2024-04-13T03:19:16.894099+0000","last_became_active":"2024-04-13T03:19:15.058305+0000","last_became_peered":"2024-04-13T03:19:15.058305+0000","last_unstale":"2024-04-13T03:19:16.894099+0000","last_undegraded":"2024-04-13T03:19:16.894099+0000","last_fullsized":"2024-04-13T03:19:16.894099+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:26:05.822687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395959+0000","last_change":"2024-04-13T03:19:15.061029+0000","last_active":"2024-04-13T03:19:17.395959+0000","last_peered":"2024-04-13T03:19:17.395959+0000","last_clean":"2024-04-13T03:19:17.395959+0000","last_became_active":"2024-04-13T03:19:15.060787+0000","last_became_peered":"2024-04-13T03:19:15.060787+0000","last_unstale":"2024-04-13T03:19:17.395959+0000","last_undegraded":"2024-04-13T03:19:17.395959+0000","last_fullsized":"2024-04-13T03:19:17.395959+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:05.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394986+0000","last_change":"2024-04-13T03:19:15.059131+0000","last_active":"2024-04-13T03:19:17.394986+0000","last_peered":"2024-04-13T03:19:17.394986+0000","last_clean":"2024-04-13T03:19:17.394986+0000","last_became_active":"2024-04-13T03:19:15.058855+0000","last_became_peered":"2024-04-13T03:19:15.058855+0000","last_unstale":"2024-04-13T03:19:17.394986+0000","last_undegraded":"2024-04-13T03:19:17.394986+0000","last_fullsized":"2024-04-13T03:19:17.394986+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:25:41.009856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395843+0000","last_change":"2024-04-13T03:19:15.063613+0000","last_active":"2024-04-13T03:19:17.395843+0000","last_peered":"2024-04-13T03:19:17.395843+0000","last_clean":"2024-04-13T03:19:17.395843+0000","last_became_active":"2024-04-13T03:19:15.063318+0000","last_became_peered":"2024-04-13T03:19:15.063318+0000","last_unstale":"2024-04-13T03:19:17.395843+0000","last_undegraded":"2024-04-13T03:19:17.395843+0000","last_fullsized":"2024-04-13T03:19:17.395843+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:00:57.216395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046180+0000","last_change":"2024-04-13T03:19:15.057157+0000","last_active":"2024-04-13T03:19:16.046180+0000","last_peered":"2024-04-13T03:19:16.046180+0000","last_clean":"2024-04-13T03:19:16.046180+0000","last_became_active":"2024-04-13T03:19:15.056878+0000","last_became_peered":"2024-04-13T03:19:15.056878+0000","last_unstale":"2024-04-13T03:19:16.046180+0000","last_undegraded":"2024-04-13T03:19:16.046180+0000","last_fullsized":"2024-04-13T03:19:16.046180+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:23:01.472885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394716+0000","last_change":"2024-04-13T03:19:15.062788+0000","last_active":"2024-04-13T03:19:17.394716+0000","last_peered":"2024-04-13T03:19:17.394716+0000","last_clean":"2024-04-13T03:19:17.394716+0000","last_became_active":"2024-04-13T03:19:15.062363+0000","last_became_peered":"2024-04-13T03:19:15.062363+0000","last_unstale":"2024-04-13T03:19:17.394716+0000","last_undegraded":"2024-04-13T03:19:17.394716+0000","last_fullsized":"2024-04-13T03:19:17.394716+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:47:24.664344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894349+0000","last_change":"2024-04-13T03:19:15.060987+0000","last_active":"2024-04-13T03:19:16.894349+0000","last_peered":"2024-04-13T03:19:16.894349+0000","last_clean":"2024-04-13T03:19:16.894349+0000","last_became_active":"2024-04-13T03:19:15.060719+0000","last_became_peered":"2024-04-13T03:19:15.060719+0000","last_unstale":"2024-04-13T03:19:16.894349+0000","last_undegraded":"2024-04-13T03:19:16.894349+0000","last_fullsized":"2024-04-13T03:19:16.894349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:00:24.616872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395170+0000","last_change":"2024-04-13T03:19:15.064123+0000","last_active":"2024-04-13T03:19:17.395170+0000","last_peered":"2024-04-13T03:19:17.395170+0000","last_clean":"2024-04-13T03:19:17.395170+0000","last_became_active":"2024-04-13T03:19:15.063828+0000","last_became_peered":"2024-04-13T03:19:15.063828+0000","last_unstale":"2024-04-13T03:19:17.395170+0000","last_undegraded":"2024-04-13T03:19:17.395170+0000","last_fullsized":"2024-04-13T03:19:17.395170+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:27:46.512706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045696+0000","last_change":"2024-04-13T03:19:15.047290+0000","last_active":"2024-04-13T03:19:16.045696+0000","last_peered":"2024-04-13T03:19:16.045696+0000","last_clean":"2024-04-13T03:19:16.045696+0000","last_became_active":"2024-04-13T03:19:15.047033+0000","last_became_peered":"2024-04-13T03:19:15.047033+0000","last_unstale":"2024-04-13T03:19:16.045696+0000","last_undegraded":"2024-04-13T03:19:16.045696+0000","last_fullsized":"2024-04-13T03:19:16.045696+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:28.630466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394869+0000","last_change":"2024-04-13T03:19:15.062616+0000","last_active":"2024-04-13T03:19:17.394869+0000","last_peered":"2024-04-13T03:19:17.394869+0000","last_clean":"2024-04-13T03:19:17.394869+0000","last_became_active":"2024-04-13T03:19:15.062342+0000","last_became_peered":"2024-04-13T03:19:15.062342+0000","last_unstale":"2024-04-13T03:19:17.394869+0000","last_undegraded":"2024-04-13T03:19:17.394869+0000","last_fullsized":"2024-04-13T03:19:17.394869+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:23.368456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894429+0000","last_change":"2024-04-13T03:19:15.060355+0000","last_active":"2024-04-13T03:19:16.894429+0000","last_peered":"2024-04-13T03:19:16.894429+0000","last_clean":"2024-04-13T03:19:16.894429+0000","last_became_active":"2024-04-13T03:19:15.060085+0000","last_became_peered":"2024-04-13T03:19:15.060085+0000","last_unstale":"2024-04-13T03:19:16.894429+0000","last_undegraded":"2024-04-13T03:19:16.894429+0000","last_fullsized":"2024-04-13T03:19:16.894429+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:24:49.422534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314106+0000","last_change":"2024-04-13T03:19:15.062992+0000","last_active":"2024-04-13T03:19:17.314106+0000","last_peered":"2024-04-13T03:19:17.314106+0000","last_clean":"2024-04-13T03:19:17.314106+0000","last_became_active":"2024-04-13T03:19:15.062724+0000","last_became_peered":"2024-04-13T03:19:15.062724+0000","last_unstale":"2024-04-13T03:19:17.314106+0000","last_undegraded":"2024-04-13T03:19:17.314106+0000","last_fullsized":"2024-04-13T03:19:17.314106+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:33.075005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044679+0000","last_change":"2024-04-13T03:19:15.049611+0000","last_active":"2024-04-13T03:19:16.044679+0000","last_peered":"2024-04-13T03:19:16.044679+0000","last_clean":"2024-04-13T03:19:16.044679+0000","last_became_active":"2024-04-13T03:19:15.049348+0000","last_became_peered":"2024-04-13T03:19:15.049348+0000","last_unstale":"2024-04-13T03:19:16.044679+0000","last_undegraded":"2024-04-13T03:19:16.044679+0000","last_fullsized":"2024-04-13T03:19:16.044679+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:04:02.011599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395876+0000","last_change":"2024-04-13T03:19:15.058262+0000","last_active":"2024-04-13T03:19:17.395876+0000","last_peered":"2024-04-13T03:19:17.395876+0000","last_clean":"2024-04-13T03:19:17.395876+0000","last_became_active":"2024-04-13T03:19:15.057989+0000","last_became_peered":"2024-04-13T03:19:15.057989+0000","last_unstale":"2024-04-13T03:19:17.395876+0000","last_undegraded":"2024-04-13T03:19:17.395876+0000","last_fullsized":"2024-04-13T03:19:17.395876+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:42.062136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395378+0000","last_change":"2024-04-13T03:19:15.057537+0000","last_active":"2024-04-13T03:19:17.395378+0000","last_peered":"2024-04-13T03:19:17.395378+0000","last_clean":"2024-04-13T03:19:17.395378+0000","last_became_active":"2024-04-13T03:19:15.057032+0000","last_became_peered":"2024-04-13T03:19:15.057032+0000","last_unstale":"2024-04-13T03:19:17.395378+0000","last_undegraded":"2024-04-13T03:19:17.395378+0000","last_fullsized":"2024-04-13T03:19:17.395378+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:11:33.698531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.396203+0000","last_change":"2024-04-13T03:19:15.061724+0000","last_active":"2024-04-13T03:19:17.396203+0000","last_peered":"2024-04-13T03:19:17.396203+0000","last_clean":"2024-04-13T03:19:17.396203+0000","last_became_active":"2024-04-13T03:19:15.061379+0000","last_became_peered":"2024-04-13T03:19:15.061379+0000","last_unstale":"2024-04-13T03:19:17.396203+0000","last_undegraded":"2024-04-13T03:19:17.396203+0000","last_fullsized":"2024-04-13T03:19:17.396203+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:45:14.495165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044945+0000","last_change":"2024-04-13T03:19:15.058408+0000","last_active":"2024-04-13T03:19:16.044945+0000","last_peered":"2024-04-13T03:19:16.044945+0000","last_clean":"2024-04-13T03:19:16.044945+0000","last_became_active":"2024-04-13T03:19:15.058124+0000","last_became_peered":"2024-04-13T03:19:15.058124+0000","last_unstale":"2024-04-13T03:19:16.044945+0000","last_undegraded":"2024-04-13T03:19:16.044945+0000","last_fullsized":"2024-04-13T03:19:16.044945+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:42.977197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045469+0000","last_change":"2024-04-13T03:19:15.051223+0000","last_active":"2024-04-13T03:19:16.045469+0000","last_peered":"2024-04-13T03:19:16.045469+0000","last_clean":"2024-04-13T03:19:16.045469+0000","last_became_active":"2024-04-13T03:19:15.050970+0000","last_became_peered":"2024-04-13T03:19:15.050970+0000","last_unstale":"2024-04-13T03:19:16.045469+0000","last_undegraded":"2024-04-13T03:19:16.045469+0000","last_fullsized":"2024-04-13T03:19:16.045469+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:11:33.698531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044820+0000","last_change":"2024-04-13T03:19:15.052325+0000","last_active":"2024-04-13T03:19:16.044820+0000","last_peered":"2024-04-13T03:19:16.044820+0000","last_clean":"2024-04-13T03:19:16.044820+0000","last_became_active":"2024-04-13T03:19:15.052085+0000","last_became_peered":"2024-04-13T03:19:15.052085+0000","last_unstale":"2024-04-13T03:19:16.044820+0000","last_undegraded":"2024-04-13T03:19:16.044820+0000","last_fullsized":"2024-04-13T03:19:16.044820+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:02:37.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314147+0000","last_change":"2024-04-13T03:19:15.063667+0000","last_active":"2024-04-13T03:19:17.314147+0000","last_peered":"2024-04-13T03:19:17.314147+0000","last_clean":"2024-04-13T03:19:17.314147+0000","last_became_active":"2024-04-13T03:19:15.063345+0000","last_became_peered":"2024-04-13T03:19:15.063345+0000","last_unstale":"2024-04-13T03:19:17.314147+0000","last_undegraded":"2024-04-13T03:19:17.314147+0000","last_fullsized":"2024-04-13T03:19:17.314147+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:46:12.179196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046210+0000","last_change":"2024-04-13T03:19:15.047146+0000","last_active":"2024-04-13T03:19:16.046210+0000","last_peered":"2024-04-13T03:19:16.046210+0000","last_clean":"2024-04-13T03:19:16.046210+0000","last_became_active":"2024-04-13T03:19:15.046875+0000","last_became_peered":"2024-04-13T03:19:15.046875+0000","last_unstale":"2024-04-13T03:19:16.046210+0000","last_undegraded":"2024-04-13T03:19:16.046210+0000","last_fullsized":"2024-04-13T03:19:16.046210+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:02:25.627888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044875+0000","last_change":"2024-04-13T03:19:15.052261+0000","last_active":"2024-04-13T03:19:16.044875+0000","last_peered":"2024-04-13T03:19:16.044875+0000","last_clean":"2024-04-13T03:19:16.044875+0000","last_became_active":"2024-04-13T03:19:15.051988+0000","last_became_peered":"2024-04-13T03:19:15.051988+0000","last_unstale":"2024-04-13T03:19:16.044875+0000","last_undegraded":"2024-04-13T03:19:16.044875+0000","last_fullsized":"2024-04-13T03:19:16.044875+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:10:43.545854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894411+0000","last_change":"2024-04-13T03:19:15.055592+0000","last_active":"2024-04-13T03:19:16.894411+0000","last_peered":"2024-04-13T03:19:16.894411+0000","last_clean":"2024-04-13T03:19:16.894411+0000","last_became_active":"2024-04-13T03:19:15.055118+0000","last_became_peered":"2024-04-13T03:19:15.055118+0000","last_unstale":"2024-04-13T03:19:16.894411+0000","last_undegraded":"2024-04-13T03:19:16.894411+0000","last_fullsized":"2024-04-13T03:19:16.894411+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:27:40.998309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045062+0000","last_change":"2024-04-13T03:19:15.047466+0000","last_active":"2024-04-13T03:19:16.045062+0000","last_peered":"2024-04-13T03:19:16.045062+0000","last_clean":"2024-04-13T03:19:16.045062+0000","last_became_active":"2024-04-13T03:19:15.047168+0000","last_became_peered":"2024-04-13T03:19:15.047168+0000","last_unstale":"2024-04-13T03:19:16.045062+0000","last_undegraded":"2024-04-13T03:19:16.045062+0000","last_fullsized":"2024-04-13T03:19:16.045062+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:51.572154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045025+0000","last_change":"2024-04-13T03:19:15.060084+0000","last_active":"2024-04-13T03:19:16.045025+0000","last_peered":"2024-04-13T03:19:16.045025+0000","last_clean":"2024-04-13T03:19:16.045025+0000","last_became_active":"2024-04-13T03:19:15.059323+0000","last_became_peered":"2024-04-13T03:19:15.059323+0000","last_unstale":"2024-04-13T03:19:16.045025+0000","last_undegraded":"2024-04-13T03:19:16.045025+0000","last_fullsized":"2024-04-13T03:19:16.045025+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:25:41.009856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394887+0000","last_change":"2024-04-13T03:19:15.060301+0000","last_active":"2024-04-13T03:19:17.394887+0000","last_peered":"2024-04-13T03:19:17.394887+0000","last_clean":"2024-04-13T03:19:17.394887+0000","last_became_active":"2024-04-13T03:19:15.060025+0000","last_became_peered":"2024-04-13T03:19:15.060025+0000","last_unstale":"2024-04-13T03:19:17.394887+0000","last_undegraded":"2024-04-13T03:19:17.394887+0000","last_fullsized":"2024-04-13T03:19:17.394887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:36.839829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314112+0000","last_change":"2024-04-13T03:19:14.300681+0000","last_active":"2024-04-13T03:19:17.314112+0000","last_peered":"2024-04-13T03:19:17.314112+0000","last_clean":"2024-04-13T03:19:17.314112+0000","last_became_active":"2024-04-13T03:19:14.300232+0000","last_became_peered":"2024-04-13T03:19:14.300232+0000","last_unstale":"2024-04-13T03:19:17.314112+0000","last_undegraded":"2024-04-13T03:19:17.314112+0000","last_fullsized":"2024-04-13T03:19:17.314112+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:04.306037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046063+0000","last_change":"2024-04-13T03:19:15.056090+0000","last_active":"2024-04-13T03:19:16.046063+0000","last_peered":"2024-04-13T03:19:16.046063+0000","last_clean":"2024-04-13T03:19:16.046063+0000","last_became_active":"2024-04-13T03:19:15.055824+0000","last_became_peered":"2024-04-13T03:19:15.055824+0000","last_unstale":"2024-04-13T03:19:16.046063+0000","last_undegraded":"2024-04-13T03:19:16.046063+0000","last_fullsized":"2024-04-13T03:19:16.046063+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:59:32.719745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394932+0000","last_change":"2024-04-13T03:19:14.296916+0000","last_active":"2024-04-13T03:19:17.394932+0000","last_peered":"2024-04-13T03:19:17.394932+0000","last_clean":"2024-04-13T03:19:17.394932+0000","last_became_active":"2024-04-13T03:19:14.296636+0000","last_became_peered":"2024-04-13T03:19:14.296636+0000","last_unstale":"2024-04-13T03:19:17.394932+0000","last_undegraded":"2024-04-13T03:19:17.394932+0000","last_fullsized":"2024-04-13T03:19:17.394932+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:20:16.466243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045366+0000","last_change":"2024-04-13T03:19:15.049024+0000","last_active":"2024-04-13T03:19:16.045366+0000","last_peered":"2024-04-13T03:19:16.045366+0000","last_clean":"2024-04-13T03:19:16.045366+0000","last_became_active":"2024-04-13T03:19:15.048641+0000","last_became_peered":"2024-04-13T03:19:15.048641+0000","last_unstale":"2024-04-13T03:19:16.045366+0000","last_undegraded":"2024-04-13T03:19:16.045366+0000","last_fullsized":"2024-04-13T03:19:16.045366+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:39.841893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395971+0000","last_change":"2024-04-13T03:19:14.307682+0000","last_active":"2024-04-13T03:19:17.395971+0000","last_peered":"2024-04-13T03:19:17.395971+0000","last_clean":"2024-04-13T03:19:17.395971+0000","last_became_active":"2024-04-13T03:19:14.307415+0000","last_became_peered":"2024-04-13T03:19:14.307415+0000","last_unstale":"2024-04-13T03:19:17.395971+0000","last_undegraded":"2024-04-13T03:19:17.395971+0000","last_fullsized":"2024-04-13T03:19:17.395971+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:24.708120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.396132+0000","last_change":"2024-04-13T03:19:15.060194+0000","last_active":"2024-04-13T03:19:17.396132+0000","last_peered":"2024-04-13T03:19:17.396132+0000","last_clean":"2024-04-13T03:19:17.396132+0000","last_became_active":"2024-04-13T03:19:15.059930+0000","last_became_peered":"2024-04-13T03:19:15.059930+0000","last_unstale":"2024-04-13T03:19:17.396132+0000","last_undegraded":"2024-04-13T03:19:17.396132+0000","last_fullsized":"2024-04-13T03:19:17.396132+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:32:47.916494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394815+0000","last_change":"2024-04-13T03:19:14.304850+0000","last_active":"2024-04-13T03:19:17.394815+0000","last_peered":"2024-04-13T03:19:17.394815+0000","last_clean":"2024-04-13T03:19:17.394815+0000","last_became_active":"2024-04-13T03:19:14.304602+0000","last_became_peered":"2024-04-13T03:19:14.304602+0000","last_unstale":"2024-04-13T03:19:17.394815+0000","last_undegraded":"2024-04-13T03:19:17.394815+0000","last_fullsized":"2024-04-13T03:19:17.394815+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:48:52.947708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394989+0000","last_change":"2024-04-13T03:19:15.059788+0000","last_active":"2024-04-13T03:19:17.394989+0000","last_peered":"2024-04-13T03:19:17.394989+0000","last_clean":"2024-04-13T03:19:17.394989+0000","last_became_active":"2024-04-13T03:19:15.059487+0000","last_became_peered":"2024-04-13T03:19:15.059487+0000","last_unstale":"2024-04-13T03:19:17.394989+0000","last_undegraded":"2024-04-13T03:19:17.394989+0000","last_fullsized":"2024-04-13T03:19:17.394989+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:51.572154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395347+0000","last_change":"2024-04-13T03:19:14.299712+0000","last_active":"2024-04-13T03:19:17.395347+0000","last_peered":"2024-04-13T03:19:17.395347+0000","last_clean":"2024-04-13T03:19:17.395347+0000","last_became_active":"2024-04-13T03:19:14.299395+0000","last_became_peered":"2024-04-13T03:19:14.299395+0000","last_unstale":"2024-04-13T03:19:17.395347+0000","last_undegraded":"2024-04-13T03:19:17.395347+0000","last_fullsized":"2024-04-13T03:19:17.395347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:59:12.674326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395226+0000","last_change":"2024-04-13T03:19:15.058979+0000","last_active":"2024-04-13T03:19:17.395226+0000","last_peered":"2024-04-13T03:19:17.395226+0000","last_clean":"2024-04-13T03:19:17.395226+0000","last_became_active":"2024-04-13T03:19:15.058521+0000","last_became_peered":"2024-04-13T03:19:15.058521+0000","last_unstale":"2024-04-13T03:19:17.395226+0000","last_undegraded":"2024-04-13T03:19:17.395226+0000","last_fullsized":"2024-04-13T03:19:17.395226+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:01:22.440765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045923+0000","last_change":"2024-04-13T03:19:14.305704+0000","last_active":"2024-04-13T03:19:16.045923+0000","last_peered":"2024-04-13T03:19:16.045923+0000","last_clean":"2024-04-13T03:19:16.045923+0000","last_became_active":"2024-04-13T03:19:14.305417+0000","last_became_peered":"2024-04-13T03:19:14.305417+0000","last_unstale":"2024-04-13T03:19:16.045923+0000","last_undegraded":"2024-04-13T03:19:16.045923+0000","last_fullsized":"2024-04-13T03:19:16.045923+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:02.070377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894511+0000","last_change":"2024-04-13T03:19:15.055370+0000","last_active":"2024-04-13T03:19:16.894511+0000","last_peered":"2024-04-13T03:19:16.894511+0000","last_clean":"2024-04-13T03:19:16.894511+0000","last_became_active":"2024-04-13T03:19:15.054785+0000","last_became_peered":"2024-04-13T03:19:15.054785+0000","last_unstale":"2024-04-13T03:19:16.894511+0000","last_undegraded":"2024-04-13T03:19:16.894511+0000","last_fullsized":"2024-04-13T03:19:16.894511+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:35:52.429859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045014+0000","last_change":"2024-04-13T03:19:14.301646+0000","last_active":"2024-04-13T03:19:16.045014+0000","last_peered":"2024-04-13T03:19:16.045014+0000","last_clean":"2024-04-13T03:19:16.045014+0000","last_became_active":"2024-04-13T03:19:14.301384+0000","last_became_peered":"2024-04-13T03:19:14.301384+0000","last_unstale":"2024-04-13T03:19:16.045014+0000","last_undegraded":"2024-04-13T03:19:16.045014+0000","last_fullsized":"2024-04-13T03:19:16.045014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:06:32.172544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394900+0000","last_change":"2024-04-13T03:19:15.055206+0000","last_active":"2024-04-13T03:19:17.394900+0000","last_peered":"2024-04-13T03:19:17.394900+0000","last_clean":"2024-04-13T03:19:17.394900+0000","last_became_active":"2024-04-13T03:19:15.054944+0000","last_became_peered":"2024-04-13T03:19:15.054944+0000","last_unstale":"2024-04-13T03:19:17.394900+0000","last_undegraded":"2024-04-13T03:19:17.394900+0000","last_fullsized":"2024-04-13T03:19:17.394900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:42.977197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045959+0000","last_change":"2024-04-13T03:19:14.307728+0000","last_active":"2024-04-13T03:19:16.045959+0000","last_peered":"2024-04-13T03:19:16.045959+0000","last_clean":"2024-04-13T03:19:16.045959+0000","last_became_active":"2024-04-13T03:19:14.307471+0000","last_became_peered":"2024-04-13T03:19:14.307471+0000","last_unstale":"2024-04-13T03:19:16.045959+0000","last_undegraded":"2024-04-13T03:19:16.045959+0000","last_fullsized":"2024-04-13T03:19:16.045959+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:56.765277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.313932+0000","last_change":"2024-04-13T03:19:15.063137+0000","last_active":"2024-04-13T03:19:17.313932+0000","last_peered":"2024-04-13T03:19:17.313932+0000","last_clean":"2024-04-13T03:19:17.313932+0000","last_became_active":"2024-04-13T03:19:15.062854+0000","last_became_peered":"2024-04-13T03:19:15.062854+0000","last_unstale":"2024-04-13T03:19:17.313932+0000","last_undegraded":"2024-04-13T03:19:17.313932+0000","last_fullsized":"2024-04-13T03:19:17.313932+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:23:56.458663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314197+0000","last_change":"2024-04-13T03:19:14.303571+0000","last_active":"2024-04-13T03:19:17.314197+0000","last_peered":"2024-04-13T03:19:17.314197+0000","last_clean":"2024-04-13T03:19:17.314197+0000","last_became_active":"2024-04-13T03:19:14.303302+0000","last_became_peered":"2024-04-13T03:19:14.303302+0000","last_unstale":"2024-04-13T03:19:17.314197+0000","last_undegraded":"2024-04-13T03:19:17.314197+0000","last_fullsized":"2024-04-13T03:19:17.314197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:22.069976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045393+0000","last_change":"2024-04-13T03:19:15.055258+0000","last_active":"2024-04-13T03:19:16.045393+0000","last_peered":"2024-04-13T03:19:16.045393+0000","last_clean":"2024-04-13T03:19:16.045393+0000","last_became_active":"2024-04-13T03:19:15.054927+0000","last_became_peered":"2024-04-13T03:19:15.054927+0000","last_unstale":"2024-04-13T03:19:16.045393+0000","last_undegraded":"2024-04-13T03:19:16.045393+0000","last_fullsized":"2024-04-13T03:19:16.045393+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:01:22.440765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045320+0000","last_change":"2024-04-13T03:19:14.298792+0000","last_active":"2024-04-13T03:19:16.045320+0000","last_peered":"2024-04-13T03:19:16.045320+0000","last_clean":"2024-04-13T03:19:16.045320+0000","last_became_active":"2024-04-13T03:19:14.298535+0000","last_became_peered":"2024-04-13T03:19:14.298535+0000","last_unstale":"2024-04-13T03:19:16.045320+0000","last_undegraded":"2024-04-13T03:19:16.045320+0000","last_fullsized":"2024-04-13T03:19:16.045320+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:27:43.310884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894572+0000","last_change":"2024-04-13T03:19:15.058348+0000","last_active":"2024-04-13T03:19:16.894572+0000","last_peered":"2024-04-13T03:19:16.894572+0000","last_clean":"2024-04-13T03:19:16.894572+0000","last_became_active":"2024-04-13T03:19:15.057901+0000","last_became_peered":"2024-04-13T03:19:15.057901+0000","last_unstale":"2024-04-13T03:19:16.894572+0000","last_undegraded":"2024-04-13T03:19:16.894572+0000","last_fullsized":"2024-04-13T03:19:16.894572+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:28:02.400865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045346+0000","last_change":"2024-04-13T03:19:14.308173+0000","last_active":"2024-04-13T03:19:16.045346+0000","last_peered":"2024-04-13T03:19:16.045346+0000","last_clean":"2024-04-13T03:19:16.045346+0000","last_became_active":"2024-04-13T03:19:14.307890+0000","last_became_peered":"2024-04-13T03:19:14.307890+0000","last_unstale":"2024-04-13T03:19:16.045346+0000","last_undegraded":"2024-04-13T03:19:16.045346+0000","last_fullsized":"2024-04-13T03:19:16.045346+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:15.074962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045539+0000","last_change":"2024-04-13T03:19:15.047943+0000","last_active":"2024-04-13T03:19:16.045539+0000","last_peered":"2024-04-13T03:19:16.045539+0000","last_clean":"2024-04-13T03:19:16.045539+0000","last_became_active":"2024-04-13T03:19:15.047685+0000","last_became_peered":"2024-04-13T03:19:15.047685+0000","last_unstale":"2024-04-13T03:19:16.045539+0000","last_undegraded":"2024-04-13T03:19:16.045539+0000","last_fullsized":"2024-04-13T03:19:16.045539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:00:57.216395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894381+0000","last_change":"2024-04-13T03:19:14.307604+0000","last_active":"2024-04-13T03:19:16.894381+0000","last_peered":"2024-04-13T03:19:16.894381+0000","last_clean":"2024-04-13T03:19:16.894381+0000","last_became_active":"2024-04-13T03:19:14.307194+0000","last_became_peered":"2024-04-13T03:19:14.307194+0000","last_unstale":"2024-04-13T03:19:16.894381+0000","last_undegraded":"2024-04-13T03:19:16.894381+0000","last_fullsized":"2024-04-13T03:19:16.894381+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:38.837643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314097+0000","last_change":"2024-04-13T03:19:15.062337+0000","last_active":"2024-04-13T03:19:17.314097+0000","last_peered":"2024-04-13T03:19:17.314097+0000","last_clean":"2024-04-13T03:19:17.314097+0000","last_became_active":"2024-04-13T03:19:15.061197+0000","last_became_peered":"2024-04-13T03:19:15.061197+0000","last_unstale":"2024-04-13T03:19:17.314097+0000","last_undegraded":"2024-04-13T03:19:17.314097+0000","last_fullsized":"2024-04-13T03:19:17.314097+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:59:13.678575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894296+0000","last_change":"2024-04-13T03:19:14.308583+0000","last_active":"2024-04-13T03:19:16.894296+0000","last_peered":"2024-04-13T03:19:16.894296+0000","last_clean":"2024-04-13T03:19:16.894296+0000","last_became_active":"2024-04-13T03:19:14.308308+0000","last_became_peered":"2024-04-13T03:19:14.308308+0000","last_unstale":"2024-04-13T03:19:16.894296+0000","last_undegraded":"2024-04-13T03:19:16.894296+0000","last_fullsized":"2024-04-13T03:19:16.894296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:01:21.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394695+0000","last_change":"2024-04-13T03:19:15.058350+0000","last_active":"2024-04-13T03:19:17.394695+0000","last_peered":"2024-04-13T03:19:17.394695+0000","last_clean":"2024-04-13T03:19:17.394695+0000","last_became_active":"2024-04-13T03:19:15.058091+0000","last_became_peered":"2024-04-13T03:19:15.058091+0000","last_unstale":"2024-04-13T03:19:17.394695+0000","last_undegraded":"2024-04-13T03:19:17.394695+0000","last_fullsized":"2024-04-13T03:19:17.394695+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:33.075005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894516+0000","last_change":"2024-04-13T03:19:14.306447+0000","last_active":"2024-04-13T03:19:16.894516+0000","last_peered":"2024-04-13T03:19:16.894516+0000","last_clean":"2024-04-13T03:19:16.894516+0000","last_became_active":"2024-04-13T03:19:14.306152+0000","last_became_peered":"2024-04-13T03:19:14.306152+0000","last_unstale":"2024-04-13T03:19:16.894516+0000","last_undegraded":"2024-04-13T03:19:16.894516+0000","last_fullsized":"2024-04-13T03:19:16.894516+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:04:37.868234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044697+0000","last_change":"2024-04-13T03:19:15.052382+0000","last_active":"2024-04-13T03:19:16.044697+0000","last_peered":"2024-04-13T03:19:16.044697+0000","last_clean":"2024-04-13T03:19:16.044697+0000","last_became_active":"2024-04-13T03:19:15.052117+0000","last_became_peered":"2024-04-13T03:19:15.052117+0000","last_unstale":"2024-04-13T03:19:16.044697+0000","last_undegraded":"2024-04-13T03:19:16.044697+0000","last_fullsized":"2024-04-13T03:19:16.044697+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:48:02.349002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894262+0000","last_change":"2024-04-13T03:19:14.307000+0000","last_active":"2024-04-13T03:19:16.894262+0000","last_peered":"2024-04-13T03:19:16.894262+0000","last_clean":"2024-04-13T03:19:16.894262+0000","last_became_active":"2024-04-13T03:19:14.306737+0000","last_became_peered":"2024-04-13T03:19:14.306737+0000","last_unstale":"2024-04-13T03:19:16.894262+0000","last_undegraded":"2024-04-13T03:19:16.894262+0000","last_fullsized":"2024-04-13T03:19:16.894262+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:11:32.694281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045232+0000","last_change":"2024-04-13T03:19:15.050239+0000","last_active":"2024-04-13T03:19:16.045232+0000","last_peered":"2024-04-13T03:19:16.045232+0000","last_clean":"2024-04-13T03:19:16.045232+0000","last_became_active":"2024-04-13T03:19:15.049984+0000","last_became_peered":"2024-04-13T03:19:15.049984+0000","last_unstale":"2024-04-13T03:19:16.045232+0000","last_undegraded":"2024-04-13T03:19:16.045232+0000","last_fullsized":"2024-04-13T03:19:16.045232+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:27:46.512706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394942+0000","last_change":"2024-04-13T03:19:14.303764+0000","last_active":"2024-04-13T03:19:17.394942+0000","last_peered":"2024-04-13T03:19:17.394942+0000","last_clean":"2024-04-13T03:19:17.394942+0000","last_became_active":"2024-04-13T03:19:14.303351+0000","last_became_peered":"2024-04-13T03:19:14.303351+0000","last_unstale":"2024-04-13T03:19:17.394942+0000","last_undegraded":"2024-04-13T03:19:17.394942+0000","last_fullsized":"2024-04-13T03:19:17.394942+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:58.456239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395992+0000","last_change":"2024-04-13T03:19:15.058033+0000","last_active":"2024-04-13T03:19:17.395992+0000","last_peered":"2024-04-13T03:19:17.395992+0000","last_clean":"2024-04-13T03:19:17.395992+0000","last_became_active":"2024-04-13T03:19:15.057425+0000","last_became_peered":"2024-04-13T03:19:15.057425+0000","last_unstale":"2024-04-13T03:19:17.395992+0000","last_undegraded":"2024-04-13T03:19:17.395992+0000","last_fullsized":"2024-04-13T03:19:17.395992+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:04:38.872483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044791+0000","last_change":"2024-04-13T03:19:14.294135+0000","last_active":"2024-04-13T03:19:16.044791+0000","last_peered":"2024-04-13T03:19:16.044791+0000","last_clean":"2024-04-13T03:19:16.044791+0000","last_became_active":"2024-04-13T03:19:14.293886+0000","last_became_peered":"2024-04-13T03:19:14.293886+0000","last_unstale":"2024-04-13T03:19:16.044791+0000","last_undegraded":"2024-04-13T03:19:16.044791+0000","last_fullsized":"2024-04-13T03:19:16.044791+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:51:42.398818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046353+0000","last_change":"2024-04-13T03:19:15.055353+0000","last_active":"2024-04-13T03:19:16.046353+0000","last_peered":"2024-04-13T03:19:16.046353+0000","last_clean":"2024-04-13T03:19:16.046353+0000","last_became_active":"2024-04-13T03:19:15.055057+0000","last_became_peered":"2024-04-13T03:19:15.055057+0000","last_unstale":"2024-04-13T03:19:16.046353+0000","last_undegraded":"2024-04-13T03:19:16.046353+0000","last_fullsized":"2024-04-13T03:19:16.046353+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:24:36.464924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.396031+0000","last_change":"2024-04-13T03:19:14.306219+0000","last_active":"2024-04-13T03:19:17.396031+0000","last_peered":"2024-04-13T03:19:17.396031+0000","last_clean":"2024-04-13T03:19:17.396031+0000","last_became_active":"2024-04-13T03:19:14.305948+0000","last_became_peered":"2024-04-13T03:19:14.305948+0000","last_unstale":"2024-04-13T03:19:17.396031+0000","last_undegraded":"2024-04-13T03:19:17.396031+0000","last_fullsized":"2024-04-13T03:19:17.396031+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:35.835580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045567+0000","last_change":"2024-04-13T03:19:15.059825+0000","last_active":"2024-04-13T03:19:16.045567+0000","last_peered":"2024-04-13T03:19:16.045567+0000","last_clean":"2024-04-13T03:19:16.045567+0000","last_became_active":"2024-04-13T03:19:15.059561+0000","last_became_peered":"2024-04-13T03:19:15.059561+0000","last_unstale":"2024-04-13T03:19:16.045567+0000","last_undegraded":"2024-04-13T03:19:16.045567+0000","last_fullsized":"2024-04-13T03:19:16.045567+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:04:38.872483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395683+0000","last_change":"2024-04-13T03:19:14.304888+0000","last_active":"2024-04-13T03:19:17.395683+0000","last_peered":"2024-04-13T03:19:17.395683+0000","last_clean":"2024-04-13T03:19:17.395683+0000","last_became_active":"2024-04-13T03:19:14.304603+0000","last_became_peered":"2024-04-13T03:19:14.304603+0000","last_unstale":"2024-04-13T03:19:17.395683+0000","last_undegraded":"2024-04-13T03:19:17.395683+0000","last_fullsized":"2024-04-13T03:19:17.395683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:13:41.972947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045732+0000","last_change":"2024-04-13T03:19:15.059209+0000","last_active":"2024-04-13T03:19:16.045732+0000","last_peered":"2024-04-13T03:19:16.045732+0000","last_clean":"2024-04-13T03:19:16.045732+0000","last_became_active":"2024-04-13T03:19:15.058952+0000","last_became_peered":"2024-04-13T03:19:15.058952+0000","last_unstale":"2024-04-13T03:19:16.045732+0000","last_undegraded":"2024-04-13T03:19:16.045732+0000","last_fullsized":"2024-04-13T03:19:16.045732+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:39:18.084175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045823+0000","last_change":"2024-04-13T03:19:14.304816+0000","last_active":"2024-04-13T03:19:16.045823+0000","last_peered":"2024-04-13T03:19:16.045823+0000","last_clean":"2024-04-13T03:19:16.045823+0000","last_became_active":"2024-04-13T03:19:14.304548+0000","last_became_peered":"2024-04-13T03:19:14.304548+0000","last_unstale":"2024-04-13T03:19:16.045823+0000","last_undegraded":"2024-04-13T03:19:16.045823+0000","last_fullsized":"2024-04-13T03:19:16.045823+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:21.554045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894149+0000","last_change":"2024-04-13T03:19:15.060373+0000","last_active":"2024-04-13T03:19:16.894149+0000","last_peered":"2024-04-13T03:19:16.894149+0000","last_clean":"2024-04-13T03:19:16.894149+0000","last_became_active":"2024-04-13T03:19:15.059891+0000","last_became_peered":"2024-04-13T03:19:15.059891+0000","last_unstale":"2024-04-13T03:19:16.894149+0000","last_undegraded":"2024-04-13T03:19:16.894149+0000","last_fullsized":"2024-04-13T03:19:16.894149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:43.099952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395683+0000","last_change":"2024-04-13T03:19:14.305923+0000","last_active":"2024-04-13T03:19:17.395683+0000","last_peered":"2024-04-13T03:19:17.395683+0000","last_clean":"2024-04-13T03:19:17.395683+0000","last_became_active":"2024-04-13T03:19:14.305647+0000","last_became_peered":"2024-04-13T03:19:14.305647+0000","last_unstale":"2024-04-13T03:19:17.395683+0000","last_undegraded":"2024-04-13T03:19:17.395683+0000","last_fullsized":"2024-04-13T03:19:17.395683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:52:22.364206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894611+0000","last_change":"2024-04-13T03:19:15.060807+0000","last_active":"2024-04-13T03:19:16.894611+0000","last_peered":"2024-04-13T03:19:16.894611+0000","last_clean":"2024-04-13T03:19:16.894611+0000","last_became_active":"2024-04-13T03:19:15.060376+0000","last_became_peered":"2024-04-13T03:19:15.060376+0000","last_unstale":"2024-04-13T03:19:16.894611+0000","last_undegraded":"2024-04-13T03:19:16.894611+0000","last_fullsized":"2024-04-13T03:19:16.894611+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:19.158915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894324+0000","last_change":"2024-04-13T03:19:14.304406+0000","last_active":"2024-04-13T03:19:16.894324+0000","last_peered":"2024-04-13T03:19:16.894324+0000","last_clean":"2024-04-13T03:19:16.894324+0000","last_became_active":"2024-04-13T03:19:14.304104+0000","last_became_peered":"2024-04-13T03:19:14.304104+0000","last_unstale":"2024-04-13T03:19:16.894324+0000","last_undegraded":"2024-04-13T03:19:16.894324+0000","last_fullsized":"2024-04-13T03:19:16.894324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:00:56.212145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045167+0000","last_change":"2024-04-13T03:19:15.047540+0000","last_active":"2024-04-13T03:19:16.045167+0000","last_peered":"2024-04-13T03:19:16.045167+0000","last_clean":"2024-04-13T03:19:16.045167+0000","last_became_active":"2024-04-13T03:19:15.047267+0000","last_became_peered":"2024-04-13T03:19:15.047267+0000","last_unstale":"2024-04-13T03:19:16.045167+0000","last_undegraded":"2024-04-13T03:19:16.045167+0000","last_fullsized":"2024-04-13T03:19:16.045167+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:06:42.202992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395265+0000","last_change":"2024-04-13T03:19:14.304736+0000","last_active":"2024-04-13T03:19:17.395265+0000","last_peered":"2024-04-13T03:19:17.395265+0000","last_clean":"2024-04-13T03:19:17.395265+0000","last_became_active":"2024-04-13T03:19:14.304454+0000","last_became_peered":"2024-04-13T03:19:14.304454+0000","last_unstale":"2024-04-13T03:19:17.395265+0000","last_undegraded":"2024-04-13T03:19:17.395265+0000","last_fullsized":"2024-04-13T03:19:17.395265+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:46:11.174947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046319+0000","last_change":"2024-04-13T03:19:15.055324+0000","last_active":"2024-04-13T03:19:16.046319+0000","last_peered":"2024-04-13T03:19:16.046319+0000","last_clean":"2024-04-13T03:19:16.046319+0000","last_became_active":"2024-04-13T03:19:15.055072+0000","last_became_peered":"2024-04-13T03:19:15.055072+0000","last_unstale":"2024-04-13T03:19:16.046319+0000","last_undegraded":"2024-04-13T03:19:16.046319+0000","last_fullsized":"2024-04-13T03:19:16.046319+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:22:37.597840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046104+0000","last_change":"2024-04-13T03:19:14.298460+0000","last_active":"2024-04-13T03:19:16.046104+0000","last_peered":"2024-04-13T03:19:16.046104+0000","last_clean":"2024-04-13T03:19:16.046104+0000","last_became_active":"2024-04-13T03:19:14.297883+0000","last_became_peered":"2024-04-13T03:19:14.297883+0000","last_unstale":"2024-04-13T03:19:16.046104+0000","last_undegraded":"2024-04-13T03:19:16.046104+0000","last_fullsized":"2024-04-13T03:19:16.046104+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:29:29.851693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394711+0000","last_change":"2024-04-13T03:19:15.058312+0000","last_active":"2024-04-13T03:19:17.394711+0000","last_peered":"2024-04-13T03:19:17.394711+0000","last_clean":"2024-04-13T03:19:17.394711+0000","last_became_active":"2024-04-13T03:19:15.058067+0000","last_became_peered":"2024-04-13T03:19:15.058067+0000","last_unstale":"2024-04-13T03:19:17.394711+0000","last_undegraded":"2024-04-13T03:19:17.394711+0000","last_fullsized":"2024-04-13T03:19:17.394711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:46:12.179196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.315528+0000","last_change":"2024-04-13T03:19:14.303560+0000","last_active":"2024-04-13T03:19:17.315528+0000","last_peered":"2024-04-13T03:19:17.315528+0000","last_clean":"2024-04-13T03:19:17.315528+0000","last_became_active":"2024-04-13T03:19:14.303258+0000","last_became_peered":"2024-04-13T03:19:14.303258+0000","last_unstale":"2024-04-13T03:19:17.315528+0000","last_undegraded":"2024-04-13T03:19:17.315528+0000","last_fullsized":"2024-04-13T03:19:17.315528+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:27:43.310884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.396149+0000","last_change":"2024-04-13T03:19:15.057783+0000","last_active":"2024-04-13T03:19:17.396149+0000","last_peered":"2024-04-13T03:19:17.396149+0000","last_clean":"2024-04-13T03:19:17.396149+0000","last_became_active":"2024-04-13T03:19:15.057495+0000","last_became_peered":"2024-04-13T03:19:15.057495+0000","last_unstale":"2024-04-13T03:19:17.396149+0000","last_undegraded":"2024-04-13T03:19:17.396149+0000","last_fullsized":"2024-04-13T03:19:17.396149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:11:33.091602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.314172+0000","last_change":"2024-04-13T03:19:14.303562+0000","last_active":"2024-04-13T03:19:17.314172+0000","last_peered":"2024-04-13T03:19:17.314172+0000","last_clean":"2024-04-13T03:19:17.314172+0000","last_became_active":"2024-04-13T03:19:14.303253+0000","last_became_peered":"2024-04-13T03:19:14.303253+0000","last_unstale":"2024-04-13T03:19:17.314172+0000","last_undegraded":"2024-04-13T03:19:17.314172+0000","last_fullsized":"2024-04-13T03:19:17.314172+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:06:32.172544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.044921+0000","last_change":"2024-04-13T03:19:15.051709+0000","last_active":"2024-04-13T03:19:16.044921+0000","last_peered":"2024-04-13T03:19:16.044921+0000","last_clean":"2024-04-13T03:19:16.044921+0000","last_became_active":"2024-04-13T03:19:15.051464+0000","last_became_peered":"2024-04-13T03:19:15.051464+0000","last_unstale":"2024-04-13T03:19:16.044921+0000","last_undegraded":"2024-04-13T03:19:16.044921+0000","last_fullsized":"2024-04-13T03:19:16.044921+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:41:30.001353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.046291+0000","last_change":"2024-04-13T03:19:14.302888+0000","last_active":"2024-04-13T03:19:16.046291+0000","last_peered":"2024-04-13T03:19:16.046291+0000","last_clean":"2024-04-13T03:19:16.046291+0000","last_became_active":"2024-04-13T03:19:14.302595+0000","last_became_peered":"2024-04-13T03:19:14.302595+0000","last_unstale":"2024-04-13T03:19:16.046291+0000","last_undegraded":"2024-04-13T03:19:16.046291+0000","last_fullsized":"2024-04-13T03:19:16.046291+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:01:12.974984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045833+0000","last_change":"2024-04-13T03:19:15.049409+0000","last_active":"2024-04-13T03:19:16.045833+0000","last_peered":"2024-04-13T03:19:16.045833+0000","last_clean":"2024-04-13T03:19:16.045833+0000","last_became_active":"2024-04-13T03:19:15.049162+0000","last_became_peered":"2024-04-13T03:19:15.049162+0000","last_unstale":"2024-04-13T03:19:16.045833+0000","last_undegraded":"2024-04-13T03:19:16.045833+0000","last_fullsized":"2024-04-13T03:19:16.045833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:42:54.570706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395511+0000","last_change":"2024-04-13T03:19:14.292947+0000","last_active":"2024-04-13T03:19:17.395511+0000","last_peered":"2024-04-13T03:19:17.395511+0000","last_clean":"2024-04-13T03:19:17.395511+0000","last_became_active":"2024-04-13T03:19:14.292589+0000","last_became_peered":"2024-04-13T03:19:14.292589+0000","last_unstale":"2024-04-13T03:19:17.395511+0000","last_undegraded":"2024-04-13T03:19:17.395511+0000","last_fullsized":"2024-04-13T03:19:17.395511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:32.070755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894207+0000","last_change":"2024-04-13T03:19:15.058565+0000","last_active":"2024-04-13T03:19:16.894207+0000","last_peered":"2024-04-13T03:19:16.894207+0000","last_clean":"2024-04-13T03:19:16.894207+0000","last_became_active":"2024-04-13T03:19:15.057952+0000","last_became_peered":"2024-04-13T03:19:15.057952+0000","last_unstale":"2024-04-13T03:19:16.894207+0000","last_undegraded":"2024-04-13T03:19:16.894207+0000","last_fullsized":"2024-04-13T03:19:16.894207+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:56:39.039010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395484+0000","last_change":"2024-04-13T03:19:14.296447+0000","last_active":"2024-04-13T03:19:17.395484+0000","last_peered":"2024-04-13T03:19:17.395484+0000","last_clean":"2024-04-13T03:19:17.395484+0000","last_became_active":"2024-04-13T03:19:14.296166+0000","last_became_peered":"2024-04-13T03:19:14.296166+0000","last_unstale":"2024-04-13T03:19:17.395484+0000","last_undegraded":"2024-04-13T03:19:17.395484+0000","last_fullsized":"2024-04-13T03:19:17.395484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:46:11.174947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395755+0000","last_change":"2024-04-13T03:19:15.058416+0000","last_active":"2024-04-13T03:19:17.395755+0000","last_peered":"2024-04-13T03:19:17.395755+0000","last_clean":"2024-04-13T03:19:17.395755+0000","last_became_active":"2024-04-13T03:19:15.058150+0000","last_became_peered":"2024-04-13T03:19:15.058150+0000","last_unstale":"2024-04-13T03:19:17.395755+0000","last_undegraded":"2024-04-13T03:19:17.395755+0000","last_fullsized":"2024-04-13T03:19:17.395755+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:19:02.736825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.045897+0000","last_change":"2024-04-13T03:19:14.303281+0000","last_active":"2024-04-13T03:19:16.045897+0000","last_peered":"2024-04-13T03:19:16.045897+0000","last_clean":"2024-04-13T03:19:16.045897+0000","last_became_active":"2024-04-13T03:19:14.302997+0000","last_became_peered":"2024-04-13T03:19:14.302997+0000","last_unstale":"2024-04-13T03:19:16.045897+0000","last_undegraded":"2024-04-13T03:19:16.045897+0000","last_fullsized":"2024-04-13T03:19:16.045897+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:01:45.221262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.394890+0000","last_change":"2024-04-13T03:19:15.057338+0000","last_active":"2024-04-13T03:19:17.394890+0000","last_peered":"2024-04-13T03:19:17.394890+0000","last_clean":"2024-04-13T03:19:17.394890+0000","last_became_active":"2024-04-13T03:19:15.057066+0000","last_became_peered":"2024-04-13T03:19:15.057066+0000","last_unstale":"2024-04-13T03:19:17.394890+0000","last_undegraded":"2024-04-13T03:19:17.394890+0000","last_fullsized":"2024-04-13T03:19:17.394890+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:27:25.712369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395407+0000","last_change":"2024-04-13T03:19:14.305719+0000","last_active":"2024-04-13T03:19:17.395407+0000","last_peered":"2024-04-13T03:19:17.395407+0000","last_clean":"2024-04-13T03:19:17.395407+0000","last_became_active":"2024-04-13T03:19:14.305279+0000","last_became_peered":"2024-04-13T03:19:14.305279+0000","last_unstale":"2024-04-13T03:19:17.395407+0000","last_undegraded":"2024-04-13T03:19:17.395407+0000","last_fullsized":"2024-04-13T03:19:17.395407+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:47:23.660095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395710+0000","last_change":"2024-04-13T03:19:15.062407+0000","last_active":"2024-04-13T03:19:17.395710+0000","last_peered":"2024-04-13T03:19:17.395710+0000","last_clean":"2024-04-13T03:19:17.395710+0000","last_became_active":"2024-04-13T03:19:15.062158+0000","last_became_peered":"2024-04-13T03:19:15.062158+0000","last_unstale":"2024-04-13T03:19:17.395710+0000","last_undegraded":"2024-04-13T03:19:17.395710+0000","last_fullsized":"2024-04-13T03:19:17.395710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:14.280820+0000","last_clean_scrub_stamp":"2024-04-13T03:19:14.280820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:13:26.030621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:16.894101+0000","last_change":"2024-04-13T03:19:14.308519+0000","last_active":"2024-04-13T03:19:16.894101+0000","last_peered":"2024-04-13T03:19:16.894101+0000","last_clean":"2024-04-13T03:19:16.894101+0000","last_became_active":"2024-04-13T03:19:14.308263+0000","last_became_peered":"2024-04-13T03:19:14.308263+0000","last_unstale":"2024-04-13T03:19:16.894101+0000","last_undegraded":"2024-04-13T03:19:16.894101+0000","last_fullsized":"2024-04-13T03:19:16.894101+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:19:13.276570+0000","last_clean_scrub_stamp":"2024-04-13T03:19:13.276570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:27:45.508456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":358,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-13T03:19:17.395654+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:17.395654+0000","last_peered":"2024-04-13T03:19:17.395654+0000","last_clean":"2024-04-13T03:19:17.395654+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:17.395654+0000","last_undegraded":"2024-04-13T03:19:17.395654+0000","last_fullsized":"2024-04-13T03:19:17.395654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2105344,"data_stored":1289836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":699274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":705544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1507328,"data_stored":700304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1523712,"data_stored":707030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1507328,"data_stored":705384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1499136,"data_stored":698714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574973,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2105344,"data_stored":1295774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:19:18.647 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T03:19:18.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 Processor -- start 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 -- start start 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe61415d400 0x7fe61413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe6140cf8f0 0x7fe614141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6141429b0 con 0x7fe61415d7e0 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe61406c0d0 con 0x7fe61415d400 2024-04-13T03:19:18.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.791+0000 7fe61c38c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe61406ba80 con 0x7fe6140cf8f0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a128640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe61415d400 0x7fe61413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe619927640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe61415d7e0 0x7fe6140cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54878/0 (socket says 172.21.15.106:54878) 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe619927640 1 -- 172.21.15.106:0/1060676409 learned_addr learned my addr 172.21.15.106:0/1060676409 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a929640 1 --2- 172.21.15.106:0/1060676409 >> v2:172.21.15.114:3300/0 conn(0x7fe6140cf8f0 0x7fe614141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe619126640 1 -- 172.21.15.106:0/1060676409 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608487919 0 0) 0x7fe6141429b0 con 0x7fe61415d7e0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe619126640 1 -- 172.21.15.106:0/1060676409 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe600003520 con 0x7fe61415d7e0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a128640 1 -- 172.21.15.106:0/1060676409 >> v2:172.21.15.114:3300/0 conn(0x7fe6140cf8f0 msgr2=0x7fe614141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a128640 1 --2- 172.21.15.106:0/1060676409 >> v2:172.21.15.114:3300/0 conn(0x7fe6140cf8f0 0x7fe614141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a128640 1 -- 172.21.15.106:0/1060676409 >> v1:172.21.15.106:6789/0 conn(0x7fe61415d7e0 legacy=0x7fe6140cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.792+0000 7fe61a128640 1 -- 172.21.15.106:0/1060676409 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61406bd60 con 0x7fe61415d400 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe61a128640 1 --2- 172.21.15.106:0/1060676409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe61415d400 0x7fe61413fc60 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe6140701c0 tx=0x7fe61002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a46b912c93ff85 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe619126640 1 -- 172.21.15.106:0/1060676409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100315a0 con 0x7fe61415d400 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe619126640 1 -- 172.21.15.106:0/1060676409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610037070 con 0x7fe61415d400 2024-04-13T03:19:18.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe619126640 1 -- 172.21.15.106:0/1060676409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610031940 con 0x7fe61415d400 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe61c38c640 1 -- 172.21.15.106:0/1060676409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe61415d400 msgr2=0x7fe61413fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.793+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1060676409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe61415d400 0x7fe61413fc60 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe6140701c0 tx=0x7fe61002b840 comp rx=0 tx=0).stop 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 -- 172.21.15.106:0/1060676409 shutdown_connections 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1060676409 >> v2:172.21.15.114:3300/0 conn(0x7fe6140cf8f0 0x7fe614141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1060676409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe61415d400 0x7fe61413fc60 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 -- 172.21.15.106:0/1060676409 >> 172.21.15.106:0/1060676409 conn(0x7fe6140b2600 msgr2=0x7fe6140d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 -- 172.21.15.106:0/1060676409 shutdown_connections 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.794+0000 7fe61c38c640 1 -- 172.21.15.106:0/1060676409 wait complete. 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 Processor -- start 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 -- start start 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe61415d400 0x7fe614157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 0x7fe6141563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe614142590 con 0x7fe6140cf8f0 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe614142410 con 0x7fe61415d7e0 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61c38c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe614142710 con 0x7fe61415d400 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.795+0000 7fe61a128640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40126/0 (socket says 172.21.15.106:40126) 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 -- 172.21.15.106:0/1034699381 learned_addr learned my addr 172.21.15.106:0/1034699381 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a929640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 0x7fe6141563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe619927640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe61415d400 0x7fe614157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 -- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe61415d400 msgr2=0x7fe614157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe61415d400 0x7fe614157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 -- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 msgr2=0x7fe6141563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 0x7fe6141563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:18.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 -- 172.21.15.106:0/1034699381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe614157b70 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a929640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 0x7fe6141563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61a128640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe61002b880 tx=0x7fe6100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610032630 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100327d0 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6141e84d0 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610032ba0 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.796+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6141e8a90 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.797+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe614068a10 con 0x7fe6140cf8f0 2024-04-13T03:19:18.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.797+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe61004e430 con 0x7fe6140cf8f0 2024-04-13T03:19:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.798+0000 7fe60affd640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 0x7fe5f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.798+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fe61002f030 con 0x7fe6140cf8f0 2024-04-13T03:19:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.798+0000 7fe619927640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 0x7fe5f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.798+0000 7fe619927640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 0x7fe5f806edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe61413e460 tx=0x7fe604009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:18.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:18.802+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe610040070 con 0x7fe6140cf8f0 2024-04-13T03:19:19.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.047+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe6140cdd80 con 0x7fe6140cf8f0 2024-04-13T03:19:19.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.050+0000 7fe60affd640 1 -- 172.21.15.106:0/1034699381 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fe61007be80 con 0x7fe6140cf8f0 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAq9xlmRiK2MhAA6axPdr/qQQt9DKD9l99bBw== 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-13T03:19:19.051 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 msgr2=0x7fe5f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 0x7fe5f806edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe61413e460 tx=0x7fe604009040 comp rx=0 tx=0).stop 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 msgr2=0x7fe6141e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe61002b880 tx=0x7fe6100046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 shutdown_connections 2024-04-13T03:19:19.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5f806c8f0 0x7fe5f806edb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe61415d400 0x7fe614157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe61415d7e0 0x7fe6141563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 --2- 172.21.15.106:0/1034699381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6140cf8f0 0x7fe6141e7dd0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 >> 172.21.15.106:0/1034699381 conn(0x7fe6140b2600 msgr2=0x7fe61415f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 shutdown_connections 2024-04-13T03:19:19.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.052+0000 7fe61c38c640 1 -- 172.21.15.106:0/1034699381 wait complete. 2024-04-13T03:19:19.064 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:19.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.163+0000 7f0254e08640 1 Processor -- start 2024-04-13T03:19:19.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.163+0000 7f0254e08640 1 -- start start 2024-04-13T03:19:19.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f0254e08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0250160c10 0x7f025006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f0254e08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 0x7f025006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f0254e08640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025015d690 con 0x7f0250156b50 2024-04-13T03:19:19.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f0254e08640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0250161080 con 0x7f0250160c10 2024-04-13T03:19:19.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f0254e08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f025015d970 con 0x7f025006e2b0 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f024dd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0250160c10 0x7f025006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f024e59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0250156b50 0x7f025015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54886/0 (socket says 172.21.15.106:54886) 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f024e59c640 1 -- 172.21.15.106:0/2109637428 learned_addr learned my addr 172.21.15.106:0/2109637428 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.164+0000 7f024ed9d640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 0x7f025006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024dd9b640 1 -- 172.21.15.106:0/2109637428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 msgr2=0x7f025006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024dd9b640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 0x7f025006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024dd9b640 1 -- 172.21.15.106:0/2109637428 >> v1:172.21.15.106:6789/0 conn(0x7f0250156b50 legacy=0x7f025015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024dd9b640 1 -- 172.21.15.106:0/2109637428 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0250160410 con 0x7f0250160c10 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024d59a640 1 -- 172.21.15.106:0/2109637428 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201602155 0 0) 0x7f025015d690 con 0x7f0250156b50 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024ed9d640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 0x7f025006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024dd9b640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006dd70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f02440097a0 tx=0x7f024402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237ec265201e87e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.165+0000 7f024d59a640 1 -- 172.21.15.106:0/2109637428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244035020 con 0x7f0250160c10 2024-04-13T03:19:19.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f024d59a640 1 -- 172.21.15.106:0/2109637428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0244031810 con 0x7f0250160c10 2024-04-13T03:19:19.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f024d59a640 1 -- 172.21.15.106:0/2109637428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244031be0 con 0x7f0250160c10 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 -- 172.21.15.106:0/2109637428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 msgr2=0x7f025006dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006dd70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f02440097a0 tx=0x7f024402b8b0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 -- 172.21.15.106:0/2109637428 shutdown_connections 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f025006e2b0 0x7f025006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 --2- 172.21.15.106:0/2109637428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006dd70 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 -- 172.21.15.106:0/2109637428 >> 172.21.15.106:0/2109637428 conn(0x7f02500b1f10 msgr2=0x7f02500b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 -- 172.21.15.106:0/2109637428 shutdown_connections 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.166+0000 7f0254e08640 1 -- 172.21.15.106:0/2109637428 wait complete. 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.167+0000 7f0254e08640 1 Processor -- start 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.167+0000 7f0254e08640 1 -- start start 2024-04-13T03:19:19.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 0x7f02501552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 0x7f025006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0250156f30 con 0x7f0250160c10 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f025015d320 con 0x7f025006e2b0 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f0254e08640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f025015eec0 con 0x7f0250156b50 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40152/0 (socket says 172.21.15.106:40152) 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024ed9d640 1 -- 172.21.15.106:0/2424942136 learned_addr learned my addr 172.21.15.106:0/2424942136 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024e59c640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 0x7f02501552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024dd9b640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 0x7f025006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.168+0000 7f024ed9d640 1 -- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 msgr2=0x7f025006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024ed9d640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 0x7f025006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024ed9d640 1 -- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 msgr2=0x7f02501552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024ed9d640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 0x7f02501552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024ed9d640 1 -- 172.21.15.106:0/2424942136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02500ca3c0 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024e59c640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 0x7f02501552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024dd9b640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 0x7f025006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f024ed9d640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f024800d800 tx=0x7f024800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248004030 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480041d0 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025015e590 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480096a0 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.169+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025015eb50 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.170+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0250068a10 con 0x7f0250160c10 2024-04-13T03:19:19.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.170+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0248009840 con 0x7f0250160c10 2024-04-13T03:19:19.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.171+0000 7f023effd640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 0x7f023006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.171+0000 7f024e59c640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 0x7f023006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.171+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f024808bf90 con 0x7f0250160c10 2024-04-13T03:19:19.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.171+0000 7f024e59c640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 0x7f023006edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f02400012e0 tx=0x7f0240007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.175+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02500ca3c0 con 0x7f0250160c10 2024-04-13T03:19:19.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.390+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f025006ba90 con 0x7f0250160c10 2024-04-13T03:19:19.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.391+0000 7f023effd640 1 -- 172.21.15.106:0/2424942136 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5195 (secure 0 0 0) 0x7f024805a2c0 con 0x7f0250160c10 2024-04-13T03:19:19.392 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:19.392 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:16.657710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9084},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9084":{"gid":9084,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1427655446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1427655446},{"type":"v1","addr":"172.21.15.106:6837","nonce":1427655446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9084,"qdb_cluster":[9084]},"id":8}]} 2024-04-13T03:19:19.392 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 86 2024-04-13T03:19:19.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 msgr2=0x7f023006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 0x7f023006edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f02400012e0 tx=0x7f0240007f90 comp rx=0 tx=0).stop 2024-04-13T03:19:19.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 msgr2=0x7f025006c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f024800d800 tx=0x7f024800dcc0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 shutdown_connections 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f023006c8f0 0x7f023006edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0250156b50 0x7f025006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0250160c10 0x7f025006c1f0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 --2- 172.21.15.106:0/2424942136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f025006e2b0 0x7f02501552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 >> 172.21.15.106:0/2424942136 conn(0x7f02500b1f10 msgr2=0x7f02500b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 shutdown_connections 2024-04-13T03:19:19.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.393+0000 7f0254e08640 1 -- 172.21.15.106:0/2424942136 wait complete. 2024-04-13T03:19:19.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:16.657710+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9084}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9084': {'gid': 9084, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/1427655446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 1427655446}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1427655446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9084, 'qdb_cluster': [9084]} 2024-04-13T03:19:19.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:19:19.809 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-13T03:19:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.911+0000 7fa69a51e640 1 Processor -- start 2024-04-13T03:19:19.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.911+0000 7fa69a51e640 1 -- start start 2024-04-13T03:19:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa69a51e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa69406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa69a51e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6940d07c0 0x7fa69406be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa69a51e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6940cef80 con 0x7fa69415ef10 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa69a51e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa6940cf260 con 0x7fa6940d07c0 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa69a51e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa6940cf540 con 0x7fa69415fb50 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa698abb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa69415ef10 0x7fa694162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54898/0 (socket says 172.21.15.106:54898) 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa698abb640 1 -- 172.21.15.106:0/4287339463 learned_addr learned my addr 172.21.15.106:0/4287339463 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 382110972 0 0) 0x7fa6940cef80 con 0x7fa69415ef10 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa6937fe640 1 --2- 172.21.15.106:0/4287339463 >> v2:172.21.15.114:3300/0 conn(0x7fa6940d07c0 0x7fa69406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa693fff640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa69406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa67c003520 con 0x7fa69415ef10 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713088529 0 0) 0x7fa67c003520 con 0x7fa69415ef10 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 msgr2=0x7fa69406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa69406b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 >> v2:172.21.15.114:3300/0 conn(0x7fa6940d07c0 msgr2=0x7fa69406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 --2- 172.21.15.106:0/4287339463 >> v2:172.21.15.114:3300/0 conn(0x7fa6940d07c0 0x7fa69406be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6940cf820 con 0x7fa69415ef10 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.912+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fa68c0019f0 con 0x7fa69415ef10 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 >> v1:172.21.15.106:6789/0 conn(0x7fa69415ef10 legacy=0x7fa694162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c0039b0 0x7fa67c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa67c004800 0x7fa67c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa67c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940cf260 con 0x7fa67c004800 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940cf540 con 0x7fa67c0039b0 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa67c00b9c0 con 0x7fa69415fb50 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa67c004800 0x7fa67c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa693fff640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c0039b0 0x7fa67c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa698abb640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa67c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 -- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 msgr2=0x7fa67c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69415fb50 0x7fa67c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 -- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c0039b0 msgr2=0x7fa67c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c0039b0 0x7fa67c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.913+0000 7fa6937fe640 1 -- 172.21.15.106:0/4287339463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c003520 con 0x7fa67c004800 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa6937fe640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa67c004800 0x7fa67c004c80 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa688009ac0 tx=0x7fa67c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d5432f5d07103a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688039070 con 0x7fa67c004800 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68802c6b0 con 0x7fa67c004800 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa692ffd640 1 -- 172.21.15.106:0/4287339463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688009690 con 0x7fa67c004800 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 -- 172.21.15.106:0/4287339463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa67c004800 msgr2=0x7fa67c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 --2- 172.21.15.106:0/4287339463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa67c004800 0x7fa67c004c80 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa688009ac0 tx=0x7fa67c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa698abb640 1 -- 172.21.15.106:0/4287339463 reap_dead start 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 -- 172.21.15.106:0/4287339463 shutdown_connections 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 -- 172.21.15.106:0/4287339463 >> 172.21.15.106:0/4287339463 conn(0x7fa6940cd6a0 msgr2=0x7fa6940cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 -- 172.21.15.106:0/4287339463 shutdown_connections 2024-04-13T03:19:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.914+0000 7fa69a51e640 1 -- 172.21.15.106:0/4287339463 wait complete. 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.915+0000 7fa69a51e640 1 Processor -- start 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 -- start start 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 0x7fa69415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 0x7fa69414e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940ced20 con 0x7fa6940b4bf0 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940ceba0 con 0x7fa67c004800 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa69a51e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa69406f6c0 con 0x7fa69414e370 2024-04-13T03:19:19.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40174/0 (socket says 172.21.15.106:40174) 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.916+0000 7fa698abb640 1 -- 172.21.15.106:0/547078248 learned_addr learned my addr 172.21.15.106:0/547078248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa693fff640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 0x7fa69415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa6937fe640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 0x7fa69414e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 -- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 msgr2=0x7fa69414e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 0x7fa69414e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 -- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 msgr2=0x7fa69415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 0x7fa69415cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 -- 172.21.15.106:0/547078248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6940b5900 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa6937fe640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 0x7fa69414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa693fff640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 0x7fa69415cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa698abb640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ce410 tx=0x7fa68c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa690ff9640 1 -- 172.21.15.106:0/547078248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00edc0 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa690ff9640 1 -- 172.21.15.106:0/547078248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c014430 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6940b5be0 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa690ff9640 1 -- 172.21.15.106:0/547078248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c014820 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.917+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6940b61a0 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.918+0000 7fa690ff9640 1 -- 172.21.15.106:0/547078248 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa68c0149c0 con 0x7fa6940b4bf0 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.919+0000 7fa690ff9640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 0x7fa68006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.919+0000 7fa690ff9640 1 -- 172.21.15.106:0/547078248 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fa68c08bb50 con 0x7fa6940b4bf0 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.919+0000 7fa693fff640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 0x7fa68006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.919+0000 7fa693fff640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 0x7fa68006ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa6840099e0 tx=0x7fa684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 Processor -- start 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 -- start start 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d716d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d746b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55628d681990 con 0x55628d7172c0 2024-04-13T03:19:19.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55628d628620 con 0x55628d743de0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.920+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55628d67d950 con 0x55628d7441c0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40182/0 (socket says 172.21.15.106:40182) 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 -- 172.21.15.106:0/1987163704 learned_addr learned my addr 172.21.15.106:0/1987163704 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa6937fe640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d746b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 -- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 msgr2=0x55628d746b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa693fff640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d716d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d746b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 -- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 msgr2=0x55628d716d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d716d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 -- 172.21.15.106:0/1987163704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628d3942d0 con 0x55628d7172c0 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa693fff640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d716d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:19.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa698abb640 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa68c089e90 tx=0x7fa68c059050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41faded672be72b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa679ffb640 1 -- 172.21.15.106:0/1987163704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0596f0 con 0x55628d7172c0 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa679ffb640 1 -- 172.21.15.106:0/1987163704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c05daa0 con 0x55628d7172c0 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.921+0000 7fa679ffb640 1 -- 172.21.15.106:0/1987163704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0622d0 con 0x55628d7172c0 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1987163704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 msgr2=0x55628d7470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa68c089e90 tx=0x7fa68c059050 comp rx=0 tx=0).stop 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1987163704 shutdown_connections 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d746b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7172c0 0x55628d7470b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1987163704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d716d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1987163704 >> 172.21.15.106:0/1987163704 conn(0x55628d718ff0 msgr2=0x55628d761b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1987163704 shutdown_connections 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1987163704 wait complete. 2024-04-13T03:19:19.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.922+0000 7fa69d3d7740 1 -- start start 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 Processor -- start 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d743de0 0x55628d73bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d73ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55628d71b590 con 0x55628d7441c0 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55628d63b9e0 con 0x55628d73c830 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55628d71a980 con 0x55628d743de0 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa69d3d7740 10 client.0 _ll_register_callbacks cb 0x55628d677220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40196/0 (socket says 172.21.15.106:40196) 2024-04-13T03:19:19.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa6937fe640 1 -- 172.21.15.106:0/2752137109 learned_addr learned my addr 172.21.15.106:0/2752137109 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.923+0000 7fa698abb640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d73ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa693fff640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d743de0 0x55628d73bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 -- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d743de0 msgr2=0x55628d73bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d743de0 0x55628d73bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 -- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 msgr2=0x55628d73ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d73ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 -- 172.21.15.106:0/2752137109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55628d45e6b0 con 0x55628d7441c0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa6937fe640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa688009be0 tx=0x7fa6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b1410aafc07e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688039030 con 0x55628d7441c0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fa68802cac0 con 0x55628d7441c0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa68c05daa0 con 0x55628d7441c0 2024-04-13T03:19:19.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa68804c050 con 0x55628d7441c0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 10 client.9240 fetch_fsmap learned FSMap version 86 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55628d73ef30 con 0x55628d7441c0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c05daa0 con 0x55628d7441c0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 86) ==== 3494+0+0 (secure 0 0 0) 0x7fa68804f020 con 0x55628d7441c0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 10 client.9240 fetch_fsmap finished waiting for FSMap version 86 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 10 client.9240 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 20 client.9240 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d775be0 0x55628d780940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 4 client.9240 mds_command: new command op to 9084 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.924+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 --> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55628d68f3b0 con 0x55628d775be0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa693fff640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d775be0 0x55628d780940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa672ffd640 10 client.9240 ms_handle_connect on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa693fff640 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d775be0 0x55628d780940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c848225ed09392 server_cookie=1336f296ac05785d in_seq=0 out_seq=0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa672ffd640 1 -- 172.21.15.106:0/2752137109 <== mds.0 v2:172.21.15.106:6835/1427655446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55628d68f3b0 con 0x55628d775be0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa672ffd640 10 client.9240 handle_command_reply: tid=0 2024-04-13T03:19:19.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.925+0000 7fa69d3d7740 1 client.9240 shutdown 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 msgr2=0x55628d7728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa688009be0 tx=0x7fa6880046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 shutdown_connections 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d775be0 0x55628d780940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d73ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d7728e0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa672ffd640 0 client.9240 ms_handle_reset on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/2752137109 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d743de0 0x55628d73bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 >> 172.21.15.106:0/2752137109 conn(0x55628d7172c0 msgr2=0x55628d742200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 shutdown_connections 2024-04-13T03:19:19.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 1 -- 172.21.15.106:0/2752137109 wait complete. 2024-04-13T03:19:19.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.926+0000 7fa69d3d7740 20 client.9240 trim_cache size 0 max 16384 2024-04-13T03:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.944+0000 7fa69d3d7740 1 Processor -- start 2024-04-13T03:19:19.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.944+0000 7fa69d3d7740 1 -- start start 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.944+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d772ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d73ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa694201b40 con 0x55628d743de0 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa694200b60 con 0x55628d73c830 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa69415b540 con 0x55628d7441c0 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40208/0 (socket says 172.21.15.106:40208) 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 -- 172.21.15.106:0/890774197 learned_addr learned my addr 172.21.15.106:0/890774197 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 -- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 msgr2=0x55628d73ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa693fff640 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d772ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d73ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 -- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 msgr2=0x55628d772ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d772ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 -- 172.21.15.106:0/890774197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628d718750 con 0x55628d743de0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6937fe640 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa68803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39865f97e3c3283e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6797fa640 1 -- 172.21.15.106:0/890774197 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688039070 con 0x55628d743de0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6797fa640 1 -- 172.21.15.106:0/890774197 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68803a040 con 0x55628d743de0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.945+0000 7fa6797fa640 1 -- 172.21.15.106:0/890774197 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688003610 con 0x55628d743de0 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- 172.21.15.106:0/890774197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 msgr2=0x55628d745a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa68803e000 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- 172.21.15.106:0/890774197 shutdown_connections 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d7441c0 0x55628d73ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d743de0 0x55628d745a70 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/890774197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d73c830 0x55628d772ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- 172.21.15.106:0/890774197 >> 172.21.15.106:0/890774197 conn(0x55628d7172c0 msgr2=0x55628d73bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- 172.21.15.106:0/890774197 shutdown_connections 2024-04-13T03:19:19.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- 172.21.15.106:0/890774197 wait complete. 2024-04-13T03:19:19.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.946+0000 7fa69d3d7740 1 -- start start 2024-04-13T03:19:19.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 Processor -- start 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d73c830 0x55628d7dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d785fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55628d73c4b0 con 0x55628d7441c0 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa694200510 con 0x55628d743de0 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55628d73f4f0 con 0x55628d73c830 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa69d3d7740 10 client.0 _ll_register_callbacks cb 0x7fa688037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:19.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa698abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40214/0 (socket says 172.21.15.106:40214) 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa698abb640 1 -- 172.21.15.106:0/1063931441 learned_addr learned my addr 172.21.15.106:0/1063931441 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa6937fe640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d785fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.947+0000 7fa693fff640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d73c830 0x55628d7dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 -- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d73c830 msgr2=0x55628d7dc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d73c830 0x55628d7dc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 -- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 msgr2=0x55628d785fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d785fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 -- 172.21.15.106:0/1063931441 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55628d73b340 con 0x55628d7441c0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa698abb640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa68c055b50 tx=0x7fa68c089dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219b782e4597646 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:19.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0622d0 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fa68c061e40 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa68803a040 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa68c0596f0 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 10 client.9246 fetch_fsmap learned FSMap version 86 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55628d742890 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68803a040 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 86) ==== 3494+0+0 (secure 0 0 0) 0x7fa68c058b40 con 0x55628d7441c0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 10 client.9246 fetch_fsmap finished waiting for FSMap version 86 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 10 client.9246 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 20 client.9246 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d71ae70 0x55628d760c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 4 client.9246 mds_command: new command op to 9084 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 --> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] -- command(tid 0: {"prefix": "status"}) -- 0x55628d73d910 con 0x55628d71ae70 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.948+0000 7fa693fff640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d71ae70 0x55628d760c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:19.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa672ffd640 10 client.9246 ms_handle_connect on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa693fff640 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d71ae70 0x55628d760c30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dc38671a87c667 server_cookie=d06de7147d06ead in_seq=0 out_seq=0 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa672ffd640 1 -- 172.21.15.106:0/1063931441 <== mds.0 v2:172.21.15.106:6835/1427655446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55628d73d910 con 0x55628d71ae70 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa672ffd640 10 client.9246 handle_command_reply: tid=0 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 client.9246 shutdown 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 msgr2=0x55628d71a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa68c055b50 tx=0x7fa68c089dc0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 shutdown_connections 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x55628d71ae70 0x55628d760c30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55628d7441c0 0x55628d71a930 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55628d743de0 0x55628d785fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa672ffd640 0 client.9246 ms_handle_reset on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 --2- 172.21.15.106:0/1063931441 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55628d73c830 0x55628d7dc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 >> 172.21.15.106:0/1063931441 conn(0x55628d7172c0 msgr2=0x55628d71dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 shutdown_connections 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 1 -- 172.21.15.106:0/1063931441 wait complete. 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.949+0000 7fa69d3d7740 20 client.9246 trim_cache size 0 max 16384 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 9084, 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.6407626469999999, 2024-04-13T03:19:19.957 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 86, 2024-04-13T03:19:19.958 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 121, 2024-04-13T03:19:19.958 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 121, 2024-04-13T03:19:19.958 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 9.6846206129999999 2024-04-13T03:19:19.958 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 msgr2=0x7fa68006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 0x7fa68006ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa6840099e0 tx=0x7fa684008040 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 msgr2=0x7fa6940b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ce410 tx=0x7fa68c00e530 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 shutdown_connections 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa68006c920 0x7fa68006ede0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa69414e370 0x7fa69414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6940b4bf0 0x7fa6940b5070 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 --2- 172.21.15.106:0/547078248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa67c004800 0x7fa69415cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 >> 172.21.15.106:0/547078248 conn(0x7fa6940cd6a0 msgr2=0x7fa69406c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 shutdown_connections 2024-04-13T03:19:19.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:19.950+0000 7fa69a51e640 1 -- 172.21.15.106:0/547078248 wait complete. 2024-04-13T03:19:19.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:19:20.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.061+0000 7f35deff0640 1 Processor -- start 2024-04-13T03:19:20.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.061+0000 7f35deff0640 1 -- start start 2024-04-13T03:19:20.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35deff0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35d815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35deff0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d8158c50 con 0x7f35d815f460 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35deff0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d80b3800 con 0x7f35d8158870 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35deff0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35d815ea90 con 0x7f35d80b32d0 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35dd58d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f35d815f460 0x7f35d815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54914/0 (socket says 172.21.15.106:54914) 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35dd58d640 1 -- 172.21.15.106:0/2994078128 learned_addr learned my addr 172.21.15.106:0/2994078128 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259999377 0 0) 0x7f35d8158c50 con 0x7f35d815f460 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35d7fff640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35d815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.062+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35c0003520 con 0x7f35d815f460 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3499526327 0 0) 0x7f35c0003520 con 0x7f35d815f460 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 msgr2=0x7f35d815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35d815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 >> v1:172.21.15.114:6789/0 conn(0x7f35d8158870 legacy=0x7f35d806faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80b3ae0 con 0x7f35d815f460 2024-04-13T03:19:20.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f35d00019f0 con 0x7f35d815f460 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 >> v1:172.21.15.106:6789/0 conn(0x7f35d815f460 legacy=0x7f35d815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35c00039e0 0x7f35c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815ea90 con 0x7f35c00039e0 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000cc30 con 0x7f35c0004800 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35c000ce30 con 0x7f35d80b32d0 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35c00039e0 0x7f35c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35d7fff640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dd58d640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 -- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 msgr2=0x7f35c0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35c0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 -- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35c0004800 msgr2=0x7f35c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:20.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dcd8c640 1 -- 172.21.15.106:0/2994078128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c0003520 con 0x7f35c00039e0 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.063+0000 7f35dd58d640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35d80b32d0 0x7f35c0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.064+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35c00039e0 0x7f35c0003e00 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f35c8001750 tx=0x7f35c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2626c7fee143f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.064+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8001370 con 0x7f35c00039e0 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.064+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c80052d0 con 0x7f35c00039e0 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.064+0000 7f35d77fe640 1 -- 172.21.15.106:0/2994078128 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8003830 con 0x7f35c00039e0 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 -- 172.21.15.106:0/2994078128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35c00039e0 msgr2=0x7f35c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 --2- 172.21.15.106:0/2994078128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35c00039e0 0x7f35c0003e00 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f35c8001750 tx=0x7f35c0005830 comp rx=0 tx=0).stop 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35dd58d640 1 -- 172.21.15.106:0/2994078128 reap_dead start 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 -- 172.21.15.106:0/2994078128 shutdown_connections 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 -- 172.21.15.106:0/2994078128 >> 172.21.15.106:0/2994078128 conn(0x7f35d80d0340 msgr2=0x7f35d80d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 -- 172.21.15.106:0/2994078128 shutdown_connections 2024-04-13T03:19:20.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.065+0000 7f35deff0640 1 -- 172.21.15.106:0/2994078128 wait complete. 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.066+0000 7f35deff0640 1 Processor -- start 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.066+0000 7f35deff0640 1 -- start start 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 0x7f35d814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 0x7f35d8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815bb00 con 0x7f35d80a9df0 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815b980 con 0x7f35d8157880 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f35d815ef20 con 0x7f35c00039e0 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40228/0 (socket says 172.21.15.106:40228) 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 -- 172.21.15.106:0/3306479199 learned_addr learned my addr 172.21.15.106:0/3306479199 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 0x7f35d814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35dd58d640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 0x7f35d8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 -- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 msgr2=0x7f35d814b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 0x7f35d814b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 -- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 msgr2=0x7f35d8157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 0x7f35d8157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 -- 172.21.15.106:0/3306479199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80aa280 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 0x7f35d814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35dd58d640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 0x7f35d8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d7fff640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0119d0 tx=0x7f35cc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00fb50 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35cc00fcf0 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d80aa560 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00fea0 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.067+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d80aab20 con 0x7f35d80a9df0 2024-04-13T03:19:20.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.068+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d814bd20 con 0x7f35d80a9df0 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.069+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35cc012040 con 0x7f35d80a9df0 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.069+0000 7f35d57fa640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 0x7f35c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.069+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f35cc08c2e0 con 0x7f35d80a9df0 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.069+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 0x7f35c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.070+0000 7f35dcd8c640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 0x7f35c406ede0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f35c8033000 tx=0x7f35c8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:20.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.074+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35cc019030 con 0x7f35d80a9df0 2024-04-13T03:19:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.287+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f35d8068a10 con 0x7f35d80a9df0 2024-04-13T03:19:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.288+0000 7f35d57fa640 1 -- 172.21.15.106:0/3306479199 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f35cc05a580 con 0x7f35d80a9df0 2024-04-13T03:19:20.289 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.289+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 msgr2=0x7f35c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.289+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 0x7f35c406ede0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f35c8033000 tx=0x7f35c8009030 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.289+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 msgr2=0x7f35d80abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.289+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0119d0 tx=0x7f35cc011e90 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 shutdown_connections 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f35c406c920 0x7f35c406ede0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d8157880 0x7f35d8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35d80a9df0 0x7f35d80abce0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 --2- 172.21.15.106:0/3306479199 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f35c00039e0 0x7f35d814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 >> 172.21.15.106:0/3306479199 conn(0x7f35d80d0340 msgr2=0x7f35d81590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 shutdown_connections 2024-04-13T03:19:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20.290+0000 7f35deff0640 1 -- 172.21.15.106:0/3306479199 wait complete. 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:19:20.301 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:20.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.302 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:19:20.317 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:20.318 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41989sec preferred_lft 41989sec 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:19:20.319 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:20.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.320 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:19:20.385 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:20.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.386 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:19:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:19:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:19:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:19:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:19:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:19:20.442 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:19:20.442 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:19:20.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:19:20.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:19:20.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:19:20.443 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:19:20.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:19:20.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.577 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:20.591 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:19:20.591 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:19:20.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:20.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.592 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:19:20.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:20.659 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:20.714 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:20.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:20.714 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:20.769 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:19:20.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.769 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:20.853 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:20.853 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:19:20.862 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:19:20.863 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:19:20.863 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:19:20.864 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:19:20.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:20.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:20.891 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:19:20.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:19:20.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:20.992 INFO:teuthology.orchestra.run.smithi106.stdout:client7768 2024-04-13T03:19:20.992 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:20.992 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> parent = os.path.join(mount_point, "parent") 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> os.mkdir(parent) 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> flushed = os.path.join(parent, "flushed") 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> os.mkdir(flushed) 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> for f in ["alpha", "bravo", "charlie"]: 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-13T03:19:21.008 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:21.131 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-13T03:19:21.192 INFO:teuthology.orchestra.run.smithi106.stdout:./ 2024-04-13T03:19:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:./parent 2024-04-13T03:19:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed 2024-04-13T03:19:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/charlie 2024-04-13T03:19:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/alpha 2024-04-13T03:19:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/bravo 2024-04-13T03:19:21.193 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.193 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.193 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.193 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.194 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-13T03:19:21.194 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.278 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:19:21.277+0000 7f2ca2ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:21.297 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:21.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.306 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.306 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.306 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.307 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.307 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-13T03:19:21.307 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.411 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:19:21.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.420 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.420 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.420 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.420 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.420 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-13T03:19:21.421 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.524 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:19:21.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.533 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.533 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.534 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.534 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.534 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-13T03:19:21.534 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.638 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628280 2024-04-13T03:19:21.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-13T03:19:21.647 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.752 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2024-04-13T03:19:21.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-13T03:19:21.761 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:21.866 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628279 2024-04-13T03:19:21.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:21.875 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:21.875 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:21.931 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:21.931 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:21.986 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:22.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.126+0000 7f37a011e640 1 Processor -- start 2024-04-13T03:19:22.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.126+0000 7f37a011e640 1 -- start start 2024-04-13T03:19:22.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f37a011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37980cdf50 0x7f37980ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f37a011e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f379806e150 0x7f379806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f37a011e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37980ce870 con 0x7f379806cb10 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f37a011e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f37980998d0 con 0x7f379806e150 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f37a011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3798073020 con 0x7f37980cdf50 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f379806e150 0x7f379806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f379806e150 0x7f379806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40244/0 (socket says 172.21.15.106:40244) 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 -- 172.21.15.106:0/2521199711 learned_addr learned my addr 172.21.15.106:0/2521199711 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 -- 172.21.15.106:0/2521199711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37980cdf50 msgr2=0x7f37980ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 --2- 172.21.15.106:0/2521199711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37980cdf50 0x7f37980ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 -- 172.21.15.106:0/2521199711 >> v1:172.21.15.114:6789/0 conn(0x7f379806cb10 legacy=0x7f3798072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.127+0000 7f379e6bb640 1 -- 172.21.15.106:0/2521199711 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798073300 con 0x7f379806e150 2024-04-13T03:19:22.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f379e6bb640 1 --2- 172.21.15.106:0/2521199711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806e150 0x7f379806e600 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f379000d150 tx=0x7f379002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46efb10fa8e56c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f379ceb8640 1 -- 172.21.15.106:0/2521199711 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379002ea80 con 0x7f379806e150 2024-04-13T03:19:22.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f379ceb8640 1 -- 172.21.15.106:0/2521199711 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379002ec20 con 0x7f379806e150 2024-04-13T03:19:22.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f379ceb8640 1 -- 172.21.15.106:0/2521199711 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790009600 con 0x7f379806e150 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f37a011e640 1 -- 172.21.15.106:0/2521199711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806e150 msgr2=0x7f379806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.128+0000 7f37a011e640 1 --2- 172.21.15.106:0/2521199711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806e150 0x7f379806e600 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f379000d150 tx=0x7f379002e0e0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 -- 172.21.15.106:0/2521199711 shutdown_connections 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 --2- 172.21.15.106:0/2521199711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806e150 0x7f379806e600 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 --2- 172.21.15.106:0/2521199711 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f37980cdf50 0x7f37980ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 -- 172.21.15.106:0/2521199711 >> 172.21.15.106:0/2521199711 conn(0x7f37980b1f10 msgr2=0x7f37980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 -- 172.21.15.106:0/2521199711 shutdown_connections 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.129+0000 7f37a011e640 1 -- 172.21.15.106:0/2521199711 wait complete. 2024-04-13T03:19:22.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 Processor -- start 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 -- start start 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f379806e150 0x7f379815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37980cdf50 0x7f379815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f379806eb40 con 0x7f379806cb10 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37980738c0 con 0x7f37980cdf50 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f37a011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f379806ecc0 con 0x7f379806e150 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.130+0000 7f379deba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40260/0 (socket says 172.21.15.106:40260) 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 -- 172.21.15.106:0/890234470 learned_addr learned my addr 172.21.15.106:0/890234470 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379d6b9640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f379806e150 0x7f379815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379e6bb640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37980cdf50 0x7f379815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 -- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f379806e150 msgr2=0x7f379815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f379806e150 0x7f379815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 -- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37980cdf50 msgr2=0x7f379815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37980cdf50 0x7f379815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 -- 172.21.15.106:0/890234470 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37980bd110 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f379deba640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f379400b440 tx=0x7f379400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794015020 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3794011890 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379815d3f0 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794011c80 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.131+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379815d9a0 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.132+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3798068a10 con 0x7f379806cb10 2024-04-13T03:19:22.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.132+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3794018020 con 0x7f379806cb10 2024-04-13T03:19:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.133+0000 7f378e7fc640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 0x7f377c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.133+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f379408be30 con 0x7f379806cb10 2024-04-13T03:19:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.133+0000 7f379d6b9640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 0x7f377c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.133+0000 7f379d6b9640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 0x7f377c06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f379815b540 tx=0x7f3788009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.137+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f379401f020 con 0x7f379806cb10 2024-04-13T03:19:22.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.317+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f379815e760 con 0x7f379806cb10 2024-04-13T03:19:22.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.318+0000 7f378e7fc640 1 -- 172.21.15.106:0/890234470 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16988 (secure 0 0 0) 0x7f379400d040 con 0x7f379806cb10 2024-04-13T03:19:22.319 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:22.319 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":121,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:16.039307+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 msgr2=0x7f377c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 0x7f377c06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f379815b540 tx=0x7f3788009040 comp rx=0 tx=0).stop 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 msgr2=0x7f37980bcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f379400b440 tx=0x7f379400b900 comp rx=0 tx=0).stop 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 shutdown_connections 2024-04-13T03:19:22.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f377c06c8f0 0x7f377c06edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f379806e150 0x7f379815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37980cdf50 0x7f379815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 --2- 172.21.15.106:0/890234470 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f379806cb10 0x7f37980bcbd0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 >> 172.21.15.106:0/890234470 conn(0x7f37980b1f10 msgr2=0x7f37980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 shutdown_connections 2024-04-13T03:19:22.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.320+0000 7f37a011e640 1 -- 172.21.15.106:0/890234470 wait complete. 2024-04-13T03:19:22.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:19:12.871192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:19:13.595223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:22.332 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:22.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.434+0000 7f3727a1c640 1 Processor -- start 2024-04-13T03:19:22.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 -- start start 2024-04-13T03:19:22.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 0x7f37200d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f372015ed90 0x7f37200d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37200cfd20 con 0x7f37200b3d60 2024-04-13T03:19:22.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3720162af0 con 0x7f372015ed90 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.435+0000 7f3727a1c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3720162dd0 con 0x7f372015e9b0 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f372015ed90 0x7f37200d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f372015ed90 0x7f37200d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40270/0 (socket says 172.21.15.106:40270) 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 -- 172.21.15.106:0/2133052955 learned_addr learned my addr 172.21.15.106:0/2133052955 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f37257b8640 1 --2- 172.21.15.106:0/2133052955 >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 0x7f37200d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 -- 172.21.15.106:0/2133052955 >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 msgr2=0x7f37200d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 --2- 172.21.15.106:0/2133052955 >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 0x7f37200d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 -- 172.21.15.106:0/2133052955 >> v1:172.21.15.114:6789/0 conn(0x7f37200b3d60 legacy=0x7f37200d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f3724fb7640 1 -- 172.21.15.106:0/2133052955 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37201630b0 con 0x7f372015ed90 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.436+0000 7f37257b8640 1 --2- 172.21.15.106:0/2133052955 >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 0x7f37200d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3724fb7640 1 --2- 172.21.15.106:0/2133052955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f372015ed90 0x7f37200d45f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f372008e4e0 tx=0x7f371c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d24d65ca65ec99 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3717fff640 1 -- 172.21.15.106:0/2133052955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004230 con 0x7f372015ed90 2024-04-13T03:19:22.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3717fff640 1 -- 172.21.15.106:0/2133052955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c0043d0 con 0x7f372015ed90 2024-04-13T03:19:22.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3717fff640 1 -- 172.21.15.106:0/2133052955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c030840 con 0x7f372015ed90 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 -- 172.21.15.106:0/2133052955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f372015ed90 msgr2=0x7f37200d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2133052955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f372015ed90 0x7f37200d45f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f372008e4e0 tx=0x7f371c02b980 comp rx=0 tx=0).stop 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 -- 172.21.15.106:0/2133052955 shutdown_connections 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2133052955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f372015ed90 0x7f37200d45f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2133052955 >> v2:172.21.15.114:3300/0 conn(0x7f372015e9b0 0x7f37200d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 -- 172.21.15.106:0/2133052955 >> 172.21.15.106:0/2133052955 conn(0x7f372008bf10 msgr2=0x7f37200b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 -- 172.21.15.106:0/2133052955 shutdown_connections 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.437+0000 7f3727a1c640 1 -- 172.21.15.106:0/2133052955 wait complete. 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.438+0000 7f3727a1c640 1 Processor -- start 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.438+0000 7f3727a1c640 1 -- start start 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37200b3d60 0x7f372014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f372015e9b0 0x7f3720160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3720162800 con 0x7f3720152f60 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3720162680 con 0x7f37200b3d60 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3727a1c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f372006eb20 con 0x7f372015e9b0 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3725fb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3725fb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40272/0 (socket says 172.21.15.106:40272) 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3725fb9640 1 -- 172.21.15.106:0/2265652692 learned_addr learned my addr 172.21.15.106:0/2265652692 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:22.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f37257b8640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37200b3d60 0x7f372014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3724fb7640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f372015e9b0 0x7f3720160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3725fb9640 1 -- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f372015e9b0 msgr2=0x7f3720160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.439+0000 7f3725fb9640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f372015e9b0 0x7f3720160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3725fb9640 1 -- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37200b3d60 msgr2=0x7f372014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3725fb9640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37200b3d60 0x7f372014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3725fb9640 1 -- 172.21.15.106:0/2265652692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372014f7d0 con 0x7f3720152f60 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3725fb9640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f371800c7e0 tx=0x7f371800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718004b60 con 0x7f3720152f60 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3718004d00 con 0x7f3720152f60 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371800aa30 con 0x7f3720152f60 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3720161fd0 con 0x7f3720152f60 2024-04-13T03:19:22.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.440+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37200b6740 con 0x7f3720152f60 2024-04-13T03:19:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.441+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37200b2730 con 0x7f3720152f60 2024-04-13T03:19:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.441+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f371800abd0 con 0x7f3720152f60 2024-04-13T03:19:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.442+0000 7f3715ffb640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 0x7f370406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.442+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f371808bea0 con 0x7f3720152f60 2024-04-13T03:19:22.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.442+0000 7f37257b8640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 0x7f370406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.442+0000 7f37257b8640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 0x7f370406ede0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f371000a990 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.446+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37200b2730 con 0x7f3720152f60 2024-04-13T03:19:22.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.623+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3720068a10 con 0x7f3720152f60 2024-04-13T03:19:22.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.623+0000 7f3715ffb640 1 -- 172.21.15.106:0/2265652692 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f3718020070 con 0x7f3720152f60 2024-04-13T03:19:22.624 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:22.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 msgr2=0x7f370406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 0x7f370406ede0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f371000a990 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-13T03:19:22.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 msgr2=0x7f37201613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f371800c7e0 tx=0x7f371800cca0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 shutdown_connections 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f370406c920 0x7f370406ede0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f372015e9b0 0x7f3720160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3720152f60 0x7f37201613b0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 --2- 172.21.15.106:0/2265652692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37200b3d60 0x7f372014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 >> 172.21.15.106:0/2265652692 conn(0x7f372008bf10 msgr2=0x7f37200b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 shutdown_connections 2024-04-13T03:19:22.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.625+0000 7f3727a1c640 1 -- 172.21.15.106:0/2265652692 wait complete. 2024-04-13T03:19:22.637 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.740+0000 7f90bffbb640 1 Processor -- start 2024-04-13T03:19:22.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.740+0000 7f90bffbb640 1 -- start start 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bffbb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f90b806bff0 0x7f90b806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bffbb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f90b80d07c0 0x7f90b806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bffbb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b80cef80 con 0x7f90b815fb50 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bffbb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f90b80cf260 con 0x7f90b80d07c0 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bffbb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f90b80cf540 con 0x7f90b806bff0 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90be558640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f90b80d07c0 0x7f90b806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bdd57640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f90b815fb50 0x7f90b806b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57014/0 (socket says 172.21.15.106:57014) 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bdd57640 1 -- 172.21.15.106:0/543666404 learned_addr learned my addr 172.21.15.106:0/543666404 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90bd556640 1 --2- 172.21.15.106:0/543666404 >> v2:172.21.15.114:3300/0 conn(0x7f90b806bff0 0x7f90b806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90be558640 1 -- 172.21.15.106:0/543666404 >> v2:172.21.15.114:3300/0 conn(0x7f90b806bff0 msgr2=0x7f90b806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:22.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90be558640 1 --2- 172.21.15.106:0/543666404 >> v2:172.21.15.114:3300/0 conn(0x7f90b806bff0 0x7f90b806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90be558640 1 -- 172.21.15.106:0/543666404 >> v1:172.21.15.106:6789/0 conn(0x7f90b815fb50 legacy=0x7f90b806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.741+0000 7f90be558640 1 -- 172.21.15.106:0/543666404 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b80cf820 con 0x7f90b80d07c0 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90be558640 1 --2- 172.21.15.106:0/543666404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b80d07c0 0x7f90b806ebf0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f90b80ce3e0 tx=0x7f90b002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf138c3dbc6d151c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bcd55640 1 -- 172.21.15.106:0/543666404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0039070 con 0x7f90b80d07c0 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bcd55640 1 -- 172.21.15.106:0/543666404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b002cbf0 con 0x7f90b80d07c0 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bcd55640 1 -- 172.21.15.106:0/543666404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0034440 con 0x7f90b80d07c0 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 -- 172.21.15.106:0/543666404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b80d07c0 msgr2=0x7f90b806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 --2- 172.21.15.106:0/543666404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b80d07c0 0x7f90b806ebf0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f90b80ce3e0 tx=0x7f90b002be40 comp rx=0 tx=0).stop 2024-04-13T03:19:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 -- 172.21.15.106:0/543666404 shutdown_connections 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 --2- 172.21.15.106:0/543666404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b80d07c0 0x7f90b806ebf0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 --2- 172.21.15.106:0/543666404 >> v2:172.21.15.114:3300/0 conn(0x7f90b806bff0 0x7f90b806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.742+0000 7f90bffbb640 1 -- 172.21.15.106:0/543666404 >> 172.21.15.106:0/543666404 conn(0x7f90b80cd6a0 msgr2=0x7f90b80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.743+0000 7f90bffbb640 1 -- 172.21.15.106:0/543666404 shutdown_connections 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.743+0000 7f90bffbb640 1 -- 172.21.15.106:0/543666404 wait complete. 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 Processor -- start 2024-04-13T03:19:22.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 -- start start 2024-04-13T03:19:22.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f90b80d07c0 0x7f90b80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b815fb50 0x7f90b80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80ced20 con 0x7f90b806bff0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80ceba0 con 0x7f90b815fb50 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bffbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f90b815ef10 con 0x7f90b80d07c0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bdd57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bdd57640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54168/0 (socket says 172.21.15.106:54168) 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.744+0000 7f90bdd57640 1 -- 172.21.15.106:0/1728685423 learned_addr learned my addr 172.21.15.106:0/1728685423 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 -- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f90b80d07c0 msgr2=0x7f90b80b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f90b80d07c0 0x7f90b80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 -- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b815fb50 msgr2=0x7f90b80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b815fb50 0x7f90b80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 -- 172.21.15.106:0/1728685423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81f4b40 con 0x7f90b806bff0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bdd57640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c3f0 tx=0x7f90b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4017070 con 0x7f90b806bff0 2024-04-13T03:19:22.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b4005ce0 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81f4e20 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b40072f0 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.745+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81f53e0 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.746+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b8068a10 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.747+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90b401a070 con 0x7f90b806bff0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.747+0000 7f90ae7fc640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.747+0000 7f90bd556640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:22.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.747+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f90b408afe0 con 0x7f90b806bff0 2024-04-13T03:19:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.747+0000 7f90bd556640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 0x7f909c06edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f90b8153d10 tx=0x7f90a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:22.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.751+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90b4059310 con 0x7f90b806bff0 2024-04-13T03:19:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.926+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90b80738b0 con 0x7f90b806bff0 2024-04-13T03:19:22.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.926+0000 7f90ae7fc640 1 -- 172.21.15.106:0/1728685423 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f90b4058c10 con 0x7f90b806bff0 2024-04-13T03:19:22.927 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 msgr2=0x7f909c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 0x7f909c06edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f90b8153d10 tx=0x7f90a8008040 comp rx=0 tx=0).stop 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 msgr2=0x7f90b81f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c3f0 tx=0x7f90b400c8b0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 shutdown_connections 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f909c06c8f0 0x7f909c06edb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f90b80d07c0 0x7f90b80b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b815fb50 0x7f90b80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 --2- 172.21.15.106:0/1728685423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90b806bff0 0x7f90b81f4410 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 >> 172.21.15.106:0/1728685423 conn(0x7f90b80cd6a0 msgr2=0x7f90b80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 shutdown_connections 2024-04-13T03:19:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:22.928+0000 7f90bffbb640 1 -- 172.21.15.106:0/1728685423 wait complete. 2024-04-13T03:19:22.939 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:23.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 Processor -- start 2024-04-13T03:19:23.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 -- start start 2024-04-13T03:19:23.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b1c15ed90 0x7f0b1c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b1c0b3d60 0x7f0b1c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1c0cfd20 con 0x7f0b1c15e9b0 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1c162af0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.042+0000 7f0b220de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1c162dd0 con 0x7f0b1c15ed90 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b1c0b3d60 0x7f0b1c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b1c15e9b0 0x7f0b1c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57022/0 (socket says 172.21.15.106:57022) 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1b7fe640 1 -- 172.21.15.106:0/533232362 learned_addr learned my addr 172.21.15.106:0/533232362 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1bfff640 1 -- 172.21.15.106:0/533232362 >> v2:172.21.15.114:3300/0 conn(0x7f0b1c15ed90 msgr2=0x7f0b1c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1bfff640 1 --2- 172.21.15.106:0/533232362 >> v2:172.21.15.114:3300/0 conn(0x7f0b1c15ed90 0x7f0b1c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1bfff640 1 -- 172.21.15.106:0/533232362 >> v1:172.21.15.106:6789/0 conn(0x7f0b1c15e9b0 legacy=0x7f0b1c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1a7fc640 1 -- 172.21.15.106:0/533232362 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3076450299 0 0) 0x7f0b1c0cfd20 con 0x7f0b1c15e9b0 2024-04-13T03:19:23.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.043+0000 7f0b1bfff640 1 -- 172.21.15.106:0/533232362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c1630b0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b1bfff640 1 --2- 172.21.15.106:0/533232362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0d0200 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c08ebd0 tx=0x7f0b1402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4283adda9c4291b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b1a7fc640 1 -- 172.21.15.106:0/533232362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14039070 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b1a7fc640 1 -- 172.21.15.106:0/533232362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1402cd00 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b1a7fc640 1 -- 172.21.15.106:0/533232362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14034660 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 -- 172.21.15.106:0/533232362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 msgr2=0x7f0b1c0d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 --2- 172.21.15.106:0/533232362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0d0200 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c08ebd0 tx=0x7f0b1402bf50 comp rx=0 tx=0).stop 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 -- 172.21.15.106:0/533232362 shutdown_connections 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 --2- 172.21.15.106:0/533232362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0d0200 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 --2- 172.21.15.106:0/533232362 >> v2:172.21.15.114:3300/0 conn(0x7f0b1c15ed90 0x7f0b1c0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 -- 172.21.15.106:0/533232362 >> 172.21.15.106:0/533232362 conn(0x7f0b1c08bf10 msgr2=0x7f0b1c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 -- 172.21.15.106:0/533232362 shutdown_connections 2024-04-13T03:19:23.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.044+0000 7f0b220de640 1 -- 172.21.15.106:0/533232362 wait complete. 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.045+0000 7f0b220de640 1 Processor -- start 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.046+0000 7f0b220de640 1 -- start start 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c15e9b0 0x7f0b1c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 0x7f0b1c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c162800 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c162680 con 0x7f0b1c15e9b0 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1c0d4d00 con 0x7f0b1c06d5c0 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54178/0 (socket says 172.21.15.106:54178) 2024-04-13T03:19:23.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 -- 172.21.15.106:0/820703940 learned_addr learned my addr 172.21.15.106:0/820703940 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1affd640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c15e9b0 0x7f0b1c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1bfff640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 0x7f0b1c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 -- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 msgr2=0x7f0b1c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 0x7f0b1c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 -- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c15e9b0 msgr2=0x7f0b1c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c15e9b0 0x7f0b1c0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 -- 172.21.15.106:0/820703940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c0cd710 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1bfff640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 0x7f0b1c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b1b7fe640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c500 tx=0x7f0b0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c018070 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c007580 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c004a20 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c0cd9f0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.047+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c0cdfb0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.048+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c068a10 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.049+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b0c004bc0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.049+0000 7f0b03fff640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 0x7f0b0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.049+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0b0c08ae90 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.049+0000 7f0b1affd640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 0x7f0b0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.050+0000 7f0b1affd640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 0x7f0b0806ede0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c15c3a0 tx=0x7f0b10009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.054+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b0c058af0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.230+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b1c0b45a0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.231+0000 7f0b03fff640 1 -- 172.21.15.106:0/820703940 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f0b0c05cdf0 con 0x7f0b1c0b3d60 2024-04-13T03:19:23.232 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 msgr2=0x7f0b0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 0x7f0b0806ede0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c15c3a0 tx=0x7f0b10009060 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 msgr2=0x7f0b1c0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c500 tx=0x7f0b0c00c9c0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 shutdown_connections 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b0806c920 0x7f0b0806ede0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b1c06d5c0 0x7f0b1c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c15e9b0 0x7f0b1c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 --2- 172.21.15.106:0/820703940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b1c0b3d60 0x7f0b1c0cc9a0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 >> 172.21.15.106:0/820703940 conn(0x7f0b1c08bf10 msgr2=0x7f0b1c0b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 shutdown_connections 2024-04-13T03:19:23.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.233+0000 7f0b220de640 1 -- 172.21.15.106:0/820703940 wait complete. 2024-04-13T03:19:23.244 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:19:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.347+0000 7f6501fbb640 1 Processor -- start 2024-04-13T03:19:23.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 -- start start 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 0x7f64fc0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64fc06e8a0 0x7f64fc15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 0x7f64fc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc0cf700 con 0x7f64fc06e8a0 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc15c400 con 0x7f64fc06f800 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f6501fbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc15c6e0 con 0x7f64fc159120 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f64faffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64fc06e8a0 0x7f64fc15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.348+0000 7f64faffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f64fc06e8a0 0x7f64fc15bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54194/0 (socket says 172.21.15.106:54194) 2024-04-13T03:19:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 -- 172.21.15.106:0/2366544977 learned_addr learned my addr 172.21.15.106:0/2366544977 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64fb7fe640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 0x7f64fc0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64fbfff640 1 --2- 172.21.15.106:0/2366544977 >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 0x7f64fc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 -- 172.21.15.106:0/2366544977 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 msgr2=0x7f64fc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 0x7f64fc0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 -- 172.21.15.106:0/2366544977 >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 msgr2=0x7f64fc06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 --2- 172.21.15.106:0/2366544977 >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 0x7f64fc06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64faffd640 1 -- 172.21.15.106:0/2366544977 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc15c9c0 con 0x7f64fc06e8a0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64fbfff640 1 --2- 172.21.15.106:0/2366544977 >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 0x7f64fc06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.349+0000 7f64fb7fe640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 0x7f64fc0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f64faffd640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc06e8a0 0x7f64fc15bb80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b5d10 tx=0x7f64e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d068f16bf45bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f64fa7fc640 1 -- 172.21.15.106:0/2366544977 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4035020 con 0x7f64fc06e8a0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f64fa7fc640 1 -- 172.21.15.106:0/2366544977 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4031870 con 0x7f64fc06e8a0 2024-04-13T03:19:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f64fa7fc640 1 -- 172.21.15.106:0/2366544977 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4031c40 con 0x7f64fc06e8a0 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 -- 172.21.15.106:0/2366544977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc06e8a0 msgr2=0x7f64fc15bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc06e8a0 0x7f64fc15bb80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b5d10 tx=0x7f64e402b910 comp rx=0 tx=0).stop 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 -- 172.21.15.106:0/2366544977 shutdown_connections 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 --2- 172.21.15.106:0/2366544977 >> v2:172.21.15.114:3300/0 conn(0x7f64fc06f800 0x7f64fc06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc06e8a0 0x7f64fc15bb80 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 --2- 172.21.15.106:0/2366544977 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc159120 0x7f64fc0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 -- 172.21.15.106:0/2366544977 >> 172.21.15.106:0/2366544977 conn(0x7f64fc0d1b60 msgr2=0x7f64fc06f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 -- 172.21.15.106:0/2366544977 shutdown_connections 2024-04-13T03:19:23.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.350+0000 7f6501fbb640 1 -- 172.21.15.106:0/2366544977 wait complete. 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.351+0000 7f6501fbb640 1 Processor -- start 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 -- start start 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc06e8a0 0x7f64fc1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64fc06f800 0x7f64fc144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc15c0c0 con 0x7f64fc159120 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc159500 con 0x7f64fc06f800 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f6501fbb640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0cf320 con 0x7f64fc06e8a0 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54206/0 (socket says 172.21.15.106:54206) 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.352+0000 7f64fbfff640 1 -- 172.21.15.106:0/883329022 learned_addr learned my addr 172.21.15.106:0/883329022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fb7fe640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc06e8a0 0x7f64fc1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64faffd640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64fc06f800 0x7f64fc144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 -- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc06e8a0 msgr2=0x7f64fc1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc06e8a0 0x7f64fc1417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 -- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64fc06f800 msgr2=0x7f64fc144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64fc06f800 0x7f64fc144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 -- 172.21.15.106:0/883329022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc142d90 con 0x7f64fc159120 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64fbfff640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d7d0 tx=0x7f64f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4004030 con 0x7f64fc159120 2024-04-13T03:19:23.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f40041d0 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc143070 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40096a0 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.353+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc143630 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.354+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc069120 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.354+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f4009840 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.355+0000 7f64f3fff640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 0x7f64dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.355+0000 7f64fb7fe640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 0x7f64dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.355+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f64f408c000 con 0x7f64fc159120 2024-04-13T03:19:23.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.355+0000 7f64fb7fe640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 0x7f64dc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009ef0 tx=0x7f64ec00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.359+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64f405a330 con 0x7f64fc159120 2024-04-13T03:19:23.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.546+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f64fc0ce900 con 0x7f64fc159120 2024-04-13T03:19:23.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.547+0000 7f64f3fff640 1 -- 172.21.15.106:0/883329022 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7f64f4059c30 con 0x7f64fc159120 2024-04-13T03:19:23.548 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 msgr2=0x7f64dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 0x7f64dc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009ef0 tx=0x7f64ec00b040 comp rx=0 tx=0).stop 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 msgr2=0x7f64fc144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d7d0 tx=0x7f64f400dc90 comp rx=0 tx=0).stop 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 shutdown_connections 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f64dc06c8f0 0x7f64dc06edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64fc159120 0x7f64fc144650 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64fc06f800 0x7f64fc144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 --2- 172.21.15.106:0/883329022 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f64fc06e8a0 0x7f64fc1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 >> 172.21.15.106:0/883329022 conn(0x7f64fc0d1b60 msgr2=0x7f64fc0b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 shutdown_connections 2024-04-13T03:19:23.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.549+0000 7f6501fbb640 1 -- 172.21.15.106:0/883329022 wait complete. 2024-04-13T03:19:23.561 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:19:23.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:23.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:23.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:19:23.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:23.611 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:19:23.637 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-13T03:19:23.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.779+0000 7f0f02036640 1 Processor -- start 2024-04-13T03:19:23.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.779+0000 7f0f02036640 1 -- start start 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0f02036640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0efc156b40 0x7f0efc06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0f02036640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0efc06ebf0 0x7f0efc15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0f02036640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0efc15d690 con 0x7f0efc15d110 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0f02036640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0efc06b8e0 con 0x7f0efc156b40 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0f02036640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0efc06bea0 con 0x7f0efc06ebf0 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0efc156b40 0x7f0efc06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efbfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0efc15d110 0x7f0efc06c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57038/0 (socket says 172.21.15.106:57038) 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efbfff640 1 -- 172.21.15.106:0/3460797155 learned_addr learned my addr 172.21.15.106:0/3460797155 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efaffd640 1 --2- 172.21.15.106:0/3460797155 >> v2:172.21.15.114:3300/0 conn(0x7f0efc06ebf0 0x7f0efc15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efb7fe640 1 -- 172.21.15.106:0/3460797155 >> v2:172.21.15.114:3300/0 conn(0x7f0efc06ebf0 msgr2=0x7f0efc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/3460797155 >> v2:172.21.15.114:3300/0 conn(0x7f0efc06ebf0 0x7f0efc15cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efb7fe640 1 -- 172.21.15.106:0/3460797155 >> v1:172.21.15.106:6789/0 conn(0x7f0efc15d110 legacy=0x7f0efc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efb7fe640 1 -- 172.21.15.106:0/3460797155 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc15d970 con 0x7f0efc156b40 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.780+0000 7f0efa7fc640 1 -- 172.21.15.106:0/3460797155 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305684979 0 0) 0x7f0efc15d690 con 0x7f0efc15d110 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.781+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/3460797155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc06e6b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0af3d0 tx=0x7f0eec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe110c592a43351e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.781+0000 7f0efa7fc640 1 -- 172.21.15.106:0/3460797155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec035020 con 0x7f0efc156b40 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.781+0000 7f0efa7fc640 1 -- 172.21.15.106:0/3460797155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec031830 con 0x7f0efc156b40 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.781+0000 7f0efa7fc640 1 -- 172.21.15.106:0/3460797155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec031c00 con 0x7f0efc156b40 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 -- 172.21.15.106:0/3460797155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 msgr2=0x7f0efc06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 --2- 172.21.15.106:0/3460797155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc06e6b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0af3d0 tx=0x7f0eec02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 -- 172.21.15.106:0/3460797155 shutdown_connections 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 --2- 172.21.15.106:0/3460797155 >> v2:172.21.15.114:3300/0 conn(0x7f0efc06ebf0 0x7f0efc15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 --2- 172.21.15.106:0/3460797155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc06e6b0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 -- 172.21.15.106:0/3460797155 >> 172.21.15.106:0/3460797155 conn(0x7f0efc0cd6a0 msgr2=0x7f0efc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 -- 172.21.15.106:0/3460797155 shutdown_connections 2024-04-13T03:19:23.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.782+0000 7f0f02036640 1 -- 172.21.15.106:0/3460797155 wait complete. 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.783+0000 7f0f02036640 1 Processor -- start 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.783+0000 7f0f02036640 1 -- start start 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0efc06ebf0 0x7f0efc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 0x7f0efc0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc06d880 con 0x7f0efc156b40 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc156f20 con 0x7f0efc06ebf0 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc06da00 con 0x7f0efc15d110 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54226/0 (socket says 172.21.15.106:54226) 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 -- 172.21.15.106:0/2337779445 learned_addr learned my addr 172.21.15.106:0/2337779445 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0efc06ebf0 0x7f0efc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efbfff640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 0x7f0efc0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 -- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 msgr2=0x7f0efc0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 0x7f0efc0b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 -- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0efc06ebf0 msgr2=0x7f0efc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0efc06ebf0 0x7f0efc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 -- 172.21.15.106:0/2337779445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc154a50 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efbfff640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 0x7f0efc0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0efaffd640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000d7d0 tx=0x7f0ef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0ee3fff640 1 -- 172.21.15.106:0/2337779445 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000b6c0 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0ee3fff640 1 -- 172.21.15.106:0/2337779445 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef0019070 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc154d30 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.784+0000 7f0ee3fff640 1 -- 172.21.15.106:0/2337779445 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000bac0 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.785+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1552f0 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.786+0000 7f0ee3fff640 1 -- 172.21.15.106:0/2337779445 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef000bc60 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.786+0000 7f0ee3fff640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.786+0000 7f0ee3fff640 1 -- 172.21.15.106:0/2337779445 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0ef008c060 con 0x7f0efc156b40 2024-04-13T03:19:23.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.786+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.787+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0eec036000 tx=0x7f0eec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.787+0000 7f0f04e45740 1 Processor -- start 2024-04-13T03:19:23.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.787+0000 7f0f04e45740 1 -- start start 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c09494d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c094654d0 0x561c094952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c093d0170 con 0x561c09491ff0 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561c093767f0 con 0x561c094654d0 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561c093cc130 con 0x561c094923d0 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54232/0 (socket says 172.21.15.106:54232) 2024-04-13T03:19:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.788+0000 7f0efb7fe640 1 -- 172.21.15.106:0/1045101185 learned_addr learned my addr 172.21.15.106:0/1045101185 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efaffd640 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c09494d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efbfff640 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c094654d0 0x561c094952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 -- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 msgr2=0x561c09494d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c09494d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 -- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c094654d0 msgr2=0x561c094952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c094654d0 0x561c094952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 -- 172.21.15.106:0/1045101185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c091ac7d0 con 0x561c09491ff0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0eec02bee0 tx=0x7f0eec030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8885ce99f21a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0ee17fa640 1 -- 172.21.15.106:0/1045101185 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec055070 con 0x561c09491ff0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0ee17fa640 1 -- 172.21.15.106:0/1045101185 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec048bc0 con 0x561c09491ff0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.789+0000 7f0ee17fa640 1 -- 172.21.15.106:0/1045101185 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec035040 con 0x561c09491ff0 2024-04-13T03:19:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- 172.21.15.106:0/1045101185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 msgr2=0x561c09464f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0eec02bee0 tx=0x7f0eec030000 comp rx=0 tx=0).stop 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- 172.21.15.106:0/1045101185 shutdown_connections 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c09494d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c094654d0 0x561c094952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1045101185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c09491ff0 0x561c09464f90 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- 172.21.15.106:0/1045101185 >> 172.21.15.106:0/1045101185 conn(0x561c09467200 msgr2=0x561c094afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- 172.21.15.106:0/1045101185 shutdown_connections 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- 172.21.15.106:0/1045101185 wait complete. 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 -- start start 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.790+0000 7f0f04e45740 1 Processor -- start 2024-04-13T03:19:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c0949d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c094698a0 con 0x561c094923d0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561c09389d00 con 0x561c0949cff0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561c0946c0e0 con 0x561c09491ff0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0f04e45740 10 client.0 _ll_register_callbacks cb 0x561c093896f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54246/0 (socket says 172.21.15.106:54246) 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 -- 172.21.15.106:0/181900665 learned_addr learned my addr 172.21.15.106:0/181900665 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efbfff640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c0949d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 -- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 msgr2=0x561c094bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094bfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 -- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 msgr2=0x561c0949d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.791+0000 7f0efaffd640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c0949d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0efaffd640 1 -- 172.21.15.106:0/181900665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c090e23f0 con 0x561c094923d0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0efaffd640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00622b0 tx=0x7f0ef005a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a963c4ca048cfb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0088060 con 0x561c094923d0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0ef0056270 con 0x561c094923d0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eec048bc0 con 0x561c094923d0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0ef0059050 con 0x561c094923d0 2024-04-13T03:19:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 10 client.9303 fetch_fsmap learned FSMap version 86 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x561c090a20f0 con 0x561c094923d0 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec048bc0 con 0x561c094923d0 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 86) ==== 3494+0+0 (secure 0 0 0) 0x7f0ef0013070 con 0x561c094923d0 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 10 client.9303 fetch_fsmap finished waiting for FSMap version 86 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 10 client.9303 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 20 client.9303 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c09482fb0 0x561c09483390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 4 client.9303 mds_command: new command op to 9084 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 --> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c093ddb90 con 0x561c09482fb0 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.792+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c09482fb0 0x561c09483390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0eda7fc640 10 client.9303 ms_handle_connect on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c09482fb0 0x561c09483390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fc112203a41fa3 server_cookie=2a4da2e9f4cf8455 in_seq=0 out_seq=0 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0eda7fc640 1 -- 172.21.15.106:0/181900665 <== mds.0 v2:172.21.15.106:6835/1427655446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c093ddb90 con 0x561c09482fb0 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0eda7fc640 10 client.9303 handle_command_reply: tid=0 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 client.9303 shutdown 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 msgr2=0x561c09497d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00622b0 tx=0x7f0ef005a440 comp rx=0 tx=0).stop 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 shutdown_connections 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c09482fb0 0x561c09483390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c0949d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0eda7fc640 0 client.9303 ms_handle_reset on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09497d90 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 --2- 172.21.15.106:0/181900665 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094bfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 >> 172.21.15.106:0/181900665 conn(0x561c094654d0 msgr2=0x561c09468fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 shutdown_connections 2024-04-13T03:19:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.793+0000 7f0f04e45740 1 -- 172.21.15.106:0/181900665 wait complete. 2024-04-13T03:19:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.794+0000 7f0f04e45740 20 client.9303 trim_cache size 0 max 16384 2024-04-13T03:19:23.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.808+0000 7f0f04e45740 1 Processor -- start 2024-04-13T03:19:23.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.808+0000 7f0f04e45740 1 -- start start 2024-04-13T03:19:23.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c09494d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1ff110 con 0x561c094923d0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1fe190 con 0x561c0949cff0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc153330 con 0x561c09491ff0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54258/0 (socket says 172.21.15.106:54258) 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 -- 172.21.15.106:0/1684417253 learned_addr learned my addr 172.21.15.106:0/1684417253 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 -- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 msgr2=0x561c094d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 -- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 msgr2=0x561c09494d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c09494d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 -- 172.21.15.106:0/1684417253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c094b0860 con 0x561c094923d0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0efaffd640 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0057eb0 tx=0x7f0ef0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5574122e98dc6b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.809+0000 7f0ee0ff9640 1 -- 172.21.15.106:0/1684417253 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0088070 con 0x561c094923d0 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0ee0ff9640 1 -- 172.21.15.106:0/1684417253 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef0056270 con 0x561c094923d0 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0ee0ff9640 1 -- 172.21.15.106:0/1684417253 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00601b0 con 0x561c094923d0 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 -- 172.21.15.106:0/1684417253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 msgr2=0x561c09494820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0057eb0 tx=0x7f0ef0020000 comp rx=0 tx=0).stop 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 -- 172.21.15.106:0/1684417253 shutdown_connections 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c0949cff0 0x561c09494d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c094923d0 0x561c09494820 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 --2- 172.21.15.106:0/1684417253 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c09491ff0 0x561c094d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 -- 172.21.15.106:0/1684417253 >> 172.21.15.106:0/1684417253 conn(0x561c094654d0 msgr2=0x561c094910c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 -- 172.21.15.106:0/1684417253 shutdown_connections 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.810+0000 7f0f04e45740 1 -- 172.21.15.106:0/1684417253 wait complete. 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.811+0000 7f0f04e45740 1 -- start start 2024-04-13T03:19:23.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.811+0000 7f0f04e45740 1 Processor -- start 2024-04-13T03:19:23.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c09491ff0 0x561c094d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c094d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c09498420 con 0x561c0949cff0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1fdd30 con 0x561c09491ff0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561c0938e720 con 0x561c094923d0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 10 client.0 _ll_register_callbacks cb 0x7f0ef00073a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54274/0 (socket says 172.21.15.106:54274) 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 -- 172.21.15.106:0/3501055201 learned_addr learned my addr 172.21.15.106:0/3501055201 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c09491ff0 0x561c094d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 -- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 msgr2=0x561c094d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c094d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 -- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c09491ff0 msgr2=0x561c094d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c09491ff0 0x561c094d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 -- 172.21.15.106:0/3501055201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c094b2f50 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0efbfff640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40079d0 tx=0x7f0ef4012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0ab064df5bfebe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0eda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401a030 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0ef4004b60 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef0056270 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0ef4023050 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 10 client.9309 fetch_fsmap learned FSMap version 86 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x561c094b39b0 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.812+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef0056270 con 0x561c0949cff0 2024-04-13T03:19:23.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 86) ==== 3494+0+0 (secure 0 0 0) 0x7f0ef4026020 con 0x561c0949cff0 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 10 client.9309 fetch_fsmap finished waiting for FSMap version 86 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 10 client.9309 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 20 client.9309 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c094917d0 0x561c09485f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 4 client.9309 mds_command: new command op to 9084 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 --> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c093d7a00 con 0x561c094917d0 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c094917d0 0x561c09485f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0eda7fc640 10 client.9309 ms_handle_connect on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:23.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.813+0000 7f0efb7fe640 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c094917d0 0x561c09485f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b659637a73c13925 server_cookie=7b38eb7ebef7d65a in_seq=0 out_seq=0 2024-04-13T03:19:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0eda7fc640 1 -- 172.21.15.106:0/3501055201 <== mds.0 v2:172.21.15.106:6835/1427655446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c093d7a00 con 0x561c094917d0 2024-04-13T03:19:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0eda7fc640 10 client.9309 handle_command_reply: tid=0 2024-04-13T03:19:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 client.9309 shutdown 2024-04-13T03:19:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 msgr2=0x561c09485a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40079d0 tx=0x7f0ef4012630 comp rx=0 tx=0).stop 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 shutdown_connections 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:6835/1427655446,v1:172.21.15.106:6837/1427655446] conn(0x561c094917d0 0x561c09485f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561c094923d0 0x561c094d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0eda7fc640 0 client.9309 ms_handle_reset on v2:172.21.15.106:6835/1427655446 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c0949cff0 0x561c09485a20 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 --2- 172.21.15.106:0/3501055201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c09491ff0 0x561c094d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 >> 172.21.15.106:0/3501055201 conn(0x561c094654d0 msgr2=0x561c094910c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 shutdown_connections 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 1 -- 172.21.15.106:0/3501055201 wait complete. 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.823+0000 7f0f04e45740 20 client.9309 trim_cache size 0 max 16384 2024-04-13T03:19:23.825 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:23.826 INFO:teuthology.orchestra.run.smithi106.stdout: "message": "", 2024-04-13T03:19:23.826 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0 2024-04-13T03:19:23.826 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 msgr2=0x7f0ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0eec036000 tx=0x7f0eec008030 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 msgr2=0x7f0efc153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000d7d0 tx=0x7f0ef000dc90 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 shutdown_connections 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0efc15d110 0x7f0efc0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0efc156b40 0x7f0efc153f20 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 --2- 172.21.15.106:0/2337779445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0efc06ebf0 0x7f0efc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 >> 172.21.15.106:0/2337779445 conn(0x7f0efc0cd6a0 msgr2=0x7f0efc15f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 shutdown_connections 2024-04-13T03:19:23.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.824+0000 7f0f02036640 1 -- 172.21.15.106:0/2337779445 wait complete. 2024-04-13T03:19:23.835 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:19:23.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.935+0000 7fdbe4280640 1 Processor -- start 2024-04-13T03:19:23.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.935+0000 7fdbe4280640 1 -- start start 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.935+0000 7fdbe4280640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdbdc1591a0 0x7fdbdc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe4280640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 0x7fdbdc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe4280640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc159580 con 0x7fdbdc0b32d0 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe4280640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc0b3ab0 con 0x7fdbdc1591a0 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe4280640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc0b3d90 con 0x7fdbdc0b5320 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe201c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdbdc1591a0 0x7fdbdc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe181b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdbdc0b32d0 0x7fdbdc0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57046/0 (socket says 172.21.15.106:57046) 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe181b640 1 -- 172.21.15.106:0/3360570518 learned_addr learned my addr 172.21.15.106:0/3360570518 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:23.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe281d640 1 --2- 172.21.15.106:0/3360570518 >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 0x7fdbdc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe201c640 1 -- 172.21.15.106:0/3360570518 >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 msgr2=0x7fdbdc06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe201c640 1 --2- 172.21.15.106:0/3360570518 >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 0x7fdbdc06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe201c640 1 -- 172.21.15.106:0/3360570518 >> v1:172.21.15.106:6789/0 conn(0x7fdbdc0b32d0 legacy=0x7fdbdc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe101a640 1 -- 172.21.15.106:0/3360570518 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221214115 0 0) 0x7fdbdc159580 con 0x7fdbdc0b32d0 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe201c640 1 -- 172.21.15.106:0/3360570518 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc0b4070 con 0x7fdbdc1591a0 2024-04-13T03:19:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.936+0000 7fdbe281d640 1 --2- 172.21.15.106:0/3360570518 >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 0x7fdbdc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe201c640 1 --2- 172.21.15.106:0/3360570518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc15bbf0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc06fe80 tx=0x7fdbd402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d6a79309d074b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe101a640 1 -- 172.21.15.106:0/3360570518 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4039070 con 0x7fdbdc1591a0 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe101a640 1 -- 172.21.15.106:0/3360570518 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd402cc80 con 0x7fdbdc1591a0 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe101a640 1 -- 172.21.15.106:0/3360570518 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4034660 con 0x7fdbdc1591a0 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 -- 172.21.15.106:0/3360570518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 msgr2=0x7fdbdc15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3360570518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc15bbf0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc06fe80 tx=0x7fdbd402bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 -- 172.21.15.106:0/3360570518 shutdown_connections 2024-04-13T03:19:23.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3360570518 >> v2:172.21.15.114:3300/0 conn(0x7fdbdc0b5320 0x7fdbdc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3360570518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc15bbf0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 -- 172.21.15.106:0/3360570518 >> 172.21.15.106:0/3360570518 conn(0x7fdbdc0d0340 msgr2=0x7fdbdc0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 -- 172.21.15.106:0/3360570518 shutdown_connections 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.937+0000 7fdbe4280640 1 -- 172.21.15.106:0/3360570518 wait complete. 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 Processor -- start 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 -- start start 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbdc0b32d0 0x7fdbdc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdbdc0b5320 0x7fdbdc15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b3720 con 0x7fdbdc1591a0 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b4630 con 0x7fdbdc0b32d0 2024-04-13T03:19:23.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe4280640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc06e310 con 0x7fdbdc0b5320 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe281d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe281d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54286/0 (socket says 172.21.15.106:54286) 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.939+0000 7fdbe281d640 1 -- 172.21.15.106:0/3604422202 learned_addr learned my addr 172.21.15.106:0/3604422202 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 -- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdbdc0b5320 msgr2=0x7fdbdc15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdbdc0b5320 0x7fdbdc15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 -- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbdc0b32d0 msgr2=0x7fdbdc150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe201c640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbdc0b32d0 0x7fdbdc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbdc0b32d0 0x7fdbdc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 -- 172.21.15.106:0/3604422202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc14fbb0 con 0x7fdbdc1591a0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe281d640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c820 tx=0x7fdbd800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8015070 con 0x7fdbdc1591a0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd80079d0 con 0x7fdbdc1591a0 2024-04-13T03:19:23.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc14fe90 con 0x7fdbdc1591a0 2024-04-13T03:19:23.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8007dc0 con 0x7fdbdc1591a0 2024-04-13T03:19:23.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.940+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc150450 con 0x7fdbdc1591a0 2024-04-13T03:19:23.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.941+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc150650 con 0x7fdbdc1591a0 2024-04-13T03:19:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.942+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbd8018070 con 0x7fdbdc1591a0 2024-04-13T03:19:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.942+0000 7fdbd2ffd640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.942+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fdbd808bc00 con 0x7fdbdc1591a0 2024-04-13T03:19:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.942+0000 7fdbe201c640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:23.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.942+0000 7fdbe201c640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 0x7fdbc006edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4036000 tx=0x7fdbd4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:23.946+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbdc150650 con 0x7fdbdc1591a0 2024-04-13T03:19:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.156+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdbdc068a10 con 0x7fdbdc1591a0 2024-04-13T03:19:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.157+0000 7fdbd2ffd640 1 -- 172.21.15.106:0/3604422202 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7fdbd8092080 con 0x7fdbdc1591a0 2024-04-13T03:19:24.158 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 msgr2=0x7fdbc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 0x7fdbc006edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4036000 tx=0x7fdbd4030000 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 msgr2=0x7fdbdc167300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c820 tx=0x7fdbd800cce0 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 shutdown_connections 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdbdc0b5320 0x7fdbdc15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbdc1591a0 0x7fdbdc167300 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 --2- 172.21.15.106:0/3604422202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbdc0b32d0 0x7fdbdc150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 >> 172.21.15.106:0/3604422202 conn(0x7fdbdc0d0340 msgr2=0x7fdbdc06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:24.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 shutdown_connections 2024-04-13T03:19:24.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24.159+0000 7fdbe4280640 1 -- 172.21.15.106:0/3604422202 wait complete. 2024-04-13T03:19:24.170 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:19:24.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:19:24.170 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:19:24.170 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:19:24.171 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:24.171 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:19:24.171 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:19:24.171 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:19:24.171 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:24.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.171 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:19:24.186 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:19:24.186 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:19:24.186 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:19:24.186 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:19:24.187 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41985sec preferred_lft 41985sec 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:24.188 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:19:24.189 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:19:24.189 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:24.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.189 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:19:24.252 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:24.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.254 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:19:24.309 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:19:24.309 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:19:24.309 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:19:24.309 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:19:24.309 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:19:24.310 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:19:24.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:19:24.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:24.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.445 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:24.459 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:19:24.459 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:19:24.460 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:24.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.460 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:19:24.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:24.526 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:24.581 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:24.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:24.582 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:24.636 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:19:24.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:24.721 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:24.721 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:24.731 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:24.731 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:24.731 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:19:24.731 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:19:24.732 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:19:24.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:19:24.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.734 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:24.759 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:19:24.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:19:24.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:24.854 INFO:teuthology.orchestra.run.smithi106.stdout:client9321 2024-04-13T03:19:24.855 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:24.855 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:24.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.871 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-13T03:19:24.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:24.929 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-13T03:19:25.009 INFO:teuthology.orchestra.run.smithi106.stderr:8+0 records in 2024-04-13T03:19:25.009 INFO:teuthology.orchestra.run.smithi106.stderr:8+0 records out 2024-04-13T03:19:25.010 INFO:teuthology.orchestra.run.smithi106.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0248779 s, 337 MB/s 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-13T03:19:25.011 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:25.115 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2024-04-13T03:19:25.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-13T03:19:25.124 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:25.229 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2024-04-13T03:19:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:25.239 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:25.239 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:25.294 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:25.294 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:25.350 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:25.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.490+0000 7fcd0ede9640 1 Processor -- start 2024-04-13T03:19:25.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.490+0000 7fcd0ede9640 1 -- start start 2024-04-13T03:19:25.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0ede9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd0806c8f0 0x7fcd081641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0ede9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd080aff20 con 0x7fcd080cadb0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0ede9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd080cbaa0 con 0x7fcd080b65f0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0ede9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcd0806fac0 con 0x7fcd0806c8f0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0cb85640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcd080cadb0 0x7fcd080cb190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57052/0 (socket says 172.21.15.106:57052) 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd0cb85640 1 -- 172.21.15.106:0/2016757233 learned_addr learned my addr 172.21.15.106:0/2016757233 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681507945 0 0) 0x7fcd080aff20 con 0x7fcd080cadb0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccf8003520 con 0x7fcd080cadb0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd07fff640 1 --2- 172.21.15.106:0/2016757233 >> v2:172.21.15.114:3300/0 conn(0x7fcd0806c8f0 0x7fcd081641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2597947523 0 0) 0x7fccf8003520 con 0x7fcd080cadb0 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 >> v2:172.21.15.114:3300/0 conn(0x7fcd0806c8f0 msgr2=0x7fcd081641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 --2- 172.21.15.106:0/2016757233 >> v2:172.21.15.114:3300/0 conn(0x7fcd0806c8f0 0x7fcd081641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 >> v1:172.21.15.114:6789/0 conn(0x7fcd080b65f0 legacy=0x7fcd08166720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:25.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd080afc40 con 0x7fcd080cadb0 2024-04-13T03:19:25.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fccfc001670 con 0x7fcd080cadb0 2024-04-13T03:19:25.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 >> v1:172.21.15.106:6789/0 conn(0x7fcd080cadb0 legacy=0x7fcd080cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fccf80039b0 0x7fccf8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fccf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf80051c0 0x7fccf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806fac0 con 0x7fccf8004800 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fccf800c0f0 con 0x7fccf80051c0 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fccf800c2f0 con 0x7fccf80039b0 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.491+0000 7fcd07fff640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fccf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd0cb85640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fccf80039b0 0x7fccf8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd0d386640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf80051c0 0x7fccf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 -- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fccf80039b0 msgr2=0x7fccf8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fccf80039b0 0x7fccf8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 -- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf80051c0 msgr2=0x7fccf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf80051c0 0x7fccf800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd0d386640 1 -- 172.21.15.106:0/2016757233 reap_dead start 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 -- 172.21.15.106:0/2016757233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf8003520 con 0x7fccf8004800 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.492+0000 7fcd07fff640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fccf8004c80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ba00 tx=0x7fccf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49393f4df5f28841 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4036020 con 0x7fccf8004800 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4032620 con 0x7fccf8004800 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd077fe640 1 -- 172.21.15.106:0/2016757233 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400b600 con 0x7fccf8004800 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 -- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 msgr2=0x7fccf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fccf8004c80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ba00 tx=0x7fccf8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 -- 172.21.15.106:0/2016757233 shutdown_connections 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/2016757233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fccf8004c80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 -- 172.21.15.106:0/2016757233 >> 172.21.15.106:0/2016757233 conn(0x7fcd080cd1e0 msgr2=0x7fcd080cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 -- 172.21.15.106:0/2016757233 shutdown_connections 2024-04-13T03:19:25.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.493+0000 7fcd0ede9640 1 -- 172.21.15.106:0/2016757233 wait complete. 2024-04-13T03:19:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.494+0000 7fcd0ede9640 1 Processor -- start 2024-04-13T03:19:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.494+0000 7fcd0ede9640 1 -- start start 2024-04-13T03:19:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd081488d0 0x7fcd081f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd0815a600 0x7fcd080b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806f750 con 0x7fccf8004800 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806f470 con 0x7fcd081488d0 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0ede9640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fcd080af6f0 con 0x7fcd0815a600 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0cb85640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0cb85640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54304/0 (socket says 172.21.15.106:54304) 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.495+0000 7fcd0cb85640 1 -- 172.21.15.106:0/3260889831 learned_addr learned my addr 172.21.15.106:0/3260889831 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd07fff640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd081488d0 0x7fcd081f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0d386640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd0815a600 0x7fcd080b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 -- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd0815a600 msgr2=0x7fcd080b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd0815a600 0x7fcd080b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 -- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd081488d0 msgr2=0x7fcd081f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd081488d0 0x7fcd081f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 -- 172.21.15.106:0/3260889831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd081f6e80 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0cb85640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007340 tx=0x7fccfc007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc01a070 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccfc00f8c0 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd081f68c0 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00fc90 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.496+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd08159100 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.497+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd080cc210 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.497+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccfc01d070 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.498+0000 7fcd057fa640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 0x7fccec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.498+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fccfc08b6a0 con 0x7fccf8004800 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.498+0000 7fcd07fff640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 0x7fccec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.498+0000 7fcd07fff640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 0x7fccec06edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fccf80057e0 tx=0x7fccf4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.502+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd080cc210 con 0x7fccf8004800 2024-04-13T03:19:25.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.677+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd08068a10 con 0x7fccf8004800 2024-04-13T03:19:25.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.678+0000 7fcd057fa640 1 -- 172.21.15.106:0/3260889831 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16988 (secure 0 0 0) 0x7fccfc0599d0 con 0x7fccf8004800 2024-04-13T03:19:25.679 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:25.679 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":121,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:16.039307+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 msgr2=0x7fccec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 0x7fccec06edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fccf80057e0 tx=0x7fccf4030000 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 msgr2=0x7fcd08159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007340 tx=0x7fccfc007800 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 shutdown_connections 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fccec06c8f0 0x7fccec06edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fcd0815a600 0x7fcd080b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd081488d0 0x7fcd081f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 --2- 172.21.15.106:0/3260889831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8004800 0x7fcd08159900 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 >> 172.21.15.106:0/3260889831 conn(0x7fcd080cd1e0 msgr2=0x7fcd0806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 shutdown_connections 2024-04-13T03:19:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.680+0000 7fcd0ede9640 1 -- 172.21.15.106:0/3260889831 wait complete. 2024-04-13T03:19:25.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:19:12.871192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:19:13.595223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:25.692 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:25.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.794+0000 7f1ddc877640 1 Processor -- start 2024-04-13T03:19:25.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.794+0000 7f1ddc877640 1 -- start start 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1ddc877640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1dd806bff0 0x7f1dd806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1ddc877640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd80d07c0 0x7f1dd806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1ddc877640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd80cef80 con 0x7f1dd815fb50 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1ddc877640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd80cf260 con 0x7f1dd806bff0 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1ddc877640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80cf540 con 0x7f1dd80d07c0 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1dd806bff0 0x7f1dd806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1dd806bff0 0x7f1dd806e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54308/0 (socket says 172.21.15.106:54308) 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 -- 172.21.15.106:0/382381017 learned_addr learned my addr 172.21.15.106:0/382381017 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dd6d9d640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd80d07c0 0x7f1dd806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 -- 172.21.15.106:0/382381017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd80d07c0 msgr2=0x7f1dd806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:25.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd80d07c0 0x7f1dd806ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 -- 172.21.15.106:0/382381017 >> v1:172.21.15.114:6789/0 conn(0x7f1dd815fb50 legacy=0x7f1dd806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.795+0000 7f1dcffff640 1 -- 172.21.15.106:0/382381017 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80cf820 con 0x7f1dd806bff0 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.796+0000 7f1dcffff640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd806bff0 0x7f1dd806e6b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009870 tx=0x7f1dc002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33aaacab45eb940 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.796+0000 7f1dd5d9b640 1 -- 172.21.15.106:0/382381017 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035020 con 0x7f1dd806bff0 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.796+0000 7f1dd5d9b640 1 -- 172.21.15.106:0/382381017 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc0031640 con 0x7f1dd806bff0 2024-04-13T03:19:25.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.796+0000 7f1dd5d9b640 1 -- 172.21.15.106:0/382381017 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc002d040 con 0x7f1dd806bff0 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 -- 172.21.15.106:0/382381017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd806bff0 msgr2=0x7f1dd806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd806bff0 0x7f1dd806e6b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009870 tx=0x7f1dc002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 -- 172.21.15.106:0/382381017 shutdown_connections 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd80d07c0 0x7f1dd806ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 --2- 172.21.15.106:0/382381017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd806bff0 0x7f1dd806e6b0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 -- 172.21.15.106:0/382381017 >> 172.21.15.106:0/382381017 conn(0x7f1dd80cd6a0 msgr2=0x7f1dd80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:25.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 -- 172.21.15.106:0/382381017 shutdown_connections 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.797+0000 7f1ddc877640 1 -- 172.21.15.106:0/382381017 wait complete. 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.798+0000 7f1ddc877640 1 Processor -- start 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.798+0000 7f1ddc877640 1 -- start start 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd806bff0 0x7f1dd8150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd80d07c0 0x7f1dd81f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80ced20 con 0x7f1dd815fb50 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80ceba0 con 0x7f1dd80d07c0 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd815ef10 con 0x7f1dd806bff0 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54320/0 (socket says 172.21.15.106:54320) 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 -- 172.21.15.106:0/2089662581 learned_addr learned my addr 172.21.15.106:0/2089662581 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:25.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dcffff640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd80d07c0 0x7f1dd81f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd659c640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd806bff0 0x7f1dd8150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 -- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd806bff0 msgr2=0x7f1dd8150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd806bff0 0x7f1dd8150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 -- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd80d07c0 msgr2=0x7f1dd81f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd80d07c0 0x7f1dd81f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 -- 172.21.15.106:0/2089662581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd814e510 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dd6d9d640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000efc0 tx=0x7f1dd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0018020 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0014480 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd81f4c20 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.799+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0014870 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.800+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd814e7b0 con 0x7f1dd815fb50 2024-04-13T03:19:25.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.801+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd81f5010 con 0x7f1dd815fb50 2024-04-13T03:19:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.801+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dd0005d10 con 0x7f1dd815fb50 2024-04-13T03:19:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.801+0000 7f1dceffd640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 0x7f1db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.801+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f1dd008ca20 con 0x7f1dd815fb50 2024-04-13T03:19:25.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.801+0000 7f1dd659c640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 0x7f1db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:25.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.802+0000 7f1dd659c640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 0x7f1db806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80016e0 tx=0x7f1dc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:25.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.806+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1dd0022030 con 0x7f1dd815fb50 2024-04-13T03:19:25.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.981+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1dd8068a10 con 0x7f1dd815fb50 2024-04-13T03:19:25.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.982+0000 7f1dceffd640 1 -- 172.21.15.106:0/2089662581 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f1dd0010040 con 0x7f1dd815fb50 2024-04-13T03:19:25.983 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 msgr2=0x7f1db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 0x7f1db806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80016e0 tx=0x7f1dc8009040 comp rx=0 tx=0).stop 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 msgr2=0x7f1dd81f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000efc0 tx=0x7f1dd000c490 comp rx=0 tx=0).stop 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 shutdown_connections 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f1db806c8f0 0x7f1db806edb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dd815fb50 0x7f1dd81f5520 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd80d07c0 0x7f1dd81f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 --2- 172.21.15.106:0/2089662581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1dd806bff0 0x7f1dd8150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:25.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 >> 172.21.15.106:0/2089662581 conn(0x7f1dd80cd6a0 msgr2=0x7f1dd80677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:25.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 shutdown_connections 2024-04-13T03:19:25.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:25.984+0000 7f1ddc877640 1 -- 172.21.15.106:0/2089662581 wait complete. 2024-04-13T03:19:25.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:26.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 Processor -- start 2024-04-13T03:19:26.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 -- start start 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06ec15e9b0 0x7f06ec0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 0x7f06ec0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ec0cfd20 con 0x7f06ec0b3d60 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f06ec162af0 con 0x7f06ec15e9b0 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.098+0000 7f06f20de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f06ec162dd0 con 0x7f06ec15ed90 2024-04-13T03:19:26.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06ec15e9b0 0x7f06ec0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06ec15e9b0 0x7f06ec0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54322/0 (socket says 172.21.15.106:54322) 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 -- 172.21.15.106:0/160792162 learned_addr learned my addr 172.21.15.106:0/160792162 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06ebfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f06ec0b3d60 0x7f06ec0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57056/0 (socket says 172.21.15.106:57056) 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eaffd640 1 --2- 172.21.15.106:0/160792162 >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 0x7f06ec0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 -- 172.21.15.106:0/160792162 >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 msgr2=0x7f06ec0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 --2- 172.21.15.106:0/160792162 >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 0x7f06ec0d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 -- 172.21.15.106:0/160792162 >> v1:172.21.15.106:6789/0 conn(0x7f06ec0b3d60 legacy=0x7f06ec0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eb7fe640 1 -- 172.21.15.106:0/160792162 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec1630b0 con 0x7f06ec15e9b0 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06eaffd640 1 --2- 172.21.15.106:0/160792162 >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 0x7f06ec0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.099+0000 7f06ea7fc640 1 -- 172.21.15.106:0/160792162 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801310446 0 0) 0x7f06ec0cfd20 con 0x7f06ec0b3d60 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06eb7fe640 1 --2- 172.21.15.106:0/160792162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec15e9b0 0x7f06ec0d2300 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f06ec08ebd0 tx=0x7f06dc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e078638efdc26b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06ea7fc640 1 -- 172.21.15.106:0/160792162 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc02f850 con 0x7f06ec15e9b0 2024-04-13T03:19:26.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06ea7fc640 1 -- 172.21.15.106:0/160792162 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc02f9f0 con 0x7f06ec15e9b0 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06ea7fc640 1 -- 172.21.15.106:0/160792162 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc02fdc0 con 0x7f06ec15e9b0 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 -- 172.21.15.106:0/160792162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec15e9b0 msgr2=0x7f06ec0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 --2- 172.21.15.106:0/160792162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec15e9b0 0x7f06ec0d2300 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f06ec08ebd0 tx=0x7f06dc02ba30 comp rx=0 tx=0).stop 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 -- 172.21.15.106:0/160792162 shutdown_connections 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 --2- 172.21.15.106:0/160792162 >> v2:172.21.15.114:3300/0 conn(0x7f06ec15ed90 0x7f06ec0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 --2- 172.21.15.106:0/160792162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec15e9b0 0x7f06ec0d2300 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 -- 172.21.15.106:0/160792162 >> 172.21.15.106:0/160792162 conn(0x7f06ec08bf10 msgr2=0x7f06ec0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 -- 172.21.15.106:0/160792162 shutdown_connections 2024-04-13T03:19:26.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.100+0000 7f06f20de640 1 -- 172.21.15.106:0/160792162 wait complete. 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.101+0000 7f06f20de640 1 Processor -- start 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 -- start start 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06ec0b3d60 0x7f06ec0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec15e9b0 0x7f06ec1613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec162800 con 0x7f06ec0cc850 2024-04-13T03:19:26.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec162680 con 0x7f06ec15e9b0 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06f20de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec06eb20 con 0x7f06ec0b3d60 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.102+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54324/0 (socket says 172.21.15.106:54324) 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 -- 172.21.15.106:0/3380135137 learned_addr learned my addr 172.21.15.106:0/3380135137 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 -- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06ec0b3d60 msgr2=0x7f06ec0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06ec0b3d60 0x7f06ec0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06eaffd640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec15e9b0 0x7f06ec1613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 -- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec15e9b0 msgr2=0x7f06ec1613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec15e9b0 0x7f06ec1613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 -- 172.21.15.106:0/3380135137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec161df0 con 0x7f06ec0cc850 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06ebfff640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c7a0 tx=0x7f06e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4018070 con 0x7f06ec0cc850 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e400a640 con 0x7f06ec0cc850 2024-04-13T03:19:26.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec0ccdf0 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e400aab0 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.103+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec0cd3b0 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.104+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ec068a10 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.105+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06e401b070 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.105+0000 7f06d3fff640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 0x7f06d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.105+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f06e408b2a0 con 0x7f06ec0cc850 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.105+0000 7f06eb7fe640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 0x7f06d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.105+0000 7f06eb7fe640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 0x7f06d806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06dc036000 tx=0x7f06dc039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.109+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06e4059540 con 0x7f06ec0cc850 2024-04-13T03:19:26.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.282+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06ec0d53a0 con 0x7f06ec0cc850 2024-04-13T03:19:26.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.283+0000 7f06d3fff640 1 -- 172.21.15.106:0/3380135137 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f06e4058e40 con 0x7f06ec0cc850 2024-04-13T03:19:26.284 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 msgr2=0x7f06d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 0x7f06d806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06dc036000 tx=0x7f06dc039040 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 msgr2=0x7f06ec0b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c7a0 tx=0x7f06e400cc60 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 shutdown_connections 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f06d806c8f0 0x7f06d806edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06ec0cc850 0x7f06ec0b6440 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec15e9b0 0x7f06ec1613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 --2- 172.21.15.106:0/3380135137 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f06ec0b3d60 0x7f06ec0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 >> 172.21.15.106:0/3380135137 conn(0x7f06ec08bf10 msgr2=0x7f06ec0b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 shutdown_connections 2024-04-13T03:19:26.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.285+0000 7f06f20de640 1 -- 172.21.15.106:0/3380135137 wait complete. 2024-04-13T03:19:26.296 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.400+0000 7f678f636640 1 Processor -- start 2024-04-13T03:19:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.400+0000 7f678f636640 1 -- start start 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678f636640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f678806f200 0x7f678806f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678f636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67880b5090 con 0x7f67880af800 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678f636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67880b3ff0 con 0x7f678806b8e0 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678f636640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f67880b42d0 con 0x7f678806f200 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678cbd1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f678806f200 0x7f678806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678d3d2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f67880af800 0x7f678815bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57062/0 (socket says 172.21.15.106:57062) 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678d3d2640 1 -- 172.21.15.106:0/366421986 learned_addr learned my addr 172.21.15.106:0/366421986 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678cbd1640 1 -- 172.21.15.106:0/366421986 >> v1:172.21.15.114:6789/0 conn(0x7f678806b8e0 legacy=0x7f678806bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678cbd1640 1 -- 172.21.15.106:0/366421986 >> v1:172.21.15.106:6789/0 conn(0x7f67880af800 legacy=0x7f678815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.401+0000 7f678cbd1640 1 -- 172.21.15.106:0/366421986 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67880b45b0 con 0x7f678806f200 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678cbd1640 1 --2- 172.21.15.106:0/366421986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806f200 0x7f678806f690 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f677800a940 tx=0x7f677802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a47c089f044f48f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f677ffff640 1 -- 172.21.15.106:0/366421986 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803a070 con 0x7f678806f200 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f677ffff640 1 -- 172.21.15.106:0/366421986 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677802db40 con 0x7f678806f200 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f677ffff640 1 -- 172.21.15.106:0/366421986 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800b040 con 0x7f678806f200 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678f636640 1 -- 172.21.15.106:0/366421986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806f200 msgr2=0x7f678806f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678f636640 1 --2- 172.21.15.106:0/366421986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806f200 0x7f678806f690 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f677800a940 tx=0x7f677802cd90 comp rx=0 tx=0).stop 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678f636640 1 -- 172.21.15.106:0/366421986 shutdown_connections 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678f636640 1 --2- 172.21.15.106:0/366421986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806f200 0x7f678806f690 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.402+0000 7f678f636640 1 -- 172.21.15.106:0/366421986 >> 172.21.15.106:0/366421986 conn(0x7f67880b2600 msgr2=0x7f67880d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.403+0000 7f678f636640 1 -- 172.21.15.106:0/366421986 shutdown_connections 2024-04-13T03:19:26.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.403+0000 7f678f636640 1 -- 172.21.15.106:0/366421986 wait complete. 2024-04-13T03:19:26.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 Processor -- start 2024-04-13T03:19:26.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 -- start start 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 0x7f6788150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 0x7f678814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67880b3cc0 con 0x7f678806b8e0 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67880b4b70 con 0x7f67880af800 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678f636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f678806fef0 con 0x7f678806f200 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678d3d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678d3d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54338/0 (socket says 172.21.15.106:54338) 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.404+0000 7f678d3d2640 1 -- 172.21.15.106:0/2753775118 learned_addr learned my addr 172.21.15.106:0/2753775118 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678cbd1640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 0x7f6788150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678dbd3640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 0x7f678814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 -- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 msgr2=0x7f6788150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 0x7f6788150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 -- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 msgr2=0x7f678814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 0x7f678814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 -- 172.21.15.106:0/2753775118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6788143300 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678cbd1640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 0x7f6788150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678dbd3640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 0x7f678814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678d3d2640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f678400cb40 tx=0x7f678400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784017070 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678400aca0 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678814f960 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784012620 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.405+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678814fc00 con 0x7f678806b8e0 2024-04-13T03:19:26.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.406+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788068a10 con 0x7f678806b8e0 2024-04-13T03:19:26.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.406+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f678401a070 con 0x7f678806b8e0 2024-04-13T03:19:26.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.407+0000 7f677dffb640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 0x7f676c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.407+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f678408ba00 con 0x7f678806b8e0 2024-04-13T03:19:26.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.407+0000 7f678cbd1640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 0x7f676c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.407+0000 7f678cbd1640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 0x7f676c06edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6778037000 tx=0x7f677800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.411+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6788143300 con 0x7f678806b8e0 2024-04-13T03:19:26.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.588+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67880b1be0 con 0x7f678806b8e0 2024-04-13T03:19:26.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.588+0000 7f677dffb640 1 -- 172.21.15.106:0/2753775118 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f6784059d30 con 0x7f678806b8e0 2024-04-13T03:19:26.589 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 msgr2=0x7f676c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 0x7f676c06edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6778037000 tx=0x7f677800c040 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 msgr2=0x7f6788142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f678400cb40 tx=0x7f678400a060 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 shutdown_connections 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.590+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f676c06c8f0 0x7f676c06edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f678806f200 0x7f6788150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67880af800 0x7f678814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 --2- 172.21.15.106:0/2753775118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f678806b8e0 0x7f6788142dc0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 >> 172.21.15.106:0/2753775118 conn(0x7f67880b2600 msgr2=0x7f678806c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 shutdown_connections 2024-04-13T03:19:26.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.591+0000 7f678f636640 1 -- 172.21.15.106:0/2753775118 wait complete. 2024-04-13T03:19:26.602 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:19:26.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:19:26.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:19:26.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.704+0000 7fdf51bcf640 1 Processor -- start 2024-04-13T03:19:26.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.704+0000 7fdf51bcf640 1 -- start start 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf51bcf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf51bcf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c0cfd20 con 0x7fdf4c15ed90 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf51bcf640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c162af0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf51bcf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdf4c162dd0 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf4b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf4b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54350/0 (socket says 172.21.15.106:54350) 2024-04-13T03:19:26.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf4affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdf4c15ed90 0x7fdf4c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57064/0 (socket says 172.21.15.106:57064) 2024-04-13T03:19:26.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.705+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/460383266 learned_addr learned my addr 172.21.15.106:0/460383266 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:26.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/460383266 >> v1:172.21.15.114:6789/0 conn(0x7fdf4c0b3d60 legacy=0x7fdf4c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:26.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/460383266 >> v1:172.21.15.106:6789/0 conn(0x7fdf4c15ed90 legacy=0x7fdf4c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/460383266 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c1630b0 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4b7fe640 1 --2- 172.21.15.106:0/460383266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea4b591a489a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4a7fc640 1 -- 172.21.15.106:0/460383266 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c02f840 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4a7fc640 1 -- 172.21.15.106:0/460383266 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c02f9e0 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.706+0000 7fdf4a7fc640 1 -- 172.21.15.106:0/460383266 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c02fdb0 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 -- 172.21.15.106:0/460383266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c15e9b0 msgr2=0x7fdf4c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/460383266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02ba20 comp rx=0 tx=0).stop 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 -- 172.21.15.106:0/460383266 shutdown_connections 2024-04-13T03:19:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/460383266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c15e9b0 0x7fdf4c0d2300 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 -- 172.21.15.106:0/460383266 >> 172.21.15.106:0/460383266 conn(0x7fdf4c08bf10 msgr2=0x7fdf4c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 -- 172.21.15.106:0/460383266 shutdown_connections 2024-04-13T03:19:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.707+0000 7fdf51bcf640 1 -- 172.21.15.106:0/460383266 wait complete. 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.708+0000 7fdf51bcf640 1 Processor -- start 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.708+0000 7fdf51bcf640 1 -- start start 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdf4c15e9b0 0x7fdf4c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf4c15f180 0x7fdf4c15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c162800 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c162680 con 0x7fdf4c15f180 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf51bcf640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0d4d00 con 0x7fdf4c15e9b0 2024-04-13T03:19:26.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54358/0 (socket says 172.21.15.106:54358) 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/525799817 learned_addr learned my addr 172.21.15.106:0/525799817 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdf4c15e9b0 msgr2=0x7fdf4c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdf4c15e9b0 0x7fdf4c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf4c15f180 msgr2=0x7fdf4c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf4c15f180 0x7fdf4c15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 -- 172.21.15.106:0/525799817 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c0c9ed0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.709+0000 7fdf4b7fe640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.710+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c041400 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.710+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c046070 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.710+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c0caeb0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.710+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0417e0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.710+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c0b6f50 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.711+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c068a10 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.711+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf3c040070 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.711+0000 7fdf33fff640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:26.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.712+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fdf3c0ae5e0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.712+0000 7fdf4affd640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:26.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.712+0000 7fdf4affd640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 0x7fdf3806edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001020 tx=0x7fdf40007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:26.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.716+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf3c033030 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.904+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf4c067fb0 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.904+0000 7fdf33fff640 1 -- 172.21.15.106:0/525799817 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7fdf3c07c880 con 0x7fdf4c0b3d60 2024-04-13T03:19:26.905 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 msgr2=0x7fdf3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 0x7fdf3806edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001020 tx=0x7fdf40007930 comp rx=0 tx=0).stop 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 msgr2=0x7fdf4c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c004730 comp rx=0 tx=0).stop 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 shutdown_connections 2024-04-13T03:19:26.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdf4c15e9b0 0x7fdf4c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf4c15f180 0x7fdf4c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 --2- 172.21.15.106:0/525799817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf4c0b3d60 0x7fdf4c0ca970 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 >> 172.21.15.106:0/525799817 conn(0x7fdf4c08bf10 msgr2=0x7fdf4c0b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 shutdown_connections 2024-04-13T03:19:26.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:26.906+0000 7fdf51bcf640 1 -- 172.21.15.106:0/525799817 wait complete. 2024-04-13T03:19:26.918 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:19:26.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:26.918 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:27.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:19:27.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:27.041 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:19:27.061 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:27.062 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:27.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:27.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:19:27.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:27.117 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:19:27.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:19:27.171 INFO:teuthology.orchestra.run.smithi106.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-13T03:19:27.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:19:27.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 Processor -- start 2024-04-13T03:19:27.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 -- start start 2024-04-13T03:19:27.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34d015e930 0x7f34d00b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d00d09f0 0x7f34d00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d006f200 con 0x7f34d006ed70 2024-04-13T03:19:27.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f34d006e280 con 0x7f34d015e930 2024-04-13T03:19:27.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.312+0000 7f34d59f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34d006e560 con 0x7f34d00d09f0 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ceffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f34d015e930 0x7f34d00b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ce7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f34d006ed70 0x7f34d00d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57074/0 (socket says 172.21.15.106:57074) 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ce7fc640 1 -- 172.21.15.106:0/1155919095 learned_addr learned my addr 172.21.15.106:0/1155919095 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34cf7fe640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d00d09f0 0x7f34d00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ceffd640 1 -- 172.21.15.106:0/1155919095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d00d09f0 msgr2=0x7f34d00cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ceffd640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d00d09f0 0x7f34d00cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ceffd640 1 -- 172.21.15.106:0/1155919095 >> v1:172.21.15.106:6789/0 conn(0x7f34d006ed70 legacy=0x7f34d00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34cdffb640 1 -- 172.21.15.106:0/1155919095 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136837129 0 0) 0x7f34d006f200 con 0x7f34d006ed70 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.313+0000 7f34ceffd640 1 -- 172.21.15.106:0/1155919095 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00cccf0 con 0x7f34d015e930 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34ceffd640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d015e930 0x7f34d00b5350 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f34c00097a0 tx=0x7f34c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be9be966e19e25a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34cdffb640 1 -- 172.21.15.106:0/1155919095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0035020 con 0x7f34d015e930 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34cdffb640 1 -- 172.21.15.106:0/1155919095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0031830 con 0x7f34d015e930 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34cdffb640 1 -- 172.21.15.106:0/1155919095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0031c00 con 0x7f34d015e930 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 -- 172.21.15.106:0/1155919095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d015e930 msgr2=0x7f34d00b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d015e930 0x7f34d00b5350 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f34c00097a0 tx=0x7f34c002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 -- 172.21.15.106:0/1155919095 shutdown_connections 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d00d09f0 0x7f34d00cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 --2- 172.21.15.106:0/1155919095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d015e930 0x7f34d00b5350 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 -- 172.21.15.106:0/1155919095 >> 172.21.15.106:0/1155919095 conn(0x7f34d00cef80 msgr2=0x7f34d00cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 -- 172.21.15.106:0/1155919095 shutdown_connections 2024-04-13T03:19:27.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.314+0000 7f34d59f7640 1 -- 172.21.15.106:0/1155919095 wait complete. 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.316+0000 7f34d59f7640 1 Processor -- start 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.316+0000 7f34d59f7640 1 -- start start 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34d006ed70 0x7f34d00ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d015e930 0x7f34d014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00ce830 con 0x7f34d00d09f0 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00cd2b0 con 0x7f34d006ed70 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f34d00ce9b0 con 0x7f34d015e930 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54366/0 (socket says 172.21.15.106:54366) 2024-04-13T03:19:27.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 -- 172.21.15.106:0/4122371821 learned_addr learned my addr 172.21.15.106:0/4122371821 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34cf7fe640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d015e930 0x7f34d014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ceffd640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34d006ed70 0x7f34d00ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 -- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d015e930 msgr2=0x7f34d014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d015e930 0x7f34d014d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 -- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34d006ed70 msgr2=0x7f34d00ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34d006ed70 0x7f34d00ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 -- 172.21.15.106:0/4122371821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00cbff0 con 0x7f34d00d09f0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34ce7fc640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c4d0 tx=0x7f34c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4018070 con 0x7f34d00d09f0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c4007500 con 0x7f34d00d09f0 2024-04-13T03:19:27.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d00c9670 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004a20 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.317+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d00c9c30 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.318+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d0068a10 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.319+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34c401b070 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.319+0000 7f34b7fff640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.319+0000 7f34ceffd640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.319+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f34c408aef0 con 0x7f34d00d09f0 2024-04-13T03:19:27.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.320+0000 7f34ceffd640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 0x7f34bc06edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34c0036000 tx=0x7f34c003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.324+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34c4058bd0 con 0x7f34d00d09f0 2024-04-13T03:19:27.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.536+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34d006fb80 con 0x7f34d00d09f0 2024-04-13T03:19:27.537 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:19:27.536+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:19:27.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.540+0000 7f34b7fff640 1 -- 172.21.15.106:0/4122371821 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) ==== 161+0+0 (secure 0 0 0) 0x7f34c405ced0 con 0x7f34d00d09f0 2024-04-13T03:19:27.540 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 msgr2=0x7f34bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 0x7f34bc06edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34c0036000 tx=0x7f34c003c040 comp rx=0 tx=0).stop 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 msgr2=0x7f34d01609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c4d0 tx=0x7f34c400c990 comp rx=0 tx=0).stop 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 shutdown_connections 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f34d015e930 0x7f34d014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34d00d09f0 0x7f34d01609f0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 --2- 172.21.15.106:0/4122371821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34d006ed70 0x7f34d00ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 >> 172.21.15.106:0/4122371821 conn(0x7f34d00cef80 msgr2=0x7f34d00cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:27.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 shutdown_connections 2024-04-13T03:19:27.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.542+0000 7f34d59f7640 1 -- 172.21.15.106:0/4122371821 wait complete. 2024-04-13T03:19:27.550 INFO:tasks.ceph.mds.c.smithi106.stderr: -5302> 2024-04-13T03:19:21.277+0000 7f2ca2ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:27.556 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:27.559 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:27.579 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:19:27.607 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-13T03:19:27.607 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-13T03:19:27.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:27.609 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-13T03:19:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.613 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:27.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-13T03:19:27.615 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:27.616 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-13T03:19:27.617 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:19:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = False 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.619 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-13T03:19:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-13T03:19:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-13T03:19:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-13T03:19:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-13T03:19:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-04-13T03:19:27.630 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = True 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = False 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.634 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:27.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:27.724 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-13T03:19:27.724 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-13T03:19:27.724 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:27.724 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:27.725 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-13T03:19:27.726 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-13T03:19:27.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.729 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:27.730 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:27.731 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.732 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-13T03:19:27.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-13T03:19:27.734 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-13T03:19:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = False 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = False 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-13T03:19:27.736 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.737 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:27.738 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-13T03:19:27.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-13T03:19:27.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-04-13T03:19:27.742 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.743 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-13T03:19:27.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-04-13T03:19:27.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.746 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = False 2024-04-13T03:19:27.747 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-04-13T03:19:27.748 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = False 2024-04-13T03:19:27.748 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:27.748 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:27.748 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:27.749 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-13T03:19:27.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.831+0000 7f462536df40 1 Processor -- start 2024-04-13T03:19:27.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.831+0000 7f462536df40 1 -- start start 2024-04-13T03:19:27.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 0x561169d74080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169d71dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561169cd9d00 con 0x561169e05ae0 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561169cddd40 con 0x561169e04930 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462536df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561169d316f0 con 0x561169deb560 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54406/0 (socket says 172.21.15.106:54406) 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 -- 172.21.15.106:0/1657564588 learned_addr learned my addr 172.21.15.106:0/1657564588 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462344c640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169d71dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f4622c4b640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 0x561169d74080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 -- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 msgr2=0x561169d71dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169d71dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 -- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 msgr2=0x561169d74080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:27.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 0x561169d74080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 -- 172.21.15.106:0/1657564588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561169e05710 con 0x561169e05ae0 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462344c640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169d71dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f4622c4b640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 0x561169d74080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f462244a640 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f46180097b0 tx=0x7f4618005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d2e335833b6d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f4621448640 1 -- 172.21.15.106:0/1657564588 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618019020 con 0x561169e05ae0 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f4621448640 1 -- 172.21.15.106:0/1657564588 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618015880 con 0x561169e05ae0 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.832+0000 7f4621448640 1 -- 172.21.15.106:0/1657564588 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618015c50 con 0x561169e05ae0 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- 172.21.15.106:0/1657564588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 msgr2=0x561169d71890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f46180097b0 tx=0x7f4618005a00 comp rx=0 tx=0).stop 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- 172.21.15.106:0/1657564588 shutdown_connections 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169d71dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e05ae0 0x561169d71890 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 --2- 172.21.15.106:0/1657564588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e04930 0x561169d74080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- 172.21.15.106:0/1657564588 >> 172.21.15.106:0/1657564588 conn(0x561169d5cae0 msgr2=0x561169d30100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- 172.21.15.106:0/1657564588 shutdown_connections 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- 172.21.15.106:0/1657564588 wait complete. 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 Processor -- start 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.833+0000 7f462536df40 1 -- start start 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169de8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 0x561169e01490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e05ae0 0x561169d79b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561169e04d10 con 0x561169e04930 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561169e05090 con 0x561169e05ae0 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462536df40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561169d5b890 con 0x561169deb560 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 0x561169e01490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 0x561169e01490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54422/0 (socket says 172.21.15.106:54422) 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 -- 172.21.15.106:0/1615012871 learned_addr learned my addr 172.21.15.106:0/1615012871 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:27.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f4622c4b640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169de8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462344c640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e05ae0 0x561169d79b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 -- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 msgr2=0x561169de8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561169deb560 0x561169de8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 -- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e05ae0 msgr2=0x561169d79b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561169e05ae0 0x561169d79b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.834+0000 7f462244a640 1 -- 172.21.15.106:0/1615012871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561169d5a940 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f462244a640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 0x561169e01490 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f4618005a30 tx=0x7f4618002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f460bfff640 1 -- 172.21.15.106:0/1615012871 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618019040 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f460bfff640 1 -- 172.21.15.106:0/1615012871 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618027680 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561169d5abe0 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f460bfff640 1 -- 172.21.15.106:0/1615012871 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618027a70 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.835+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561169e02270 con 0x561169e04930 2024-04-13T03:19:27.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.836+0000 7f460bfff640 1 -- 172.21.15.106:0/1615012871 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f461801c020 con 0x561169e04930 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.836+0000 7f460bfff640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f461006b2d0 0x7f461006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f460bfff640 1 -- 172.21.15.106:0/1615012871 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f461808c450 con 0x561169e04930 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f4622c4b640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f461006b2d0 0x7f461006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f462536df40 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561169d7a810 0x561169e02b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x561169e019d0 con 0x561169d7a810 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f462344c640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561169d7a810 0x561169e02b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f4622c4b640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f461006b2d0 0x7f461006d790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f461c0013d0 tx=0x7f461c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.837+0000 7f462344c640 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561169d7a810 0x561169e02b50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:27.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462344c640 1 -- 172.21.15.106:0/1615012871 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v122'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46140527c0 con 0x561169d7a810 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561169d7a810 msgr2=0x561169e02b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561169d7a810 0x561169e02b50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f461006b2d0 msgr2=0x7f461006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f461006b2d0 0x7f461006d790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f461c0013d0 tx=0x7f461c008040 comp rx=0 tx=0).stop 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 msgr2=0x561169e01490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 --2- 172.21.15.106:0/1615012871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561169e04930 0x561169e01490 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f4618005a30 tx=0x7f4618002fb0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462344c640 1 -- 172.21.15.106:0/1615012871 reap_dead start 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.838+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 shutdown_connections 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.839+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 >> 172.21.15.106:0/1615012871 conn(0x561169d5cae0 msgr2=0x561169d74010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.839+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 shutdown_connections 2024-04-13T03:19:27.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.839+0000 7f462536df40 1 -- 172.21.15.106:0/1615012871 wait complete. 2024-04-13T03:19:27.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:19:27.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.987+0000 7fb1b81aa640 1 Processor -- start 2024-04-13T03:19:27.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.987+0000 7fb1b81aa640 1 -- start start 2024-04-13T03:19:27.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b81aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb1b0156b40 0x7fb1b006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b81aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b81aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b015d690 con 0x7fb1b006ebf0 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b81aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b006b8e0 con 0x7fb1b0156b40 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b81aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b006bea0 con 0x7fb1b006c330 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5745640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb1b006ebf0 0x7fb1b015cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57086/0 (socket says 172.21.15.106:57086) 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5745640 1 -- 172.21.15.106:0/3343352357 learned_addr learned my addr 172.21.15.106:0/3343352357 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:27.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5f46640 1 --2- 172.21.15.106:0/3343352357 >> v2:172.21.15.106:3300/0 conn(0x7fb1b0156b40 0x7fb1b006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b6747640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5f46640 1 -- 172.21.15.106:0/3343352357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 msgr2=0x7fb1b006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5f46640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5f46640 1 -- 172.21.15.106:0/3343352357 >> v1:172.21.15.106:6789/0 conn(0x7fb1b006ebf0 legacy=0x7fb1b015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b5f46640 1 -- 172.21.15.106:0/3343352357 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b015d970 con 0x7fb1b0156b40 2024-04-13T03:19:27.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.988+0000 7fb1b4f44640 1 -- 172.21.15.106:0/3343352357 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2868865436 0 0) 0x7fb1b015d690 con 0x7fb1b006ebf0 2024-04-13T03:19:28.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.989+0000 7fb1b5f46640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b0156b40 0x7fb1b006e6b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0097a0 tx=0x7fb1ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f030a46855ad17 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.989+0000 7fb1b4f44640 1 -- 172.21.15.106:0/3343352357 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac035020 con 0x7fb1b0156b40 2024-04-13T03:19:28.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.989+0000 7fb1b4f44640 1 -- 172.21.15.106:0/3343352357 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ac031830 con 0x7fb1b0156b40 2024-04-13T03:19:28.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.989+0000 7fb1b4f44640 1 -- 172.21.15.106:0/3343352357 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac031c00 con 0x7fb1b0156b40 2024-04-13T03:19:28.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3343352357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b0156b40 msgr2=0x7fb1b006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b0156b40 0x7fb1b006e6b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0097a0 tx=0x7fb1ac02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3343352357 shutdown_connections 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3343352357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b0156b40 0x7fb1b006e6b0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3343352357 >> 172.21.15.106:0/3343352357 conn(0x7fb1b00cd6a0 msgr2=0x7fb1b00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:28.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3343352357 shutdown_connections 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.990+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3343352357 wait complete. 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.991+0000 7fb1b81aa640 1 Processor -- start 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.991+0000 7fb1b81aa640 1 -- start start 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b00b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b0156b40 0x7fb1b01e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b015d2e0 con 0x7fb1b006ebf0 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0156f20 con 0x7fb1b0156b40 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b00b3f60 con 0x7fb1b006c330 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54446/0 (socket says 172.21.15.106:54446) 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 -- 172.21.15.106:0/3459274487 learned_addr learned my addr 172.21.15.106:0/3459274487 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 -- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 msgr2=0x7fb1b00b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b00b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 -- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b0156b40 msgr2=0x7fb1b01e83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b0156b40 0x7fb1b01e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 -- 172.21.15.106:0/3459274487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b01e7030 con 0x7fb1b006ebf0 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b5745640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c4b0 tx=0x7fb1a000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00074e0 con 0x7fb1b006ebf0 2024-04-13T03:19:28.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a00047e0 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0004bd0 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.992+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b01e7310 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.993+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01e78d0 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.993+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b0068a10 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.994+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1a0011070 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.994+0000 7fb1a67fc640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.995+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fb1a008bf40 con 0x7fb1b006ebf0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.995+0000 7fb1b5f46640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.995+0000 7fb1b5f46640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 0x7fb19406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac036000 tx=0x7fb1ac030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:27.998+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1a000e280 con 0x7fb1b006ebf0 2024-04-13T03:19:28.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.210+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb1b00ce300 con 0x7fb1b006ebf0 2024-04-13T03:19:28.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.541+0000 7fb1a67fc640 1 -- 172.21.15.106:0/3459274487 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v88) ==== 162+0+0 (secure 0 0 0) 0x7fb1a005a330 con 0x7fb1b006ebf0 2024-04-13T03:19:28.542 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 msgr2=0x7fb19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 0x7fb19406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac036000 tx=0x7fb1ac030000 comp rx=0 tx=0).stop 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 msgr2=0x7fb1b00b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c4b0 tx=0x7fb1a000c970 comp rx=0 tx=0).stop 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 shutdown_connections 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b0156b40 0x7fb1b01e83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1b006ebf0 0x7fb1b00b3a20 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 --2- 172.21.15.106:0/3459274487 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb1b006c330 0x7fb1b00b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 >> 172.21.15.106:0/3459274487 conn(0x7fb1b00cd6a0 msgr2=0x7fb1b015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 shutdown_connections 2024-04-13T03:19:28.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.543+0000 7fb1b81aa640 1 -- 172.21.15.106:0/3459274487 wait complete. 2024-04-13T03:19:28.558 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.656+0000 7fc9976de640 1 Processor -- start 2024-04-13T03:19:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 -- start start 2024-04-13T03:19:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 0x7fc9900d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9900b3d60 0x7fc9900d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900cfd20 con 0x7fc99015e9b0 2024-04-13T03:19:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc990162af0 con 0x7fc9900b3d60 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.657+0000 7fc9976de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc990162dd0 con 0x7fc99015ed90 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9900b3d60 0x7fc9900d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9900b3d60 0x7fc9900d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54452/0 (socket says 172.21.15.106:54452) 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 -- 172.21.15.106:0/1783631392 learned_addr learned my addr 172.21.15.106:0/1783631392 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc994c79640 1 --2- 172.21.15.106:0/1783631392 >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 0x7fc9900d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 -- 172.21.15.106:0/1783631392 >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 msgr2=0x7fc9900d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 --2- 172.21.15.106:0/1783631392 >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 0x7fc9900d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 -- 172.21.15.106:0/1783631392 >> v1:172.21.15.114:6789/0 conn(0x7fc99015e9b0 legacy=0x7fc9900d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc995c7b640 1 -- 172.21.15.106:0/1783631392 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901630b0 con 0x7fc9900b3d60 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.658+0000 7fc994c79640 1 --2- 172.21.15.106:0/1783631392 >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 0x7fc9900d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc995c7b640 1 --2- 172.21.15.106:0/1783631392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc9900d0200 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc99008ebd0 tx=0x7fc98c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb4efaf96f6d7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc987fff640 1 -- 172.21.15.106:0/1783631392 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c034070 con 0x7fc9900b3d60 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc987fff640 1 -- 172.21.15.106:0/1783631392 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c030860 con 0x7fc9900b3d60 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc987fff640 1 -- 172.21.15.106:0/1783631392 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c030c30 con 0x7fc9900b3d60 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 -- 172.21.15.106:0/1783631392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 msgr2=0x7fc9900d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 --2- 172.21.15.106:0/1783631392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc9900d0200 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc99008ebd0 tx=0x7fc98c02b9a0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 -- 172.21.15.106:0/1783631392 shutdown_connections 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 --2- 172.21.15.106:0/1783631392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc9900d0200 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 --2- 172.21.15.106:0/1783631392 >> v2:172.21.15.114:3300/0 conn(0x7fc99015ed90 0x7fc9900d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 -- 172.21.15.106:0/1783631392 >> 172.21.15.106:0/1783631392 conn(0x7fc99008bf10 msgr2=0x7fc9900b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 -- 172.21.15.106:0/1783631392 shutdown_connections 2024-04-13T03:19:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.659+0000 7fc9976de640 1 -- 172.21.15.106:0/1783631392 wait complete. 2024-04-13T03:19:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 Processor -- start 2024-04-13T03:19:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 -- start start 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 0x7fc9900ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 0x7fc9900b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc990162800 con 0x7fc9900b3d60 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc990162680 con 0x7fc99015e9b0 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc9976de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900d4d00 con 0x7fc9900b4f30 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc99547a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc99547a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54454/0 (socket says 172.21.15.106:54454) 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.661+0000 7fc99547a640 1 -- 172.21.15.106:0/1780022806 learned_addr learned my addr 172.21.15.106:0/1780022806 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc994c79640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 0x7fc9900ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc995c7b640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 0x7fc9900b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 -- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 msgr2=0x7fc9900b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 0x7fc9900b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 -- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 msgr2=0x7fc9900ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 0x7fc9900ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 -- 172.21.15.106:0/1780022806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900caaa0 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc995c7b640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 0x7fc9900b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc994c79640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 0x7fc9900ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc99547a640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c7a0 tx=0x7fc98800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988017070 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc988003da0 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9900c94c0 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880052b0 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.662+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9900c9a80 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.663+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990069190 con 0x7fc9900b3d60 2024-04-13T03:19:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.663+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98801a070 con 0x7fc9900b3d60 2024-04-13T03:19:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.664+0000 7fc985ffb640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.664+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fc98808b3a0 con 0x7fc9900b3d60 2024-04-13T03:19:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.664+0000 7fc994c79640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.664+0000 7fc994c79640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 0x7fc97406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cdfd0 tx=0x7fc980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:28.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.668+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc990069190 con 0x7fc9900b3d60 2024-04-13T03:19:28.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.880+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc990068a10 con 0x7fc9900b3d60 2024-04-13T03:19:28.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.881+0000 7fc985ffb640 1 -- 172.21.15.106:0/1780022806 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5190 (secure 0 0 0) 0x7fc988059680 con 0x7fc9900b3d60 2024-04-13T03:19:28.882 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:28.882 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:28.544366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.106:6837/3985285527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3985285527},{"type":"v1","addr":"172.21.15.106:6837","nonce":3985285527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-13T03:19:28.882 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 89 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 msgr2=0x7fc97406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 0x7fc97406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cdfd0 tx=0x7fc980008040 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 msgr2=0x7fc99015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c7a0 tx=0x7fc98800cc60 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 shutdown_connections 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc9900b4f30 0x7fc9900b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc99015e9b0 0x7fc9900ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 --2- 172.21.15.106:0/1780022806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9900b3d60 0x7fc99015b7f0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 >> 172.21.15.106:0/1780022806 conn(0x7fc99008bf10 msgr2=0x7fc9900b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:28.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 shutdown_connections 2024-04-13T03:19:28.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:28.883+0000 7fc9976de640 1 -- 172.21.15.106:0/1780022806 wait complete. 2024-04-13T03:19:29.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:28.544366+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/3985285527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 3985285527}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3985285527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:19:29.297 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9375:up:replay 2024-04-13T03:19:30.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:30.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.409+0000 7f5e139aa640 1 Processor -- start 2024-04-13T03:19:30.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.410+0000 7f5e139aa640 1 -- start start 2024-04-13T03:19:30.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.410+0000 7f5e139aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5e0c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.410+0000 7f5e139aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c0cfd20 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.410+0000 7f5e139aa640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c162af0 con 0x7f5e0c15ed90 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.410+0000 7f5e139aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c162dd0 con 0x7f5e0c15e9b0 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e11f47640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5e0c0b3d60 0x7f5e0c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57096/0 (socket says 172.21.15.106:57096) 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e11f47640 1 -- 172.21.15.106:0/1969576501 learned_addr learned my addr 172.21.15.106:0/1969576501 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e11746640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5e0c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576907048 0 0) 0x7f5e0c0cfd20 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df8003520 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109817569 0 0) 0x7f5e0c162af0 con 0x7f5e0c15ed90 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e0c0cfd20 con 0x7f5e0c15ed90 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2127602450 0 0) 0x7f5df8003520 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 msgr2=0x7f5e0c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5e0c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 >> v1:172.21.15.114:6789/0 conn(0x7f5e0c15ed90 legacy=0x7f5e0c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1630b0 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5e04001a10 con 0x7f5e0c0b3d60 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 >> v1:172.21.15.106:6789/0 conn(0x7f5e0c0b3d60 legacy=0x7f5e0c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5df8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5df8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5df8004d40 0x7f5df80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c162dd0 con 0x7f5df8003b70 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5df800ba70 con 0x7f5df8004d40 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5df800bc70 con 0x7f5e0c15e9b0 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.411+0000 7f5e11746640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5df8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e10f45640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5df8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11f47640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5df8004d40 0x7f5df80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 -- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 msgr2=0x7f5df8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c15e9b0 0x7f5df8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 -- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5df8004d40 msgr2=0x7f5df80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5df8004d40 0x7f5df80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 -- 172.21.15.106:0/1969576501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8003520 con 0x7f5df8003b70 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11f47640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5df8004d40 0x7f5df80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.412+0000 7f5e11746640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5df8003f70 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009ac0 tx=0x7f5df8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e34c58a5c59fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08039070 con 0x7f5df8003b70 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e0802c6b0 con 0x7f5df8003b70 2024-04-13T03:19:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e03fff640 1 -- 172.21.15.106:0/1969576501 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08009690 con 0x7f5df8003b70 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 -- 172.21.15.106:0/1969576501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 msgr2=0x7f5df8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1969576501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5df8003f70 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009ac0 tx=0x7f5df8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e11f47640 1 -- 172.21.15.106:0/1969576501 reap_dead start 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 -- 172.21.15.106:0/1969576501 shutdown_connections 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 -- 172.21.15.106:0/1969576501 >> 172.21.15.106:0/1969576501 conn(0x7f5e0c08bf10 msgr2=0x7f5e0c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 -- 172.21.15.106:0/1969576501 shutdown_connections 2024-04-13T03:19:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.413+0000 7f5e139aa640 1 -- 172.21.15.106:0/1969576501 wait complete. 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.414+0000 7f5e139aa640 1 Processor -- start 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.414+0000 7f5e139aa640 1 -- start start 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 0x7f5e0c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c161350 0x7f5e0c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c162800 con 0x7f5df8003b70 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c162680 con 0x7f5e0c15cfa0 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e139aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c0d4d00 con 0x7f5e0c161350 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54472/0 (socket says 172.21.15.106:54472) 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 -- 172.21.15.106:0/1827412432 learned_addr learned my addr 172.21.15.106:0/1827412432 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e10f45640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 0x7f5e0c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11f47640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c161350 0x7f5e0c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 -- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c161350 msgr2=0x7f5e0c1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c161350 0x7f5e0c1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 -- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 msgr2=0x7f5e0c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 0x7f5e0c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e11746640 1 -- 172.21.15.106:0/1827412432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c06e700 con 0x7f5df8003b70 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.415+0000 7f5e10f45640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 0x7f5e0c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e11746640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009ac0 tx=0x7f5e08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08039050 con 0x7f5df8003b70 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e08045070 con 0x7f5df8003b70 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c06cfc0 con 0x7f5df8003b70 2024-04-13T03:19:30.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003a50 con 0x7f5df8003b70 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.416+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c06d580 con 0x7f5df8003b70 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.417+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c161d10 con 0x7f5df8003b70 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.417+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e08003bf0 con 0x7f5df8003b70 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.418+0000 7f5e01ffb640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 0x7f5df006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.418+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f5e080ada20 con 0x7f5df8003b70 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.418+0000 7f5e10f45640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 0x7f5df006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.418+0000 7f5e10f45640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 0x7f5df006edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c162080 tx=0x7f5dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.422+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e0804e020 con 0x7f5df8003b70 2024-04-13T03:19:30.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.640+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0c0b4160 con 0x7f5df8003b70 2024-04-13T03:19:30.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.641+0000 7f5e01ffb640 1 -- 172.21.15.106:0/1827412432 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5190 (secure 0 0 0) 0x7f5e0c0b4160 con 0x7f5df8003b70 2024-04-13T03:19:30.642 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:30.642 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:29.549989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.106:6837/3985285527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3985285527},{"type":"v1","addr":"172.21.15.106:6837","nonce":3985285527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-13T03:19:30.642 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 91 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 msgr2=0x7f5df006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 0x7f5df006edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c162080 tx=0x7f5dfc009040 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 msgr2=0x7f5e0c06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009ac0 tx=0x7f5e08004730 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 shutdown_connections 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5df006c8f0 0x7f5df006edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5e0c161350 0x7f5e0c1617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e0c15cfa0 0x7f5e0c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 --2- 172.21.15.106:0/1827412432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5df8003b70 0x7f5e0c06c930 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:30.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 >> 172.21.15.106:0/1827412432 conn(0x7f5e0c08bf10 msgr2=0x7f5e0c0b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:30.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 shutdown_connections 2024-04-13T03:19:30.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:30.643+0000 7f5e139aa640 1 -- 172.21.15.106:0/1827412432 wait complete. 2024-04-13T03:19:31.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:29.549989+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.106:6837/3985285527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 3985285527}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3985285527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:19:31.057 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9375:up:rejoin 2024-04-13T03:19:32.058 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.169+0000 7f037a51e640 1 Processor -- start 2024-04-13T03:19:32.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.169+0000 7f037a51e640 1 -- start start 2024-04-13T03:19:32.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f037a51e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f03740cdf50 0x7f03740ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f037a51e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03740ce870 con 0x7f037406e150 2024-04-13T03:19:32.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f037a51e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03740998d0 con 0x7f037406cb10 2024-04-13T03:19:32.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f037a51e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0374073020 con 0x7f03740cdf50 2024-04-13T03:19:32.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0378abb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f037406e150 0x7f037406e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57100/0 (socket says 172.21.15.106:57100) 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0378abb640 1 -- 172.21.15.106:0/856933680 learned_addr learned my addr 172.21.15.106:0/856933680 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680531720 0 0) 0x7f03740ce870 con 0x7f037406e150 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f035c003520 con 0x7f037406e150 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0373fff640 1 --2- 172.21.15.106:0/856933680 >> v2:172.21.15.114:3300/0 conn(0x7f03740cdf50 0x7f03740ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.170+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1285287062 0 0) 0x7f035c003520 con 0x7f037406e150 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 >> v2:172.21.15.114:3300/0 conn(0x7f03740cdf50 msgr2=0x7f03740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 --2- 172.21.15.106:0/856933680 >> v2:172.21.15.114:3300/0 conn(0x7f03740cdf50 0x7f03740ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 >> v1:172.21.15.114:6789/0 conn(0x7f037406cb10 legacy=0x7f0374072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0374073300 con 0x7f037406e150 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f036c0019f0 con 0x7f037406e150 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 >> v1:172.21.15.106:6789/0 conn(0x7f037406e150 legacy=0x7f037406e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0039b0 0x7f035c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f035c004800 0x7f035c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 0x7f035c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0374073020 con 0x7f035c0051c0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f035c00c060 con 0x7f035c0039b0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f035c00c260 con 0x7f035c004800 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 0x7f035c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0373fff640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0039b0 0x7f035c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f03737fe640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f035c004800 0x7f035c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 -- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f035c004800 msgr2=0x7f035c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f035c004800 0x7f035c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 -- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0039b0 msgr2=0x7f035c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0039b0 0x7f035c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f0378abb640 1 -- 172.21.15.106:0/856933680 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c003520 con 0x7f035c0051c0 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.171+0000 7f03737fe640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f035c004800 0x7f035c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f0378abb640 1 -- 172.21.15.106:0/856933680 reap_dead start 2024-04-13T03:19:32.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f0378abb640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 0x7f035c00b9c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a860 tx=0x7f035c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a6d7bd38a36dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00a480 con 0x7f035c0051c0 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036c034630 con 0x7f035c0051c0 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f0372ffd640 1 -- 172.21.15.106:0/856933680 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c034a20 con 0x7f035c0051c0 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 -- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 msgr2=0x7f035c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 0x7f035c00b9c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a860 tx=0x7f035c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 -- 172.21.15.106:0/856933680 shutdown_connections 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 --2- 172.21.15.106:0/856933680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f035c0051c0 0x7f035c00b9c0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 -- 172.21.15.106:0/856933680 >> 172.21.15.106:0/856933680 conn(0x7f03740b1f10 msgr2=0x7f03740b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.172+0000 7f037a51e640 1 -- 172.21.15.106:0/856933680 shutdown_connections 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.173+0000 7f037a51e640 1 -- 172.21.15.106:0/856933680 wait complete. 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.174+0000 7f037a51e640 1 Processor -- start 2024-04-13T03:19:32.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.174+0000 7f037a51e640 1 -- start start 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0051c0 0x7f037409dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f037415adf0 0x7f03740c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03740ce330 con 0x7f037406b8e0 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f03740738c0 con 0x7f035c0051c0 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f03740ce4b0 con 0x7f037415adf0 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54498/0 (socket says 172.21.15.106:54498) 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 -- 172.21.15.106:0/3480147974 learned_addr learned my addr 172.21.15.106:0/3480147974 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 -- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f037415adf0 msgr2=0x7f03740c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f03737fe640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f037415adf0 0x7f03740c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0373fff640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0051c0 0x7f037409dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f037415adf0 0x7f03740c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 -- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0051c0 msgr2=0x7f037409dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0051c0 0x7f037409dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 -- 172.21.15.106:0/3480147974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037409cce0 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0378abb640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a980 tx=0x7f03740734d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00a480 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036c042070 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037409cfc0 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0019c0 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.175+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037409d580 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.177+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f037406c2a0 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.177+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f036c03c070 con 0x7f037406b8e0 2024-04-13T03:19:32.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.177+0000 7f0370ff9640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 0x7f036006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.177+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f036c0ad420 con 0x7f037406b8e0 2024-04-13T03:19:32.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.177+0000 7f0373fff640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 0x7f036006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.178+0000 7f0373fff640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 0x7f036006ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0364009980 tx=0x7f0364008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.182+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f036c07b670 con 0x7f037406b8e0 2024-04-13T03:19:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.395+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f037406f360 con 0x7f037406b8e0 2024-04-13T03:19:32.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.395+0000 7f0370ff9640 1 -- 172.21.15.106:0/3480147974 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5197 (secure 0 0 0) 0x7f037406f360 con 0x7f037406b8e0 2024-04-13T03:19:32.396 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:32.396 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":92,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:31.554133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/3985285527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3985285527},{"type":"v1","addr":"172.21.15.106:6837","nonce":3985285527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9375,"qdb_cluster":[9375]},"id":8}]} 2024-04-13T03:19:32.397 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 92 2024-04-13T03:19:32.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 msgr2=0x7f036006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 0x7f036006ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0364009980 tx=0x7f0364008040 comp rx=0 tx=0).stop 2024-04-13T03:19:32.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 msgr2=0x7f037406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a980 tx=0x7f03740734d0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 shutdown_connections 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f036006c920 0x7f036006ede0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f037415adf0 0x7f03740c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f037406b8e0 0x7f037406bd60 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 --2- 172.21.15.106:0/3480147974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f035c0051c0 0x7f037409dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.397+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 >> 172.21.15.106:0/3480147974 conn(0x7f03740b1f10 msgr2=0x7f03740afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.398+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 shutdown_connections 2024-04-13T03:19:32.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.398+0000 7f037a51e640 1 -- 172.21.15.106:0/3480147974 wait complete. 2024-04-13T03:19:32.812 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:31.554133+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.106:6837/3985285527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 3985285527}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3985285527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9375, 'qdb_cluster': [9375]} 2024-04-13T03:19:32.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:19:32.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-13T03:19:32.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.917+0000 7f4dff0de640 1 Processor -- start 2024-04-13T03:19:32.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.917+0000 7f4dff0de640 1 -- start start 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dff0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4df806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dff0de640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4df806f6c0 0x7f4df806be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dff0de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df80cef80 con 0x7f4df80d07c0 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dff0de640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4df80cf260 con 0x7f4df806f6c0 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dff0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4df80cf540 con 0x7f4df815fb50 2024-04-13T03:19:32.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4df7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4df80d07c0 0x7f4df8162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43726/0 (socket says 172.21.15.106:43726) 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4df7fff640 1 -- 172.21.15.106:0/2774166270 learned_addr learned my addr 172.21.15.106:0/2774166270 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377727189 0 0) 0x7f4df80cef80 con 0x7f4df80d07c0 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de0003520 con 0x7f4df80d07c0 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/2774166270 >> v2:172.21.15.114:3300/0 conn(0x7f4df806f6c0 0x7f4df806be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.918+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4df806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 541217742 0 0) 0x7f4de0003520 con 0x7f4df80d07c0 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 msgr2=0x7f4df806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4df806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 >> v2:172.21.15.114:3300/0 conn(0x7f4df806f6c0 msgr2=0x7f4df806be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 --2- 172.21.15.106:0/2774166270 >> v2:172.21.15.114:3300/0 conn(0x7f4df806f6c0 0x7f4df806be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80cf820 con 0x7f4df80d07c0 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4dec001670 con 0x7f4df80d07c0 2024-04-13T03:19:32.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 >> v1:172.21.15.106:6789/0 conn(0x7f4df80d07c0 legacy=0x7f4df8162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4de00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de0003f30 0x7f4de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4de0004d40 0x7f4de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80cf260 con 0x7f4de0003f30 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80cf540 con 0x7f4de0004d40 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4de000ba10 con 0x7f4df815fb50 2024-04-13T03:19:32.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de0003f30 0x7f4de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4de00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4de0004d40 0x7f4de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 -- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 msgr2=0x7f4de00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4df815fb50 0x7f4de00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 -- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4de0004d40 msgr2=0x7f4de00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4de0004d40 0x7f4de00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.919+0000 7f4df7fff640 1 -- 172.21.15.106:0/2774166270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de0003520 con 0x7f4de0003f30 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4df7fff640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de0003f30 0x7f4de0004800 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4dec007fd0 tx=0x7f4de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae119c9f5442f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007d50 con 0x7f4de0003f30 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec030070 con 0x7f4de0003f30 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4df77fe640 1 -- 172.21.15.106:0/2774166270 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec005d10 con 0x7f4de0003f30 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4dff0de640 1 -- 172.21.15.106:0/2774166270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de0003f30 msgr2=0x7f4de0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.920+0000 7f4dff0de640 1 --2- 172.21.15.106:0/2774166270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de0003f30 0x7f4de0004800 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4dec007fd0 tx=0x7f4de0005830 comp rx=0 tx=0).stop 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.921+0000 7f4dfd67b640 1 -- 172.21.15.106:0/2774166270 reap_dead start 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.921+0000 7f4dff0de640 1 -- 172.21.15.106:0/2774166270 shutdown_connections 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.921+0000 7f4dff0de640 1 -- 172.21.15.106:0/2774166270 >> 172.21.15.106:0/2774166270 conn(0x7f4df80cd6a0 msgr2=0x7f4df80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.921+0000 7f4dff0de640 1 -- 172.21.15.106:0/2774166270 shutdown_connections 2024-04-13T03:19:32.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.921+0000 7f4dff0de640 1 -- 172.21.15.106:0/2774166270 wait complete. 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.922+0000 7f4dff0de640 1 Processor -- start 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.922+0000 7f4dff0de640 1 -- start start 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4de0003f30 0x7f4df8160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 0x7f4df80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80ced20 con 0x7f4df8157620 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80ceba0 con 0x7f4df81f4a60 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4df815ef10 con 0x7f4de0003f30 2024-04-13T03:19:32.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56696/0 (socket says 172.21.15.106:56696) 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 -- 172.21.15.106:0/4148645728 learned_addr learned my addr 172.21.15.106:0/4148645728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4de0003f30 0x7f4df8160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 0x7f4df80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 -- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4de0003f30 msgr2=0x7f4df8160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4de0003f30 0x7f4df8160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 -- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 msgr2=0x7f4df80b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 0x7f4df80b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 -- 172.21.15.106:0/4148645728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81f54a0 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 0x7f4df80b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c300 tx=0x7f4dec000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df57fa640 1 -- 172.21.15.106:0/4148645728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec001cb0 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df57fa640 1 -- 172.21.15.106:0/4148645728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec004bf0 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81f4ee0 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4df57fa640 1 -- 172.21.15.106:0/4148645728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007070 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.923+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df80b3cc0 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.925+0000 7f4df57fa640 1 -- 172.21.15.106:0/4148645728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dec048020 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.925+0000 7f4df57fa640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.925+0000 7f4df57fa640 1 -- 172.21.15.106:0/4148645728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f4dec0ad050 con 0x7f4df8157620 2024-04-13T03:19:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.925+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.926+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 0x7f4de406edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4df80b1380 tx=0x7f4de8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.926+0000 7f4e01fb9740 1 Processor -- start 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.926+0000 7f4e01fb9740 1 -- start start 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf41d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf422c0 0x55933cf720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55933ceac990 con 0x55933cf6f1c0 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55933ce53620 con 0x55933cf6ede0 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55933cea8950 con 0x55933cf422c0 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56700/0 (socket says 172.21.15.106:56700) 2024-04-13T03:19:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 -- 172.21.15.106:0/4284492674 learned_addr learned my addr 172.21.15.106:0/4284492674 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 -- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf422c0 msgr2=0x55933cf720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf422c0 0x55933cf720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 -- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 msgr2=0x55933cf41d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf41d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 -- 172.21.15.106:0/4284492674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933cbbf2d0 con 0x55933cf6f1c0 2024-04-13T03:19:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4df7fff640 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0ab340 tx=0x7f4dec07ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311fbc15f06c560c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4dde7fc640 1 -- 172.21.15.106:0/4284492674 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec078ea0 con 0x55933cf6f1c0 2024-04-13T03:19:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4dde7fc640 1 -- 172.21.15.106:0/4284492674 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec030040 con 0x55933cf6f1c0 2024-04-13T03:19:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.927+0000 7f4dde7fc640 1 -- 172.21.15.106:0/4284492674 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0832f0 con 0x55933cf6f1c0 2024-04-13T03:19:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 -- 172.21.15.106:0/4284492674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 msgr2=0x55933cf71b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0ab340 tx=0x7f4dec07ac60 comp rx=0 tx=0).stop 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 -- 172.21.15.106:0/4284492674 shutdown_connections 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf422c0 0x55933cf720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf71b70 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/4284492674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf41d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 -- 172.21.15.106:0/4284492674 >> 172.21.15.106:0/4284492674 conn(0x55933cf43ff0 msgr2=0x55933cf8cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 -- 172.21.15.106:0/4284492674 shutdown_connections 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.928+0000 7f4e01fb9740 1 -- 172.21.15.106:0/4284492674 wait complete. 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.929+0000 7f4e01fb9740 1 -- start start 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.929+0000 7f4e01fb9740 1 Processor -- start 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf74ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf6f1c0 0x55933cf86530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55933ce53620 con 0x55933cf632d0 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55933ce669e0 con 0x55933cf6ede0 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55933ceac990 con 0x55933cf6f1c0 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 10 client.0 _ll_register_callbacks cb 0x55933cea2220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:32.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56706/0 (socket says 172.21.15.106:56706) 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 -- 172.21.15.106:0/194974787 learned_addr learned my addr 172.21.15.106:0/194974787 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 -- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf6f1c0 msgr2=0x55933cf86530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4df7fff640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf6f1c0 0x55933cf86530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf74ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf6f1c0 0x55933cf86530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 -- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 msgr2=0x55933cf74ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf74ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 -- 172.21.15.106:0/194974787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933cc896b0 con 0x55933cf632d0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf74ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009fd0 tx=0x7f4df0007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70115cce47843a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0003da0 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f4df0012900 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec030040 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4df0025050 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 10 client.9423 fetch_fsmap learned FSMap version 92 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.930+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55933cf84250 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec030040 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f4df0028020 con 0x55933cf632d0 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 10 client.9423 fetch_fsmap finished waiting for FSMap version 92 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 10 client.9423 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 20 client.9423 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933cf9d3b0 0x55933cfab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 4 client.9423 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:19:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 --> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933ceba3b0 con 0x55933cf9d3b0 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933cf9d3b0 0x55933cfab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4dd77fe640 10 client.9423 ms_handle_connect on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.931+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933cf9d3b0 0x55933cfab230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d695400066e82e3e server_cookie=a52c2138d3c43a72 in_seq=0 out_seq=0 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4dd77fe640 1 -- 172.21.15.106:0/194974787 <== mds.0 v2:172.21.15.106:6835/3985285527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55933ceba3b0 con 0x55933cf9d3b0 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4dd77fe640 10 client.9423 handle_command_reply: tid=0 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 client.9423 shutdown 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 msgr2=0x55933cf86a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009fd0 tx=0x7f4df0007f10 comp rx=0 tx=0).stop 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 shutdown_connections 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933cf9d3b0 0x55933cfab230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf6f1c0 0x55933cf86530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf632d0 0x55933cf86a70 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4dd77fe640 0 client.9423 ms_handle_reset on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/194974787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf74ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 >> 172.21.15.106:0/194974787 conn(0x55933cf422c0 msgr2=0x55933cf90080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 shutdown_connections 2024-04-13T03:19:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 1 -- 172.21.15.106:0/194974787 wait complete. 2024-04-13T03:19:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.932+0000 7f4e01fb9740 20 client.9423 trim_cache size 0 max 16384 2024-04-13T03:19:32.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.950+0000 7f4e01fb9740 1 Processor -- start 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.950+0000 7f4e01fb9740 1 -- start start 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf40c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8201cb0 con 0x55933cf6f1c0 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8200d30 con 0x55933cf6ede0 2024-04-13T03:19:32.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4df8200090 con 0x55933cf632d0 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56720/0 (socket says 172.21.15.106:56720) 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 -- 172.21.15.106:0/418737318 learned_addr learned my addr 172.21.15.106:0/418737318 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 -- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 msgr2=0x55933cf40c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf40c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 -- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 msgr2=0x55933cf411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4df7fff640 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 -- 172.21.15.106:0/418737318 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933cf9d9c0 con 0x55933cf6f1c0 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4df7fff640 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.951+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4df0012b40 tx=0x7f4df0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780a49ef76d0b11b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4dddffb640 1 -- 172.21.15.106:0/418737318 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0003da0 con 0x55933cf6f1c0 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4dddffb640 1 -- 172.21.15.106:0/418737318 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df001d070 con 0x55933cf6f1c0 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4dddffb640 1 -- 172.21.15.106:0/418737318 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004220 con 0x55933cf6f1c0 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 -- 172.21.15.106:0/418737318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 msgr2=0x55933cf61a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4df0012b40 tx=0x7f4df0018000 comp rx=0 tx=0).stop 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 -- 172.21.15.106:0/418737318 shutdown_connections 2024-04-13T03:19:32.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6f1c0 0x55933cf61a70 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6ede0 0x55933cf411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/418737318 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf40c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 -- 172.21.15.106:0/418737318 >> 172.21.15.106:0/418737318 conn(0x55933cf422c0 msgr2=0x55933cf8d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 -- 172.21.15.106:0/418737318 shutdown_connections 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.952+0000 7f4e01fb9740 1 -- 172.21.15.106:0/418737318 wait complete. 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.953+0000 7f4e01fb9740 1 -- start start 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.953+0000 7f4e01fb9740 1 Processor -- start 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6f1c0 0x55933cfec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55933cf75010 con 0x55933cf6ede0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4df82006e0 con 0x55933cf6f1c0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55933cf9cdb0 con 0x55933cf632d0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 10 client.0 _ll_register_callbacks cb 0x7f4df0004ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56724/0 (socket says 172.21.15.106:56724) 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 -- 172.21.15.106:0/1130480208 learned_addr learned my addr 172.21.15.106:0/1130480208 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dfd67b640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6f1c0 0x55933cfec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 -- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 msgr2=0x55933cf41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 -- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6f1c0 msgr2=0x55933cfec560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6f1c0 0x55933cfec560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 -- 172.21.15.106:0/1130480208 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933cf415d0 con 0x55933cf6ede0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4df7fff640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4dec07a8a0 tx=0x7f4dec006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e421a8d92b3be55 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec099060 con 0x55933cf6ede0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f4dec077210 con 0x55933cf6ede0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.954+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df001d070 con 0x55933cf6ede0 2024-04-13T03:19:32.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4dec07b2a0 con 0x55933cf6ede0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 10 client.9429 fetch_fsmap learned FSMap version 92 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55933cf415d0 con 0x55933cf6ede0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df001d070 con 0x55933cf6ede0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f4dec0832f0 con 0x55933cf6ede0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 10 client.9429 fetch_fsmap finished waiting for FSMap version 92 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 10 client.9429 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 20 client.9429 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933d00a900 0x55933d00ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 4 client.9429 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 --> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] -- command(tid 0: {"prefix": "status"}) -- 0x55933cf884f0 con 0x55933d00a900 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933d00a900 0x55933d00ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 10 client.9429 ms_handle_connect on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dfce7a640 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933d00a900 0x55933d00ace0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219951d3ab318766 server_cookie=97ca6f54f77f14d0 in_seq=0 out_seq=0 2024-04-13T03:19:32.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 1 -- 172.21.15.106:0/1130480208 <== mds.0 v2:172.21.15.106:6835/3985285527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55933cf884f0 con 0x55933d00a900 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4dd77fe640 10 client.9429 handle_command_reply: tid=0 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.955+0000 7f4e01fb9740 1 client.9429 shutdown 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 msgr2=0x55933cfa5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4dec07a8a0 tx=0x7f4dec006040 comp rx=0 tx=0).stop 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 shutdown_connections 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55933cf6f1c0 0x55933cfec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55933cf6ede0 0x55933cfa5c00 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x55933d00a900 0x55933d00ace0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 --2- 172.21.15.106:0/1130480208 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55933cf632d0 0x55933cf41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dd77fe640 0 client.9429 ms_handle_reset on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 >> 172.21.15.106:0/1130480208 conn(0x55933cf422c0 msgr2=0x55933cf8d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 shutdown_connections 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 1 -- 172.21.15.106:0/1130480208 wait complete. 2024-04-13T03:19:32.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4e01fb9740 20 client.9429 trim_cache size 0 max 16384 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 9375, 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.4087677049999998, 2024-04-13T03:19:32.982 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 92, 2024-04-13T03:19:32.983 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 122, 2024-04-13T03:19:32.983 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 122, 2024-04-13T03:19:32.983 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 5.375465105 2024-04-13T03:19:32.983 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:32.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 msgr2=0x7f4de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 0x7f4de406edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4df80b1380 tx=0x7f4de8008040 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 msgr2=0x7f4df81f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c300 tx=0x7f4dec000fe0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 shutdown_connections 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4df81f4a60 0x7f4df80b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df8157620 0x7f4df81f4520 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 --2- 172.21.15.106:0/4148645728 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4de0003f30 0x7f4df8160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.956+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 >> 172.21.15.106:0/4148645728 conn(0x7f4df80cd6a0 msgr2=0x7f4df806c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.957+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 shutdown_connections 2024-04-13T03:19:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:32.957+0000 7f4dff0de640 1 -- 172.21.15.106:0/4148645728 wait complete. 2024-04-13T03:19:32.986 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:19:33.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.085+0000 7fc4d0bb4640 1 Processor -- start 2024-04-13T03:19:33.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.085+0000 7fc4d0bb4640 1 -- start start 2024-04-13T03:19:33.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4d0bb4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:33.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4d0bb4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc1635b0 con 0x7fc4cc0d3b10 2024-04-13T03:19:33.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4d0bb4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc1627f0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4d0bb4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc0b7100 con 0x7fc4cc15ea80 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4ca59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4c9d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc4cc0d3b10 0x7fc4cc0b63b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43736/0 (socket says 172.21.15.106:43736) 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4ca59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56736/0 (socket says 172.21.15.106:56736) 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.086+0000 7fc4c9d9b640 1 -- 172.21.15.106:0/787579608 learned_addr learned my addr 172.21.15.106:0/787579608 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4ca59c640 1 -- 172.21.15.106:0/787579608 >> v1:172.21.15.114:6789/0 conn(0x7fc4cc0b6ac0 legacy=0x7fc4cc1689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4ca59c640 1 -- 172.21.15.106:0/787579608 >> v1:172.21.15.106:6789/0 conn(0x7fc4cc0d3b10 legacy=0x7fc4cc0b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4ca59c640 1 -- 172.21.15.106:0/787579608 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc162ad0 con 0x7fc4cc15ea80 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4c959a640 1 -- 172.21.15.106:0/787579608 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317259341 0 0) 0x7fc4cc1635b0 con 0x7fc4cc0d3b10 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4ca59c640 1 --2- 172.21.15.106:0/787579608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d90 tx=0x7fc4bc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3538c2f16e557b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4c959a640 1 -- 172.21.15.106:0/787579608 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc039070 con 0x7fc4cc15ea80 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.087+0000 7fc4c959a640 1 -- 172.21.15.106:0/787579608 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc02cbd0 con 0x7fc4cc15ea80 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4c959a640 1 -- 172.21.15.106:0/787579608 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc034440 con 0x7fc4cc15ea80 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/787579608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc15ea80 msgr2=0x7fc4cc0d35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/787579608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d90 tx=0x7fc4bc02be20 comp rx=0 tx=0).stop 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/787579608 shutdown_connections 2024-04-13T03:19:33.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/787579608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc0d35d0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/787579608 >> 172.21.15.106:0/787579608 conn(0x7fc4cc0b97a0 msgr2=0x7fc4cc0b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:33.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/787579608 shutdown_connections 2024-04-13T03:19:33.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.088+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/787579608 wait complete. 2024-04-13T03:19:33.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.089+0000 7fc4d0bb4640 1 Processor -- start 2024-04-13T03:19:33.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.089+0000 7fc4d0bb4640 1 -- start start 2024-04-13T03:19:33.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:33.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4cc0d3b10 0x7fc4cc15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:33.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc0b33e0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc163090 con 0x7fc4cc15ea80 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4d0bb4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc15ee60 con 0x7fc4cc0d3b10 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4ca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4ca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56750/0 (socket says 172.21.15.106:56750) 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4ca59c640 1 -- 172.21.15.106:0/1410102345 learned_addr learned my addr 172.21.15.106:0/1410102345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4cad9d640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4ca59c640 1 -- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4cc0d3b10 msgr2=0x7fc4cc15fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.090+0000 7fc4ca59c640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4cc0d3b10 0x7fc4cc15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4ca59c640 1 -- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 msgr2=0x7fc4cc15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4ca59c640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4ca59c640 1 -- 172.21.15.106:0/1410102345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc06c880 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4cad9d640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:33.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4ca59c640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d90 tx=0x7fc4bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc044020 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc03f670 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc06c5a0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc039040 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.091+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc15f4b0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.092+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc068a10 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.092+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4bc047020 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.093+0000 7fc4baffd640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 0x7fc4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.093+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fc4bc0ae8f0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.093+0000 7fc4c9d9b640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 0x7fc4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:33.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.093+0000 7fc4c9d9b640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 0x7fc4ac06edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c00079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:33.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.097+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4bc04e030 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.311+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4cc0b9e30 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.311+0000 7fc4baffd640 1 -- 172.21.15.106:0/1410102345 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7fc4bc07cbd0 con 0x7fc4cc0b6ac0 2024-04-13T03:19:33.312 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 msgr2=0x7fc4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 0x7fc4ac06edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c00079c0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 msgr2=0x7fc4cc160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d90 tx=0x7fc4bc004a40 comp rx=0 tx=0).stop 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 shutdown_connections 2024-04-13T03:19:33.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc4ac06c8f0 0x7fc4ac06edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc4cc0d3b10 0x7fc4cc15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4cc15ea80 0x7fc4cc15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 --2- 172.21.15.106:0/1410102345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4cc0b6ac0 0x7fc4cc160bb0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 >> 172.21.15.106:0/1410102345 conn(0x7fc4cc0b97a0 msgr2=0x7fc4cc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 shutdown_connections 2024-04-13T03:19:33.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33.313+0000 7fc4d0bb4640 1 -- 172.21.15.106:0/1410102345 wait complete. 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:19:33.325 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:33.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.326 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:19:33.341 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:19:33.341 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:19:33.341 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:19:33.341 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.341 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:33.342 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41976sec preferred_lft 41976sec 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.343 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:33.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.345 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:19:33.406 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:33.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.408 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:19:33.463 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:19:33.464 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:19:33.464 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:19:33.464 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:19:33.464 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:19:33.464 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:19:33.548 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:19:33.548+0000 7f36deff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:33.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:19:33.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:33.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.599 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:33.614 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:19:33.614 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:19:33.614 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:33.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:19:33.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:33.681 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:33.736 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:33.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:33.736 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:33.790 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:19:33.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:33.874 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:33.874 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:33.881 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:33.881 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:19:33.882 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:19:33.884 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:19:33.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:33.884 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:33.952 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:19:34.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:19:34.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.047 INFO:teuthology.orchestra.run.smithi106.stdout:client7849 2024-04-13T03:19:34.048 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:34.048 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:34.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:34.063 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-13T03:19:34.121 INFO:teuthology.orchestra.run.smithi106.stdout:./ 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/charlie 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/alpha 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/unflushed 2024-04-13T03:19:34.122 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/unflushed/jfile 2024-04-13T03:19:34.122 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.122 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.122 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.123 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.123 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-13T03:19:34.123 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.226 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:34.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.235 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.235 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.235 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.235 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.236 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-13T03:19:34.236 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.340 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:19:34.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.348 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.348 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.348 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.348 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.348 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-13T03:19:34.349 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.453 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:19:34.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.462 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.463 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.463 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.463 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.463 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-13T03:19:34.463 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.568 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628280 2024-04-13T03:19:34.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-13T03:19:34.579 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.683 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2024-04-13T03:19:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-13T03:19:34.692 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.798 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2024-04-13T03:19:34.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-13T03:19:34.807 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:34.911 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2024-04-13T03:19:34.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:34.920 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:34.920 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:34.975 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:34.975 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:35.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:35.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.168+0000 7f588be2d640 1 Processor -- start 2024-04-13T03:19:35.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 -- start start 2024-04-13T03:19:35.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58840cdf50 0x7f58840ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406cb10 0x7f5884072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58840ce870 con 0x7f588406b8e0 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f58840998d0 con 0x7f58840cdf50 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.169+0000 7f588be2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5884073020 con 0x7f588406cb10 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5889bc9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58840cdf50 0x7f58840ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f588a3ca640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f588406b8e0 0x7f588406bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43744/0 (socket says 172.21.15.106:43744) 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f588a3ca640 1 -- 172.21.15.106:0/3217550272 learned_addr learned my addr 172.21.15.106:0/3217550272 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5889bc9640 1 -- 172.21.15.106:0/3217550272 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406cb10 msgr2=0x7f5884072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5889bc9640 1 --2- 172.21.15.106:0/3217550272 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406cb10 0x7f5884072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5889bc9640 1 -- 172.21.15.106:0/3217550272 >> v1:172.21.15.106:6789/0 conn(0x7f588406b8e0 legacy=0x7f588406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5889bc9640 1 -- 172.21.15.106:0/3217550272 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5884073300 con 0x7f58840cdf50 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.170+0000 7f5888bc7640 1 -- 172.21.15.106:0/3217550272 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180393820 0 0) 0x7f58840ce870 con 0x7f588406b8e0 2024-04-13T03:19:35.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f5889bc9640 1 --2- 172.21.15.106:0/3217550272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58840cdf50 0x7f58840ce330 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f587c009d90 tx=0x7f587c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6045c3d3878981d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f5888bc7640 1 -- 172.21.15.106:0/3217550272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c039070 con 0x7f58840cdf50 2024-04-13T03:19:35.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f5888bc7640 1 -- 172.21.15.106:0/3217550272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587c02cc50 con 0x7f58840cdf50 2024-04-13T03:19:35.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f5888bc7640 1 -- 172.21.15.106:0/3217550272 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c034440 con 0x7f58840cdf50 2024-04-13T03:19:35.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 -- 172.21.15.106:0/3217550272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58840cdf50 msgr2=0x7f58840ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 --2- 172.21.15.106:0/3217550272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58840cdf50 0x7f58840ce330 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f587c009d90 tx=0x7f587c02bea0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 -- 172.21.15.106:0/3217550272 shutdown_connections 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 --2- 172.21.15.106:0/3217550272 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406cb10 0x7f5884072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 --2- 172.21.15.106:0/3217550272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58840cdf50 0x7f58840ce330 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 -- 172.21.15.106:0/3217550272 >> 172.21.15.106:0/3217550272 conn(0x7f58840b1f10 msgr2=0x7f58840b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 -- 172.21.15.106:0/3217550272 shutdown_connections 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.171+0000 7f588be2d640 1 -- 172.21.15.106:0/3217550272 wait complete. 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 Processor -- start 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 -- start start 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406b8e0 0x7f58840c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58840cdf50 0x7f588409dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5884072c60 con 0x7f588406cb10 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58840738c0 con 0x7f58840cdf50 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f588be2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5884072de0 con 0x7f588406b8e0 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f58893c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f58893c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56760/0 (socket says 172.21.15.106:56760) 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.173+0000 7f58893c8640 1 -- 172.21.15.106:0/2254668724 learned_addr learned my addr 172.21.15.106:0/2254668724 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 -- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406b8e0 msgr2=0x7f58840c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406b8e0 0x7f58840c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 -- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58840cdf50 msgr2=0x7f588409dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58840cdf50 0x7f588409dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 -- 172.21.15.106:0/2254668724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588409e2b0 con 0x7f588406cb10 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58893c8640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f587800b450 tx=0x7f587800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878015020 con 0x7f588406cb10 2024-04-13T03:19:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878011890 con 0x7f588406cb10 2024-04-13T03:19:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878011c80 con 0x7f588406cb10 2024-04-13T03:19:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58840c9f90 con 0x7f588406cb10 2024-04-13T03:19:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.174+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58840c9cb0 con 0x7f588406cb10 2024-04-13T03:19:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.175+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5884068a10 con 0x7f588406cb10 2024-04-13T03:19:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.175+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5878018020 con 0x7f588406cb10 2024-04-13T03:19:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.176+0000 7f58827fc640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.176+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f587808be10 con 0x7f588406cb10 2024-04-13T03:19:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.176+0000 7f5889bc9640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.176+0000 7f5889bc9640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 0x7f587006edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f587c036000 tx=0x7f587c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.180+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f587801f020 con 0x7f588406cb10 2024-04-13T03:19:35.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.355+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58840ce330 con 0x7f588406cb10 2024-04-13T03:19:35.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.356+0000 7f58827fc640 1 -- 172.21.15.106:0/2254668724 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+17119 (secure 0 0 0) 0x7f587800d040 con 0x7f588406cb10 2024-04-13T03:19:35.357 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:35.357 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":122,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:27.537479+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 msgr2=0x7f587006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 0x7f587006edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f587c036000 tx=0x7f587c02e000 comp rx=0 tx=0).stop 2024-04-13T03:19:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 msgr2=0x7f588409a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f587800b450 tx=0x7f587800b910 comp rx=0 tx=0).stop 2024-04-13T03:19:35.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 shutdown_connections 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58840cdf50 0x7f588409dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f588406cb10 0x7f588409a300 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 --2- 172.21.15.106:0/2254668724 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f588406b8e0 0x7f58840c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 >> 172.21.15.106:0/2254668724 conn(0x7f58840b1f10 msgr2=0x7f58840b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 shutdown_connections 2024-04-13T03:19:35.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.358+0000 7f588be2d640 1 -- 172.21.15.106:0/2254668724 wait complete. 2024-04-13T03:19:35.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:19:12.871192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:19:13.595223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:35.370 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 Processor -- start 2024-04-13T03:19:35.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 -- start start 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f079415c820 0x7f079406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f07940ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 0x7f0794160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f07940cd510 con 0x7f079415c820 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f07940706d0 con 0x7f07940cd0a0 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.473+0000 7f079a420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f07940709b0 con 0x7f07940b7f00 2024-04-13T03:19:35.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f079415c820 0x7f079406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f079415c820 0x7f079406fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56776/0 (socket says 172.21.15.106:56776) 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 -- 172.21.15.106:0/1029991981 learned_addr learned my addr 172.21.15.106:0/1029991981 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f07989bd640 1 --2- 172.21.15.106:0/1029991981 >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 0x7f0794160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f07937fe640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f07940ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 -- 172.21.15.106:0/1029991981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 msgr2=0x7f07940ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f07940ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 -- 172.21.15.106:0/1029991981 >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 msgr2=0x7f0794160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 --2- 172.21.15.106:0/1029991981 >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 0x7f0794160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f0793fff640 1 -- 172.21.15.106:0/1029991981 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0794070c90 con 0x7f079415c820 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f07937fe640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f07940ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.474+0000 7f07989bd640 1 --2- 172.21.15.106:0/1029991981 >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 0x7f0794160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f0793fff640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079406fea0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0794073ea0 tx=0x7f078402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9713ff88e94d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f0792ffd640 1 -- 172.21.15.106:0/1029991981 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784035020 con 0x7f079415c820 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f0792ffd640 1 -- 172.21.15.106:0/1029991981 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0784031870 con 0x7f079415c820 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f0792ffd640 1 -- 172.21.15.106:0/1029991981 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784031c40 con 0x7f079415c820 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 -- 172.21.15.106:0/1029991981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 msgr2=0x7f079406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079406fea0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0794073ea0 tx=0x7f078402b910 comp rx=0 tx=0).stop 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 -- 172.21.15.106:0/1029991981 shutdown_connections 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 --2- 172.21.15.106:0/1029991981 >> v2:172.21.15.114:3300/0 conn(0x7f07940cd0a0 0x7f0794160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f07940ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 --2- 172.21.15.106:0/1029991981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079406fea0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 -- 172.21.15.106:0/1029991981 >> 172.21.15.106:0/1029991981 conn(0x7f07940d2510 msgr2=0x7f07940d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 -- 172.21.15.106:0/1029991981 shutdown_connections 2024-04-13T03:19:35.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.475+0000 7f079a420640 1 -- 172.21.15.106:0/1029991981 wait complete. 2024-04-13T03:19:35.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.476+0000 7f079a420640 1 Processor -- start 2024-04-13T03:19:35.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 -- start start 2024-04-13T03:19:35.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f0794155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07940cd0a0 0x7f07940c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07940703e0 con 0x7f079415c820 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07940d03c0 con 0x7f07940cd0a0 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.477+0000 7f079a420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f07941607c0 con 0x7f07940b7f00 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56780/0 (socket says 172.21.15.106:56780) 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 -- 172.21.15.106:0/1801602110 learned_addr learned my addr 172.21.15.106:0/1801602110 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f0793fff640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f0794155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07937fe640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07940cd0a0 0x7f07940c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 -- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 msgr2=0x7f0794155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f0794155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 -- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07940cd0a0 msgr2=0x7f07940c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07940cd0a0 0x7f07940c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 -- 172.21.15.106:0/1801602110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07940c6c80 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f07989bd640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f078c009fa0 tx=0x7f078c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c01a070 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c003bb0 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c004030 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07940c6f60 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.478+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07940c7520 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.479+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0794069120 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.479+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f078c01d070 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.480+0000 7f0790ff9640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.480+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f078c08c040 con 0x7f079415c820 2024-04-13T03:19:35.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.480+0000 7f0793fff640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.480+0000 7f0793fff640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0784036000 tx=0x7f078403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.484+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f078c010070 con 0x7f079415c820 2024-04-13T03:19:35.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.660+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07940cc730 con 0x7f079415c820 2024-04-13T03:19:35.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.660+0000 7f0790ff9640 1 -- 172.21.15.106:0/1801602110 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f078c059bc0 con 0x7f079415c820 2024-04-13T03:19:35.661 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 msgr2=0x7f078006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0784036000 tx=0x7f078403c040 comp rx=0 tx=0).stop 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 msgr2=0x7f079415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f078c009fa0 tx=0x7f078c00d560 comp rx=0 tx=0).stop 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 shutdown_connections 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f078006c8f0 0x7f078006edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f079415c820 0x7f079415f1f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07940cd0a0 0x7f07940c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 --2- 172.21.15.106:0/1801602110 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f07940b7f00 0x7f0794155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 >> 172.21.15.106:0/1801602110 conn(0x7f07940d2510 msgr2=0x7f07940cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 shutdown_connections 2024-04-13T03:19:35.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.662+0000 7f079a420640 1 -- 172.21.15.106:0/1801602110 wait complete. 2024-04-13T03:19:35.674 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:35.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.776+0000 7f278c405640 1 Processor -- start 2024-04-13T03:19:35.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.776+0000 7f278c405640 1 -- start start 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f278c405640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f278415e9b0 0x7f27840d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f278c405640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f278415ed90 0x7f27840d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f278c405640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840cfd20 con 0x7f27840b3d60 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f278c405640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2784162af0 con 0x7f278415ed90 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f278c405640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2784162dd0 con 0x7f278415e9b0 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.777+0000 7f27899a0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f278415ed90 0x7f27840d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f278a9a2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f27840b3d60 0x7f27840d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43760/0 (socket says 172.21.15.106:43760) 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f278a9a2640 1 -- 172.21.15.106:0/3164982618 learned_addr learned my addr 172.21.15.106:0/3164982618 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f278a1a1640 1 --2- 172.21.15.106:0/3164982618 >> v2:172.21.15.114:3300/0 conn(0x7f278415e9b0 0x7f27840d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f27899a0640 1 -- 172.21.15.106:0/3164982618 >> v2:172.21.15.114:3300/0 conn(0x7f278415e9b0 msgr2=0x7f27840d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f27899a0640 1 --2- 172.21.15.106:0/3164982618 >> v2:172.21.15.114:3300/0 conn(0x7f278415e9b0 0x7f27840d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f27899a0640 1 -- 172.21.15.106:0/3164982618 >> v1:172.21.15.106:6789/0 conn(0x7f27840b3d60 legacy=0x7f27840d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f278919f640 1 -- 172.21.15.106:0/3164982618 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695988446 0 0) 0x7f27840cfd20 con 0x7f27840b3d60 2024-04-13T03:19:35.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.778+0000 7f27899a0640 1 -- 172.21.15.106:0/3164982618 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841630b0 con 0x7f278415ed90 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f27899a0640 1 --2- 172.21.15.106:0/3164982618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278415ed90 0x7f27840d45f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f278408e4e0 tx=0x7f277802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b440a34e2e42e4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278919f640 1 -- 172.21.15.106:0/3164982618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778034070 con 0x7f278415ed90 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278919f640 1 -- 172.21.15.106:0/3164982618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2778030610 con 0x7f278415ed90 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278919f640 1 -- 172.21.15.106:0/3164982618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27780309e0 con 0x7f278415ed90 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 -- 172.21.15.106:0/3164982618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278415ed90 msgr2=0x7f27840d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 --2- 172.21.15.106:0/3164982618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278415ed90 0x7f27840d45f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f278408e4e0 tx=0x7f277802b960 comp rx=0 tx=0).stop 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 -- 172.21.15.106:0/3164982618 shutdown_connections 2024-04-13T03:19:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 --2- 172.21.15.106:0/3164982618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278415ed90 0x7f27840d45f0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 --2- 172.21.15.106:0/3164982618 >> v2:172.21.15.114:3300/0 conn(0x7f278415e9b0 0x7f27840d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 -- 172.21.15.106:0/3164982618 >> 172.21.15.106:0/3164982618 conn(0x7f278408bf10 msgr2=0x7f27840b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 -- 172.21.15.106:0/3164982618 shutdown_connections 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.779+0000 7f278c405640 1 -- 172.21.15.106:0/3164982618 wait complete. 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.780+0000 7f278c405640 1 Processor -- start 2024-04-13T03:19:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.780+0000 7f278c405640 1 -- start start 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f27840b3d60 0x7f27841f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f278415e9b0 0x7f2784155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2784162800 con 0x7f278406b8e0 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2784162680 con 0x7f278415e9b0 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278c405640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f278406eb20 con 0x7f27840b3d60 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278a9a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278a9a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56802/0 (socket says 172.21.15.106:56802) 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278a9a2640 1 -- 172.21.15.106:0/605456896 learned_addr learned my addr 172.21.15.106:0/605456896 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278a1a1640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f27840b3d60 0x7f27841f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f27899a0640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f278415e9b0 0x7f2784155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.781+0000 7f278a9a2640 1 -- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f27840b3d60 msgr2=0x7f27841f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278a9a2640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f27840b3d60 0x7f27841f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278a9a2640 1 -- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f278415e9b0 msgr2=0x7f2784155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278a9a2640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f278415e9b0 0x7f2784155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278a9a2640 1 -- 172.21.15.106:0/605456896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278406bd70 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278a9a2640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c450 tx=0x7f277c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c018070 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c007500 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c004a20 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841f4fd0 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.782+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841f5590 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.783+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784068a10 con 0x7f278406b8e0 2024-04-13T03:19:35.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.783+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f277c01b070 con 0x7f278406b8e0 2024-04-13T03:19:35.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.784+0000 7f2782ffd640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 0x7f277006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:35.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.784+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f277c08b090 con 0x7f278406b8e0 2024-04-13T03:19:35.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.784+0000 7f278a1a1640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 0x7f277006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:35.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.784+0000 7f278a1a1640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 0x7f277006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2774009a60 tx=0x7f2774003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:35.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.788+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f277c059250 con 0x7f278406b8e0 2024-04-13T03:19:35.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.961+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f27840d53a0 con 0x7f278406b8e0 2024-04-13T03:19:35.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.962+0000 7f2782ffd640 1 -- 172.21.15.106:0/605456896 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7f277c058b50 con 0x7f278406b8e0 2024-04-13T03:19:35.963 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 msgr2=0x7f277006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 0x7f277006edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2774009a60 tx=0x7f2774003990 comp rx=0 tx=0).stop 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 msgr2=0x7f27840b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c450 tx=0x7f277c00c910 comp rx=0 tx=0).stop 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 shutdown_connections 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f277006c8f0 0x7f277006edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278406b8e0 0x7f27840b5df0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f278415e9b0 0x7f2784155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 --2- 172.21.15.106:0/605456896 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f27840b3d60 0x7f27841f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:35.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 >> 172.21.15.106:0/605456896 conn(0x7f278408bf10 msgr2=0x7f27840d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:35.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 shutdown_connections 2024-04-13T03:19:35.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:35.964+0000 7f278c405640 1 -- 172.21.15.106:0/605456896 wait complete. 2024-04-13T03:19:35.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:36.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.078+0000 7f603011e640 1 Processor -- start 2024-04-13T03:19:36.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.078+0000 7f603011e640 1 -- start start 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f603011e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6028073520 0x7f60280ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f603011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815eca0 0x7f60280b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f603011e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60280b5060 con 0x7f602815e8c0 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f603011e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f60280b5340 con 0x7f6028073520 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f603011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f602806c500 con 0x7f602815eca0 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602d6b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6028073520 0x7f60280ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602deba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f602815e8c0 0x7f6028072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43762/0 (socket says 172.21.15.106:43762) 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602deba640 1 -- 172.21.15.106:0/762707338 learned_addr learned my addr 172.21.15.106:0/762707338 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602e6bb640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815eca0 0x7f60280b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602d6b9640 1 -- 172.21.15.106:0/762707338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815eca0 msgr2=0x7f60280b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602d6b9640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815eca0 0x7f60280b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602d6b9640 1 -- 172.21.15.106:0/762707338 >> v1:172.21.15.106:6789/0 conn(0x7f602815e8c0 legacy=0x7f6028072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602d6b9640 1 -- 172.21.15.106:0/762707338 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602806c7e0 con 0x7f6028073520 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.079+0000 7f602ceb8640 1 -- 172.21.15.106:0/762707338 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969532160 0 0) 0x7f60280b5060 con 0x7f602815e8c0 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.080+0000 7f602d6b9640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6028073520 0x7f60280ce250 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f60180097a0 tx=0x7f601802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd74a45073cf804b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.080+0000 7f602ceb8640 1 -- 172.21.15.106:0/762707338 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018034070 con 0x7f6028073520 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.080+0000 7f602ceb8640 1 -- 172.21.15.106:0/762707338 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018030610 con 0x7f6028073520 2024-04-13T03:19:36.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.080+0000 7f602ceb8640 1 -- 172.21.15.106:0/762707338 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180309e0 con 0x7f6028073520 2024-04-13T03:19:36.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 -- 172.21.15.106:0/762707338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6028073520 msgr2=0x7f60280ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6028073520 0x7f60280ce250 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f60180097a0 tx=0x7f601802b960 comp rx=0 tx=0).stop 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 -- 172.21.15.106:0/762707338 shutdown_connections 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815eca0 0x7f60280b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 --2- 172.21.15.106:0/762707338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6028073520 0x7f60280ce250 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 -- 172.21.15.106:0/762707338 >> 172.21.15.106:0/762707338 conn(0x7f602806f0d0 msgr2=0x7f602806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 -- 172.21.15.106:0/762707338 shutdown_connections 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.081+0000 7f603011e640 1 -- 172.21.15.106:0/762707338 wait complete. 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.082+0000 7f603011e640 1 Processor -- start 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.082+0000 7f603011e640 1 -- start start 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6028073520 0x7f60281541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 0x7f6028155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60280b4d80 con 0x7f602815eca0 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60280b4c00 con 0x7f6028073520 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f602806cd00 con 0x7f602815e8c0 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56824/0 (socket says 172.21.15.106:56824) 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 -- 172.21.15.106:0/2296858780 learned_addr learned my addr 172.21.15.106:0/2296858780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602deba640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6028073520 0x7f60281541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602d6b9640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 0x7f6028155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 -- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 msgr2=0x7f6028155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 0x7f6028155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 -- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6028073520 msgr2=0x7f60281541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6028073520 0x7f60281541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 -- 172.21.15.106:0/2296858780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028156340 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602d6b9640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 0x7f6028155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f602e6bb640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f602000ebe0 tx=0x7f602000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020018020 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602000cd40 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6028156620 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200146c0 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.083+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6028156be0 con 0x7f602815eca0 2024-04-13T03:19:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.084+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6028068a10 con 0x7f602815eca0 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.085+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6020005d10 con 0x7f602815eca0 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.085+0000 7f601e7fc640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 0x7f600c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.085+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f602008ca30 con 0x7f602815eca0 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.085+0000 7f602deba640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 0x7f600c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.086+0000 7f602deba640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 0x7f600c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6024000f60 tx=0x7f6024007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.089+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6020022030 con 0x7f602815eca0 2024-04-13T03:19:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.263+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60280cdbf0 con 0x7f602815eca0 2024-04-13T03:19:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.263+0000 7f601e7fc640 1 -- 172.21.15.106:0/2296858780 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7f6020010040 con 0x7f602815eca0 2024-04-13T03:19:36.264 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 msgr2=0x7f600c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 0x7f600c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6024000f60 tx=0x7f6024007d50 comp rx=0 tx=0).stop 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 msgr2=0x7f60280b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f602000ebe0 tx=0x7f602000c490 comp rx=0 tx=0).stop 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 shutdown_connections 2024-04-13T03:19:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f600c06c8f0 0x7f600c06edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f602815e8c0 0x7f6028155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f602815eca0 0x7f60280b1d30 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 --2- 172.21.15.106:0/2296858780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6028073520 0x7f60281541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 >> 172.21.15.106:0/2296858780 conn(0x7f602806f0d0 msgr2=0x7f6028162a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 shutdown_connections 2024-04-13T03:19:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.265+0000 7f603011e640 1 -- 172.21.15.106:0/2296858780 wait complete. 2024-04-13T03:19:36.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 Processor -- start 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 -- start start 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f05b015cbd0 0x7f05b015d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b015d690 con 0x7f05b0156b40 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b015d970 con 0x7f05b006c330 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05b6295640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f05b006b8e0 con 0x7f05b015cbd0 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05affff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f05b0156b40 0x7f05b006e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43772/0 (socket says 172.21.15.106:43772) 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.378+0000 7f05affff640 1 -- 172.21.15.106:0/2160348134 learned_addr learned my addr 172.21.15.106:0/2160348134 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:36.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2160348134 >> v2:172.21.15.114:3300/0 conn(0x7f05b015cbd0 0x7f05b015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771863158 0 0) 0x7f05b015d690 con 0x7f05b0156b40 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0598003520 con 0x7f05b0156b40 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397977175 0 0) 0x7f05b015d970 con 0x7f05b006c330 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b015d690 con 0x7f05b006c330 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3817645373 0 0) 0x7f0598003520 con 0x7f05b0156b40 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 >> v2:172.21.15.114:3300/0 conn(0x7f05b015cbd0 msgr2=0x7f05b015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 --2- 172.21.15.106:0/2160348134 >> v2:172.21.15.114:3300/0 conn(0x7f05b015cbd0 0x7f05b015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 >> v1:172.21.15.114:6789/0 conn(0x7f05b006c330 legacy=0x7f05b00afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b006bb30 con 0x7f05b0156b40 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f05a0001670 con 0x7f05b0156b40 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 >> v1:172.21.15.106:6789/0 conn(0x7f05b0156b40 legacy=0x7f05b006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0598003b40 0x7f0598003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f0598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05980051c0 0x7f059800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05b006b8e0 con 0x7f0598004800 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f059800c060 con 0x7f05980051c0 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f059800c260 con 0x7f0598003b40 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f0598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05b4832640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05980051c0 0x7f059800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05affff640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0598003b40 0x7f0598003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 -- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0598003b40 msgr2=0x7f0598003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0598003b40 0x7f0598003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 -- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05980051c0 msgr2=0x7f059800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05980051c0 0x7f059800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05b4832640 1 -- 172.21.15.106:0/2160348134 reap_dead start 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05af7fe640 1 -- 172.21.15.106:0/2160348134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0598003520 con 0x7f0598004800 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.379+0000 7f05affff640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0598003b40 0x7f0598003f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:36.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.380+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f0598004c80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009a90 tx=0x7f0598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd65943a5a71d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.380+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4039070 con 0x7f0598004800 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.380+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a402c680 con 0x7f0598004800 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.380+0000 7f05aeffd640 1 -- 172.21.15.106:0/2160348134 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4009690 con 0x7f0598004800 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 -- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 msgr2=0x7f0598004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f0598004c80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009a90 tx=0x7f0598005830 comp rx=0 tx=0).stop 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 -- 172.21.15.106:0/2160348134 shutdown_connections 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 --2- 172.21.15.106:0/2160348134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f0598004c80 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 -- 172.21.15.106:0/2160348134 >> 172.21.15.106:0/2160348134 conn(0x7f05b00cd6a0 msgr2=0x7f05b00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 -- 172.21.15.106:0/2160348134 shutdown_connections 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.381+0000 7f05b6295640 1 -- 172.21.15.106:0/2160348134 wait complete. 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.382+0000 7f05b6295640 1 Processor -- start 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.382+0000 7f05b6295640 1 -- start start 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05b014efc0 0x7f05b014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b014f980 0x7f05b013fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05b006d880 con 0x7f0598004800 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05b006bea0 con 0x7f05b014f980 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b6295640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f05b006da00 con 0x7f05b014efc0 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56850/0 (socket says 172.21.15.106:56850) 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 -- 172.21.15.106:0/2766937331 learned_addr learned my addr 172.21.15.106:0/2766937331 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05b014efc0 0x7f05b014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05b4832640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b014f980 0x7f05b013fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 -- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05b014efc0 msgr2=0x7f05b014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05b014efc0 0x7f05b014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 -- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b014f980 msgr2=0x7f05b013fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b014f980 0x7f05b013fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 -- 172.21.15.106:0/2766937331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b013f080 con 0x7f0598004800 2024-04-13T03:19:36.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.383+0000 7f05affff640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e710 tx=0x7f05a000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.384+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000c6c0 con 0x7f0598004800 2024-04-13T03:19:36.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.384+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a0018070 con 0x7f0598004800 2024-04-13T03:19:36.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.384+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b013f360 con 0x7f0598004800 2024-04-13T03:19:36.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.384+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000ca80 con 0x7f0598004800 2024-04-13T03:19:36.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.384+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b013f920 con 0x7f0598004800 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.385+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b013fb20 con 0x7f0598004800 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.385+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05a001d460 con 0x7f0598004800 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.386+0000 7f05acff9640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.386+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.386+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f05a008bbe0 con 0x7f0598004800 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.386+0000 7f05af7fe640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 0x7f059c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05a4036000 tx=0x7f05a4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.390+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05b013fb20 con 0x7f0598004800 2024-04-13T03:19:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.580+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05b0068a10 con 0x7f0598004800 2024-04-13T03:19:36.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.580+0000 7f05acff9640 1 -- 172.21.15.106:0/2766937331 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) ==== 80+0+124 (secure 0 0 0) 0x7f05a0059870 con 0x7f0598004800 2024-04-13T03:19:36.581 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/1427655446 2024-04-14T03:19:27.537455+0000 2024-04-13T03:19:36.581 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/1427655446 2024-04-14T03:19:27.537455+0000 2024-04-13T03:19:36.581 INFO:teuthology.orchestra.run.smithi106.stderr:listed 2 entries 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 msgr2=0x7f059c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 0x7f059c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05a4036000 tx=0x7f05a4030000 comp rx=0 tx=0).stop 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 msgr2=0x7f05b01417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e710 tx=0x7f05a000ebd0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 shutdown_connections 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f059c06c8f0 0x7f059c06edb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05b014efc0 0x7f05b014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b014f980 0x7f05b013fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 --2- 172.21.15.106:0/2766937331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0598004800 0x7f05b01417b0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 >> 172.21.15.106:0/2766937331 conn(0x7f05b00cd6a0 msgr2=0x7f05b006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 shutdown_connections 2024-04-13T03:19:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.582+0000 7f05b6295640 1 -- 172.21.15.106:0/2766937331 wait complete. 2024-04-13T03:19:36.594 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:19:36.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:36.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:36.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:19:36.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:36.624 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:19:36.679 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:36.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.818+0000 7f494478c640 1 Processor -- start 2024-04-13T03:19:36.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 -- start start 2024-04-13T03:19:36.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f493c15ed90 0x7f493c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f493c0b3d60 0x7f493c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f493c0cfd20 con 0x7f493c15e9b0 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f493c162af0 con 0x7f493c0b3d60 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.819+0000 7f494478c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f493c162dd0 con 0x7f493c15ed90 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942d29640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f493c0b3d60 0x7f493c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942528640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f493c15e9b0 0x7f493c0d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43780/0 (socket says 172.21.15.106:43780) 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942528640 1 -- 172.21.15.106:0/3712699813 learned_addr learned my addr 172.21.15.106:0/3712699813 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942d29640 1 -- 172.21.15.106:0/3712699813 >> v2:172.21.15.114:3300/0 conn(0x7f493c15ed90 msgr2=0x7f493c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942d29640 1 --2- 172.21.15.106:0/3712699813 >> v2:172.21.15.114:3300/0 conn(0x7f493c15ed90 0x7f493c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942d29640 1 -- 172.21.15.106:0/3712699813 >> v1:172.21.15.106:6789/0 conn(0x7f493c15e9b0 legacy=0x7f493c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4942d29640 1 -- 172.21.15.106:0/3712699813 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1630b0 con 0x7f493c0b3d60 2024-04-13T03:19:36.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.820+0000 7f4941526640 1 -- 172.21.15.106:0/3712699813 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445456298 0 0) 0x7f493c0cfd20 con 0x7f493c15e9b0 2024-04-13T03:19:36.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f4942d29640 1 --2- 172.21.15.106:0/3712699813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c0b3d60 0x7f493c0d0200 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f493c08ebd0 tx=0x7f492c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1793170315853ca server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f4941526640 1 -- 172.21.15.106:0/3712699813 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c034070 con 0x7f493c0b3d60 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f4941526640 1 -- 172.21.15.106:0/3712699813 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c030610 con 0x7f493c0b3d60 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f4941526640 1 -- 172.21.15.106:0/3712699813 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0309e0 con 0x7f493c0b3d60 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 -- 172.21.15.106:0/3712699813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c0b3d60 msgr2=0x7f493c0d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 --2- 172.21.15.106:0/3712699813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c0b3d60 0x7f493c0d0200 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f493c08ebd0 tx=0x7f492c02b960 comp rx=0 tx=0).stop 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 -- 172.21.15.106:0/3712699813 shutdown_connections 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 --2- 172.21.15.106:0/3712699813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c0b3d60 0x7f493c0d0200 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 --2- 172.21.15.106:0/3712699813 >> v2:172.21.15.114:3300/0 conn(0x7f493c15ed90 0x7f493c0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 -- 172.21.15.106:0/3712699813 >> 172.21.15.106:0/3712699813 conn(0x7f493c08bf10 msgr2=0x7f493c0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 -- 172.21.15.106:0/3712699813 shutdown_connections 2024-04-13T03:19:36.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.821+0000 7f494478c640 1 -- 172.21.15.106:0/3712699813 wait complete. 2024-04-13T03:19:36.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.822+0000 7f494478c640 1 Processor -- start 2024-04-13T03:19:36.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.822+0000 7f494478c640 1 -- start start 2024-04-13T03:19:36.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 0x7f493c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 0x7f493c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f493c162800 con 0x7f493c1e8380 2024-04-13T03:19:36.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f493c162680 con 0x7f493c15e9b0 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f494478c640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f493c0d4d00 con 0x7f493c0b3d60 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56874/0 (socket says 172.21.15.106:56874) 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 -- 172.21.15.106:0/2894895468 learned_addr learned my addr 172.21.15.106:0/2894895468 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4941d27640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 0x7f493c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942528640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 0x7f493c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 -- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 msgr2=0x7f493c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 0x7f493c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 -- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 msgr2=0x7f493c0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.823+0000 7f4942d29640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 0x7f493c0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f4942d29640 1 -- 172.21.15.106:0/2894895468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1e8890 con 0x7f493c1e8380 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f4941d27640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 0x7f493c0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:36.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f4942528640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 0x7f493c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f4942d29640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f492c009860 tx=0x7f492c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c034040 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c039990 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c0b4250 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c039d60 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.824+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1e8b70 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.825+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c068a10 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.825+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f492c037070 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.826+0000 7f493affd640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.826+0000 7f4942528640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.826+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f492c0adf10 con 0x7f493c1e8380 2024-04-13T03:19:36.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.826+0000 7f4942528640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 0x7f492806edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f49340012e0 tx=0x7f4934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:36.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:36.830+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f492c07c160 con 0x7f493c1e8380 2024-04-13T03:19:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.040+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f493c0d5090 con 0x7f493c1e8380 2024-04-13T03:19:37.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.041+0000 7f493affd640 1 -- 172.21.15.106:0/2894895468 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5197 (secure 0 0 0) 0x7f492c07ba60 con 0x7f493c1e8380 2024-04-13T03:19:37.042 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:37.042 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":92,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:31.554133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/3985285527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3985285527},{"type":"v1","addr":"172.21.15.106:6837","nonce":3985285527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9375,"qdb_cluster":[9375]},"id":8}]} 2024-04-13T03:19:37.042 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 92 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 msgr2=0x7f492806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 0x7f492806edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f49340012e0 tx=0x7f4934009040 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 msgr2=0x7f493c0ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f492c009860 tx=0x7f492c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 shutdown_connections 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f493c1e8380 0x7f493c0ccd90 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f493c15e9b0 0x7f493c0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 --2- 172.21.15.106:0/2894895468 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f493c0b3d60 0x7f493c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 >> 172.21.15.106:0/2894895468 conn(0x7f493c08bf10 msgr2=0x7f493c0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 shutdown_connections 2024-04-13T03:19:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.043+0000 7f494478c640 1 -- 172.21.15.106:0/2894895468 wait complete. 2024-04-13T03:19:37.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-13T03:19:37.175 INFO:teuthology.orchestra.run.smithi106.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-13T03:19:37.176 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-13T03:19:37.176 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-13T03:19:37.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.217+0000 7f6b59d3ef40 1 Processor -- start 2024-04-13T03:19:37.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.217+0000 7f6b59d3ef40 1 -- start start 2024-04-13T03:19:37.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a2a170 0x562f179e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aaefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562f17a12db0 con 0x562f17a12590 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562f1798ad00 con 0x562f17a2a170 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x562f17a13050 con 0x562f17a2c050 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56896/0 (socket says 172.21.15.106:56896) 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 -- 172.21.15.106:0/2836165274 learned_addr learned my addr 172.21.15.106:0/2836165274 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 -- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 msgr2=0x562f17aaefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aaefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 -- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a2a170 msgr2=0x562f179e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a2a170 0x562f179e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 -- 172.21.15.106:0/2836165274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f17a249f0 con 0x562f17a12590 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b575a3640 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097c0 tx=0x7f6b50009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9231583782a5433 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b55da0640 1 -- 172.21.15.106:0/2836165274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50017070 con 0x562f17a12590 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b55da0640 1 -- 172.21.15.106:0/2836165274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b500090d0 con 0x562f17a12590 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.218+0000 7f6b55da0640 1 -- 172.21.15.106:0/2836165274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50006b60 con 0x562f17a12590 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/2836165274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 msgr2=0x562f17a243c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097c0 tx=0x7f6b50009c80 comp rx=0 tx=0).stop 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/2836165274 shutdown_connections 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aaefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a2a170 0x562f179e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/2836165274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a12590 0x562f17a243c0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/2836165274 >> 172.21.15.106:0/2836165274 conn(0x562f17a10a20 msgr2=0x562f17abb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/2836165274 shutdown_connections 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/2836165274 wait complete. 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 Processor -- start 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.219+0000 7f6b59d3ef40 1 -- start start 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a12590 0x562f17aacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 0x562f17a10070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aa2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562f17a24f30 con 0x562f17a2a170 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562f17a24c90 con 0x562f17a12590 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b59d3ef40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x562f17a28440 con 0x562f17a2c050 2024-04-13T03:19:37.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b56da2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 0x562f17a10070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b56da2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 0x562f17a10070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56898/0 (socket says 172.21.15.106:56898) 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b56da2640 1 -- 172.21.15.106:0/3466665924 learned_addr learned my addr 172.21.15.106:0/3466665924 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b575a3640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a12590 0x562f17aacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.220+0000 7f6b57da4640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aa2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 -- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 msgr2=0x562f17aa2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aa2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 -- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a12590 msgr2=0x562f17aacfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a12590 0x562f17aacfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 -- 172.21.15.106:0/3466665924 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f17b4ee60 con 0x562f17a2a170 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b575a3640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f17a12590 0x562f17aacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b57da4640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x562f17a2c050 0x562f17aa2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b56da2640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 0x562f17a10070 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c057040 tx=0x7f6b4c04d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b54d9e640 1 -- 172.21.15.106:0/3466665924 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c065020 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b54d9e640 1 -- 172.21.15.106:0/3466665924 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c04d740 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f17aad6a0 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b54d9e640 1 -- 172.21.15.106:0/3466665924 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c061ac0 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.221+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f17aadc60 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.222+0000 7f6b54d9e640 1 -- 172.21.15.106:0/3466665924 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b4c068020 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.222+0000 7f6b54d9e640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6b4406b2d0 0x7f6b4406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b54d9e640 1 -- 172.21.15.106:0/3466665924 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f6b4c0cf330 con 0x562f17a2a170 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b575a3640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6b4406b2d0 0x7f6b4406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x562f17b4f360 0x562f17b4f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x562f17b4fcb0 con 0x562f17b4f360 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b57da4640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x562f17b4f360 0x562f17b4f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b575a3640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6b4406b2d0 0x7f6b4406d790 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000afd0 tx=0x7f6b5000f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b57da4640 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x562f17b4f360 0x562f17b4f770 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b57da4640 1 -- 172.21.15.106:0/3466665924 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f6b4800d430 con 0x562f17b4f360 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x562f17b4f360 msgr2=0x562f17b4f770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.223+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x562f17b4f360 0x562f17b4f770 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6b4406b2d0 msgr2=0x7f6b4406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6b4406b2d0 0x7f6b4406d790 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000afd0 tx=0x7f6b5000f000 comp rx=0 tx=0).stop 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 msgr2=0x562f17a10070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 --2- 172.21.15.106:0/3466665924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562f17a2a170 0x562f17a10070 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c057040 tx=0x7f6b4c04d550 comp rx=0 tx=0).stop 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b57da4640 1 -- 172.21.15.106:0/3466665924 reap_dead start 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 shutdown_connections 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 >> 172.21.15.106:0/3466665924 conn(0x562f17a10a20 msgr2=0x562f179e6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 shutdown_connections 2024-04-13T03:19:37.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.224+0000 7f6b59d3ef40 1 -- 172.21.15.106:0/3466665924 wait complete. 2024-04-13T03:19:37.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:19:37.234 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-13T03:19:37.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.316+0000 7f7a2a97ff40 1 Processor -- start 2024-04-13T03:19:37.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.316+0000 7f7a2a97ff40 1 -- start start 2024-04-13T03:19:37.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a14002c80 0x55846ca65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 0x55846cb1ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55846ca0cd00 con 0x55846ca66530 2024-04-13T03:19:37.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55846ca10d40 con 0x55846cb39520 2024-04-13T03:19:37.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55846cb39c30 con 0x7f7a14002c80 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56914/0 (socket says 172.21.15.106:56914) 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 -- 172.21.15.106:0/4221307635 learned_addr learned my addr 172.21.15.106:0/4221307635 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 0x55846cb1ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a2824b640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a14002c80 0x55846ca65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 -- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a14002c80 msgr2=0x55846ca65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a14002c80 0x55846ca65110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 -- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 msgr2=0x55846cb1ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 0x55846cb1ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 -- 172.21.15.106:0/4221307635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846cb20760 con 0x55846ca66530 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 0x55846cb1ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006c70 tx=0x7f7a20003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20e51db8c0a63e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a26a48640 1 -- 172.21.15.106:0/4221307635 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20018070 con 0x55846ca66530 2024-04-13T03:19:37.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a26a48640 1 -- 172.21.15.106:0/4221307635 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a20006e20 con 0x55846ca66530 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.317+0000 7f7a26a48640 1 -- 172.21.15.106:0/4221307635 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200016b0 con 0x55846ca66530 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/4221307635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 msgr2=0x55846cb1b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006c70 tx=0x7f7a20003aa0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/4221307635 shutdown_connections 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb1b3a0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55846cb39520 0x55846cb1ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/4221307635 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f7a14002c80 0x55846ca65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/4221307635 >> 172.21.15.106:0/4221307635 conn(0x55846ca8fae0 msgr2=0x55846cb3d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/4221307635 shutdown_connections 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/4221307635 wait complete. 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 Processor -- start 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.318+0000 7f7a2a97ff40 1 -- start start 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a14002c80 0x55846cb2e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55846cb1f250 0x55846cb374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55846cb18a90 con 0x55846ca66530 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55846ca65650 con 0x7f7a14002c80 2024-04-13T03:19:37.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2a97ff40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55846cb18230 con 0x55846cb1f250 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2824b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb29ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2824b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb29ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56916/0 (socket says 172.21.15.106:56916) 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2824b640 1 -- 172.21.15.106:0/728622879 learned_addr learned my addr 172.21.15.106:0/728622879 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a14002c80 0x55846cb2e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55846cb1f250 0x55846cb374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2824b640 1 -- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55846cb1f250 msgr2=0x55846cb374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.319+0000 7f7a2824b640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55846cb1f250 0x55846cb374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2824b640 1 -- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a14002c80 msgr2=0x55846cb2e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2824b640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a14002c80 0x55846cb2e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2824b640 1 -- 172.21.15.106:0/728622879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846cb29ff0 con 0x55846ca66530 2024-04-13T03:19:37.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a14002c80 0x55846cb2e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2824b640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb29ab0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009f90 tx=0x7f7a100084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a25a46640 1 -- 172.21.15.106:0/728622879 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1001a070 con 0x55846ca66530 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a25a46640 1 -- 172.21.15.106:0/728622879 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a10016610 con 0x55846ca66530 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55846cb2f270 con 0x55846ca66530 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a25a46640 1 -- 172.21.15.106:0/728622879 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10016a00 con 0x55846ca66530 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.320+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846cb2f830 con 0x55846ca66530 2024-04-13T03:19:37.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.321+0000 7f7a25a46640 1 -- 172.21.15.106:0/728622879 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a1001d070 con 0x55846ca66530 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.321+0000 7f7a25a46640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a0806b2d0 0x7f7a0806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a0806b2d0 0x7f7a0806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a25a46640 1 -- 172.21.15.106:0/728622879 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f7a1008b150 con 0x55846ca66530 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55846cb317d0 0x55846cb31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x55846cb32130 con 0x55846cb317d0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55846cb317d0 0x55846cb31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a27a4a640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a0806b2d0 0x7f7a0806d790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009bc0 tx=0x7f7a18050000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a28a4c640 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55846cb317d0 0x55846cb31bf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.322+0000 7f7a28a4c640 1 -- 172.21.15.106:0/728622879 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f7a200016b0 con 0x55846cb317d0 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55846cb317d0 msgr2=0x55846cb31bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55846cb317d0 0x55846cb31bf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a0806b2d0 msgr2=0x7f7a0806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f7a0806b2d0 0x7f7a0806d790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009bc0 tx=0x7f7a18050000 comp rx=0 tx=0).stop 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 msgr2=0x55846cb29ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 --2- 172.21.15.106:0/728622879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55846ca66530 0x55846cb29ab0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009f90 tx=0x7f7a100084c0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a28a4c640 1 -- 172.21.15.106:0/728622879 reap_dead start 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 shutdown_connections 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 >> 172.21.15.106:0/728622879 conn(0x55846ca8fae0 msgr2=0x55846caad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 shutdown_connections 2024-04-13T03:19:37.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.323+0000 7f7a2a97ff40 1 -- 172.21.15.106:0/728622879 wait complete. 2024-04-13T03:19:37.336 INFO:teuthology.orchestra.run.smithi106.stdout:mytag123 2024-04-13T03:19:37.336 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-13T03:19:37.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.418+0000 7f0691772f40 1 Processor -- start 2024-04-13T03:19:37.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.418+0000 7f0691772f40 1 -- start start 2024-04-13T03:19:37.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 0x56306ff09ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x563070042b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56306fedc050 con 0x56306ff967c0 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56306fe82d00 con 0x56306ff23220 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f0691772f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56306fedc1d0 con 0x56306ff96ba0 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56944/0 (socket says 172.21.15.106:56944) 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 -- 172.21.15.106:0/4040833319 learned_addr learned my addr 172.21.15.106:0/4040833319 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068f84c640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x563070042b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068f04b640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 0x56306ff09ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 -- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 msgr2=0x563070042b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x563070042b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 -- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 msgr2=0x56306ff09ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 0x56306ff09ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 -- 172.21.15.106:0/4040833319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306ff945f0 con 0x56306ff967c0 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068f84c640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x563070042b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068f04b640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 0x56306ff09ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068e84a640 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f068400a590 tx=0x7f068400aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e62569f7ac1ff9b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068d848640 1 -- 172.21.15.106:0/4040833319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068400b980 con 0x56306ff967c0 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068d848640 1 -- 172.21.15.106:0/4040833319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068400f070 con 0x56306ff967c0 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.419+0000 7f068d848640 1 -- 172.21.15.106:0/4040833319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068400bd80 con 0x56306ff967c0 2024-04-13T03:19:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- 172.21.15.106:0/4040833319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 msgr2=0x56306ff99bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f068400a590 tx=0x7f068400aa50 comp rx=0 tx=0).stop 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- 172.21.15.106:0/4040833319 shutdown_connections 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x563070042b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff967c0 0x56306ff99bb0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 --2- 172.21.15.106:0/4040833319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff23220 0x56306ff09ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- 172.21.15.106:0/4040833319 >> 172.21.15.106:0/4040833319 conn(0x56306ff05ae0 msgr2=0x56306ff07e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- 172.21.15.106:0/4040833319 shutdown_connections 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- 172.21.15.106:0/4040833319 wait complete. 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 Processor -- start 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.420+0000 7f0691772f40 1 -- start start 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 0x56307003f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff967c0 0x56307003f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x56307003fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56306fec8ea0 con 0x56306ff23220 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56306ff9a0f0 con 0x56306ff967c0 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f0691772f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x56306ffb0180 con 0x56306ff96ba0 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 0x56307003f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 0x56307003f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56962/0 (socket says 172.21.15.106:56962) 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 -- 172.21.15.106:0/1969240779 learned_addr learned my addr 172.21.15.106:0/1969240779 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f84c640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x56307003fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068e84a640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff967c0 0x56307003f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 -- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 msgr2=0x56307003fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x56306ff96ba0 0x56307003fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 -- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff967c0 msgr2=0x56307003f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56306ff967c0 0x56307003f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.421+0000 7f068f04b640 1 -- 172.21.15.106:0/1969240779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306ffa9960 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f068f04b640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 0x56307003f420 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f068001f5d0 tx=0x7f0680003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f068c846640 1 -- 172.21.15.106:0/1969240779 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680060830 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f068c846640 1 -- 172.21.15.106:0/1969240779 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680066070 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56306ffa9c00 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f068c846640 1 -- 172.21.15.106:0/1969240779 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680060bd0 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.422+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306ffaa1c0 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.423+0000 7f068c846640 1 -- 172.21.15.106:0/1969240779 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0680053070 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.423+0000 7f068c846640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f067c06b300 0x7f067c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f068c846640 1 -- 172.21.15.106:0/1969240779 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f06800ceec0 con 0x56306ff23220 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f068e84a640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f067c06b300 0x7f067c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f0691772f40 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56306ff8d5f0 0x56306ff8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x56306ffaa9c0 con 0x56306ff8d5f0 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f068f84c640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56306ff8d5f0 0x56306ff8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.424+0000 7f068e84a640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f067c06b300 0x7f067c06d7c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684002000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f068f84c640 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56306ff8d5f0 0x56306ff8d9d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f068f84c640 1 -- 172.21.15.106:0/1969240779 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f06880075a0 con 0x56306ff8d5f0 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56306ff8d5f0 msgr2=0x56306ff8d9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x56306ff8d5f0 0x56306ff8d9d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f067c06b300 msgr2=0x7f067c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f067c06b300 0x7f067c06d7c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684002000 comp rx=0 tx=0).stop 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 msgr2=0x56307003f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 --2- 172.21.15.106:0/1969240779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56306ff23220 0x56307003f420 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f068001f5d0 tx=0x7f0680003330 comp rx=0 tx=0).stop 2024-04-13T03:19:37.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f068f84c640 1 -- 172.21.15.106:0/1969240779 reap_dead start 2024-04-13T03:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 shutdown_connections 2024-04-13T03:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 >> 172.21.15.106:0/1969240779 conn(0x56306ff05ae0 msgr2=0x56306ff998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 shutdown_connections 2024-04-13T03:19:37.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.425+0000 7f0691772f40 1 -- 172.21.15.106:0/1969240779 wait complete. 2024-04-13T03:19:37.434 INFO:teuthology.orchestra.run.smithi106.stdout:mytag123 2024-04-13T03:19:37.434 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-13T03:19:37.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 Processor -- start 2024-04-13T03:19:37.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 -- start start 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 0x560a5c45a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 0x560a5c50bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560a5c4387f0 con 0x560a5c4ef280 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560a5c3ded00 con 0x560a5c4ef7d0 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd62649f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560a5c438a90 con 0x7fbd4c0030f0 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd5ff50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd5ff50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56978/0 (socket says 172.21.15.106:56978) 2024-04-13T03:19:37.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.517+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1116790075 learned_addr learned my addr 172.21.15.106:0/1116790075 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5f74f640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 0x560a5c45a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ef4e640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 0x560a5c50bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 msgr2=0x560a5c45a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 0x560a5c45a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 msgr2=0x560a5c50bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 0x560a5c50bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1116790075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5c47f5a0 con 0x560a5c4ef280 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ef4e640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 0x560a5c50bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5f74f640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 0x560a5c45a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fbd58001700 tx=0x7fbd58009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73c1691b6ef0bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5df4c640 1 -- 172.21.15.106:0/1116790075 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800f7d0 con 0x560a5c4ef280 2024-04-13T03:19:37.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5df4c640 1 -- 172.21.15.106:0/1116790075 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5800f970 con 0x560a5c4ef280 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd5df4c640 1 -- 172.21.15.106:0/1116790075 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800fd20 con 0x560a5c4ef280 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd62649f40 1 -- 172.21.15.106:0/1116790075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 msgr2=0x560a5c464eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.518+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fbd58001700 tx=0x7fbd58009a30 comp rx=0 tx=0).stop 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 -- 172.21.15.106:0/1116790075 shutdown_connections 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c464eb0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560a5c4ef7d0 0x560a5c50bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1116790075 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fbd4c0030f0 0x560a5c45a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 -- 172.21.15.106:0/1116790075 >> 172.21.15.106:0/1116790075 conn(0x560a5c461ae0 msgr2=0x560a5c4f5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 -- 172.21.15.106:0/1116790075 shutdown_connections 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 -- 172.21.15.106:0/1116790075 wait complete. 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 Processor -- start 2024-04-13T03:19:37.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.519+0000 7fbd62649f40 1 -- start start 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c4ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560a5c4ef7d0 0x560a5c4efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd4c0030f0 0x560a5c4f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560a5c4599c0 con 0x560a5c4ef280 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560a5c47f840 con 0x7fbd4c0030f0 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd62649f40 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560a5c459c60 con 0x560a5c4ef7d0 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c4ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c4ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56982/0 (socket says 172.21.15.106:56982) 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 -- 172.21.15.106:0/1859969549 learned_addr learned my addr 172.21.15.106:0/1859969549 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5ef4e640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560a5c4ef7d0 0x560a5c4efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd4c0030f0 0x560a5c4f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 -- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560a5c4ef7d0 msgr2=0x560a5c4efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560a5c4ef7d0 0x560a5c4efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 -- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd4c0030f0 msgr2=0x560a5c4f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd4c0030f0 0x560a5c4f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 -- 172.21.15.106:0/1859969549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5c45e750 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.520+0000 7fbd5f74f640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c4ee260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000a640 tx=0x7fbd5000ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.521+0000 7fbd5cf4a640 1 -- 172.21.15.106:0/1859969549 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50018070 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.521+0000 7fbd5cf4a640 1 -- 172.21.15.106:0/1859969549 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd500108c0 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.521+0000 7fbd5cf4a640 1 -- 172.21.15.106:0/1859969549 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50010c70 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.521+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a5c45ea30 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.521+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5c45eff0 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.522+0000 7fbd5cf4a640 1 -- 172.21.15.106:0/1859969549 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd50011040 con 0x560a5c4ef280 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.522+0000 7fbd5cf4a640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbd4006b2d0 0x7fbd4006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.522+0000 7fbd5cf4a640 1 -- 172.21.15.106:0/1859969549 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fbd5008b550 con 0x560a5c4ef280 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.522+0000 7fbd5ef4e640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbd4006b2d0 0x7fbd4006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560a5c464f10 0x560a5c5a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x560a5c5a5f30 con 0x560a5c464f10 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560a5c464f10 0x560a5c5a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd5ef4e640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbd4006b2d0 0x7fbd4006d790 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbd48056000 tx=0x7fbd48051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd5ff50640 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560a5c464f10 0x560a5c5a59f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1859969549 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fbd5800fb20 con 0x560a5c464f10 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560a5c464f10 msgr2=0x560a5c5a59f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560a5c464f10 0x560a5c5a59f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbd4006b2d0 msgr2=0x7fbd4006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fbd4006b2d0 0x7fbd4006d790 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbd48056000 tx=0x7fbd48051000 comp rx=0 tx=0).stop 2024-04-13T03:19:37.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 msgr2=0x560a5c4ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd62649f40 1 --2- 172.21.15.106:0/1859969549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560a5c4ef280 0x560a5c4ee260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000a640 tx=0x7fbd5000ab00 comp rx=0 tx=0).stop 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.523+0000 7fbd5ff50640 1 -- 172.21.15.106:0/1859969549 reap_dead start 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.524+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 shutdown_connections 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.524+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 >> 172.21.15.106:0/1859969549 conn(0x560a5c461ae0 msgr2=0x560a5c4f5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.524+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 shutdown_connections 2024-04-13T03:19:37.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.524+0000 7fbd62649f40 1 -- 172.21.15.106:0/1859969549 wait complete. 2024-04-13T03:19:37.532 INFO:teuthology.orchestra.run.smithi106.stdout:mytag123 2024-04-13T03:19:37.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-13T03:19:37.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.673+0000 7f2228636640 1 Processor -- start 2024-04-13T03:19:37.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.673+0000 7f2228636640 1 -- start start 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2228636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f222006edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2228636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22200cef80 con 0x7f222006bff0 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2228636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22200cf260 con 0x7f222015fb50 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2228636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f22200cf540 con 0x7f22200d07c0 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2225bd1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f222006bff0 0x7f222006e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43830/0 (socket says 172.21.15.106:43830) 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.674+0000 7f2225bd1640 1 -- 172.21.15.106:0/1612036822 learned_addr learned my addr 172.21.15.106:0/1612036822 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600094376 0 0) 0x7f22200cef80 con 0x7f222006bff0 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f222006edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f220c003520 con 0x7f222006bff0 2024-04-13T03:19:37.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2254231523 0 0) 0x7f220c003520 con 0x7f222006bff0 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 msgr2=0x7f222006edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f222006edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 >> v1:172.21.15.114:6789/0 conn(0x7f222015fb50 legacy=0x7f222006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200cf820 con 0x7f222006bff0 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f2210001670 con 0x7f222006bff0 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 >> v1:172.21.15.106:6789/0 conn(0x7f222006bff0 legacy=0x7f222006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f220c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220c003f60 0x7f220c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22200cf540 con 0x7f220c004d40 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f220c00bb50 con 0x7f220c003f60 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f220c00bd50 con 0x7f22200d07c0 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f22263d2640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f220c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2225bd1640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220c003f60 0x7f220c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 -- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 msgr2=0x7f220c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f22200d07c0 0x7f220c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 -- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220c003f60 msgr2=0x7f220c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220c003f60 0x7f220c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.675+0000 7f2226bd3640 1 -- 172.21.15.106:0/1612036822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c003520 con 0x7f220c004d40 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.676+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2218009a90 tx=0x7f220c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6988bd51fa696433 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.676+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218039070 con 0x7f220c004d40 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.676+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221802c680 con 0x7f220c004d40 2024-04-13T03:19:37.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.676+0000 7f22253d0640 1 -- 172.21.15.106:0/1612036822 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218009690 con 0x7f220c004d40 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 -- 172.21.15.106:0/1612036822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220c004d40 msgr2=0x7f220c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 --2- 172.21.15.106:0/1612036822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2218009a90 tx=0x7f220c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2226bd3640 1 -- 172.21.15.106:0/1612036822 reap_dead start 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 -- 172.21.15.106:0/1612036822 shutdown_connections 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 -- 172.21.15.106:0/1612036822 >> 172.21.15.106:0/1612036822 conn(0x7f22200cd6a0 msgr2=0x7f22200cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 -- 172.21.15.106:0/1612036822 shutdown_connections 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.677+0000 7f2228636640 1 -- 172.21.15.106:0/1612036822 wait complete. 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.678+0000 7f2228636640 1 Processor -- start 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.678+0000 7f2228636640 1 -- start start 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 0x7f222014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22200b3940 0x7f22201e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22200ced20 con 0x7f22200b3da0 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f22200ceba0 con 0x7f22200b3940 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f222015ef10 con 0x7f220c004d40 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57006/0 (socket says 172.21.15.106:57006) 2024-04-13T03:19:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 -- 172.21.15.106:0/3220239390 learned_addr learned my addr 172.21.15.106:0/3220239390 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f22263d2640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 0x7f222014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 -- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 msgr2=0x7f222014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2225bd1640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22200b3940 0x7f22201e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 0x7f222014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 -- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22200b3940 msgr2=0x7f22201e9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22200b3940 0x7f22201e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 -- 172.21.15.106:0/3220239390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200b42b0 con 0x7f22200b3da0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f22263d2640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 0x7f222014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2226bd3640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f221802c2f0 tx=0x7f2218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2216ffd640 1 -- 172.21.15.106:0/3220239390 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218039040 con 0x7f22200b3da0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2216ffd640 1 -- 172.21.15.106:0/3220239390 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2218044070 con 0x7f22200b3da0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2216ffd640 1 -- 172.21.15.106:0/3220239390 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180044c0 con 0x7f22200b3da0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22200b4590 con 0x7f22200b3da0 2024-04-13T03:19:37.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.679+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22200b4870 con 0x7f22200b3da0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.681+0000 7f2216ffd640 1 -- 172.21.15.106:0/3220239390 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2218005990 con 0x7f22200b3da0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.681+0000 7f2216ffd640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.681+0000 7f2216ffd640 1 -- 172.21.15.106:0/3220239390 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f22180acba0 con 0x7f22200b3da0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.681+0000 7f22263d2640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.682+0000 7f22263d2640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f221c003fc0 tx=0x7f221c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.682+0000 7f222b44a740 1 Processor -- start 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 -- start start 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c0297f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c53d0 0x5621c02c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5621c0203170 con 0x5621c02984d0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621c01a97f0 con 0x5621c02c53d0 2024-04-13T03:19:37.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5621c01ff130 con 0x5621c02c4ff0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57008/0 (socket says 172.21.15.106:57008) 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f2226bd3640 1 -- 172.21.15.106:0/1723553255 learned_addr learned my addr 172.21.15.106:0/1723553255 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f22263d2640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c0297f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.683+0000 7f2225bd1640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c53d0 0x5621c02c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 -- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 msgr2=0x5621c0297f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c0297f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 -- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c53d0 msgr2=0x5621c02c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c53d0 0x5621c02c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 -- 172.21.15.106:0/1723553255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621bffdf7d0 con 0x5621c02984d0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f22263d2640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c0297f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f22180aaec0 tx=0x7f2218009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d68ff347b0f5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2203fff640 1 -- 172.21.15.106:0/1723553255 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218000e60 con 0x5621c02984d0 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2203fff640 1 -- 172.21.15.106:0/1723553255 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221807a780 con 0x5621c02984d0 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.684+0000 7f2203fff640 1 -- 172.21.15.106:0/1723553255 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221807eb30 con 0x5621c02984d0 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- 172.21.15.106:0/1723553255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 msgr2=0x5621c02c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f22180aaec0 tx=0x7f2218009750 comp rx=0 tx=0).stop 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- 172.21.15.106:0/1723553255 shutdown_connections 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02984d0 0x5621c02c82c0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c53d0 0x5621c02c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 --2- 172.21.15.106:0/1723553255 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c0297f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- 172.21.15.106:0/1723553255 >> 172.21.15.106:0/1723553255 conn(0x5621c029a200 msgr2=0x5621c02e2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- 172.21.15.106:0/1723553255 shutdown_connections 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- 172.21.15.106:0/1723553255 wait complete. 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 -- start start 2024-04-13T03:19:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.685+0000 7f222b44a740 1 Processor -- start 2024-04-13T03:19:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02e3520 0x5621c02f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5621c01ff130 con 0x5621c02c53d0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621c01bcd00 con 0x5621c02c4ff0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5621c0203170 con 0x5621c02e3520 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f222b44a740 10 client.0 _ll_register_callbacks cb 0x5621c01bc6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f2225bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f2225bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57010/0 (socket says 172.21.15.106:57010) 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f2225bd1640 1 -- 172.21.15.106:0/2690613881 learned_addr learned my addr 172.21.15.106:0/2690613881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.686+0000 7f22263d2640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2226bd3640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02e3520 0x5621c02f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 -- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02e3520 msgr2=0x5621c02f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02e3520 0x5621c02f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 -- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 msgr2=0x5621c02ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 -- 172.21.15.106:0/2690613881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621bff153f0 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f2225bd1640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2210004a40 tx=0x7f2210004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4900386e36f1eb46 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210016440 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f2210010070 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221807a780 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f22100165e0 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 10 client.9537 fetch_fsmap learned FSMap version 92 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5621bfed50f0 con 0x5621c02c53d0 2024-04-13T03:19:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221807a780 con 0x5621c02c53d0 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f2210016780 con 0x5621c02c53d0 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 10 client.9537 fetch_fsmap finished waiting for FSMap version 92 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 10 client.9537 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 20 client.9537 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02c6930 0x5621c02c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 4 client.9537 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 --> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c0210b90 con 0x5621c02c6930 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.687+0000 7f22263d2640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02c6930 0x5621c02c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f22263d2640 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02c6930 0x5621c02c6d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9b3b32742dc958 server_cookie=e447c367b5c02eb6 in_seq=0 out_seq=0 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f22017fa640 10 client.9537 ms_handle_connect on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f22017fa640 1 -- 172.21.15.106:0/2690613881 <== mds.0 v2:172.21.15.106:6835/3985285527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621c0210b90 con 0x5621c02c6930 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f22017fa640 10 client.9537 handle_command_reply: tid=0 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f222b44a740 1 client.9537 shutdown 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 msgr2=0x5621c02f2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.688+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2210004a40 tx=0x7f2210004f00 comp rx=0 tx=0).stop 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 shutdown_connections 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02c6930 0x5621c02c6d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02e3520 0x5621c02f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2fc0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f22017fa640 0 client.9537 ms_handle_reset on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 --2- 172.21.15.106:0/2690613881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 >> 172.21.15.106:0/2690613881 conn(0x5621c02984d0 msgr2=0x5621c029bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 shutdown_connections 2024-04-13T03:19:37.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 1 -- 172.21.15.106:0/2690613881 wait complete. 2024-04-13T03:19:37.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.689+0000 7f222b44a740 20 client.9537 trim_cache size 0 max 16384 2024-04-13T03:19:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.703+0000 7f222b44a740 1 Processor -- start 2024-04-13T03:19:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.703+0000 7f222b44a740 1 -- start start 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c53d0 0x5621c0306a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2220201d70 con 0x5621c02e3520 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2220200e40 con 0x5621c02c4ff0 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f22202001a0 con 0x5621c02c53d0 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f2226bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57022/0 (socket says 172.21.15.106:57022) 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f2226bd3640 1 -- 172.21.15.106:0/4078888721 learned_addr learned my addr 172.21.15.106:0/4078888721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.704+0000 7f2225bd1640 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c53d0 0x5621c0306a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f22263d2640 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 -- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c53d0 msgr2=0x5621c0306a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c53d0 0x5621c0306a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 -- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 msgr2=0x5621c02e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02e4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 -- 172.21.15.106:0/4078888721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c02f5bc0 con 0x5621c02e3520 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f2226bd3640 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f22180aaec0 tx=0x7f2218081050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575917b856e5a507 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f22037fe640 1 -- 172.21.15.106:0/4078888721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218000e60 con 0x5621c02e3520 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f22037fe640 1 -- 172.21.15.106:0/4078888721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221807a780 con 0x5621c02e3520 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f22037fe640 1 -- 172.21.15.106:0/4078888721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218083360 con 0x5621c02e3520 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 -- 172.21.15.106:0/4078888721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 msgr2=0x5621c0306fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f22180aaec0 tx=0x7f2218081050 comp rx=0 tx=0).stop 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 -- 172.21.15.106:0/4078888721 shutdown_connections 2024-04-13T03:19:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c53d0 0x5621c0306a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02e3520 0x5621c0306fc0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 --2- 172.21.15.106:0/4078888721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02c4ff0 0x5621c02e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.705+0000 7f222b44a740 1 -- 172.21.15.106:0/4078888721 >> 172.21.15.106:0/4078888721 conn(0x5621c02984d0 msgr2=0x5621c02ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.706+0000 7f222b44a740 1 -- 172.21.15.106:0/4078888721 shutdown_connections 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.706+0000 7f222b44a740 1 -- 172.21.15.106:0/4078888721 wait complete. 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.706+0000 7f222b44a740 1 -- start start 2024-04-13T03:19:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.706+0000 7f222b44a740 1 Processor -- start 2024-04-13T03:19:37.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c035bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02e3520 0x5621c02f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5621c01a97f0 con 0x5621c02c53d0 2024-04-13T03:19:37.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f22202007f0 con 0x5621c02e3520 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5621c0306920 con 0x5621c02c4ff0 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f222b44a740 10 client.0 _ll_register_callbacks cb 0x7f2210007cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57032/0 (socket says 172.21.15.106:57032) 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 -- 172.21.15.106:0/1969743841 learned_addr learned my addr 172.21.15.106:0/1969743841 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f22263d2640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c035bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2226bd3640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02e3520 0x5621c02f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 -- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 msgr2=0x5621c035bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c035bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 -- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02e3520 msgr2=0x5621c02f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02e3520 0x5621c02f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 -- 172.21.15.106:0/1969743841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c029b690 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f2225bd1640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f22100078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ee78e376cc12c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.707+0000 7f22017fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210016440 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f2210022670 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221807a780 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2210001c10 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 10 client.9543 fetch_fsmap learned FSMap version 92 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5621c0306660 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221807a780 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f22100228b0 con 0x5621c02c53d0 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 10 client.9543 fetch_fsmap finished waiting for FSMap version 92 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 10 client.9543 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 20 client.9543 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:37.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02f24c0 0x5621c0305a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 4 client.9543 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-13T03:19:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 --> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621c020aa00 con 0x5621c02f24c0 2024-04-13T03:19:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.708+0000 7f22263d2640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02f24c0 0x5621c0305a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.709+0000 7f22017fa640 10 client.9543 ms_handle_connect on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.709+0000 7f22263d2640 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02f24c0 0x5621c0305a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae06ca6dc7e3ab server_cookie=4e9ff2e9938b0b3e in_seq=0 out_seq=0 2024-04-13T03:19:37.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f22017fa640 1 -- 172.21.15.106:0/1969743841 <== mds.0 v2:172.21.15.106:6835/3985285527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621c020aa00 con 0x5621c02f24c0 2024-04-13T03:19:37.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f22017fa640 10 client.9543 handle_command_reply: tid=0 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 client.9543 shutdown 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 msgr2=0x5621c02f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f22100078f0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 shutdown_connections 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621c02e3520 0x5621c02f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5621c02c53d0 0x5621c02f2c00 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.106:6835/3985285527,v1:172.21.15.106:6837/3985285527] conn(0x5621c02f24c0 0x5621c0305a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 --2- 172.21.15.106:0/1969743841 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5621c02c4ff0 0x5621c035bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f22017fa640 0 client.9543 ms_handle_reset on v2:172.21.15.106:6835/3985285527 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 >> 172.21.15.106:0/1969743841 conn(0x5621c02984d0 msgr2=0x5621c02bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 shutdown_connections 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 1 -- 172.21.15.106:0/1969743841 wait complete. 2024-04-13T03:19:37.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.717+0000 7f222b44a740 20 client.9543 trim_cache size 0 max 16384 2024-04-13T03:19:37.720 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:37.720 INFO:teuthology.orchestra.run.smithi106.stdout: "message": "", 2024-04-13T03:19:37.720 INFO:teuthology.orchestra.run.smithi106.stdout: "return_code": 0 2024-04-13T03:19:37.720 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 msgr2=0x7f220406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f221c003fc0 tx=0x7f221c009040 comp rx=0 tx=0).stop 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 msgr2=0x7f22201e9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f221802c2f0 tx=0x7f2218004730 comp rx=0 tx=0).stop 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 shutdown_connections 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22200b3da0 0x7f22201e9e70 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22200b3940 0x7f22201e9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 --2- 172.21.15.106:0/3220239390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f220c004d40 0x7f222014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 >> 172.21.15.106:0/3220239390 conn(0x7f22200cd6a0 msgr2=0x7f2220072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 shutdown_connections 2024-04-13T03:19:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.718+0000 7f2228636640 1 -- 172.21.15.106:0/3220239390 wait complete. 2024-04-13T03:19:37.729 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:19:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.830+0000 7f5a4a2c8640 1 Processor -- start 2024-04-13T03:19:37.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.830+0000 7f5a4a2c8640 1 -- start start 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a4a2c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a4406db90 0x7f5a44072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a4a2c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4406c880 con 0x7f5a4406e020 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a4a2c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4406caf0 con 0x7f5a44145f60 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a4a2c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4406cdd0 con 0x7f5a4406db90 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a48865640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5a4406e020 0x7f5a4415ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43846/0 (socket says 172.21.15.106:43846) 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a48865640 1 -- 172.21.15.106:0/4022758492 learned_addr learned my addr 172.21.15.106:0/4022758492 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861570477 0 0) 0x7f5a4406c880 con 0x7f5a4406e020 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a2c003520 con 0x7f5a4406e020 2024-04-13T03:19:37.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a437fe640 1 --2- 172.21.15.106:0/4022758492 >> v2:172.21.15.114:3300/0 conn(0x7f5a4406db90 0x7f5a44072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3884780939 0 0) 0x7f5a2c003520 con 0x7f5a4406e020 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 >> v2:172.21.15.114:3300/0 conn(0x7f5a4406db90 msgr2=0x7f5a44072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 --2- 172.21.15.106:0/4022758492 >> v2:172.21.15.114:3300/0 conn(0x7f5a4406db90 0x7f5a44072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 >> v1:172.21.15.114:6789/0 conn(0x7f5a44145f60 legacy=0x7f5a4406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4406d0b0 con 0x7f5a4406e020 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.831+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f5a3c0019f0 con 0x7f5a4406e020 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 >> v1:172.21.15.106:6789/0 conn(0x7f5a4406e020 legacy=0x7f5a4415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 0x7f5a2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c004800 0x7f5a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a2c005210 0x7f5a2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4406cdd0 con 0x7f5a2c003a00 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c00c140 con 0x7f5a2c005210 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c00c340 con 0x7f5a2c004800 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 0x7f5a2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a48865640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a2c005210 0x7f5a2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a437fe640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c004800 0x7f5a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 -- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c004800 msgr2=0x7f5a2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c004800 0x7f5a2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 -- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a2c005210 msgr2=0x7f5a2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a2c005210 0x7f5a2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a43fff640 1 -- 172.21.15.106:0/4022758492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c003520 con 0x7f5a2c003a00 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a437fe640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c004800 0x7f5a2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a48865640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a2c005210 0x7f5a2c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.832+0000 7f5a48865640 1 -- 172.21.15.106:0/4022758492 reap_dead start 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a43fff640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 0x7f5a2c003e00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f5a34003d80 tx=0x7f5a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3771cc91459b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037070 con 0x7f5a2c003a00 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34032600 con 0x7f5a2c003a00 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a42ffd640 1 -- 172.21.15.106:0/4022758492 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34003980 con 0x7f5a2c003a00 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 msgr2=0x7f5a2c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 0x7f5a2c003e00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f5a34003d80 tx=0x7f5a2c005830 comp rx=0 tx=0).stop 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/4022758492 shutdown_connections 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/4022758492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a2c003a00 0x7f5a2c003e00 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/4022758492 >> 172.21.15.106:0/4022758492 conn(0x7f5a4406f0d0 msgr2=0x7f5a4406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:37.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/4022758492 shutdown_connections 2024-04-13T03:19:37.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.833+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/4022758492 wait complete. 2024-04-13T03:19:37.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.834+0000 7f5a4a2c8640 1 Processor -- start 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 -- start start 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 0x7f5a4413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a4413ef70 0x7f5a440c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5a440cdaa0 con 0x7f5a4415bdd0 2024-04-13T03:19:37.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44146340 con 0x7f5a4413ef70 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a4a2c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5a440cdc20 con 0x7f5a2c003a00 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.835+0000 7f5a48865640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57050/0 (socket says 172.21.15.106:57050) 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 -- 172.21.15.106:0/796245792 learned_addr learned my addr 172.21.15.106:0/796245792 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a43fff640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 0x7f5a4413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a437fe640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a4413ef70 0x7f5a440c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 -- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 msgr2=0x7f5a4413cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 0x7f5a4413cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 -- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a4413ef70 msgr2=0x7f5a440c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a4413ef70 0x7f5a440c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:37.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 -- 172.21.15.106:0/796245792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4413d500 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a43fff640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 0x7f5a4413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a48865640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5a4406e490 tx=0x7f5a3c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00ec40 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3c00ede0 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c014650 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4413d7a0 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.836+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4413dd60 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.837+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a44068a10 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.837+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a3c0147f0 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.838+0000 7f5a40ff9640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.838+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f5a3c08bb70 con 0x7f5a4415bdd0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.838+0000 7f5a43fff640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:37.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.838+0000 7f5a43fff640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 0x7f5a3006ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a34009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:37.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:37.842+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a3c059e50 con 0x7f5a4415bdd0 2024-04-13T03:19:38.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.052+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a44067aa0 con 0x7f5a4415bdd0 2024-04-13T03:19:38.053 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:19:38.053+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:19:38.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.054+0000 7f5a40ff9640 1 -- 172.21.15.106:0/796245792 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f5a3c059750 con 0x7f5a4415bdd0 2024-04-13T03:19:38.054 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.057+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 msgr2=0x7f5a3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.057+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 0x7f5a3006ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a34009030 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.057+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 msgr2=0x7f5a44143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.057+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5a4406e490 tx=0x7f5a3c00e530 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 shutdown_connections 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a4415bdd0 0x7f5a44143bf0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a4413ef70 0x7f5a440c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 --2- 172.21.15.106:0/796245792 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5a2c003a00 0x7f5a4413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 >> 172.21.15.106:0/796245792 conn(0x7f5a4406f0d0 msgr2=0x7f5a440cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 shutdown_connections 2024-04-13T03:19:38.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.058+0000 7f5a4a2c8640 1 -- 172.21.15.106:0/796245792 wait complete. 2024-04-13T03:19:38.060 INFO:tasks.ceph.mds.c.smithi106.stderr: -1809> 2024-04-13T03:19:33.548+0000 7f36deff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:38.070 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-13T03:19:38.086 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:19:38.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 -- start start 2024-04-13T03:19:38.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 Processor -- start 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc772cc00 0x561dc789a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc772dc00 0x561dc789ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561dc85da180 con 0x561dc772d800 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561dc85da300 con 0x561dc772dc00 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561dc85da480 con 0x561dc772cc00 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a43fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a43fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57078/0 (socket says 172.21.15.106:57078) 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a43fb640 1 -- 172.21.15.106:0/1664773639 learned_addr learned my addr 172.21.15.106:0/1664773639 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc772cc00 0x561dc789a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.111+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc772dc00 0x561dc789ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 -- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc772cc00 msgr2=0x561dc789a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc772cc00 0x561dc789a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 -- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc772dc00 msgr2=0x561dc789ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc772dc00 0x561dc789ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 -- 172.21.15.106:0/1664773639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc7764780 con 0x561dc772d800 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x561dc8541350 tx=0x561dc84c2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a0ce0ee1ef9e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a23f7640 1 -- 172.21.15.106:0/1664773639 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc77624e0 con 0x561dc772d800 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a23f7640 1 -- 172.21.15.106:0/1664773639 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561dc7758a80 con 0x561dc772d800 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a23f7640 1 -- 172.21.15.106:0/1664773639 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc7762b60 con 0x561dc772d800 2024-04-13T03:19:38.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc77643c0 con 0x561dc772d800 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561dc7765680 con 0x561dc772d800 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 4 waiting for MDS map... 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a23f7640 1 -- 172.21.15.106:0/1664773639 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x561dc8602600 con 0x561dc772d800 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a23f7640 1 -- 172.21.15.106:0/1664773639 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 93) ==== 2885+0+0 (secure 0 0 0) 0x561dc7789180 con 0x561dc772d800 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 4 Got MDS map 93 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 10 main: JournalTool::main 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.112+0000 7fa5a6729280 4 main: JournalTool: connecting to RADOS... 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 Processor -- start 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 -- start start 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561dc85da600 con 0x561dc85e5800 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561dc85da780 con 0x561dc85e5c00 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.113+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561dc85da900 con 0x561dc85e5400 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57080/0 (socket says 172.21.15.106:57080) 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/2942118045 learned_addr learned my addr 172.21.15.106:0/2942118045 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 msgr2=0x561dc789b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 msgr2=0x561dc789bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/2942118045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc77654a0 con 0x561dc85e5800 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x561dc85f8660 tx=0x561dc8617800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44afde44599458b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a03f3640 1 -- 172.21.15.106:0/2942118045 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc77624e0 con 0x561dc85e5800 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a03f3640 1 -- 172.21.15.106:0/2942118045 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561dc7758fc0 con 0x561dc85e5800 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a03f3640 1 -- 172.21.15.106:0/2942118045 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc7762b60 con 0x561dc85e5800 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a6729280 1 -- 172.21.15.106:0/2942118045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 msgr2=0x561dc789b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a6729280 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x561dc85f8660 tx=0x561dc8617800 comp rx=0 tx=0).stop 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a6729280 1 -- 172.21.15.106:0/2942118045 shutdown_connections 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a6729280 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.114+0000 7fa5a6729280 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 --2- 172.21.15.106:0/2942118045 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 -- 172.21.15.106:0/2942118045 >> 172.21.15.106:0/2942118045 conn(0x561dc85e5000 msgr2=0x561dc7819000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 -- 172.21.15.106:0/2942118045 shutdown_connections 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 -- 172.21.15.106:0/2942118045 wait complete. 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 Processor -- start 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.115+0000 7fa5a6729280 1 -- start start 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561dc85dbc80 con 0x561dc85e5400 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561dc85dbb00 con 0x561dc85e5800 2024-04-13T03:19:38.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561dc85db980 con 0x561dc85e5c00 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57092/0 (socket says 172.21.15.106:57092) 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 -- 172.21.15.106:0/35192369 learned_addr learned my addr 172.21.15.106:0/35192369 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 -- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5c00 msgr2=0x561dc789bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 -- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5800 msgr2=0x561dc789b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 -- 172.21.15.106:0/35192369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc7764780 con 0x561dc85e5400 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x561dc85f9290 tx=0x561dc84c0660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa59ebf0640 1 -- 172.21.15.106:0/35192369 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc77629c0 con 0x561dc85e5400 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa59ebf0640 1 -- 172.21.15.106:0/35192369 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561dc7758e00 con 0x561dc85e5400 2024-04-13T03:19:38.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa59ebf0640 1 -- 172.21.15.106:0/35192369 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc77624e0 con 0x561dc85e5400 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561dc77643c0 con 0x561dc85e5400 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc7765680 con 0x561dc85e5400 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.116+0000 7fa5a6729280 4 main: JournalTool: resolving pool 17 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa59ebf0640 1 -- 172.21.15.106:0/35192369 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x561dc8676c00 con 0x561dc85e5400 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa59ebf0640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 0x561dc789c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa59ebf0640 1 -- 172.21.15.106:0/35192369 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x561dc8670c00 con 0x561dc85e5400 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa5a6729280 4 main: JournalTool: creating IoCtx.. 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa5a6729280 4 main: Executing for rank 0 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 0x561dc789c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.118+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 0x561dc789c100 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x561dc85f8660 tx=0x561dc84c0600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 -- start start 2024-04-13T03:19:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 Processor -- start 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc866f400 0x561dc789c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 0x561dc789cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc866fc00 0x561dc789d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561dc85db680 con 0x561dc866f800 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561dc85da600 con 0x561dc866f400 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a6729280 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561dc85da780 con 0x561dc866fc00 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a4bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 0x561dc789cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a4bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 0x561dc789cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57096/0 (socket says 172.21.15.106:57096) 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.119+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 learned_addr learned my addr 172.21.15.106:0/3878013708 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc866fc00 0x561dc789d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc866f400 0x561dc789c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc866fc00 msgr2=0x561dc789d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc866fc00 0x561dc789d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc866f400 msgr2=0x561dc789c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc866f400 0x561dc789c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc7764000 con 0x561dc866f800 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc866f400 0x561dc789c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 0x561dc789cc00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x561dc7a676b0 tx=0x561dc84c0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8e10caf3a91a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa59bbea640 1 -- 172.21.15.106:0/3878013708 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc7762000 con 0x561dc866f800 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa59bbea640 1 -- 172.21.15.106:0/3878013708 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561dc77581c0 con 0x561dc866f800 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa59bbea640 1 -- 172.21.15.106:0/3878013708 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561dc869c1a0 con 0x561dc866f800 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc77654a0 con 0x561dc866f800 2024-04-13T03:19:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561dc7764000 con 0x561dc866f800 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 4 waiting for MDS map... 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa59bbea640 1 -- 172.21.15.106:0/3878013708 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x561dc7aeca00 con 0x561dc866f800 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa59bbea640 1 -- 172.21.15.106:0/3878013708 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 93) ==== 2885+0+0 (secure 0 0 0) 0x561dc7789500 con 0x561dc866f800 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 4 Got MDS map 93 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x561dc7afb400 0x561dc789d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7afb800 con 0x561dc7afb400 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.120+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x561dc7afb400 0x561dc789d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.121+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x561dc7afb400 0x561dc789d700 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d98f36e62c8141c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.122+0000 7fa5a53fd640 1 -- 172.21.15.106:0/3878013708 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v123'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x561dc782e000 con 0x561dc7afb400 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.122+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x561dc7afbc00 0x561dc7b24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.122+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b28000 con 0x561dc7afbc00 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.122+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x561dc7afbc00 0x561dc7b24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.122+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x561dc7afbc00 0x561dc7b24000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=29062ec985bb5020 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v123'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x561dc782e500 con 0x561dc7afbc00 2024-04-13T03:19:38.134 INFO:teuthology.orchestra.run.smithi106.stdout:writing EResetJournal entry 2024-04-13T03:19:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b28400 con 0x561dc7afbc00 2024-04-13T03:19:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x561dc7b28800 0x561dc7b24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b28c00 con 0x561dc7b28800 2024-04-13T03:19:38.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561dc7b29000 0x561dc7b24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b29400 con 0x561dc7b29000 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x561dc7b29800 0x561dc7b25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b29c00 con 0x561dc7b29800 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b2e000 con 0x561dc7b29000 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.125+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b2e400 con 0x561dc7afb400 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.126+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561dc7b29000 0x561dc7b24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.126+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x561dc7b28800 0x561dc7b24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.126+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x561dc7b29800 0x561dc7b25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.126+0000 7fa5a4bfc640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561dc7b29000 0x561dc7b24b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=4ec0fcc105212a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.126+0000 7fa5a53fd640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x561dc7b28800 0x561dc7b24580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e322a0a260390ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.127+0000 7fa5a43fb640 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x561dc7b29800 0x561dc7b25080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=6f6eb8f7deaa1a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.127+0000 7fa5a53fd640 1 -- 172.21.15.106:0/3878013708 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(8 200.00000006 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x561dc782e000 con 0x561dc7afb400 2024-04-13T03:19:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.127+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(3 200.00000001 [delete] v123'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x561dc782e500 con 0x561dc7afbc00 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.128+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(5 200.00000003 [delete] v123'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x561dc782e780 con 0x561dc7b29000 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.128+0000 7fa5a4bfc640 1 -- 172.21.15.106:0/3878013708 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(7 200.00000005 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x561dc782e780 con 0x561dc7b29000 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.128+0000 7fa5a53fd640 1 -- 172.21.15.106:0/3878013708 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(4 200.00000002 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x561dc782e000 con 0x561dc7b28800 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.128+0000 7fa59a3e7640 1 -- 172.21.15.106:0/3878013708 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561dc7b30000 con 0x561dc7afbc00 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.130+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v123'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x561dc782e500 con 0x561dc7afbc00 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(6 200.00000004 [delete] v123'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x561dc782e500 con 0x561dc7b29800 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561dc7b29000 msgr2=0x561dc7b24b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x561dc7b29000 0x561dc7b24b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x561dc7afb400 msgr2=0x561dc789d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x561dc7afb400 0x561dc789d700 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x561dc7afbc00 msgr2=0x561dc7b24000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x561dc7afbc00 0x561dc7b24000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x561dc7b28800 msgr2=0x561dc7b24580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x561dc7b28800 0x561dc7b24580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 reap_dead start 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x561dc7b29800 msgr2=0x561dc7b25080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x561dc7b29800 0x561dc7b25080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 msgr2=0x561dc789cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/3878013708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc866f800 0x561dc789cc00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x561dc7a676b0 tx=0x561dc84c0a80 comp rx=0 tx=0).stop 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a43fb640 1 -- 172.21.15.106:0/3878013708 reap_dead start 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 shutdown_connections 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 >> 172.21.15.106:0/3878013708 conn(0x561dc866f000 msgr2=0x561dc7819800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 shutdown_connections 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/3878013708 wait complete. 2024-04-13T03:19:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 msgr2=0x561dc789c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 0x561dc789c100 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x561dc85f8660 tx=0x561dc84c0600 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 msgr2=0x561dc789b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x561dc85f9290 tx=0x561dc84c0660 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 shutdown_connections 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x561dc866ec00 0x561dc789c100 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc85e5c00 0x561dc789bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc85e5800 0x561dc789b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 --2- 172.21.15.106:0/35192369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc85e5400 0x561dc789b080 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 >> 172.21.15.106:0/35192369 conn(0x561dc85e5000 msgr2=0x561dc7819000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 shutdown_connections 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.131+0000 7fa5a6729280 1 -- 172.21.15.106:0/35192369 wait complete. 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 msgr2=0x561dc789a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x561dc8541350 tx=0x561dc84c2120 comp rx=0 tx=0).stop 2024-04-13T03:19:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 shutdown_connections 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561dc772dc00 0x561dc789ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561dc772d800 0x561dc789a580 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 --2- 172.21.15.106:0/1664773639 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561dc772cc00 0x561dc789a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 >> 172.21.15.106:0/1664773639 conn(0x561dc772d000 msgr2=0x561dc7818800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 shutdown_connections 2024-04-13T03:19:38.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.132+0000 7fa5a6729280 1 -- 172.21.15.106:0/1664773639 wait complete. 2024-04-13T03:19:38.144 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.074349, result: writing EResetJournal entry 2024-04-13T03:19:38.144 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-13T03:19:38.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.217+0000 7f5bfd825740 1 -- start start 2024-04-13T03:19:38.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.217+0000 7f5bfd825740 1 Processor -- start 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f20167800 0x560f20f3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f20167c00 0x560f20f3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3e180 con 0x560f20166c00 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3e300 con 0x560f20167c00 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560f20f3e480 con 0x560f20167800 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57116/0 (socket says 172.21.15.106:57116) 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2514656432 learned_addr learned my addr 172.21.15.106:0/2514656432 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f20167c00 0x560f20f3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f20167800 0x560f20f3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f20167800 msgr2=0x560f20f3a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f20167800 0x560f20f3a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f20167c00 msgr2=0x560f20f3ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f20167c00 0x560f20f3ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2514656432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f20193680 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x560f20f898c0 tx=0x560f20f04120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf035e09f032b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bf89f7640 1 -- 172.21.15.106:0/2514656432 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019c4e0 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bf89f7640 1 -- 172.21.15.106:0/2514656432 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f201b21c0 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bf89f7640 1 -- 172.21.15.106:0/2514656432 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019cb60 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f20193860 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560f201932c0 con 0x560f20166c00 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bfd825740 4 waiting for MDS map... 2024-04-13T03:19:38.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.218+0000 7f5bf89f7640 1 -- 172.21.15.106:0/2514656432 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x560f21042600 con 0x560f20166c00 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bf89f7640 1 -- 172.21.15.106:0/2514656432 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 93) ==== 2885+0+0 (secure 0 0 0) 0x560f201c1180 con 0x560f20166c00 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bfd825740 4 Got MDS map 93 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bfd825740 4 datascan.main: Using metadata pool output 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bfd825740 4 datascan.main: connecting to RADOS... 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bfd825740 1 Processor -- start 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.219+0000 7f5bfd825740 1 -- start start 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3e600 con 0x560f21045c00 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3e780 con 0x560f21045400 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560f20f3e900 con 0x560f21045800 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57130/0 (socket says 172.21.15.106:57130) 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfb9fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/4246736390 learned_addr learned my addr 172.21.15.106:0/4246736390 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045800 msgr2=0x560f20f3b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045400 msgr2=0x560f20f3b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/4246736390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f201934a0 con 0x560f21045c00 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x560f2102ec90 tx=0x560f20f00a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29611fcca981c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bf69f3640 1 -- 172.21.15.106:0/4246736390 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019c9c0 con 0x560f21045c00 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.220+0000 7f5bf69f3640 1 -- 172.21.15.106:0/4246736390 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f201b2e00 con 0x560f21045c00 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bf69f3640 1 -- 172.21.15.106:0/4246736390 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019c4e0 con 0x560f21045c00 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 -- 172.21.15.106:0/4246736390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 msgr2=0x560f20f3bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x560f2102ec90 tx=0x560f20f00a20 comp rx=0 tx=0).stop 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 -- 172.21.15.106:0/4246736390 shutdown_connections 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 --2- 172.21.15.106:0/4246736390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 -- 172.21.15.106:0/4246736390 >> 172.21.15.106:0/4246736390 conn(0x560f21045000 msgr2=0x560f2024d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 -- 172.21.15.106:0/4246736390 shutdown_connections 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 -- 172.21.15.106:0/4246736390 wait complete. 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.221+0000 7f5bfd825740 1 Processor -- start 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 -- start start 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3fb00 con 0x560f21045800 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560f20f3f980 con 0x560f21045c00 2024-04-13T03:19:38.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfd825740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x560f20f3f800 con 0x560f21045400 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57132/0 (socket says 172.21.15.106:57132) 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 learned_addr learned my addr 172.21.15.106:0/2000732658 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045400 msgr2=0x560f20f3b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f21045400 0x560f20f3b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045c00 msgr2=0x560f20f3bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f21045c00 0x560f20f3bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.222+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f20193680 con 0x560f21045800 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x560f2102f9e0 tx=0x560f20f00600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bf51f0640 1 -- 172.21.15.106:0/2000732658 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019c340 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bf51f0640 1 -- 172.21.15.106:0/2000732658 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f201b2c40 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f20193860 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bf51f0640 1 -- 172.21.15.106:0/2000732658 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f2019c9c0 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f201930e0 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.223+0000 7f5bfd825740 4 datascan.init: resolving metadata pool 17 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bf51f0640 1 -- 172.21.15.106:0/2000732658 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x560f2103ac00 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bf51f0640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 0x560f20f3c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bf51f0640 1 -- 172.21.15.106:0/2000732658 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x560f20f6a600 con 0x560f21045800 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfd825740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 0x560f20f3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfd825740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfd825740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x560f20f6d400 0x560f20f3c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6d800 con 0x560f20f6d400 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.224+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x560f20f6d400 0x560f20f3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.225+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 0x560f20f3c100 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x560f20f02f00 tx=0x560f2047aa80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.225+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x560f20f6d400 0x560f20f3c680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.225+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.225+0000 7f5bf61f2640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x560f20f55400 0x560f204f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.225+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f55000 con 0x560f20f55400 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x560f20f55400 0x560f204f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x560f20f55400 0x560f204f8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bf69f3640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x560f20f54800 0x560f204f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20472c00 con 0x560f20f54800 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.226+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x560f20f54800 0x560f204f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.227+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x560f20f54800 0x560f204f8580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.227+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20f54800 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.227+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f54c00 con 0x560f20f6d400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.227+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.227+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6d800 con 0x560f20f6d400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20f55400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bf69f3640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560f20473c00 0x560f204f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a000 con 0x560f20473c00 2024-04-13T03:19:38.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560f20473c00 0x560f204f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560f20473c00 0x560f204f8b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bf61f2640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x560f20473400 0x560f204f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.228+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473800 con 0x560f20473400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x560f20473400 0x560f204f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x560f20473400 0x560f204f9600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a400 con 0x560f20f6d400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.229+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20472800 con 0x560f20f6d400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a800 con 0x560f20f54800 2024-04-13T03:19:38.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20f54800 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20f6d400 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050ac00 con 0x560f20f55400 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(13 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bf61f2640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x560f20473800 0x560f204f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f55000 con 0x560f20473800 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.230+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x560f20473800 0x560f204f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bfb1fc640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x560f20473800 0x560f204f9b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a000 con 0x560f20473c00 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(15 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bf61f2640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560f2050a400 0x560f204fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.231+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050b000 con 0x560f2050a400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.232+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560f2050a400 0x560f204fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.232+0000 7f5bfa9fb640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560f2050a400 0x560f204fa100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.232+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.232+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20472800 con 0x560f2050a400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050b400 con 0x560f20473400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20473800 2024-04-13T03:19:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(19 [pgnls start_epoch 123 out=81b] v122'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6dc00 con 0x560f20473800 2024-04-13T03:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfd825740 10 datascan.operator(): handling object 1099511627778.0 2024-04-13T03:19:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.233+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x560f20472c00 con 0x560f20473800 2024-04-13T03:19:38.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(21 10000000002.00000000 [call] v123'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20473400 2024-04-13T03:19:38.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6d800 con 0x560f20473400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(23 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bf61f2640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x560f2050b800 0x560f204fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050bc00 con 0x560f2050b800 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.235+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x560f2050b800 0x560f204fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bfb9fd640 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x560f2050b800 0x560f204fa680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(24 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f54c00 con 0x560f20473400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 5 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050ac00 con 0x560f20f6d400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(26 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.236+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c000 con 0x560f2050a400 2024-04-13T03:19:38.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(27 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050b400 con 0x560f20473800 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 5 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c400 con 0x560f20473400 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.237+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050bc00 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.238+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=81b] v121'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.238+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560f20f55000 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.238+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.238+0000 7f5bfd825740 10 datascan.operator(): handling object 1099511628279.0 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.238+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a800 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.239+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v123'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.239+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6cc00 con 0x560f20f6d400 2024-04-13T03:19:38.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.239+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(33 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.239+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6d800 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.239+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 7 ==== osd_op_reply(34 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c800 con 0x560f20f55400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(35 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 8 ==== osd_op_reply(36 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052cc00 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 9 ==== osd_op_reply(37 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.240+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a000 con 0x560f20f55400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.241+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(38 [pgnls start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.241+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052d000 con 0x560f20473c00 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.241+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(39 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.241+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050b000 con 0x560f2050b800 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.242+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(40 [pgnls start_epoch 123 out=81b] v121'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.242+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6c800 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.242+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.242+0000 7f5bfd825740 10 datascan.operator(): handling object 1099511627780.1 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.242+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c400 con 0x560f20473c00 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(42 10000000004.00000000 [call] v123'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c800 con 0x560f20473400 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 10 ==== osd_op_reply(43 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050ac00 con 0x560f2050a400 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(44 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.244+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052d400 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.245+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(45 [pgnls start_epoch 123 out=81b] v122'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.245+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560f2052cc00 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.245+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.245+0000 7f5bfd825740 10 datascan.operator(): handling object 1099511628280.0 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.245+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c000 con 0x560f2050b800 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.246+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v123'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.246+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052d800 con 0x560f20f6d400 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 9 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c800 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(49 [pgnls start_epoch 123 out=81b] v123'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560f20f54c00 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfd825740 10 datascan.operator(): handling object 1099511627780.0 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.247+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x560f20f55000 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.248+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 10 ==== osd_op_reply(51 10000000004.00000000 [call] v123'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.248+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052d000 con 0x560f20f54800 2024-04-13T03:19:38.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.248+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20f54800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.248+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052d400 con 0x560f20f55400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.249+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(53 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.249+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050bc00 con 0x560f2050b800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.249+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 7 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.249+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052cc00 con 0x560f2050b800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 8 ==== osd_op_reply(55 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20473000 con 0x560f20f54800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20f54800 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052c000 con 0x560f20f6d400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.0 v2:172.21.15.106:6808/3573709684 10 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20f6d400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20472800 con 0x560f20473400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.250+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.3 v2:172.21.15.106:6811/987945743 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20473400 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6c800 con 0x560f20473c00 2024-04-13T03:19:38.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20f6d800 con 0x560f20473800 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050a000 con 0x560f2050a400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.251+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2052dc00 con 0x560f20473c00 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 12 ==== osd_op_reply(62 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f2050b000 con 0x560f20f55400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20532000 con 0x560f2050a400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 6 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.252+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20534000 con 0x560f20473c00 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 13 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20532400 con 0x560f2050b800 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 9 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20534400 con 0x560f20473800 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bfb1fc640 1 -- 172.21.15.106:0/2000732658 <== osd.2 v2:172.21.15.106:6824/4087145286 7 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8780 con 0x560f20473800 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.253+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20532800 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20534800 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 9 ==== osd_op_reply(69 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20532c00 con 0x560f2050a400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 7 ==== osd_op_reply(70 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.254+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20534c00 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.5 v2:172.21.15.114:6808/1239989021 10 ==== osd_op_reply(71 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f20f55400 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20533000 con 0x560f20473c00 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.1 v2:172.21.15.106:6800/1701180838 14 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f20473c00 2024-04-13T03:19:38.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bf69f3640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20535000 con 0x560f2050a400 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 <== osd.7 v2:172.21.15.114:6824/300957724 8 ==== osd_op_reply(73 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8000 con 0x560f2050a400 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.255+0000 7f5bf61f2640 1 -- 172.21.15.106:0/2000732658 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x560f20533400 con 0x560f2050b800 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfb9fd640 1 -- 172.21.15.106:0/2000732658 <== osd.4 v2:172.21.15.114:6800/1124906481 10 ==== osd_op_reply(74 [pgnls start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560f20ef8500 con 0x560f2050b800 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x560f20f6d400 msgr2=0x560f20f3c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x560f20f6d400 0x560f20f3c680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560f20473c00 msgr2=0x560f204f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x560f20473c00 0x560f204f8b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x560f20473800 msgr2=0x560f204f9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x560f20473800 0x560f204f9b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x560f20473400 msgr2=0x560f204f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x560f20473400 0x560f204f9600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x560f2050b800 msgr2=0x560f204fa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x560f2050b800 0x560f204fa680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x560f20f55400 msgr2=0x560f204f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x560f20f55400 0x560f204f8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x560f20f54800 msgr2=0x560f204f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x560f20f54800 0x560f204f8580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 reap_dead start 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560f2050a400 msgr2=0x560f204fa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x560f2050a400 0x560f204fa100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfa9fb640 1 -- 172.21.15.106:0/2000732658 reap_dead start 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 msgr2=0x560f20f3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 0x560f20f3c100 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x560f20f02f00 tx=0x560f2047aa80 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 msgr2=0x560f20f3b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x560f2102f9e0 tx=0x560f20f00600 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 shutdown_connections 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x560f20f6d000 0x560f20f3c100 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2000732658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f21045800 0x560f20f3b600 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 >> 172.21.15.106:0/2000732658 conn(0x560f21045000 msgr2=0x560f2024d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 shutdown_connections 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2000732658 wait complete. 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 msgr2=0x560f20f3a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x560f20f898c0 tx=0x560f20f04120 comp rx=0 tx=0).stop 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 shutdown_connections 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x560f20167800 0x560f20f3a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560f20167c00 0x560f20f3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 --2- 172.21.15.106:0/2514656432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f20166c00 0x560f20f3a000 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 >> 172.21.15.106:0/2514656432 conn(0x560f20167000 msgr2=0x560f2024c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 shutdown_connections 2024-04-13T03:19:38.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.256+0000 7f5bfd825740 1 -- 172.21.15.106:0/2514656432 wait complete. 2024-04-13T03:19:38.278 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.133353, result: 2024-04-13T03:19:38.278 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-13T03:19:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 -- start start 2024-04-13T03:19:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 Processor -- start 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84bf12c00 0x55c84cce2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84bf13c00 0x55c84cce2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c84ccfa600 con 0x55c84bf13800 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c84ccfa480 con 0x55c84bf13c00 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c84ccfa300 con 0x55c84bf12c00 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f583fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f583fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57154/0 (socket says 172.21.15.106:57154) 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.346+0000 7f5f583fb640 1 -- 172.21.15.106:0/192130259 learned_addr learned my addr 172.21.15.106:0/192130259 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f593fd640 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84bf13c00 0x55c84cce2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 -- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84bf12c00 msgr2=0x55c84cce2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84bf12c00 0x55c84cce2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 -- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84bf13c00 msgr2=0x55c84cce2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84bf13c00 0x55c84cce2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 -- 172.21.15.106:0/192130259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84bf3f680 con 0x55c84bf13800 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f583fb640 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x55c84cd31980 tx=0x55c84ccacc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4236ecf93f8418 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f563f7640 1 -- 172.21.15.106:0/192130259 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf484e0 con 0x55c84bf13800 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f563f7640 1 -- 172.21.15.106:0/192130259 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c84bf5ec40 con 0x55c84bf13800 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f563f7640 1 -- 172.21.15.106:0/192130259 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf48b60 con 0x55c84bf13800 2024-04-13T03:19:38.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c84bf3f860 con 0x55c84bf13800 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c84bf3f2c0 con 0x55c84bf13800 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 4 waiting for MDS map... 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f563f7640 1 -- 172.21.15.106:0/192130259 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55c84cde6600 con 0x55c84bf13800 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f563f7640 1 -- 172.21.15.106:0/192130259 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 93) ==== 2885+0+0 (secure 0 0 0) 0x55c84bf6d180 con 0x55c84bf13800 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 4 Got MDS map 93 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 4 datascan.main: Using metadata pool output 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.347+0000 7f5f5b01e740 4 datascan.main: connecting to RADOS... 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 Processor -- start 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 -- start start 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c84ccfa180 con 0x55c84cdedc00 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c84bf39800 con 0x55c84cded800 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.348+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c84bf38a80 con 0x55c84cded400 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57166/0 (socket says 172.21.15.106:57166) 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 -- 172.21.15.106:0/3075714119 learned_addr learned my addr 172.21.15.106:0/3075714119 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f593fd640 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 -- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 msgr2=0x55c84cce3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 -- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cded800 msgr2=0x55c84cce3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 -- 172.21.15.106:0/3075714119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84bf3f4a0 con 0x55c84cdedc00 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f583fb640 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x55c84cddeb70 tx=0x55c84bfc8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee7d28e8fc26bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f543f3640 1 -- 172.21.15.106:0/3075714119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf48340 con 0x55c84cdedc00 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f543f3640 1 -- 172.21.15.106:0/3075714119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c84bf5ea80 con 0x55c84cdedc00 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f543f3640 1 -- 172.21.15.106:0/3075714119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf484e0 con 0x55c84cdedc00 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f5b01e740 1 -- 172.21.15.106:0/3075714119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 msgr2=0x55c84cce3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x55c84cddeb70 tx=0x55c84bfc8cc0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f5b01e740 1 -- 172.21.15.106:0/3075714119 shutdown_connections 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.349+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/3075714119 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 -- 172.21.15.106:0/3075714119 >> 172.21.15.106:0/3075714119 conn(0x55c84cded000 msgr2=0x55c84bff6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 -- 172.21.15.106:0/3075714119 shutdown_connections 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 -- 172.21.15.106:0/3075714119 wait complete. 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 Processor -- start 2024-04-13T03:19:38.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.350+0000 7f5f5b01e740 1 -- start start 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c84ccfb380 con 0x55c84cded800 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c84ccfb500 con 0x55c84cdedc00 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x55c84ccfa180 con 0x55c84cded400 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57168/0 (socket says 172.21.15.106:57168) 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 learned_addr learned my addr 172.21.15.106:0/2862973320 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 msgr2=0x55c84cce3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84cded400 0x55c84cce3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cdedc00 msgr2=0x55c84cce3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84cdedc00 0x55c84cce3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84bf3f680 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x55c84cddf830 tx=0x55c84ce06d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f52bf0640 1 -- 172.21.15.106:0/2862973320 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf489c0 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f52bf0640 1 -- 172.21.15.106:0/2862973320 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c84bf5f500 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c84bf3f860 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f52bf0640 1 -- 172.21.15.106:0/2862973320 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c84bf48340 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c84bf3f0e0 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.351+0000 7f5f5b01e740 4 datascan.init: resolving metadata pool 17 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f52bf0640 1 -- 172.21.15.106:0/2862973320 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55c84ce54c00 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f52bf0640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 0x55c84cce4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f52bf0640 1 -- 172.21.15.106:0/2862973320 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55c84ce6e600 con 0x55c84cded800 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f5b01e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:19:38.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f5b01e740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f5b01e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 0x55c84cce4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c84ce53400 0x55c84cce4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce53800 con 0x55c84ce53400 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c84ce53400 0x55c84cce4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 0x55c84cce4100 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55c84cd31890 tx=0x55c84c266540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.353+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c84ce53400 0x55c84cce4680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.354+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.354+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c84ce53c00 0x55c84cce4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.354+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84ce53c00 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.354+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c84ce53c00 0x55c84cce4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.354+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c84ce53c00 0x55c84cce4c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c84ce52800 0x55c84cce5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84ce52800 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c84ce52800 0x55c84cce5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c84ce52800 0x55c84cce5180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f53bf2640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c84c2a0c00 0x55c84c2b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.355+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1000 con 0x55c84c2a0c00 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.356+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c84c2a0c00 0x55c84c2b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.356+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c84c2a0c00 0x55c84c2b8000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.356+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.356+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1800 con 0x55c84ce53c00 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1400 con 0x55c84ce52800 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1c00 con 0x55c84ce52800 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16000 con 0x55c84c2a0c00 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f543f3640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c84cd16c00 0x55c84c2b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17000 con 0x55c84cd16c00 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.357+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c84cd16c00 0x55c84c2b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c84cd16c00 0x55c84c2b8580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16400 con 0x55c84ce53400 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17400 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.358+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16800 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1400 con 0x55c84ce53c00 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17400 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.359+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84c2a0c00 2024-04-13T03:19:38.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f53bf2640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c84c2a1c00 0x55c84c2b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a0800 con 0x55c84c2a1c00 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c84c2a1c00 0x55c84c2b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f58bfc640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c84c2a1c00 0x55c84c2b8b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84cd16c00 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f53bf2640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c84c2a1800 0x55c84c2b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.360+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17000 con 0x55c84c2a1800 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.361+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c84c2a1800 0x55c84c2b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.361+0000 7f5f583fb640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c84c2a1800 0x55c84c2b9080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.361+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.361+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ccfd400 con 0x55c84c2a1800 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.361+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17800 con 0x55c84ce53400 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16000 con 0x55c84c2a1c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 123 out=81b] v123'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52000 con 0x55c84ce52800 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.362+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16400 con 0x55c84c2a0c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17c00 con 0x55c84ce53c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ccfd400 con 0x55c84ce52800 2024-04-13T03:19:38.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16800 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.363+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84c2a0c00 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17400 con 0x55c84cd16c00 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1000 con 0x55c84ce53400 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16000 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.364+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17000 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ccfd400 con 0x55c84ce53c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17800 con 0x55c84ce52800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.365+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52000 con 0x55c84c2a0c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ccfd400 con 0x55c84c2a1c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a0800 con 0x55c84cd16c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17c00 con 0x55c84c2a1800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.366+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce000 con 0x55c84c2a1800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17400 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce400 con 0x55c84c2a1c00 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 123 out=49b] v123'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce800 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f543f3640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c84c2a1400 0x55c84c2b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.367+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce000 con 0x55c84c2a1400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.368+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c84c2a1400 0x55c84c2b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.368+0000 7f5f593fd640 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c84c2a1400 0x55c84c2b9600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.368+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.368+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cec00 con 0x55c84ce53400 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84ce52800 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf000 con 0x55c84c2a1800 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52000 con 0x55c84c2a1c00 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.369+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf400 con 0x55c84ce53400 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84cd16c00 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 123 out=81b] v123'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 10 datascan.operator(): handling object 100000001f7.0 2024-04-13T03:19:38.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84cdec800 con 0x55c84cd16c00 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.370+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce400 con 0x55c84ce52800 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84ce53400 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-13T03:19:38.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.get_frag_of: frag is * 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84ccfd400 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16800 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 20 datascan.read_dentry: bravo_head not found in result 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.371+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16000 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v123'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f5b01e740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-13T03:19:38.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17000 con 0x55c84ce52800 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16400 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.373+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf800 con 0x55c84c2a0c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17c00 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cfc00 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.374+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17400 con 0x55c84c2a0c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce800 con 0x55c84cd16c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1000 con 0x55c84c2a1400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 123 out=49b] v121'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.375+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cec00 con 0x55c84ce53400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce000 con 0x55c84c2a1800 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf000 con 0x55c84c2a1400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 123 out=49b] v123'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.376+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce800 con 0x55c84ce52800 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf400 con 0x55c84cd16c00 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 123 out=49b] v123'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84ce53c00 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.377+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf800 con 0x55c84c2a0c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce53800 con 0x55c84c2a1400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cfc00 con 0x55c84c2a1400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cdec800 con 0x55c84ce53c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.6 v2:172.21.15.114:6816/1761123940 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84ce53c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.378+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16400 con 0x55c84ce52800 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.0 v2:172.21.15.106:6808/3573709684 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84ce52800 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a0800 con 0x55c84ce53400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.3 v2:172.21.15.106:6811/987945743 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84ce53400 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17c00 con 0x55c84cd16c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce400 con 0x55c84c2a1c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.379+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52400 con 0x55c84c2a1800 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52000 con 0x55c84cd16c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cdec800 con 0x55c84c2a0c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.380+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce52c00 con 0x55c84c2a1800 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2ce800 con 0x55c84cd16c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84ce53800 con 0x55c84c2a1400 2024-04-13T03:19:38.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.381+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf000 con 0x55c84c2a1c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 <== osd.2 v2:172.21.15.106:6824/4087145286 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4780 con 0x55c84c2a1c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2a1000 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cf400 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16800 con 0x55c84c2a1800 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.382+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cfc00 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.5 v2:172.21.15.114:6808/1239989021 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a0c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd16000 con 0x55c84cd16c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.1 v2:172.21.15.106:6800/1701180838 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84cd16c00 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f53bf2640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84c2cec00 con 0x55c84c2a1800 2024-04-13T03:19:38.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f583fb640 1 -- 172.21.15.106:0/2862973320 <== osd.7 v2:172.21.15.114:6824/300957724 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4000 con 0x55c84c2a1800 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f543f3640 1 -- 172.21.15.106:0/2862973320 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55c84cd17000 con 0x55c84c2a1400 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.383+0000 7f5f593fd640 1 -- 172.21.15.106:0/2862973320 <== osd.4 v2:172.21.15.114:6800/1124906481 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c84cca4500 con 0x55c84c2a1400 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c84ce52800 msgr2=0x55c84cce5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x55c84ce52800 0x55c84cce5180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c84cd16c00 msgr2=0x55c84c2b8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x55c84cd16c00 0x55c84c2b8580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c84c2a1c00 msgr2=0x55c84c2b8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x55c84c2a1c00 0x55c84c2b8b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c84ce53400 msgr2=0x55c84cce4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x55c84ce53400 0x55c84cce4680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 reap_dead start 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c84c2a1400 msgr2=0x55c84c2b9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x55c84c2a1400 0x55c84c2b9600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c84c2a0c00 msgr2=0x55c84c2b8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x55c84c2a0c00 0x55c84c2b8000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c84ce53c00 msgr2=0x55c84cce4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x55c84ce53c00 0x55c84cce4c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c84c2a1800 msgr2=0x55c84c2b9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x55c84c2a1800 0x55c84c2b9080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f58bfc640 1 -- 172.21.15.106:0/2862973320 reap_dead start 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 msgr2=0x55c84cce4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 0x55c84cce4100 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55c84cd31890 tx=0x55c84c266540 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 msgr2=0x55c84cce3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x55c84cddf830 tx=0x55c84ce06d20 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 shutdown_connections 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x55c84ce53000 0x55c84cce4100 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/2862973320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84cded800 0x55c84cce3600 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 >> 172.21.15.106:0/2862973320 conn(0x55c84cded000 msgr2=0x55c84bff6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 shutdown_connections 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/2862973320 wait complete. 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 msgr2=0x55c84cce2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x55c84cd31980 tx=0x55c84ccacc60 comp rx=0 tx=0).stop 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 shutdown_connections 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c84bf13c00 0x55c84cce2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c84bf13800 0x55c84cce2580 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 --2- 172.21.15.106:0/192130259 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x55c84bf12c00 0x55c84cce2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 >> 172.21.15.106:0/192130259 conn(0x55c84bf13000 msgr2=0x55c84bff7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 shutdown_connections 2024-04-13T03:19:38.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.384+0000 7f5f5b01e740 1 -- 172.21.15.106:0/192130259 wait complete. 2024-04-13T03:19:38.407 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.128636, result: 2024-04-13T03:19:38.407 DEBUG:teuthology.orchestra.run.smithi106:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-13T03:19:38.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.467+0000 7f7c85c2c740 1 -- start start 2024-04-13T03:19:38.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 Processor -- start 2024-04-13T03:19:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ee59f800 0x5594ef372580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ee59fc00 0x5594ef372b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37a180 con 0x5594ee59ec00 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37a300 con 0x5594ee59fc00 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5594ef37a480 con 0x5594ee59f800 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c833fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c833fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57188/0 (socket says 172.21.15.106:57188) 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c833fc640 1 -- 172.21.15.106:0/847738141 learned_addr learned my addr 172.21.15.106:0/847738141 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ee59fc00 0x5594ef372b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.468+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ee59f800 0x5594ef372580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 -- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ee59f800 msgr2=0x5594ef372580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ee59f800 0x5594ef372580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 -- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ee59fc00 msgr2=0x5594ef372b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ee59fc00 0x5594ef372b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 -- 172.21.15.106:0/847738141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee5cb680 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c833fc640 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x5594ef3c54a0 tx=0x5594ef33b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbc210fdd4e255f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c80bf7640 1 -- 172.21.15.106:0/847738141 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d44e0 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c80bf7640 1 -- 172.21.15.106:0/847738141 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5594ee5ea1c0 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c80bf7640 1 -- 172.21.15.106:0/847738141 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d4b60 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ee5cb4a0 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5594ee5cba40 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 4 waiting for MDS map... 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c80bf7640 1 -- 172.21.15.106:0/847738141 <== mon.0 v2:172.21.15.106:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x5594ef474600 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c80bf7640 1 -- 172.21.15.106:0/847738141 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 93) ==== 2885+0+0 (secure 0 0 0) 0x5594ee5f9180 con 0x5594ee59ec00 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 4 Got MDS map 93 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 4 datascan.main: Using metadata pool output 2024-04-13T03:19:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.469+0000 7f7c85c2c740 4 datascan.main: connecting to RADOS... 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 Processor -- start 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 -- start start 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37a600 con 0x5594ef46dc00 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37a780 con 0x5594ef46d800 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5594ef37a900 con 0x5594ef46d400 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c82bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c82bfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57194/0 (socket says 172.21.15.106:57194) 2024-04-13T03:19:38.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.470+0000 7f7c82bfb640 1 -- 172.21.15.106:0/1784980416 learned_addr learned my addr 172.21.15.106:0/1784980416 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c833fc640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 -- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 msgr2=0x5594ef373080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 -- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 msgr2=0x5594ef373600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 -- 172.21.15.106:0/1784980416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee5cb860 con 0x5594ef46dc00 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c833fc640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x5594ef47a8d0 tx=0x5594ef33af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c83a154799d3b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c7ebf3640 1 -- 172.21.15.106:0/1784980416 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d49c0 con 0x5594ef46dc00 2024-04-13T03:19:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c7ebf3640 1 -- 172.21.15.106:0/1784980416 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5594ee5eac40 con 0x5594ef46dc00 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c7ebf3640 1 -- 172.21.15.106:0/1784980416 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d44e0 con 0x5594ef46dc00 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 -- 172.21.15.106:0/1784980416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 msgr2=0x5594ef373b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x5594ef47a8d0 tx=0x5594ef33af00 comp rx=0 tx=0).stop 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 -- 172.21.15.106:0/1784980416 shutdown_connections 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/1784980416 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 -- 172.21.15.106:0/1784980416 >> 172.21.15.106:0/1784980416 conn(0x5594ef46d000 msgr2=0x5594ee683800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.471+0000 7f7c85c2c740 1 -- 172.21.15.106:0/1784980416 shutdown_connections 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.472+0000 7f7c85c2c740 1 -- 172.21.15.106:0/1784980416 wait complete. 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.472+0000 7f7c85c2c740 1 Processor -- start 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.472+0000 7f7c85c2c740 1 -- start start 2024-04-13T03:19:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.472+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37b980 con 0x5594ef46d800 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594ef37b800 con 0x5594ef46d400 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5594ef37b680 con 0x5594ef46dc00 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 0x5594ef373600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57208/0 (socket says 172.21.15.106:57208) 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 learned_addr learned my addr 172.21.15.106:0/68388561 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46dc00 msgr2=0x5594ef373b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ef46dc00 0x5594ef373b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d400 msgr2=0x5594ef373080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ef46d400 0x5594ef373080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee5cb680 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 0x5594ef373600 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x5594ef47bc20 tx=0x5594ef33a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c7d3f0640 1 -- 172.21.15.106:0/68388561 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d4340 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c7d3f0640 1 -- 172.21.15.106:0/68388561 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5594ee5eae00 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5594ee5cb4a0 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c7d3f0640 1 -- 172.21.15.106:0/68388561 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5594ee5d49c0 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ee5cba40 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.473+0000 7f7c85c2c740 4 datascan.init: resolving metadata pool 17 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.474+0000 7f7c7d3f0640 1 -- 172.21.15.106:0/68388561 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5594ef49cc00 con 0x5594ef46d800 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c7d3f0640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5594ef4ef000 0x5594ef374100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c7d3f0640 1 -- 172.21.15.106:0/68388561 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x5594ef4ec600 con 0x5594ef46d800 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5594ef4ef000 0x5594ef374100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c85c2c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c85c2c740 4 datascan.main: resolving metadata pool 17 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5594ef4ef400 0x5594ef374680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4ef800 con 0x5594ef4ef400 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5594ef4ef400 0x5594ef374680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5594ef4ef000 0x5594ef374100 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5594ef33cde0 tx=0x5594ee8b6a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.475+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5594ef4ef400 0x5594ef374680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x5594ef4ef800 0x5594ee92c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377400 con 0x5594ef4ef800 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x5594ef4ef800 0x5594ee92c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x5594ef4ef800 0x5594ee92c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.476+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377400 con 0x5594ef4ef400 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x5594ef377400 0x5594ee92c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377000 con 0x5594ef377400 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x5594ef377400 0x5594ee92c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x5594ef377400 0x5594ee92c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.477+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377000 con 0x5594ef377400 2024-04-13T03:19:38.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377000 con 0x5594ef4ef800 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef377000 con 0x5594ef4ef400 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 3 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5594ef377000 0x5594ee92cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef376c00 con 0x5594ef377000 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.478+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5594ef377000 0x5594ee92cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.479+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5594ef377000 0x5594ee92cb00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.479+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.479+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x5594ef376c00 0x5594ee92d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.479+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef376800 con 0x5594ef376c00 2024-04-13T03:19:38.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.479+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x5594ef376c00 0x5594ee92d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x5594ef376c00 0x5594ee92d080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef376800 con 0x5594ef4ef800 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 3 ==== osd_op_reply(10 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef376800 con 0x5594ef4ef400 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 4 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.480+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef376800 con 0x5594ef377400 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 3 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5594ef376800 0x5594ee92e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4ee800 con 0x5594ef376800 2024-04-13T03:19:38.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5594ef376800 0x5594ee92e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5594ef376800 0x5594ee92e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== osd_op_reply(13 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x5594ef4ee800 0x5594ee92e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.481+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4eec00 con 0x5594ef4ee800 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x5594ef4ee800 0x5594ee92e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x5594ef4ee800 0x5594ee92e680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c82bfb640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x5594ef4eec00 0x5594ee92d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4eec00 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.482+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x5594ef4eec00 0x5594ee92d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c83bfd640 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x5594ef4eec00 0x5594ee92d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== osd_op_reply(15 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4eec00 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef377000 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef377400 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 4 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.483+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ef800 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 4 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ef800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 5 ==== osd_op_reply(19 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ee800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef376800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== osd_op_reply(21 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.484+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ee800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ef400 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 5 ==== osd_op_reply(23 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ef400 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 6 ==== osd_op_reply(24 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.485+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef376c00 2024-04-13T03:19:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef4ef800 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 6 ==== osd_op_reply(26 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef377400 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 5 ==== osd_op_reply(27 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef376c00 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 3 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.486+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef377400 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef376800 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef377000 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 3 ==== osd_op_reply(31 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.487+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92ac00 con 0x5594ef376800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 4 ==== osd_op_reply(32 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4efc00 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a000 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a400 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.488+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a800 con 0x5594ef4ef800 2024-04-13T03:19:38.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94ac00 con 0x5594ef4ef800 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b000 con 0x5594ef4ef800 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-13T03:19:38.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.489+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a000 con 0x5594ef4ef800 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a400 con 0x5594ef4eec00 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4eec00 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.490+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b800 con 0x5594ef4ee800 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-13T03:19:38.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94bc00 con 0x5594ef4ef400 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4efc00 con 0x5594ef4ef800 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.491+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-13T03:19:38.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee92a800 con 0x5594ef377400 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a800 con 0x5594ef377400 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 20 datascan.scan_links: injected ino detected 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-13T03:19:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94ac00 con 0x5594ef376800 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.492+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b000 con 0x5594ef4ef400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.493+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 8 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.493+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.493+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 15 ==== osd_op_reply(49 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.493+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 9 ==== osd_op_reply(50 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 9 ==== osd_op_reply(51 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 10 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 16 ==== osd_op_reply(53 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.494+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef400 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 10 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377000 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 5 ==== osd_op_reply(55 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376c00 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.495+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 17 ==== osd_op_reply(57 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef400 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376800 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.496+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ee800 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4eec00 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.2 v2:172.21.15.106:6824/4087145286 3 ==== osd_op_reply(62 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4eec00 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377000 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 6 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 12 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.497+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 18 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 19 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ee800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 6 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 7 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.498+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ee800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 7 ==== osd_op_reply(69 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef400 2024-04-13T03:19:38.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 12 ==== osd_op_reply(70 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef400 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 13 ==== osd_op_reply(71 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.499+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376c00 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 5 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 20 ==== osd_op_reply(73 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 13 ==== osd_op_reply(74 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.500+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376c00 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.4 v2:172.21.15.114:6800/1124906481 6 ==== osd_op_reply(75 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330500 con 0x5594ef376c00 2024-04-13T03:19:38.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377400 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 14 ==== osd_op_reply(76 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376800 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 8 ==== osd_op_reply(77 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef377000 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 7 ==== osd_op_reply(78 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.501+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef376800 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 9 ==== osd_op_reply(79 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a000 con 0x5594ef4ef800 2024-04-13T03:19:38.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a400 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956000 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.502+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956400 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956800 con 0x5594ef4ef800 2024-04-13T03:19:38.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956c00 con 0x5594ef4ef800 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee957000 con 0x5594ef377000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.1 v2:172.21.15.106:6800/1701180838 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef377000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-13T03:19:38.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.503+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee957400 con 0x5594ef4ef800 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee957800 con 0x5594ef4eec00 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.2 v2:172.21.15.106:6824/4087145286 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4eec00 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee957c00 con 0x5594ef4ee800 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.5 v2:172.21.15.114:6808/1239989021 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ee800 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-13T03:19:38.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.504+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956000 con 0x5594ef4ef400 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956400 con 0x5594ef4ef800 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-13T03:19:38.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee956800 con 0x5594ef377400 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.505+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a000 con 0x5594ef377400 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 20 datascan.scan_links: injected ino detected 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-13T03:19:38.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a400 con 0x5594ef376800 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.6 v2:172.21.15.114:6816/1761123940 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5594ef330780 con 0x5594ef376800 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: processing 1 injected_inos 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.506+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ef4efc00 con 0x5594ef377400 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.507+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.507+0000 7f7c85c2c740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.507+0000 7f7c85c2c740 20 datascan.scan_links: first is now 2 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.507+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94a800 con 0x5594ef377400 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c833fc640 1 -- 172.21.15.106:0/68388561 <== osd.0 v2:172.21.15.106:6808/3573709684 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v123'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5594ef330780 con 0x5594ef377400 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 10 datascan.scan_links: updating inotable 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94ac00 con 0x5594ef4ef800 2024-04-13T03:19:38.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.508+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b400 con 0x5594ef4ef800 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.509+0000 7f7c83bfd640 1 -- 172.21.15.106:0/68388561 <== osd.3 v2:172.21.15.106:6811/987945743 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v123'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5594ef330500 con 0x5594ef4ef800 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.509+0000 7f7c85c2c740 10 datascan.scan_links: updating snaptable 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.509+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5594ee94b000 con 0x5594ef4ef400 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 <== osd.7 v2:172.21.15.114:6824/300957724 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5594ef330000 con 0x5594ef4ef400 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x5594ef377400 msgr2=0x5594ee92c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x5594ef377400 0x5594ee92c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5594ef377000 msgr2=0x5594ee92cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x5594ef377000 0x5594ee92cb00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x5594ef4eec00 msgr2=0x5594ee92d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x5594ef4eec00 0x5594ee92d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x5594ef4ef800 msgr2=0x5594ee92c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x5594ef4ef800 0x5594ee92c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x5594ef376c00 msgr2=0x5594ee92d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x5594ef376c00 0x5594ee92d080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x5594ef4ee800 msgr2=0x5594ee92e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 reap_dead start 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x5594ef4ee800 0x5594ee92e680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5594ef376800 msgr2=0x5594ee92e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x5594ef376800 0x5594ee92e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5594ef4ef400 msgr2=0x5594ef374680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x5594ef4ef400 0x5594ef374680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5594ef4ef000 msgr2=0x5594ef374100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x5594ef4ef000 0x5594ef374100 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5594ef33cde0 tx=0x5594ee8b6a80 comp rx=0 tx=0).stop 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 msgr2=0x5594ef373600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/68388561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ef46d800 0x5594ef373600 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x5594ef47bc20 tx=0x5594ef33a000 comp rx=0 tx=0).stop 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c82bfb640 1 -- 172.21.15.106:0/68388561 reap_dead start 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 shutdown_connections 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 >> 172.21.15.106:0/68388561 conn(0x5594ef46d000 msgr2=0x5594ee683800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 shutdown_connections 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/68388561 wait complete. 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 msgr2=0x5594ef372000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x5594ef3c54a0 tx=0x5594ef33b0e0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 shutdown_connections 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x5594ee59f800 0x5594ef372580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594ee59fc00 0x5594ef372b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 --2- 172.21.15.106:0/847738141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594ee59ec00 0x5594ef372000 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 >> 172.21.15.106:0/847738141 conn(0x5594ee59f000 msgr2=0x5594ee683000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 shutdown_connections 2024-04-13T03:19:38.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.510+0000 7f7c85c2c740 1 -- 172.21.15.106:0/847738141 wait complete. 2024-04-13T03:19:38.543 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.136242, result: 2024-04-13T03:19:38.544 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:38.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:38.610 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.611 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:38.612 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.613 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.614 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-13T03:19:38.615 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-13T03:19:38.616 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.617 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-13T03:19:38.618 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.619 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.620 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = False 2024-04-13T03:19:38.621 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-13T03:19:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-13T03:19:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-13T03:19:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.629 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.631 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2024-04-13T03:19:38.632 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-04-13T03:19:38.633 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = False 2024-04-13T03:19:38.634 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-04-13T03:19:38.634 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = True 2024-04-13T03:19:38.634 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:38.634 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.634 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:38.678 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-13T03:19:38.726 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-13T03:19:38.726 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-13T03:19:38.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:38.728 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-13T03:19:38.729 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-13T03:19:38.730 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2024-04-13T03:19:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.733 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.734 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-13T03:19:38.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-13T03:19:38.736 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-13T03:19:38.737 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-13T03:19:38.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.739 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-13T03:19:38.740 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.106,172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790]" 2024-04-13T03:19:38.741 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d7ff0368-2781-4238-b477-50485b08a223 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-13T03:19:38.742 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.743 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-13T03:19:38.744 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:38.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.746 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-13T03:19:38.747 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-04-13T03:19:38.748 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = True 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = True 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-04-13T03:19:38.749 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-04-13T03:19:38.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-13T03:19:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.892+0000 7fa5951b4640 1 Processor -- start 2024-04-13T03:19:38.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.892+0000 7fa5951b4640 1 -- start start 2024-04-13T03:19:38.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa5951b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa59006db90 0x7fa590072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa5951b4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa59006e020 0x7fa590073350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa5951b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59006c880 con 0x7fa590145f60 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa5951b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa59006caf0 con 0x7fa59006db90 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa5951b4640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa59006cdd0 con 0x7fa59006e020 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa59006db90 0x7fa590072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa590145f60 0x7fa59006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43884/0 (socket says 172.21.15.106:43884) 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58ed9d640 1 -- 172.21.15.106:0/2102975212 learned_addr learned my addr 172.21.15.106:0/2102975212 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58e59c640 1 -- 172.21.15.106:0/2102975212 >> v2:172.21.15.114:3300/0 conn(0x7fa59006e020 msgr2=0x7fa590073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58e59c640 1 --2- 172.21.15.106:0/2102975212 >> v2:172.21.15.114:3300/0 conn(0x7fa59006e020 0x7fa590073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58e59c640 1 -- 172.21.15.106:0/2102975212 >> v1:172.21.15.106:6789/0 conn(0x7fa590145f60 legacy=0x7fa59006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58e59c640 1 -- 172.21.15.106:0/2102975212 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59006d0b0 con 0x7fa59006db90 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.893+0000 7fa58dd9b640 1 -- 172.21.15.106:0/2102975212 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3975410873 0 0) 0x7fa59006c880 con 0x7fa590145f60 2024-04-13T03:19:38.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa58e59c640 1 --2- 172.21.15.106:0/2102975212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59006db90 0x7fa590072e10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa5901601b0 tx=0x7fa58402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4bb0ee5f783fb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa58dd9b640 1 -- 172.21.15.106:0/2102975212 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584035020 con 0x7fa59006db90 2024-04-13T03:19:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa58dd9b640 1 -- 172.21.15.106:0/2102975212 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa584031830 con 0x7fa59006db90 2024-04-13T03:19:38.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa58dd9b640 1 -- 172.21.15.106:0/2102975212 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584031c00 con 0x7fa59006db90 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa5951b4640 1 -- 172.21.15.106:0/2102975212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59006db90 msgr2=0x7fa590072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.894+0000 7fa5951b4640 1 --2- 172.21.15.106:0/2102975212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59006db90 0x7fa590072e10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa5901601b0 tx=0x7fa58402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 -- 172.21.15.106:0/2102975212 shutdown_connections 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 --2- 172.21.15.106:0/2102975212 >> v2:172.21.15.114:3300/0 conn(0x7fa59006e020 0x7fa590073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 --2- 172.21.15.106:0/2102975212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa59006db90 0x7fa590072e10 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 -- 172.21.15.106:0/2102975212 >> 172.21.15.106:0/2102975212 conn(0x7fa59006f0d0 msgr2=0x7fa59006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 -- 172.21.15.106:0/2102975212 shutdown_connections 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.895+0000 7fa5951b4640 1 -- 172.21.15.106:0/2102975212 wait complete. 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.896+0000 7fa5951b4640 1 Processor -- start 2024-04-13T03:19:38.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.896+0000 7fa5951b4640 1 -- start start 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59006db90 0x7fa59013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 0x7fa5901f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5900cdaa0 con 0x7fa590145f60 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa590146340 con 0x7fa59006db90 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa5951b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fa5900cdc20 con 0x7fa59006e020 2024-04-13T03:19:38.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57212/0 (socket says 172.21.15.106:57212) 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 -- 172.21.15.106:0/113717419 learned_addr learned my addr 172.21.15.106:0/113717419 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58ed9d640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59006db90 0x7fa59013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58e59c640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 0x7fa5901f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 -- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 msgr2=0x7fa5901f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 0x7fa5901f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 -- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59006db90 msgr2=0x7fa59013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59006db90 0x7fa59013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 -- 172.21.15.106:0/113717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59013e4f0 con 0x7fa590145f60 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58e59c640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 0x7fa5901f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.897+0000 7fa58f59e640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b3f0 tx=0x7fa58800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.898+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588015020 con 0x7fa590145f60 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.898+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa588011830 con 0x7fa590145f60 2024-04-13T03:19:38.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.898+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59013e7d0 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.898+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588011c20 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.898+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59013ed90 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.899+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5901e51d0 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.899+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa588018020 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.899+0000 7fa5777fe640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 0x7fa57c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.899+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fa58808bc00 con 0x7fa590145f60 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.899+0000 7fa58ed9d640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 0x7fa57c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:38.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.900+0000 7fa58ed9d640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 0x7fa57c06ede0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa5800012e0 tx=0x7fa580001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:38.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:38.904+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa588092080 con 0x7fa590145f60 2024-04-13T03:19:39.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:39.123+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa590068a10 con 0x7fa590145f60 2024-04-13T03:19:40.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.041+0000 7fa5777fe640 1 -- 172.21.15.106:0/113717419 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v95) ==== 162+0+0 (secure 0 0 0) 0x7fa58801f020 con 0x7fa590145f60 2024-04-13T03:19:40.042 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 msgr2=0x7fa57c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 0x7fa57c06ede0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa5800012e0 tx=0x7fa580001630 comp rx=0 tx=0).stop 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 msgr2=0x7fa5901f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b3f0 tx=0x7fa58800b8b0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 shutdown_connections 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fa57c06c920 0x7fa57c06ede0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fa59006e020 0x7fa5901f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa590145f60 0x7fa5901f62e0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 --2- 172.21.15.106:0/113717419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59006db90 0x7fa59013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 >> 172.21.15.106:0/113717419 conn(0x7fa59006f0d0 msgr2=0x7fa5900b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:40.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 shutdown_connections 2024-04-13T03:19:40.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.044+0000 7fa5951b4640 1 -- 172.21.15.106:0/113717419 wait complete. 2024-04-13T03:19:40.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:40.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.163+0000 7f8b53a36640 1 Processor -- start 2024-04-13T03:19:40.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.164+0000 7f8b53a36640 1 -- start start 2024-04-13T03:19:40.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.164+0000 7f8b53a36640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b4c06f180 0x7f8b4c166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:40.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.164+0000 7f8b53a36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c072de0 con 0x7f8b4c06cb80 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.164+0000 7f8b53a36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c073090 con 0x7f8b4c0cdf50 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.164+0000 7f8b53a36640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4c073370 con 0x7f8b4c06f180 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b51fd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b4c06f180 0x7f8b4c166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b50fd1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8b4c06cb80 0x7f8b4c0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43886/0 (socket says 172.21.15.106:43886) 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b50fd1640 1 -- 172.21.15.106:0/1178204562 learned_addr learned my addr 172.21.15.106:0/1178204562 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b51fd3640 1 -- 172.21.15.106:0/1178204562 >> v1:172.21.15.114:6789/0 conn(0x7f8b4c0cdf50 legacy=0x7f8b4c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b51fd3640 1 -- 172.21.15.106:0/1178204562 >> v1:172.21.15.106:6789/0 conn(0x7f8b4c06cb80 legacy=0x7f8b4c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.165+0000 7f8b51fd3640 1 -- 172.21.15.106:0/1178204562 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c073650 con 0x7f8b4c06f180 2024-04-13T03:19:40.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b51fd3640 1 --2- 172.21.15.106:0/1178204562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c166170 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c61b22a964b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:40.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b4bfff640 1 -- 172.21.15.106:0/1178204562 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c034070 con 0x7f8b4c06f180 2024-04-13T03:19:40.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b4bfff640 1 -- 172.21.15.106:0/1178204562 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c030610 con 0x7f8b4c06f180 2024-04-13T03:19:40.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b4bfff640 1 -- 172.21.15.106:0/1178204562 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0309e0 con 0x7f8b4c06f180 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 -- 172.21.15.106:0/1178204562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 msgr2=0x7f8b4c166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 --2- 172.21.15.106:0/1178204562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c166170 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02b960 comp rx=0 tx=0).stop 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 -- 172.21.15.106:0/1178204562 shutdown_connections 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 --2- 172.21.15.106:0/1178204562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c166170 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 -- 172.21.15.106:0/1178204562 >> 172.21.15.106:0/1178204562 conn(0x7f8b4c0b1f10 msgr2=0x7f8b4c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 -- 172.21.15.106:0/1178204562 shutdown_connections 2024-04-13T03:19:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.166+0000 7f8b53a36640 1 -- 172.21.15.106:0/1178204562 wait complete. 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 Processor -- start 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 -- start start 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c06cb80 0x7f8b4c1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 0x7f8b4c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c099930 con 0x7f8b4c06f180 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0997b0 con 0x7f8b4c06cb80 2024-04-13T03:19:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b53a36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c1666b0 con 0x7f8b4c0cdf50 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b50fd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b50fd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57240/0 (socket says 172.21.15.106:57240) 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.168+0000 7f8b50fd1640 1 -- 172.21.15.106:0/2457609349 learned_addr learned my addr 172.21.15.106:0/2457609349 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b517d2640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c06cb80 0x7f8b4c1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b51fd3640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 0x7f8b4c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 -- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 msgr2=0x7f8b4c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 0x7f8b4c158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 -- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c06cb80 msgr2=0x7f8b4c1f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c06cb80 0x7f8b4c1f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 -- 172.21.15.106:0/2457609349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c0c62b0 con 0x7f8b4c06f180 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b51fd3640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 0x7f8b4c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b50fd1640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c4d0 tx=0x7f8b4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40017070 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b40005ce0 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c0c6590 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40007270 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.169+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c0c6b50 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.170+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c068a10 con 0x7f8b4c06f180 2024-04-13T03:19:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.170+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b4001a070 con 0x7f8b4c06f180 2024-04-13T03:19:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.171+0000 7f8b49ffb640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 0x7f8b3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.171+0000 7f8b517d2640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 0x7f8b3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.171+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f8b4008af50 con 0x7f8b4c06f180 2024-04-13T03:19:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.171+0000 7f8b517d2640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 0x7f8b3806ede0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15e300 tx=0x7f8b44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:40.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.175+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b40058b80 con 0x7f8b4c06f180 2024-04-13T03:19:40.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.388+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b4c0ce8c0 con 0x7f8b4c06f180 2024-04-13T03:19:40.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.388+0000 7f8b49ffb640 1 -- 172.21.15.106:0/2457609349 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5187 (secure 0 0 0) 0x7f8b4005ce80 con 0x7f8b4c06f180 2024-04-13T03:19:40.389 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:40.390 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:40.041055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:replay","state_seq":1,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-13T03:19:40.390 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 95 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.390+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 msgr2=0x7f8b3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.390+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 0x7f8b3806ede0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15e300 tx=0x7f8b44009040 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.390+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 msgr2=0x7f8b4c0c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.390+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c4d0 tx=0x7f8b4000c990 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 shutdown_connections 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f8b3806c920 0x7f8b3806ede0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8b4c0cdf50 0x7f8b4c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b4c06f180 0x7f8b4c0c4330 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 --2- 172.21.15.106:0/2457609349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c06cb80 0x7f8b4c1f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 >> 172.21.15.106:0/2457609349 conn(0x7f8b4c0b1f10 msgr2=0x7f8b4c0b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:40.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 shutdown_connections 2024-04-13T03:19:40.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:40.391+0000 7f8b53a36640 1 -- 172.21.15.106:0/2457609349 wait complete. 2024-04-13T03:19:40.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:40.041055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9555}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9555': {'gid': 9555, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/856679728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 856679728}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 856679728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:19:40.806 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9555:up:replay 2024-04-13T03:19:41.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:41.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.916+0000 7f5368420640 1 Processor -- start 2024-04-13T03:19:41.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.916+0000 7f5368420640 1 -- start start 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53600cdf50 0x7f53600ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f536006cb10 0x7f5360072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f536006b8e0 0x7f536006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f53600ce870 con 0x7f536006b8e0 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f53600998d0 con 0x7f536006cb10 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f5368420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5360073020 con 0x7f53600cdf50 2024-04-13T03:19:41.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f536006b8e0 0x7f536006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f536006b8e0 0x7f536006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57244/0 (socket says 172.21.15.106:57244) 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 -- 172.21.15.106:0/276593491 learned_addr learned my addr 172.21.15.106:0/276593491 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53659bb640 1 --2- 172.21.15.106:0/276593491 >> v2:172.21.15.114:3300/0 conn(0x7f536006cb10 0x7f5360072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53661bc640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53600cdf50 0x7f53600ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 -- 172.21.15.106:0/276593491 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53600cdf50 msgr2=0x7f53600ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53600cdf50 0x7f53600ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 -- 172.21.15.106:0/276593491 >> v2:172.21.15.114:3300/0 conn(0x7f536006cb10 msgr2=0x7f5360072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 --2- 172.21.15.106:0/276593491 >> v2:172.21.15.114:3300/0 conn(0x7f536006cb10 0x7f5360072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.917+0000 7f53669bd640 1 -- 172.21.15.106:0/276593491 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5360073300 con 0x7f536006b8e0 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f53669bd640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f536006b8e0 0x7f536006bd90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f53600b2fd0 tx=0x7f535802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b39fc9c4b2f58b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f53651ba640 1 -- 172.21.15.106:0/276593491 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535802ca70 con 0x7f536006b8e0 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f53651ba640 1 -- 172.21.15.106:0/276593491 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535802cc10 con 0x7f536006b8e0 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f53651ba640 1 -- 172.21.15.106:0/276593491 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53580343f0 con 0x7f536006b8e0 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f5368420640 1 -- 172.21.15.106:0/276593491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f536006b8e0 msgr2=0x7f536006bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.918+0000 7f5368420640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f536006b8e0 0x7f536006bd90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f53600b2fd0 tx=0x7f535802bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 -- 172.21.15.106:0/276593491 shutdown_connections 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f536006b8e0 0x7f536006bd90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 --2- 172.21.15.106:0/276593491 >> v2:172.21.15.114:3300/0 conn(0x7f536006cb10 0x7f5360072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 --2- 172.21.15.106:0/276593491 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f53600cdf50 0x7f53600ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 -- 172.21.15.106:0/276593491 >> 172.21.15.106:0/276593491 conn(0x7f53600b1f10 msgr2=0x7f53600b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 -- 172.21.15.106:0/276593491 shutdown_connections 2024-04-13T03:19:41.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.919+0000 7f5368420640 1 -- 172.21.15.106:0/276593491 wait complete. 2024-04-13T03:19:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.920+0000 7f5368420640 1 Processor -- start 2024-04-13T03:19:41.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.920+0000 7f5368420640 1 -- start start 2024-04-13T03:19:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.920+0000 7f5368420640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f536006b8e0 0x7f536015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.920+0000 7f5368420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f536006cb10 0x7f536015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5360072c60 con 0x7f53600cdf50 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f53600738c0 con 0x7f536006cb10 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5360072de0 con 0x7f536006b8e0 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57256/0 (socket says 172.21.15.106:57256) 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 -- 172.21.15.106:0/656334390 learned_addr learned my addr 172.21.15.106:0/656334390 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53659bb640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f536006cb10 0x7f536015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53661bc640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f536006b8e0 0x7f536015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 -- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f536006b8e0 msgr2=0x7f536015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f536006b8e0 0x7f536015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 -- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f536006cb10 msgr2=0x7f536015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:41.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f536006cb10 0x7f536015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 -- 172.21.15.106:0/656334390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536015d440 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f53669bd640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f5358036040 tx=0x7f5358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358043020 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535803e450 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536015d720 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358009510 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.921+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536015dce0 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.922+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5360068a10 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.923+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5358046020 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.923+0000 7f5356ffd640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 0x7f534406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.923+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f53580ae4f0 con 0x7f53600cdf50 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.923+0000 7f53661bc640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 0x7f534406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:41.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.924+0000 7f53661bc640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 0x7f534406edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:41.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:41.928+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f535804d030 con 0x7f53600cdf50 2024-04-13T03:19:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.143+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f536006f020 con 0x7f53600cdf50 2024-04-13T03:19:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.144+0000 7f5356ffd640 1 -- 172.21.15.106:0/656334390 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5187 (secure 0 0 0) 0x7f535807c6e0 con 0x7f53600cdf50 2024-04-13T03:19:42.145 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:42.145 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:41.046605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-13T03:19:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 97 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 msgr2=0x7f534406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 0x7f534406edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350008040 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 msgr2=0x7f536015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f5358036040 tx=0x7f5358004a40 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 shutdown_connections 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f534406c8f0 0x7f534406edb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53600cdf50 0x7f536015b1f0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f536006cb10 0x7f536015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 --2- 172.21.15.106:0/656334390 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f536006b8e0 0x7f536015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 >> 172.21.15.106:0/656334390 conn(0x7f53600b1f10 msgr2=0x7f53600b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 shutdown_connections 2024-04-13T03:19:42.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:42.146+0000 7f5368420640 1 -- 172.21.15.106:0/656334390 wait complete. 2024-04-13T03:19:42.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:41.046605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9555}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9555': {'gid': 9555, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.106:6837/856679728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 856679728}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 856679728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-13T03:19:42.560 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9555:up:rejoin 2024-04-13T03:19:43.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:43.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.673+0000 7fc812d15640 1 Processor -- start 2024-04-13T03:19:43.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.673+0000 7fc812d15640 1 -- start start 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc812d15640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc80c15ebf0 0x7fc80c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc812d15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0ce1f0 0x7fc80c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc812d15640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80c0ce6f0 con 0x7fc80c0b5bf0 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc812d15640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc80c15fa70 con 0x7fc80c15ebf0 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc812d15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c15f420 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc810ab1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc80c15ebf0 0x7fc80c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc8112b2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc80c0b5bf0 0x7fc80c06e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58834/0 (socket says 172.21.15.106:58834) 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc8112b2640 1 -- 172.21.15.106:0/3609574321 learned_addr learned my addr 172.21.15.106:0/3609574321 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc80bfff640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0ce1f0 0x7fc80c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc810ab1640 1 -- 172.21.15.106:0/3609574321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0ce1f0 msgr2=0x7fc80c06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc810ab1640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0ce1f0 0x7fc80c06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc810ab1640 1 -- 172.21.15.106:0/3609574321 >> v1:172.21.15.106:6789/0 conn(0x7fc80c0b5bf0 legacy=0x7fc80c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc810ab1640 1 -- 172.21.15.106:0/3609574321 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c15f700 con 0x7fc80c15ebf0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.674+0000 7fc80b7fe640 1 -- 172.21.15.106:0/3609574321 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3584824021 0 0) 0x7fc80c0ce6f0 con 0x7fc80c0b5bf0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.675+0000 7fc810ab1640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06f780 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fc8000097a0 tx=0x7fc80002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d120a3de29cefc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.675+0000 7fc80b7fe640 1 -- 172.21.15.106:0/3609574321 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000315a0 con 0x7fc80c15ebf0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.675+0000 7fc80b7fe640 1 -- 172.21.15.106:0/3609574321 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800037070 con 0x7fc80c15ebf0 2024-04-13T03:19:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.675+0000 7fc80b7fe640 1 -- 172.21.15.106:0/3609574321 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800031960 con 0x7fc80c15ebf0 2024-04-13T03:19:43.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 -- 172.21.15.106:0/3609574321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c15ebf0 msgr2=0x7fc80c06f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06f780 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fc8000097a0 tx=0x7fc80002b840 comp rx=0 tx=0).stop 2024-04-13T03:19:43.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 -- 172.21.15.106:0/3609574321 shutdown_connections 2024-04-13T03:19:43.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0ce1f0 0x7fc80c06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 --2- 172.21.15.106:0/3609574321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06f780 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 -- 172.21.15.106:0/3609574321 >> 172.21.15.106:0/3609574321 conn(0x7fc80c0cd6d0 msgr2=0x7fc80c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 -- 172.21.15.106:0/3609574321 shutdown_connections 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.676+0000 7fc812d15640 1 -- 172.21.15.106:0/3609574321 wait complete. 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.677+0000 7fc812d15640 1 Processor -- start 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.677+0000 7fc812d15640 1 -- start start 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0b5bf0 0x7fc80c1f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0b0650 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0b04d0 con 0x7fc80c15ebf0 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc812d15640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c15efd0 con 0x7fc80c0b5bf0 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37714/0 (socket says 172.21.15.106:37714) 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 -- 172.21.15.106:0/3510652151 learned_addr learned my addr 172.21.15.106:0/3510652151 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc810ab1640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0b5bf0 0x7fc80c1f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc8112b2640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 -- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0b5bf0 msgr2=0x7fc80c1f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0b5bf0 0x7fc80c1f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 -- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 msgr2=0x7fc80c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc80bfff640 1 -- 172.21.15.106:0/3510652151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c06c290 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.678+0000 7fc8112b2640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc80bfff640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d5f0 tx=0x7fc7f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:43.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80125f0 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8019070 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c06c570 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80129d0 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.679+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1f4cc0 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.680+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c068a10 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.680+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7f8012b70 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.680+0000 7fc8097fa640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 0x7fc7f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:43.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.681+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fc7f808c0e0 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.681+0000 7fc810ab1640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 0x7fc7f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:43.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.681+0000 7fc810ab1640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 0x7fc7f006edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc800036000 tx=0x7fc800008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:43.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.685+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7f805a360 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.898+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc80c0d07d0 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.898+0000 7fc8097fa640 1 -- 172.21.15.106:0/3510652151 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5194 (secure 0 0 0) 0x7fc7f8059c60 con 0x7fc80c0ce1f0 2024-04-13T03:19:43.899 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:43.899 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:43.045274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9555,"qdb_cluster":[9555]},"id":8}]} 2024-04-13T03:19:43.900 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 98 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 msgr2=0x7fc7f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 0x7fc7f006edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc800036000 tx=0x7fc800008030 comp rx=0 tx=0).stop 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 msgr2=0x7fc80c1f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d5f0 tx=0x7fc7f800dab0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 shutdown_connections 2024-04-13T03:19:43.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fc7f006c8f0 0x7fc7f006edb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc80c15ebf0 0x7fc80c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc80c0ce1f0 0x7fc80c1f8720 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 --2- 172.21.15.106:0/3510652151 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fc80c0b5bf0 0x7fc80c1f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 >> 172.21.15.106:0/3510652151 conn(0x7fc80c0cd6d0 msgr2=0x7fc80c1591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 shutdown_connections 2024-04-13T03:19:43.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:43.900+0000 7fc812d15640 1 -- 172.21.15.106:0/3510652151 wait complete. 2024-04-13T03:19:44.065 INFO:tasks.ceph.mds.c.smithi106.stderr:2024-04-13T03:19:44.064+0000 7f0472ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:44.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T03:19:14.646491+0000', 'modified': '2024-04-13T03:19:43.045274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9555}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9555': {'gid': 9555, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.106:6837/856679728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 856679728}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 856679728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9555, 'qdb_cluster': [9555]} 2024-04-13T03:19:44.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T03:19:44.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-13T03:19:44.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.416+0000 7fe2d5ac8640 1 Processor -- start 2024-04-13T03:19:44.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.416+0000 7fe2d5ac8640 1 -- start start 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2d5ac8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe2d0145d20 0x7fe2d006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2d5ac8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 0x7fe2d00cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2d5ac8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d00b3740 con 0x7fe2d00b47e0 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2d5ac8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d00b3a20 con 0x7fe2d0145d20 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2d5ac8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d00b3d00 con 0x7fe2d00b4c20 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cf7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe2d0145d20 0x7fe2d006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2ceffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe2d00b47e0 0x7fe2d015cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58848/0 (socket says 172.21.15.106:58848) 2024-04-13T03:19:44.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2ceffd640 1 -- 172.21.15.106:0/1063310371 learned_addr learned my addr 172.21.15.106:0/1063310371 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cffff640 1 --2- 172.21.15.106:0/1063310371 >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 0x7fe2d00cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1063310371 >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 msgr2=0x7fe2d00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/1063310371 >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 0x7fe2d00cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1063310371 >> v1:172.21.15.106:6789/0 conn(0x7fe2d00b47e0 legacy=0x7fe2d015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1063310371 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d00b3fe0 con 0x7fe2d0145d20 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.417+0000 7fe2cffff640 1 --2- 172.21.15.106:0/1063310371 >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 0x7fe2d00cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.418+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/1063310371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d006cb10 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00afef0 tx=0x7fe2c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fa0ff4687370bd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.418+0000 7fe2ce7fc640 1 -- 172.21.15.106:0/1063310371 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00315a0 con 0x7fe2d0145d20 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.418+0000 7fe2ce7fc640 1 -- 172.21.15.106:0/1063310371 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0037070 con 0x7fe2d0145d20 2024-04-13T03:19:44.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.418+0000 7fe2ce7fc640 1 -- 172.21.15.106:0/1063310371 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0031960 con 0x7fe2d0145d20 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/1063310371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 msgr2=0x7fe2d006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/1063310371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d006cb10 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00afef0 tx=0x7fe2c002b840 comp rx=0 tx=0).stop 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/1063310371 shutdown_connections 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/1063310371 >> v2:172.21.15.114:3300/0 conn(0x7fe2d00b4c20 0x7fe2d00cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/1063310371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d006cb10 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/1063310371 >> 172.21.15.106:0/1063310371 conn(0x7fe2d00b2600 msgr2=0x7fe2d00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/1063310371 shutdown_connections 2024-04-13T03:19:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.419+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/1063310371 wait complete. 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.420+0000 7fe2d5ac8640 1 Processor -- start 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.420+0000 7fe2d5ac8640 1 -- start start 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 0x7fe2d0140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 0x7fe2d01413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0146280 con 0x7fe2d0145d20 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0146100 con 0x7fe2d00b47e0 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2d5ac8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d006d050 con 0x7fe2d00b4c20 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37722/0 (socket says 172.21.15.106:37722) 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 -- 172.21.15.106:0/300665060 learned_addr learned my addr 172.21.15.106:0/300665060 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 0x7fe2d01413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 0x7fe2d0140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 -- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 msgr2=0x7fe2d01413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 0x7fe2d01413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 -- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 msgr2=0x7fe2d0140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 0x7fe2d0140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 -- 172.21.15.106:0/300665060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d014bde0 con 0x7fe2d0145d20 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 0x7fe2d0140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 0x7fe2d01413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.421+0000 7fe2cffff640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800cd50 tx=0x7fe2c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.422+0000 7fe2b7fff640 1 -- 172.21.15.106:0/300665060 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800f680 con 0x7fe2d0145d20 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.422+0000 7fe2b7fff640 1 -- 172.21.15.106:0/300665060 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c800f840 con 0x7fe2d0145d20 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.422+0000 7fe2b7fff640 1 -- 172.21.15.106:0/300665060 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800fc30 con 0x7fe2d0145d20 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.422+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d014c0c0 con 0x7fe2d0145d20 2024-04-13T03:19:44.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.422+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d013e4e0 con 0x7fe2d0145d20 2024-04-13T03:19:44.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.423+0000 7fe2b7fff640 1 -- 172.21.15.106:0/300665060 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2c80213f0 con 0x7fe2d0145d20 2024-04-13T03:19:44.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.424+0000 7fe2b7fff640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.424+0000 7fe2b7fff640 1 -- 172.21.15.106:0/300665060 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fe2c808cb20 con 0x7fe2d0145d20 2024-04-13T03:19:44.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.424+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.424+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0036000 tx=0x7fe2c003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.425+0000 7fe2d898d740 1 Processor -- start 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.425+0000 7fe2d898d740 1 -- start start 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a55d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246a85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612469c0990 con 0x561246a562c0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561246967620 con 0x561246a831c0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x5612469bc950 con 0x561246a82de0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37730/0 (socket says 172.21.15.106:37730) 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 -- 172.21.15.106:0/2371129506 learned_addr learned my addr 172.21.15.106:0/2371129506 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246a85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a55d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 -- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 msgr2=0x561246a55d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a55d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 -- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 msgr2=0x561246a85b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246a85b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 -- 172.21.15.106:0/2371129506 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612466d32d0 con 0x561246a562c0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fe2c808adb0 tx=0x7fe2c805a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba83a0c745787e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2b57fa640 1 -- 172.21.15.106:0/2371129506 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80586b0 con 0x561246a562c0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2b57fa640 1 -- 172.21.15.106:0/2371129506 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c8058910 con 0x561246a562c0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.426+0000 7fe2b57fa640 1 -- 172.21.15.106:0/2371129506 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8062d60 con 0x561246a562c0 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 -- 172.21.15.106:0/2371129506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 msgr2=0x561246a860b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fe2c808adb0 tx=0x7fe2c805a6d0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 -- 172.21.15.106:0/2371129506 shutdown_connections 2024-04-13T03:19:44.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a562c0 0x561246a860b0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246a85b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2371129506 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a55d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 -- 172.21.15.106:0/2371129506 >> 172.21.15.106:0/2371129506 conn(0x561246a57ff0 msgr2=0x561246aa0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 -- 172.21.15.106:0/2371129506 shutdown_connections 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.427+0000 7fe2d898d740 1 -- 172.21.15.106:0/2371129506 wait complete. 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 -- start start 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 Processor -- start 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246ab5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561246a5a590 con 0x561246a7ded0 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56124697a9e0 con 0x561246a831c0 2024-04-13T03:19:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561246a59980 con 0x561246a82de0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2d898d740 10 client.0 _ll_register_callbacks cb 0x5612469b6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.428+0000 7fe2cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37732/0 (socket says 172.21.15.106:37732) 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 -- 172.21.15.106:0/2320278789 learned_addr learned my addr 172.21.15.106:0/2320278789 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246ab5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 -- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 msgr2=0x561246a7b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a7b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 -- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 msgr2=0x561246ab5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246ab5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 -- 172.21.15.106:0/2320278789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124679d6b0 con 0x561246a7ded0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2cffff640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fe2c808ae60 tx=0x7fe2c800af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d343c87ec29224 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8076030 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fe2c8062d60 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c8058910 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8062f60 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 10 client.9663 fetch_fsmap learned FSMap version 98 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x561246a7ed50 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c8058910 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7fe2c805fb50 con 0x561246a7ded0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 10 client.9663 fetch_fsmap finished waiting for FSMap version 98 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 10 client.9663 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.429+0000 7fe2d898d740 20 client.9663 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a84ac0 0x561246a84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2d898d740 4 client.9663 mds_command: new command op to 9555 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 --> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612469ce3b0 con 0x561246a84ac0 2024-04-13T03:19:44.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a84ac0 0x561246a84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2ae7fc640 10 client.9663 ms_handle_connect on v2:172.21.15.106:6835/856679728 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a84ac0 0x561246a84ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5e76e5f627323 server_cookie=ecafd8a866b69f20 in_seq=0 out_seq=0 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.430+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/2320278789 <== mds.0 v2:172.21.15.106:6835/856679728 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612469ce3b0 con 0x561246a84ac0 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2ae7fc640 10 client.9663 handle_command_reply: tid=0 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 client.9663 shutdown 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 msgr2=0x561246a7e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fe2c808ae60 tx=0x7fe2c800af70 comp rx=0 tx=0).stop 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 shutdown_connections 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a84ac0 0x561246a84ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a7e350 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a831c0 0x561246ab5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 --2- 172.21.15.106:0/2320278789 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a82de0 0x561246a7b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2ae7fc640 0 client.9663 ms_handle_reset on v2:172.21.15.106:6835/856679728 2024-04-13T03:19:44.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 >> 172.21.15.106:0/2320278789 conn(0x561246a562c0 msgr2=0x561246aa4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 shutdown_connections 2024-04-13T03:19:44.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 1 -- 172.21.15.106:0/2320278789 wait complete. 2024-04-13T03:19:44.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.431+0000 7fe2d898d740 20 client.9663 trim_cache size 0 max 16384 2024-04-13T03:19:44.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.449+0000 7fe2d898d740 1 Processor -- start 2024-04-13T03:19:44.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.449+0000 7fe2d898d740 1 -- start start 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246a597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246a59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01fe530 con 0x561246a7ded0 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d013e100 con 0x561246a82de0 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d01fdee0 con 0x561246a831c0 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37740/0 (socket says 172.21.15.106:37740) 2024-04-13T03:19:44.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1407480675 learned_addr learned my addr 172.21.15.106:0/1407480675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 msgr2=0x561246a59d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246a59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 msgr2=0x561246a597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246a597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/1407480675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561246aa1360 con 0x561246a7ded0 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002be50 tx=0x7fe2c0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a448893e9cb0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2b4ff9640 1 -- 172.21.15.106:0/1407480675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0058070 con 0x561246a7ded0 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2b4ff9640 1 -- 172.21.15.106:0/1407480675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0037040 con 0x561246a7ded0 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.450+0000 7fe2b4ff9640 1 -- 172.21.15.106:0/1407480675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0032d40 con 0x561246a7ded0 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- 172.21.15.106:0/1407480675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 msgr2=0x561246aa0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002be50 tx=0x7fe2c0030000 comp rx=0 tx=0).stop 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- 172.21.15.106:0/1407480675 shutdown_connections 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246a59d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246a597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 --2- 172.21.15.106:0/1407480675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246aa0550 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- 172.21.15.106:0/1407480675 >> 172.21.15.106:0/1407480675 conn(0x561246a562c0 msgr2=0x561246a7e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- 172.21.15.106:0/1407480675 shutdown_connections 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- 172.21.15.106:0/1407480675 wait complete. 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.451+0000 7fe2d898d740 1 -- start start 2024-04-13T03:19:44.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 Processor -- start 2024-04-13T03:19:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246b1aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246b1b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561246a5a590 con 0x561246a7ded0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56124697a9e0 con 0x561246a82de0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x561246ab10b0 con 0x561246a831c0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2d898d740 10 client.0 _ll_register_callbacks cb 0x7fe2c805ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37754/0 (socket says 172.21.15.106:37754) 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/345857671 learned_addr learned my addr 172.21.15.106:0/345857671 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2cffff640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246b1b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.452+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246b1aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 msgr2=0x561246b1b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246b1b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 msgr2=0x561246b1aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246b1aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 -- 172.21.15.106:0/345857671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561246a83e90 con 0x561246a7ded0 2024-04-13T03:19:44.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cffff640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246b1b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246b1aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2cf7fe640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006ce0 tx=0x7fe2c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4e28436334da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0058040 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fe2c0053030 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c0037040 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fe2c0053230 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 10 client.9669 fetch_fsmap learned FSMap version 98 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x561246a86920 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0037040 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7fe2c0031070 con 0x561246a7ded0 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 10 client.9669 fetch_fsmap finished waiting for FSMap version 98 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 10 client.9669 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-13T03:19:44.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 20 client.9669 populate_metadata read hostname 'smithi106' 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a59ec0 0x561246b26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 4 client.9669 mds_command: new command op to 9555 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 --> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] -- command(tid 0: {"prefix": "status"}) -- 0x5612469c8220 con 0x561246a59ec0 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.453+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a59ec0 0x561246b26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2ae7fc640 10 client.9669 ms_handle_connect on v2:172.21.15.106:6835/856679728 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2ceffd640 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a59ec0 0x561246b26530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b491071068a5a5e server_cookie=1ddd190645b7efaa in_seq=0 out_seq=0 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2ae7fc640 1 -- 172.21.15.106:0/345857671 <== mds.0 v2:172.21.15.106:6835/856679728 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5612469c8220 con 0x561246a59ec0 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2ae7fc640 10 client.9669 handle_command_reply: tid=0 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 client.9669 shutdown 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 msgr2=0x561246a86de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006ce0 tx=0x7fe2c0004730 comp rx=0 tx=0).stop 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 shutdown_connections 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:6835/856679728,v1:172.21.15.106:6837/856679728] conn(0x561246a59ec0 0x561246b26530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x561246a831c0 0x561246b1b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561246a82de0 0x561246b1aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2ae7fc640 0 client.9669 ms_handle_reset on v2:172.21.15.106:6835/856679728 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 --2- 172.21.15.106:0/345857671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561246a7ded0 0x561246a86de0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 >> 172.21.15.106:0/345857671 conn(0x561246a562c0 msgr2=0x561246a54ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 shutdown_connections 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 1 -- 172.21.15.106:0/345857671 wait complete. 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.454+0000 7fe2d898d740 20 client.9669 trim_cache size 0 max 16384 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d7ff0368-2781-4238-b477-50485b08a223", 2024-04-13T03:19:44.462 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 9555, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 4.4118721809999997, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 98, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 123, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 123, 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 6.3665350780000001 2024-04-13T03:19:44.463 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 msgr2=0x7fe2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0036000 tx=0x7fe2c003d040 comp rx=0 tx=0).stop 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 msgr2=0x7fe2d014dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800cd50 tx=0x7fe2c800a650 comp rx=0 tx=0).stop 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 shutdown_connections 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe2d00b4c20 0x7fe2d01413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2d0145d20 0x7fe2d014dcf0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 --2- 172.21.15.106:0/300665060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2d00b47e0 0x7fe2d0140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 >> 172.21.15.106:0/300665060 conn(0x7fe2d00b2600 msgr2=0x7fe2d00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 shutdown_connections 2024-04-13T03:19:44.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.455+0000 7fe2d5ac8640 1 -- 172.21.15.106:0/300665060 wait complete. 2024-04-13T03:19:44.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-13T03:19:44.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.566+0000 7f6cdb0c8640 1 Processor -- start 2024-04-13T03:19:44.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.566+0000 7f6cdb0c8640 1 -- start start 2024-04-13T03:19:44.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cdb0c8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6cd4158870 0x7f6cd406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cdb0c8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd4158c50 con 0x7f6cd415b270 2024-04-13T03:19:44.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cdb0c8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd40b3800 con 0x7f6cd40b32d0 2024-04-13T03:19:44.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cdb0c8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd415ea90 con 0x7f6cd4158870 2024-04-13T03:19:44.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cd9665640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6cd415b270 0x7f6cd406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58862/0 (socket says 172.21.15.106:58862) 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cd9665640 1 -- 172.21.15.106:0/3786734436 learned_addr learned my addr 172.21.15.106:0/3786734436 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3786734436 >> v2:172.21.15.114:3300/0 conn(0x7f6cd4158870 0x7f6cd406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65314054 0 0) 0x7f6cd4158c50 con 0x7f6cd415b270 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.567+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cbc003520 con 0x7f6cd415b270 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1210289464 0 0) 0x7f6cbc003520 con 0x7f6cd415b270 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 >> v2:172.21.15.114:3300/0 conn(0x7f6cd4158870 msgr2=0x7f6cd406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 --2- 172.21.15.106:0/3786734436 >> v2:172.21.15.114:3300/0 conn(0x7f6cd4158870 0x7f6cd406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 >> v1:172.21.15.114:6789/0 conn(0x7f6cd40b32d0 legacy=0x7f6cd415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd40b3ae0 con 0x7f6cd415b270 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f6ccc0019f0 con 0x7f6cd415b270 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3786734436 >> v2:172.21.15.114:3300/0 conn(0x7f6cd4158870 0x7f6cd406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 >> v1:172.21.15.106:6789/0 conn(0x7f6cd415b270 legacy=0x7f6cd406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cbc0039b0 0x7f6cbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 0x7f6cbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd415ea90 con 0x7f6cbc0051c0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc00c060 con 0x7f6cbc0039b0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc00e470 con 0x7f6cbc004800 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 0x7f6cbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cbc0039b0 0x7f6cbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd3fff640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 -- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc004800 msgr2=0x7f6cbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 -- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cbc0039b0 msgr2=0x7f6cbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cbc0039b0 0x7f6cbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.568+0000 7f6cd9665640 1 -- 172.21.15.106:0/3786734436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc003520 con 0x7f6cbc0051c0 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.569+0000 7f6cd9665640 1 -- 172.21.15.106:0/3786734436 reap_dead start 2024-04-13T03:19:44.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.569+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 0x7f6cbc00b9c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00a840 tx=0x7f6cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52d942bb2032659 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.569+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00a460 con 0x7f6cbc0051c0 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.569+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc034650 con 0x7f6cbc0051c0 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.569+0000 7f6cd37fe640 1 -- 172.21.15.106:0/3786734436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc034a40 con 0x7f6cbc0051c0 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 msgr2=0x7f6cbc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 0x7f6cbc00b9c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00a840 tx=0x7f6cbc005830 comp rx=0 tx=0).stop 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3786734436 shutdown_connections 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3786734436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cbc0051c0 0x7f6cbc00b9c0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3786734436 >> 172.21.15.106:0/3786734436 conn(0x7f6cd40d0340 msgr2=0x7f6cd40d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3786734436 shutdown_connections 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.570+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3786734436 wait complete. 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.571+0000 7f6cdb0c8640 1 Processor -- start 2024-04-13T03:19:44.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.571+0000 7f6cdb0c8640 1 -- start start 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc0051c0 0x7f6cd41f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd40cfb50 0x7f6cd41576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd415b870 con 0x7f6cd41f41e0 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd415b6f0 con 0x7f6cd40cfb50 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd415b9f0 con 0x7f6cbc0051c0 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37768/0 (socket says 172.21.15.106:37768) 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 -- 172.21.15.106:0/3051164327 learned_addr learned my addr 172.21.15.106:0/3051164327 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc0051c0 0x7f6cd41f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd3fff640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd40cfb50 0x7f6cd41576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 -- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc0051c0 msgr2=0x7f6cd41f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc0051c0 0x7f6cd41f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 -- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd40cfb50 msgr2=0x7f6cd41576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd40cfb50 0x7f6cd41576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 -- 172.21.15.106:0/3051164327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd41f0c30 con 0x7f6cd41f41e0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd9665640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02cc50 tx=0x7f6cd40b3cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc039040 con 0x7f6cd41f41e0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc043070 con 0x7f6cd41f41e0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd41f0f10 con 0x7f6cd41f41e0 2024-04-13T03:19:44.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.572+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004d60 con 0x7f6cd41f41e0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.573+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd40cffb0 con 0x7f6cd41f41e0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.574+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd41f1430 con 0x7f6cd41f41e0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.574+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ccc034650 con 0x7f6cd41f41e0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.574+0000 7f6cd17fa640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.574+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f6ccc0ada70 con 0x7f6cd41f41e0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.574+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.575+0000 7f6cd8e64640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 0x7f6cc006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009c10 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:44.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.579+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cd41f1430 con 0x7f6cd41f41e0 2024-04-13T03:19:44.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.790+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6cd415a5e0 con 0x7f6cd41f41e0 2024-04-13T03:19:44.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.791+0000 7f6cd17fa640 1 -- 172.21.15.106:0/3051164327 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f6cd415a5e0 con 0x7f6cd41f41e0 2024-04-13T03:19:44.791 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-13T03:19:44.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 msgr2=0x7f6cc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 0x7f6cc006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009c10 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 msgr2=0x7f6cd41e3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02cc50 tx=0x7f6cd40b3cb0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 shutdown_connections 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cd41f41e0 0x7f6cd41e3ba0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cd40cfb50 0x7f6cd41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 --2- 172.21.15.106:0/3051164327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cbc0051c0 0x7f6cd41f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 >> 172.21.15.106:0/3051164327 conn(0x7f6cd40d0340 msgr2=0x7f6cd40b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 shutdown_connections 2024-04-13T03:19:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:44.792+0000 7f6cdb0c8640 1 -- 172.21.15.106:0/3051164327 wait complete. 2024-04-13T03:19:44.804 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T03:19:44.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T03:19:44.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-13T03:19:44.804 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T03:19:44.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:44.805 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T03:19:44.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-13T03:19:44.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T03:19:44.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:44.805 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-13T03:19:44.820 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-13T03:19:44.821 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41965sec preferred_lft 41965sec 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9406:73ff:fe29:7efa/64 scope link 2024-04-13T03:19:44.822 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0a:ed:1c:ac:17:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::8ed:1cff:feac:17b5/64 scope link 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 16:8f:b1:4e:a0:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::148f:b1ff:fe4e:a0f7/64 scope link 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-13T03:19:44.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:44.823 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T03:19:44.886 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:44.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:44.887 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-13T03:19:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-13T03:19:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T03:19:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-13T03:19:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-13T03:19:44.943 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-13T03:19:44.944 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-13T03:19:45.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T03:19:45.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:45.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:45.079 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:45.093 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:19:45.093 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:19:45.093 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:45.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:45.094 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-13T03:19:45.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:45.160 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:45.215 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T03:19:45.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T03:19:45.215 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:45.270 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-13T03:19:45.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:45.271 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:45.354 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:45.355 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:45.362 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.114:3300,172.21.15.114:3301" 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d7ff0368-2781-4238-b477-50485b08a223.cephfs=/ 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.114:3300/172.21.15.114:3301" will pass to kernel 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-13T03:19:45.363 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-13T03:19:45.364 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-13T03:19:45.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:45.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:45.397 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-13T03:19:45.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T03:19:45.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:45.493 INFO:teuthology.orchestra.run.smithi106.stdout:client9678 2024-04-13T03:19:45.493 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:45.493 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:45.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:45.509 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./ 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/charlie 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/alpha 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/flushed/bravo 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/unflushed 2024-04-13T03:19:45.568 INFO:teuthology.orchestra.run.smithi106.stdout:./parent/unflushed/jfile 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-13T03:19:45.569 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:45.673 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-13T03:19:45.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-13T03:19:45.683 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:45.787 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627776 2024-04-13T03:19:45.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-13T03:19:45.796 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:45.899 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2024-04-13T03:19:45.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-13T03:19:45.908 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:46.012 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628280 2024-04-13T03:19:46.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:46.021 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:46.022 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:46.022 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:46.022 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:46.022 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-13T03:19:46.022 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:46.128 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2024-04-13T03:19:46.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:46.136 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:46.137 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:46.137 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:46.137 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:46.137 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-13T03:19:46.137 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:46.241 INFO:teuthology.orchestra.run.smithi106.stdout:1099511628279 2024-04-13T03:19:46.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-13T03:19:46.255 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:46.359 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2024-04-13T03:19:46.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:46.368 DEBUG:teuthology.orchestra.run.smithi106:> stdin-killer -- python3 -c ' 2024-04-13T03:19:46.369 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-13T03:19:46.369 DEBUG:teuthology.orchestra.run.smithi106:> import stat 2024-04-13T03:19:46.369 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-13T03:19:46.369 DEBUG:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-13T03:19:46.369 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-13T03:19:46.474 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2024-04-13T03:19:46.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T03:19:46.482 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-13T03:19:46.482 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:19:46.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-13T03:19:46.620 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-13T03:19:46.620 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:19:46.620 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:46.620 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:46.635 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:46.636 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:46.694 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.837+0000 7fca3b4b7640 1 Processor -- start 2024-04-13T03:19:46.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.838+0000 7fca3b4b7640 1 -- start start 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.838+0000 7fca3b4b7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fca3406f180 0x7fca34166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.838+0000 7fca3b4b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca34072de0 con 0x7fca340cdf50 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.838+0000 7fca3b4b7640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca34073090 con 0x7fca3406cb80 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.838+0000 7fca3b4b7640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fca34073370 con 0x7fca3406f180 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39253640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fca340cdf50 0x7fca340ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58872/0 (socket says 172.21.15.106:58872) 2024-04-13T03:19:46.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39253640 1 -- 172.21.15.106:0/3511484483 learned_addr learned my addr 172.21.15.106:0/3511484483 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 --2- 172.21.15.106:0/3511484483 >> v2:172.21.15.114:3300/0 conn(0x7fca3406f180 0x7fca34166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768424769 0 0) 0x7fca34072de0 con 0x7fca340cdf50 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca20003520 con 0x7fca340cdf50 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3883516707 0 0) 0x7fca20003520 con 0x7fca340cdf50 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 >> v2:172.21.15.114:3300/0 conn(0x7fca3406f180 msgr2=0x7fca34166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 --2- 172.21.15.106:0/3511484483 >> v2:172.21.15.114:3300/0 conn(0x7fca3406f180 0x7fca34166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 >> v1:172.21.15.114:6789/0 conn(0x7fca3406cb80 legacy=0x7fca340726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca34073650 con 0x7fca340cdf50 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fca300019f0 con 0x7fca340cdf50 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 >> v1:172.21.15.106:6789/0 conn(0x7fca340cdf50 legacy=0x7fca340ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200039b0 0x7fca20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca20004800 0x7fca20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 0x7fca2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca34073370 con 0x7fca200051c0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fca2000c060 con 0x7fca20004800 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fca2000c260 con 0x7fca200039b0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 0x7fca2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39253640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200039b0 0x7fca20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca38a52640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca20004800 0x7fca20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 -- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200039b0 msgr2=0x7fca20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200039b0 0x7fca20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 -- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca20004800 msgr2=0x7fca20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca20004800 0x7fca20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.839+0000 7fca39a54640 1 -- 172.21.15.106:0/3511484483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca20003520 con 0x7fca200051c0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.840+0000 7fca39a54640 1 -- 172.21.15.106:0/3511484483 reap_dead start 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.840+0000 7fca39a54640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 0x7fca2000b9c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ac0 tx=0x7fca20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cae645067c08c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.840+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009690 con 0x7fca200051c0 2024-04-13T03:19:46.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.840+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c004020 con 0x7fca200051c0 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.840+0000 7fca2bfff640 1 -- 172.21.15.106:0/3511484483 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004410 con 0x7fca200051c0 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 -- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 msgr2=0x7fca2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 0x7fca2000b9c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ac0 tx=0x7fca20005830 comp rx=0 tx=0).stop 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 -- 172.21.15.106:0/3511484483 shutdown_connections 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/3511484483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca200051c0 0x7fca2000b9c0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 -- 172.21.15.106:0/3511484483 >> 172.21.15.106:0/3511484483 conn(0x7fca340b1f10 msgr2=0x7fca340b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 -- 172.21.15.106:0/3511484483 shutdown_connections 2024-04-13T03:19:46.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.841+0000 7fca3b4b7640 1 -- 172.21.15.106:0/3511484483 wait complete. 2024-04-13T03:19:46.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.842+0000 7fca3b4b7640 1 Processor -- start 2024-04-13T03:19:46.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.842+0000 7fca3b4b7640 1 -- start start 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200051c0 0x7fca341527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 0x7fca340c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca34099930 con 0x7fca34150780 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fca340997b0 con 0x7fca340c2e40 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fca340726d0 con 0x7fca200051c0 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37780/0 (socket says 172.21.15.106:37780) 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 -- 172.21.15.106:0/4212227755 learned_addr learned my addr 172.21.15.106:0/4212227755 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca39a54640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 0x7fca340c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca39253640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200051c0 0x7fca341527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 -- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200051c0 msgr2=0x7fca341527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200051c0 0x7fca341527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 -- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 msgr2=0x7fca340c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 0x7fca340c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 -- 172.21.15.106:0/4212227755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca34152cf0 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca39a54640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 0x7fca340c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca38a52640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fca2400f620 tx=0x7fca2400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240052e0 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca24018070 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.843+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca34150d40 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.844+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24003810 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.844+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3415d140 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.845+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34068a10 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.845+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca240039b0 con 0x7fca34150780 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.845+0000 7fca29ffb640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 0x7fca1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:46.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.845+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fca2408bab0 con 0x7fca34150780 2024-04-13T03:19:46.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.845+0000 7fca39253640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 0x7fca1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:46.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.846+0000 7fca39253640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 0x7fca1806edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fca300013d0 tx=0x7fca30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:46.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:46.850+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca24059d30 con 0x7fca34150780 2024-04-13T03:19:47.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.027+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca340680d0 con 0x7fca34150780 2024-04-13T03:19:47.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.028+0000 7fca29ffb640 1 -- 172.21.15.106:0/4212227755 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17251 (secure 0 0 0) 0x7fca24059630 con 0x7fca34150780 2024-04-13T03:19:47.029 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:47.029 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":123,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:38.053684+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6835/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 msgr2=0x7fca1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 0x7fca1806edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fca300013d0 tx=0x7fca30009040 comp rx=0 tx=0).stop 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 msgr2=0x7fca340ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fca2400f620 tx=0x7fca2400fae0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 shutdown_connections 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fca1806c8f0 0x7fca1806edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca340c2e40 0x7fca340c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca34150780 0x7fca340ae580 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 --2- 172.21.15.106:0/4212227755 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fca200051c0 0x7fca341527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 >> 172.21.15.106:0/4212227755 conn(0x7fca340b1f10 msgr2=0x7fca340ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 shutdown_connections 2024-04-13T03:19:47.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.030+0000 7fca3b4b7640 1 -- 172.21.15.106:0/4212227755 wait complete. 2024-04-13T03:19:47.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:19:12.871192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:19:13.595223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:47.043 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:47.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.146+0000 7f6e3155e640 1 Processor -- start 2024-04-13T03:19:47.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.146+0000 7f6e3155e640 1 -- start start 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e3155e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c15ee40 0x7f6e2c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e3155e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e2c0709e0 0x7f6e2c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e3155e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0b5cf0 con 0x7f6e2c0b1c30 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e3155e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2c0b5fd0 con 0x7f6e2c0709e0 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e3155e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c0b62b0 con 0x7f6e2c15ee40 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e2c0709e0 0x7f6e2c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6e2c0b1c30 0x7f6e2c0b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58874/0 (socket says 172.21.15.106:58874) 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2affd640 1 -- 172.21.15.106:0/3665744832 learned_addr learned my addr 172.21.15.106:0/3665744832 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2a7fc640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c15ee40 0x7f6e2c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3665744832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c15ee40 msgr2=0x7f6e2c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2b7fe640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c15ee40 0x7f6e2c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3665744832 >> v1:172.21.15.106:6789/0 conn(0x7f6e2c0b1c30 legacy=0x7f6e2c0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3665744832 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c0b6590 con 0x7f6e2c0709e0 2024-04-13T03:19:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.147+0000 7f6e29ffb640 1 -- 172.21.15.106:0/3665744832 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388670654 0 0) 0x7f6e2c0b5cf0 con 0x7f6e2c0b1c30 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.148+0000 7f6e2b7fe640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c0709e0 0x7f6e2c06d660 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f6e240097d0 tx=0x7f6e2402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5690de7b2d90b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.148+0000 7f6e29ffb640 1 -- 172.21.15.106:0/3665744832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24035020 con 0x7f6e2c0709e0 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.148+0000 7f6e29ffb640 1 -- 172.21.15.106:0/3665744832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e24031800 con 0x7f6e2c0709e0 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.148+0000 7f6e29ffb640 1 -- 172.21.15.106:0/3665744832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24031bd0 con 0x7f6e2c0709e0 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 -- 172.21.15.106:0/3665744832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c0709e0 msgr2=0x7f6e2c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c0709e0 0x7f6e2c06d660 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f6e240097d0 tx=0x7f6e2402b8a0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 -- 172.21.15.106:0/3665744832 shutdown_connections 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c0709e0 0x7f6e2c06d660 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3665744832 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c15ee40 0x7f6e2c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 -- 172.21.15.106:0/3665744832 >> 172.21.15.106:0/3665744832 conn(0x7f6e2c071870 msgr2=0x7f6e2c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 -- 172.21.15.106:0/3665744832 shutdown_connections 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.149+0000 7f6e3155e640 1 -- 172.21.15.106:0/3665744832 wait complete. 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.150+0000 7f6e3155e640 1 Processor -- start 2024-04-13T03:19:47.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.150+0000 7f6e3155e640 1 -- start start 2024-04-13T03:19:47.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c0709e0 0x7f6e2c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e2c0b1c30 0x7f6e2c0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c06df30 con 0x7f6e2c15ee40 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c06ddb0 con 0x7f6e2c0b1c30 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c06e0b0 con 0x7f6e2c0709e0 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37802/0 (socket says 172.21.15.106:37802) 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3342514079 learned_addr learned my addr 172.21.15.106:0/3342514079 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c0709e0 msgr2=0x7f6e2c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c0709e0 0x7f6e2c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e2c0b1c30 msgr2=0x7f6e2c0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2a7fc640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e2c0b1c30 0x7f6e2c0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e2c0b1c30 0x7f6e2c0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 -- 172.21.15.106:0/3342514079 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c0cd190 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e2b7fe640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f6e2402beb0 tx=0x7f6e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24041070 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2403ac80 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24035040 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.151+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c0cd470 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.152+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c0cda30 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.152+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c068a10 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.153+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e24038020 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.153+0000 7f6e137fe640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 0x7f6e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.153+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f6e240ad8a0 con 0x7f6e2c15ee40 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.153+0000 7f6e2affd640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 0x7f6e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.154+0000 7f6e2affd640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 0x7f6e1806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c001560 tx=0x7f6e1c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.158+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e2407bb20 con 0x7f6e2c15ee40 2024-04-13T03:19:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.331+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e2c0711f0 con 0x7f6e2c15ee40 2024-04-13T03:19:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.332+0000 7f6e137fe640 1 -- 172.21.15.106:0/3342514079 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f6e2407b420 con 0x7f6e2c15ee40 2024-04-13T03:19:47.333 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 msgr2=0x7f6e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 0x7f6e1806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c001560 tx=0x7f6e1c007ca0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 msgr2=0x7f6e2c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f6e2402beb0 tx=0x7f6e240046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 shutdown_connections 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6e1806c8f0 0x7f6e1806edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e2c15ee40 0x7f6e2c0b5000 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e2c0b1c30 0x7f6e2c0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 --2- 172.21.15.106:0/3342514079 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6e2c0709e0 0x7f6e2c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 >> 172.21.15.106:0/3342514079 conn(0x7f6e2c071870 msgr2=0x7f6e2c0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 shutdown_connections 2024-04-13T03:19:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.334+0000 7f6e3155e640 1 -- 172.21.15.106:0/3342514079 wait complete. 2024-04-13T03:19:47.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.447+0000 7f548171e640 1 Processor -- start 2024-04-13T03:19:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.447+0000 7f548171e640 1 -- start start 2024-04-13T03:19:47.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f548171e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f547c06e890 0x7f547c06dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f548171e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c15bb70 con 0x7f547c15b790 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f548171e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c0cfea0 con 0x7f547c0704f0 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f548171e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f547c06d280 con 0x7f547c06e890 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f547a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f547c06e890 0x7f547c06dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f547a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f547c06e890 0x7f547c06dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37816/0 (socket says 172.21.15.106:37816) 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.448+0000 7f547affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f547c15b790 0x7f547c073750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58882/0 (socket says 172.21.15.106:58882) 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f547a7fc640 1 -- 172.21.15.106:0/588598442 learned_addr learned my addr 172.21.15.106:0/588598442 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f547a7fc640 1 -- 172.21.15.106:0/588598442 >> v1:172.21.15.114:6789/0 conn(0x7f547c0704f0 legacy=0x7f547c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f547a7fc640 1 -- 172.21.15.106:0/588598442 >> v1:172.21.15.106:6789/0 conn(0x7f547c15b790 legacy=0x7f547c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f5479ffb640 1 -- 172.21.15.106:0/588598442 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103296418 0 0) 0x7f547c15bb70 con 0x7f547c15b790 2024-04-13T03:19:47.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f547a7fc640 1 -- 172.21.15.106:0/588598442 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c0d0180 con 0x7f547c06e890 2024-04-13T03:19:47.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.449+0000 7f547a7fc640 1 --2- 172.21.15.106:0/588598442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c06e890 0x7f547c06dcd0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5464009870 tx=0x7f546402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee34457a1aad49bd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f5479ffb640 1 -- 172.21.15.106:0/588598442 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546402f840 con 0x7f547c06e890 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f5479ffb640 1 -- 172.21.15.106:0/588598442 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546402f9e0 con 0x7f547c06e890 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f5479ffb640 1 -- 172.21.15.106:0/588598442 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546402fdb0 con 0x7f547c06e890 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 -- 172.21.15.106:0/588598442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c06e890 msgr2=0x7f547c06dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 --2- 172.21.15.106:0/588598442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c06e890 0x7f547c06dcd0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5464009870 tx=0x7f546402ba20 comp rx=0 tx=0).stop 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 -- 172.21.15.106:0/588598442 shutdown_connections 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 --2- 172.21.15.106:0/588598442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c06e890 0x7f547c06dcd0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 -- 172.21.15.106:0/588598442 >> 172.21.15.106:0/588598442 conn(0x7f547c0d13f0 msgr2=0x7f547c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 -- 172.21.15.106:0/588598442 shutdown_connections 2024-04-13T03:19:47.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.450+0000 7f548171e640 1 -- 172.21.15.106:0/588598442 wait complete. 2024-04-13T03:19:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 Processor -- start 2024-04-13T03:19:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 -- start start 2024-04-13T03:19:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f547c0704f0 0x7f547c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f547c155800 0x7f547c0aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f547c06e210 con 0x7f547c15b790 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f547c06d7b0 con 0x7f547c155800 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f548171e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f547c06e390 con 0x7f547c0704f0 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f547a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f547a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37818/0 (socket says 172.21.15.106:37818) 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.452+0000 7f547a7fc640 1 -- 172.21.15.106:0/3554728097 learned_addr learned my addr 172.21.15.106:0/3554728097 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547affd640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f547c0704f0 0x7f547c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547b7fe640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f547c155800 0x7f547c0aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 -- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f547c0704f0 msgr2=0x7f547c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f547c0704f0 0x7f547c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 -- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f547c155800 msgr2=0x7f547c0aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f547c155800 0x7f547c0aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 -- 172.21.15.106:0/3554728097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c155d10 con 0x7f547c15b790 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f547a7fc640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5464000c00 tx=0x7f5464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464041400 con 0x7f547c15b790 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5464046070 con 0x7f547c15b790 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464041800 con 0x7f547c15b790 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c155ff0 con 0x7f547c15b790 2024-04-13T03:19:47.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.453+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1565b0 con 0x7f547c15b790 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.454+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c069120 con 0x7f547c15b790 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.455+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5464040070 con 0x7f547c15b790 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.455+0000 7f54737fe640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.455+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f54640ae4f0 con 0x7f547c15b790 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.455+0000 7f547affd640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.455+0000 7f547affd640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 0x7f545c06edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f54740012e0 tx=0x7f5474004d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.459+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5464033030 con 0x7f547c15b790 2024-04-13T03:19:47.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.634+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f547c071020 con 0x7f547c15b790 2024-04-13T03:19:47.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.634+0000 7f54737fe640 1 -- 172.21.15.106:0/3554728097 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f546407c090 con 0x7f547c15b790 2024-04-13T03:19:47.635 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 msgr2=0x7f545c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 0x7f545c06edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f54740012e0 tx=0x7f5474004d40 comp rx=0 tx=0).stop 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 msgr2=0x7f547c157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5464000c00 tx=0x7f5464004730 comp rx=0 tx=0).stop 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 shutdown_connections 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f547c155800 0x7f547c0aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f547c15b790 0x7f547c157ff0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 --2- 172.21.15.106:0/3554728097 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f547c0704f0 0x7f547c156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 >> 172.21.15.106:0/3554728097 conn(0x7f547c0d13f0 msgr2=0x7f547c073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 shutdown_connections 2024-04-13T03:19:47.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.636+0000 7f548171e640 1 -- 172.21.15.106:0/3554728097 wait complete. 2024-04-13T03:19:47.648 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:47.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.750+0000 7fee04311640 1 Processor -- start 2024-04-13T03:19:47.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.750+0000 7fee04311640 1 -- start start 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee04311640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fedfc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee04311640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fedfc073520 0x7fedfc15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee04311640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedfc06caf0 con 0x7fedfc06db90 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee04311640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc06cdd0 con 0x7fedfc073520 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee04311640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc06d0b0 con 0x7fedfc145f60 2024-04-13T03:19:47.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee018ac640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fedfc06db90 0x7fedfc072e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58896/0 (socket says 172.21.15.106:58896) 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee018ac640 1 -- 172.21.15.106:0/260100570 learned_addr learned my addr 172.21.15.106:0/260100570 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee020ad640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fedfc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645872684 0 0) 0x7fedfc06caf0 con 0x7fedfc06db90 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee028ae640 1 --2- 172.21.15.106:0/260100570 >> v2:172.21.15.114:3300/0 conn(0x7fedfc073520 0x7fedfc15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede8003520 con 0x7fedfc06db90 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880720971 0 0) 0x7fede8003520 con 0x7fedfc06db90 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 msgr2=0x7fedfc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fedfc06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 >> v2:172.21.15.114:3300/0 conn(0x7fedfc073520 msgr2=0x7fedfc15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 --2- 172.21.15.106:0/260100570 >> v2:172.21.15.114:3300/0 conn(0x7fedfc073520 0x7fedfc15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc06d390 con 0x7fedfc06db90 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.751+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fedec001670 con 0x7fedfc06db90 2024-04-13T03:19:47.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 >> v1:172.21.15.106:6789/0 conn(0x7fedfc06db90 legacy=0x7fedfc072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fede8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede8004d40 0x7fede80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc06cdd0 con 0x7fede80039b0 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc06d0b0 con 0x7fede8004d40 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fede800b9c0 con 0x7fedfc145f60 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fede8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee018ac640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fede8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee028ae640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede8004d40 0x7fede80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 -- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 msgr2=0x7fede8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:47.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc145f60 0x7fede8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 -- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede8004d40 msgr2=0x7fede80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede8004d40 0x7fede80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.752+0000 7fee020ad640 1 -- 172.21.15.106:0/260100570 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8003520 con 0x7fede80039b0 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee020ad640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fede8003dd0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ac0 tx=0x7fede8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec4a8b2e72fc153 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8039070 con 0x7fede80039b0 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf802c6b0 con 0x7fede80039b0 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee010ab640 1 -- 172.21.15.106:0/260100570 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8009690 con 0x7fede80039b0 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 -- 172.21.15.106:0/260100570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 msgr2=0x7fede8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 --2- 172.21.15.106:0/260100570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fede8003dd0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ac0 tx=0x7fede8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee028ae640 1 -- 172.21.15.106:0/260100570 reap_dead start 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 -- 172.21.15.106:0/260100570 shutdown_connections 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 -- 172.21.15.106:0/260100570 >> 172.21.15.106:0/260100570 conn(0x7fedfc06f0d0 msgr2=0x7fedfc06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 -- 172.21.15.106:0/260100570 shutdown_connections 2024-04-13T03:19:47.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.753+0000 7fee04311640 1 -- 172.21.15.106:0/260100570 wait complete. 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 Processor -- start 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 -- start start 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 0x7fedfc0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedfc0cc540 0x7fedfc1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0cdc20 con 0x7fede80039b0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0cdaa0 con 0x7fedfc0cc540 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee04311640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc06dff0 con 0x7fedfc0ca9c0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee020ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee020ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37846/0 (socket says 172.21.15.106:37846) 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.755+0000 7fee020ad640 1 -- 172.21.15.106:0/2619790857 learned_addr learned my addr 172.21.15.106:0/2619790857 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee028ae640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedfc0cc540 0x7fedfc1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee018ac640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 0x7fedfc0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 -- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 msgr2=0x7fedfc0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 0x7fedfc0cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 -- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedfc0cc540 msgr2=0x7fedfc1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedfc0cc540 0x7fedfc1f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 -- 172.21.15.106:0/2619790857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0c69a0 con 0x7fede80039b0 2024-04-13T03:19:47.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee018ac640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 0x7fedfc0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee020ad640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fedf8000c00 tx=0x7fedf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8042020 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf803a040 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8039070 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc0c9b90 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.756+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc0cb1b0 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.757+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc068a10 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.757+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedf8045020 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.758+0000 7fedf2ffd640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 0x7fede006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.758+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fedf80ad4c0 con 0x7fede80039b0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.758+0000 7fee018ac640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 0x7fede006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:47.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.758+0000 7fee018ac640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 0x7fede006ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1426a0 tx=0x7fedec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:47.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.762+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedf804c030 con 0x7fede80039b0 2024-04-13T03:19:47.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.937+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedfc0cc920 con 0x7fede80039b0 2024-04-13T03:19:47.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.938+0000 7fedf2ffd640 1 -- 172.21.15.106:0/2619790857 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7fedf807b740 con 0x7fede80039b0 2024-04-13T03:19:47.938 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:47.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 msgr2=0x7fede006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 0x7fede006ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1426a0 tx=0x7fedec009040 comp rx=0 tx=0).stop 2024-04-13T03:19:47.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 msgr2=0x7fedfc0c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fedf8000c00 tx=0x7fedf8004730 comp rx=0 tx=0).stop 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 shutdown_connections 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fede006c920 0x7fede006ede0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fedfc0ca9c0 0x7fedfc0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedfc0cc540 0x7fedfc1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.939+0000 7fee04311640 1 --2- 172.21.15.106:0/2619790857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede80039b0 0x7fedfc0c9650 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.940+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 >> 172.21.15.106:0/2619790857 conn(0x7fedfc06f0d0 msgr2=0x7fedfc144430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.940+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 shutdown_connections 2024-04-13T03:19:47.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:47.940+0000 7fee04311640 1 -- 172.21.15.106:0/2619790857 wait complete. 2024-04-13T03:19:47.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T03:19:48.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.052+0000 7f20c5895640 1 Processor -- start 2024-04-13T03:19:48.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.052+0000 7f20c5895640 1 -- start start 2024-04-13T03:19:48.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20c5895640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20c00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20c5895640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c00cfd20 con 0x7f20c015e9b0 2024-04-13T03:19:48.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20c5895640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c0162af0 con 0x7f20c015ed90 2024-04-13T03:19:48.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20c5895640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20c0162dd0 con 0x7f20c00b3d60 2024-04-13T03:19:48.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20beffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f20c015e9b0 0x7f20c00d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58898/0 (socket says 172.21.15.106:58898) 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20beffd640 1 -- 172.21.15.106:0/2479006715 learned_addr learned my addr 172.21.15.106:0/2479006715 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20c00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884551643 0 0) 0x7f20c00cfd20 con 0x7f20c015e9b0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20a8003520 con 0x7f20c015e9b0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1252196120 0 0) 0x7f20a8003520 con 0x7f20c015e9b0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 msgr2=0x7f20c00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20c00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 >> v1:172.21.15.114:6789/0 conn(0x7f20c015ed90 legacy=0x7f20c00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c01630b0 con 0x7f20c015e9b0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f20b0001670 con 0x7f20c015e9b0 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.053+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 >> v1:172.21.15.106:6789/0 conn(0x7f20c015e9b0 legacy=0x7f20c00d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20a80039e0 0x7f20a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20a8004d40 0x7f20a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0162dd0 con 0x7f20a8004d40 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20a800b9c0 con 0x7f20a80039e0 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20a800bbc0 con 0x7f20c00b3d60 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20a8004d40 0x7f20a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20beffd640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20a80039e0 0x7f20a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20be7fc640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 -- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 msgr2=0x7f20a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20c00b3d60 0x7f20a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 -- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20a80039e0 msgr2=0x7f20a8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20a80039e0 0x7f20a8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.054+0000 7f20bf7fe640 1 -- 172.21.15.106:0/2479006715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a8003520 con 0x7f20a8004d40 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20a8004d40 0x7f20a80051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009fc0 tx=0x7f20a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44f7150c5f5fa8f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8039070 con 0x7f20a8004d40 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b802cc20 con 0x7f20a8004d40 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20bdffb640 1 -- 172.21.15.106:0/2479006715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8009c10 con 0x7f20a8004d40 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 -- 172.21.15.106:0/2479006715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20a8004d40 msgr2=0x7f20a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 --2- 172.21.15.106:0/2479006715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20a8004d40 0x7f20a80051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009fc0 tx=0x7f20a8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20bf7fe640 1 -- 172.21.15.106:0/2479006715 reap_dead start 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 -- 172.21.15.106:0/2479006715 shutdown_connections 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 -- 172.21.15.106:0/2479006715 >> 172.21.15.106:0/2479006715 conn(0x7f20c008bf10 msgr2=0x7f20c00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 -- 172.21.15.106:0/2479006715 shutdown_connections 2024-04-13T03:19:48.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.055+0000 7f20c5895640 1 -- 172.21.15.106:0/2479006715 wait complete. 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.056+0000 7f20c5895640 1 Processor -- start 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 -- start start 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 0x7f20c01612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 0x7f20c015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0162800 con 0x7f20c006d9b0 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0162680 con 0x7f20c015c560 2024-04-13T03:19:48.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20c5895640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00d4d00 con 0x7f20a8004d40 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20be7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20be7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37866/0 (socket says 172.21.15.106:37866) 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.057+0000 7f20be7fc640 1 -- 172.21.15.106:0/464558935 learned_addr learned my addr 172.21.15.106:0/464558935 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 0x7f20c015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20beffd640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 0x7f20c01612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 -- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 msgr2=0x7f20c01612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 0x7f20c01612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 -- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 msgr2=0x7f20c015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 0x7f20c015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 -- 172.21.15.106:0/464558935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c006de70 con 0x7f20c006d9b0 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20bf7fe640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 0x7f20c015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20beffd640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 0x7f20c01612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20be7fc640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f20b400f860 tx=0x7f20b400fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4018020 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b400d8e0 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c006e150 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b400dcb0 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.058+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c006e710 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.059+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c0068a10 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.060+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20b401b020 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.060+0000 7f20a7fff640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.060+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f20b408c2f0 con 0x7f20c006d9b0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.060+0000 7f20beffd640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.060+0000 7f20beffd640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 0x7f20ac06ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007d90 tx=0x7f20b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.064+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20b4022020 con 0x7f20c006d9b0 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.256+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20c00d53a0 con 0x7f20c006d9b0 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.256+0000 7f20a7fff640 1 -- 172.21.15.106:0/464558935 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v123) ==== 80+0+248 (secure 0 0 0) 0x7f20b405a4e0 con 0x7f20c006d9b0 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/3985285527 2024-04-14T03:19:38.053659+0000 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/3985285527 2024-04-14T03:19:38.053659+0000 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6837/1427655446 2024-04-14T03:19:27.537455+0000 2024-04-13T03:19:48.257 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6835/1427655446 2024-04-14T03:19:27.537455+0000 2024-04-13T03:19:48.258 INFO:teuthology.orchestra.run.smithi106.stderr:listed 4 entries 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.258+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 msgr2=0x7f20ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.258+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 0x7f20ac06ede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007d90 tx=0x7f20b0009040 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.258+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 msgr2=0x7f20c00b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.258+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f20b400f860 tx=0x7f20b400fd20 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 shutdown_connections 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c015c560 0x7f20c015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c006d9b0 0x7f20c00b6950 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 --2- 172.21.15.106:0/464558935 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20a8004d40 0x7f20c01612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 >> 172.21.15.106:0/464558935 conn(0x7f20c008bf10 msgr2=0x7f20c00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:48.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 shutdown_connections 2024-04-13T03:19:48.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.259+0000 7f20c5895640 1 -- 172.21.15.106:0/464558935 wait complete. 2024-04-13T03:19:48.270 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-13T03:19:48.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:48.270 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-13T03:19:48.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:19:48.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:48.301 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T03:19:48.355 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:19:48.356 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:48.356 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:48.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:48.374 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:19:48.374 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:48.375 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:48.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:48.410 INFO:tasks.cephfs.mount:Terminating background process 2024-04-13T03:19:48.410 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:48.410 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:48.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:48.433 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:48.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.550+0000 7f10ad42d640 1 Processor -- start 2024-04-13T03:19:48.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.550+0000 7f10ad42d640 1 -- start start 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10ad42d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10ad42d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10a80734c0 0x7f10a80cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10ad42d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a80ccd00 con 0x7f10a80cc540 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10ad42d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10a806ec10 con 0x7f10a80734c0 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10ad42d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10a80ccfe0 con 0x7f10a815c3f0 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a67fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f10a80cc540 0x7f10a8072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58910/0 (socket says 172.21.15.106:58910) 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a67fc640 1 -- 172.21.15.106:0/2210550507 learned_addr learned my addr 172.21.15.106:0/2210550507 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10a80734c0 0x7f10a80cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a6ffd640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a77fe640 1 -- 172.21.15.106:0/2210550507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 msgr2=0x7f10a815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.551+0000 7f10a77fe640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a815eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a77fe640 1 -- 172.21.15.106:0/2210550507 >> v1:172.21.15.106:6789/0 conn(0x7f10a80cc540 legacy=0x7f10a8072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a77fe640 1 -- 172.21.15.106:0/2210550507 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80cd2c0 con 0x7f10a80734c0 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a5ffb640 1 -- 172.21.15.106:0/2210550507 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2917718257 0 0) 0x7f10a80ccd00 con 0x7f10a80cc540 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a77fe640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80734c0 0x7f10a80cf530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f10a00097a0 tx=0x7f10a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fd9058522c6e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a5ffb640 1 -- 172.21.15.106:0/2210550507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0035020 con 0x7f10a80734c0 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a5ffb640 1 -- 172.21.15.106:0/2210550507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0031830 con 0x7f10a80734c0 2024-04-13T03:19:48.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.552+0000 7f10a5ffb640 1 -- 172.21.15.106:0/2210550507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0031c00 con 0x7f10a80734c0 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 -- 172.21.15.106:0/2210550507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80734c0 msgr2=0x7f10a80cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80734c0 0x7f10a80cf530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f10a00097a0 tx=0x7f10a002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 -- 172.21.15.106:0/2210550507 shutdown_connections 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80734c0 0x7f10a80cf530 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 --2- 172.21.15.106:0/2210550507 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 -- 172.21.15.106:0/2210550507 >> 172.21.15.106:0/2210550507 conn(0x7f10a80cdef0 msgr2=0x7f10a80ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 -- 172.21.15.106:0/2210550507 shutdown_connections 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.553+0000 7f10ad42d640 1 -- 172.21.15.106:0/2210550507 wait complete. 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.554+0000 7f10ad42d640 1 Processor -- start 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.554+0000 7f10ad42d640 1 -- start start 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a80734c0 0x7f10a81f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a80c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80cc9c0 con 0x7f10a80cc540 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80cd880 con 0x7f10a80734c0 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10ad42d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f10a815c7d0 con 0x7f10a815c3f0 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37872/0 (socket says 172.21.15.106:37872) 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 -- 172.21.15.106:0/3203303740 learned_addr learned my addr 172.21.15.106:0/3203303740 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a77fe640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a80c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a6ffd640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a80734c0 0x7f10a81f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 -- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 msgr2=0x7f10a80c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a80c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 -- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a80734c0 msgr2=0x7f10a81f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a80734c0 0x7f10a81f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 -- 172.21.15.106:0/3203303740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80c8f00 con 0x7f10a80cc540 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.555+0000 7f10a67fc640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c4a0 tx=0x7f109c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.556+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c017070 con 0x7f10a80cc540 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.556+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c005ce0 con 0x7f10a80cc540 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.556+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c0072b0 con 0x7f10a80cc540 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.556+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81f8450 con 0x7f10a80cc540 2024-04-13T03:19:48.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.556+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81f8a10 con 0x7f10a80cc540 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.557+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a8068bc0 con 0x7f10a80cc540 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.557+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f109c01a070 con 0x7f10a80cc540 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.557+0000 7f108f7fe640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 0x7f109406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.557+0000 7f10a6ffd640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 0x7f109406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.557+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f109c08b0c0 con 0x7f10a80cc540 2024-04-13T03:19:48.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.558+0000 7f10a6ffd640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 0x7f109406edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e90 tx=0x7f109800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.562+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10a8068bc0 con 0x7f10a80cc540 2024-04-13T03:19:48.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.775+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a8069120 con 0x7f10a80cc540 2024-04-13T03:19:48.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.776+0000 7f108f7fe640 1 -- 172.21.15.106:0/3203303740 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5194 (secure 0 0 0) 0x7f109c059340 con 0x7f10a80cc540 2024-04-13T03:19:48.777 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:48.777 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:43.045274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9555,"qdb_cluster":[9555]},"id":8}]} 2024-04-13T03:19:48.777 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 98 2024-04-13T03:19:48.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 msgr2=0x7f109406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 0x7f109406edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e90 tx=0x7f109800b040 comp rx=0 tx=0).stop 2024-04-13T03:19:48.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 msgr2=0x7f10a80cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c4a0 tx=0x7f109c00c960 comp rx=0 tx=0).stop 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 shutdown_connections 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f109406c8f0 0x7f109406edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f10a815c3f0 0x7f10a80c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a80cc540 0x7f10a80cae80 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 --2- 172.21.15.106:0/3203303740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a80734c0 0x7f10a81f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 >> 172.21.15.106:0/3203303740 conn(0x7f10a80cdef0 msgr2=0x7f10a8067720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 shutdown_connections 2024-04-13T03:19:48.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.778+0000 7f10ad42d640 1 -- 172.21.15.106:0/3203303740 wait complete. 2024-04-13T03:19:48.790 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:48.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.888+0000 7f2d8835e640 1 Processor -- start 2024-04-13T03:19:48.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.888+0000 7f2d8835e640 1 -- start start 2024-04-13T03:19:48.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d8835e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2d8006e8e0 0x7f2d8015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d8835e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d80070c80 0x7f2d800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d8835e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8015f420 con 0x7f2d800cd770 2024-04-13T03:19:48.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d8835e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8006ecc0 con 0x7f2d80070c80 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d8835e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2d800707f0 con 0x7f2d8006e8e0 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d868fb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d80070c80 0x7f2d800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d868fb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2d80070c80 0x7f2d800710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37874/0 (socket says 172.21.15.106:37874) 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d868fb640 1 -- 172.21.15.106:0/3231534621 learned_addr learned my addr 172.21.15.106:0/3231534621 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.889+0000 7f2d860fa640 1 --2- 172.21.15.106:0/3231534621 >> v2:172.21.15.114:3300/0 conn(0x7f2d8006e8e0 0x7f2d8015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d868fb640 1 -- 172.21.15.106:0/3231534621 >> v2:172.21.15.114:3300/0 conn(0x7f2d8006e8e0 msgr2=0x7f2d8015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d868fb640 1 --2- 172.21.15.106:0/3231534621 >> v2:172.21.15.114:3300/0 conn(0x7f2d8006e8e0 0x7f2d8015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d868fb640 1 -- 172.21.15.106:0/3231534621 >> v1:172.21.15.114:6789/0 conn(0x7f2d800cd770 legacy=0x7f2d800cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:48.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d868fb640 1 -- 172.21.15.106:0/3231534621 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8015f700 con 0x7f2d80070c80 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d868fb640 1 --2- 172.21.15.106:0/3231534621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d80070c80 0x7f2d800710b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f2d80070970 tx=0x7f2d7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953bfa37d46360cf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d850f8640 1 -- 172.21.15.106:0/3231534621 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7802ca70 con 0x7f2d80070c80 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d850f8640 1 -- 172.21.15.106:0/3231534621 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7802cc10 con 0x7f2d80070c80 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.890+0000 7f2d850f8640 1 -- 172.21.15.106:0/3231534621 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780343f0 con 0x7f2d80070c80 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 -- 172.21.15.106:0/3231534621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d80070c80 msgr2=0x7f2d800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 --2- 172.21.15.106:0/3231534621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d80070c80 0x7f2d800710b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f2d80070970 tx=0x7f2d7802bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 -- 172.21.15.106:0/3231534621 shutdown_connections 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 --2- 172.21.15.106:0/3231534621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d80070c80 0x7f2d800710b0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 --2- 172.21.15.106:0/3231534621 >> v2:172.21.15.114:3300/0 conn(0x7f2d8006e8e0 0x7f2d8015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 -- 172.21.15.106:0/3231534621 >> 172.21.15.106:0/3231534621 conn(0x7f2d8006f180 msgr2=0x7f2d8006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 -- 172.21.15.106:0/3231534621 shutdown_connections 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.891+0000 7f2d8835e640 1 -- 172.21.15.106:0/3231534621 wait complete. 2024-04-13T03:19:48.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.892+0000 7f2d8835e640 1 Processor -- start 2024-04-13T03:19:48.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.892+0000 7f2d8835e640 1 -- start start 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 0x7f2d80157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 0x7f2d800b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8015fcc0 con 0x7f2d8006e8e0 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2d800715f0 con 0x7f2d800cd770 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d8835e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2d800cdbd0 con 0x7f2d80070c80 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37882/0 (socket says 172.21.15.106:37882) 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 -- 172.21.15.106:0/1216370843 learned_addr learned my addr 172.21.15.106:0/1216370843 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 -- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 msgr2=0x7f2d80157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d868fb640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 0x7f2d800b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d858f9640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 0x7f2d80157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 0x7f2d80157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 -- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 msgr2=0x7f2d800b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 0x7f2d800b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d860fa640 1 -- 172.21.15.106:0/1216370843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d800b6b90 con 0x7f2d8006e8e0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d868fb640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 0x7f2d800b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.893+0000 7f2d858f9640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 0x7f2d80157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d860fa640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00d5f0 tx=0x7f2d7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0125f0 con 0x7f2d8006e8e0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c019070 con 0x7f2d8006e8e0 2024-04-13T03:19:48.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d800b55b0 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0129d0 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.894+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d800b5b70 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.895+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d80068a10 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.895+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d7c012b70 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.896+0000 7f2d76ffd640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 0x7f2d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.896+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f2d7c08c150 con 0x7f2d8006e8e0 2024-04-13T03:19:48.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.896+0000 7f2d858f9640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 0x7f2d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:48.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.896+0000 7f2d858f9640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 0x7f2d6406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d800b4b40 tx=0x7f2d70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:48.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:48.900+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d7c05a3d0 con 0x7f2d8006e8e0 2024-04-13T03:19:49.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.076+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d800cb780 con 0x7f2d8006e8e0 2024-04-13T03:19:49.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.078+0000 7f2d76ffd640 1 -- 172.21.15.106:0/1216370843 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17251 (secure 0 0 0) 0x7f2d800cb780 con 0x7f2d8006e8e0 2024-04-13T03:19:49.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:49.079 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":123,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:38.053684+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6835/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 msgr2=0x7f2d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 0x7f2d6406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d800b4b40 tx=0x7f2d70009040 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 msgr2=0x7f2d800b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00d5f0 tx=0x7f2d7c00dab0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 shutdown_connections 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2d6406c8f0 0x7f2d6406edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2d80070c80 0x7f2d80157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d800cd770 0x7f2d800b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 --2- 172.21.15.106:0/1216370843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d8006e8e0 0x7f2d800b6650 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 >> 172.21.15.106:0/1216370843 conn(0x7f2d8006f180 msgr2=0x7f2d80072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 shutdown_connections 2024-04-13T03:19:49.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.080+0000 7f2d8835e640 1 -- 172.21.15.106:0/1216370843 wait complete. 2024-04-13T03:19:49.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T03:09:12.377322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T03:19:12.871192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T03:19:13.595223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T03:19:49.092 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T03:19:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.195+0000 7f562355e640 1 Processor -- start 2024-04-13T03:19:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.195+0000 7f562355e640 1 -- start start 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f562355e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 0x7f561c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f562355e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f561c070bb0 0x7f561c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f562355e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f561c0b5cf0 con 0x7f561c0b54a0 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f562355e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f561c0b5fd0 con 0x7f561c070bb0 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f562355e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f561c0b62b0 con 0x7f561c0b1c30 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5621afb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f561c070bb0 0x7f561c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5620af9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f561c0b54a0 0x7f561c0704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58916/0 (socket says 172.21.15.106:58916) 2024-04-13T03:19:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5620af9640 1 -- 172.21.15.106:0/3216710008 learned_addr learned my addr 172.21.15.106:0/3216710008 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f56212fa640 1 --2- 172.21.15.106:0/3216710008 >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 0x7f561c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5621afb640 1 -- 172.21.15.106:0/3216710008 >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 msgr2=0x7f561c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5621afb640 1 --2- 172.21.15.106:0/3216710008 >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 0x7f561c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5621afb640 1 -- 172.21.15.106:0/3216710008 >> v1:172.21.15.106:6789/0 conn(0x7f561c0b54a0 legacy=0x7f561c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f5621afb640 1 -- 172.21.15.106:0/3216710008 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c0b6590 con 0x7f561c070bb0 2024-04-13T03:19:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.196+0000 7f56212fa640 1 --2- 172.21.15.106:0/3216710008 >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 0x7f561c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f5621afb640 1 --2- 172.21.15.106:0/3216710008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c070bb0 0x7f561c06d660 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f561c0687a0 tx=0x7f561402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab20df9b4ae248 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f5613fff640 1 -- 172.21.15.106:0/3216710008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614039070 con 0x7f561c070bb0 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f5613fff640 1 -- 172.21.15.106:0/3216710008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561402cc80 con 0x7f561c070bb0 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f5613fff640 1 -- 172.21.15.106:0/3216710008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614034660 con 0x7f561c070bb0 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f562355e640 1 -- 172.21.15.106:0/3216710008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c070bb0 msgr2=0x7f561c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.197+0000 7f562355e640 1 --2- 172.21.15.106:0/3216710008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c070bb0 0x7f561c06d660 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f561c0687a0 tx=0x7f561402bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 -- 172.21.15.106:0/3216710008 shutdown_connections 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 --2- 172.21.15.106:0/3216710008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c070bb0 0x7f561c06d660 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 --2- 172.21.15.106:0/3216710008 >> v2:172.21.15.114:3300/0 conn(0x7f561c0b1c30 0x7f561c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 -- 172.21.15.106:0/3216710008 >> 172.21.15.106:0/3216710008 conn(0x7f561c071870 msgr2=0x7f561c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 -- 172.21.15.106:0/3216710008 shutdown_connections 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.198+0000 7f562355e640 1 -- 172.21.15.106:0/3216710008 wait complete. 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 Processor -- start 2024-04-13T03:19:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 -- start start 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f561c070bb0 0x7f561c161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f561c0b54a0 0x7f561c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f561c06dd20 con 0x7f561c0b1c30 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f561c06dba0 con 0x7f561c070bb0 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.199+0000 7f562355e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f561c06dea0 con 0x7f561c0b54a0 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37910/0 (socket says 172.21.15.106:37910) 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 -- 172.21.15.106:0/4123029111 learned_addr learned my addr 172.21.15.106:0/4123029111 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5621afb640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f561c0b54a0 0x7f561c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 -- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f561c0b54a0 msgr2=0x7f561c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f56212fa640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f561c070bb0 0x7f561c161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f561c0b54a0 0x7f561c153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 -- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f561c070bb0 msgr2=0x7f561c161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f561c070bb0 0x7f561c161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 -- 172.21.15.106:0/4123029111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c161c30 con 0x7f561c0b1c30 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5620af9640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c3f0 tx=0x7f560c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c017070 con 0x7f561c0b1c30 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560c005ce0 con 0x7f561c0b1c30 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c1f7e60 con 0x7f561c0b1c30 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c0072f0 con 0x7f561c0b1c30 2024-04-13T03:19:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.200+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1f8420 con 0x7f561c0b1c30 2024-04-13T03:19:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.201+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561c068a10 con 0x7f561c0b1c30 2024-04-13T03:19:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.202+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f560c01a070 con 0x7f561c0b1c30 2024-04-13T03:19:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.202+0000 7f5611ffb640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.202+0000 7f56212fa640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.202+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f560c08b110 con 0x7f561c0b1c30 2024-04-13T03:19:49.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.203+0000 7f56212fa640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 0x7f560006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5618008850 tx=0x7f5618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.207+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f560c059300 con 0x7f561c0b1c30 2024-04-13T03:19:49.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.384+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f561c0cef20 con 0x7f561c0b1c30 2024-04-13T03:19:49.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.384+0000 7f5611ffb640 1 -- 172.21.15.106:0/4123029111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f560c058c00 con 0x7f561c0b1c30 2024-04-13T03:19:49.385 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 msgr2=0x7f560006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 0x7f560006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5618008850 tx=0x7f5618008040 comp rx=0 tx=0).stop 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 msgr2=0x7f561c0cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c3f0 tx=0x7f560c00c8b0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 shutdown_connections 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f561c0b54a0 0x7f561c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f561c0b1c30 0x7f561c0cdde0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 --2- 172.21.15.106:0/4123029111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f561c070bb0 0x7f561c161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 >> 172.21.15.106:0/4123029111 conn(0x7f561c071870 msgr2=0x7f561c15c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 shutdown_connections 2024-04-13T03:19:49.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.386+0000 7f562355e640 1 -- 172.21.15.106:0/4123029111 wait complete. 2024-04-13T03:19:49.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T03:19:49.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.500+0000 7f91a231e640 1 Processor -- start 2024-04-13T03:19:49.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.500+0000 7f91a231e640 1 -- start start 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a231e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f919c15e930 0x7f919c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a231e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c06f200 con 0x7f919c0d09f0 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a231e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c06e280 con 0x7f919c06ed70 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a231e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f919c06e560 con 0x7f919c15e930 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a08bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f919c0d09f0 0x7f919c0cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58918/0 (socket says 172.21.15.106:58918) 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f919bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f919c15e930 0x7f919c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f91a08bb640 1 -- 172.21.15.106:0/2404290960 learned_addr learned my addr 172.21.15.106:0/2404290960 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f919bfff640 1 -- 172.21.15.106:0/2404290960 >> v1:172.21.15.114:6789/0 conn(0x7f919c06ed70 legacy=0x7f919c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f919bfff640 1 -- 172.21.15.106:0/2404290960 >> v1:172.21.15.106:6789/0 conn(0x7f919c0d09f0 legacy=0x7f919c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f919affd640 1 -- 172.21.15.106:0/2404290960 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965134531 0 0) 0x7f919c06f200 con 0x7f919c0d09f0 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.501+0000 7f919bfff640 1 -- 172.21.15.106:0/2404290960 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0cccf0 con 0x7f919c15e930 2024-04-13T03:19:49.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.502+0000 7f919bfff640 1 --2- 172.21.15.106:0/2404290960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c15e930 0x7f919c0b5350 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097a0 tx=0x7f918c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a0dc1e3925a572 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.502+0000 7f919affd640 1 -- 172.21.15.106:0/2404290960 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c035020 con 0x7f919c15e930 2024-04-13T03:19:49.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.502+0000 7f919affd640 1 -- 172.21.15.106:0/2404290960 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c031830 con 0x7f919c15e930 2024-04-13T03:19:49.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.502+0000 7f919affd640 1 -- 172.21.15.106:0/2404290960 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c031c00 con 0x7f919c15e930 2024-04-13T03:19:49.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 -- 172.21.15.106:0/2404290960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c15e930 msgr2=0x7f919c0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 --2- 172.21.15.106:0/2404290960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c15e930 0x7f919c0b5350 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097a0 tx=0x7f918c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 -- 172.21.15.106:0/2404290960 shutdown_connections 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 --2- 172.21.15.106:0/2404290960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c15e930 0x7f919c0b5350 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 -- 172.21.15.106:0/2404290960 >> 172.21.15.106:0/2404290960 conn(0x7f919c0cef80 msgr2=0x7f919c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 -- 172.21.15.106:0/2404290960 shutdown_connections 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.503+0000 7f91a231e640 1 -- 172.21.15.106:0/2404290960 wait complete. 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.504+0000 7f91a231e640 1 Processor -- start 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.504+0000 7f91a231e640 1 -- start start 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f919c0d09f0 0x7f919c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f919c15e930 0x7f919c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0ce830 con 0x7f919c06ed70 2024-04-13T03:19:49.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0cd2b0 con 0x7f919c15e930 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f919c0ce9b0 con 0x7f919c0d09f0 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37938/0 (socket says 172.21.15.106:37938) 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 -- 172.21.15.106:0/2083478124 learned_addr learned my addr 172.21.15.106:0/2083478124 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 -- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f919c0d09f0 msgr2=0x7f919c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f919c0d09f0 0x7f919c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 -- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f919c15e930 msgr2=0x7f919c0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f919c15e930 0x7f919c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 -- 172.21.15.106:0/2083478124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0cab30 con 0x7f919c06ed70 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f919bfff640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f918c034040 tx=0x7f918c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c004230 con 0x7f919c06ed70 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c0043f0 con 0x7f919c06ed70 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c0cae10 con 0x7f919c06ed70 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c003850 con 0x7f919c06ed70 2024-04-13T03:19:49.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.505+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c0cb3d0 con 0x7f919c06ed70 2024-04-13T03:19:49.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.506+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c068a10 con 0x7f919c06ed70 2024-04-13T03:19:49.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.507+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f918c038020 con 0x7f919c06ed70 2024-04-13T03:19:49.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.507+0000 7f9198ff9640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 0x7f918806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.507+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f918c0ad980 con 0x7f919c06ed70 2024-04-13T03:19:49.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.507+0000 7f919b7fe640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 0x7f918806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.508+0000 7f919b7fe640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 0x7f918806edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f919c160d10 tx=0x7f9190007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.511+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f918c040070 con 0x7f919c06ed70 2024-04-13T03:19:49.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.685+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f919c0cfce0 con 0x7f919c06ed70 2024-04-13T03:19:49.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.685+0000 7f9198ff9640 1 -- 172.21.15.106:0/2083478124 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f918c07bc90 con 0x7f919c06ed70 2024-04-13T03:19:49.687 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-13T03:19:49.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 msgr2=0x7f918806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 0x7f918806edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f919c160d10 tx=0x7f9190007990 comp rx=0 tx=0).stop 2024-04-13T03:19:49.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 msgr2=0x7f919c14c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f918c034040 tx=0x7f918c0046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 shutdown_connections 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f918806c8f0 0x7f918806edb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f919c0d09f0 0x7f919c0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f919c15e930 0x7f919c0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 --2- 172.21.15.106:0/2083478124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f919c06ed70 0x7f919c14c8d0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 >> 172.21.15.106:0/2083478124 conn(0x7f919c0cef80 msgr2=0x7f919c0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 shutdown_connections 2024-04-13T03:19:49.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.687+0000 7f91a231e640 1 -- 172.21.15.106:0/2083478124 wait complete. 2024-04-13T03:19:49.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T03:19:49.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.801+0000 7f691d508640 1 Processor -- start 2024-04-13T03:19:49.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.801+0000 7f691d508640 1 -- start start 2024-04-13T03:19:49.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f691d508640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6918145f60 0x7f691806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f691d508640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6918073520 0x7f691815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f691d508640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691806caf0 con 0x7f691806db90 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f691d508640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f691806cdd0 con 0x7f6918145f60 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f691d508640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f691806d0b0 con 0x7f6918073520 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6918145f60 0x7f691806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6918145f60 0x7f691806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37952/0 (socket says 172.21.15.106:37952) 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 -- 172.21.15.106:0/351014804 learned_addr learned my addr 172.21.15.106:0/351014804 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f69177fe640 1 --2- 172.21.15.106:0/351014804 >> v2:172.21.15.114:3300/0 conn(0x7f6918073520 0x7f691815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 -- 172.21.15.106:0/351014804 >> v2:172.21.15.114:3300/0 conn(0x7f6918073520 msgr2=0x7f691815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 --2- 172.21.15.106:0/351014804 >> v2:172.21.15.114:3300/0 conn(0x7f6918073520 0x7f691815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 -- 172.21.15.106:0/351014804 >> v1:172.21.15.114:6789/0 conn(0x7f691806db90 legacy=0x7f6918072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.802+0000 7f6916ffd640 1 -- 172.21.15.106:0/351014804 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691806d390 con 0x7f6918145f60 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.803+0000 7f6916ffd640 1 --2- 172.21.15.106:0/351014804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918145f60 0x7f691806c170 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f69181601b0 tx=0x7f690802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ccdb1a215ef54 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.803+0000 7f69167fc640 1 -- 172.21.15.106:0/351014804 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908035020 con 0x7f6918145f60 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.803+0000 7f69167fc640 1 -- 172.21.15.106:0/351014804 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6908031870 con 0x7f6918145f60 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.803+0000 7f69167fc640 1 -- 172.21.15.106:0/351014804 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908031c40 con 0x7f6918145f60 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 -- 172.21.15.106:0/351014804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918145f60 msgr2=0x7f691806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 --2- 172.21.15.106:0/351014804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918145f60 0x7f691806c170 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f69181601b0 tx=0x7f690802b910 comp rx=0 tx=0).stop 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 -- 172.21.15.106:0/351014804 shutdown_connections 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 --2- 172.21.15.106:0/351014804 >> v2:172.21.15.114:3300/0 conn(0x7f6918073520 0x7f691815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 --2- 172.21.15.106:0/351014804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918145f60 0x7f691806c170 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 -- 172.21.15.106:0/351014804 >> 172.21.15.106:0/351014804 conn(0x7f691806f0d0 msgr2=0x7f691806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 -- 172.21.15.106:0/351014804 shutdown_connections 2024-04-13T03:19:49.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.804+0000 7f691d508640 1 -- 172.21.15.106:0/351014804 wait complete. 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.805+0000 7f691d508640 1 Processor -- start 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.805+0000 7f691d508640 1 -- start start 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f691806db90 0x7f691813e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6918145f60 0x7f69181444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f691806c830 con 0x7f6918073520 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f691806c6b0 con 0x7f691806db90 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f69180cdaa0 con 0x7f6918145f60 2024-04-13T03:19:49.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37966/0 (socket says 172.21.15.106:37966) 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 -- 172.21.15.106:0/3302861010 learned_addr learned my addr 172.21.15.106:0/3302861010 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f69177fe640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6918145f60 0x7f69181444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f6916ffd640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f691806db90 0x7f691813e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 -- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6918145f60 msgr2=0x7f69181444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6918145f60 0x7f69181444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 -- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f691806db90 msgr2=0x7f691813e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f691806db90 0x7f691813e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 -- 172.21.15.106:0/3302861010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69181e5c50 con 0x7f6918073520 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690ffff640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f690000c430 tx=0x7f690000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011790 con 0x7f6918073520 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900017070 con 0x7f6918073520 2024-04-13T03:19:49.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69181e5f30 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011bb0 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.806+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69181e64f0 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.808+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180d12a0 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.808+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f690001c460 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.808+0000 7f690f7fe640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 0x7f68f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.808+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f690008bab0 con 0x7f6918073520 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.808+0000 7f6916ffd640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 0x7f68f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:49.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.809+0000 7f6916ffd640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 0x7f68f806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6908036000 tx=0x7f690803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:49.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.813+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69180d12a0 con 0x7f6918073520 2024-04-13T03:19:49.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.988+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6918068a10 con 0x7f6918073520 2024-04-13T03:19:49.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.989+0000 7f690f7fe640 1 -- 172.21.15.106:0/3302861010 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7f6900059650 con 0x7f6918073520 2024-04-13T03:19:49.991 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 msgr2=0x7f68f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 0x7f68f806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6908036000 tx=0x7f690803c040 comp rx=0 tx=0).stop 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 msgr2=0x7f69181e6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f690000c430 tx=0x7f690000c8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 shutdown_connections 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f68f806c8f0 0x7f68f806edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6918145f60 0x7f69181444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6918073520 0x7f69181e6cf0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 --2- 172.21.15.106:0/3302861010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f691806db90 0x7f691813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:49.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 >> 172.21.15.106:0/3302861010 conn(0x7f691806f0d0 msgr2=0x7f69180cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:49.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 shutdown_connections 2024-04-13T03:19:49.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:49.991+0000 7f691d508640 1 -- 172.21.15.106:0/3302861010 wait complete. 2024-04-13T03:19:50.004 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.103+0000 7f81e27aa640 1 Processor -- start 2024-04-13T03:19:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.103+0000 7f81e27aa640 1 -- start start 2024-04-13T03:19:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81dc15e9c0 0x7f81dc0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 0x7f81dc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc0b6700 0x7f81dc06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc0d27f0 con 0x7f81dc15e9c0 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc15eda0 con 0x7f81dc1624b0 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e27aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc162c20 con 0x7f81dc0b6700 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81dc15e9c0 0x7f81dc0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f81dc15e9c0 0x7f81dc0ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37974/0 (socket says 172.21.15.106:37974) 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 -- 172.21.15.106:0/470698542 learned_addr learned my addr 172.21.15.106:0/470698542 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81e0d47640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc0b6700 0x7f81dc06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81db7fe640 1 --2- 172.21.15.106:0/470698542 >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 0x7f81dc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 -- 172.21.15.106:0/470698542 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc0b6700 msgr2=0x7f81dc06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc0b6700 0x7f81dc06b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 -- 172.21.15.106:0/470698542 >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 msgr2=0x7f81dc0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 --2- 172.21.15.106:0/470698542 >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 0x7f81dc0b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81dbfff640 1 -- 172.21.15.106:0/470698542 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc162f00 con 0x7f81dc15e9c0 2024-04-13T03:19:50.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.104+0000 7f81db7fe640 1 --2- 172.21.15.106:0/470698542 >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 0x7f81dc0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.105+0000 7f81dbfff640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0ba290 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0b7330 tx=0x7f81cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575c5c97041568db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.105+0000 7f81daffd640 1 -- 172.21.15.106:0/470698542 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc035020 con 0x7f81dc15e9c0 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.105+0000 7f81daffd640 1 -- 172.21.15.106:0/470698542 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81cc031870 con 0x7f81dc15e9c0 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.105+0000 7f81daffd640 1 -- 172.21.15.106:0/470698542 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc031c40 con 0x7f81dc15e9c0 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 -- 172.21.15.106:0/470698542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 msgr2=0x7f81dc0ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0ba290 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0b7330 tx=0x7f81cc02b910 comp rx=0 tx=0).stop 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 -- 172.21.15.106:0/470698542 shutdown_connections 2024-04-13T03:19:50.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc0b6700 0x7f81dc06b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 --2- 172.21.15.106:0/470698542 >> v2:172.21.15.114:3300/0 conn(0x7f81dc1624b0 0x7f81dc0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 --2- 172.21.15.106:0/470698542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0ba290 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 -- 172.21.15.106:0/470698542 >> 172.21.15.106:0/470698542 conn(0x7f81dc0d11c0 msgr2=0x7f81dc0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 -- 172.21.15.106:0/470698542 shutdown_connections 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.106+0000 7f81e27aa640 1 -- 172.21.15.106:0/470698542 wait complete. 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.107+0000 7f81e27aa640 1 Processor -- start 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.107+0000 7f81e27aa640 1 -- start start 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81dc0b6700 0x7f81dc148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc1624b0 0x7f81dc0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc162910 con 0x7f81dc15e9c0 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2a40 con 0x7f81dc0b6700 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc06be20 con 0x7f81dc1624b0 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37990/0 (socket says 172.21.15.106:37990) 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 -- 172.21.15.106:0/3568630764 learned_addr learned my addr 172.21.15.106:0/3568630764 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 -- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc1624b0 msgr2=0x7f81dc0c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e0d47640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc1624b0 0x7f81dc0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc1624b0 0x7f81dc0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 -- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81dc0b6700 msgr2=0x7f81dc148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81dbfff640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81dc0b6700 0x7f81dc148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81dc0b6700 0x7f81dc148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 -- 172.21.15.106:0/3568630764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0ccf60 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81db7fe640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c820 tx=0x7f81d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0015070 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d0007a30 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc0cd240 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0007de0 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.108+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc0cd800 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.109+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc068a10 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.110+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81d0018070 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.110+0000 7f81d8ff9640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 0x7f81c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.110+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f81d008bbf0 con 0x7f81dc15e9c0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.110+0000 7f81dbfff640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 0x7f81c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.110+0000 7f81dbfff640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 0x7f81c806edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.114+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81d0059de0 con 0x7f81dc15e9c0 2024-04-13T03:19:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.324+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81dc0b4590 con 0x7f81dc15e9c0 2024-04-13T03:19:50.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.325+0000 7f81d8ff9640 1 -- 172.21.15.106:0/3568630764 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5194 (secure 0 0 0) 0x7f81d00596e0 con 0x7f81dc15e9c0 2024-04-13T03:19:50.327 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:50.327 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:43.045274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9555,"qdb_cluster":[9555]},"id":8}]} 2024-04-13T03:19:50.327 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 98 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 msgr2=0x7f81c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 0x7f81c806edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc008030 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 msgr2=0x7f81dc0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c820 tx=0x7f81d000cce0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 shutdown_connections 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f81c806c8f0 0x7f81c806edb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f81dc1624b0 0x7f81dc0c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81dc15e9c0 0x7f81dc0c8270 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 --2- 172.21.15.106:0/3568630764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81dc0b6700 0x7f81dc148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 >> 172.21.15.106:0/3568630764 conn(0x7f81dc0d11c0 msgr2=0x7f81dc06c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 shutdown_connections 2024-04-13T03:19:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.327+0000 7f81e27aa640 1 -- 172.21.15.106:0/3568630764 wait complete. 2024-04-13T03:19:50.340 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:50.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.438+0000 7f0f1f088640 1 Processor -- start 2024-04-13T03:19:50.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.438+0000 7f0f1f088640 1 -- start start 2024-04-13T03:19:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.438+0000 7f0f1f088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 0x7f0f18072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.438+0000 7f0f1f088640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f1806e150 0x7f0f1806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1f088640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f180998d0 con 0x7f0f180cdf50 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1f088640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0f18073020 con 0x7f0f1806e150 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1f088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0f18073300 con 0x7f0f1806cb10 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1d625640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f1806e150 0x7f0f1806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1ce24640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0f180cdf50 0x7f0f180ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58934/0 (socket says 172.21.15.106:58934) 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1ce24640 1 -- 172.21.15.106:0/1956130186 learned_addr learned my addr 172.21.15.106:0/1956130186 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f17fff640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 0x7f0f18072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1d625640 1 -- 172.21.15.106:0/1956130186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 msgr2=0x7f0f18072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1d625640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 0x7f0f18072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1d625640 1 -- 172.21.15.106:0/1956130186 >> v1:172.21.15.106:6789/0 conn(0x7f0f180cdf50 legacy=0x7f0f180ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f177fe640 1 -- 172.21.15.106:0/1956130186 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379152915 0 0) 0x7f0f180998d0 con 0x7f0f180cdf50 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f1d625640 1 -- 172.21.15.106:0/1956130186 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f180735e0 con 0x7f0f1806e150 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.439+0000 7f0f17fff640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 0x7f0f18072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:50.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1d625640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806e150 0x7f0f1806e600 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009d90 tx=0x7f0f1002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117102fadf31a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f177fe640 1 -- 172.21.15.106:0/1956130186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10039070 con 0x7f0f1806e150 2024-04-13T03:19:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f177fe640 1 -- 172.21.15.106:0/1956130186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1002cc50 con 0x7f0f1806e150 2024-04-13T03:19:50.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f177fe640 1 -- 172.21.15.106:0/1956130186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10034440 con 0x7f0f1806e150 2024-04-13T03:19:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 -- 172.21.15.106:0/1956130186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806e150 msgr2=0x7f0f1806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806e150 0x7f0f1806e600 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009d90 tx=0x7f0f1002bea0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 -- 172.21.15.106:0/1956130186 shutdown_connections 2024-04-13T03:19:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806e150 0x7f0f1806e600 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 --2- 172.21.15.106:0/1956130186 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f1806cb10 0x7f0f18072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 -- 172.21.15.106:0/1956130186 >> 172.21.15.106:0/1956130186 conn(0x7f0f180b1f10 msgr2=0x7f0f180b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 -- 172.21.15.106:0/1956130186 shutdown_connections 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.440+0000 7f0f1f088640 1 -- 172.21.15.106:0/1956130186 wait complete. 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.441+0000 7f0f1f088640 1 Processor -- start 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 -- start start 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f1806e150 0x7f0f180c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f180cdf50 0x7f0f1815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18072de0 con 0x7f0f1806cb10 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18072c60 con 0x7f0f1806e150 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1f088640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1806eb40 con 0x7f0f180cdf50 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1ce24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1ce24640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38002/0 (socket says 172.21.15.106:38002) 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.442+0000 7f0f1ce24640 1 -- 172.21.15.106:0/2011654516 learned_addr learned my addr 172.21.15.106:0/2011654516 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 -- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f180cdf50 msgr2=0x7f0f1815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f180cdf50 0x7f0f1815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 -- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f1806e150 msgr2=0x7f0f180c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f1806e150 0x7f0f180c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 -- 172.21.15.106:0/2011654516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f18152300 con 0x7f0f1806cb10 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1ce24640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c430 tx=0x7f0f0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0074e0 con 0x7f0f1806cb10 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0c0047e0 con 0x7f0f1806cb10 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181525e0 con 0x7f0f1806cb10 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c004bd0 con 0x7f0f1806cb10 2024-04-13T03:19:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.443+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f18152ba0 con 0x7f0f1806cb10 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.444+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18068a10 con 0x7f0f1806cb10 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.444+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f0c011070 con 0x7f0f1806cb10 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.445+0000 7f0f157fa640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 0x7f0efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.445+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f0f0c08b9c0 con 0x7f0f1806cb10 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.445+0000 7f0f17fff640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 0x7f0efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.445+0000 7f0f17fff640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 0x7f0efc06edb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.449+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f0c08de20 con 0x7f0f1806cb10 2024-04-13T03:19:50.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.627+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f1815a3d0 con 0x7f0f1806cb10 2024-04-13T03:19:50.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.628+0000 7f0f157fa640 1 -- 172.21.15.106:0/2011654516 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17251 (secure 0 0 0) 0x7f0f0c00e280 con 0x7f0f1806cb10 2024-04-13T03:19:50.630 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:50.630 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":123,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:38.053684+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6835/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 msgr2=0x7f0efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 0x7f0efc06edb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04008040 comp rx=0 tx=0).stop 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 msgr2=0x7f0f180cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c430 tx=0x7f0f0c00c8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 shutdown_connections 2024-04-13T03:19:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0efc06c8f0 0x7f0efc06edb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0f180cdf50 0x7f0f1815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f1806e150 0x7f0f180c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 --2- 172.21.15.106:0/2011654516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f1806cb10 0x7f0f180cb530 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 >> 172.21.15.106:0/2011654516 conn(0x7f0f180b1f10 msgr2=0x7f0f180723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 shutdown_connections 2024-04-13T03:19:50.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.630+0000 7f0f1f088640 1 -- 172.21.15.106:0/2011654516 wait complete. 2024-04-13T03:19:50.643 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:19:50.643 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.743+0000 7f9faeb9f640 1 Processor -- start 2024-04-13T03:19:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.743+0000 7f9faeb9f640 1 -- start start 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9faeb9f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9fa80b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9faeb9f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa80d2f30 con 0x7f9fa80d0480 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9faeb9f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa80cf0f0 con 0x7f9fa80ceb70 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9faeb9f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80cf390 con 0x7f9fa815e8a0 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9fae39e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9fa80d0480 0x7f9fa80d33f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58936/0 (socket says 172.21.15.106:58936) 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9fae39e640 1 -- 172.21.15.106:0/601138612 learned_addr learned my addr 172.21.15.106:0/601138612 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4127691214 0 0) 0x7f9fa0001490 con 0x7f9fa80d0480 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9fa80b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f94003520 con 0x7f9fa80d0480 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2013966203 0 0) 0x7f9f94003520 con 0x7f9fa80d0480 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 msgr2=0x7f9fa80b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9fa80b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 >> v1:172.21.15.114:6789/0 conn(0x7f9fa80ceb70 legacy=0x7f9fa80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80cf630 con 0x7f9fa80d0480 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f9fa0001bc0 con 0x7f9fa80d0480 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 >> v1:172.21.15.106:6789/0 conn(0x7f9fa80d0480 legacy=0x7f9fa80d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9f94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94004390 0x7f9f94004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9f940050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80cf390 con 0x7f9f940039e0 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9400b9c0 con 0x7f9f94004390 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9400bbc0 con 0x7f9fa815e8a0 2024-04-13T03:19:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.744+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9f94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fad39c640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94004390 0x7f9f94004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fae39e640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9f940050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 -- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 msgr2=0x7f9f940050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa815e8a0 0x7f9f940050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 -- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94004390 msgr2=0x7f9f94004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94004390 0x7f9f94004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 -- 172.21.15.106:0/601138612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f94003520 con 0x7f9f940039e0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.745+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9f94003dc0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ac0 tx=0x7f9f94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8183c159dfde24 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c039070 con 0x7f9f940039e0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c02c6b0 con 0x7f9f940039e0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9facb9b640 1 -- 172.21.15.106:0/601138612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c009690 con 0x7f9f940039e0 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 -- 172.21.15.106:0/601138612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 msgr2=0x7f9f94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/601138612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9f94003dc0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ac0 tx=0x7f9f94005830 comp rx=0 tx=0).stop 2024-04-13T03:19:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9fae39e640 1 -- 172.21.15.106:0/601138612 reap_dead start 2024-04-13T03:19:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 -- 172.21.15.106:0/601138612 shutdown_connections 2024-04-13T03:19:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 -- 172.21.15.106:0/601138612 >> 172.21.15.106:0/601138612 conn(0x7f9fa809d8c0 msgr2=0x7f9fa80b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 -- 172.21.15.106:0/601138612 shutdown_connections 2024-04-13T03:19:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.746+0000 7f9faeb9f640 1 -- 172.21.15.106:0/601138612 wait complete. 2024-04-13T03:19:50.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.747+0000 7f9faeb9f640 1 Processor -- start 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.747+0000 7f9faeb9f640 1 -- start start 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa80b1f70 0x7f9fa815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 0x7f9fa8152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80b9d70 con 0x7f9f940039e0 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80b9bf0 con 0x7f9fa80b1f70 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9faeb9f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80b9ef0 con 0x7f9fa80b23d0 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38010/0 (socket says 172.21.15.106:38010) 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 -- 172.21.15.106:0/2347037333 learned_addr learned my addr 172.21.15.106:0/2347037333 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fad39c640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa80b1f70 0x7f9fa815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fae39e640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 0x7f9fa8152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 -- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 msgr2=0x7f9fa8152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 0x7f9fa8152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 -- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa80b1f70 msgr2=0x7f9fa815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa80b1f70 0x7f9fa815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fadb9d640 1 -- 172.21.15.106:0/2347037333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa815bdd0 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.748+0000 7f9fae39e640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 0x7f9fa8152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9fadb9d640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c042020 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c03a040 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa806cb40 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c039070 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.749+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa806d100 con 0x7f9f940039e0 2024-04-13T03:19:50.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.750+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa804fd50 con 0x7f9f940039e0 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.750+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f9c045020 con 0x7f9f940039e0 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.750+0000 7f9f9a7fc640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.751+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f9f9c0ad210 con 0x7f9f940039e0 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.751+0000 7f9fad39c640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.751+0000 7f9fad39c640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 0x7f9f8c06ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9fa81f7960 tx=0x7f9fa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:50.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.755+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f9c04c070 con 0x7f9f940039e0 2024-04-13T03:19:50.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.966+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fa806efb0 con 0x7f9f940039e0 2024-04-13T03:19:50.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.966+0000 7f9f9a7fc640 1 -- 172.21.15.106:0/2347037333 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5194 (secure 0 0 0) 0x7f9f9c07b370 con 0x7f9f940039e0 2024-04-13T03:19:50.968 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:50.968 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:43.045274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9555,"qdb_cluster":[9555]},"id":8}]} 2024-04-13T03:19:50.969 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 98 2024-04-13T03:19:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 msgr2=0x7f9f8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 0x7f9f8c06ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9fa81f7960 tx=0x7f9fa4009040 comp rx=0 tx=0).stop 2024-04-13T03:19:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 msgr2=0x7f9fa80c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:50.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c004730 comp rx=0 tx=0).stop 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 shutdown_connections 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9fa80b23d0 0x7f9fa8152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fa80b1f70 0x7f9fa815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 --2- 172.21.15.106:0/2347037333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f940039e0 0x7f9fa80c7ee0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 >> 172.21.15.106:0/2347037333 conn(0x7f9fa809d8c0 msgr2=0x7f9fa80cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 shutdown_connections 2024-04-13T03:19:50.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:50.968+0000 7f9faeb9f640 1 -- 172.21.15.106:0/2347037333 wait complete. 2024-04-13T03:19:50.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:19:51.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.080+0000 7feb77915640 1 Processor -- start 2024-04-13T03:19:51.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 -- start start 2024-04-13T03:19:51.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb70156b50 0x7feb70160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb7015cc10 0x7feb7015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb7006dd70 0x7feb7006e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feb7015d690 con 0x7feb70156b50 2024-04-13T03:19:51.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7feb7015d970 con 0x7feb7006dd70 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb77915640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feb70160410 con 0x7feb7015cc10 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.081+0000 7feb756b1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb70156b50 0x7feb70160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb70156b50 0x7feb70160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38026/0 (socket says 172.21.15.106:38026) 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 -- 172.21.15.106:0/4060497345 learned_addr learned my addr 172.21.15.106:0/4060497345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 -- 172.21.15.106:0/4060497345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb7015cc10 msgr2=0x7feb7015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb74eb0640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb7015cc10 0x7feb7015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb75eb2640 1 --2- 172.21.15.106:0/4060497345 >> v2:172.21.15.114:3300/0 conn(0x7feb7006dd70 0x7feb7006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb7015cc10 0x7feb7015d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 -- 172.21.15.106:0/4060497345 >> v2:172.21.15.114:3300/0 conn(0x7feb7006dd70 msgr2=0x7feb7006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 --2- 172.21.15.106:0/4060497345 >> v2:172.21.15.114:3300/0 conn(0x7feb7006dd70 0x7feb7006e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.082+0000 7feb756b1640 1 -- 172.21.15.106:0/4060497345 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701606f0 con 0x7feb70156b50 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb756b1640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb70156b50 0x7feb70160c10 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7feb7015a750 tx=0x7feb6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d66c6587735f513 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb67fff640 1 -- 172.21.15.106:0/4060497345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c035020 con 0x7feb70156b50 2024-04-13T03:19:51.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb67fff640 1 -- 172.21.15.106:0/4060497345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c031870 con 0x7feb70156b50 2024-04-13T03:19:51.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb67fff640 1 -- 172.21.15.106:0/4060497345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c031c40 con 0x7feb70156b50 2024-04-13T03:19:51.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 -- 172.21.15.106:0/4060497345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb70156b50 msgr2=0x7feb70160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb70156b50 0x7feb70160c10 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7feb7015a750 tx=0x7feb6c02b910 comp rx=0 tx=0).stop 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 -- 172.21.15.106:0/4060497345 shutdown_connections 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 --2- 172.21.15.106:0/4060497345 >> v2:172.21.15.114:3300/0 conn(0x7feb7006dd70 0x7feb7006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb7015cc10 0x7feb7015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 --2- 172.21.15.106:0/4060497345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb70156b50 0x7feb70160c10 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 -- 172.21.15.106:0/4060497345 >> 172.21.15.106:0/4060497345 conn(0x7feb700b1f10 msgr2=0x7feb700b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 -- 172.21.15.106:0/4060497345 shutdown_connections 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.083+0000 7feb77915640 1 -- 172.21.15.106:0/4060497345 wait complete. 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.084+0000 7feb77915640 1 Processor -- start 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.084+0000 7feb77915640 1 -- start start 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 0x7feb70156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb70156b50 0x7feb700ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feb700afc20 con 0x7feb7015cc10 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb700afaa0 con 0x7feb7006dd70 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb77915640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7feb7006bcb0 con 0x7feb70156b50 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb75eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb75eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38032/0 (socket says 172.21.15.106:38032) 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb75eb2640 1 -- 172.21.15.106:0/1183514395 learned_addr learned my addr 172.21.15.106:0/1183514395 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb756b1640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 0x7feb70156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.085+0000 7feb74eb0640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb70156b50 0x7feb700ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 -- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb70156b50 msgr2=0x7feb700ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb70156b50 0x7feb700ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 -- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 msgr2=0x7feb70156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 0x7feb70156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 -- 172.21.15.106:0/1183514395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb700cff00 con 0x7feb7015cc10 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb756b1640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 0x7feb70156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb75eb2640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7feb6800cbb0 tx=0x7feb6800a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68015070 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6800abd0 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800d040 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb700cee60 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.086+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb700cf420 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.087+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb70068a10 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.087+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb68018070 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.088+0000 7feb65ffb640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 0x7feb5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.088+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7feb6808b920 con 0x7feb7015cc10 2024-04-13T03:19:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.088+0000 7feb756b1640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 0x7feb5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.088+0000 7feb756b1640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 0x7feb5406edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7feb6c036000 tx=0x7feb6c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.092+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb68059ba0 con 0x7feb7015cc10 2024-04-13T03:19:51.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.304+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb7006ba90 con 0x7feb7015cc10 2024-04-13T03:19:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.304+0000 7feb65ffb640 1 -- 172.21.15.106:0/1183514395 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5194 (secure 0 0 0) 0x7feb680594a0 con 0x7feb7015cc10 2024-04-13T03:19:51.306 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:51.306 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T03:19:14.646491+0000","modified":"2024-04-13T03:19:43.045274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9555":{"gid":9555,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6837/856679728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":856679728},{"type":"v1","addr":"172.21.15.106:6837","nonce":856679728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9555,"qdb_cluster":[9555]},"id":8}]} 2024-04-13T03:19:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 98 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 msgr2=0x7feb5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 0x7feb5406edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7feb6c036000 tx=0x7feb6c03c040 comp rx=0 tx=0).stop 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 msgr2=0x7feb700ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7feb6800cbb0 tx=0x7feb6800a4c0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 shutdown_connections 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7feb5406c8f0 0x7feb5406edb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7feb70156b50 0x7feb700ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb7015cc10 0x7feb700ae710 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 --2- 172.21.15.106:0/1183514395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb7006dd70 0x7feb70156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 >> 172.21.15.106:0/1183514395 conn(0x7feb700b1f10 msgr2=0x7feb700b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 shutdown_connections 2024-04-13T03:19:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.306+0000 7feb77915640 1 -- 172.21.15.106:0/1183514395 wait complete. 2024-04-13T03:19:51.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:51.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.417+0000 7efce1c2d640 1 Processor -- start 2024-04-13T03:19:51.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.417+0000 7efce1c2d640 1 -- start start 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efce1c2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcdc156b50 0x7efcdc160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efce1c2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 0x7efcdc15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efce1c2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc15d690 con 0x7efcdc15cc10 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efce1c2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efcdc15d970 con 0x7efcdc156b50 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efce1c2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc160410 con 0x7efcdc06e480 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcdc156b50 0x7efcdc160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdaffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efcdc15cc10 0x7efcdc06dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58940/0 (socket says 172.21.15.106:58940) 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdaffd640 1 -- 172.21.15.106:0/3338757316 learned_addr learned my addr 172.21.15.106:0/3338757316 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcda7fc640 1 -- 172.21.15.106:0/3338757316 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74940548 0 0) 0x7efcdc15d690 con 0x7efcdc15cc10 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcda7fc640 1 -- 172.21.15.106:0/3338757316 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcc4003520 con 0x7efcdc15cc10 2024-04-13T03:19:51.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdbfff640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 0x7efcdc15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3338757316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 msgr2=0x7efcdc15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdb7fe640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 0x7efcdc15d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3338757316 >> v1:172.21.15.106:6789/0 conn(0x7efcdc15cc10 legacy=0x7efcdc06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3338757316 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc1606f0 con 0x7efcdc156b50 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.418+0000 7efcdbfff640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 0x7efcdc15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.419+0000 7efcdb7fe640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc156b50 0x7efcdc160c10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7efccc0097a0 tx=0x7efccc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0b2bf0e8b960d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.419+0000 7efcda7fc640 1 -- 172.21.15.106:0/3338757316 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc035020 con 0x7efcdc156b50 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.419+0000 7efcda7fc640 1 -- 172.21.15.106:0/3338757316 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc031830 con 0x7efcdc156b50 2024-04-13T03:19:51.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.419+0000 7efcda7fc640 1 -- 172.21.15.106:0/3338757316 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc031be0 con 0x7efcdc156b50 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 -- 172.21.15.106:0/3338757316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc156b50 msgr2=0x7efcdc160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc156b50 0x7efcdc160c10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7efccc0097a0 tx=0x7efccc02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 -- 172.21.15.106:0/3338757316 shutdown_connections 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc06e480 0x7efcdc15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3338757316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc156b50 0x7efcdc160c10 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 -- 172.21.15.106:0/3338757316 >> 172.21.15.106:0/3338757316 conn(0x7efcdc0b1f10 msgr2=0x7efcdc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 -- 172.21.15.106:0/3338757316 shutdown_connections 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.420+0000 7efce1c2d640 1 -- 172.21.15.106:0/3338757316 wait complete. 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.421+0000 7efce1c2d640 1 Processor -- start 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.421+0000 7efce1c2d640 1 -- start start 2024-04-13T03:19:51.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 0x7efcdc153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc15cc10 0x7efcdc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0afc20 con 0x7efcdc06e480 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0afaa0 con 0x7efcdc156b50 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efce1c2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc06bcb0 con 0x7efcdc15cc10 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38054/0 (socket says 172.21.15.106:38054) 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3611218784 learned_addr learned my addr 172.21.15.106:0/3611218784 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdbfff640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc15cc10 0x7efcdc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc15cc10 msgr2=0x7efcdc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdaffd640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 0x7efcdc153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc15cc10 0x7efcdc157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 msgr2=0x7efcdc153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 0x7efcdc153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 -- 172.21.15.106:0/3611218784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc159ea0 con 0x7efcdc06e480 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdaffd640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 0x7efcdc153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcdb7fe640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7efccc0098c0 tx=0x7efccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.422+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc041070 con 0x7efcdc06e480 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.423+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc03b900 con 0x7efcdc06e480 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.423+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc158db0 con 0x7efcdc06e480 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.423+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc035040 con 0x7efcdc06e480 2024-04-13T03:19:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.423+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc154480 con 0x7efcdc06e480 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.424+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc1592d0 con 0x7efcdc06e480 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.424+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efccc038020 con 0x7efcdc06e480 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.424+0000 7efcc3fff640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 0x7efcc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.424+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7efccc0ad7c0 con 0x7efcdc06e480 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.424+0000 7efcdaffd640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 0x7efcc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.425+0000 7efcdaffd640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 0x7efcc806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efcdc072a50 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.429+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcdc1592d0 con 0x7efcdc06e480 2024-04-13T03:19:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.605+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcdc068a10 con 0x7efcdc06e480 2024-04-13T03:19:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.607+0000 7efcc3fff640 1 -- 172.21.15.106:0/3611218784 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17251 (secure 0 0 0) 0x7efccc031800 con 0x7efcdc06e480 2024-04-13T03:19:51.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:51.609 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":123,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:38.053684+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-13T03:19:12.871192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-13T03:19:13.595223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6835/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.608+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 msgr2=0x7efcc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.608+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 0x7efcc806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efcdc072a50 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.608+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 msgr2=0x7efcdc0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.608+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7efccc0098c0 tx=0x7efccc0046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 shutdown_connections 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcc806c8f0 0x7efcc806edb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcdc15cc10 0x7efcdc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcdc156b50 0x7efcdc153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 --2- 172.21.15.106:0/3611218784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcdc06e480 0x7efcdc0cd1f0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 >> 172.21.15.106:0/3611218784 conn(0x7efcdc0b1f10 msgr2=0x7efcdc072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 shutdown_connections 2024-04-13T03:19:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.609+0000 7efce1c2d640 1 -- 172.21.15.106:0/3611218784 wait complete. 2024-04-13T03:19:51.622 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T03:19:51.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.722+0000 7f60c391e640 1 Processor -- start 2024-04-13T03:19:51.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.722+0000 7f60c391e640 1 -- start start 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c391e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 0x7f60bc0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c391e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 0x7f60bc0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c391e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60bc06e280 con 0x7f60bc0d0820 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c391e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0d0c90 con 0x7f60bc15e930 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c391e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc06e560 con 0x7f60bc06ed70 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c16ba640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 0x7f60bc0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c0eb9640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 0x7f60bc0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c16ba640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 0x7f60bc0b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:52226/0 (socket says 172.21.15.106:52226) 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c1ebb640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f60bc0d0820 0x7f60bc0cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46598/0 (socket says 172.21.15.106:46598) 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c16ba640 1 -- 172.21.15.106:0/2796023689 learned_addr learned my addr 172.21.15.106:0/2796023689 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c0eb9640 1 -- 172.21.15.106:0/2796023689 >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 msgr2=0x7f60bc0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c0eb9640 1 --2- 172.21.15.106:0/2796023689 >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 0x7f60bc0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c0eb9640 1 -- 172.21.15.106:0/2796023689 >> v1:172.21.15.114:6789/0 conn(0x7f60bc0d0820 legacy=0x7f60bc0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.723+0000 7f60c0eb9640 1 -- 172.21.15.106:0/2796023689 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0cccf0 con 0x7f60bc06ed70 2024-04-13T03:19:51.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.724+0000 7f60c0eb9640 1 --2- 172.21.15.106:0/2796023689 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 0x7f60bc0d02e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0097a0 tx=0x7f60ac02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b66754919d7945 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.724+0000 7f60b3fff640 1 -- 172.21.15.106:0/2796023689 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac035020 con 0x7f60bc06ed70 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.724+0000 7f60b3fff640 1 -- 172.21.15.106:0/2796023689 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac031830 con 0x7f60bc06ed70 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.724+0000 7f60b3fff640 1 -- 172.21.15.106:0/2796023689 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac031c00 con 0x7f60bc06ed70 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 -- 172.21.15.106:0/2796023689 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 msgr2=0x7f60bc0d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 --2- 172.21.15.106:0/2796023689 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 0x7f60bc0d02e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0097a0 tx=0x7f60ac02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 -- 172.21.15.106:0/2796023689 shutdown_connections 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 --2- 172.21.15.106:0/2796023689 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc06ed70 0x7f60bc0d02e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 --2- 172.21.15.106:0/2796023689 >> v2:172.21.15.114:3300/0 conn(0x7f60bc15e930 0x7f60bc0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 -- 172.21.15.106:0/2796023689 >> 172.21.15.106:0/2796023689 conn(0x7f60bc0cef80 msgr2=0x7f60bc0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 -- 172.21.15.106:0/2796023689 shutdown_connections 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.725+0000 7f60c391e640 1 -- 172.21.15.106:0/2796023689 wait complete. 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.726+0000 7f60c391e640 1 Processor -- start 2024-04-13T03:19:51.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.726+0000 7f60c391e640 1 -- start start 2024-04-13T03:19:51.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 0x7f60bc159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc15e930 0x7f60bc1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc06f1d0 con 0x7f60bc0d0820 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0cd2b0 con 0x7f60bc06ed70 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c391e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0ce830 con 0x7f60bc15e930 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38068/0 (socket says 172.21.15.106:38068) 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 -- 172.21.15.106:0/4038514240 learned_addr learned my addr 172.21.15.106:0/4038514240 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c16ba640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 0x7f60bc159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c1ebb640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc15e930 0x7f60bc1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 -- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc15e930 msgr2=0x7f60bc1581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc15e930 0x7f60bc1581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 -- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 msgr2=0x7f60bc159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 0x7f60bc159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 -- 172.21.15.106:0/4038514240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc158c20 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c16ba640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 0x7f60bc159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60c0eb9640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009890 tx=0x7f60ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.727+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac035040 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.728+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac03b920 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.728+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc158f00 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.728+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac03bd10 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.728+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc1594c0 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.729+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc068a10 con 0x7f60bc0d0820 2024-04-13T03:19:51.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.729+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60ac038020 con 0x7f60bc0d0820 2024-04-13T03:19:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.729+0000 7f60b1ffb640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 0x7f60a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.730+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f60ac0ade30 con 0x7f60bc0d0820 2024-04-13T03:19:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.730+0000 7f60c16ba640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 0x7f60a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:51.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.730+0000 7f60c16ba640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 0x7f60a006ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009b80 tx=0x7f60b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:51.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.734+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60ac031800 con 0x7f60bc0d0820 2024-04-13T03:19:51.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.943+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f60bc0b7690 con 0x7f60bc0d0820 2024-04-13T03:19:51.945 INFO:tasks.ceph.mon.a.smithi106.stderr:2024-04-13T03:19:51.943+0000 7f7078bf1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T03:19:51.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.945+0000 7f60b1ffb640 1 -- 172.21.15.106:0/4038514240 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7f60ac07b9e0 con 0x7f60bc0d0820 2024-04-13T03:19:51.946 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 msgr2=0x7f60a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 0x7f60a006ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009b80 tx=0x7f60b8008040 comp rx=0 tx=0).stop 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 msgr2=0x7f60bc14b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009890 tx=0x7f60ac0046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 shutdown_connections 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f60a006c920 0x7f60a006ede0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f60bc15e930 0x7f60bc1581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60bc0d0820 0x7f60bc14b9a0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 --2- 172.21.15.106:0/4038514240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60bc06ed70 0x7f60bc159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:51.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 >> 172.21.15.106:0/4038514240 conn(0x7f60bc0cef80 msgr2=0x7f60bc0b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:51.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 shutdown_connections 2024-04-13T03:19:51.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:51.948+0000 7f60c391e640 1 -- 172.21.15.106:0/4038514240 wait complete. 2024-04-13T03:19:51.951 INFO:tasks.ceph.mds.c.smithi106.stderr: -1315> 2024-04-13T03:19:44.064+0000 7f0472ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T03:19:51.962 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T03:19:51.977 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-13T03:19:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.062+0000 7f20b65a3640 1 Processor -- start 2024-04-13T03:19:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 -- start start 2024-04-13T03:19:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 0x7f20b00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20b015ed90 0x7f20b00d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00cfd20 con 0x7f20b00b3d60 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f20b0162af0 con 0x7f20b015ed90 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.063+0000 7f20b65a3640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20b0162dd0 con 0x7f20b015e9b0 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20affff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 0x7f20b00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20af7fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20b015ed90 0x7f20b00d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20b4b40640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f20b00b3d60 0x7f20b00d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46614/0 (socket says 172.21.15.106:46614) 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20affff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 0x7f20b00d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:48186/0 (socket says 172.21.15.106:48186) 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20b4b40640 1 -- 172.21.15.106:0/58345941 learned_addr learned my addr 172.21.15.106:0/58345941 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20af7fe640 1 -- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 msgr2=0x7f20b00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20af7fe640 1 --2- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 0x7f20b00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20af7fe640 1 -- 172.21.15.106:0/58345941 >> v1:172.21.15.114:6789/0 conn(0x7f20b00b3d60 legacy=0x7f20b00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.064+0000 7f20af7fe640 1 -- 172.21.15.106:0/58345941 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01630b0 con 0x7f20b015ed90 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.065+0000 7f20af7fe640 1 --2- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015ed90 0x7f20b00d45f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009790 tx=0x7f20a402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cbf620b7e97581 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.065+0000 7f20aeffd640 1 -- 172.21.15.106:0/58345941 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4035020 con 0x7f20b015ed90 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.065+0000 7f20aeffd640 1 -- 172.21.15.106:0/58345941 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a4031890 con 0x7f20b015ed90 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.065+0000 7f20aeffd640 1 -- 172.21.15.106:0/58345941 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4031c60 con 0x7f20b015ed90 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 -- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015ed90 msgr2=0x7f20b00d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 --2- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015ed90 0x7f20b00d45f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009790 tx=0x7f20a402b930 comp rx=0 tx=0).stop 2024-04-13T03:19:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 -- 172.21.15.106:0/58345941 shutdown_connections 2024-04-13T03:19:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 --2- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015ed90 0x7f20b00d45f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 --2- 172.21.15.106:0/58345941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015e9b0 0x7f20b00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 -- 172.21.15.106:0/58345941 >> 172.21.15.106:0/58345941 conn(0x7f20b008bf10 msgr2=0x7f20b00b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 -- 172.21.15.106:0/58345941 shutdown_connections 2024-04-13T03:19:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.066+0000 7f20b65a3640 1 -- 172.21.15.106:0/58345941 wait complete. 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.067+0000 7f20b65a3640 1 Processor -- start 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.067+0000 7f20b65a3640 1 -- start start 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015e9b0 0x7f20b0159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 0x7f20b00ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0162800 con 0x7f20b00b3d60 2024-04-13T03:19:52.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0162680 con 0x7f20b015e9b0 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b65a3640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f20b006eb20 con 0x7f20b015ff40 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38086/0 (socket says 172.21.15.106:38086) 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 -- 172.21.15.106:0/4229690010 learned_addr learned my addr 172.21.15.106:0/4229690010 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b4b40640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 0x7f20b00ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20af7fe640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015e9b0 0x7f20b0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 -- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 msgr2=0x7f20b00ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 0x7f20b00ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 -- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015e9b0 msgr2=0x7f20b0159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015e9b0 0x7f20b0159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 -- 172.21.15.106:0/4229690010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b015a3c0 con 0x7f20b00b3d60 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20b4b40640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 0x7f20b00ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20affff640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9b0 tx=0x7f20a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.068+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b00b3d60 2024-04-13T03:19:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.069+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a000c9a0 con 0x7f20b00b3d60 2024-04-13T03:19:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.069+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000cd90 con 0x7f20b00b3d60 2024-04-13T03:19:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.069+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b00b66e0 con 0x7f20b00b3d60 2024-04-13T03:19:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.069+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b015f460 con 0x7f20b00b3d60 2024-04-13T03:19:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.070+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b015fb80 con 0x7f20b00b3d60 2024-04-13T03:19:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.070+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20a001a070 con 0x7f20b00b3d60 2024-04-13T03:19:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.070+0000 7f20acff9640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 0x7f209c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.070+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6359+0+0 (secure 0 0 0) 0x7f20a008bcd0 con 0x7f20b00b3d60 2024-04-13T03:19:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.071+0000 7f20af7fe640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 0x7f209c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:52.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.071+0000 7f20af7fe640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 0x7f209c06ede0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036000 tx=0x7f20a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:52.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.075+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20a0059820 con 0x7f20b00b3d60 2024-04-13T03:19:52.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.292+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20b00d53a0 con 0x7f20b00b3d60 2024-04-13T03:19:52.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.946+0000 7f20acff9640 1 -- 172.21.15.106:0/4229690010 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) ==== 104+0+0 (secure 0 0 0) 0x7f20a005db20 con 0x7f20b00b3d60 2024-04-13T03:19:52.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 msgr2=0x7f209c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 0x7f209c06ede0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036000 tx=0x7f20a4008030 comp rx=0 tx=0).stop 2024-04-13T03:19:52.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 msgr2=0x7f20b00b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:52.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9b0 tx=0x7f20a000ee70 comp rx=0 tx=0).stop 2024-04-13T03:19:52.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 shutdown_connections 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f209c06c920 0x7f209c06ede0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f20b015ff40 0x7f20b00ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20b015e9b0 0x7f20b0159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 --2- 172.21.15.106:0/4229690010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b00b3d60 0x7f20b00b6050 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 >> 172.21.15.106:0/4229690010 conn(0x7f20b008bf10 msgr2=0x7f20b00b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 shutdown_connections 2024-04-13T03:19:52.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:52.949+0000 7f20b65a3640 1 -- 172.21.15.106:0/4229690010 wait complete. 2024-04-13T03:19:52.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T03:19:53.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.109+0000 7f3f44bf0640 1 Processor -- start 2024-04-13T03:19:53.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.109+0000 7f3f44bf0640 1 -- start start 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.110+0000 7f3f44bf0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f400d3490 0x7f3f400d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.110+0000 7f3f44bf0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4015c870 con 0x7f3f400b3920 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.110+0000 7f3f44bf0640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4015ee70 con 0x7f3f400d2370 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.110+0000 7f3f44bf0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3f400b1b00 con 0x7f3f400d3490 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f400d3490 0x7f3f400d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3f400b3920 0x7f3f4015f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60870/0 (socket says 172.21.15.106:60870) 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3ed9d640 1 -- 172.21.15.106:0/3835606617 learned_addr learned my addr 172.21.15.106:0/3835606617 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3e59c640 1 -- 172.21.15.106:0/3835606617 >> v1:172.21.15.114:6789/0 conn(0x7f3f400d2370 legacy=0x7f3f400b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3e59c640 1 -- 172.21.15.106:0/3835606617 >> v1:172.21.15.106:6789/0 conn(0x7f3f400b3920 legacy=0x7f3f4015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3e59c640 1 -- 172.21.15.106:0/3835606617 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f400d3db0 con 0x7f3f400d3490 2024-04-13T03:19:53.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.111+0000 7f3f3d59a640 1 -- 172.21.15.106:0/3835606617 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553371715 0 0) 0x7f3f4015c870 con 0x7f3f400b3920 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f3e59c640 1 --2- 172.21.15.106:0/3835606617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400d3870 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3f340097a0 tx=0x7f3f3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b914f0a937d75f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f3d59a640 1 -- 172.21.15.106:0/3835606617 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34035020 con 0x7f3f400d3490 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f3d59a640 1 -- 172.21.15.106:0/3835606617 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34031830 con 0x7f3f400d3490 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f3d59a640 1 -- 172.21.15.106:0/3835606617 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34031c00 con 0x7f3f400d3490 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 -- 172.21.15.106:0/3835606617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 msgr2=0x7f3f400d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/3835606617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400d3870 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3f340097a0 tx=0x7f3f3402b8d0 comp rx=0 tx=0).stop 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 -- 172.21.15.106:0/3835606617 shutdown_connections 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/3835606617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400d3870 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 -- 172.21.15.106:0/3835606617 >> 172.21.15.106:0/3835606617 conn(0x7f3f400b9760 msgr2=0x7f3f400b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 -- 172.21.15.106:0/3835606617 shutdown_connections 2024-04-13T03:19:53.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.112+0000 7f3f44bf0640 1 -- 172.21.15.106:0/3835606617 wait complete. 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.114+0000 7f3f44bf0640 1 Processor -- start 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.114+0000 7f3f44bf0640 1 -- start start 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 0x7f3f40156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 0x7f3f400b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f400d2ae0 con 0x7f3f400d3490 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3f400d5420 con 0x7f3f400b3920 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f3f400d2dc0 con 0x7f3f400ce250 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:53.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60764/0 (socket says 172.21.15.106:60764) 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 -- 172.21.15.106:0/730204297 learned_addr learned my addr 172.21.15.106:0/730204297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3e59c640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 0x7f3f40156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3ed9d640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 0x7f3f400b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 -- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 msgr2=0x7f3f400b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 0x7f3f400b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 -- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 msgr2=0x7f3f40156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 0x7f3f40156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 -- 172.21.15.106:0/730204297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f400b5a80 con 0x7f3f400d3490 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3ed9d640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 0x7f3f400b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3e59c640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 0x7f3f40156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f3dd9b640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c480 tx=0x7f3f3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300052d0 con 0x7f3f400d3490 2024-04-13T03:19:53.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30005490 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30003880 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f400b5d60 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.115+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f400cbcd0 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.116+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f400ce660 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.117+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f30003a20 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.117+0000 7f3f2effd640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.117+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6359+0+0 (secure 0 0 0) 0x7f3f3008ba20 con 0x7f3f400d3490 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.117+0000 7f3f3e59c640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:53.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.118+0000 7f3f3e59c640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 0x7f3f2006edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f34036000 tx=0x7f3f34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:53.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.122+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f400ce660 con 0x7f3f400d3490 2024-04-13T03:19:53.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:53.296+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3f40068a10 con 0x7f3f400d3490 2024-04-13T03:19:54.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.108+0000 7f3f2effd640 1 -- 172.21.15.106:0/730204297 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) ==== 181+0+0 (secure 0 0 0) 0x7f3f30059c50 con 0x7f3f400d3490 2024-04-13T03:19:54.110 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-13T03:19:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 msgr2=0x7f3f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 0x7f3f2006edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f34036000 tx=0x7f3f34008030 comp rx=0 tx=0).stop 2024-04-13T03:19:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 msgr2=0x7f3f400cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c480 tx=0x7f3f3000c940 comp rx=0 tx=0).stop 2024-04-13T03:19:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 shutdown_connections 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f3f400ce250 0x7f3f400b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f400d3490 0x7f3f400cdd10 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 --2- 172.21.15.106:0/730204297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f400b3920 0x7f3f40156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 >> 172.21.15.106:0/730204297 conn(0x7f3f400b9760 msgr2=0x7f3f400b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 shutdown_connections 2024-04-13T03:19:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.110+0000 7f3f44bf0640 1 -- 172.21.15.106:0/730204297 wait complete. 2024-04-13T03:19:54.129 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T03:19:54.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.238+0000 7f2e45a6d640 1 Processor -- start 2024-04-13T03:19:54.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.238+0000 7f2e45a6d640 1 -- start start 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e45a6d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e400cf7d0 0x7f2e4006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e45a6d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 0x7f2e400d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e45a6d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e400cfc60 con 0x7f2e4015e880 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e45a6d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2e400d30a0 con 0x7f2e400cf7d0 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e45a6d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2e400d2a50 con 0x7f2e400d3530 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e400cf7d0 0x7f2e4006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e400cf7d0 0x7f2e4006bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60778/0 (socket says 172.21.15.106:60778) 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 -- 172.21.15.106:0/1055428182 learned_addr learned my addr 172.21.15.106:0/1055428182 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3ffff640 1 --2- 172.21.15.106:0/1055428182 >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 0x7f2e400d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 -- 172.21.15.106:0/1055428182 >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 msgr2=0x7f2e400d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:54.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 --2- 172.21.15.106:0/1055428182 >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 0x7f2e400d39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 -- 172.21.15.106:0/1055428182 >> v1:172.21.15.114:6789/0 conn(0x7f2e4015e880 legacy=0x7f2e4006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3effd640 1 -- 172.21.15.106:0/1055428182 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e400d2d30 con 0x7f2e400cf7d0 2024-04-13T03:19:54.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.239+0000 7f2e3ffff640 1 --2- 172.21.15.106:0/1055428182 >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 0x7f2e400d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:54.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.240+0000 7f2e3effd640 1 --2- 172.21.15.106:0/1055428182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e400cf7d0 0x7f2e4006bff0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f2e400b6fa0 tx=0x7f2e3402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11bcff30d8c565 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:54.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.240+0000 7f2e3e7fc640 1 -- 172.21.15.106:0/1055428182 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34034070 con 0x7f2e400cf7d0 2024-04-13T03:19:54.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.240+0000 7f2e3e7fc640 1 -- 172.21.15.106:0/1055428182 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34030860 con 0x7f2e400cf7d0 2024-04-13T03:19:54.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.240+0000 7f2e3e7fc640 1 -- 172.21.15.106:0/1055428182 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34030c30 con 0x7f2e400cf7d0 2024-04-13T03:19:54.248 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 -- 172.21.15.106:0/1055428182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e400cf7d0 msgr2=0x7f2e4006bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/1055428182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e400cf7d0 0x7f2e4006bff0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f2e400b6fa0 tx=0x7f2e3402b9a0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 -- 172.21.15.106:0/1055428182 shutdown_connections 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/1055428182 >> v2:172.21.15.114:3300/0 conn(0x7f2e400d3530 0x7f2e400d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/1055428182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e400cf7d0 0x7f2e4006bff0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 -- 172.21.15.106:0/1055428182 >> 172.21.15.106:0/1055428182 conn(0x7f2e400b6280 msgr2=0x7f2e400b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 -- 172.21.15.106:0/1055428182 shutdown_connections 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.241+0000 7f2e45a6d640 1 -- 172.21.15.106:0/1055428182 wait complete. 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.242+0000 7f2e45a6d640 1 Processor -- start 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.242+0000 7f2e45a6d640 1 -- start start 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e400d3530 0x7f2e400ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 0x7f2e4015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400b3bd0 con 0x7f2e4015cfa0 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400b3a50 con 0x7f2e400d3530 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2e400b78c0 con 0x7f2e4015e880 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60790/0 (socket says 172.21.15.106:60790) 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 -- 172.21.15.106:0/5621095 learned_addr learned my addr 172.21.15.106:0/5621095 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3f7fe640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e400d3530 0x7f2e400ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3effd640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 0x7f2e4015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 -- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 msgr2=0x7f2e4015a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 0x7f2e4015a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 -- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e400d3530 msgr2=0x7f2e400ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e400d3530 0x7f2e400ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 -- 172.21.15.106:0/5621095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e401612e0 con 0x7f2e4015cfa0 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3effd640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 0x7f2e4015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:54.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e3ffff640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009fa0 tx=0x7f2e3800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3801a070 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38003bb0 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004030 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.243+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e40161000 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.244+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e400cdbd0 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.244+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e40068a10 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.245+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e3801d070 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.245+0000 7f2e27fff640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 0x7f2e2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.245+0000 7f2e3f7fe640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 0x7f2e2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.245+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5921+0+0 (secure 0 0 0) 0x7f2e3808bee0 con 0x7f2e4015cfa0 2024-04-13T03:19:54.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.246+0000 7f2e3f7fe640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 0x7f2e2c06edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2e300016e0 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:54.252 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.250+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e38010070 con 0x7f2e4015cfa0 2024-04-13T03:19:54.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:54.425+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e400b7e20 con 0x7f2e4015cfa0 2024-04-13T03:19:55.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.122+0000 7f2e27fff640 1 -- 172.21.15.106:0/5621095 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) ==== 169+0+0 (secure 0 0 0) 0x7f2e38059be0 con 0x7f2e4015cfa0 2024-04-13T03:19:55.124 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 msgr2=0x7f2e2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 0x7f2e2c06edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2e300016e0 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 msgr2=0x7f2e4015d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009fa0 tx=0x7f2e3800d560 comp rx=0 tx=0).stop 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 shutdown_connections 2024-04-13T03:19:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2e2c06c8f0 0x7f2e2c06edb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2e4015e880 0x7f2e4015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e4015cfa0 0x7f2e4015d430 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 --2- 172.21.15.106:0/5621095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e400d3530 0x7f2e400ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.129+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 >> 172.21.15.106:0/5621095 conn(0x7f2e400b6280 msgr2=0x7f2e400b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.130+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 shutdown_connections 2024-04-13T03:19:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.130+0000 7f2e45a6d640 1 -- 172.21.15.106:0/5621095 wait complete. 2024-04-13T03:19:55.145 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-13T03:19:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 Processor -- start 2024-04-13T03:19:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 -- start start 2024-04-13T03:19:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 0x7f2c500d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c500b3d60 0x7f2c500d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c500cfd20 con 0x7f2c5015ed90 2024-04-13T03:19:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2c50162af0 con 0x7f2c500b3d60 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.250+0000 7f2c56e2d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2c50162dd0 con 0x7f2c5015e9b0 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c553ca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c500b3d60 0x7f2c500d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c4ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2c5015ed90 0x7f2c500d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60884/0 (socket says 172.21.15.106:60884) 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c4ffff640 1 -- 172.21.15.106:0/4112653900 learned_addr learned my addr 172.21.15.106:0/4112653900 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c54bc9640 1 --2- 172.21.15.106:0/4112653900 >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 0x7f2c500d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c553ca640 1 -- 172.21.15.106:0/4112653900 >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 msgr2=0x7f2c500d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c553ca640 1 --2- 172.21.15.106:0/4112653900 >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 0x7f2c500d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c553ca640 1 -- 172.21.15.106:0/4112653900 >> v1:172.21.15.106:6789/0 conn(0x7f2c5015ed90 legacy=0x7f2c500d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c553ca640 1 -- 172.21.15.106:0/4112653900 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501630b0 con 0x7f2c500b3d60 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c4f7fe640 1 -- 172.21.15.106:0/4112653900 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315374729 0 0) 0x7f2c500cfd20 con 0x7f2c5015ed90 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.251+0000 7f2c54bc9640 1 --2- 172.21.15.106:0/4112653900 >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 0x7f2c500d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c553ca640 1 --2- 172.21.15.106:0/4112653900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c500b3d60 0x7f2c500d0200 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2c5008e4e0 tx=0x7f2c4002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13dd1c37e3903b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c4f7fe640 1 -- 172.21.15.106:0/4112653900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40034070 con 0x7f2c500b3d60 2024-04-13T03:19:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c4f7fe640 1 -- 172.21.15.106:0/4112653900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40030610 con 0x7f2c500b3d60 2024-04-13T03:19:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c4f7fe640 1 -- 172.21.15.106:0/4112653900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400309e0 con 0x7f2c500b3d60 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 -- 172.21.15.106:0/4112653900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c500b3d60 msgr2=0x7f2c500d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/4112653900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c500b3d60 0x7f2c500d0200 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2c5008e4e0 tx=0x7f2c4002b960 comp rx=0 tx=0).stop 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 -- 172.21.15.106:0/4112653900 shutdown_connections 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/4112653900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c500b3d60 0x7f2c500d0200 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/4112653900 >> v2:172.21.15.114:3300/0 conn(0x7f2c5015e9b0 0x7f2c500d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 -- 172.21.15.106:0/4112653900 >> 172.21.15.106:0/4112653900 conn(0x7f2c5008bf10 msgr2=0x7f2c500b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 -- 172.21.15.106:0/4112653900 shutdown_connections 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.252+0000 7f2c56e2d640 1 -- 172.21.15.106:0/4112653900 wait complete. 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.253+0000 7f2c56e2d640 1 Processor -- start 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.253+0000 7f2c56e2d640 1 -- start start 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c500b3d60 0x7f2c500b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c5015e9b0 0x7f2c5015c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50162800 con 0x7f2c501e99e0 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50162680 con 0x7f2c500b3d60 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c56e2d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500d4d00 con 0x7f2c5015e9b0 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c553ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c553ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60814/0 (socket says 172.21.15.106:60814) 2024-04-13T03:19:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.254+0000 7f2c553ca640 1 -- 172.21.15.106:0/1926126849 learned_addr learned my addr 172.21.15.106:0/1926126849 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 -- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c5015e9b0 msgr2=0x7f2c5015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c5015e9b0 0x7f2c5015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 -- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c500b3d60 msgr2=0x7f2c500b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c54bc9640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c500b3d60 0x7f2c500b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c500b3d60 0x7f2c500b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 -- 172.21.15.106:0/1926126849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c500ccd50 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c553ca640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009860 tx=0x7f2c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40046020 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40039990 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c500cd030 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40034040 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.255+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c500cd5f0 con 0x7f2c501e99e0 2024-04-13T03:19:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.256+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c50068a10 con 0x7f2c501e99e0 2024-04-13T03:19:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.256+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c40037070 con 0x7f2c501e99e0 2024-04-13T03:19:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.257+0000 7f2c4d7fa640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 0x7f2c3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.257+0000 7f2c54bc9640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 0x7f2c3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.257+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f2c400ad8b0 con 0x7f2c501e99e0 2024-04-13T03:19:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.257+0000 7f2c54bc9640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 0x7f2c3c06edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006dd80 tx=0x7f2c480084f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.262+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c40050020 con 0x7f2c501e99e0 2024-04-13T03:19:55.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.436+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2c500d3730 con 0x7f2c501e99e0 2024-04-13T03:19:55.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.441+0000 7f2c4d7fa640 1 -- 172.21.15.106:0/1926126849 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f2c40037360 con 0x7f2c501e99e0 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 msgr2=0x7f2c3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 0x7f2c3c06edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006dd80 tx=0x7f2c480084f0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 msgr2=0x7f2c500b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009860 tx=0x7f2c400046c0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 shutdown_connections 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f2c3c06c8f0 0x7f2c3c06edb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f2c5015e9b0 0x7f2c5015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c501e99e0 0x7f2c500b65c0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 --2- 172.21.15.106:0/1926126849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c500b3d60 0x7f2c500b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 >> 172.21.15.106:0/1926126849 conn(0x7f2c5008bf10 msgr2=0x7f2c500b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 shutdown_connections 2024-04-13T03:19:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.443+0000 7f2c56e2d640 1 -- 172.21.15.106:0/1926126849 wait complete. 2024-04-13T03:19:55.455 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-13T03:19:55.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.553+0000 7fb4f360f640 1 Processor -- start 2024-04-13T03:19:55.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 -- start start 2024-04-13T03:19:55.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4ec0d1c00 0x7fb4ec0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4ec1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4ec0d3530 con 0x7fb4ec1622d0 2024-04-13T03:19:55.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb4ec1626b0 con 0x7fb4ec0d1c00 2024-04-13T03:19:55.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.554+0000 7fb4f360f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb4ec0d0b60 con 0x7fb4ec0b8080 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb4ec1622d0 0x7fb4ec0b7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60896/0 (socket says 172.21.15.106:60896) 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 -- 172.21.15.106:0/3493290467 learned_addr learned my addr 172.21.15.106:0/3493290467 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368655162 0 0) 0x7fb4ec0d3530 con 0x7fb4ec1622d0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/3493290467 >> v2:172.21.15.114:3300/0 conn(0x7fb4ec0d1c00 0x7fb4ec0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f1bac640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4ec1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4d8003520 con 0x7fb4ec1622d0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3824517861 0 0) 0x7fb4d8003520 con 0x7fb4ec1622d0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 msgr2=0x7fb4ec1688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4ec1688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 >> v2:172.21.15.114:3300/0 conn(0x7fb4ec0d1c00 msgr2=0x7fb4ec0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 --2- 172.21.15.106:0/3493290467 >> v2:172.21.15.114:3300/0 conn(0x7fb4ec0d1c00 0x7fb4ec0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec0d0e40 con 0x7fb4ec1622d0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7fb4dc001670 con 0x7fb4ec1622d0 2024-04-13T03:19:55.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 >> v1:172.21.15.106:6789/0 conn(0x7fb4ec1622d0 legacy=0x7fb4ec0b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4d8004d40 0x7fb4d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0d0b60 con 0x7fb4d80039b0 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d800b9c0 con 0x7fb4d8004d40 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d800bb40 con 0x7fb4ec0b8080 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f1bac640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4d8004d40 0x7fb4d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 -- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 msgr2=0x7fb4d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec0b8080 0x7fb4d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 -- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4d8004d40 msgr2=0x7fb4d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4d8004d40 0x7fb4d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.555+0000 7fb4f13ab640 1 -- 172.21.15.106:0/3493290467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d8003520 con 0x7fb4d80039b0 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.556+0000 7fb4f1bac640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4d8004d40 0x7fb4d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.556+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4d8003dd0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc007fd0 tx=0x7fb4d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f410a6691207af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.556+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc007d50 con 0x7fb4d80039b0 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.556+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc030070 con 0x7fb4d80039b0 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.556+0000 7fb4e3fff640 1 -- 172.21.15.106:0/3493290467 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc02cd40 con 0x7fb4d80039b0 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 -- 172.21.15.106:0/3493290467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 msgr2=0x7fb4d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 --2- 172.21.15.106:0/3493290467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4d8003dd0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc007fd0 tx=0x7fb4d8005830 comp rx=0 tx=0).stop 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f1bac640 1 -- 172.21.15.106:0/3493290467 reap_dead start 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 -- 172.21.15.106:0/3493290467 shutdown_connections 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 -- 172.21.15.106:0/3493290467 >> 172.21.15.106:0/3493290467 conn(0x7fb4ec0d25d0 msgr2=0x7fb4ec0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 -- 172.21.15.106:0/3493290467 shutdown_connections 2024-04-13T03:19:55.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.557+0000 7fb4f360f640 1 -- 172.21.15.106:0/3493290467 wait complete. 2024-04-13T03:19:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.558+0000 7fb4f360f640 1 Processor -- start 2024-04-13T03:19:55.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.558+0000 7fb4f360f640 1 -- start start 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 0x7fb4ec15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 0x7fb4ec15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0d16e0 con 0x7fb4d80039b0 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0d1400 con 0x7fb4ec149a50 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f360f640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb4ec0cc6f0 con 0x7fb4ec15c3d0 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60818/0 (socket says 172.21.15.106:60818) 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 -- 172.21.15.106:0/1704093803 learned_addr learned my addr 172.21.15.106:0/1704093803 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f1bac640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 0x7fb4ec15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 0x7fb4ec15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 -- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 msgr2=0x7fb4ec15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 0x7fb4ec15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 -- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 msgr2=0x7fb4ec15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 0x7fb4ec15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 -- 172.21.15.106:0/1704093803 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec149ed0 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f1bac640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 0x7fb4ec15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 0x7fb4ec15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.559+0000 7fb4f13ab640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02de10 tx=0x7fb4ec073d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.560+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc045070 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.560+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc004bf0 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.560+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec0b30e0 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.560+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc007d50 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.560+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec0b36a0 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.561+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ec068a10 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.561+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4dc007070 con 0x7fb4d80039b0 2024-04-13T03:19:55.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.562+0000 7fb4e1ffb640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 0x7fb4d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:55.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.562+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fb4dc0acb40 con 0x7fb4d80039b0 2024-04-13T03:19:55.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.562+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 0x7fb4d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:55.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.562+0000 7fb4f0baa640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 0x7fb4d006edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec15b770 tx=0x7fb4e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:55.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.566+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4dc07b1a0 con 0x7fb4d80039b0 2024-04-13T03:19:55.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:55.805+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fb4ec15e090 con 0x7fb4d80039b0 2024-04-13T03:19:56.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.125+0000 7fb4e1ffb640 1 -- 172.21.15.106:0/1704093803 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v500) ==== 144+0+0 (secure 0 0 0) 0x7fb4dc07aaa0 con 0x7fb4d80039b0 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 msgr2=0x7fb4d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 0x7fb4d006edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec15b770 tx=0x7fb4e8008040 comp rx=0 tx=0).stop 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 msgr2=0x7fb4ec0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02de10 tx=0x7fb4ec073d00 comp rx=0 tx=0).stop 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 shutdown_connections 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb4d006c8f0 0x7fb4d006edb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:56.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb4ec15c3d0 0x7fb4ec15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:56.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4ec149a50 0x7fb4ec15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:56.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 --2- 172.21.15.106:0/1704093803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4d80039b0 0x7fb4ec0b2ad0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:56.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 >> 172.21.15.106:0/1704093803 conn(0x7fb4ec0d25d0 msgr2=0x7fb4ec160ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:56.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 shutdown_connections 2024-04-13T03:19:56.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:56.127+0000 7fb4f360f640 1 -- 172.21.15.106:0/1704093803 wait complete. 2024-04-13T03:19:56.139 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-13T03:19:56.140 INFO:tasks.cephfs_test_runner: 2024-04-13T03:19:56.140 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-13T03:19:56.140 INFO:tasks.cephfs_test_runner:Ran 7 tests in 583.908s 2024-04-13T03:19:56.140 INFO:tasks.cephfs_test_runner: 2024-04-13T03:19:56.140 INFO:tasks.cephfs_test_runner:OK 2024-04-13T03:19:56.141 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-13T03:19:56.151 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-13T03:19:56.177 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-13T03:19:56.178 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:56.178 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:56.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:56.194 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-13T03:19:56.194 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:56.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=257 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-13T03:19:56.249 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:56.249 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:56.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:56.268 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-13T03:19:56.268 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T03:19:56.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-13T03:19:56.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.327 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:56.341 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-13T03:19:56.342 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T03:19:56.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-13T03:19:56.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.409 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T03:19:56.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-13T03:19:56.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.572 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-13T03:19:56.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.587 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-13T03:19:56.658 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2024-04-13T03:19:56.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:19:56.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.660 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:19:56.678 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-13T03:19:56.679 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-13T03:19:56.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.679 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-13T03:19:56.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.752 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-13T03:19:56.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:19:56.764 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:19:56.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-04-13T03:19:56.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.915 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-04-13T03:19:56.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T03:19:56.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-04-13T03:19:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2024-04-13T03:19:57.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:19:57.004 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-13T03:19:57.013 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-13T03:19:57.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:19:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.114+0000 7f975e036640 1 Processor -- start 2024-04-13T03:19:57.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.114+0000 7f975e036640 1 -- start start 2024-04-13T03:19:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f975e036640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f975e036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f975e036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97580d0890 con 0x7f97580ce580 2024-04-13T03:19:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f975e036640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f97580d1be0 con 0x7f97580b5400 2024-04-13T03:19:57.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f975e036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f97580d0b40 con 0x7f975815e220 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f9757fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f9756ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9756ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:34484/0 (socket says 172.21.15.106:34484) 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f9757fff640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:34782/0 (socket says 172.21.15.106:34782) 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.115+0000 7f97577fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f97580ce580 0x7f97580d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46308/0 (socket says 172.21.15.106:46308) 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9756ffd640 1 -- 172.21.15.106:0/4221011941 learned_addr learned my addr 172.21.15.106:0/4221011941 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9757fff640 1 -- 172.21.15.106:0/4221011941 >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 msgr2=0x7f97580b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9757fff640 1 --2- 172.21.15.106:0/4221011941 >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9757fff640 1 -- 172.21.15.106:0/4221011941 >> v1:172.21.15.114:6789/0 conn(0x7f97580ce580 legacy=0x7f97580d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9757fff640 1 -- 172.21.15.106:0/4221011941 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580d0e20 con 0x7f975815e220 2024-04-13T03:19:57.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.116+0000 7f9756ffd640 1 --2- 172.21.15.106:0/4221011941 >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:57.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f9757fff640 1 --2- 172.21.15.106:0/4221011941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9750009d90 tx=0x7f975002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7b876f7004ef54 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f97567fc640 1 -- 172.21.15.106:0/4221011941 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750039070 con 0x7f975815e220 2024-04-13T03:19:57.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f97567fc640 1 -- 172.21.15.106:0/4221011941 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975002cc50 con 0x7f975815e220 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f97567fc640 1 -- 172.21.15.106:0/4221011941 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750034440 con 0x7f975815e220 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 -- 172.21.15.106:0/4221011941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 msgr2=0x7f975815e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 --2- 172.21.15.106:0/4221011941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9750009d90 tx=0x7f975002bea0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 -- 172.21.15.106:0/4221011941 shutdown_connections 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 --2- 172.21.15.106:0/4221011941 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f975815e670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 --2- 172.21.15.106:0/4221011941 >> v2:172.21.15.114:3300/0 conn(0x7f97580b5400 0x7f97580b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 -- 172.21.15.106:0/4221011941 >> 172.21.15.106:0/4221011941 conn(0x7f97580d25b0 msgr2=0x7f97580d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 -- 172.21.15.106:0/4221011941 shutdown_connections 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.117+0000 7f975e036640 1 -- 172.21.15.106:0/4221011941 wait complete. 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 Processor -- start 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 -- start start 2024-04-13T03:19:57.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97580ce580 0x7f97580c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f97580c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97580d16c0 con 0x7f97580b5400 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97580d13e0 con 0x7f97580ce580 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.119+0000 7f975e036640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f97580ce960 con 0x7f975815e220 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60828/0 (socket says 172.21.15.106:60828) 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 -- 172.21.15.106:0/2865404081 learned_addr learned my addr 172.21.15.106:0/2865404081 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f9757fff640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f97580c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f9756ffd640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97580ce580 0x7f97580c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 -- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 msgr2=0x7f97580c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f97580c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 -- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97580ce580 msgr2=0x7f97580c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97580ce580 0x7f97580c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 -- 172.21.15.106:0/2865404081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580c61b0 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f97577fe640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f974800c400 tx=0x7f974800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480074e0 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97480047e0 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748004c10 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97580c6490 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.120+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97580c6a50 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.121+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9758074750 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.122+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f974801a020 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.122+0000 7f973ffff640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.122+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f974808b180 con 0x7f97580b5400 2024-04-13T03:19:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.122+0000 7f9756ffd640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.123+0000 7f9756ffd640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 0x7f974406edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f974c009630 tx=0x7f974c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.126+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9748059020 con 0x7f97580b5400 2024-04-13T03:19:57.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.299+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97580ca140 con 0x7f974406c8f0 2024-04-13T03:19:57.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.300+0000 7f973ffff640 1 -- 172.21.15.106:0/2865404081 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13412 (secure 0 0 0) 0x7f97580ca140 con 0x7f974406c8f0 2024-04-13T03:19:57.301 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:57.301 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 msgr2=0x7f974406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 0x7f974406edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f974c009630 tx=0x7f974c009240 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 msgr2=0x7f9758155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f974800c400 tx=0x7f974800c8c0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 shutdown_connections 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f975815e220 0x7f97580c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97580ce580 0x7f97580c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 --2- 172.21.15.106:0/2865404081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97580b5400 0x7f9758155370 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 >> 172.21.15.106:0/2865404081 conn(0x7f97580d25b0 msgr2=0x7f97580ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 shutdown_connections 2024-04-13T03:19:57.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.302+0000 7f975e036640 1 -- 172.21.15.106:0/2865404081 wait complete. 2024-04-13T03:19:57.313 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-13T03:19:56.503412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279032,"kb_used_data":29612,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695688,"statfs":{"total":773094113280,"available":772808384512,"internally_reserved":0,"allocated":30322688,"data_stored":23719213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-172,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001013"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-13T03:19:38.129853+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:38.129853+0000","last_peered":"2024-04-13T03:19:38.129853+0000","last_clean":"2024-04-13T03:19:38.129853+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:38.129853+0000","last_undegraded":"2024-04-13T03:19:38.129853+0000","last_fullsized":"2024-04-13T03:19:38.129853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4912880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1548288,"data_stored":721279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1536000,"data_stored":718421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5734400,"data_stored":4912747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:19:57.314 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:19:57.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.413+0000 7fb870636640 1 Processor -- start 2024-04-13T03:19:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.413+0000 7fb870636640 1 -- start start 2024-04-13T03:19:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb870636640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 0x7fb868070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb870636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb870636640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86806fd00 con 0x7fb86815ce80 2024-04-13T03:19:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb870636640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb86806ffe0 con 0x7fb86815caa0 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb870636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb8680702c0 con 0x7fb8680b7f70 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86e3d2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 0x7fb868070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86ebd3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86e3d2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 0x7fb868070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:34498/0 (socket says 172.21.15.106:34498) 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86ebd3640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:34798/0 (socket says 172.21.15.106:34798) 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86e3d2640 1 -- 172.21.15.106:0/682633155 learned_addr learned my addr 172.21.15.106:0/682633155 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.414+0000 7fb86dbd1640 1 --1- 172.21.15.106:0/682633155 >> v1:172.21.15.114:6789/0 conn(0x7fb86815ce80 0x7fb8680b5780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46318/0 (socket says 172.21.15.106:46318) 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86d3d0640 1 -- 172.21.15.106:0/682633155 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398713618 0 0) 0x7fb86806fd00 con 0x7fb86815ce80 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86ebd3640 1 -- 172.21.15.106:0/682633155 >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 msgr2=0x7fb868070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86ebd3640 1 --2- 172.21.15.106:0/682633155 >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 0x7fb868070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86ebd3640 1 -- 172.21.15.106:0/682633155 >> v1:172.21.15.114:6789/0 conn(0x7fb86815ce80 legacy=0x7fb8680b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86ebd3640 1 -- 172.21.15.106:0/682633155 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8680705a0 con 0x7fb8680b7f70 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86ebd3640 1 --2- 172.21.15.106:0/682633155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb860009d90 tx=0x7fb86002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf942e6974cade5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86d3d0640 1 -- 172.21.15.106:0/682633155 <== mon.2 v2:172.21.15.114:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860039070 con 0x7fb8680b7f70 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86d3d0640 1 -- 172.21.15.106:0/682633155 <== mon.2 v2:172.21.15.114:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86002cc50 con 0x7fb8680b7f70 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.415+0000 7fb86d3d0640 1 -- 172.21.15.106:0/682633155 <== mon.2 v2:172.21.15.114:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860034440 con 0x7fb8680b7f70 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 -- 172.21.15.106:0/682633155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 msgr2=0x7fb8680b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 --2- 172.21.15.106:0/682633155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb860009d90 tx=0x7fb86002bea0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 -- 172.21.15.106:0/682633155 shutdown_connections 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 --2- 172.21.15.106:0/682633155 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb8680b7f70 0x7fb8680b5e90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 --2- 172.21.15.106:0/682633155 >> v2:172.21.15.114:3300/0 conn(0x7fb86815caa0 0x7fb868070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 -- 172.21.15.106:0/682633155 >> 172.21.15.106:0/682633155 conn(0x7fb8680d25d0 msgr2=0x7fb8680d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 -- 172.21.15.106:0/682633155 shutdown_connections 2024-04-13T03:19:57.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.416+0000 7fb870636640 1 -- 172.21.15.106:0/682633155 wait complete. 2024-04-13T03:19:57.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.417+0000 7fb870636640 1 Processor -- start 2024-04-13T03:19:57.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.417+0000 7fb870636640 1 -- start start 2024-04-13T03:19:57.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 0x7fb86814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 0x7fb8680b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680712e0 con 0x7fb8681f5b20 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb86806f990 con 0x7fb8680b7f70 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb870636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb868071460 con 0x7fb86815caa0 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb86ebd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb86ebd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60830/0 (socket says 172.21.15.106:60830) 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb86ebd3640 1 -- 172.21.15.106:0/2629248327 learned_addr learned my addr 172.21.15.106:0/2629248327 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb86e3d2640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 0x7fb86814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.418+0000 7fb86dbd1640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 0x7fb8680b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 -- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 msgr2=0x7fb8680b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 0x7fb8680b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 -- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 msgr2=0x7fb86814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 0x7fb86814b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 -- 172.21.15.106:0/2629248327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8681f5fe0 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86dbd1640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 0x7fb8680b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86e3d2640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 0x7fb86814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb86ebd3640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb860009d90 tx=0x7fb860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860044020 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86003a040 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8681f62c0 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860039050 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.419+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681f6880 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.420+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb868068a10 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.420+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb860003990 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.421+0000 7fb85effd640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.421+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fb8600ade20 con 0x7fb8681f5b20 2024-04-13T03:19:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.421+0000 7fb86e3d2640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.421+0000 7fb86e3d2640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb864009c20 tx=0x7fb864008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.425+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb86004e060 con 0x7fb8681f5b20 2024-04-13T03:19:57.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.597+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8680cbb70 con 0x7fb84c06c8f0 2024-04-13T03:19:57.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.598+0000 7fb85effd640 1 -- 172.21.15.106:0/2629248327 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13412 (secure 0 0 0) 0x7fb8680cbb70 con 0x7fb84c06c8f0 2024-04-13T03:19:57.600 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:57.600 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:19:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 msgr2=0x7fb84c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb864009c20 tx=0x7fb864008040 comp rx=0 tx=0).stop 2024-04-13T03:19:57.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 msgr2=0x7fb8680b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb860009d90 tx=0x7fb860004a40 comp rx=0 tx=0).stop 2024-04-13T03:19:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 shutdown_connections 2024-04-13T03:19:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb86815caa0 0x7fb8680b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8681f5b20 0x7fb8680b75b0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 --2- 172.21.15.106:0/2629248327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8680b7f70 0x7fb86814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 >> 172.21.15.106:0/2629248327 conn(0x7fb8680d25d0 msgr2=0x7fb86815bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.600+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 shutdown_connections 2024-04-13T03:19:57.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.601+0000 7fb870636640 1 -- 172.21.15.106:0/2629248327 wait complete. 2024-04-13T03:19:57.612 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-13T03:19:56.503412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279032,"kb_used_data":29612,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695688,"statfs":{"total":773094113280,"available":772808384512,"internally_reserved":0,"allocated":30322688,"data_stored":23719213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-172,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001013"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-13T03:19:38.129853+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:38.129853+0000","last_peered":"2024-04-13T03:19:38.129853+0000","last_clean":"2024-04-13T03:19:38.129853+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:38.129853+0000","last_undegraded":"2024-04-13T03:19:38.129853+0000","last_fullsized":"2024-04-13T03:19:38.129853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4912880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1548288,"data_stored":721279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1536000,"data_stored":718421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5734400,"data_stored":4912747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:19:57.612 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-13T03:19:57.613 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:19:57.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.713+0000 7f71105f0640 1 Processor -- start 2024-04-13T03:19:57.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.713+0000 7f71105f0640 1 -- start start 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f71105f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f710815ebf0 0x7f710806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f71105f0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f71080ce1f0 0x7f710806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f71105f0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71080ce6f0 con 0x7f71080b5bf0 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f71105f0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f710815fa70 con 0x7f71080ce1f0 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f71105f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f710815f420 con 0x7f710815ebf0 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710db8b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f71080ce1f0 0x7f710806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710eb8d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f71080b5bf0 0x7f710806e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60906/0 (socket says 172.21.15.106:60906) 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710eb8d640 1 -- 172.21.15.106:0/3890724987 learned_addr learned my addr 172.21.15.106:0/3890724987 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710db8b640 1 -- 172.21.15.106:0/3890724987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f710815ebf0 msgr2=0x7f710806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710db8b640 1 --2- 172.21.15.106:0/3890724987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f710815ebf0 0x7f710806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710db8b640 1 -- 172.21.15.106:0/3890724987 >> v1:172.21.15.106:6789/0 conn(0x7f71080b5bf0 legacy=0x7f710806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710d38a640 1 -- 172.21.15.106:0/3890724987 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199031544 0 0) 0x7f71080ce6f0 con 0x7f71080b5bf0 2024-04-13T03:19:57.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.714+0000 7f710db8b640 1 -- 172.21.15.106:0/3890724987 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710815f700 con 0x7f71080ce1f0 2024-04-13T03:19:57.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f710db8b640 1 --2- 172.21.15.106:0/3890724987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080ce1f0 0x7f710806fcc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6e78aca9b2ed0b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f710d38a640 1 -- 172.21.15.106:0/3890724987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80315a0 con 0x7f71080ce1f0 2024-04-13T03:19:57.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f710d38a640 1 -- 172.21.15.106:0/3890724987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f8037070 con 0x7f71080ce1f0 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f710d38a640 1 -- 172.21.15.106:0/3890724987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8031960 con 0x7f71080ce1f0 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f71105f0640 1 -- 172.21.15.106:0/3890724987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080ce1f0 msgr2=0x7f710806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f71105f0640 1 --2- 172.21.15.106:0/3890724987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080ce1f0 0x7f710806fcc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f70f80097a0 tx=0x7f70f802b840 comp rx=0 tx=0).stop 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f71105f0640 1 -- 172.21.15.106:0/3890724987 shutdown_connections 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f71105f0640 1 --2- 172.21.15.106:0/3890724987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080ce1f0 0x7f710806fcc0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.715+0000 7f71105f0640 1 --2- 172.21.15.106:0/3890724987 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f710815ebf0 0x7f710806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.716+0000 7f71105f0640 1 -- 172.21.15.106:0/3890724987 >> 172.21.15.106:0/3890724987 conn(0x7f71080cd6d0 msgr2=0x7f71080cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.716+0000 7f71105f0640 1 -- 172.21.15.106:0/3890724987 shutdown_connections 2024-04-13T03:19:57.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.716+0000 7f71105f0640 1 -- 172.21.15.106:0/3890724987 wait complete. 2024-04-13T03:19:57.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 Processor -- start 2024-04-13T03:19:57.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 -- start start 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71080ce1f0 0x7f71081f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f710815ebf0 0x7f710815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71080b0650 con 0x7f71080b5bf0 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71080b04d0 con 0x7f710815ebf0 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f71105f0640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f710815efd0 con 0x7f71080ce1f0 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f710e38c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f710e38c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60850/0 (socket says 172.21.15.106:60850) 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.717+0000 7f710e38c640 1 -- 172.21.15.106:0/3701014251 learned_addr learned my addr 172.21.15.106:0/3701014251 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 -- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71080ce1f0 msgr2=0x7f71081f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71080ce1f0 0x7f71081f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 -- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f710815ebf0 msgr2=0x7f710815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f710815ebf0 0x7f710815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 -- 172.21.15.106:0/3701014251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081f5790 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f710e38c640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f710400b4a0 tx=0x7f710400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104014070 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7104010610 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081f4b10 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104010a00 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.718+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081f5090 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.719+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7108068a10 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.719+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7104017070 con 0x7f71080b5bf0 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.720+0000 7f70feffd640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 0x7f70ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:57.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.720+0000 7f710db8b640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 0x7f70ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:57.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.720+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f710408b520 con 0x7f71080b5bf0 2024-04-13T03:19:57.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.720+0000 7f710db8b640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 0x7f70ec06edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f70f8036000 tx=0x7f70f8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:57.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.724+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7104059a10 con 0x7f71080b5bf0 2024-04-13T03:19:57.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.895+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7108068140 con 0x7f70ec06c8f0 2024-04-13T03:19:57.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.895+0000 7f70feffd640 1 -- 172.21.15.106:0/3701014251 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13412 (secure 0 0 0) 0x7f7108068140 con 0x7f70ec06c8f0 2024-04-13T03:19:57.896 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:57.897 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:19:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 msgr2=0x7f70ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 0x7f70ec06edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f70f8036000 tx=0x7f70f8030000 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 msgr2=0x7f7108148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f710400b4a0 tx=0x7f710400b960 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 shutdown_connections 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70ec06c8f0 0x7f70ec06edb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f71080ce1f0 0x7f71081f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f710815ebf0 0x7f710815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 --2- 172.21.15.106:0/3701014251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71080b5bf0 0x7f7108148850 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 >> 172.21.15.106:0/3701014251 conn(0x7f71080cd6d0 msgr2=0x7f71081592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 shutdown_connections 2024-04-13T03:19:57.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:57.897+0000 7f71105f0640 1 -- 172.21.15.106:0/3701014251 wait complete. 2024-04-13T03:19:57.909 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-13T03:19:56.503412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279032,"kb_used_data":29612,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695688,"statfs":{"total":773094113280,"available":772808384512,"internally_reserved":0,"allocated":30322688,"data_stored":23719213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-172,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001013"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-13T03:19:38.129853+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:38.129853+0000","last_peered":"2024-04-13T03:19:38.129853+0000","last_clean":"2024-04-13T03:19:38.129853+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:38.129853+0000","last_undegraded":"2024-04-13T03:19:38.129853+0000","last_fullsized":"2024-04-13T03:19:38.129853+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4912880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1548288,"data_stored":721279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1536000,"data_stored":718421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5734400,"data_stored":4912747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:19:57.910 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T03:19:58.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 Processor -- start 2024-04-13T03:19:58.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 -- start start 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f520815ed90 0x7f52080d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 0x7f52080d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080cfd20 con 0x7f520815e9b0 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5208162af0 con 0x7f520815ed90 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.010+0000 7f520ccde640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5208162dd0 con 0x7f52080b3d60 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f5206d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 0x7f52080d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f520815ed90 0x7f52080d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f5206d9d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 0x7f52080d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3301/0 says I am v2:172.21.15.106:34844/0 (socket says 172.21.15.106:34844) 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f520815ed90 0x7f52080d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.106:34530/0 (socket says 172.21.15.106:34530) 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f5206d9d640 1 -- 172.21.15.106:0/3167269984 learned_addr learned my addr 172.21.15.106:0/3167269984 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f520659c640 1 --1- 172.21.15.106:0/3167269984 >> v1:172.21.15.114:6789/0 conn(0x7f520815e9b0 0x7f52080d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.106:46326/0 (socket says 172.21.15.106:46326) 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 -- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 msgr2=0x7f52080d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 --2- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 0x7f52080d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 -- 172.21.15.106:0/3167269984 >> v1:172.21.15.114:6789/0 conn(0x7f520815e9b0 legacy=0x7f52080d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.011+0000 7f51fffff640 1 -- 172.21.15.106:0/3167269984 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081630b0 con 0x7f520815ed90 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.012+0000 7f51fffff640 1 --2- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815ed90 0x7f52080d2300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009790 tx=0x7f51f002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449ac6ac0868beb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.012+0000 7f5205d9b640 1 -- 172.21.15.106:0/3167269984 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0035020 con 0x7f520815ed90 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.012+0000 7f5205d9b640 1 -- 172.21.15.106:0/3167269984 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0031890 con 0x7f520815ed90 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.012+0000 7f5205d9b640 1 -- 172.21.15.106:0/3167269984 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0031c60 con 0x7f520815ed90 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 -- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815ed90 msgr2=0x7f52080d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 --2- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815ed90 0x7f52080d2300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009790 tx=0x7f51f002b930 comp rx=0 tx=0).stop 2024-04-13T03:19:58.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 -- 172.21.15.106:0/3167269984 shutdown_connections 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 --2- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f52080b3d60 0x7f52080d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 --2- 172.21.15.106:0/3167269984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815ed90 0x7f52080d2300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 -- 172.21.15.106:0/3167269984 >> 172.21.15.106:0/3167269984 conn(0x7f520808bf10 msgr2=0x7f52080b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 -- 172.21.15.106:0/3167269984 shutdown_connections 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.013+0000 7f520ccde640 1 -- 172.21.15.106:0/3167269984 wait complete. 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.014+0000 7f520ccde640 1 Processor -- start 2024-04-13T03:19:58.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.014+0000 7f520ccde640 1 -- start start 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 0x7f52080cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 0x7f52080ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5208162800 con 0x7f52080b3d60 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5208162680 con 0x7f520815fe30 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520ccde640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f52080d4d00 con 0x7f520815e9b0 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60854/0 (socket says 172.21.15.106:60854) 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 -- 172.21.15.106:0/3065144636 learned_addr learned my addr 172.21.15.106:0/3065144636 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f51fffff640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 0x7f52080cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f5206d9d640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 0x7f52080ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 -- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 msgr2=0x7f52080cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 0x7f52080cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 -- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 msgr2=0x7f52080ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 0x7f52080ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 -- 172.21.15.106:0/3065144636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081606b0 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f51fffff640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 0x7f52080cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f5206d9d640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 0x7f52080ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f520659c640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c430 tx=0x7f51f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.015+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80074e0 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.016+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8007680 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.016+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8004a00 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.016+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5208160990 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.016+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081602d0 con 0x7f52080b3d60 2024-04-13T03:19:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.017+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208160eb0 con 0x7f52080b3d60 2024-04-13T03:19:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.017+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51f801a020 con 0x7f52080b3d60 2024-04-13T03:19:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.017+0000 7f51feffd640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.017+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f51f808b2c0 con 0x7f52080b3d60 2024-04-13T03:19:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.017+0000 7f51fffff640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.018+0000 7f51fffff640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 0x7f51e806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.022+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51f80597b0 con 0x7f52080b3d60 2024-04-13T03:19:58.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.197+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5208068a10 con 0x7f52080b3d60 2024-04-13T03:19:58.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.198+0000 7f51feffd640 1 -- 172.21.15.106:0/3065144636 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+13603 (secure 0 0 0) 0x7f51f808b0e0 con 0x7f52080b3d60 2024-04-13T03:19:58.198 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:19:58.199 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":126,"fsid":"d7ff0368-2781-4238-b477-50485b08a223","created":"2024-04-13T03:09:00.163499+0000","modified":"2024-04-13T03:19:55.120076+0000","last_up_change":"2024-04-13T03:09:11.402783+0000","last_in_change":"2024-04-13T03:09:02.422340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T03:09:12.377322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f0d52df-7c5b-4676-8ea0-28475c30a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6809","nonce":3573709684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6812","nonce":3573709684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6820","nonce":3573709684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3573709684},{"type":"v1","addr":"172.21.15.106:6816","nonce":3573709684}]},"public_addr":"172.21.15.106:6809/3573709684","cluster_addr":"172.21.15.106:6812/3573709684","heartbeat_back_addr":"172.21.15.106:6820/3573709684","heartbeat_front_addr":"172.21.15.106:6816/3573709684","state":["exists","up"]},{"osd":1,"uuid":"611ca47e-a863-4c90-baef-28b486fd0bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6801","nonce":1701180838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6803","nonce":1701180838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6807","nonce":1701180838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1701180838},{"type":"v1","addr":"172.21.15.106:6805","nonce":1701180838}]},"public_addr":"172.21.15.106:6801/1701180838","cluster_addr":"172.21.15.106:6803/1701180838","heartbeat_back_addr":"172.21.15.106:6807/1701180838","heartbeat_front_addr":"172.21.15.106:6805/1701180838","state":["exists","up"]},{"osd":2,"uuid":"b2cfad75-5479-41fc-b4a5-8f481ce3d6ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6825","nonce":4087145286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6827","nonce":4087145286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6831","nonce":4087145286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4087145286},{"type":"v1","addr":"172.21.15.106:6829","nonce":4087145286}]},"public_addr":"172.21.15.106:6825/4087145286","cluster_addr":"172.21.15.106:6827/4087145286","heartbeat_back_addr":"172.21.15.106:6831/4087145286","heartbeat_front_addr":"172.21.15.106:6829/4087145286","state":["exists","up"]},{"osd":3,"uuid":"687c55b8-bccf-42fe-b20c-85d0fc53095b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6813","nonce":987945743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6815","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6817","nonce":987945743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6823","nonce":987945743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6819","nonce":987945743},{"type":"v1","addr":"172.21.15.106:6821","nonce":987945743}]},"public_addr":"172.21.15.106:6813/987945743","cluster_addr":"172.21.15.106:6817/987945743","heartbeat_back_addr":"172.21.15.106:6823/987945743","heartbeat_front_addr":"172.21.15.106:6821/987945743","state":["exists","up"]},{"osd":4,"uuid":"8daeed25-5121-4623-86b7-f96990e6f838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6801","nonce":1124906481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6803","nonce":1124906481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6807","nonce":1124906481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1124906481},{"type":"v1","addr":"172.21.15.114:6805","nonce":1124906481}]},"public_addr":"172.21.15.114:6801/1124906481","cluster_addr":"172.21.15.114:6803/1124906481","heartbeat_back_addr":"172.21.15.114:6807/1124906481","heartbeat_front_addr":"172.21.15.114:6805/1124906481","state":["exists","up"]},{"osd":5,"uuid":"6590374b-3603-439d-ab79-ce2e0094e7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6809","nonce":1239989021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6811","nonce":1239989021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6815","nonce":1239989021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1239989021},{"type":"v1","addr":"172.21.15.114:6813","nonce":1239989021}]},"public_addr":"172.21.15.114:6809/1239989021","cluster_addr":"172.21.15.114:6811/1239989021","heartbeat_back_addr":"172.21.15.114:6815/1239989021","heartbeat_front_addr":"172.21.15.114:6813/1239989021","state":["exists","up"]},{"osd":6,"uuid":"f882fbdd-eb82-4ae8-bbf1-ca8ea466ae09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6817","nonce":1761123940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6819","nonce":1761123940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6823","nonce":1761123940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1761123940},{"type":"v1","addr":"172.21.15.114:6821","nonce":1761123940}]},"public_addr":"172.21.15.114:6817/1761123940","cluster_addr":"172.21.15.114:6819/1761123940","heartbeat_back_addr":"172.21.15.114:6823/1761123940","heartbeat_front_addr":"172.21.15.114:6821/1761123940","state":["exists","up"]},{"osd":7,"uuid":"d90993dc-97b5-49ab-803c-c0ba0225626f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6825","nonce":300957724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6827","nonce":300957724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6831","nonce":300957724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":300957724},{"type":"v1","addr":"172.21.15.114:6829","nonce":300957724}]},"public_addr":"172.21.15.114:6825/300957724","cluster_addr":"172.21.15.114:6827/300957724","heartbeat_back_addr":"172.21.15.114:6831/300957724","heartbeat_front_addr":"172.21.15.114:6829/300957724","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629429+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.499070+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.605206+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.649099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.612830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.532168+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.629471+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T03:09:07.575539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/856679728":"2024-04-14T03:19:51.945519+0000","172.21.15.106:6837/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6835/856679728":"2024-04-14T03:19:51.945519+0000","172.21.15.106:6835/3985285527":"2024-04-14T03:19:38.053659+0000","172.21.15.106:6837/1427655446":"2024-04-14T03:19:27.537455+0000","172.21.15.106:6835/1427655446":"2024-04-14T03:19:27.537455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 msgr2=0x7f51e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 0x7f51e806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f003c040 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 msgr2=0x7f520815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c430 tx=0x7f51f800c8f0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 shutdown_connections 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f520815e9b0 0x7f52080cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520815fe30 0x7f52080ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 --2- 172.21.15.106:0/3065144636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52080b3d60 0x7f520815f8f0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 >> 172.21.15.106:0/3065144636 conn(0x7f520808bf10 msgr2=0x7f52080b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 shutdown_connections 2024-04-13T03:19:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:58.200+0000 7f520ccde640 1 -- 172.21.15.106:0/3065144636 wait complete. 2024-04-13T03:19:59.212 INFO:tasks.ceph:Scrubbing osd.0 2024-04-13T03:19:59.212 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:19:59.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.330+0000 7f73eef1e640 1 Processor -- start 2024-04-13T03:19:59.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.330+0000 7f73eef1e640 1 -- start start 2024-04-13T03:19:59.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eef1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73e8162470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eef1e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f73e80ce100 0x7f73e8070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eef1e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e806db70 con 0x7f73e80b4510 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eef1e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f73e806c380 con 0x7f73e80ce100 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eef1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f73e8162ce0 con 0x7f73e815e890 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73ed4bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f73e80b4510 0x7f73e80b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60912/0 (socket says 172.21.15.106:60912) 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73ed4bb640 1 -- 172.21.15.106:0/1055498651 learned_addr learned my addr 172.21.15.106:0/1055498651 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251941140 0 0) 0x7f73e806db70 con 0x7f73e80b4510 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73eccba640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73e8162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d0003520 con 0x7f73e80b4510 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.331+0000 7f73e7fff640 1 --2- 172.21.15.106:0/1055498651 >> v2:172.21.15.114:3300/0 conn(0x7f73e80ce100 0x7f73e8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3828789997 0 0) 0x7f73d0003520 con 0x7f73e80b4510 2024-04-13T03:19:59.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 msgr2=0x7f73e8162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73e8162470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 >> v2:172.21.15.114:3300/0 conn(0x7f73e80ce100 msgr2=0x7f73e8070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 --2- 172.21.15.106:0/1055498651 >> v2:172.21.15.114:3300/0 conn(0x7f73e80ce100 0x7f73e8070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e8162fc0 con 0x7f73e80b4510 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f73e00019f0 con 0x7f73e80b4510 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 >> v1:172.21.15.106:6789/0 conn(0x7f73e80b4510 legacy=0x7f73e80b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73d0004d40 0x7f73d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73e806c380 con 0x7f73d00039b0 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8162ce0 con 0x7f73d0004d40 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f73d000b9c0 con 0x7f73e815e890 2024-04-13T03:19:59.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73ed4bb640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73d0004d40 0x7f73d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73e7fff640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 -- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 msgr2=0x7f73d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73e815e890 0x7f73d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 -- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73d0004d40 msgr2=0x7f73d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73d0004d40 0x7f73d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.332+0000 7f73eccba640 1 -- 172.21.15.106:0/1055498651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d0003520 con 0x7f73d00039b0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.333+0000 7f73eccba640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009ac0 tx=0x7f73d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d421891c7af3676 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.333+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8039070 con 0x7f73d00039b0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.333+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d802c6b0 con 0x7f73d00039b0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.333+0000 7f73e77fe640 1 -- 172.21.15.106:0/1055498651 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8009690 con 0x7f73d00039b0 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 -- 172.21.15.106:0/1055498651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73d00039b0 msgr2=0x7f73d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 --2- 172.21.15.106:0/1055498651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009ac0 tx=0x7f73d0005830 comp rx=0 tx=0).stop 2024-04-13T03:19:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73ed4bb640 1 -- 172.21.15.106:0/1055498651 reap_dead start 2024-04-13T03:19:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 -- 172.21.15.106:0/1055498651 shutdown_connections 2024-04-13T03:19:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 -- 172.21.15.106:0/1055498651 >> 172.21.15.106:0/1055498651 conn(0x7f73e806f290 msgr2=0x7f73e806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 -- 172.21.15.106:0/1055498651 shutdown_connections 2024-04-13T03:19:59.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.334+0000 7f73eef1e640 1 -- 172.21.15.106:0/1055498651 wait complete. 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.335+0000 7f73eef1e640 1 Processor -- start 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.335+0000 7f73eef1e640 1 -- start start 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73d00039b0 0x7f73e8146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 0x7f73e815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73e80cb8e0 0x7f73e80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73e815ec70 con 0x7f73e815ace0 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f73e806c640 con 0x7f73e80cb8e0 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eef1e640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f73e8070c70 con 0x7f73d00039b0 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 0x7f73e815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 0x7f73e815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60880/0 (socket says 172.21.15.106:60880) 2024-04-13T03:19:59.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 -- 172.21.15.106:0/1046364250 learned_addr learned my addr 172.21.15.106:0/1046364250 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73eccba640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73d00039b0 0x7f73e8146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73ed4bb640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73e80cb8e0 0x7f73e80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 -- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73d00039b0 msgr2=0x7f73e8146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f73d00039b0 0x7f73e8146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 -- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73e80cb8e0 msgr2=0x7f73e80cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73e80cb8e0 0x7f73e80cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80cc580 con 0x7f73e815ace0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.336+0000 7f73e7fff640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 0x7f73e815b160 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f73dc011980 tx=0x7f73dc011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.337+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00f9d0 con 0x7f73e815ace0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.337+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc00fb70 con 0x7f73e815ace0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.337+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e80cc860 con 0x7f73e815ace0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.337+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc014440 con 0x7f73e815ace0 2024-04-13T03:19:59.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.337+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e80cce20 con 0x7f73e815ace0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.338+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73e8068a10 con 0x7f73e815ace0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.338+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73dc012040 con 0x7f73e815ace0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73e57fa640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73d406c8f0 0x7f73d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f73dc08bfb0 con 0x7f73e815ace0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73eccba640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73d406c8f0 0x7f73d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73e57fa640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7f73d4072ed0 0x7f73d4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73dc025ec0 con 0x7f73d4072ed0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f73dc08e5f0 con 0x7f73e815ace0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73ed4bb640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7f73d4072ed0 0x7f73d4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73eccba640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73d406c8f0 0x7f73d406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f73d8036000 tx=0x7f73d8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.339+0000 7f73ed4bb640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7f73d4072ed0 0x7f73d4075310 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.340+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== osd.0 v2:172.21.15.106:6808/3573709684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73dc025ec0 con 0x7f73d4072ed0 2024-04-13T03:19:59.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.353+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 --> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f73e8071790 con 0x7f73d4072ed0 2024-04-13T03:19:59.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.353+0000 7f73e57fa640 1 -- 172.21.15.106:0/1046364250 <== osd.0 v2:172.21.15.106:6808/3573709684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f73e8071790 con 0x7f73d4072ed0 2024-04-13T03:19:59.354 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:59.354 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:19:59.354 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.353+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7f73d4072ed0 msgr2=0x7f73d4075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6808/3573709684,v1:172.21.15.106:6809/3573709684] conn(0x7f73d4072ed0 0x7f73d4075310 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73d406c8f0 msgr2=0x7f73d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f73d406c8f0 0x7f73d406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f73d8036000 tx=0x7f73d8030000 comp rx=0 tx=0).stop 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 msgr2=0x7f73e815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 --2- 172.21.15.106:0/1046364250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73e815ace0 0x7f73e815b160 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f73dc011980 tx=0x7f73dc011d80 comp rx=0 tx=0).stop 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73ed4bb640 1 -- 172.21.15.106:0/1046364250 reap_dead start 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 shutdown_connections 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 >> 172.21.15.106:0/1046364250 conn(0x7f73e806f290 msgr2=0x7f73e806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 shutdown_connections 2024-04-13T03:19:59.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.354+0000 7f73eef1e640 1 -- 172.21.15.106:0/1046364250 wait complete. 2024-04-13T03:19:59.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-13T03:19:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.467+0000 7f5da41cc640 1 Processor -- start 2024-04-13T03:19:59.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 -- start start 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 0x7f5d9c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 0x7f5d9c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5d9c15c5d0 0x7f5d9c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9c06d180 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9c0ce180 con 0x7f5d9c1592b0 2024-04-13T03:19:59.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da41cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c06ec90 con 0x7f5d9c06e740 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.468+0000 7f5da2769640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5d9c15c5d0 0x7f5d9c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5d9c15c5d0 0x7f5d9c15f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60888/0 (socket says 172.21.15.106:60888) 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 -- 172.21.15.106:0/2956076478 learned_addr learned my addr 172.21.15.106:0/2956076478 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da1767640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 0x7f5d9c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da1f68640 1 --2- 172.21.15.106:0/2956076478 >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 0x7f5d9c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 -- 172.21.15.106:0/2956076478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 msgr2=0x7f5d9c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 0x7f5d9c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 -- 172.21.15.106:0/2956076478 >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 msgr2=0x7f5d9c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 --2- 172.21.15.106:0/2956076478 >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 0x7f5d9c15bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da2769640 1 -- 172.21.15.106:0/2956076478 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c06ef70 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da1f68640 1 --2- 172.21.15.106:0/2956076478 >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 0x7f5d9c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.469+0000 7f5da1767640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 0x7f5d9c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da2769640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c15c5d0 0x7f5d9c15f140 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0e80 tx=0x7f5d9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce17a6afad3ec45 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da0f66640 1 -- 172.21.15.106:0/2956076478 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9402ca70 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da0f66640 1 -- 172.21.15.106:0/2956076478 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9402cc10 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da0f66640 1 -- 172.21.15.106:0/2956076478 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940343f0 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 -- 172.21.15.106:0/2956076478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c15c5d0 msgr2=0x7f5d9c15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c15c5d0 0x7f5d9c15f140 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0e80 tx=0x7f5d9402bed0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 -- 172.21.15.106:0/2956076478 shutdown_connections 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c15c5d0 0x7f5d9c15f140 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 --2- 172.21.15.106:0/2956076478 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c06e740 0x7f5d9c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 --2- 172.21.15.106:0/2956076478 >> v2:172.21.15.114:3300/0 conn(0x7f5d9c1592b0 0x7f5d9c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 -- 172.21.15.106:0/2956076478 >> 172.21.15.106:0/2956076478 conn(0x7f5d9c0b2600 msgr2=0x7f5d9c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 -- 172.21.15.106:0/2956076478 shutdown_connections 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.470+0000 7f5da41cc640 1 -- 172.21.15.106:0/2956076478 wait complete. 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.471+0000 7f5da41cc640 1 Processor -- start 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.471+0000 7f5da41cc640 1 -- start start 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 0x7f5d9c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 0x7f5d9c140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c15f680 con 0x7f5d9c06e740 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c06f530 con 0x7f5d9c1592b0 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da41cc640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c15f800 con 0x7f5d9c15c5d0 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60894/0 (socket says 172.21.15.106:60894) 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 -- 172.21.15.106:0/1657098166 learned_addr learned my addr 172.21.15.106:0/1657098166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da2769640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 0x7f5d9c140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1767640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 0x7f5d9c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 -- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 msgr2=0x7f5d9c140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 0x7f5d9c140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 -- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 msgr2=0x7f5d9c157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 0x7f5d9c157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da1f68640 1 -- 172.21.15.106:0/1657098166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c156ef0 con 0x7f5d9c06e740 2024-04-13T03:19:59.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.472+0000 7f5da2769640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 0x7f5d9c140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5da1767640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 0x7f5d9c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5da1f68640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c7f0 tx=0x7f5d8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c015070 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c007a30 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c007e00 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c155e50 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.473+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c156410 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.474+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c068a10 con 0x7f5d9c06e740 2024-04-13T03:19:59.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.474+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d8c018070 con 0x7f5d9c06e740 2024-04-13T03:19:59.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.475+0000 7f5d927fc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 0x7f5d8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.475+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f5d8c08ba60 con 0x7f5d9c06e740 2024-04-13T03:19:59.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.475+0000 7f5da1767640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 0x7f5d8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.475+0000 7f5da1767640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 0x7f5d8006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006fd0 tx=0x7f5d98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.479+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d9c156ef0 con 0x7f5d9c06e740 2024-04-13T03:19:59.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.654+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f5d9c0b30c0 con 0x7f5d8006c8f0 2024-04-13T03:19:59.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.654+0000 7f5d927fc640 1 -- 172.21.15.106:0/1657098166 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5d9c0b30c0 con 0x7f5d8006c8f0 2024-04-13T03:19:59.655 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 0 to deep-scrub 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 msgr2=0x7f5d8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 0x7f5d8006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006fd0 tx=0x7f5d98008040 comp rx=0 tx=0).stop 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 msgr2=0x7f5d9c155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c7f0 tx=0x7f5d8c00cbf0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 shutdown_connections 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f5d8006c8f0 0x7f5d8006edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f5d9c15c5d0 0x7f5d9c140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d9c1592b0 0x7f5d9c157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 --2- 172.21.15.106:0/1657098166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d9c06e740 0x7f5d9c155700 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 >> 172.21.15.106:0/1657098166 conn(0x7f5d9c0b2600 msgr2=0x7f5d9c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 shutdown_connections 2024-04-13T03:19:59.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.656+0000 7f5da41cc640 1 -- 172.21.15.106:0/1657098166 wait complete. 2024-04-13T03:19:59.668 INFO:tasks.ceph:Scrubbing osd.1 2024-04-13T03:19:59.668 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:19:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.771+0000 7fcea3799640 1 Processor -- start 2024-04-13T03:19:59.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea3799640 1 -- start start 2024-04-13T03:19:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea3799640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce9c0d2370 0x7fce9c0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea3799640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9c15c870 con 0x7fce9c0d3490 2024-04-13T03:19:59.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea3799640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9c15ee70 con 0x7fce9c0b3920 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea3799640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c0d5420 con 0x7fce9c0d2370 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea0d34640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fce9c0d2370 0x7fce9c0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea1535640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fce9c0d3490 0x7fce9c0d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60916/0 (socket says 172.21.15.106:60916) 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.772+0000 7fcea1535640 1 -- 172.21.15.106:0/2633817833 learned_addr learned my addr 172.21.15.106:0/2633817833 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.773+0000 7fcea0d34640 1 -- 172.21.15.106:0/2633817833 >> v1:172.21.15.114:6789/0 conn(0x7fce9c0b3920 legacy=0x7fce9c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.773+0000 7fcea0d34640 1 -- 172.21.15.106:0/2633817833 >> v1:172.21.15.106:6789/0 conn(0x7fce9c0d3490 legacy=0x7fce9c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.773+0000 7fcea0d34640 1 -- 172.21.15.106:0/2633817833 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0d2800 con 0x7fce9c0d2370 2024-04-13T03:19:59.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.773+0000 7fce93fff640 1 -- 172.21.15.106:0/2633817833 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1723541602 0 0) 0x7fce9c15c870 con 0x7fce9c0d3490 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.773+0000 7fcea0d34640 1 --2- 172.21.15.106:0/2633817833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0d2370 0x7fce9c0b13f0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009790 tx=0x7fce8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cde23d704ca03d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fce93fff640 1 -- 172.21.15.106:0/2633817833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0315a0 con 0x7fce9c0d2370 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fce93fff640 1 -- 172.21.15.106:0/2633817833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c037070 con 0x7fce9c0d2370 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fce93fff640 1 -- 172.21.15.106:0/2633817833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c031960 con 0x7fce9c0d2370 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 -- 172.21.15.106:0/2633817833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0d2370 msgr2=0x7fce9c0b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 --2- 172.21.15.106:0/2633817833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0d2370 0x7fce9c0b13f0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009790 tx=0x7fce8c02b840 comp rx=0 tx=0).stop 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 -- 172.21.15.106:0/2633817833 shutdown_connections 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 --2- 172.21.15.106:0/2633817833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0d2370 0x7fce9c0b13f0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 -- 172.21.15.106:0/2633817833 >> 172.21.15.106:0/2633817833 conn(0x7fce9c0b9760 msgr2=0x7fce9c0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 -- 172.21.15.106:0/2633817833 shutdown_connections 2024-04-13T03:19:59.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.774+0000 7fcea3799640 1 -- 172.21.15.106:0/2633817833 wait complete. 2024-04-13T03:19:59.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.775+0000 7fcea3799640 1 Processor -- start 2024-04-13T03:19:59.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.775+0000 7fcea3799640 1 -- start start 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 0x7fce9c0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0d3490 0x7fce9c1f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce9c0b5e90 0x7fce9c1f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0d30a0 con 0x7fce9c0b3920 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0d2dc0 con 0x7fce9c0d3490 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea3799640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0b1930 con 0x7fce9c0b5e90 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 0x7fce9c0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 0x7fce9c0b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60912/0 (socket says 172.21.15.106:60912) 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 -- 172.21.15.106:0/1612622773 learned_addr learned my addr 172.21.15.106:0/1612622773 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea0d34640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0d3490 0x7fce9c1f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1d36640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce9c0b5e90 0x7fce9c1f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 -- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce9c0b5e90 msgr2=0x7fce9c1f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fce9c0b5e90 0x7fce9c1f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 -- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0d3490 msgr2=0x7fce9c1f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0d3490 0x7fce9c1f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.776+0000 7fcea1535640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c06bb80 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fcea1535640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 0x7fce9c0b5950 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c8c0 tx=0x7fce9800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98018070 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9800a640 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9800aa30 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c0b6340 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.777+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c0b6900 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.778+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce9c068a10 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.778+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce9801b070 con 0x7fce9c0b3920 2024-04-13T03:19:59.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.778+0000 7fce91ffb640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce8006c8f0 0x7fce8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fce9808b100 con 0x7fce9c0b3920 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fcea0d34640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce8006c8f0 0x7fce8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fce91ffb640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7fce80072ed0 0x7fce80075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce800759e0 con 0x7fce80072ed0 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7fce98092050 con 0x7fce9c0b3920 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fcea1d36640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7fce80072ed0 0x7fce80075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fcea0d34640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce8006c8f0 0x7fce8006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.779+0000 7fcea1d36640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7fce80072ed0 0x7fce80075310 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.780+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== osd.1 v2:172.21.15.106:6800/1701180838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce800759e0 con 0x7fce80072ed0 2024-04-13T03:19:59.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 --> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fce9c157260 con 0x7fce80072ed0 2024-04-13T03:19:59.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fce91ffb640 1 -- 172.21.15.106:0/1612622773 <== osd.1 v2:172.21.15.106:6800/1701180838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fce9c157260 con 0x7fce80072ed0 2024-04-13T03:19:59.794 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:19:59.794 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:19:59.794 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7fce80072ed0 msgr2=0x7fce80075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6800/1701180838,v1:172.21.15.106:6801/1701180838] conn(0x7fce80072ed0 0x7fce80075310 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce8006c8f0 msgr2=0x7fce8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fce8006c8f0 0x7fce8006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c03d040 comp rx=0 tx=0).stop 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 msgr2=0x7fce9c0b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 --2- 172.21.15.106:0/1612622773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce9c0b3920 0x7fce9c0b5950 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c8c0 tx=0x7fce9800cd80 comp rx=0 tx=0).stop 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea1d36640 1 -- 172.21.15.106:0/1612622773 reap_dead start 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 shutdown_connections 2024-04-13T03:19:59.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 >> 172.21.15.106:0/1612622773 conn(0x7fce9c0b9760 msgr2=0x7fce9c15dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 shutdown_connections 2024-04-13T03:19:59.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.793+0000 7fcea3799640 1 -- 172.21.15.106:0/1612622773 wait complete. 2024-04-13T03:19:59.804 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-13T03:19:59.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.904+0000 7f4cace36640 1 Processor -- start 2024-04-13T03:19:59.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.904+0000 7f4cace36640 1 -- start start 2024-04-13T03:19:59.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4cace36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4ca815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4cace36640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca806d180 con 0x7f4ca806e740 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4cace36640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca80ce180 con 0x7f4ca815c7a0 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4cace36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca806ec90 con 0x7f4ca81592b0 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4c9ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4ca806e740 0x7f4ca815c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60920/0 (socket says 172.21.15.106:60920) 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4c9ffff640 1 -- 172.21.15.106:0/2490607672 learned_addr learned my addr 172.21.15.106:0/2490607672 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4ca815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763173185 0 0) 0x7f4ca806d180 con 0x7f4ca806e740 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c94003520 con 0x7f4ca806e740 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.1 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520169430 0 0) 0x7f4ca80ce180 con 0x7f4ca815c7a0 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ca806d180 con 0x7f4ca815c7a0 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1469178433 0 0) 0x7f4c94003520 con 0x7f4ca806e740 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 msgr2=0x7f4ca815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4ca815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 >> v1:172.21.15.114:6789/0 conn(0x7f4ca815c7a0 legacy=0x7f4ca80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca806ef70 con 0x7f4ca806e740 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4c90001670 con 0x7f4ca806e740 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4ca815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 >> v1:172.21.15.106:6789/0 conn(0x7f4ca806e740 legacy=0x7f4ca815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c94003b70 0x7f4c94003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4c94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c94004d40 0x7f4c940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca806ec90 con 0x7f4c94003b70 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400ba70 con 0x7f4c94004d40 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9400bc70 con 0x7f4ca81592b0 2024-04-13T03:19:59.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.905+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c94003b70 0x7f4c94003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4c9ffff640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4c94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca6d9d640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c94004d40 0x7f4c940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 -- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 msgr2=0x7f4c94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4ca81592b0 0x7f4c94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 -- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c94004d40 msgr2=0x7f4c940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c94004d40 0x7f4c940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 -- 172.21.15.106:0/2490607672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c94003520 con 0x7f4c94003b70 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.906+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c94003b70 0x7f4c94003f70 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009c90 tx=0x7f4c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a02be6aec6571b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98039070 con 0x7f4c94003b70 2024-04-13T03:19:59.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9802c8f0 con 0x7f4c94003b70 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4ca5d9b640 1 -- 172.21.15.106:0/2490607672 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98009860 con 0x7f4c94003b70 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 -- 172.21.15.106:0/2490607672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c94003b70 msgr2=0x7f4c94003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 --2- 172.21.15.106:0/2490607672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c94003b70 0x7f4c94003f70 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009c90 tx=0x7f4c94005830 comp rx=0 tx=0).stop 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4ca6d9d640 1 -- 172.21.15.106:0/2490607672 reap_dead start 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 -- 172.21.15.106:0/2490607672 shutdown_connections 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 -- 172.21.15.106:0/2490607672 >> 172.21.15.106:0/2490607672 conn(0x7f4ca80b2600 msgr2=0x7f4ca80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 -- 172.21.15.106:0/2490607672 shutdown_connections 2024-04-13T03:19:59.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.907+0000 7f4cace36640 1 -- 172.21.15.106:0/2490607672 wait complete. 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.908+0000 7f4cace36640 1 Processor -- start 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.908+0000 7f4cace36640 1 -- start start 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c94003b70 0x7f4ca80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 0x7f4ca812f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca815bb50 con 0x7f4ca80b39c0 2024-04-13T03:19:59.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca806f530 con 0x7f4ca815b120 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4cace36640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca815bcd0 con 0x7f4c94003b70 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60928/0 (socket says 172.21.15.106:60928) 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4c9ffff640 1 -- 172.21.15.106:0/2568746228 learned_addr learned my addr 172.21.15.106:0/2568746228 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4ca6d9d640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 0x7f4ca812f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.909+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c94003b70 0x7f4ca80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 -- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c94003b70 msgr2=0x7f4ca80b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c94003b70 0x7f4ca80b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 -- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 msgr2=0x7f4ca812f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 0x7f4ca812f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 -- 172.21.15.106:0/2568746228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca8130600 con 0x7f4ca80b39c0 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4ca6d9d640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 0x7f4ca812f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:19:59.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9ffff640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4c900072c0 tx=0x7f4c90007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90001670 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c90018070 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca81308e0 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000fc00 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.910+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca812dce0 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.911+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca8068a10 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.911+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c9001c410 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.912+0000 7f4c9effd640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.912+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f4c9008b6d0 con 0x7f4ca80b39c0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.912+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:19:59.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.912+0000 7f4ca659c640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 0x7f4c8806ede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c98030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:19:59.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:19:59.916+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c90059bc0 con 0x7f4ca80b39c0 2024-04-13T03:20:00.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.092+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f4ca815b560 con 0x7f4c8806c920 2024-04-13T03:20:00.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.093+0000 7f4c9effd640 1 -- 172.21.15.106:0/2568746228 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4ca815b560 con 0x7f4c8806c920 2024-04-13T03:20:00.093 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 1 to deep-scrub 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 msgr2=0x7f4c8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 0x7f4c8806ede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c98030000 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 msgr2=0x7f4ca815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4c900072c0 tx=0x7f4c90007780 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 shutdown_connections 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4c8806c920 0x7f4c8806ede0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca815b120 0x7f4ca812f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca80b39c0 0x7f4ca815abe0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 --2- 172.21.15.106:0/2568746228 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4c94003b70 0x7f4ca80b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 >> 172.21.15.106:0/2568746228 conn(0x7f4ca80b2600 msgr2=0x7f4ca806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 shutdown_connections 2024-04-13T03:20:00.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.095+0000 7f4cace36640 1 -- 172.21.15.106:0/2568746228 wait complete. 2024-04-13T03:20:00.106 INFO:tasks.ceph:Scrubbing osd.2 2024-04-13T03:20:00.106 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:00.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.213+0000 7f93bb488640 1 Processor -- start 2024-04-13T03:20:00.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.213+0000 7f93bb488640 1 -- start start 2024-04-13T03:20:00.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93bb488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93b40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93bb488640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b4161170 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93bb488640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b4160bb0 con 0x7f93b4146010 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93bb488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f93b4160950 con 0x7f93b40cd100 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93b9224640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f93b4145c30 0x7f93b4160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60936/0 (socket says 172.21.15.106:60936) 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93b9224640 1 -- 172.21.15.106:0/3561406985 learned_addr learned my addr 172.21.15.106:0/3561406985 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166863236 0 0) 0x7f93b4161170 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93a0003520 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93b8a23640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93b40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116915227 0 0) 0x7f93a0003520 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 msgr2=0x7f93b40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93b40cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 >> v1:172.21.15.114:6789/0 conn(0x7f93b4146010 legacy=0x7f93b40b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4160e90 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f93b0001670 con 0x7f93b4145c30 2024-04-13T03:20:00.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 >> v1:172.21.15.106:6789/0 conn(0x7f93b4145c30 legacy=0x7f93b4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93a00039e0 0x7f93a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0004d40 0x7f93a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4160950 con 0x7f93a00039e0 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93a000ba70 con 0x7f93a0004d40 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.214+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f93a000bc70 con 0x7f93b40cd100 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93a00039e0 0x7f93a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b8a23640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9a25640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0004d40 0x7f93a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 -- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 msgr2=0x7f93a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 -- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0004d40 msgr2=0x7f93a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0004d40 0x7f93a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b9224640 1 -- 172.21.15.106:0/3561406985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a0003520 con 0x7f93a00039e0 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.215+0000 7f93b8a23640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93b40cd100 0x7f93a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93b9224640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93a00039e0 0x7f93a0003e00 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c210 tx=0x7f93a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30b2eedda765f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0007cf0 con 0x7f93a00039e0 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b0030070 con 0x7f93a00039e0 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93abfff640 1 -- 172.21.15.106:0/3561406985 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0005d30 con 0x7f93a00039e0 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 -- 172.21.15.106:0/3561406985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93a00039e0 msgr2=0x7f93a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 --2- 172.21.15.106:0/3561406985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93a00039e0 0x7f93a0003e00 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c210 tx=0x7f93a0005830 comp rx=0 tx=0).stop 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93b9a25640 1 -- 172.21.15.106:0/3561406985 reap_dead start 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 -- 172.21.15.106:0/3561406985 shutdown_connections 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 -- 172.21.15.106:0/3561406985 >> 172.21.15.106:0/3561406985 conn(0x7f93b40d0370 msgr2=0x7f93b40d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 -- 172.21.15.106:0/3561406985 shutdown_connections 2024-04-13T03:20:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.216+0000 7f93bb488640 1 -- 172.21.15.106:0/3561406985 wait complete. 2024-04-13T03:20:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.217+0000 7f93bb488640 1 Processor -- start 2024-04-13T03:20:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.217+0000 7f93bb488640 1 -- start start 2024-04-13T03:20:00.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93a00039e0 0x7f93b413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93b413a280 0x7f93b4140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 0x7f93b415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93b406b8e0 con 0x7f93b415ed00 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40b3c00 con 0x7f93b413a280 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93bb488640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f93b406ba60 con 0x7f93a00039e0 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 0x7f93b415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 0x7f93b415f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60950/0 (socket says 172.21.15.106:60950) 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 -- 172.21.15.106:0/2221543996 learned_addr learned my addr 172.21.15.106:0/2221543996 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b8a23640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93b413a280 0x7f93b4140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9224640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93a00039e0 0x7f93b413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 -- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93a00039e0 msgr2=0x7f93b413ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f93a00039e0 0x7f93b413ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 -- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93b413a280 msgr2=0x7f93b4140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93b413a280 0x7f93b4140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.218+0000 7f93b9a25640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4140e00 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93b9a25640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 0x7f93b415f180 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f93b4070e30 tx=0x7f93ac00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac003970 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac018070 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41410e0 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac003d50 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.219+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b413bfe0 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.220+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93b4141320 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.220+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93ac0050d0 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93a9ffb640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f939806c920 0x7f939806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f93ac08b680 con 0x7f93b415ed00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93b9224640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f939806c920 0x7f939806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93a9ffb640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f9398072f00 0x7f9398075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9398075a10 con 0x7f9398072f00 2024-04-13T03:20:00.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f93ac092050 con 0x7f93b415ed00 2024-04-13T03:20:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93b8a23640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f9398072f00 0x7f9398075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93b9224640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f939806c920 0x7f939806ede0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034000 tx=0x7f93b0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.221+0000 7f93b8a23640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f9398072f00 0x7f9398075340 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.222+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== osd.2 v2:172.21.15.106:6824/4087145286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9398075a10 con 0x7f9398072f00 2024-04-13T03:20:00.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 --> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93b4068a10 con 0x7f9398072f00 2024-04-13T03:20:00.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93a9ffb640 1 -- 172.21.15.106:0/2221543996 <== osd.2 v2:172.21.15.106:6824/4087145286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93b4068a10 con 0x7f9398072f00 2024-04-13T03:20:00.236 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:00.236 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:00.236 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:00.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f9398072f00 msgr2=0x7f9398075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6824/4087145286,v1:172.21.15.106:6825/4087145286] conn(0x7f9398072f00 0x7f9398075340 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f939806c920 msgr2=0x7f939806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f939806c920 0x7f939806ede0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034000 tx=0x7f93b0007000 comp rx=0 tx=0).stop 2024-04-13T03:20:00.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 msgr2=0x7f93b415f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 --2- 172.21.15.106:0/2221543996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b415ed00 0x7f93b415f180 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f93b4070e30 tx=0x7f93ac00eb90 comp rx=0 tx=0).stop 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93b9a25640 1 -- 172.21.15.106:0/2221543996 reap_dead start 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 shutdown_connections 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.235+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 >> 172.21.15.106:0/2221543996 conn(0x7f93b40d0370 msgr2=0x7f93b406ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.236+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 shutdown_connections 2024-04-13T03:20:00.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.236+0000 7f93bb488640 1 -- 172.21.15.106:0/2221543996 wait complete. 2024-04-13T03:20:00.246 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-13T03:20:00.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.348+0000 7fdb8837b640 1 Processor -- start 2024-04-13T03:20:00.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.348+0000 7fdb8837b640 1 -- start start 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb8837b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb8015e930 0x7fdb800b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb8837b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8006ed70 0x7fdb800d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb8837b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb8006e280 con 0x7fdb800d0820 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb8837b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdb800d0c90 con 0x7fdb8015e930 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb8837b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8006e560 con 0x7fdb8006ed70 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86117640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdb8015e930 0x7fdb800b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86918640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdb800d0820 0x7fdb800cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60948/0 (socket says 172.21.15.106:60948) 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86918640 1 -- 172.21.15.106:0/4015916704 learned_addr learned my addr 172.21.15.106:0/4015916704 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb85916640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8006ed70 0x7fdb800d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86117640 1 -- 172.21.15.106:0/4015916704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8006ed70 msgr2=0x7fdb800d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86117640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8006ed70 0x7fdb800d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86117640 1 -- 172.21.15.106:0/4015916704 >> v1:172.21.15.106:6789/0 conn(0x7fdb800d0820 legacy=0x7fdb800cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.349+0000 7fdb86117640 1 -- 172.21.15.106:0/4015916704 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb800cccf0 con 0x7fdb8015e930 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.350+0000 7fdb86117640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb8015e930 0x7fdb800b5350 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0097a0 tx=0x7fdb7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225be451746618f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.350+0000 7fdb85115640 1 -- 172.21.15.106:0/4015916704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c035020 con 0x7fdb8015e930 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.350+0000 7fdb85115640 1 -- 172.21.15.106:0/4015916704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c031830 con 0x7fdb8015e930 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.350+0000 7fdb85115640 1 -- 172.21.15.106:0/4015916704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c031c00 con 0x7fdb8015e930 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 -- 172.21.15.106:0/4015916704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb8015e930 msgr2=0x7fdb800b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb8015e930 0x7fdb800b5350 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0097a0 tx=0x7fdb7c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 -- 172.21.15.106:0/4015916704 shutdown_connections 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8006ed70 0x7fdb800d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 --2- 172.21.15.106:0/4015916704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb8015e930 0x7fdb800b5350 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 -- 172.21.15.106:0/4015916704 >> 172.21.15.106:0/4015916704 conn(0x7fdb800cef80 msgr2=0x7fdb800cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 -- 172.21.15.106:0/4015916704 shutdown_connections 2024-04-13T03:20:00.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.351+0000 7fdb8837b640 1 -- 172.21.15.106:0/4015916704 wait complete. 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.352+0000 7fdb8837b640 1 Processor -- start 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.352+0000 7fdb8837b640 1 -- start start 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb8006ed70 0x7fdb8015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8015e930 0x7fdb8014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8006f1d0 con 0x7fdb800d0820 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800cd2b0 con 0x7fdb8006ed70 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800ce830 con 0x7fdb8015e930 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60960/0 (socket says 172.21.15.106:60960) 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 -- 172.21.15.106:0/3795720132 learned_addr learned my addr 172.21.15.106:0/3795720132 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 -- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8015e930 msgr2=0x7fdb8014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8015e930 0x7fdb8014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 -- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb8006ed70 msgr2=0x7fdb8015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb8006ed70 0x7fdb8015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 -- 172.21.15.106:0/3795720132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb800cafa0 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb85916640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d800 tx=0x7fdb7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000b6c0 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb70019070 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb800cb280 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000ba80 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.353+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb800cb840 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.354+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80068a10 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.355+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb7000bc20 con 0x7fdb800d0820 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.355+0000 7fdb76ffd640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 0x7fdb6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.355+0000 7fdb86117640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 0x7fdb6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.355+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fdb7008bf60 con 0x7fdb800d0820 2024-04-13T03:20:00.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.356+0000 7fdb86117640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 0x7fdb6406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c036000 tx=0x7fdb7c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.360+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb7005a450 con 0x7fdb800d0820 2024-04-13T03:20:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.537+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdb800b7400 con 0x7fdb6406c8f0 2024-04-13T03:20:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.538+0000 7fdb76ffd640 1 -- 172.21.15.106:0/3795720132 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdb800b7400 con 0x7fdb6406c8f0 2024-04-13T03:20:00.538 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 2 to deep-scrub 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 msgr2=0x7fdb6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 0x7fdb6406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c036000 tx=0x7fdb7c030000 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 msgr2=0x7fdb800cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d800 tx=0x7fdb7000dcc0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 shutdown_connections 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fdb6406c8f0 0x7fdb6406edb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fdb8015e930 0x7fdb8014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb800d0820 0x7fdb800cc040 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 --2- 172.21.15.106:0/3795720132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb8006ed70 0x7fdb8015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 >> 172.21.15.106:0/3795720132 conn(0x7fdb800cef80 msgr2=0x7fdb800b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 shutdown_connections 2024-04-13T03:20:00.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.540+0000 7fdb8837b640 1 -- 172.21.15.106:0/3795720132 wait complete. 2024-04-13T03:20:00.551 INFO:tasks.ceph:Scrubbing osd.3 2024-04-13T03:20:00.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:00.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.655+0000 7f187e308640 1 Processor -- start 2024-04-13T03:20:00.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.655+0000 7f187e308640 1 -- start start 2024-04-13T03:20:00.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187e308640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1878145c30 0x7f1878160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187e308640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1878161170 con 0x7f1878146010 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187e308640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1878160bb0 con 0x7f18780cd100 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187e308640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1878160e90 con 0x7f1878145c30 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187c8a5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1878146010 0x7f18780b4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60964/0 (socket says 172.21.15.106:60964) 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f187c8a5640 1 -- 172.21.15.106:0/1899568018 learned_addr learned my addr 172.21.15.106:0/1899568018 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1877fff640 1 --2- 172.21.15.106:0/1899568018 >> v2:172.21.15.114:3300/0 conn(0x7f1878145c30 0x7f1878160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510915571 0 0) 0x7f1878161170 con 0x7f1878146010 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1860003520 con 0x7f1878146010 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3547252124 0 0) 0x7f1860003520 con 0x7f1878146010 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 >> v2:172.21.15.114:3300/0 conn(0x7f1878145c30 msgr2=0x7f1878160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 --2- 172.21.15.106:0/1899568018 >> v2:172.21.15.114:3300/0 conn(0x7f1878145c30 0x7f1878160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 >> v1:172.21.15.114:6789/0 conn(0x7f18780cd100 legacy=0x7f18780cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18780b3920 con 0x7f1878146010 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.656+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f18700019f0 con 0x7f1878146010 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 >> v1:172.21.15.106:6789/0 conn(0x7f1878146010 legacy=0x7f18780b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f1860003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1860004800 0x7f1860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18600051c0 0x7f186000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1878160e90 con 0x7f18600039b0 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f186000c060 con 0x7f18600051c0 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f186000c260 con 0x7f1860004800 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f1860003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f187c8a5640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18600051c0 0x7f186000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f18777fe640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1860004800 0x7f1860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 -- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1860004800 msgr2=0x7f1860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1860004800 0x7f1860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 -- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18600051c0 msgr2=0x7f186000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18600051c0 0x7f186000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f187c8a5640 1 -- 172.21.15.106:0/1899568018 reap_dead start 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.657+0000 7f1877fff640 1 -- 172.21.15.106:0/1899568018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1860003520 con 0x7f18600039b0 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f1877fff640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f1860003dd0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f1868009ac0 tx=0x7f1860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439d4ced89d7111b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868039070 con 0x7f18600039b0 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f186802c6b0 con 0x7f18600039b0 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f1876ffd640 1 -- 172.21.15.106:0/1899568018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868009690 con 0x7f18600039b0 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 -- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 msgr2=0x7f1860003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f1860003dd0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f1868009ac0 tx=0x7f1860005830 comp rx=0 tx=0).stop 2024-04-13T03:20:00.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 -- 172.21.15.106:0/1899568018 shutdown_connections 2024-04-13T03:20:00.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 --2- 172.21.15.106:0/1899568018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f1860003dd0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 -- 172.21.15.106:0/1899568018 >> 172.21.15.106:0/1899568018 conn(0x7f18780d0370 msgr2=0x7f18780d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 -- 172.21.15.106:0/1899568018 shutdown_connections 2024-04-13T03:20:00.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.658+0000 7f187e308640 1 -- 172.21.15.106:0/1899568018 wait complete. 2024-04-13T03:20:00.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.659+0000 7f187e308640 1 Processor -- start 2024-04-13T03:20:00.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 -- start start 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f187815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18780af3a0 0x7f187806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18780ae6e0 0x7f18780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187806ba60 con 0x7f18600039b0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f187806b8e0 con 0x7f18780ae6e0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f187e308640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f1878160240 con 0x7f18780af3a0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f1877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f187815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f1877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f187815beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60978/0 (socket says 172.21.15.106:60978) 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.660+0000 7f1877fff640 1 -- 172.21.15.106:0/11713907 learned_addr learned my addr 172.21.15.106:0/11713907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f187c8a5640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18780ae6e0 0x7f18780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f18777fe640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18780af3a0 0x7f187806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 -- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18780af3a0 msgr2=0x7f187806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18780af3a0 0x7f187806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 -- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18780ae6e0 msgr2=0x7f18780aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18780ae6e0 0x7f18780aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187815c3f0 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f18777fe640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f18780af3a0 0x7f187806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1877fff640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f187815beb0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f1868000c00 tx=0x7f1868004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868042020 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f186803a040 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187806ea30 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868039070 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.661+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187806eff0 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.662+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f187806e7e0 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1868045020 con 0x7f18600039b0 2024-04-13T03:20:00.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f186406c920 0x7f186406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f18680ad010 con 0x7f18600039b0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f18777fe640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f186406c920 0x7f186406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f1864072f00 0x7f1864075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f186803de50 con 0x7f1864072f00 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f18680b4050 con 0x7f18600039b0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f187c8a5640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f1864072f00 0x7f1864075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.663+0000 7f18777fe640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f186406c920 0x7f186406ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f186c000c00 tx=0x7f186c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.664+0000 7f187c8a5640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f1864072f00 0x7f1864075340 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.664+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== osd.3 v2:172.21.15.106:6811/987945743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f186803de50 con 0x7f1864072f00 2024-04-13T03:20:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.677+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 --> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1878068a10 con 0x7f1864072f00 2024-04-13T03:20:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f1874ff9640 1 -- 172.21.15.106:0/11713907 <== osd.3 v2:172.21.15.106:6811/987945743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1878068a10 con 0x7f1864072f00 2024-04-13T03:20:00.678 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:00.679 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:00.679 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f1864072f00 msgr2=0x7f1864075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6811/987945743,v1:172.21.15.106:6813/987945743] conn(0x7f1864072f00 0x7f1864075340 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f186406c920 msgr2=0x7f186406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f186406c920 0x7f186406ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f186c000c00 tx=0x7f186c009040 comp rx=0 tx=0).stop 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 msgr2=0x7f187815beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 --2- 172.21.15.106:0/11713907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18600039b0 0x7f187815beb0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f1868000c00 tx=0x7f1868004730 comp rx=0 tx=0).stop 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187c8a5640 1 -- 172.21.15.106:0/11713907 reap_dead start 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 shutdown_connections 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 >> 172.21.15.106:0/11713907 conn(0x7f18780d0370 msgr2=0x7f18780b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 shutdown_connections 2024-04-13T03:20:00.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.678+0000 7f187e308640 1 -- 172.21.15.106:0/11713907 wait complete. 2024-04-13T03:20:00.690 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-13T03:20:00.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.790+0000 7f850f1b4640 1 Processor -- start 2024-04-13T03:20:00.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.790+0000 7f850f1b4640 1 -- start start 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850f1b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f850815d380 0x7f850815d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850f1b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f850806f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850f1b4640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85080d0640 con 0x7f85080ccd40 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850f1b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f85080d0f70 con 0x7f850815d380 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850f1b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f85080d0920 con 0x7f85080b7f40 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850cf50640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f850815d380 0x7f850815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850cf50640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f850815d380 0x7f850815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60988/0 (socket says 172.21.15.106:60988) 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.791+0000 7f850cf50640 1 -- 172.21.15.106:0/2924829135 learned_addr learned my addr 172.21.15.106:0/2924829135 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.792+0000 7f850cf50640 1 -- 172.21.15.106:0/2924829135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 msgr2=0x7f850806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.792+0000 7f850cf50640 1 --2- 172.21.15.106:0/2924829135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f850806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.792+0000 7f850cf50640 1 -- 172.21.15.106:0/2924829135 >> v1:172.21.15.114:6789/0 conn(0x7f85080ccd40 legacy=0x7f85080d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.792+0000 7f850cf50640 1 -- 172.21.15.106:0/2924829135 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85080d0c00 con 0x7f850815d380 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.792+0000 7f850cf50640 1 --2- 172.21.15.106:0/2924829135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f850815d760 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009870 tx=0x7f84f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad76edcf8014ab9a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f85077fe640 1 -- 172.21.15.106:0/2924829135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80317c0 con 0x7f850815d380 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f85077fe640 1 -- 172.21.15.106:0/2924829135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8037070 con 0x7f850815d380 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f85077fe640 1 -- 172.21.15.106:0/2924829135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8031b80 con 0x7f850815d380 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 -- 172.21.15.106:0/2924829135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 msgr2=0x7f850815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 --2- 172.21.15.106:0/2924829135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f850815d760 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009870 tx=0x7f84f802b860 comp rx=0 tx=0).stop 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 -- 172.21.15.106:0/2924829135 shutdown_connections 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 --2- 172.21.15.106:0/2924829135 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f850806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 --2- 172.21.15.106:0/2924829135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f850815d760 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 -- 172.21.15.106:0/2924829135 >> 172.21.15.106:0/2924829135 conn(0x7f85080d2530 msgr2=0x7f85080d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 -- 172.21.15.106:0/2924829135 shutdown_connections 2024-04-13T03:20:00.796 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.793+0000 7f850f1b4640 1 -- 172.21.15.106:0/2924829135 wait complete. 2024-04-13T03:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.794+0000 7f850f1b4640 1 Processor -- start 2024-04-13T03:20:00.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.794+0000 7f850f1b4640 1 -- start start 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f8508153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85080ccd40 0x7f85080cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85080cd920 con 0x7f850815d380 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85080cd1d0 con 0x7f85080ccd40 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850f1b4640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f85080c7010 con 0x7f85080b7f40 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850d751640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850d751640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60994/0 (socket says 172.21.15.106:60994) 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850d751640 1 -- 172.21.15.106:0/1348314968 learned_addr learned my addr 172.21.15.106:0/1348314968 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:00.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f850cf50640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f8508153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.795+0000 7f8507fff640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85080ccd40 0x7f85080cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 -- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 msgr2=0x7f8508153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f8508153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 -- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85080ccd40 msgr2=0x7f85080cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85080ccd40 0x7f85080cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 -- 172.21.15.106:0/1348314968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85080c72f0 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850d751640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f850000c780 tx=0x7f850000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500004b60 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500004d20 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85080c75d0 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500005270 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.796+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85080c62a0 con 0x7f850815d380 2024-04-13T03:20:00.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.797+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8508069120 con 0x7f850815d380 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.797+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8500005410 con 0x7f850815d380 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.798+0000 7f85057fa640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 0x7f84f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.798+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f850008bbd0 con 0x7f850815d380 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.798+0000 7f850cf50640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 0x7f84f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.798+0000 7f850cf50640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 0x7f84f406edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:00.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.802+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8500020070 con 0x7f850815d380 2024-04-13T03:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.976+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f85080d3190 con 0x7f84f406c8f0 2024-04-13T03:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.977+0000 7f85057fa640 1 -- 172.21.15.106:0/1348314968 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f85080d3190 con 0x7f84f406c8f0 2024-04-13T03:20:00.977 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 3 to deep-scrub 2024-04-13T03:20:00.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.978+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 msgr2=0x7f84f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.978+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 0x7f84f406edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803d040 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.978+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 msgr2=0x7f85080cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.978+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f850000c780 tx=0x7f850000cb80 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 shutdown_connections 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f84f406c8f0 0x7f84f406edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f850815d380 0x7f85080cbb50 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85080ccd40 0x7f85080cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 --2- 172.21.15.106:0/1348314968 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f85080b7f40 0x7f8508153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 >> 172.21.15.106:0/1348314968 conn(0x7f85080d2530 msgr2=0x7f85080b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 shutdown_connections 2024-04-13T03:20:00.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:00.979+0000 7f850f1b4640 1 -- 172.21.15.106:0/1348314968 wait complete. 2024-04-13T03:20:00.990 INFO:tasks.ceph:Scrubbing osd.4 2024-04-13T03:20:00.990 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:01.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.094+0000 7f0b3fe80640 1 Processor -- start 2024-04-13T03:20:01.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.094+0000 7f0b3fe80640 1 -- start start 2024-04-13T03:20:01.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3fe80640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b38070730 0x7f0b38070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3fe80640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 0x7f0b3806c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3fe80640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b380d11e0 con 0x7f0b38145470 2024-04-13T03:20:01.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3fe80640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0b380d03a0 con 0x7f0b38070730 2024-04-13T03:20:01.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3fe80640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b380d05d0 con 0x7f0b3806c450 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3d41b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b38070730 0x7f0b38070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3dc1c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b38145470 0x7f0b38145af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60974/0 (socket says 172.21.15.106:60974) 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3dc1c640 1 -- 172.21.15.106:0/2205959930 learned_addr learned my addr 172.21.15.106:0/2205959930 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3e41d640 1 --2- 172.21.15.106:0/2205959930 >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 0x7f0b3806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3d41b640 1 -- 172.21.15.106:0/2205959930 >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 msgr2=0x7f0b3806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3d41b640 1 --2- 172.21.15.106:0/2205959930 >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 0x7f0b3806c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3d41b640 1 -- 172.21.15.106:0/2205959930 >> v1:172.21.15.106:6789/0 conn(0x7f0b38145470 legacy=0x7f0b38145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3cc1a640 1 -- 172.21.15.106:0/2205959930 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2536818009 0 0) 0x7f0b380d11e0 con 0x7f0b38145470 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3d41b640 1 -- 172.21.15.106:0/2205959930 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b380d08b0 con 0x7f0b38070730 2024-04-13T03:20:01.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.095+0000 7f0b3e41d640 1 --2- 172.21.15.106:0/2205959930 >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 0x7f0b3806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.096+0000 7f0b3d41b640 1 --2- 172.21.15.106:0/2205959930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b38070730 0x7f0b38070bc0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cdbd0 tx=0x7f0b2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd493ea3323cd6d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.096+0000 7f0b3cc1a640 1 -- 172.21.15.106:0/2205959930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28035020 con 0x7f0b38070730 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.096+0000 7f0b3cc1a640 1 -- 172.21.15.106:0/2205959930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28031830 con 0x7f0b38070730 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.096+0000 7f0b3cc1a640 1 -- 172.21.15.106:0/2205959930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28031c00 con 0x7f0b38070730 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 -- 172.21.15.106:0/2205959930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b38070730 msgr2=0x7f0b38070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/2205959930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b38070730 0x7f0b38070bc0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cdbd0 tx=0x7f0b2802b8d0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 -- 172.21.15.106:0/2205959930 shutdown_connections 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/2205959930 >> v2:172.21.15.114:3300/0 conn(0x7f0b3806c450 0x7f0b3806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/2205959930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b38070730 0x7f0b38070bc0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 -- 172.21.15.106:0/2205959930 >> 172.21.15.106:0/2205959930 conn(0x7f0b3806f290 msgr2=0x7f0b3806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 -- 172.21.15.106:0/2205959930 shutdown_connections 2024-04-13T03:20:01.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.097+0000 7f0b3fe80640 1 -- 172.21.15.106:0/2205959930 wait complete. 2024-04-13T03:20:01.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.098+0000 7f0b3fe80640 1 Processor -- start 2024-04-13T03:20:01.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.098+0000 7f0b3fe80640 1 -- start start 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 0x7f0b380aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b38070730 0x7f0b381f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b38145470 0x7f0b380b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38145850 con 0x7f0b3806c450 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380d0e70 con 0x7f0b38070730 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f0b380cee30 con 0x7f0b38145470 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 0x7f0b380aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 0x7f0b380aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32778/0 (socket says 172.21.15.106:32778) 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 -- 172.21.15.106:0/313043640 learned_addr learned my addr 172.21.15.106:0/313043640 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 -- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b38145470 msgr2=0x7f0b380b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0b38145470 0x7f0b380b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 -- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b38070730 msgr2=0x7f0b381f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b38070730 0x7f0b381f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3815f3b0 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3dc1c640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 0x7f0b380aa630 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400c7c0 tx=0x7f0b3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34018070 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3400a640 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400aa30 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3815f690 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.099+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381f3830 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.100+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b38075180 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b3401b070 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b1c06c8f0 0x7f0b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f0b3408b140 con 0x7f0b3806c450 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b3d41b640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b1c06c8f0 0x7f0b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7f0b1c072ed0 0x7f0b1c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b1c0759e0 con 0x7f0b1c072ed0 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.101+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f0b34092050 con 0x7f0b3806c450 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.102+0000 7f0b3e41d640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7f0b1c072ed0 0x7f0b1c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.102+0000 7f0b3d41b640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b1c06c8f0 0x7f0b1c06edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b28036000 tx=0x7f0b28030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.102+0000 7f0b3e41d640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7f0b1c072ed0 0x7f0b1c075310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.103+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== osd.4 v2:172.21.15.114:6800/1124906481 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b1c0759e0 con 0x7f0b1c072ed0 2024-04-13T03:20:01.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.116+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 --> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0b38068a10 con 0x7f0b1c072ed0 2024-04-13T03:20:01.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.116+0000 7f0b2e7fc640 1 -- 172.21.15.106:0/313043640 <== osd.4 v2:172.21.15.114:6800/1124906481 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0b38068a10 con 0x7f0b1c072ed0 2024-04-13T03:20:01.117 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:01.117 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:01.117 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:01.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7f0b1c072ed0 msgr2=0x7f0b1c075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.114:6800/1124906481,v1:172.21.15.114:6801/1124906481] conn(0x7f0b1c072ed0 0x7f0b1c075310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b1c06c8f0 msgr2=0x7f0b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f0b1c06c8f0 0x7f0b1c06edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b28036000 tx=0x7f0b28030000 comp rx=0 tx=0).stop 2024-04-13T03:20:01.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 msgr2=0x7f0b380aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 --2- 172.21.15.106:0/313043640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b3806c450 0x7f0b380aa630 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400c7c0 tx=0x7f0b3400cc80 comp rx=0 tx=0).stop 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3e41d640 1 -- 172.21.15.106:0/313043640 reap_dead start 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 shutdown_connections 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 >> 172.21.15.106:0/313043640 conn(0x7f0b3806f290 msgr2=0x7f0b38072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 shutdown_connections 2024-04-13T03:20:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.117+0000 7f0b3fe80640 1 -- 172.21.15.106:0/313043640 wait complete. 2024-04-13T03:20:01.128 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-13T03:20:01.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.229+0000 7fb668f99640 1 Processor -- start 2024-04-13T03:20:01.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.229+0000 7fb668f99640 1 -- start start 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb668f99640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 0x7fb66415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb668f99640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb664070bb0 0x7fb66406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb668f99640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6640b5cf0 con 0x7fb6640b54a0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb668f99640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb6640b5fd0 con 0x7fb664070bb0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb668f99640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb6640b62b0 con 0x7fb6640b1c30 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb662d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb664070bb0 0x7fb66406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb661d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb6640b54a0 0x7fb6640704a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60986/0 (socket says 172.21.15.106:60986) 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb661d9b640 1 -- 172.21.15.106:0/1166004199 learned_addr learned my addr 172.21.15.106:0/1166004199 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb66259c640 1 --2- 172.21.15.106:0/1166004199 >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 0x7fb66415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb66159a640 1 -- 172.21.15.106:0/1166004199 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400849945 0 0) 0x7fb6640b5cf0 con 0x7fb6640b54a0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb66159a640 1 -- 172.21.15.106:0/1166004199 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb64c003520 con 0x7fb6640b54a0 2024-04-13T03:20:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb662d9d640 1 -- 172.21.15.106:0/1166004199 >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 msgr2=0x7fb66415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb662d9d640 1 --2- 172.21.15.106:0/1166004199 >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 0x7fb66415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb662d9d640 1 -- 172.21.15.106:0/1166004199 >> v1:172.21.15.106:6789/0 conn(0x7fb6640b54a0 legacy=0x7fb6640704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb662d9d640 1 -- 172.21.15.106:0/1166004199 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6640b6590 con 0x7fb664070bb0 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.230+0000 7fb66259c640 1 --2- 172.21.15.106:0/1166004199 >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 0x7fb66415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb662d9d640 1 --2- 172.21.15.106:0/1166004199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb664070bb0 0x7fb66406d660 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb6640687a0 tx=0x7fb65c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba250786c1a4083 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb66159a640 1 -- 172.21.15.106:0/1166004199 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c039070 con 0x7fb664070bb0 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb66159a640 1 -- 172.21.15.106:0/1166004199 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c02cc70 con 0x7fb664070bb0 2024-04-13T03:20:01.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb66159a640 1 -- 172.21.15.106:0/1166004199 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c034640 con 0x7fb664070bb0 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb668f99640 1 -- 172.21.15.106:0/1166004199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb664070bb0 msgr2=0x7fb66406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.231+0000 7fb668f99640 1 --2- 172.21.15.106:0/1166004199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb664070bb0 0x7fb66406d660 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb6640687a0 tx=0x7fb65c02bec0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 -- 172.21.15.106:0/1166004199 shutdown_connections 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 --2- 172.21.15.106:0/1166004199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb664070bb0 0x7fb66406d660 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 --2- 172.21.15.106:0/1166004199 >> v2:172.21.15.114:3300/0 conn(0x7fb6640b1c30 0x7fb66415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 -- 172.21.15.106:0/1166004199 >> 172.21.15.106:0/1166004199 conn(0x7fb664071870 msgr2=0x7fb664071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 -- 172.21.15.106:0/1166004199 shutdown_connections 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.232+0000 7fb668f99640 1 -- 172.21.15.106:0/1166004199 wait complete. 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.233+0000 7fb668f99640 1 Processor -- start 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.233+0000 7fb668f99640 1 -- start start 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb664070bb0 0x7fb6641f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6640b54a0 0x7fb664152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb66406dd20 con 0x7fb6640b1c30 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb66406dba0 con 0x7fb664070bb0 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fb66406dea0 con 0x7fb6640b54a0 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32792/0 (socket says 172.21.15.106:32792) 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 -- 172.21.15.106:0/331782140 learned_addr learned my addr 172.21.15.106:0/331782140 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 -- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6640b54a0 msgr2=0x7fb664152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6640b54a0 0x7fb664152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 -- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb664070bb0 msgr2=0x7fb6641f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb66259c640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb664070bb0 0x7fb6641f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb664070bb0 0x7fb6641f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 -- 172.21.15.106:0/331782140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb664154d70 con 0x7fb6640b1c30 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb661d9b640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c390 tx=0x7fb65800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658017070 con 0x7fb6640b1c30 2024-04-13T03:20:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658005ce0 con 0x7fb6640b1c30 2024-04-13T03:20:01.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb664161260 con 0x7fb6640b1c30 2024-04-13T03:20:01.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.234+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580072f0 con 0x7fb6640b1c30 2024-04-13T03:20:01.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.235+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb664161820 con 0x7fb6640b1c30 2024-04-13T03:20:01.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.235+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb664068a10 con 0x7fb6640b1c30 2024-04-13T03:20:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.236+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb65801a070 con 0x7fb6640b1c30 2024-04-13T03:20:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.236+0000 7fb652ffd640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.236+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fb65808ae80 con 0x7fb6640b1c30 2024-04-13T03:20:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.236+0000 7fb66259c640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.237+0000 7fb66259c640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 0x7fb64406edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb65400d8b0 tx=0x7fb654008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.241+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb658059370 con 0x7fb6640b1c30 2024-04-13T03:20:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.416+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb6640711f0 con 0x7fb64406c8f0 2024-04-13T03:20:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.416+0000 7fb652ffd640 1 -- 172.21.15.106:0/331782140 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb6640711f0 con 0x7fb64406c8f0 2024-04-13T03:20:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 4 to deep-scrub 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 msgr2=0x7fb64406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 0x7fb64406edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb65400d8b0 tx=0x7fb654008040 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 msgr2=0x7fb664150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c390 tx=0x7fb65800c850 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 shutdown_connections 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fb6640b54a0 0x7fb664152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6640b1c30 0x7fb664150bf0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 --2- 172.21.15.106:0/331782140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb664070bb0 0x7fb6641f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 >> 172.21.15.106:0/331782140 conn(0x7fb664071870 msgr2=0x7fb66415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 shutdown_connections 2024-04-13T03:20:01.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.418+0000 7fb668f99640 1 -- 172.21.15.106:0/331782140 wait complete. 2024-04-13T03:20:01.430 INFO:tasks.ceph:Scrubbing osd.5 2024-04-13T03:20:01.430 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 Processor -- start 2024-04-13T03:20:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 -- start start 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5dc0d3490 0x7fe5dc0d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc15c870 con 0x7fe5dc0b3920 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc0d5420 con 0x7fe5dc0d2370 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.534+0000 7fe5e2620640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc15ee70 con 0x7fe5dc0d3490 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32802/0 (socket says 172.21.15.106:32802) 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 -- 172.21.15.106:0/2545731710 learned_addr learned my addr 172.21.15.106:0/2545731710 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/2545731710 >> v2:172.21.15.114:3300/0 conn(0x7fe5dc0d3490 0x7fe5dc0d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 -- 172.21.15.106:0/2545731710 >> v2:172.21.15.114:3300/0 conn(0x7fe5dc0d3490 msgr2=0x7fe5dc0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/2545731710 >> v2:172.21.15.114:3300/0 conn(0x7fe5dc0d3490 0x7fe5dc0d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 -- 172.21.15.106:0/2545731710 >> v1:172.21.15.114:6789/0 conn(0x7fe5dc0b3920 legacy=0x7fe5dc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.535+0000 7fe5db7fe640 1 -- 172.21.15.106:0/2545731710 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc0d3db0 con 0x7fe5dc0d2370 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/2545731710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0b8700 tx=0x7fe5d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf119dd93f09016 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5daffd640 1 -- 172.21.15.106:0/2545731710 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002d040 con 0x7fe5dc0d2370 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5daffd640 1 -- 172.21.15.106:0/2545731710 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d0004230 con 0x7fe5dc0d2370 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5daffd640 1 -- 172.21.15.106:0/2545731710 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0003610 con 0x7fe5dc0d2370 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 -- 172.21.15.106:0/2545731710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d2370 msgr2=0x7fe5dc0b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 --2- 172.21.15.106:0/2545731710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0b8700 tx=0x7fe5d002b8f0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 -- 172.21.15.106:0/2545731710 shutdown_connections 2024-04-13T03:20:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 --2- 172.21.15.106:0/2545731710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d2370 0x7fe5dc0b13f0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 --2- 172.21.15.106:0/2545731710 >> v2:172.21.15.114:3300/0 conn(0x7fe5dc0d3490 0x7fe5dc0d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 -- 172.21.15.106:0/2545731710 >> 172.21.15.106:0/2545731710 conn(0x7fe5dc0b9760 msgr2=0x7fe5dc0b9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 -- 172.21.15.106:0/2545731710 shutdown_connections 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.536+0000 7fe5e2620640 1 -- 172.21.15.106:0/2545731710 wait complete. 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.537+0000 7fe5e2620640 1 Processor -- start 2024-04-13T03:20:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 -- start start 2024-04-13T03:20:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5dc0b3920 0x7fe5dc160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 0x7fe5dc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5dc06d1b0 0x7fe5dc1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d2dc0 con 0x7fe5dc0d3490 2024-04-13T03:20:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d2ae0 con 0x7fe5dc06d1b0 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5e2620640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d30a0 con 0x7fe5dc0b3920 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 0x7fe5dc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 0x7fe5dc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32804/0 (socket says 172.21.15.106:32804) 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.538+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3007786370 learned_addr learned my addr 172.21.15.106:0/3007786370 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5e0bbd640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5dc06d1b0 0x7fe5dc1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5dc0b3920 0x7fe5dc160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5dc0b3920 msgr2=0x7fe5dc160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe5dc0b3920 0x7fe5dc160520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5dc06d1b0 msgr2=0x7fe5dc1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5dc06d1b0 0x7fe5dc1588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc06d730 con 0x7fe5dc0d3490 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 0x7fe5dc06cc70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00090b0 tx=0x7fe5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0003c50 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d0003e10 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0006280 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc06da10 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.539+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc06dfd0 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.540+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5dc06e3b0 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5d0006420 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5c806c8f0 0x7fe5c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fe5d00adca0 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5c806c8f0 0x7fe5c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7fe5c8072ed0 0x7fe5c8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5d0008f00 con 0x7fe5c8072ed0 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7fe5d00b03a0 con 0x7fe5dc0d3490 2024-04-13T03:20:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5c806c8f0 0x7fe5c806edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006fd0 tx=0x7fe5cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.541+0000 7fe5e0bbd640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7fe5c8072ed0 0x7fe5c8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.542+0000 7fe5e0bbd640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7fe5c8072ed0 0x7fe5c8075310 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.543+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== osd.5 v2:172.21.15.114:6808/1239989021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5d0008f00 con 0x7fe5c8072ed0 2024-04-13T03:20:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.555+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 --> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe5dc068a10 con 0x7fe5c8072ed0 2024-04-13T03:20:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.556+0000 7fe5d8ff9640 1 -- 172.21.15.106:0/3007786370 <== osd.5 v2:172.21.15.114:6808/1239989021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe5dc068a10 con 0x7fe5c8072ed0 2024-04-13T03:20:01.557 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:01.557 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:01.557 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7fe5c8072ed0 msgr2=0x7fe5c8075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.114:6808/1239989021,v1:172.21.15.114:6809/1239989021] conn(0x7fe5c8072ed0 0x7fe5c8075310 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5c806c8f0 msgr2=0x7fe5c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe5c806c8f0 0x7fe5c806edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006fd0 tx=0x7fe5cc008040 comp rx=0 tx=0).stop 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 msgr2=0x7fe5dc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 --2- 172.21.15.106:0/3007786370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5dc0d3490 0x7fe5dc06cc70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00090b0 tx=0x7fe5d00046c0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e0bbd640 1 -- 172.21.15.106:0/3007786370 reap_dead start 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 shutdown_connections 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 >> 172.21.15.106:0/3007786370 conn(0x7fe5dc0b9760 msgr2=0x7fe5dc0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 shutdown_connections 2024-04-13T03:20:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.557+0000 7fe5e2620640 1 -- 172.21.15.106:0/3007786370 wait complete. 2024-04-13T03:20:01.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-13T03:20:01.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.667+0000 7ff3fb898640 1 Processor -- start 2024-04-13T03:20:01.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 -- start start 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f415e9b0 0x7ff3f40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 0x7ff3f40d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3f40b3d60 0x7ff3f40d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f40d5070 con 0x7ff3f40b3d60 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f40cfd20 con 0x7ff3f415ed90 2024-04-13T03:20:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.668+0000 7ff3fb898640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f4162af0 con 0x7ff3f415e9b0 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3f40b3d60 0x7ff3f40d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3f40b3d60 0x7ff3f40d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32812/0 (socket says 172.21.15.106:32812) 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 -- 172.21.15.106:0/4156560347 learned_addr learned my addr 172.21.15.106:0/4156560347 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f8e33640 1 --2- 172.21.15.106:0/4156560347 >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 0x7ff3f40d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9634640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f415e9b0 0x7ff3f40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 -- 172.21.15.106:0/4156560347 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f415e9b0 msgr2=0x7ff3f40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f415e9b0 0x7ff3f40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 -- 172.21.15.106:0/4156560347 >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 msgr2=0x7ff3f40d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/4156560347 >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 0x7ff3f40d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f9e35640 1 -- 172.21.15.106:0/4156560347 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f4162dd0 con 0x7ff3f40b3d60 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.669+0000 7ff3f8e33640 1 --2- 172.21.15.106:0/4156560347 >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 0x7ff3f40d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b3d60 0x7ff3f40d4b30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4162d00 tx=0x7ff3ec02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af6a0e13f9fac0f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3ebfff640 1 -- 172.21.15.106:0/4156560347 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec02caf0 con 0x7ff3f40b3d60 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3ebfff640 1 -- 172.21.15.106:0/4156560347 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec02cc90 con 0x7ff3f40b3d60 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3ebfff640 1 -- 172.21.15.106:0/4156560347 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec034610 con 0x7ff3f40b3d60 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 -- 172.21.15.106:0/4156560347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b3d60 msgr2=0x7ff3f40d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b3d60 0x7ff3f40d4b30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4162d00 tx=0x7ff3ec02bf50 comp rx=0 tx=0).stop 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 -- 172.21.15.106:0/4156560347 shutdown_connections 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b3d60 0x7ff3f40d4b30 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 --2- 172.21.15.106:0/4156560347 >> v2:172.21.15.114:3300/0 conn(0x7ff3f415ed90 0x7ff3f40d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 --2- 172.21.15.106:0/4156560347 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f415e9b0 0x7ff3f40d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 -- 172.21.15.106:0/4156560347 >> 172.21.15.106:0/4156560347 conn(0x7ff3f408bf10 msgr2=0x7ff3f40b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 -- 172.21.15.106:0/4156560347 shutdown_connections 2024-04-13T03:20:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.670+0000 7ff3fb898640 1 -- 172.21.15.106:0/4156560347 wait complete. 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 Processor -- start 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 -- start start 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f40b3d60 0x7ff3f41f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f415e9b0 0x7ff3f406bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4162680 con 0x7ff3f40b61e0 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4163390 con 0x7ff3f415e9b0 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3fb898640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f4162800 con 0x7ff3f40b3d60 2024-04-13T03:20:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3f9e35640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3f9e35640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32824/0 (socket says 172.21.15.106:32824) 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.672+0000 7ff3f9e35640 1 -- 172.21.15.106:0/3616108377 learned_addr learned my addr 172.21.15.106:0/3616108377 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 -- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f40b3d60 msgr2=0x7ff3f41f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f40b3d60 0x7ff3f41f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 -- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f415e9b0 msgr2=0x7ff3f406bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f415e9b0 0x7ff3f406bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 -- 172.21.15.106:0/3616108377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f415fc30 con 0x7ff3f40b61e0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3f9e35640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009f10 tx=0x7ff3ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec043020 con 0x7ff3f40b61e0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec03e450 con 0x7ff3f40b61e0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f415ff10 con 0x7ff3f40b61e0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec03e840 con 0x7ff3f40b61e0 2024-04-13T03:20:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.673+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41604d0 con 0x7ff3f40b61e0 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.674+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f4068a10 con 0x7ff3f40b61e0 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.674+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3ec046020 con 0x7ff3f40b61e0 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.675+0000 7ff3e9ffb640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.675+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7ff3ec0ae420 con 0x7ff3f40b61e0 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.675+0000 7ff3f9634640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.675+0000 7ff3f9634640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 0x7ff3d806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.679+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3ec04d030 con 0x7ff3f40b61e0 2024-04-13T03:20:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.853+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ff3f415f9e0 con 0x7ff3d806c8f0 2024-04-13T03:20:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.854+0000 7ff3e9ffb640 1 -- 172.21.15.106:0/3616108377 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff3f415f9e0 con 0x7ff3d806c8f0 2024-04-13T03:20:01.855 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 5 to deep-scrub 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 msgr2=0x7ff3d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 0x7ff3d806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e4008040 comp rx=0 tx=0).stop 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 msgr2=0x7ff3f40b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009f10 tx=0x7ff3ec004a40 comp rx=0 tx=0).stop 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 shutdown_connections 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7ff3d806c8f0 0x7ff3d806edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3f40b61e0 0x7ff3f40b6690 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f415e9b0 0x7ff3f406bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 --2- 172.21.15.106:0/3616108377 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7ff3f40b3d60 0x7ff3f41f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 >> 172.21.15.106:0/3616108377 conn(0x7ff3f408bf10 msgr2=0x7ff3f40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 shutdown_connections 2024-04-13T03:20:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.856+0000 7ff3fb898640 1 -- 172.21.15.106:0/3616108377 wait complete. 2024-04-13T03:20:01.868 INFO:tasks.ceph:Scrubbing osd.6 2024-04-13T03:20:01.868 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:01.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.971+0000 7f4f51e76640 1 Processor -- start 2024-04-13T03:20:01.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.971+0000 7f4f51e76640 1 -- start start 2024-04-13T03:20:01.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.971+0000 7f4f51e76640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f4c15ad10 0x7f4f4c06f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f51e76640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c15b0f0 con 0x7f4f4c06c170 2024-04-13T03:20:01.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f51e76640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c0b4d40 con 0x7f4f4c06f9a0 2024-04-13T03:20:01.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f51e76640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4c06e980 con 0x7f4f4c15ad10 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4f4c06c170 0x7f4f4c0cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60994/0 (socket says 172.21.15.106:60994) 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4bfff640 1 -- 172.21.15.106:0/1394327816 learned_addr learned my addr 172.21.15.106:0/1394327816 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252725556 0 0) 0x7f4f4c15b0f0 con 0x7f4f4c06c170 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f34003520 con 0x7f4f4c06c170 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4b7fe640 1 --2- 172.21.15.106:0/1394327816 >> v2:172.21.15.114:3300/0 conn(0x7f4f4c15ad10 0x7f4f4c06f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732415643 0 0) 0x7f4f34003520 con 0x7f4f4c06c170 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 >> v2:172.21.15.114:3300/0 conn(0x7f4f4c15ad10 msgr2=0x7f4f4c06f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 --2- 172.21.15.106:0/1394327816 >> v2:172.21.15.114:3300/0 conn(0x7f4f4c15ad10 0x7f4f4c06f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 >> v1:172.21.15.114:6789/0 conn(0x7f4f4c06f9a0 legacy=0x7f4f4c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c06ec60 con 0x7f4f4c06c170 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f4f440019f0 con 0x7f4f4c06c170 2024-04-13T03:20:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 >> v1:172.21.15.106:6789/0 conn(0x7f4f4c06c170 legacy=0x7f4f4c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f340051c0 0x7f4f3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06e980 con 0x7f4f34004800 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400c060 con 0x7f4f340051c0 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3400c260 con 0x7f4f340039b0 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4affd640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4b7fe640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.972+0000 7f4f4bfff640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f340051c0 0x7f4f3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 -- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f340039b0 msgr2=0x7f4f34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 -- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f340051c0 msgr2=0x7f4f3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f340051c0 0x7f4f3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4bfff640 1 -- 172.21.15.106:0/1394327816 reap_dead start 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 -- 172.21.15.106:0/1394327816 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f34003520 con 0x7f4f34004800 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.973+0000 7f4f4affd640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4f40001720 tx=0x7f4f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d36ba8f53659ee server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40001370 con 0x7f4f34004800 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f400052d0 con 0x7f4f34004800 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f4a7fc640 1 -- 172.21.15.106:0/1394327816 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40003830 con 0x7f4f34004800 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 -- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 msgr2=0x7f4f34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4f40001720 tx=0x7f4f34005830 comp rx=0 tx=0).stop 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 -- 172.21.15.106:0/1394327816 shutdown_connections 2024-04-13T03:20:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 --2- 172.21.15.106:0/1394327816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 -- 172.21.15.106:0/1394327816 >> 172.21.15.106:0/1394327816 conn(0x7f4f4c0d0370 msgr2=0x7f4f4c0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 -- 172.21.15.106:0/1394327816 shutdown_connections 2024-04-13T03:20:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.974+0000 7f4f51e76640 1 -- 172.21.15.106:0/1394327816 wait complete. 2024-04-13T03:20:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.975+0000 7f4f51e76640 1 Processor -- start 2024-04-13T03:20:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 -- start start 2024-04-13T03:20:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f34004800 0x7f4f4c0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f4c0ad8b0 0x7f4f4c0abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 0x7f4f4c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06f460 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c0b3dd0 con 0x7f4f4c0ad8b0 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f51e76640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c06f5e0 con 0x7f4f34004800 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 0x7f4f4c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 0x7f4f4c14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32842/0 (socket says 172.21.15.106:32842) 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.976+0000 7f4f4bfff640 1 -- 172.21.15.106:0/2454501581 learned_addr learned my addr 172.21.15.106:0/2454501581 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4affd640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f4c0ad8b0 0x7f4f4c0abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4b7fe640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f34004800 0x7f4f4c0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 -- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f34004800 msgr2=0x7f4f4c0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f4f34004800 0x7f4f4c0aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 -- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f4c0ad8b0 msgr2=0x7f4f4c0abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f4c0ad8b0 0x7f4f4c0abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0af330 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f4bfff640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 0x7f4f4c14f8d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000f80 tx=0x7f4f440079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44017020 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4400ab00 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c0add70 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440133f0 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.977+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c0ae330 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.978+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f4c068a10 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.978+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f4401a020 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f33fff640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3806c8f0 0x7f4f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f4f4408c070 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f4b7fe640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3806c8f0 0x7f4f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f33fff640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f4f38072ed0 0x7f4f38075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f4401de50 con 0x7f4f38072ed0 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f4f44093050 con 0x7f4f4c0ce9e0 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f4affd640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f4f38072ed0 0x7f4f38075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.979+0000 7f4f4b7fe640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3806c8f0 0x7f4f3806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00acc0 tx=0x7f4f3c00a2a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.980+0000 7f4f4affd640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f4f38072ed0 0x7f4f38075310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.980+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== osd.6 v2:172.21.15.114:6816/1761123940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f4401de50 con 0x7f4f38072ed0 2024-04-13T03:20:01.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.993+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 --> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4f4c070090 con 0x7f4f38072ed0 2024-04-13T03:20:01.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f33fff640 1 -- 172.21.15.106:0/2454501581 <== osd.6 v2:172.21.15.114:6816/1761123940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4f4c070090 con 0x7f4f38072ed0 2024-04-13T03:20:01.994 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:01.995 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:01.995 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f4f38072ed0 msgr2=0x7f4f38075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.114:6816/1761123940,v1:172.21.15.114:6817/1761123940] conn(0x7f4f38072ed0 0x7f4f38075310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3806c8f0 msgr2=0x7f4f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f4f3806c8f0 0x7f4f3806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00acc0 tx=0x7f4f3c00a2a0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 msgr2=0x7f4f4c14f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 --2- 172.21.15.106:0/2454501581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f4c0ce9e0 0x7f4f4c14f8d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000f80 tx=0x7f4f440079d0 comp rx=0 tx=0).stop 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f4bfff640 1 -- 172.21.15.106:0/2454501581 reap_dead start 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 shutdown_connections 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 >> 172.21.15.106:0/2454501581 conn(0x7f4f4c0d0370 msgr2=0x7f4f4c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 shutdown_connections 2024-04-13T03:20:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:01.994+0000 7f4f51e76640 1 -- 172.21.15.106:0/2454501581 wait complete. 2024-04-13T03:20:02.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-13T03:20:02.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.105+0000 7f25dbede640 1 Processor -- start 2024-04-13T03:20:02.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.105+0000 7f25dbede640 1 -- start start 2024-04-13T03:20:02.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25dbede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25d4072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25dbede640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25d406e020 0x7f25d4073350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25dbede640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d406c880 con 0x7f25d4145f60 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25dbede640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f25d406caf0 con 0x7f25d406e020 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25dbede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25d406cdd0 con 0x7f25d406db90 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25d9c7a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f25d4145f60 0x7f25d406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32768/0 (socket says 172.21.15.106:32768) 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25d9c7a640 1 -- 172.21.15.106:0/886104222 learned_addr learned my addr 172.21.15.106:0/886104222 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231803414 0 0) 0x7f25d406c880 con 0x7f25d4145f60 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25c0003520 con 0x7f25d4145f60 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25d9479640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25d4072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.106+0000 7f25da47b640 1 --2- 172.21.15.106:0/886104222 >> v2:172.21.15.114:3300/0 conn(0x7f25d406e020 0x7f25d4073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 273913444 0 0) 0x7f25c0003520 con 0x7f25d4145f60 2024-04-13T03:20:02.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 msgr2=0x7f25d4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25d4072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 >> v2:172.21.15.114:3300/0 conn(0x7f25d406e020 msgr2=0x7f25d4073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 --2- 172.21.15.106:0/886104222 >> v2:172.21.15.114:3300/0 conn(0x7f25d406e020 0x7f25d4073350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d406d0b0 con 0x7f25d4145f60 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2378869408 0 0) 0x7f25d0001670 con 0x7f25d4145f60 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 >> v1:172.21.15.106:6789/0 conn(0x7f25d4145f60 legacy=0x7f25d406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0003f30 0x7f25c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c0004d40 0x7f25c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25d406caf0 con 0x7f25c0004d40 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25d406cdd0 con 0x7f25c0003f30 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25c000b9c0 con 0x7f25d406db90 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c0004d40 0x7f25c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d9479640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0003f30 0x7f25c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25d9c7a640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 -- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 msgr2=0x7f25c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d406db90 0x7f25c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 -- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0003f30 msgr2=0x7f25c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0003f30 0x7f25c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.107+0000 7f25da47b640 1 -- 172.21.15.106:0/886104222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c0003520 con 0x7f25c0004d40 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25da47b640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c0004d40 0x7f25c00051c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009fc0 tx=0x7f25c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ce5b92ce8476ae server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc039070 con 0x7f25c0004d40 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc02cc40 con 0x7f25c0004d40 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25d8c78640 1 -- 172.21.15.106:0/886104222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc009c10 con 0x7f25c0004d40 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25dbede640 1 -- 172.21.15.106:0/886104222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c0004d40 msgr2=0x7f25c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25dbede640 1 --2- 172.21.15.106:0/886104222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c0004d40 0x7f25c00051c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009fc0 tx=0x7f25c0005830 comp rx=0 tx=0).stop 2024-04-13T03:20:02.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25da47b640 1 -- 172.21.15.106:0/886104222 reap_dead start 2024-04-13T03:20:02.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25dbede640 1 -- 172.21.15.106:0/886104222 shutdown_connections 2024-04-13T03:20:02.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.108+0000 7f25dbede640 1 -- 172.21.15.106:0/886104222 >> 172.21.15.106:0/886104222 conn(0x7f25d406f0d0 msgr2=0x7f25d406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.109+0000 7f25dbede640 1 -- 172.21.15.106:0/886104222 shutdown_connections 2024-04-13T03:20:02.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.109+0000 7f25dbede640 1 -- 172.21.15.106:0/886104222 wait complete. 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 Processor -- start 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 -- start start 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0004d40 0x7f25d40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d40c6d10 0x7f25d415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40cdaa0 con 0x7f25d40b5d30 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4146340 con 0x7f25c0004d40 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.110+0000 7f25dbede640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f25d40cdc20 con 0x7f25d40c6d10 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32864/0 (socket says 172.21.15.106:32864) 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 -- 172.21.15.106:0/1875723878 learned_addr learned my addr 172.21.15.106:0/1875723878 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25d9c7a640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0004d40 0x7f25d40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25d9479640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d40c6d10 0x7f25d415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 -- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d40c6d10 msgr2=0x7f25d415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d40c6d10 0x7f25d415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 -- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0004d40 msgr2=0x7f25d40c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0004d40 0x7f25d40c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 -- 172.21.15.106:0/1875723878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d40b4fe0 con 0x7f25d40b5d30 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25da47b640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00e230 tx=0x7f25cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc039030 con 0x7f25d40b5d30 2024-04-13T03:20:02.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc03a040 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc009c10 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d41425d0 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.111+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d40b5530 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.112+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d415cbb0 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.113+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25cc045030 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.113+0000 7f25ca7fc640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.113+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f25cc0ad460 con 0x7f25d40b5d30 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.113+0000 7f25d9c7a640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.113+0000 7f25d9c7a640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001050 tx=0x7f25d0007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.117+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25d415cbb0 con 0x7f25d40b5d30 2024-04-13T03:20:02.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.293+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f25d4068a10 con 0x7f25b806c8f0 2024-04-13T03:20:02.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.294+0000 7f25ca7fc640 1 -- 172.21.15.106:0/1875723878 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f25d4068a10 con 0x7f25b806c8f0 2024-04-13T03:20:02.295 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 6 to deep-scrub 2024-04-13T03:20:02.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 msgr2=0x7f25b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001050 tx=0x7f25d0007970 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 msgr2=0x7f25d40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00e230 tx=0x7f25cc009750 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 shutdown_connections 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f25d40c6d10 0x7f25d415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25d40b5d30 0x7f25d40b61b0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 --2- 172.21.15.106:0/1875723878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25c0004d40 0x7f25d40c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 >> 172.21.15.106:0/1875723878 conn(0x7f25d406f0d0 msgr2=0x7f25d40b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 shutdown_connections 2024-04-13T03:20:02.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.296+0000 7f25dbede640 1 -- 172.21.15.106:0/1875723878 wait complete. 2024-04-13T03:20:02.307 INFO:tasks.ceph:Scrubbing osd.7 2024-04-13T03:20:02.307 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-13T03:20:02.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.411+0000 7fe178ade640 1 Processor -- start 2024-04-13T03:20:02.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.411+0000 7fe178ade640 1 -- start start 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe178ade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe17415e890 0x7fe174162470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe178ade640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1740ce100 0x7fe174070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe178ade640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17406db70 con 0x7fe1740b4510 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe178ade640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe17406c380 con 0x7fe1740ce100 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe178ade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe174162ce0 con 0x7fe17415e890 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1740ce100 0x7fe174070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1740ce100 0x7fe174070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32878/0 (socket says 172.21.15.106:32878) 2024-04-13T03:20:02.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 -- 172.21.15.106:0/4164821436 learned_addr learned my addr 172.21.15.106:0/4164821436 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 -- 172.21.15.106:0/4164821436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe17415e890 msgr2=0x7fe174162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 --2- 172.21.15.106:0/4164821436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe17415e890 0x7fe174162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 -- 172.21.15.106:0/4164821436 >> v1:172.21.15.114:6789/0 conn(0x7fe1740b4510 legacy=0x7fe1740b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.412+0000 7fe169d9b640 1 -- 172.21.15.106:0/4164821436 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174162fc0 con 0x7fe1740ce100 2024-04-13T03:20:02.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.413+0000 7fe169d9b640 1 --2- 172.21.15.106:0/4164821436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1740ce100 0x7fe174070730 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7d16305c860d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.413+0000 7fe171d9b640 1 -- 172.21.15.106:0/4164821436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c034070 con 0x7fe1740ce100 2024-04-13T03:20:02.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.413+0000 7fe171d9b640 1 -- 172.21.15.106:0/4164821436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15c030840 con 0x7fe1740ce100 2024-04-13T03:20:02.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.413+0000 7fe171d9b640 1 -- 172.21.15.106:0/4164821436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c030c10 con 0x7fe1740ce100 2024-04-13T03:20:02.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 -- 172.21.15.106:0/4164821436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1740ce100 msgr2=0x7fe174070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 --2- 172.21.15.106:0/4164821436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1740ce100 0x7fe174070730 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c02b980 comp rx=0 tx=0).stop 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 -- 172.21.15.106:0/4164821436 shutdown_connections 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 --2- 172.21.15.106:0/4164821436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1740ce100 0x7fe174070730 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 --2- 172.21.15.106:0/4164821436 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe17415e890 0x7fe174162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 -- 172.21.15.106:0/4164821436 >> 172.21.15.106:0/4164821436 conn(0x7fe17406f290 msgr2=0x7fe17406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 -- 172.21.15.106:0/4164821436 shutdown_connections 2024-04-13T03:20:02.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.414+0000 7fe178ade640 1 -- 172.21.15.106:0/4164821436 wait complete. 2024-04-13T03:20:02.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.415+0000 7fe178ade640 1 Processor -- start 2024-04-13T03:20:02.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.415+0000 7fe178ade640 1 -- start start 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1740b4510 0x7fe17414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1740ce100 0x7fe174155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 0x7fe174156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe17415ec70 con 0x7fe17415e890 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe17406c640 con 0x7fe1740ce100 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe178ade640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7fe1741629b0 con 0x7fe1740b4510 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 0x7fe174156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 0x7fe174156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32892/0 (socket says 172.21.15.106:32892) 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 -- 172.21.15.106:0/2123256567 learned_addr learned my addr 172.21.15.106:0/2123256567 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe17259c640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1740b4510 0x7fe17414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe169d9b640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1740ce100 0x7fe174155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 -- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1740b4510 msgr2=0x7fe17414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7fe1740b4510 0x7fe17414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 -- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1740ce100 msgr2=0x7fe174155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1740ce100 0x7fe174155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17414cbf0 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.416+0000 7fe172d9d640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 0x7fe174156060 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c780 tx=0x7fe16c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.417+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c017070 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.417+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c003da0 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.417+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1741f5720 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.417+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0052b0 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.417+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17415c200 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.418+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe17415c8c0 con 0x7fe17415e890 2024-04-13T03:20:02.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.418+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe16c01a070 con 0x7fe17415e890 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe16b7fe640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe15406c920 0x7fe15406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fe16c08afe0 con 0x7fe17415e890 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe17259c640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe15406c920 0x7fe15406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe16b7fe640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7fe154072f00 0x7fe154075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe154075a10 con 0x7fe154072f00 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7fe16c092050 con 0x7fe17415e890 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe17259c640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe15406c920 0x7fe15406ede0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe1640098e0 tx=0x7fe164009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.419+0000 7fe169d9b640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7fe154072f00 0x7fe154075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.420+0000 7fe169d9b640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7fe154072f00 0x7fe154075340 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.420+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== osd.7 v2:172.21.15.114:6824/300957724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe154075a10 con 0x7fe154072f00 2024-04-13T03:20:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.433+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 --> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe174068a10 con 0x7fe154072f00 2024-04-13T03:20:02.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe16b7fe640 1 -- 172.21.15.106:0/2123256567 <== osd.7 v2:172.21.15.114:6824/300957724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe174068a10 con 0x7fe154072f00 2024-04-13T03:20:02.434 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-13T03:20:02.435 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-13T03:20:02.435 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7fe154072f00 msgr2=0x7fe154075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.114:6824/300957724,v1:172.21.15.114:6825/300957724] conn(0x7fe154072f00 0x7fe154075340 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe15406c920 msgr2=0x7fe15406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7fe15406c920 0x7fe15406ede0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe1640098e0 tx=0x7fe164009060 comp rx=0 tx=0).stop 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 msgr2=0x7fe174156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 --2- 172.21.15.106:0/2123256567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe17415e890 0x7fe174156060 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c780 tx=0x7fe16c00cc40 comp rx=0 tx=0).stop 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe172d9d640 1 -- 172.21.15.106:0/2123256567 reap_dead start 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 shutdown_connections 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 >> 172.21.15.106:0/2123256567 conn(0x7fe17406f290 msgr2=0x7fe1740b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 shutdown_connections 2024-04-13T03:20:02.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.434+0000 7fe178ade640 1 -- 172.21.15.106:0/2123256567 wait complete. 2024-04-13T03:20:02.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-13T03:20:02.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.546+0000 7f3077355640 1 Processor -- start 2024-04-13T03:20:02.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.546+0000 7f3077355640 1 -- start start 2024-04-13T03:20:02.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f3077355640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f30700cf480 0x7f307015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f3077355640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30700d3220 0x7f30700d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f3077355640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307015f380 con 0x7f30700ba170 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f3077355640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f30700d2d90 con 0x7f30700d3220 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f3077355640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f307015f600 con 0x7f30700cf480 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30700d3220 0x7f30700d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30748f0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f30700ba170 0x7f30700cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32782/0 (socket says 172.21.15.106:32782) 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30700d3220 0x7f30700d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32898/0 (socket says 172.21.15.106:32898) 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30748f0640 1 -- 172.21.15.106:0/498746296 learned_addr learned my addr 172.21.15.106:0/498746296 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30750f1640 1 --2- 172.21.15.106:0/498746296 >> v2:172.21.15.114:3300/0 conn(0x7f30700cf480 0x7f307015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 -- 172.21.15.106:0/498746296 >> v2:172.21.15.114:3300/0 conn(0x7f30700cf480 msgr2=0x7f307015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 --2- 172.21.15.106:0/498746296 >> v2:172.21.15.114:3300/0 conn(0x7f30700cf480 0x7f307015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 -- 172.21.15.106:0/498746296 >> v1:172.21.15.106:6789/0 conn(0x7f30700ba170 legacy=0x7f30700cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.547+0000 7f30758f2640 1 -- 172.21.15.106:0/498746296 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307015f8e0 con 0x7f30700d3220 2024-04-13T03:20:02.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f30758f2640 1 --2- 172.21.15.106:0/498746296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700d3220 0x7f30700d36d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f307006e580 tx=0x7f306802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abecc2ac5a1148e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3067fff640 1 -- 172.21.15.106:0/498746296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068039070 con 0x7f30700d3220 2024-04-13T03:20:02.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3067fff640 1 -- 172.21.15.106:0/498746296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306802cc80 con 0x7f30700d3220 2024-04-13T03:20:02.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3067fff640 1 -- 172.21.15.106:0/498746296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068034660 con 0x7f30700d3220 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 -- 172.21.15.106:0/498746296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700d3220 msgr2=0x7f30700d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 --2- 172.21.15.106:0/498746296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700d3220 0x7f30700d36d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f307006e580 tx=0x7f306802bed0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 -- 172.21.15.106:0/498746296 shutdown_connections 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 --2- 172.21.15.106:0/498746296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700d3220 0x7f30700d36d0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 --2- 172.21.15.106:0/498746296 >> v2:172.21.15.114:3300/0 conn(0x7f30700cf480 0x7f307015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.548+0000 7f3077355640 1 -- 172.21.15.106:0/498746296 >> 172.21.15.106:0/498746296 conn(0x7f30700d47d0 msgr2=0x7f30700d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.549+0000 7f3077355640 1 -- 172.21.15.106:0/498746296 shutdown_connections 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.549+0000 7f3077355640 1 -- 172.21.15.106:0/498746296 wait complete. 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 Processor -- start 2024-04-13T03:20:02.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 -- start start 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30700cf480 0x7f30700ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30700d3220 0x7f30700ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30701592f0 con 0x7f30700ba170 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3070159170 con 0x7f30700d3220 2024-04-13T03:20:02.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f3077355640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f30700d3c10 con 0x7f30700cf480 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.550+0000 7f30750f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32900/0 (socket says 172.21.15.106:32900) 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 -- 172.21.15.106:0/3993869093 learned_addr learned my addr 172.21.15.106:0/3993869093 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 -- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30700cf480 msgr2=0x7f30700ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30700cf480 0x7f30700ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 -- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30700d3220 msgr2=0x7f30700ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30700d3220 0x7f30700ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 -- 172.21.15.106:0/3993869093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30701e7800 con 0x7f30700ba170 2024-04-13T03:20:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f30750f1640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d830 tx=0x7f306c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00b6c0 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c019070 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30701e7ae0 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00ba80 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.551+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30701e80a0 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.552+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3070068a10 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.552+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f306c01e460 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.553+0000 7f3065ffb640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.553+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f306c08c240 con 0x7f30700ba170 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.553+0000 7f30748f0640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.553+0000 7f30748f0640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 0x7f305406edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f30700ceb40 tx=0x7f3060007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.557+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f306c05a6a0 con 0x7f30700ba170 2024-04-13T03:20:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.731+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f3070156e50 con 0x7f305406c8f0 2024-04-13T03:20:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.732+0000 7f3065ffb640 1 -- 172.21.15.106:0/3993869093 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3070156e50 con 0x7f305406c8f0 2024-04-13T03:20:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 7 to deep-scrub 2024-04-13T03:20:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.733+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 msgr2=0x7f305406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.733+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 0x7f305406edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f30700ceb40 tx=0x7f3060007a40 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 msgr2=0x7f30701e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d830 tx=0x7f306c00dcf0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 shutdown_connections 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f30700cf480 0x7f30700ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30700d3220 0x7f30700ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 --2- 172.21.15.106:0/3993869093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30700ba170 0x7f30701e88a0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 >> 172.21.15.106:0/3993869093 conn(0x7f30700d47d0 msgr2=0x7f30701617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 shutdown_connections 2024-04-13T03:20:02.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.734+0000 7f3077355640 1 -- 172.21.15.106:0/3993869093 wait complete. 2024-04-13T03:20:02.745 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:20:02.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.844+0000 7f70f342d640 1 Processor -- start 2024-04-13T03:20:02.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 -- start start 2024-04-13T03:20:02.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 0x7f70ec0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70ec06e170 0x7f70ec06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70ec0ce870 con 0x7f70ec06cb80 2024-04-13T03:20:02.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f70ec072de0 con 0x7f70ec06e170 2024-04-13T03:20:02.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.845+0000 7f70f342d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f70ec073090 con 0x7f70ec0cdf50 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70ec06e170 0x7f70ec06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70ec06e170 0x7f70ec06e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32904/0 (socket says 172.21.15.106:32904) 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 -- 172.21.15.106:0/4151217362 learned_addr learned my addr 172.21.15.106:0/4151217362 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f11c9640 1 --2- 172.21.15.106:0/4151217362 >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 0x7f70ec0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 -- 172.21.15.106:0/4151217362 >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 msgr2=0x7f70ec0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 --2- 172.21.15.106:0/4151217362 >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 0x7f70ec0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 -- 172.21.15.106:0/4151217362 >> v1:172.21.15.114:6789/0 conn(0x7f70ec06cb80 legacy=0x7f70ec0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f19ca640 1 -- 172.21.15.106:0/4151217362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec073370 con 0x7f70ec06e170 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.846+0000 7f70f11c9640 1 --2- 172.21.15.106:0/4151217362 >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 0x7f70ec0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f19ca640 1 --2- 172.21.15.106:0/4151217362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06e170 0x7f70ec06e5e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f70ec06d680 tx=0x7f70dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405249c7b38bb2be server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70ebfff640 1 -- 172.21.15.106:0/4151217362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc02e070 con 0x7f70ec06e170 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70ebfff640 1 -- 172.21.15.106:0/4151217362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc037070 con 0x7f70ec06e170 2024-04-13T03:20:02.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70ebfff640 1 -- 172.21.15.106:0/4151217362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc031a00 con 0x7f70ec06e170 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 -- 172.21.15.106:0/4151217362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06e170 msgr2=0x7f70ec06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 --2- 172.21.15.106:0/4151217362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06e170 0x7f70ec06e5e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f70ec06d680 tx=0x7f70dc02b880 comp rx=0 tx=0).stop 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 -- 172.21.15.106:0/4151217362 shutdown_connections 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 --2- 172.21.15.106:0/4151217362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06e170 0x7f70ec06e5e0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 --2- 172.21.15.106:0/4151217362 >> v2:172.21.15.114:3300/0 conn(0x7f70ec0cdf50 0x7f70ec0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 -- 172.21.15.106:0/4151217362 >> 172.21.15.106:0/4151217362 conn(0x7f70ec0b1f10 msgr2=0x7f70ec0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 -- 172.21.15.106:0/4151217362 shutdown_connections 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.847+0000 7f70f342d640 1 -- 172.21.15.106:0/4151217362 wait complete. 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 Processor -- start 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 -- start start 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 0x7f70ec0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 0x7f70ec1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec0997b0 con 0x7f70ec06cb80 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec073930 con 0x7f70ec0cdf50 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f342d640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f70ec099930 con 0x7f70ec06e170 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f11c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f11c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32912/0 (socket says 172.21.15.106:32912) 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f11c9640 1 -- 172.21.15.106:0/56669152 learned_addr learned my addr 172.21.15.106:0/56669152 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f09c8640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 0x7f70ec0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.849+0000 7f70f19ca640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 0x7f70ec1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 -- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 msgr2=0x7f70ec0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 0x7f70ec0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 -- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 msgr2=0x7f70ec1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 0x7f70ec1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:02.858 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 -- 172.21.15.106:0/56669152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec1f5250 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f09c8640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 0x7f70ec0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f19ca640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 0x7f70ec1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f11c9640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ce30 tx=0x7f70e400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400f890 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e400fa30 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec1f5530 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400fe00 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.850+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec156e30 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.851+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ec068a10 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.852+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70e4018400 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.852+0000 7f70e9ffb640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 0x7f70d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.852+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f70e408c870 con 0x7f70ec06cb80 2024-04-13T03:20:02.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.852+0000 7f70f09c8640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 0x7f70d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:02.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.852+0000 7f70f09c8640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 0x7f70d806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f70ec15c840 tx=0x7f70e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:02.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:02.856+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70e4014030 con 0x7f70ec06cb80 2024-04-13T03:20:03.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.027+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70ec1f5700 con 0x7f70d806c8f0 2024-04-13T03:20:03.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.027+0000 7f70e9ffb640 1 -- 172.21.15.106:0/56669152 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13390 (secure 0 0 0) 0x7f70ec1f5700 con 0x7f70d806c8f0 2024-04-13T03:20:03.028 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:20:03.029 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 msgr2=0x7f70d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 0x7f70d806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f70ec15c840 tx=0x7f70e0009040 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 msgr2=0x7f70ec0c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ce30 tx=0x7f70e400a650 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 shutdown_connections 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f70d806c8f0 0x7f70d806edb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f70ec06e170 0x7f70ec0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70ec0cdf50 0x7f70ec1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 --2- 172.21.15.106:0/56669152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70ec06cb80 0x7f70ec0c6200 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 >> 172.21.15.106:0/56669152 conn(0x7f70ec0b1f10 msgr2=0x7f70ec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:03.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 shutdown_connections 2024-04-13T03:20:03.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:03.030+0000 7f70f342d640 1 -- 172.21.15.106:0/56669152 wait complete. 2024-04-13T03:20:03.042 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-04-13T03:20:02.504420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262944,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711776,"statfs":{"total":773094113280,"available":772824858624,"internally_reserved":0,"allocated":13803520,"data_stored":7082792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.391567"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":368,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-13T03:19:55.929874+0000","last_change":"2024-04-13T03:09:13.415254+0000","last_active":"2024-04-13T03:19:55.929874+0000","last_peered":"2024-04-13T03:19:55.929874+0000","last_clean":"2024-04-13T03:19:55.929874+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:19:55.929874+0000","last_undegraded":"2024-04-13T03:19:55.929874+0000","last_fullsized":"2024-04-13T03:19:55.929874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T03:09:12.404710+0000","last_clean_scrub_stamp":"2024-04-13T03:09:12.404710+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:47:43.578742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2170880,"data_stored":1328125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1328125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:20:03.045 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-13T03:09:12.404710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=13, tm_hour=3, tm_min=9, tm_sec=12, tm_wday=5, tm_yday=104, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=13, tm_hour=3, tm_min=19, tm_sec=58, tm_wday=5, tm_yday=104, tm_isdst=0) 2024-04-13T03:20:03.045 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-13T03:20:23.046 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T03:20:23.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.157+0000 7f05a8636640 1 Processor -- start 2024-04-13T03:20:23.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.157+0000 7f05a8636640 1 -- start start 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a8636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a00cdf50 0x7f05a00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a8636640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f05a006cb10 0x7f05a0072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a8636640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a00ce870 con 0x7f05a006b8e0 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a8636640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f05a00998d0 con 0x7f05a006cb10 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a8636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f05a0073020 con 0x7f05a00cdf50 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a6bd3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f05a006b8e0 0x7f05a006bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42588/0 (socket says 172.21.15.106:42588) 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a6bd3640 1 -- 172.21.15.106:0/129597732 learned_addr learned my addr 172.21.15.106:0/129597732 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a63d2640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a00cdf50 0x7f05a00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.158+0000 7f05a5bd1640 1 --2- 172.21.15.106:0/129597732 >> v2:172.21.15.114:3300/0 conn(0x7f05a006cb10 0x7f05a0072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a53d0640 1 -- 172.21.15.106:0/129597732 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041024423 0 0) 0x7f05a00ce870 con 0x7f05a006b8e0 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a53d0640 1 -- 172.21.15.106:0/129597732 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f058c003520 con 0x7f05a006b8e0 2024-04-13T03:20:23.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a5bd1640 1 -- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a00cdf50 msgr2=0x7f05a00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:23.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a5bd1640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a00cdf50 0x7f05a00ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a5bd1640 1 -- 172.21.15.106:0/129597732 >> v1:172.21.15.106:6789/0 conn(0x7f05a006b8e0 legacy=0x7f05a006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.159+0000 7f05a5bd1640 1 -- 172.21.15.106:0/129597732 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a0073300 con 0x7f05a006cb10 2024-04-13T03:20:23.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a5bd1640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006cb10 0x7f05a0072720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f05900097a0 tx=0x7f059002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53aa23e63fcaf3f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a53d0640 1 -- 172.21.15.106:0/129597732 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590035020 con 0x7f05a006cb10 2024-04-13T03:20:23.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a53d0640 1 -- 172.21.15.106:0/129597732 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590031830 con 0x7f05a006cb10 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a53d0640 1 -- 172.21.15.106:0/129597732 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590031be0 con 0x7f05a006cb10 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a8636640 1 -- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006cb10 msgr2=0x7f05a0072720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.160+0000 7f05a8636640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006cb10 0x7f05a0072720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f05900097a0 tx=0x7f059002b8d0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 -- 172.21.15.106:0/129597732 shutdown_connections 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006cb10 0x7f05a0072720 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 --2- 172.21.15.106:0/129597732 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a00cdf50 0x7f05a00ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 -- 172.21.15.106:0/129597732 >> 172.21.15.106:0/129597732 conn(0x7f05a00b1f10 msgr2=0x7f05a00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 -- 172.21.15.106:0/129597732 shutdown_connections 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.161+0000 7f05a8636640 1 -- 172.21.15.106:0/129597732 wait complete. 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.162+0000 7f05a8636640 1 Processor -- start 2024-04-13T03:20:23.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.162+0000 7f05a8636640 1 -- start start 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006b8e0 0x7f05a01f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a006cb10 0x7f05a00c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0072c60 con 0x7f05a00cdf50 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05a00738c0 con 0x7f05a006b8e0 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f05a0072de0 con 0x7f05a006cb10 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44616/0 (socket says 172.21.15.106:44616) 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 -- 172.21.15.106:0/1260547578 learned_addr learned my addr 172.21.15.106:0/1260547578 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a5bd1640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a006cb10 0x7f05a00c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a63d2640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006b8e0 0x7f05a01f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 -- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a006cb10 msgr2=0x7f05a00c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a006cb10 0x7f05a00c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 -- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006b8e0 msgr2=0x7f05a01f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006b8e0 0x7f05a01f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 -- 172.21.15.106:0/1260547578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a00c6340 con 0x7f05a00cdf50 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a6bd3640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f059800caa0 tx=0x7f059800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598017070 con 0x7f05a00cdf50 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059800ac00 con 0x7f05a00cdf50 2024-04-13T03:20:23.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a00ad430 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598012400 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.163+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a00ad9f0 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.165+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a0068a10 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.165+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f059801a070 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.165+0000 7f0596ffd640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 0x7f058406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.165+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f059808b850 con 0x7f05a00cdf50 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.165+0000 7f05a63d2640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 0x7f058406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.166+0000 7f05a63d2640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 0x7f058406edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f05a01571f0 tx=0x7f059c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.170+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05a00c6340 con 0x7f05a00cdf50 2024-04-13T03:20:23.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.342+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 --> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05a00b25d0 con 0x7f058406c8f0 2024-04-13T03:20:23.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.343+0000 7f0596ffd640 1 -- 172.21.15.106:0/1260547578 <== mgr.4104 v2:172.21.15.106:6832/3543688068 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f05a00b25d0 con 0x7f058406c8f0 2024-04-13T03:20:23.344 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:20:23.344 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 msgr2=0x7f058406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 0x7f058406edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f05a01571f0 tx=0x7f059c008040 comp rx=0 tx=0).stop 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 msgr2=0x7f05a00c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f059800caa0 tx=0x7f059800cf60 comp rx=0 tx=0).stop 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 shutdown_connections 2024-04-13T03:20:23.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f058406c8f0 0x7f058406edb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f05a006cb10 0x7f05a00c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a00cdf50 0x7f05a00c9e50 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 --2- 172.21.15.106:0/1260547578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05a006b8e0 0x7f05a01f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 >> 172.21.15.106:0/1260547578 conn(0x7f05a00b1f10 msgr2=0x7f05a006e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 shutdown_connections 2024-04-13T03:20:23.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.345+0000 7f05a8636640 1 -- 172.21.15.106:0/1260547578 wait complete. 2024-04-13T03:20:23.356 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-04-13T03:20:22.508808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262932,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711788,"statfs":{"total":773094113280,"available":772824870912,"internally_reserved":0,"allocated":13803520,"data_stored":7082792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002712"},"pg_stats":[{"pgid":"1.0","version":"126'119","reported_seq":379,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-13T03:20:03.052461+0000","last_change":"2024-04-13T03:20:03.052496+0000","last_active":"2024-04-13T03:20:03.052461+0000","last_peered":"2024-04-13T03:20:03.052461+0000","last_clean":"2024-04-13T03:20:03.052461+0000","last_became_active":"2024-04-13T03:09:13.414394+0000","last_became_peered":"2024-04-13T03:09:13.414394+0000","last_unstale":"2024-04-13T03:20:03.052461+0000","last_undegraded":"2024-04-13T03:20:03.052461+0000","last_fullsized":"2024-04-13T03:20:03.052461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'119","last_scrub_stamp":"2024-04-13T03:20:03.052380+0000","last_deep_scrub":"126'119","last_deep_scrub_stamp":"2024-04-13T03:20:03.052380+0000","last_clean_scrub_stamp":"2024-04-13T03:20:03.052380+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:16:19.094597+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1328125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1328125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T03:20:23.357 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-13T03:20:23.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.457+0000 7f6d17d11640 1 Processor -- start 2024-04-13T03:20:23.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.457+0000 7f6d17d11640 1 -- start start 2024-04-13T03:20:23.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.457+0000 7f6d17d11640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d10072d40 0x7f6d1006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.457+0000 7f6d17d11640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d100d02b0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d17d11640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1006ba60 con 0x7f6d100cf9b0 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d17d11640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6d100d0590 con 0x7f6d10072d40 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d15aad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d10072d40 0x7f6d1006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d152ac640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6d1015fbb0 0x7f6d10073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42594/0 (socket says 172.21.15.106:42594) 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d152ac640 1 -- 172.21.15.106:0/3457301156 learned_addr learned my addr 172.21.15.106:0/3457301156 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d15aad640 1 -- 172.21.15.106:0/3457301156 >> v1:172.21.15.114:6789/0 conn(0x7f6d100cf9b0 legacy=0x7f6d100ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d15aad640 1 -- 172.21.15.106:0/3457301156 >> v1:172.21.15.106:6789/0 conn(0x7f6d1015fbb0 legacy=0x7f6d10073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d14aab640 1 -- 172.21.15.106:0/3457301156 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215075478 0 0) 0x7f6d100d02b0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.458+0000 7f6d15aad640 1 -- 172.21.15.106:0/3457301156 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1006bd40 con 0x7f6d10072d40 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d15aad640 1 --2- 172.21.15.106:0/3457301156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d10072d40 0x7f6d1006c540 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0097d0 tx=0x7f6d0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4484d4b9c7d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d14aab640 1 -- 172.21.15.106:0/3457301156 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c035020 con 0x7f6d10072d40 2024-04-13T03:20:23.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d14aab640 1 -- 172.21.15.106:0/3457301156 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c031830 con 0x7f6d10072d40 2024-04-13T03:20:23.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d14aab640 1 -- 172.21.15.106:0/3457301156 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c031c00 con 0x7f6d10072d40 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d17d11640 1 -- 172.21.15.106:0/3457301156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d10072d40 msgr2=0x7f6d1006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d17d11640 1 --2- 172.21.15.106:0/3457301156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d10072d40 0x7f6d1006c540 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0097d0 tx=0x7f6d0c02b8d0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d17d11640 1 -- 172.21.15.106:0/3457301156 shutdown_connections 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d17d11640 1 --2- 172.21.15.106:0/3457301156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d10072d40 0x7f6d1006c540 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.459+0000 7f6d17d11640 1 -- 172.21.15.106:0/3457301156 >> 172.21.15.106:0/3457301156 conn(0x7f6d10072220 msgr2=0x7f6d10072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.460+0000 7f6d17d11640 1 -- 172.21.15.106:0/3457301156 shutdown_connections 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.460+0000 7f6d17d11640 1 -- 172.21.15.106:0/3457301156 wait complete. 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 Processor -- start 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 -- start start 2024-04-13T03:20:23.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d10072d40 0x7f6d100cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 0x7f6d10146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d100cfe00 con 0x7f6d1015fbb0 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101486b0 con 0x7f6d100cf9b0 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d17d11640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7f6d100cff80 con 0x7f6d10072d40 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d162ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d162ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44640/0 (socket says 172.21.15.106:44640) 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.461+0000 7f6d162ae640 1 -- 172.21.15.106:0/796735549 learned_addr learned my addr 172.21.15.106:0/796735549 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:23.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d152ac640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 0x7f6d10146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d15aad640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d10072d40 0x7f6d100cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 -- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d10072d40 msgr2=0x7f6d100cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d10072d40 0x7f6d100cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 -- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 msgr2=0x7f6d10146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 0x7f6d10146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 -- 172.21.15.106:0/796735549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d10134b60 con 0x7f6d1015fbb0 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d152ac640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 0x7f6d10146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T03:20:23.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d162ae640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c7b0 tx=0x7f6d0800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08004b60 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d08004d00 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d10133ac0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800aa90 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.462+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d10134080 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.463+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d100cc930 con 0x7f6d1015fbb0 2024-04-13T03:20:23.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.463+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d0800ac30 con 0x7f6d1015fbb0 2024-04-13T03:20:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.464+0000 7f6d067fc640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 0x7f6cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.464+0000 7f6d15aad640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 0x7f6cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.464+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f6d0808c000 con 0x7f6d1015fbb0 2024-04-13T03:20:23.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.464+0000 7f6d15aad640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 0x7f6cf406edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c036000 tx=0x7f6d0c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.468+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d0805a4f0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.645+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f6d100732a0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.648+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f6d08059df0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.650+0000 7f6d067fc640 1 -- 172.21.15.106:0/796735549 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6d0805e1a0 con 0x7f6d1015fbb0 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 msgr2=0x7f6cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 0x7f6cf406edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c036000 tx=0x7f6d0c03c040 comp rx=0 tx=0).stop 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 msgr2=0x7f6d10146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c7b0 tx=0x7f6d0800cc70 comp rx=0 tx=0).stop 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 shutdown_connections 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7f6cf406c8f0 0x7f6cf406edb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d1015fbb0 0x7f6d10146b90 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d100cf9b0 0x7f6d10146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 --2- 172.21.15.106:0/796735549 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6d10072d40 0x7f6d100cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 >> 172.21.15.106:0/796735549 conn(0x7f6d10072220 msgr2=0x7f6d1006c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 shutdown_connections 2024-04-13T03:20:23.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.651+0000 7f6d17d11640 1 -- 172.21.15.106:0/796735549 wait complete. 2024-04-13T03:20:23.662 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T03:20:23.663 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T03:20:23.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.762+0000 7efcf15f7640 1 Processor -- start 2024-04-13T03:20:23.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.762+0000 7efcf15f7640 1 -- start start 2024-04-13T03:20:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcf15f7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcec0ce580 0x7efcec0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcf15f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcf15f7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcec0b5d60 con 0x7efcec0b5940 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcf15f7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efcec0d1be0 con 0x7efcec0ce580 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcf15f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efcec0d0b40 con 0x7efcec0d06c0 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcec0ce580 0x7efcec0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efcec0b5940 0x7efcec1627f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42604/0 (socket says 172.21.15.106:42604) 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceb7fe640 1 -- 172.21.15.106:0/981516958 learned_addr learned my addr 172.21.15.106:0/981516958 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efcea7fc640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efce9ffb640 1 -- 172.21.15.106:0/981516958 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514789268 0 0) 0x7efcec0b5d60 con 0x7efcec0b5940 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efce9ffb640 1 -- 172.21.15.106:0/981516958 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd4003520 con 0x7efcec0b5940 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceaffd640 1 -- 172.21.15.106:0/981516958 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 msgr2=0x7efcec0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceaffd640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceaffd640 1 -- 172.21.15.106:0/981516958 >> v1:172.21.15.106:6789/0 conn(0x7efcec0b5940 legacy=0x7efcec1627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.763+0000 7efceaffd640 1 -- 172.21.15.106:0/981516958 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec0d0e20 con 0x7efcec0ce580 2024-04-13T03:20:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.764+0000 7efceaffd640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec0d0180 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009d90 tx=0x7efcdc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766f1ebd18db9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.764+0000 7efce9ffb640 1 -- 172.21.15.106:0/981516958 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc039070 con 0x7efcec0ce580 2024-04-13T03:20:23.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.764+0000 7efce9ffb640 1 -- 172.21.15.106:0/981516958 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efcdc02cc50 con 0x7efcec0ce580 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.764+0000 7efce9ffb640 1 -- 172.21.15.106:0/981516958 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc034440 con 0x7efcec0ce580 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 -- 172.21.15.106:0/981516958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 msgr2=0x7efcec0d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec0d0180 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009d90 tx=0x7efcdc02bea0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 -- 172.21.15.106:0/981516958 shutdown_connections 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 --2- 172.21.15.106:0/981516958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec0d0180 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 -- 172.21.15.106:0/981516958 >> 172.21.15.106:0/981516958 conn(0x7efcec0d25b0 msgr2=0x7efcec0d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 -- 172.21.15.106:0/981516958 shutdown_connections 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.765+0000 7efcf15f7640 1 -- 172.21.15.106:0/981516958 wait complete. 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.766+0000 7efcf15f7640 1 Processor -- start 2024-04-13T03:20:23.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.766+0000 7efcf15f7640 1 -- start start 2024-04-13T03:20:23.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 0x7efcec0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 --2- >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efcec0d16c0 con 0x7efcec0ce580 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efcec0d13e0 con 0x7efcec0b5940 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcf15f7640 1 -- --> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] -- mon_getmap magic: 0 -- 0x7efcec0ce960 con 0x7efcec0d06c0 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44652/0 (socket says 172.21.15.106:44652) 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 -- 172.21.15.106:0/1146484771 learned_addr learned my addr 172.21.15.106:0/1146484771 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efceb7fe640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efceaffd640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 0x7efcec0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 -- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 msgr2=0x7efcec0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 -- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 msgr2=0x7efcec0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 0x7efcec0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 -- 172.21.15.106:0/1146484771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec14f4b0 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efceb7fe640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efceaffd640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 0x7efcec0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.767+0000 7efcea7fc640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7efce000e8a0 tx=0x7efce000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.768+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0017070 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.768+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efce000c940 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.768+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000cd50 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.768+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcec0b15f0 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.768+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcec14fa00 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.769+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcec150400 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.769+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efce001a070 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.769+0000 7efcd37fe640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 0x7efcd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.770+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7efce008b9b0 con 0x7efcec0ce580 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.770+0000 7efceaffd640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 0x7efcd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T03:20:23.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.770+0000 7efceaffd640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 0x7efcd806edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efcdc036000 tx=0x7efcdc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T03:20:23.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.774+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efce0059ea0 con 0x7efcec0ce580 2024-04-13T03:20:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.986+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcec0cd840 con 0x7efcec0ce580 2024-04-13T03:20:23.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.987+0000 7efcd37fe640 1 -- 172.21.15.106:0/1146484771 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+3883 (secure 0 0 0) 0x7efce00597a0 con 0x7efcec0ce580 2024-04-13T03:20:23.989 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-13T03:20:23.989 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":100,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4116242919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":4116242919},{"type":"v1","addr":"172.21.15.114:6835","nonce":4116242919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/3481273799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3481273799},{"type":"v1","addr":"172.21.15.114:6834","nonce":3481273799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3845003921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3845003921},{"type":"v1","addr":"172.21.15.106:6836","nonce":3845003921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9798,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2967271200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2967271200},{"type":"v1","addr":"172.21.15.106:6837","nonce":2967271200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[]} 2024-04-13T03:20:23.990 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 100 2024-04-13T03:20:23.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 msgr2=0x7efcd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 0x7efcd806edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efcdc036000 tx=0x7efcdc030000 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 msgr2=0x7efcec14e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7efce000e8a0 tx=0x7efce000ed60 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 shutdown_connections 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:6832/3543688068,v1:172.21.15.106:6833/3543688068] conn(0x7efcd806c8f0 0x7efcd806edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7efcec0d06c0 0x7efcec0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcec0ce580 0x7efcec14e120 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 --2- 172.21.15.106:0/1146484771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec0b5940 0x7efcec0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 >> 172.21.15.106:0/1146484771 conn(0x7efcec0d25b0 msgr2=0x7efcec0cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 shutdown_connections 2024-04-13T03:20:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-13T03:20:23.989+0000 7efcf15f7640 1 -- 172.21.15.106:0/1146484771 wait complete. 2024-04-13T03:20:24.000 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-13T03:20:24.001 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-13T03:20:24.001 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-13T03:20:24.001 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.010 INFO:tasks.ceph.mds.a:Stopped 2024-04-13T03:20:24.010 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-13T03:20:24.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.059 INFO:tasks.ceph.mds.c:Stopped 2024-04-13T03:20:24.060 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-13T03:20:24.060 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.072 INFO:tasks.ceph.mds.b:Stopped 2024-04-13T03:20:24.073 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-13T03:20:24.073 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.124 INFO:tasks.ceph.mds.d:Stopped 2024-04-13T03:20:24.125 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-13T03:20:24.125 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-13T03:20:24.125 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.155 INFO:tasks.ceph.osd.0:Stopped 2024-04-13T03:20:24.156 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-13T03:20:24.156 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.226 INFO:tasks.ceph.osd.1:Stopped 2024-04-13T03:20:24.226 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-13T03:20:24.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.293 INFO:tasks.ceph.osd.2:Stopped 2024-04-13T03:20:24.293 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-13T03:20:24.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.361 INFO:tasks.ceph.osd.3:Stopped 2024-04-13T03:20:24.361 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-13T03:20:24.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.403 INFO:tasks.ceph.osd.4:Stopped 2024-04-13T03:20:24.403 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-13T03:20:24.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.441 INFO:tasks.ceph.osd.5:Stopped 2024-04-13T03:20:24.442 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-13T03:20:24.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.512 INFO:tasks.ceph.osd.6:Stopped 2024-04-13T03:20:24.512 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-13T03:20:24.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.589 INFO:tasks.ceph.osd.7:Stopped 2024-04-13T03:20:24.589 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-13T03:20:24.589 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-13T03:20:24.590 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.615 INFO:tasks.ceph.mgr.y:Stopped 2024-04-13T03:20:24.615 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-13T03:20:24.615 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.659 INFO:tasks.ceph.mgr.x:Stopped 2024-04-13T03:20:24.659 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-13T03:20:24.659 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-13T03:20:24.659 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.671 INFO:tasks.ceph.mon.a:Stopped 2024-04-13T03:20:24.671 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-13T03:20:24.671 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.711 INFO:tasks.ceph.mon.b:Stopped 2024-04-13T03:20:24.711 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-13T03:20:24.711 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T03:20:24.766 INFO:tasks.ceph.mon.c:Stopped 2024-04-13T03:20:24.766 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-13T03:20:24.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-13T03:20:24.805 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:13:57.348562+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-13T03:20:24.805 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-13T03:20:24.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-13T03:20:24.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-13T03:20:24.953 INFO:teuthology.orchestra.run.smithi106.stdout:2024-04-13T03:15:35.247546+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-13T03:20:24.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:20:24.954 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-13T03:20:25.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:20:25.199 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-13T03:20:25.287 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:20:25.287 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-13T03:20:25.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:20:25.354 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-13T03:20:25.412 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:20:25.412 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-13T03:20:25.571 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:20:25.571 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-13T03:20:25.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:20:25.643 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-13T03:20:25.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:20:25.708 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-13T03:20:25.756 INFO:tasks.ceph:Archiving mon data... 2024-04-13T03:20:25.758 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/data/mon.a.tgz 2024-04-13T03:20:25.758 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-13T03:20:25.773 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.rSv2kWtMPE 2024-04-13T03:20:25.773 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rSv2kWtMPE 2024-04-13T03:20:25.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.rSv2kWtMPE 2024-04-13T03:20:26.010 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.rSv2kWtMPE is 405KB 2024-04-13T03:20:26.108 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.rSv2kWtMPE 2024-04-13T03:20:26.124 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/data/mon.b.tgz 2024-04-13T03:20:26.124 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-04-13T03:20:26.140 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.uB39O7mK2Y 2024-04-13T03:20:26.140 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.uB39O7mK2Y 2024-04-13T03:20:26.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.uB39O7mK2Y 2024-04-13T03:20:26.381 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.uB39O7mK2Y is 409KB 2024-04-13T03:20:26.478 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.uB39O7mK2Y 2024-04-13T03:20:26.492 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/data/mon.c.tgz 2024-04-13T03:20:26.492 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-04-13T03:20:26.546 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.t56TetO5jP 2024-04-13T03:20:26.547 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.t56TetO5jP 2024-04-13T03:20:26.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.t56TetO5jP 2024-04-13T03:20:26.772 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.t56TetO5jP is 409KB 2024-04-13T03:20:26.868 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.t56TetO5jP 2024-04-13T03:20:26.882 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-13T03:20:26.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-13T03:20:26.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-13T03:20:26.919 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:26.956 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:26.969 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-13T03:20:27.005 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-13T03:20:27.005 INFO:tasks.ceph:Archiving crash dumps... 2024-04-13T03:20:27.009 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi106/crash 2024-04-13T03:20:27.010 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-13T03:20:27.073 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi114/crash 2024-04-13T03:20:27.074 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-13T03:20:27.100 INFO:tasks.ceph:Shutting down logrotate 2024-04-13T03:20:27.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:27.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:27.202 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:27.229 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-13T03:20:27.256 INFO:tasks.ceph:Compressing logs... 2024-04-13T03:20:27.256 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-13T03:20:27.272 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-13T03:20:27.347 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43656.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37647.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.43656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37727.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43656.log.gz 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44054.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37647.log.gz 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37727.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.3.44054.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37727.log.gz -- 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-mds.b.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44054.log.gz 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37567.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37607.log 2024-04-13T03:20:27.348 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37567.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37607.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37567.log.gz 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37607.log.gz 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37687.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37687.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37687.log.gz 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37807.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37807.log.gz 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-13T03:20:27.349 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-13T03:20:27.350 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-13T03:20:27.350 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-13T03:20:27.350 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-13T03:20:27.352 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37847.log 2024-04-13T03:20:27.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.37847.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37847.log.gz 2024-04-13T03:20:27.391 INFO:teuthology.orchestra.run.smithi114.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-13T03:20:27.478 INFO:teuthology.orchestra.run.smithi114.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-13T03:20:27.571 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80931.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78970.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66235.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80181.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78970.log.gz 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.80931.log.gz 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60139.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66235.log.gz 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46694.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80181.log.gz 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69736.log 2024-04-13T03:20:27.572 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46694.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60139.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71555.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46694.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69736.log: -- /var/log/ceph/ceph-client.admin.83222.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54388.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69736.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71555.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54766.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64474.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83222.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54388.log.gz 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52520.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70312.log 2024-04-13T03:20:27.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54766.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52520.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70312.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52520.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67223.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64474.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70312.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49111.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.67223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67139.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49111.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.46349.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67223.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50434.log 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49111.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46349.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67139.log.gz 2024-04-13T03:20:27.574 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82758.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75276.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70062.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50434.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75276.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82758.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47202.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75276.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79305.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47202.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70062.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48817.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47202.log.gz 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79689.log 2024-04-13T03:20:27.575 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69170.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79305.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.79689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71597.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% 89.3% -- replaced with /var/log/ceph/ceph-client.0.79689.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48817.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70001.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71597.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69170.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65138.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58727.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71597.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.65138.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.0.65138.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.60874.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57865.log 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60874.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58727.log.gz 2024-04-13T03:20:27.576 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81470.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61492.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57865.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.70001.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60874.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81470.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57865.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74791.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66277.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81470.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74791.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61492.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62162.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66109.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.74791.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66277.log.gz 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77330.log 2024-04-13T03:20:27.577 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62162.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63553.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66109.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77330.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5/var/log/ceph/ceph-client.admin.50608.log: --verbose -- /var/log/ceph/ceph-client.admin.64648.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45672.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71387.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63553.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45672.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.64648.log.gz 2024-04-13T03:20:27.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71970.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71387.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82632.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57991.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71970.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70645.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57991.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82632.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79978.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57991.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61822.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58086.log 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61822.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.79978.log.gz 2024-04-13T03:20:27.579 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70645.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68495.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61822.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68495.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58086.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.55531.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79393.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54346.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68495.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55531.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66498.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54346.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79393.log.gz 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62794.log 2024-04-13T03:20:27.580 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54346.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50350.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.62794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59406.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66498.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62794.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71741.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59406.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50350.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63341.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53024.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59406.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63341.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71741.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62120.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63341.log.gz 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45074.log 2024-04-13T03:20:27.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53024.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45032.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45074.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62120.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75463.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45074.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82109.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75463.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45032.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55867.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82109.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.75463.log.gz -5 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60562.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82109.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55867.log.gz 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-04-13T03:20:27.582 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44058.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60562.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66066.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63141.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51834.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63141.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66066.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55443.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63141.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79936.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51834.log: /var/log/ceph/ceph-client.admin.55443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67457.log 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51834.log.gz 2024-04-13T03:20:27.583 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55443.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53719.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79936.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67457.log: /var/log/ceph/ceph-client.admin.53719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47058.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67457.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60916.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53719.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76173.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47058.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70579.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60916.log: /var/log/ceph/ceph-client.admin.76173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70579.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60916.log.gz 2024-04-13T03:20:27.584 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76173.log.gz 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54682.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75136.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70579.log.gz 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54682.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz /var/log/ceph/ceph-client.admin.68244.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54682.log.gz 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47016.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75136.log: /var/log/ceph/ceph-client.admin.68244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63015.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47016.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.75136.log.gz 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66971.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47016.log.gz 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63015.log: 91.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.68244.log.gz -5 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55487.log 2024-04-13T03:20:27.585 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63285.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63015.log.gz/var/log/ceph/ceph-client.admin.66971.log: 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66971.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69652.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55487.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74698.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63285.log: /var/log/ceph/ceph-client.admin.69652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78928.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63285.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69652.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59947.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59731.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74698.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59947.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78928.log.gz 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76892.log 2024-04-13T03:20:27.586 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72522.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80062.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59731.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59947.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45798.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76892.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81983.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72522.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76257.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80062.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45798.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61355.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81983.log.gz 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.76257.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55275.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58504.log 2024-04-13T03:20:27.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76257.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61355.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58504.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55275.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46932.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58504.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72652.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55615.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72652.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46932.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52688.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72652.log.gz 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66872.log 2024-04-13T03:20:27.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49405.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55615.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66872.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52688.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72192.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66872.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73806.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49405.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73806.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72192.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52856.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73806.log.gz 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46890.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52856.log: gzip 88.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz /var/log/ceph/ceph-client.admin.60384.log 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.589 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46890.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52856.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67645.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46890.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60384.log: /var/log/ceph/ceph-client.0.67645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51586.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67645.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60384.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48775.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78293.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70500.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51586.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48775.log.gz 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49783.log 2024-04-13T03:20:27.590 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78293.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.70500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73883.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70500.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63219.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6%/var/log/ceph/ceph-client.admin.73883.log: -- replaced with /var/log/ceph/ceph-client.admin.49783.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61952.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2%/var/log/ceph/ceph-client.admin.63219.log: -- replaced with /var/log/ceph/ceph-client.admin.73883.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61952.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63219.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56527.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61952.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82211.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69876.log 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-04-13T03:20:27.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82211.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82211.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69876.log: gzip -5 --verbose 89.0% -- -- replaced with /var/log/ceph/ceph-client.admin.69876.log.gz /var/log/ceph/ceph-client.admin.64907.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64907.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62498.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50188.log: /var/log/ceph/ceph-client.admin.46554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71167.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62498.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2024-04-13T03:20:27.592 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62498.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71345.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71167.log: /var/log/ceph/ceph-client.admin.50392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56528.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71167.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71345.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43804.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71345.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71303.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56528.log: /var/log/ceph/ceph-client.admin.43804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72736.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.56528.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43804.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80747.log 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71303.log.gz 2024-04-13T03:20:27.593 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72438.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72736.log: gzip/var/log/ceph/ceph-client.admin.80747.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46014.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72736.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72438.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80747.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74579.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72438.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75879.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46014.log: /var/log/ceph/ceph-client.admin.74579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57907.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74579.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75879.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46014.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75634.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58685.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57907.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75879.log.gz 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75700.log 2024-04-13T03:20:27.594 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57907.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53677.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75700.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75634.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79195.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58685.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75700.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51292.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53677.log: /var/log/ceph/ceph-client.admin.79195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81857.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53677.log.gz/var/log/ceph/ceph-client.admin.51292.log: 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 90.2% -- replaced with /var/log/ceph/ceph-client.admin.79195.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73638.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51292.log.gz 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55825.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81512.log 2024-04-13T03:20:27.595 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81857.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55825.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73638.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55825.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71041.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81512.log: /var/log/ceph/ceph-client.admin.45300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81512.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71041.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61060.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71041.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70116.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49978.log: /var/log/ceph/ceph-client.admin.61060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62288.log 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70116.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61060.log.gz 2024-04-13T03:20:27.596 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62540.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70116.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57319.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62288.log: /var/log/ceph/ceph-client.admin.62540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83514.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62288.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71928.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57319.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.62540.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70999.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81554.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83514.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70999.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71928.log.gz 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68911.log 2024-04-13T03:20:27.597 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70999.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81554.log: /var/log/ceph/ceph-client.admin.49573.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81554.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72480.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68911.log: /var/log/ceph/ceph-client.admin.48901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80480.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78436.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80480.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68911.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72480.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56973.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68166.log 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78436.log.gz 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56973.log: gzip 91.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.80480.log.gz --verbose 2024-04-13T03:20:27.598 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.58811.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56973.log.gz 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77690.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68166.log: /var/log/ceph/ceph-client.admin.58811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63808.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68166.log.gz 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77690.log: 88.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.58811.log.gz --verbose 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.76425.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3%/var/log/ceph/ceph-client.0.63808.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.77690.log.gz 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75192.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63808.log.gz 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50566.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7%/var/log/ceph/ceph-client.admin.75192.log: -- replaced with /var/log/ceph/ceph-client.admin.76425.log.gz 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76383.log 2024-04-13T03:20:27.599 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75192.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57193.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50566.log: /var/log/ceph/ceph-client.admin.76383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50566.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57193.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76383.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78886.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57193.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77540.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78886.log: /var/log/ceph/ceph-client.admin.47160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77061.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78886.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77540.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66744.log 2024-04-13T03:20:27.600 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77540.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82674.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77061.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44184.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66744.log: /var/log/ceph/ceph-client.admin.82674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76731.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66744.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82674.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61229.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76731.log: 91.3%/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66828.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44184.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76731.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-04-13T03:20:27.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66828.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61229.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58769.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66828.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64012.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54556.log: /var/log/ceph/ceph-client.admin.58769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61696.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 89.8%/var/log/ceph/ceph-client.admin.64012.log: 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58769.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68453.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64012.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49657.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61696.log: /var/log/ceph/ceph-client.admin.68453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68561.log 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61696.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68453.log.gz 2024-04-13T03:20:27.602 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79098.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49657.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48985.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68561.log: /var/log/ceph/ceph-client.admin.79098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53963.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68561.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79098.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49489.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82254.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53963.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48985.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74792.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.53963.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49489.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83264.log 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74792.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82254.log.gz 2024-04-13T03:20:27.603 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74792.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57277.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83264.log: /var/log/ceph/ceph-client.admin.51376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77105.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83264.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57277.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76005.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5%/var/log/ceph/ceph-client.admin.77105.log: -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57487.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76005.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77105.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46691.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57487.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76005.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57949.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57487.log.gz 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57739.log 2024-04-13T03:20:27.604 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68365.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57949.log: /var/log/ceph/ceph-client.admin.57739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9%/var/log/ceph/ceph-client.admin.68365.log: -- replaced with /var/log/ceph/ceph-client.admin.57949.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57739.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59045.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68365.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46974.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43445.log: /var/log/ceph/ceph-client.admin.59045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79347.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46974.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59045.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61866.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46974.log.gz 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73198.log 2024-04-13T03:20:27.605 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79347.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5/var/log/ceph/ceph-client.admin.61866.log: --verbose -- /var/log/ceph/ceph-client.admin.43614.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52436.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61866.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43614.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73198.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76509.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43614.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81638.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52436.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76509.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76509.log.gz 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49321.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56783.log 2024-04-13T03:20:27.606 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81638.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82716.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56783.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49321.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58895.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56783.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62246.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58895.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82716.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69470.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58895.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55573.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62246.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64281.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62973.log 2024-04-13T03:20:27.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55573.log: /var/log/ceph/ceph-client.admin.64281.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55573.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69470.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.73072.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43134.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62973.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73072.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64281.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73072.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58937.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43134.log: /var/log/ceph/ceph-client.admin.54220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65493.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43134.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64054.log 2024-04-13T03:20:27.608 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65493.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58937.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58166.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65493.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9%/var/log/ceph/ceph-client.admin.58166.log: -- replaced with /var/log/ceph/ceph-client.admin.64054.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66786.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58166.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46016.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47370.log: /var/log/ceph/ceph-client.admin.66786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58290.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46016.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66786.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66702.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58290.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73925.log 2024-04-13T03:20:27.609 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58290.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54262.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66702.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73925.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71886.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54262.log: /var/log/ceph/ceph-client.admin.57235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68311.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71886.log: -- replaced with /var/log/ceph/ceph-client.admin.54262.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80457.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71886.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60772.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68311.log: /var/log/ceph/ceph-client.admin.80457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75921.log 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68311.log.gz 2024-04-13T03:20:27.610 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60772.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.80457.log.gz --verbose 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.73240.log 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60772.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43226.log 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.80391.log -- replaced with /var/log/ceph/ceph-client.admin.75921.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73240.log: /var/log/ceph/ceph-client.admin.43226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78844.log 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80391.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43226.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70796.log 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73240.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78844.log: -5 --verbose -- 91.1% -- replaced with /var/log/ceph/ceph-client.admin.80391.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.83012.log 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78844.log.gz 2024-04-13T03:20:27.611 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54808.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70796.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77498.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83012.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72820.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54808.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77498.log: /var/log/ceph/ceph-client.admin.72820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77456.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77498.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46685.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72820.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46353.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75837.log 2024-04-13T03:20:27.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77456.log: /var/log/ceph/ceph-client.admin.46353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66660.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77456.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75837.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46353.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63436.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66660.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75837.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63436.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66660.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52604.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63436.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60342.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44100.log: /var/log/ceph/ceph-client.admin.52604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74537.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60342.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52604.log.gz 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54850.log 2024-04-13T03:20:27.613 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60342.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60832.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74537.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66916.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54850.log: /var/log/ceph/ceph-client.admin.60832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54976.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60832.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66916.log: -- replaced with /var/log/ceph/ceph-client.admin.54850.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46356.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54976.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66916.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82800.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46356.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54976.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52982.log 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46356.log.gz 2024-04-13T03:20:27.614 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66403.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82800.log: /var/log/ceph/ceph-client.admin.52982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72694.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.82800.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66403.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52982.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82067.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66403.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83348.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72694.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48943.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82067.log: /var/log/ceph/ceph-client.admin.83348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48943.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82067.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83348.log.gz 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71826.log 2024-04-13T03:20:27.615 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48943.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72354.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5/var/log/ceph/ceph-client.admin.71826.log: --verbose -- /var/log/ceph/ceph-client.admin.67391.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70730.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71826.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72354.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65325.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64733.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70730.log: /var/log/ceph/ceph-client.admin.65325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59665.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67391.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64733.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65325.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79435.log 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59665.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70730.log.gz 2024-04-13T03:20:27.616 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58853.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79435.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64733.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64949.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59665.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49195.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58853.log: /var/log/ceph/ceph-client.admin.64949.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58853.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64582.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79435.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64949.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52306.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49195.log: /var/log/ceph/ceph-client.admin.64582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72012.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49195.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69694.log 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9%/var/log/ceph/ceph-client.admin.72012.log: -- replaced with /var/log/ceph/ceph-client.admin.64582.log.gz 2024-04-13T03:20:27.617 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52306.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72012.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52814.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69694.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60205.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80020.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52814.log: /var/log/ceph/ceph-client.admin.60205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67307.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52814.log.gz/var/log/ceph/ceph-client.admin.80020.log: 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60205.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67055.log 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80020.log.gz 2024-04-13T03:20:27.618 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43322.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67307.log: /var/log/ceph/ceph-client.admin.67055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54000.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67307.log.gz 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43322.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67055.log.gz 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43322.log.gz 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49615.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54000.log: /var/log/ceph/ceph-client.admin.49153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60097.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54000.log.gz/var/log/ceph/ceph-client.admin.49615.log: 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56615.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52898.log 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60097.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49615.log.gz 2024-04-13T03:20:27.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60097.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51712.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52898.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56615.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72988.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52898.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55741.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50104.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51712.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72988.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58460.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83556.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55741.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72988.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58460.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55741.log.gz 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75029.log 2024-04-13T03:20:27.620 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.58460.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60958.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83556.log: /var/log/ceph/ceph-client.admin.75029.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83556.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78209.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.75029.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46341.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60958.log: /var/log/ceph/ceph-client.admin.78209.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60958.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76593.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78209.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46341.log: /var/log/ceph/ceph-client.admin.79251.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76593.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46341.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67013.log 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76593.log.gz 2024-04-13T03:20:27.621 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72396.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79251.log: /var/log/ceph/ceph-client.admin.67013.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79251.log.gz 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.80630.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6%/var/log/ceph/ceph-client.admin.72396.log: -- replaced with /var/log/ceph/ceph-client.admin.67013.log.gz 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.72396.log.gz 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57057.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46692.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4%/var/log/ceph/ceph-client.admin.46692.log: -- replaced with /var/log/ceph/ceph-client.admin.57057.log.gz 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49447.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46692.log.gz 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61000.log 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log/var/log/ceph/ceph-client.admin.49447.log: 2024-04-13T03:20:27.622 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.80630.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61000.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49447.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46357.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61000.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74879.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53635.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.74879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60426.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53635.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60426.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74879.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47412.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60426.log.gz 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56699.log 2024-04-13T03:20:27.623 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47412.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58555.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83390.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46020.log: /var/log/ceph/ceph-client.admin.58555.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr: -- 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56699.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.62204.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83390.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58555.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59581.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83390.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83180.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59581.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62204.log.gz 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72566.log 2024-04-13T03:20:27.624 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49027.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83180.log: /var/log/ceph/ceph-client.admin.72566.log: gzip -5 --verbose -- 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59581.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83180.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.59881.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49027.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72566.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43656.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49027.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79501.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63262.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.79501.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43656.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68189.log 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59881.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79501.log.gz 2024-04-13T03:20:27.625 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69212.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63262.log: /var/log/ceph/ceph-client.admin.68189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76215.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63262.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68189.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65670.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69212.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76215.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52222.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77582.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65670.log.gz 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2024-04-13T03:20:27.626 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52222.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59459.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77582.log: /var/log/ceph/ceph-client.admin.49363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61103.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77582.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59459.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46351.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61103.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.59459.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83306.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61103.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52096.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46351.log: /var/log/ceph/ceph-client.admin.83306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52562.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46351.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63394.log 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83306.log.gz 2024-04-13T03:20:27.627 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8%/var/log/ceph/ceph-client.admin.52562.log: -- replaced with /var/log/ceph/ceph-client.admin.52096.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65796.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52562.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55657.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63394.log: /var/log/ceph/ceph-client.admin.65796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69814.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63394.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55657.log: gzip 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65796.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80801.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55657.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73030.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69814.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80801.log: /var/log/ceph/ceph-client.admin.73030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70246.log 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80801.log.gz 2024-04-13T03:20:27.628 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73030.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45342.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54892.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70246.log: /var/log/ceph/ceph-client.admin.45342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63619.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45342.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52264.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr: 90.8%/var/log/ceph/ceph-client.admin.63619.log: -- replaced with /var/log/ceph/ceph-client.admin.70246.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54892.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81731.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63619.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57361.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81731.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52264.log.gz 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49237.log 2024-04-13T03:20:27.629 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81731.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57361.log: /var/log/ceph/ceph-client.admin.49237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50266.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57361.log.gz/var/log/ceph/ceph-client.admin.43762.log: 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62456.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49237.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44671.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50266.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63501.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62456.log: /var/log/ceph/ceph-client.admin.44671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82338.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62456.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52348.log 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63501.log.gz 2024-04-13T03:20:27.630 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82338.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.72778.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37297.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52348.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80843.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37297.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72778.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56247.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37297.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81899.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80843.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.56247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56247.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71784.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81899.log.gz 2024-04-13T03:20:27.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73722.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56806.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51502.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71784.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52730.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73722.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52730.log: gzip -5 --verbose -- 90.0% -- replaced with /var/log/ceph/ceph-client.admin.56806.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.61145.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51106.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52730.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59773.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gz 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69610.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46686.log 2024-04-13T03:20:27.632 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59773.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61145.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61994.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46686.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69610.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59773.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.51962.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46686.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71681.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61994.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73494.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51962.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69062.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5%/var/log/ceph/ceph-client.0.73494.log: -- replaced with /var/log/ceph/ceph-client.admin.71681.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73494.log.gz 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62036.log 2024-04-13T03:20:27.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73948.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62036.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69062.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57781.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7%/var/log/ceph/ceph-client.admin.73948.log: -- replaced with /var/log/ceph/ceph-client.admin.62036.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57781.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73114.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73948.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74963.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73114.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76934.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73114.log.gz 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75550.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71429.log 2024-04-13T03:20:27.634 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76934.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49069.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74963.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66006.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49069.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.75550.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71429.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49069.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78251.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43530.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66006.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78251.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65451.log 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43530.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78251.log.gz 2024-04-13T03:20:27.635 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61738.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43530.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83054.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61738.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65451.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82884.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83054.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61738.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82927.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83054.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82884.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82927.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77149.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82927.log.gz -5 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-04-13T03:20:27.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47622.log: /var/log/ceph/ceph-client.admin.77149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55113.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.37650.log: gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77149.log.gz 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56377.log 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55113.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55113.log.gz 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47244.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56377.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56377.log.gz 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64096.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3%/var/log/ceph/ceph-client.admin.49741.log: -- replaced with /var/log/ceph/ceph-client.admin.47244.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75052.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.64096.log: 89.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.74495.log 2024-04-13T03:20:27.637 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67265.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64096.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74495.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.75052.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68104.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74495.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71083.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67265.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51670.log: /var/log/ceph/ceph-client.admin.68104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49699.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71083.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68104.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56931.log 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-04-13T03:20:27.638 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49699.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.71083.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61397.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56931.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59839.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56931.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59363.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59839.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61397.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81941.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59363.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59839.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82151.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59363.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81941.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31266.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81941.log.gz 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66151.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.tmp-client.admin.31266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45462.log 2024-04-13T03:20:27.639 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31266.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82151.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55699.log/var/log/ceph/ceph-client.admin.66151.log: 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45462.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66151.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58639.log 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55699.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45462.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64799.log 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58639.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55699.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77878.log 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58639.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64215.log 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 87.5%/var/log/ceph/ceph-client.0.77878.log: -- replaced with /var/log/ceph/ceph-client.admin.64799.log.gz 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72946.log 2024-04-13T03:20:27.640 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77878.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82970.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72946.log: gzip -5 --verbose 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64215.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.46018.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82970.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72946.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80997.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82970.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77019.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76551.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80997.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64432.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77019.log.gz 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48105.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.64432.log: 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.49825.log 2024-04-13T03:20:27.641 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76551.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58416.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48105.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64432.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.49825.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58979.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58416.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49825.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.76976.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58979.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58416.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68845.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62372.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2%/var/log/ceph/ceph-client.admin.68845.log: -- replaced with /var/log/ceph/ceph-client.admin.58979.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76976.log.gz 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58374.log 2024-04-13T03:20:27.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62372.log: -- /var/log/ceph/ceph-client.admin.65409.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68845.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62372.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68027.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58374.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80588.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65409.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55909.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68027.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58248.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58248.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55909.log.gz 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53880.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81773.log 2024-04-13T03:20:27.643 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44250.log: 89.2% 91.9% -- replaced with /var/log/ceph/ceph-client.admin.58248.log.gz -- replaced with /var/log/ceph/ceph-client.admin.80588.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53880.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65712.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-client.admin.81773.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56400.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53880.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65712.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81773.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44864.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65712.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46022.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56400.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48859.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46022.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47100.log 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44864.log.gz 2024-04-13T03:20:27.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48859.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46022.log.gzgzip 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44906.log 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48859.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57823.log 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47100.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44906.log: /var/log/ceph/ceph-client.admin.74453.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47100.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57823.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44906.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80298.log 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57823.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62414.log 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74453.log: /var/log/ceph/ceph-client.admin.80298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74004.log 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74453.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62414.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.80298.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74621.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62414.log.gz 2024-04-13T03:20:27.645 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44948.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74004.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.74621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61313.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44948.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74621.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44948.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.69959.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70158.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61313.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72054.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70158.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82296.log 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72054.log.gz 2024-04-13T03:20:27.646 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75094.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45840.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.69959.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76131.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3%/var/log/ceph/ceph-client.admin.82296.log: -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6%/var/log/ceph/ceph-client.admin.76131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55401.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82296.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.75094.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76131.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43572.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51418.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55401.log.gz 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67349.log 2024-04-13T03:20:27.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43572.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64540.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67349.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43572.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48242.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67349.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64540.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64540.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53803.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mgr.y.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5/var/log/ceph/ceph-client.admin.53803.log: --verbose -- /var/log/ceph/ceph-client.admin.61910.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50062.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53803.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81815.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2024-04-13T03:20:27.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61910.log: /var/log/ceph/ceph-client.admin.81815.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61910.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81815.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65880.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54724.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60468.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1%/var/log/ceph/ceph-client.admin.54724.log: -- replaced with /var/log/ceph/ceph-client.admin.65880.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58597.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54724.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69362.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60468.log: /var/log/ceph/ceph-client.admin.58597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60468.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58597.log.gz 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65922.log 2024-04-13T03:20:27.649 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69362.log: /var/log/ceph/ceph-client.admin.50854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65922.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65922.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69362.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74192.log 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55783.log 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log/var/log/ceph/ceph-client.0.74192.log: 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74192.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73680.log 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55783.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80335.log 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73680.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73680.log.gz 2024-04-13T03:20:27.650 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.77624.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49867.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 90.3%/var/log/ceph/ceph-client.admin.77624.log: -- replaced with /var/log/ceph/ceph-client.admin.80335.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55951.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49867.log: -- /var/log/ceph/ceph-client.admin.45174.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.77624.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49867.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46690.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5%/var/log/ceph/ceph-client.admin.45174.log: -- replaced with /var/log/ceph/ceph-client.admin.55951.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46690.log.gz 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-04-13T03:20:27.651 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45630.log: /var/log/ceph/ceph-client.admin.54472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67097.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43974.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67097.log: -- /var/log/ceph/ceph-client.admin.66193.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52138.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75336.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67097.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76811.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66193.log: /var/log/ceph/ceph-client.admin.75336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48691.log 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76811.log: 89.3% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75336.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66193.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76811.log.gz 2024-04-13T03:20:27.652 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56059.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48691.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57403.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56059.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45546.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69568.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47496.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57403.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69104.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69568.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77200.log 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69104.log: 87.7%/var/log/ceph/ceph-client.admin.37547.log: -- replaced with /var/log/ceph/ceph-client.admin.47496.log.gz 2024-04-13T03:20:27.653 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.69104.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61271.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61780.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77200.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62078.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61271.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80889.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61780.log: /var/log/ceph/ceph-client.admin.62078.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61780.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62078.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60604.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80889.log: /var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68749.log 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60604.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80889.log.gz 2024-04-13T03:20:27.654 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60031.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60604.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64841.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.68749.log: /var/log/ceph/ceph-client.admin.60031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66578.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68749.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64841.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.60031.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64841.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52053.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80104.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66578.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61654.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52053.log.gz 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51334.log 2024-04-13T03:20:27.655 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80104.log: /var/log/ceph/ceph-client.admin.61654.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80104.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61654.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76341.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70838.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76341.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51334.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52180.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76341.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70838.log: /var/log/ceph/ceph-client.admin.52180.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52180.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.70838.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77242.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78624.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72904.log 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77242.log.gz 2024-04-13T03:20:27.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.78624.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78624.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47538.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50146.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72904.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50146.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47538.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50146.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62330.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62330.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62330.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70204.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67181.log 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz 2024-04-13T03:20:27.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54037.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70204.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67181.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67181.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46358.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50308.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54037.log.gz/var/log/ceph/ceph-client.admin.46358.log: 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82590.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46358.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50308.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55359.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76089.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82590.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83474.log 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55359.log.gz 2024-04-13T03:20:27.658 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76089.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73156.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83474.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65628.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73156.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73156.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65367.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76635.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65628.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47786.log 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76635.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65367.log.gz 2024-04-13T03:20:27.659 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76635.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57015.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52010.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47786.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71513.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57015.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52010.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83096.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69428.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71513.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69899.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69428.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83096.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69428.log.gz 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55193.log 2024-04-13T03:20:27.660 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83432.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55193.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55193.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.69899.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72272.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83432.log: /var/log/ceph/ceph.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83432.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76299.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70904.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72272.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.70904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76299.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70904.log.gz 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-04-13T03:20:27.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83138.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.83138.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83138.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56453.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82025.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.56453.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56453.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82025.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82025.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67901.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68996.log 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67901.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67901.log.gz 2024-04-13T03:20:27.662 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68996.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68996.log.gz 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80261.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77284.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77414.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77284.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.80261.log.gz 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77284.log.gz 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53320.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77414.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77414.log.gz 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-04-13T03:20:27.663 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.53320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53320.log.gz 2024-04-13T03:20:27.664 INFO:teuthology.orchestra.run.smithi114.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-13T03:20:27.664 INFO:teuthology.orchestra.run.smithi114.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-13T03:20:27.749 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62606.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50020.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71261.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.62606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63970.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71261.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62606.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71261.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55993.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63970.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63970.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55993.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55993.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57655.log 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45588.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2024-04-13T03:20:27.750 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59515.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57655.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57655.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48557.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.59515.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59515.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.2.48557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.48557.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63057.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47286.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52478.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63057.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63057.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75378.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52478.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52478.log.gz 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72610.log 2024-04-13T03:20:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75378.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75378.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54430.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72610.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72610.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66319.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54430.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52772.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66319.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66319.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52772.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52772.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78125.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57697.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78370.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78125.log.gz 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82380.log 2024-04-13T03:20:27.752 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78370.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.78370.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60646.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82380.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82380.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51916.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60646.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60646.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76467.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51916.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51916.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76467.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76467.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65964.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54304.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71125.log 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65964.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65964.log.gz 2024-04-13T03:20:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52940.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71125.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71125.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53922.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52940.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52940.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82842.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53922.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53922.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72096.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.82842.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82842.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80224.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72096.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72096.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57445.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80224.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80224.log.gz 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-04-13T03:20:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57445.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57445.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50692.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60300.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65754.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.53761.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45756.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65754.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65754.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78167.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45756.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45756.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55018.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78167.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78167.log.gz 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75462.log 2024-04-13T03:20:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55018.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55018.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59233.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75462.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75462.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50776.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.59233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59233.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50776.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50776.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76047.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49531.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69278.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.76047.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76047.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54640.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.69278.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69278.log.gz 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54934.log 2024-04-13T03:20:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54640.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54640.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47580.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54934.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47580.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47580.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64366.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51544.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64366.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64366.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61187.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46021.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80546.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61187.log.gz 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48733.log 2024-04-13T03:20:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.80546.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.80546.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48733.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48733.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63099.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46019.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81596.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.63099.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63099.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77372.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.81596.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81596.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57613.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.77372.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77372.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57613.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57613.log.gz 2024-04-13T03:20:27.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78802.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43932.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.78802.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78802.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50734.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45420.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71639.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50734.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50734.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68407.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71639.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71639.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.68407.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68407.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67943.log 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54111.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-04-13T03:20:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65586.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67943.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67943.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55317.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65586.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61572.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.55317.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55317.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71471.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.61572.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61572.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.71471.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71471.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75723.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73306.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75723.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.75723.log.gz 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65215.log 2024-04-13T03:20:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73306.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73306.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65838.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65215.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65215.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.65838.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65838.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60730.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75234.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60730.log: /var/log/ceph/ceph-client.admin.43890.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60730.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50476.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75234.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75234.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81185.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.50476.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50476.log.gz 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73764.log 2024-04-13T03:20:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.81185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81185.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72862.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.73764.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73764.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57529.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.72862.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72862.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58332.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.57529.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57529.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.58332.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58332.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.51022.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67985.log 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43403.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2024-04-13T03:20:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75963.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67985.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66361.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.75963.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75963.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52646.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.66361.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66361.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.52646.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52646.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60688.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.54074.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64138.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.60688.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60688.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67859.log 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.64138.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64138.log.gz 2024-04-13T03:20:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.67859.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67859.log.gz 2024-04-13T03:20:27.797 INFO:teuthology.orchestra.run.smithi106.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-13T03:20:28.213 INFO:teuthology.orchestra.run.smithi106.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-13T03:20:28.548 INFO:teuthology.orchestra.run.smithi114.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-13T03:20:28.631 INFO:teuthology.orchestra.run.smithi114.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-13T03:20:28.663 INFO:teuthology.orchestra.run.smithi114.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-13T03:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-13T03:20:28.681 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-04-13T03:20:28.682 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m1.368s 2024-04-13T03:20:28.682 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m5.827s 2024-04-13T03:20:28.682 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m0.213s 2024-04-13T03:20:28.776 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-13T03:20:28.888 INFO:teuthology.orchestra.run.smithi106.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-13T03:20:28.889 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-13T03:20:28.984 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-13T03:20:28.986 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-13T03:20:28.986 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m1.703s 2024-04-13T03:20:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m7.149s 2024-04-13T03:20:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.645s 2024-04-13T03:20:28.987 INFO:tasks.ceph:Archiving logs... 2024-04-13T03:20:28.987 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi106/log 2024-04-13T03:20:28.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-13T03:20:30.576 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi114/log 2024-04-13T03:20:30.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-13T03:20:31.453 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-13T03:20:31.463 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-13T03:20:31.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-13T03:20:31.466 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-13T03:20:31.553 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-13T03:20:31.564 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-13T03:20:31.564 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T03:20:31.567 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T03:20:31.581 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-13T03:20:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-13T03:20:31.585 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T03:20:31.585 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-13T03:20:31.586 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +1218us[+1203us] +/- 61ms 2024-04-13T03:20:31.586 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -1036us[-1051us] +/- 51ms 2024-04-13T03:20:31.586 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -100us[ -115us] +/- 65ms 2024-04-13T03:20:31.586 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1233us[+1229us] +/- 61ms 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1014us[-1018us] +/- 51ms 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -57us[ -57us] +/- 65ms 2024-04-13T03:20:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T03:20:31.590 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-13T03:20:31.599 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-13T03:20:31.599 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-13T03:20:31.634 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-13T03:20:31.726 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-13T03:20:31.803 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T03:20:31.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:20:31.836 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T03:20:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:20:31.868 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-13T03:20:31.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-13T03:20:31.925 INFO:teuthology.task.internal:Duration was 1166.560094 seconds 2024-04-13T03:20:31.926 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-13T03:20:31.963 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-13T03:20:31.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-13T03:20:31.966 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-13T03:20:32.007 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T03:20:32.012 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T03:20:32.348 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-13T03:20:32.348 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-13T03:20:32.348 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-13T03:20:32.408 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-04-13T03:20:32.408 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-13T03:20:32.432 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-13T03:20:32.433 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-13T03:20:32.451 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-13T03:20:32.503 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-13T03:20:32.504 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-13T03:20:32.519 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-13T03:20:32.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-13T03:20:32.661 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-13T03:20:32.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-13T03:20:32.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-13T03:20:32.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-13T03:20:32.726 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-13T03:20:32.730 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-13T03:20:32.754 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-13T03:20:32.782 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-04-13T03:20:32.797 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-13T03:20:32.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:20:32.824 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-13T03:20:32.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T03:20:32.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-13T03:20:32.862 INFO:teuthology.task.internal:Transferring archived files... 2024-04-13T03:20:32.863 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi106 2024-04-13T03:20:32.863 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-13T03:20:32.906 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237/remote/smithi114 2024-04-13T03:20:32.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-13T03:20:32.943 INFO:teuthology.task.internal:Removing archive directory... 2024-04-13T03:20:32.944 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-13T03:20:32.946 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-13T03:20:32.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-13T03:20:33.010 INFO:teuthology.task.internal:Not uploading archives. 2024-04-13T03:20:33.010 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-13T03:20:33.027 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-13T03:20:33.027 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-13T03:20:33.029 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-13T03:20:33.041 INFO:teuthology.orchestra.run.smithi106.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 13 03:20 /home/ubuntu/cephtest 2024-04-13T03:20:33.047 INFO:teuthology.orchestra.run.smithi114.stdout: 265845 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 13 03:20 /home/ubuntu/cephtest 2024-04-13T03:20:33.049 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-13T03:20:33.063 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-13T03:20:33.127 INFO:teuthology.nuke:Checking targets against current locks 2024-04-13T03:20:33.154 DEBUG:teuthology.nuke:shortname: smithi106 2024-04-13T03:20:33.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T03:20:33.182 DEBUG:teuthology.nuke:shortname: smithi114 2024-04-13T03:20:33.183 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T03:20:33.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 02:51:04.234176', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyYDU5m3TDDQA01tQdtRqc5190+eS7O5PDuIy4C3rpyY48TgQcZeo44AnjuDICd/T+RwJqL3qnqghlndxPkuK8='} 2024-04-13T03:20:33.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654237', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 02:51:04.232760', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrnsrNDJBSUskISoAmoVnGd/SkC+aeSpHNjd8AYVLKsGuh8E4E6f1RCpN0JYFOVOBR1GbmNXdlypbubhUzF36Y='} 2024-04-13T03:20:33.241 INFO:teuthology.orchestra.console.smithi106:Power off 2024-04-13T03:20:33.242 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T03:20:33.268 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-04-13T03:20:33.268 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:33.289 INFO:teuthology.orchestra.console.smithi114:Power off 2024-04-13T03:20:33.289 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T03:20:33.303 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-13T03:20:33.317 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2024-04-13T03:20:33.317 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:33.345 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-04-13T03:20:37.303 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:37.346 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:37.418 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-13T03:20:37.457 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-04-13T03:20:41.420 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:41.458 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:41.531 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-13T03:20:41.570 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2024-04-13T03:20:41.671 INFO:teuthology.orchestra.console.smithi114:Power off completed 2024-04-13T03:20:45.535 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T03:20:45.646 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-04-13T03:20:45.746 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-04-13T03:20:45.851 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1166.5600938796997 failure_reason: '"2024-04-13T03:15:35.247546+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-04-13T03:20:45.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T03:20:45.940 INFO:teuthology.run:FAIL